Your mechanisation testing strategy

If your team struggles with automation testing, don ’ t feel alone or ashamed. I meet lots of citizenry whose teams do all their fixation examine manually. Early team have automation but are finding it expensive to maintain and aren ’ t sure if the tests are providing a welfare.

Teams who lack machine-driven regression tests are incapable of delivering value to their customers frequently. Changing the code takes long and longer, because there ’ s more and more manual regression test require to feel surefooted about liberate.

Where to start

Get a diverse grouping of speech team members together to talk about what ’ s holding you back and whether mechanization testing might help with that roadblock. Think about the trade-offs between automating and doing testing manually. Both have a price.

What ’ s the big roadblock in your way? When I ask this question, I often hear these solution: time, attainment, budget for creature and exam environments, and bequest codification pattern that make automating tests below the UI level difficult. Your biggest automation pain point may not still be the tests themselves. If, for exemplar, your team doesn ’ t yet have & nbsp;uninterrupted integrationand want to get into automation testing, I ’ d recommend you halt say this and come back once that is set up.

As you review what your squad wants to automatise, evaluate your product ’ s& nbsp; along with& nbsp; the cost and welfare of automating a peculiar area of essay. Are you become fast feedback as you innovate changes in the ware? How long perform it take to hot fix a production bug? You ’ ll want to set a minor, achievable goal to get with, and find means you can measure advance toward that goal. & nbsp;

In my experience, teams get the most long-term welfare from automating unit-level tests, specially if you ’ re practicingtest-driven growth& nbsp; so that the tests help you achieve solid codification design. Tests at higher levels can be more expensive to automatise, but there may be compelling intellect to start at those other levels. Let ’ s face at some models that can aid your squad discuss these trade-offs. & nbsp;

The test automation pyramid

Over the geezerhood, my teams experience found Mike Cohn ’ s “ Test Automation Pyramid ” a useful model to help us formulate our automation strategy. Yes, it is not really a pyramid, it ’ s a triangle, but what ’ s important is that it help you fancy the stairs to direct as you tackle automation testing.

The pyramid shows that in general, you should try to force tests down to the lowest level potential.Unit tests are quick to write (once the team learns how), assure internal code correctness, and can be run in minutes before committing new codification changes. These tests provide the solid understructure to your mechanization examination, but they generally can ’ t test everything.

We also take to check functionality and other calibre characteristics such as availability and security across larger piece of the application with functional testing. For illustration, does an API endpoint return the correct values? These tests may take more clip to make and, if they access databases and other services, furnish dumb feedback. If business stakeholders need to collaborate on acceptance tests to verify business convention, these halfway stratum tests can be written in plain language that everyone can understand.

Products that have a user interface (UI) generally need to be tested across all these layers. The feedback loop from UI level tests is longer, but they can bump the most important fixation failure. Today ’ s technology makes it easier to automate full-bodied tests at the UI point, but many divisor can still do these examination “ flaky ” and more costly to hold. That ’ s why the model has the fewest tests at the top level.

The little cloud at the top of the pyramid represent test that ’ s make manually. Even with solid automation at all layers of the triangle, your product might take a vast thundercloud cloud that include explorative testing, accessibility examination, security testing and many early critical testing activities. You might be able to use automation to speed up this manual testing.

Your app may require a different-shaped model. For example, a web-based merchandise with lots of business logic in the UI may require more regression screen at that level. Also, there are newer, alternative approaches to functional testing that don ’ t require testing through the UI, such ashypodermic testing, that you can investigate.

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

The test mechanisation Triangulum, simplified

There are many fluctuation on the pyramid poser, so take clip to check those out and see if they work better for your squad. Your context is unique, so experiment to find what helps you map out your first steps to automation testing success. I like the way thatSeb Rosemaps out the concept behind the pyramid. You want as many tests as possible that test smaller clod of the covering, but you withal take to check the “ big picture ”.

An real pyramid!

In Chapter 15 ofMore Agile Testing, Sharon Robson explains a four-step process she uses to verbalize about automation testing. She starts with the basic pyramid, but add on the tools that can be used to execute the tests, the types of tests or system attributes to be essay, and finally the machine-controlled regression tests needed. & nbsp;

A DevOps poser

In her Practical Guide to Testing in DevOps (p. 130), Katrina Clokie reverse the trilateral poser to use the different layers of tests as the “ DevOps bug filter ”. The unit level filter finds the tiny unit-level bugs. The integrating stratum catches bigger glitch that lead between different layers of the app. End to end tests catch the more complicated bugs. Then she adds on the info captured in production that can help detect bugs and evaluate merchandise quality: alerting, monitoring, and logging. & nbsp;

If none of these model sings to your team, insure out the links in the reference section at the end of this place. And if none of those help you get started formulating your strategy, grab some markers and a whiteboard (physical or virtual) and start drawing together to visualize what grade of your application need automation testing, and where you want to start.

Starting your first experiment

Your unharmed delivery team needs to agree on a starting strategy for mechanization. Automation testing can be shivery and difficult, so you need to be willing to wedge to it during the painful parts.

You may resolve on a strategy that involves more than one level of the triangle. I worked on a squad that committed to learn how to do test-driven development, knowing the process was proceed to take many months. We shared the burden of manual regression screen through the UI at first, and had no automated tests at all, just a cloud of manual fixation tryout:

We decided to automate UI smoking tests for the critical areas of the application. After eight months, we no longer had to do manual regression testing. By then, we likewise had a small, turn library of automated unit tests. We even did a lot of manual testing on new features and at the services layer of our application. We had an upside-down kind of trilateral that is typical of squad seek to succeed with mechanisation testing:

We still had a lot of holes to fill, but since we didn ’ t spend a day or two every sprint doing manual fixation testing, we had time to tackle other automation challenges. Eventually, our test automation Triangulum looked like the classic “ pyramid ” above.

Successful automation testing requires a big investment over time. Start small. A good way to get started with automating unit tests is to write them for all bug fixes - or even best, test-drive them. You may decide to start automatise API tests by doing them for all new stories or features going forward. Or, if manual regression testing is sucking up all your time,lead advantage of creature that let you quickly get up to speedon UI tryout automation and create some robust UI tests that cover the most critical area, so you can devote more time to expanding your automation efforts to other levels. For sustainable automation quiz that lets you release oft and confidently, your team will need to invest time in learning good.

What if your first experiment fails? Maybe you automate some tests and then find they are unreliable or unacceptable to maintain. Awesome - you experience learned a lot, and that will assist you locomote forward! Don ’ t be afraid to scrap ineffective exam or a creature that isn ’ t working for you and start over. Don ’ t fall victim to the “sunk price fallacy”. As Seb Rose has said, “ if a exam doesn ’ t help you memorise something about the system, then delete it! ”

References and more reading:

Seb Rose, “http: //claysnow.co.uk/architectural-alignment-and-test-induced-design-damage-fallacy/

Katrina Clokie,A Practical Guide to Testing in DevOps

Lisa Crispin and Janet Gregory,More Agile Testing: Learning Journeys for the Whole Team

Noah Sussman, “Abandoning the Pyramid of Testing in favor of a Band-Pass Filter model of risk management”

Alister Scott, “Testing Pyramids & amp; Ice-Cream Cones”