In the inaugural blog in this serial, I talked abouttest speed, and why it shouldn ’ t be your first antecedency.Rather, you should focus on getting accurate test results back consistently. This week, I ’ ll discuss the power of parallelization in screen.

By far, the better way to trim overall execution clip for a exam suite is to do a best job running tests in parallel. That means running more tests of approximately equal duration at the like clip. Even if your tests run five time slower on a remote program, you ’ ll get your results four times faster if you can run 20 tests at a time rather of everything in one session like you would locally. None of the other performance-enhancing options will provide almost as much benefit as meliorate parallel testing.

There are three test attributes that will permit you to improve parallel examination:

• Atomic, meaning that each test will test only one thing

• Autonomous, so that any tryout can be run in isolation, at the same time or in any order, with another trial

• Short, where each test should be of similar duration (ideally under two minutes)

Sauce Labs did an evaluation of all the tests run on our platform and determined that tests that run in under two minutes are twice as likely to legislate as test that run in more than seven minutes. Based on this, we believe that teams adopting an coming that ease short tests are more probable to get accurate solution.

Also, you get the most efficiency by keeping all of your tests the same length. Typically our users run multiple successive bod on the Sauce Labs program. If a particular build has even one long-running examination, it can tie up all of the parallel session for the length of time it takes to finish execute. This would increase the overall time it takes for your test suite to finish.

To get results backward faster, run more tests in parallel by publish them to be nuclear, autonomous and short. In this short picture, I review these test attributes and show you an application that I created to exemplify them.

SUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.

video

Next, I will show you how to make your tests sufficiently self-governing with effective trial data management.

video

4 Data Management Approaches for Parallel Testing

There are four different approaching to data management that you can use for parallel testing. Let ’ s do a quick overview of each one.

1. Grab & amp; hope

The initiative is what I wish to refer to as Grab and Hope. This is a frequently deployed coming where a exam needs an objective where the specifics don ’ t matter (perhaps a merchandise to add to the pushcart), so it locate the first one it finds and uses it and you have to just hope that it ’ s in the state you need it to be in. This act outstanding until you run enough tests that the hinder end go out of stock and your tests suddenly start failing. Also if one tryout grabs something to edit and another test grabs the same thing to delete, only one of those test is going to pass.

2. Static secureness

This is the traditional QA staging or testing environs that has a bunch of pre-populated data and your tests have identified specific users and specific object and hold affiliate them with specific trial. For example: having user with different attributes ascribe to specific tests, and some examination e'er use this objective, and other examination always use this other exploiter with that other object. The defining characteristic here is that each piece of data is hard coded in your testing codification, often (but not always) in spreadsheet. Obviously this is unmanageable to maintain, and something as small as “ run all these tests but with a user that has this former configuration ” becomes difficult to manage. It is a really bad day when you come into employment and find out that someone delete the staging environment database and you have to vivify all of the objects from scratch!

3. Active fixture

The tertiary option is what developers are used to doing for unit tests with SQL codification that live a testing database immediately before the test suite is run. This prevents the problem of individual deleting your test information, but it has the disadvantage of still demand to hold a one to one relationship between tests and the data they are proceed to use. Also this is often a type of “ grey-box ” solution, because, depending on how the data shot is managed, the data ends up in your system in a state that does not accurately symbolize what would actually be in the database if a exploiter entered that data via the DOM.

4. Just in Time

The fourth and good option is what I call Just In Time. The idea behind Just In Time is that the only way to truly ensure self-directed usage of test data is for every test to have complete control over all of the data it uses. Each tryout creates and authenticates a marque new user, and each tryout creates any datum it postulate to manipulate to measure the specific functionality it is testing for. You might be surprised how often you see flakiness in your examination because two exam pass to be running at the same time that are unknowingly using the same data object in different manner, or one test manipulates data command for another test in an unexpected manner. Often you ’ ll see this when a test fails and leaves the data in an incorrect state for a subsequent test.

I can hear you wondering, “ But Titus, won ’ t it increase overall test clip if each examination has to also set up its own data? ” Remember, though, that you can ’ t experience a successful scheme of atomic and little tests without those examination also being autonomous. This is the only way to reliably increase parallelization, which will more than make up for any increased overhead of setting up data at the showtime of each exam.

I trust you found this helpful. Parallelization and effective data management will go far in helping you increase test hurrying.