A little more than a year ago today, aiming to direct what we saw as a significant gap in the screen market, we first launched the Sauce Labs Continuous Testing Benchmark, a new study organizations could use to see how their continuous examine exertion heap up against both critical topper practices and the testing efforts of other initiative. In the time since, hundreds of organisation have leverage the report to better understand and amend upon the province of their continuous examine initiatives. It has go an all-important resource for the testing community.

The report looks at real customer datum from the Sauce Labs Continuous Testing Cloud, a platform on which more than 3 million tests are run each day, and on which more than 3 billion tests have been run since its origin. It articulates the four benchmark prosody critical to continuous testing excellence and mensurate how administration ’ corporate examination efforts stack up against these defined standards. Those metrics include test calibre, test run time, test platform coverage, and test efficiency, the latter of which looks nearly at the extent to which organizations are leveraging usable testing capability to achieve parallelization.

New for 2020, the story see the execution of mobile tests (for mobile apps and websites) in addition to desktop examination (for desktop apps and websites). As organizations progressively adopt a mobile-first mindset, understanding and improving the efficaciousness of our roving testing try is overriding. Equally overriding (as is laid bare in the 2020 benchmark report) is the need for organizations to improve test character. Test quality is, after all, the fundament on which successful continuous testing is built. Though governance ’ corporate performance against the test caliber benchmark did improve in comparison to 2019, we still have a long way to go.

SUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.

That ’ s why the report also looks for the first clip at the potential impact of, a new method of applying machine-learning to pass/fail data in order to surface the most common underlying causes of failures within a test suite. Given the pressing motive to improve test quality, failure analysis stand as one of the most promising new developments the testing manufacture has seen in some time.

More important than what the 2020 benchmark study reveals, however, is what it represents: an opportunity for tester and developers to continue learning, growing, and ultimately, improving. Amid the day-to-day pressures of acquire great software, it ’ s easy to bury that most organizations are just beginning their uninterrupted quiz journey, and most will face challenge as those journeys evolve. The best way for us to jointly overcome those challenge is to continue the sense of communion, support, and community that has arrive to delimitate the examination industry. We hope this report will keep to play a role in those endeavor, making it that much easy for us to test better, together.