Lessons From Four Billion Tests

Sauce AI for Test Authoring: Move from intent to execution in minutes.|xBack to ResourcesBlogPosted

January 06, 2026 · 3 min read · Testing Guide

Sauce AI for Test Authoring: Move from intent to execution in minutes.

|

x

Back to Resources

Blog

Posted June 9, 2020

Lessons From Four Billion Tests

quote

Did you see the news? Sauce Labs has now surpassed fourbilliontrial run on our cloud-based continuous testing program! Users are now running more than three million tests per day on Sauce Labs. As we celebrate this milestone, we ’ re reflecting back on some lessons we ’ ve larn along the way. Here ’ s a roundup of our thoughts and advice after working with customer across the world to help them acquire with confidence and ensure their web and mobile apps looking, function and perform exactly as they should.

Ensure stability with a consistent examination environment

After four billion tests, we know that environmental unbalance is a big source of uncertainty in testing. Tests that fail due to factors such as misdeployed APIs and incorrectly configure network/web infrastructure are often due to a couple of key constituent:

  • A test environment that & # x27; s different from production, both in level of resourcing and complexity of configuration.

  • Teams being dependent on former team environments in order for tryout to run correctly.

--Marcus Merrell, Head of Technology Strategy

For autonomous testing across multiple user personas, check out SUSATest — it explores your app like 10 different real users.

Smaller is better

Browser tests have a tendency to be too long (more than 30 seconds locally). The datum clearly exhibit that the long a test accomplish, the more probable it is to fail from a false positive.Atomicand self-directed browser-based tests are urge to alleviate false positives.

--Nikolay Advolodkin, Principal Solutions Architect

Innovation in quality can truly affect people ’ s lives

Companies that provide a remarkable digital experience understand that ’ s where hope are felt most acutely by client. When users have an gratifying experience with a company, they are more potential to preserve doing business with them. I ’ ve seen this in my own life, too—when I needed movie tickets, I download the movie theater ’ s app and the buying procedure was so easy and seamless that I decided to buy an annual membership! Leaders in organizations that empower their development teams to innovate and ensure calibre have the power to create meaningful moments in people ’ s lives, and I love partnering with them.

--Jasmine Williams, Director, Customer Success

Don ’ t get comfortable with failure

Teams have a eminent tolerance for failing/error-prone tests. We ofttimes see teams execute the same failing tests over and over, indicating that they are not fixing the problems, and that the severity of these failure isn & # x27; t important enough to trigger a alteration in the summons.

--Marcus Merrell, Head of Technology Strategy

Get better reportage with more types of testing

There continues to be way too much focus on browser-based functional testing. Many modern web applications should also leverage unit tests, component tests, visual exam, and API tests for authentic and efficient examination reportage.

--Nikolay Advolodkin, Principal Solutions Architect

Sauce Labs will be here for the next four billion

We & # x27; ve seen the browser ecosystem change significantly, and our client alter with it. Internet Explorer was once the king, only then to be subsumed by Firefox, which so yield way to the upstart Chrome. Edge replaced Internet Explorer, and so Google & # x27; s rendering engine replaced Edge & # x27; s supply engine. Meanwhile, Safari has proceed to dominate a large part of the Apple ecosystem. Through it all, Sauce has be at the forefront, not only providing these platforms to our customers, but also guiding customer as they choose what kinds of coverage they need. We ’ ll be here for our client with what ’ s next, too.

--Marcus Merrell, Head of Technology Strategy

Published:
Jun 9, 2020
Share this post
Copy Share Link
LinkedIn
© 2026 Sauce Labs Inc., all right reserved. SAUCE and SAUCE LABS are registered trademarks own by Sauce Labs Inc. in the United States, EU, and may be registered in other jurisdictions.
robot
quote

Automate This With SUSA

Upload your APK or URL. SUSA explores like 10 real users — finds bugs, accessibility violations, and security issues. No scripts needed.

Try SUSA Free

Test Your App Autonomously

Upload your APK or URL. SUSA explores like 10 real users — finds bugs, accessibility violations, and security issues. No scripts.

Try SUSA Free