Pricelineis the home of modern travel experience. Whether it ’ s finding the perfect place to stay for a much-needed getaway, snagging a reserve at the hottest place in town, or conduct to the open route in a rental car, Priceline can get you there.

Managing quality across this wide range of personalized experience necessitate an extensive and adaptable software test strategy. Dynamic content dominates the customer experience, from sorting algorithms to recommendations and close user group. The intense level of customization delivers an special experience for users, but creates challenges for the QA and development squad.

When Test Automation Architect Antony Robertson joined Priceline in 2015, their package examine strategy was 100 % manual. Regression testing, characteristic (A/B) examination, web/API testing, and end-to-end testing be all execute step-by-step by a dedicated squad of QA professionals. Between ware complexness and rapid-fire delivery cycles, the team simply didn ’ t have the bandwidth to build or buy a examination mechanisation tool that would have a steep learning bender and ask ongoing maintenance. Yet manual testing was increasingly proving to be too time-intensive and ineffective for scaling quality across active user experiences.

From Manual to Automated Testing

Antony begin building towards Priceline ’ s automated examine future by gathering requirements from the QA team. At the clip, only lineament engineer were hire in testing, supporting an all-embracing testing scheme that include core functional testing, GUI prove, end-to-testing, regression testing, cross browser testing, and more. As Antony dove into the challenges limiting testing, he found that Priceline ’ s complex proficient needs had prevented them from investing in a test automation instrument that could reduce the time take for testing. So he decided to make his own.

The Rise of Autobot and Early Automated Testing Efforts

The earliest loop of Priceline ’ s homegrown testing framework, which came to be lovingly known as the Autobots, was the first step towards test automation. As the entire Priceline organization became more gift in the value of test automation, Antony created new, more open iterations of the Autobot that incorporated a turn number of requests from the squad.

The first edition of Antony ’ s homegrown examination automation framework was CLI-only, which was dispute for those new to automated testing. Though the Priceline squad was use to act with their existing technology mass - Node.js, WebdriverIO, Mocha, Chai, and page objective models - they weren ’ t used to leveraging these tools for testing, which added friction to adoption. V0 was too hard encipher, which made it difficult to adapt to quickly changing user needs as they looked for different hotels, restaurant, or travel options.

With that feedback in brain, Antony created V1 of his automated testing framework. He began by create abstraction layers that reduced how often his teammates demand to re-write the like code over and over as they created tests. This include make parameterized functions that the team could just simulate and paste into their testing script. This effort to streamline essay workflow paid off as more people adopted automated testing. More automated testing, however, introduce a new challenge: test maintenance.

To manage test maintenance as automated testing increase, Antony insert examination templet for common trial types. For example, if a Priceline QA or developer desire to test a hotel lookup glad way, they had a standardized template to use. This mired encapsulating the homepage as one entity in a page object poser, then creating a single tryout script that continue the homepage, the lookup page, the checkout, and the intact customer journeying. This get creating an automated exam as easy as introducing conditional logic and incorporating item-by-item test measure.

The test templates resulted in Priceline ’ s “ Prometheus moment ” as automated testing reached parity with manual testing, at least in terms of exertion and truth. Having simple, clear-cut scenario meant that the team was capable to shift from 100 % manual examination to 20 % automated testing. Antony explicate the outlook transformation that occupy place:

“ People realized their boring work could be done with the pushing of a button, and the conversation go from ‘ what ’ s go to happen to my job ’ to ‘ what more can I do in my job. ’ They could spend more time doing higher impact tasks; the harder edge cases, the corner case, the undertaking that are really worthful and receive true monetary value when they create outages, but are besides hard to test. ”

Integrations, Maintenance, and Scaling Automated Testing

For autonomous testing across multiple user personas, check out SUSATest — it explores your app like 10 different real users.

Once the V1 Autobot proved the viability of machine-driven examination, Antony was deluged with further requests from the Priceline team. The first step was locomote off the CLI and into a full test automation program. Then, Antony get adding further abstraction layers that made it easier to configure different permutations of tests, including virtual machine for different type of mobile devices. He also introduced new ways to import new information into the testing framework for more realistic testing scenario. Each merchandise squad had expanded test templates and new levels of data to inform and execute their quality strategies.

But again, more testing introduce more test maintenance challenges and demand for greater functionality from the Autobot. Antony wanted to take any human bottlenecks to testing, i.e. be called when a developer or product owner make a particular environment. So his team began brainstorming how to integrate automated testing into their CI/CD line. As a GitHub team, they construct a GitHub Action that allowed people to trigger tests as needed. Additional tooling pulled test answer, actuate alerts to Splunk or Slack, ingested the info, and then allowed code to pass to the succeeding logical environment if it met minimum pass percentages set by Priceline QA engineers.

New Abstraction Layers Democratize Automated Testing, But Infrastructure Challenges Build

These advance and expansion authorize everyone in the Priceline administration to enter in software screen. Different teams had access to data from their specific product, the power to trigger trial as needed, and could share results to their preferred toolset.

The final lingering challenges to a fully scalable, high-performing automated testing scheme were in the final requirements to accelerate examination. Antony require his entire squad, from QA and developers to ware owners to the CTO, to be capable to test whatever they need. The infrastructure to enable this, however, is complex. Priceline used Sauce Labs for virtual machine, which let for parallelization, an indispensable component for software character strategy at the scale of Priceline ’ s. But those parallelizations were be maxed out as more people get participating in automated testing. Antony calculated how much time was be spent waiting for try capacity and its true cost to the fellowship, and realized this was an unsustainable restriction. So he created an individualised type of parameterization by injecting environmental variable from the UI, which enabled the Priceline team to change examination exceedingly quickly.

Parameterization resolved the parallelization issues, but unleash a new wave of automated screen that strained Priceline ’ s infrastructure. Testing was befall hard and fast, and not all service in non-production surround could handle the cargo. To get around this, Antony built a queuing mechanics that offset and ran each test within a test plan at randomised intervals. The workaround was effective at mitigating the capacity challenge. & nbsp;

Building a Culture of Quality with Mabl

Mabl shared Antony ’ s allegiance to making automated testing as leisurely and accessible as possible. Thanks to the simmpleness of low-code, Priceline was able to promptly onboard dozens of team member. Just as new versions of Autobot give new levels of excitement, Priceline team members were eager to see how they explore new areas of testing with mabl.

Once initial test cases were progress out and the QA team felt comfy testing complex exploiter journeys, the team migrate the entirety of their web examination (~5K tests) into mabl. That impulse ensue in developer asking for the same deploy triggers and automatic gating they had had with Autobot.

Luckily, mabl made it easygoing to accommodate these (many) requests. Antony was able to make machine-driven triggers and deployment action with Priceline ’ s CI/CD line. With only a few carapace hand, the squad eliminated several human touch points that accounted for multiple employment hours per deployment. Several teams have developed such highly tuned tests that they can deploy to production with zero human intervention (so long as all tests pass).

Armed with a reliable and intuitive trial automation platform, engineers began attending Antony ’ s weekly bureau hr to learn more about how they could use mabl, especially flows, JavaScript Snippets, and end-to-end examination templates. Once they understood the basics of mabl and understood how to use reclaimable component, they be able to now start contributing to software testing efforts and establish a culture of quality. Hundreds of user running UI and API tryout are centralise in mabl ’ s unified test automation platform, making it easier to collaborate on quality even as deployment increase.

Antony summarize the importance and impact of this acculturation of quality:

“ In the existence of test automation, our effectiveness is measured by the combined results of speed and caliber. Meaning to say, we want to deploy our code as often and as tight as potential with no bugs. Rushing codification to product without proper examination is a recipe for disaster and, conversely, be slow to grocery with new characteristic leaves you in the dust.

At Priceline, fostering a culture of quality has be one of the master reasons for our success with mabl. Make test creation a zephyr, make deployment quicker and safer by gate free-base on test results, and, lastly, create test results actionable. Build it and they will come, or, rather, examination. ”

Teams can start progress their own acculturation of quality with mabl by file for acostless 14-day test.