Getting Started With Visual Regression Testing Using Intern and Sauce Labs
Sauce AI for Test Authoring: Move from intent to executing in minutes.|xBack to ResourcesBlogPosted
Sauce AI for Test Authoring: Move from intent to executing in minutes.
|
x
Blog
Getting Started With Visual Regression Testing Using Intern and Sauce Labs
Intern is a modernistic, pliable, open origin JavaScript testing stack from SitePen and the JS Foundation for exam author in JavaScript, ES6+, or TypeScript. Intern support the following feature set:
Full essay lifecycle support: Unit, functional, integration, and system testing
Author: Mod JS (ES5, ES6+, TypeScript, etc.) with any toolkit or framework
Multiple surroundings: Run examination in browsers, Node.js, and in the cloud (e.g. Sauce Labs, Selenium Grid)
Code coverage: Full statement, line, branch, and function coverage data
CI: Jenkins, SonarQube, TeamCity, Travis CI, Bamboo,Codeship, Codecov, etc.
Visual and performance fixation: Visual diffs and performance degradations
Accessibility submission: Verify compliance expend Tenon and aXe
Thanks to a late grant from the Mozilla MOSS program, Intern recently added support for performance benchmarking, accessibility examination, and visual regression testing. Here we will focus on ocular regression testing with Intern while leverage Sauce Labs.
Visual fixation test
Optical change are notoriously difficult to test. Even the best functional trial suite can not tell you if a page looks correct. Instead, Ocular Regression testing takes a screenshot of a web page (or state within a webapp) and compares it to a previously taken screenshot (referred to as a baseline) to see if the contents feature changed, make a report straightaway unveil what is different. This facilitate technologist to:
know when a change to CSS solution in undesired visual changes elsewhere in the coating
ensure a page is rendered consistently across browsers
put a quick set of exam around a legacy system to identify regression, where it may be hard to write tests for old codification that is not well engineered
Setup
Sauce Labs with Intern
With late versions of Intern, it ’ s now easy to use the intern-cli tool to get setup.
First, you ’ ll need an account with Sauce Labs. Assuming you have a Sauce Labs account, you can find your access key in your user settings page, under “ Access Key ”. It should be a 32-character string, like “ 1234abcd-12ab-34cd-12cd-1234abcd1234 ”.
Once you get a cloud examination account, you can use intern-cli to quickly make a test config and a few elementary tests:
Install intern-cli:
npm install -g intern-cliSUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.
Create a new, empty labor directory and install Intern:
npm install internUse intern-cli to create some starter tests:
intern initEdit the intern config:
Set the tunnel belongings to ‘ SauceLabsTunnel ’
Set the tunnelOptions property to
If your Sauce Labs credentials are already provided via environmental variables, thetunnelOptionsholding for credentials is not necessary. In general and as a reminder, furnish these credentials via surroundings variable is advisable in order to avoid unintentionally checking them into a rootage codification repository. The username and access key are being added to the configuration file for model purposes only.
The project directory should now contain a tests directory with a starter Intern configuration, along with one unit test suite and one functional tryout suite.
project_directory/
tests/
intern.js
unit/
hello.js
functional/
hello.js
page.html
To run the sample functional exam suite with Intern, run the commandintern run -w. The first time you run the tests, you ’ ll see a burrow download progress message, then something similar to the following:
Listening on 0.0.0.0:9000
Tunnel started
*Created session chrome on any platform (a2b118 ... d54)
These content indicate that the service tunnel has successfully begin and that a session was created on the removed examination service. At this point, Intern will start running tests on the test virtual machine. The nonremittal newsperson will display the IDs of tests as they legislate, prove a summary of the test results when the trial have completed.
And with that, you ’ re running Intern examination on Sauce Labs!
For further reading, the Intern guidebook contain a important amount of information onsetup and configuration.
intern-visual
Ocular Regression testing is ply via a plugin to Intern. To get started, first usenpm install intern-visualto instal the plugin. Second, configure the plugin as a parcel with your module loader.
Writing visual regression tests
Visual Regression tests are publish for the WebDriver engine and can be written programmatically or via configuration.
Configuration-driven tests are ideal for situations where the goal is simply to control that a web page hasn ’ t visually change. These exam are very simple to author and are a great way to quickly create tests.
The above model learn intern-visual to make a test that visits the provided url, resize the browser to the providedwidth and heightvalue, and make a baseline icon snapshot if one does not already exist with themissingBaseline property.
However, there are times when we need more control over the page, the application we are try, or when additional steps need to be taken during a test. These extra steps could include inputting and responding to text prompts, hiding parts of the DOM that alteration (e.g. ads), or interact with the page. In these cases, the programmatic API provide a command benefactor that expect a screenshot as a argument.
In the above model,assertVisuals is a command helperthat accepts a cite to the current test and various options, returning a function that wait a screenshot from the takeScreenshot command.
The programmatic code shown above does virtually the same thing as the configuration-based visual test, except the WebDriver bid are defined explicitly. Both of the above tests search for a baseline image on the local machine, yield one if necessary, and provide a pass/fail to Intern along with some additional information about the visual comparison.
Visual fixation test reports
Reporters are Intern ’ s mechanism for providing test results from a variety of source in a variety of formats. The intern-visual test and assert faculty are part of the exam and asseveration layer of Intern, providing a pass or a fail notice. To get a optical HTML report, simply add theVisualRegressionreport to Intern ’ s lean of newsperson:
The VisualRegression report will write an HTML report showing any visual regression in the visual-test directory. It will not output any failures to the console, so be certain to include your preferred console newsperson if you would care to see pass/fail information in the console. When a ocular regression exam fails, it will look in the story with the differences overlaid in red, as shown in the below example.
Future
This is an other release of intern-visual and we have plans to further ameliorate our approach. Our plan include improving the configuration-based ocular examination to make it easier and quicker to publish full visual tests using less code. Also, the current image comparison algorithms are best for compare two identically sized images due to how image differences are identified. We require to improve these algorithms to be more tolerant of minor differences caused by anti-aliasing, allowing for minor differences between browsers or between different window sizes, ignoring all but relevant visual divergence.
Ocular fixation test provide a nimble and easy way to establish baseline for how your application should appear. They are especially valuable when you ask to put tests around a legacy application or to ensure that a CSS change has not make unsought changes. For more information, conduct a look atintern-visualon Github. Visual fixation test examples are available under the tests/visual directory.
Dylan Schiemann is an early adopter and proponent of JavaScript and Open Source web technology. As the CEO ofSitePen, he ’ s be creating leading edge software for nearly two decades. Along with an impressive squad of engineers, SitePen has create tools and frameworks to expedite developer progress while cater weighing and professional services to enterprise teams worldwide. Surrounded by a passionate team of forward-thinking people, Dylan is currently concenter on meliorate the approach to development for the next version of the exposed web, enabling SitePen ’ s customers to modernise their apps, tools and team, and be ready for what ’ s to get.
LinkedIn: https: //www.linkedin.com/in/dylans/Email: dylan @ sitepen.com Twitter: @dylans
Share this office
Automate This With SUSA
Upload your APK or URL. SUSA explores like 10 real users — finds bugs, accessibility violations, and security issues. No scripts needed.
Try SUSA FreeTest Your App Autonomously
Upload your APK or URL. SUSA explores like 10 real users — finds bugs, accessibility violations, and security issues. No scripts.
Try SUSA Free