This office is the 1st in a 3-part series on optical testing with Appium. Optical testing is an crucial class of testing and can get visual fixation as easily as functional glitch. The brainchild for this series comes from a I produced for. Check it out!

Why do we like about visual testing? Visual examination is a eccentric of testing that overlaps with but is largely orthogonal to functional examination. Here & # x27; s how I think about the deviation:

• Functional testingensures that there are no bugs in the relationship between the user inputs and the codification yield of your app.
• Visual examinationensures that there are no optical discrepancies between different test test. Visual discrepancy could be a sign of a routine of bugs, or a visual/aesthetic regression.

Visual regressions can be just as bad news as functional fixation, especially when they surface bugs that a functional testsuite wouldn & # x27; t catch. Here & # x27; s a great representative found in @ techgirl1908 & # x27; s Twitter feed:

‍

Steps for visual validation

What do we need for visual establishment? The basic principle is that, as we navigate through our app (in the course of a functional test, maybe), we occupy screenshots of each view we encounter. We then compare these screenshots with a previous screenshot of the same view, using some kind of image analysis software. If we bump any notable differences, we may receive found a visual regression, and at that point can either betray the exam or simply droop the difference in some kind of database for a squad to look at down the road.

‍

As pointed out in the diagram above, the persona we use as a mention check are phone & quot; baselines. & quot; Typically we gather baseline on the first visual exam run, and then compare against the baseline on subsequent runs.

Visual comparison with Appium

We already know how to get screenshots with Appium, so that constituent is taken care of. But how do we equate two screenshots to check for visual conflict? As a human be, this is pretty easy (unless you & # x27; re playing one of those difficult spot-the-difference games), but how on earth can we instruct a figurer to do this?

The little answer is that there are scads of smart people out there who have come up with clever algorithms for detecting visual differences. A naive approach to visual divergence (checking individuality of each pixel across the two images) would be way too brittle, since even compression algorithms might result in nigh unnoticeable differences. But luckily for us in the open source existence, we have something calledOpenCV, which is a collection of image processing tools we can use without having to really interpret in depth how they are implemented.

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

Additional system setup

Appium has incorporate support for OpenCV, though it does not get turned on by default, since building OpenCV and its Node.js binding takes a lot of time and must take place on individual platforms. The easygoing way to get things set up for use with Appium is to only runnpm install -g opencv4nodejs.This will attempt to globally establish the Node bindings, which will also take care of downloading and building OpenCV on your system. If that doesn & # x27; t work, you can install OpenCV using Homebrew and then install the Node bindings with theOPENCV4NODEJS_DISABLE_AUTOBUILD=1env flag in forepart, to tell it to use the system-installed binary.

Once you & # x27; ve got the opencv4nodejs package installed, you also need to do certain it & # x27; s accessible to Appium when it runs. One way of do this is to run thenpm install command withoutthe -g flag, inside the Appium directory itself. Another solution is to ensure that your orbicularnode_modulesfolder is listed in an environment variable calledNODE_PATH.

Image comparison commands

OK, we & # x27; ve got all this OpenCV business set up, so how do we really use it from our Appium test codification? With this handy command:

SimilarityMatchingResult res = driver.getImagesSimilarity (img1, img2, opts);

When you name this bid with the appropriate image byte arrays (hither, img1 would be our baseline, and img2 would be the screenshot we like to validate), and with the appropriate options object (it needs to be an example ofSimilarityMatchingOptions), what you get back is aSimilarityMatchingResult object.

The all-important method on the result objective isgetScore,which will tell us how similar the two images we sent in were, in terms of a figure between 0 and 1. To turn this into actual validation, we simply check whether the score is above a certain doorway (which should be experimentally take based on your app). If so, we see there to be no significant differences. If not, we can either cast an exception to fail the test or droop the divergence in some other way.

if (res.getScore ()< MATCH_THRESHOLD) {
    throw new Exception("Visual check failed!");
}

What & # x27; s a good match threshold? I & # x27; ve been pretty successful with a value of 0.99, but your milage may depart. Why do we need a value lower than 1.0? Because on any device screen there will be element unrelated to the visual design of the app which are constantly changing -- the clip on the clock, if nothing else!

Visualizing the departure

Knowing that two images differ in some way is great, but that & # x27; s not actionable information for a developer. What we desire is the two picture side-by-side, with any differences highlighted to get clear what the potential issues are. Thankfully we can direct thegetImagesSimilaritycommand to yield us one of these human-readable images by setting the option target appropriately:

SimilarityMatchingOptions opts = new SimilarityMatchingOptions (); opts.withEnabledVisualization ();

If we & # x27; ve done this, so our issue target will get another method on it:

res.storeVisualization (fileObj);

This method allows us to preserve the visualized deviation icon into aFile object.

Hacking together a ocular validation framework

All of what we & # x27; ve see so far is adequate to insure the similarity between screenshots, but this is not yet a visual proofsystem. For that, we need baseline screenshot management. The simplest way we can implement this is as follows:

• Specify a directory on the scheme to check baseline picture.
• Make sure that each optic check we do has a unique gens, so multiple baselines can exist for different survey.
• Whenever we trigger a ocular check, first look to see whether a baseline exist. If it does not, then simply save the screenshot as the baseline. If it does, then compare the current screenshot with the baseline.

In code, we might construct adoVisualCheckmethod that implements this scheme (assuming all the static class variables hold been set correctly):

private nullity doVisualCheck (String checkName) throws Exception {String baselineFilename = VALIDATION_PATH + `` / '' + BASELINE + checkName + `` .png ''; File baselineImg = new File (baselineFilename); // If no baseline image exists for this check, we should create a baseline image if (! baselineImg.exists ()) {System.out.println (String.format (`` No baseline plant for ' % s ' chit; capturing baseline instead of checking '', checkName)); File newBaseline = driver.getScreenshotAs (OutputType.FILE); FileUtils.copyFile (newBaseline, new File (baselineFilename)); return;} // Otherwise, if we found a baseline, get the image similarity from Appium. In getting the similarity, // we also turn on visualisation so we can see what go wrong if something did. SimilarityMatchingOptions opts = new SimilarityMatchingOptions (); opts.withEnabledVisualization (); SimilarityMatchingResult res = driver.getImagesSimilarity (baselineImg, driver.getScreenshotAs (OutputType.FILE), opts); // If the similarity is not eminent plenty, consider the assay to have failed if (res.getScore ()< MATCH_THRESHOLD) {
        File failViz = new File(VALIDATION_PATH + "/FAIL_" + checkName + ".png");
        res.storeVisualization(failViz);
        throw new Exception(
            String.format("Visual check of '%s' failed; similarity match was only %f, and below the threshold of %f. Visualization written to %s.",
                checkName, res.getScore(), MATCH_THRESHOLD, failViz.getAbsolutePath()));
    }

    // Otherwise, it passed!
    System.out.println(String.format("Visual check of '%s' passed; similarity match was %f",
        checkName, res.getScore()));
}

Now, in our functional test event, we can mix in visual checks however we like. Here & # x27; s an example of a test that navigates and performs visual checks on a few views in the course of a introductory functional verification:

@ Test public void testAppDesign () throws Exception {WebDriverWait wait = new WebDriverWait (driver, 5); // delay for an element that 's on the home blind WebElement loginScreen = waitForElement (wait, LOGIN_SCREEN); // now we know the place blind is loaded, so do a visual cheque doVisualCheck (CHECK_HOME); // nav to the login screen, and wait for an element that 's on the login screen loginScreen.click (); waitForElement (waiting, USERNAME_FIELD); // execute our second optic check, this time of the login screen doVisualCheck (CHECK_LOGIN);}

This is what we need! Of course, there are a lot of rough edges hither, which you would run into if you tried to build an entire visualtestsuite on top of this miniskirt & quot; model & quot; we & # x27; ve developed. So don & # x27; t forget to check out episode 2 in this little serial on visual testing with Appium. Oh, and if you want to see the accomplished code example in the context of a working project, you can alwayschance it on GitHub.

As it stands, the tests will surpass, but that & # x27; s because they & # x27; re running against precisely one version of an app. To see the existent power of this approach in action, you will of course need to run the test against multiple versions of the app, where the second version actually differs visually from the maiden!

4 Parts

Connet Now