This post is the 2d in a 3-part series on optic testing with Appium. Ocular examination is an important category of testing and can catch optic fixation as well as functional bugs. The inspiration for this series comes from a free online video course on mobile optic testing I produced for Test Automation University. Check it out! (And don & # x27; t forget to say the first part in this series first).

In the of this series, we look at how to put together a uncomplicated but efficacious homebrew visual examination fabric for mobile apps using Appium and Appium & # x27; s built-in image processing capabilities (thanks, OpenCV!). Our approach did have a turn of limitation, however:

Accelerate Appium test cycles with the HeadSpin, a solution for mobile app mechanization.!

• We had to manage our own baseline images by storing them in a directory on the machine running Appium. This is obviously not very scalable, since whole teams would need access to these images, we & # x27; d probably want to keep a revision history in case we require to roll rearwards to a previous baseline, etc ...
• We had to manage the logic for comparing screenshots to baselines, including settle what to do when we detect differences. In the event of the logic we enforce, we just threw an exception when deviation be detected. But this would kill our functional exam as easily! It & # x27; s nice for the functional trial to continue and for visual differences to be merelyflaggedfor afterwards reexamination rather than cause the whole test to fail.
• In our old scheme, if we detected a visual difference, an persona would be render for us that prove us where the differences might be. It would look something like this, which is an example of how things show up when encountering a failure:

• As you can see, it & # x27; s not just the dislocated & quot; Login & quot; button which is highlighted. Various chip of the device UI have also changed. How long until there are * enough * changes in this piece of the UI for a optical test to neglect, when in reality nothing changed in our app? In general, we have to care a lot about * mistaken positives *, which is when we are alerted to a potential job when none in fact exists. Other typical causes for false positives include, for example, dynamical text whose content is irrelevant to the visual check but which does visually change every clip the exam is run.
• The current approach doesn & # x27; t at all handle scrolling content. What happens when we need to visually verify a blind which has a list or a curl view? For this to work, we & # x27; d need to use Appium to incrementally scroll, conduct screenshots, and stitch them together into one long image for comparison. This is certainly possible, but it can be pretty finicky.

HeadSpin -

Making visual testing robust and maintainable

Thankfully, we can solve moderately much all of these challenges. My favored way to handle them is with a service called Applitools, which specializes in exactly this kind of optic testing. Let & # x27; s see how I can eliminate a bunch of code from the last article and at the same time amend the quality and developer experience of our visual testing.

First, I have to ensure that I have the Applitools Eyes SDK set up in my project:

testImplementation group: 'com.applitools ', name: 'eyes-appium-java4 ', edition: ' 4.2.1 '

Now, I can adjust my setUp and tearDown methods to manage the state of the Eyes service, represented by a class battleground nominate eyes:

@ Before public void setUp () throws Exception {//detonator and driver instantiation// set up Eyes SDK eyes = new Eyes (); eyes.setLogHandler (new StdoutLogHandler ()); eyes.setApiKey (System.getenv (`` APPLITOOLS_API_KEY ''));} @ After public void tearDown () {eyes.abortIfNotClosed (); if (driver! = null) {driver.quit ();}}

Here I am just telling Eyes to log its output to stdout so I can say it from the console, and likewise registering my API key (which I & # x27; ve store as the value of an surroundings variable) so that the service knows to associate checks and screenshots with me and not someone else.

Now, for each discrete visual test session, I ring eyes.open and eyes.close, like so:

eyes.open (driver, `` TheApp '', `` appium pro basic designing test ''); // actual functional test logic and visual chit go here eyes.close ();

Notice that this is where we really afford the eyes object access to our driver, which is how it will be able to tell Appium to direct screenshots at the appropriate multiplication.

Check out:

In the previous article, we had implemented a method for performing visual checks, called doVisualCheck. Now, we can completely delete that entire method. Instead, any time we need to do a visual assay base on the current screenshot, we call eyes.checkWindow. Like our own method, it takes a parameter which is the gens of the check. So, here & # x27; s how our actual test method could look, mixing Appium & # x27; s functional commands with Applitool & # x27; s visual cheque commands:

@ Test world void testAppDesign () {eyes.open (driver, `` TheApp '', `` appium pro basic plan exam ''); WebDriverWait wait = new WebDriverWait (driver, 5); // delay for an element that 's on the abode screen WebElement loginScreen = waitForElement (delay, LOGIN_SCREEN); // now we know the habitation blind is loaded, so do a optical chit eyes.checkWindow (CHECK_HOME); // nav to the login screen, and wait for an element that 's on the login screen loginScreen.click (); waitForElement (wait, USERNAME_FIELD); // execute our 2d visual check, this time of the login screen eyes.checkWindow (CHECK_LOGIN); eyes.close ();}

That & # x27; s it!

Optical testing in activeness

To see how this new approach addresses some of the challenges, we & # x27; ll need to actually run the test using two different version of the app, to sham two builds occurring at different points in time with app changes having occurred between them. I have upload two versions of The App hither:

String APP_V1 = `` https: //github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-VR-v1.apk ''; String APP_V2 = `` https: //github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-VR-v2.apk '';

If we first run the tryout with APP_V1 as the value of the app potentiality, the examination will pass. (And if you want to follow on, recollect that thefull code for this articleis available). That & # x27; s great! If I wanted, I could log into the Applitools dashboard and see that baselines have be captured from the run. But at this point, nothing interesting has happened because we & # x27; ve just set the baselines. So let & # x27; s make things more interesting, and swop the app potentiality to APP_V2, and run the exam again. This clip, I & # x27; ll get a failure, and the next output:

Edition099_Visual_Testing_2 > testAppDesign FAILED com.applitools.eyes.exceptions.DiffsFoundException: Test 'appium pro basic design test ' of 'TheApp ' detected differences! See item at: https: //eyes.applitools.com/app/batches//at com.applitools.eyes.EyesBase.close (EyesBase.java:793) at com.applitools.eyes.EyesBase.close (EyesBase.java:735) at Edition099_Visual_Testing_2.testAppDesign (Edition099_Visual_Testing_2.java:81)

That & # x27; s actually a pretty handy error message. We even get a tie-in so we can straight hop into the web app and start exploring the differences that were found. Once I log in, I can see the resolution of the most recent run:

It & # x27; s tag as & quot; unresolved & quot;, and sure enough there are some manifest deviation. I can tick on the initiative departure, and I & # x27; m greeted with an interface that allows me to see the differences in more detail:

Notice first of all that there are no twist position bars or clock time in these picture. They don & # x27; t matter to the app and so they aren & # x27; t included at all. With this visual divergence, it & # x27; s fairly easy to tell what happened -- the second version of the app impart some list items! So this is not really a fixation, just a deviation that & # x27; s actually fine. I can let the tool know that this is all nominal by click the & quot; thumbs up & quot; button at the top rightfield.

The next dispute is a little more filthy:

We can see hither that a push has moved up. This doesn & # x27; t look as good, so it & # x27; s plausibly a visual regression. I can therefore click the & quot; thumbs down & quot; button, and the check as well as the overall visual test will be marked as failed. This is good because it accurately reflects the state of the app & # x27; s quality. I can of trend send this failure report straightaway to my design or growth team. Once they provide a new build, we can re-run the exact same trial code, and verify that the new adaptation is visually similar enough to the original baseline.

Also check:

That & # x27; s all there is to it! Now that I & # x27; m done walking through all the detected dispute, I can simply relieve my choices, and Applitools will view the new home screen image as the baseline for all succeeding comparability. Stay tune for the following and net instalment of this series, where we research some advanced techniques that help clear even more of the problematic subject we described above about our original framework.

4 Parts

Connet Now