Review of Visual vs. Functional Testing with pDiff and Applitools

Sauce AI for Test Authoring: Move from intent to execution in min.|xBack to ResourcesBlogPosted December 3, 2015

Review of Visual vs. Functional Testing with pDiff and Applitools

Update: Sauce Labs recently assume Screener. To register your interest and memorize more please fill out the form on ourVisual Testing page.

As development teams eliminate manual testing and focus only on test automation, everyone lives and dies by automated testing. It is quotable, faster (than manual), and should be prevent bugs, flop? How does your automated examination reportage compare to manual testing? You hold more automated exam for Unit, API, Integration, and Functional UI. But the forgotten test coverage is ofttimes Visual quiz! Typically, automated testing does not extend chit for visual components of your covering.

I have found that teams writing automated Functional UI tests do not understand the departure between these and ocular tests. Do you translate the difference? Your functional coverage could be that you & # x27; ve identified the master workflows in the app, and optical coverage validates how specific visual components appear in the app. Or it could be more. Without a clear picture of when to use what and why, you could be lose out, or even wasting time. It is important to disunite the two levels of tests. Don & # x27; t flux the two.

What is Visual Testing?

The goal of visual testing is to catch unintended optical bugs when update UI factor. This activity verifies that the UI appears correctly from the user ’ s perspective. There is no way to cognise that a visual component that has been updated breaks by just running a functional machine-driven test. Here is my initial checklist when defining optical test coverage:

Each UI component appears in the right color, shape, position and sizing
Ensure that a component doesn ’ t hide or overlap any other UI elements
Antiphonal content at different viewport sizing to ensure changes made at one screen size won ’ t break a layout on another screen size
No broken images

What is Functional Testing?

The destination of functional testing is to extend all workflows of the system by fully practise the GUI itself. Here is my initial checklist when delimit Functional UI test reportage:

Form substantiation, i.e. all required field
Navigation
Links on a page
Search and expose the correct resultant
Pop-up or confirmation message
Sorting
Create, edit, delete, and update (CRUD) tasks
Javascript is properly working in all browsers
Negative

To expand, check out thiscomplete web application checklist.

Why do you need BOTH?

As software development shops adopt automated functional UI examination, they tend to eradicate manual testing. How often are you changing or tweaking CSS? Today ’ s web and mobile applications are designed with data-rich interface, a responsive layout, AJAX, Javascript or complex UI. It is critical when changes are get to action both visual and functional tests to control everything works with browsers, different screen resolve, and responsive designs.

For autonomous testing across multiple user personas, check out SUSATest — it explores your app like 10 different real users.

Browsers. Vendors are pushing update automatically and often.
Screen Resolutions. People are browsing web covering from a variety of devices (mobile, tablet, desktop) for non-responsive applications.
Responsive Designs. Content at different viewport sizes ensures alteration made at one screen size won ’ t break a layout at another blind size.

Functional modification are easily observe. One UI component change can have unexpected consequences, making visual mistakes hard to spot with only automated Functional UI tests. Visual examination is as significant as Functional UI essay. Users present are using all types of devices to access web applications. So testing ocular components and functional workflows is important because the user experience on nomadic devices is quite different from background.

Tool Overload

It turns out that there are a lot of Visual screen model available. They all work in different ways — some guide screenshots while others aren ’ t even visual (CSS attributes). Of the many fabric out there, you ’ ll find, nigh notably, DPXDT, Applitools, and BackstopJS.

Applitoolsis a cloud-based automated visual screen solution that mechanically validates all the visual aspects of web, mobile and desktop apps.

Cloud-based
Provides native SDK for development and testing
Supports multiple browsers and multiple device
Commercial

DPXDTDPXDT(pronounced Depicted) is a tool that liken before and after website screenshots for each release. It shows when any visual and/or perceptual differences are found.

From Google, written in Python
Support of PhantomJS by nonpayment
Restful API style of development and testing
Report support manual diff confirmation
Free and open source

Others

To satisfy your missing Optic test coverage, I recommend checking some of these out to see if they fit what you ’ re do in your application. What do you require out of a Visual testing model? I personally want to liken between iteration or freeing to test our CSS, layout, art and other visual alteration on our application.

Let ’ s see it in activity

The perceptual testing fabric compare between loop or releases — the new loop screenshot (arrange surround) compare against the golden screenshot (production environs). The instrument execute a graphical comparison by return rendering differences, not utilise the traditional pel comparison method. The release manager still needs to manually inspect all difference flagged as failures. The manager will confirm if miscarry dispute are unexpected or pass them as require changes (make a new prosperous image). Takeaways As you can see, visual tests are an important part of test coverage. Every prove strategy should include visual testing. It is critical to coach team members on the difference between Visual and Functional UI examination.

Greg Sypolt (@ gregsypolt) is a senior engineer at Gannett and co-founder of Quality Element. He is a passionate automation engineer attempt to optimise software development quality, while train team appendage on how to write great mechanization scripts and helping the testing community go better testers. Greg has spent most of his career working on software caliber — concentrating on web browsers, APIs, and mobile. For the past five years, he has focused on the creation and deployment of automated tryout strategies, frameworks, tool and platforms.