Welcome to our series of articles introducing you to tried-and-true practices and design that help team make worthful automated trial. Your team can utilise these basic concepts to build up tests that quickly inform you about the outcomes of the latest alteration to your package product, with a sensible investing of time.

Knock-down Test Automation Practices

Part 2: Great Expectations

In, we dreamed of the outstanding test case ever, and name elements that it would assure with assertions to provide that all-important feedback: is our application behaving as want? We examined how to apply full testing skills into the context of automation. Now, we ’ ll conduct a look at how to use those skills to formulate assertions that optimise feedback. Once you hold started to consciously translate your natural scanning into detailed observations, you ’ re ready to make assertions.

A story of statement triumphing over regressions

Let ’ s say, by some random happenstance, that we wanted to test an app telephone. What would be the initiative thing you see? That would be the login blind.

Where to start

First, let ’ s design what we want to screen. The login page is a key piece of user experience. Our team wants confidence that code or configuration modification don ’ t cause any important elements to go lose or functionality to break.

As a tester, you probably part by thinking what ’ s important about this page. When you search it manually, you ’ re likely to look to see all the expected elements look correct, and that the most valuable functionality is present. The login capability is the page ’ s main purpose. There are four different shipway to login: username/password, Google, GitHub, and single signal on consolidation. There are numerous failure scenario for login, but we ’ re not locomote to address those in our first set of tests. yet.

Additionally, the page has marketing and branding elements that are important to the companionship. Our team worked difficult to decide on the brand colors, company logo, and messaging to make the page both inviting and functional. Another key component is the picture in the bottom left corner that lets people get quick helper via chat with client support staff. We want to make sure all of these branding elements are present.

Prioritize on value

We ’ ve resolve our priorities, we ’ ll start by verifying the following:

• The customer support chat and marketing/branding elements are present

• The “ happy paths ” for several manner of logging in work aright

We ’ ll address more feature of good tests in a later post, but for the time being, let ’ s get a destination of creating tests that are short, that essay one characteristic or panorama, and that are real intentional in their purpose. We ’ ll apply automation patterns such as“ one open purpose ” , “ keep it simple ”, and “ maintainable testware ”. We require assertions that cause a exam failure when an unexpected outcome occurs, and that help document the expected behavior of the user interface. We desire to deflect assertions that fail for reasons that we don ’ t attention about, for things that don ’ t affect value to our customers.
 

Balancing averment posture and test maintainability

Potent, specific assertions may give us definite feedback that we haven ’ t do fixation failures. However, especially for teams practice continuous delivery with frequent production deploys, web application pages are likely to change frequently in minor agency, such as rewording messages, changing colors or appearance that doesn ’ t affect functionality. The more specific we get with our affirmation, the more probable they are to neglect tests mistakenly, because of a change that might not be important to us. & nbsp; However, if our assertions are too loose, the tests may signal everything is fine when in fact a regression failure has occurred. Let ’ s look at some of the options along with guideline on when to use them.
 

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

Equality versus Contains assertions

A common format for averment is to compare the expected outcome, such as a text thread value, with the literal outcome. For optimum test reliability, find the nonindulgent condition that doesn ’ t over-specify.

In the following representative, we want to make sure there ’ s a friendly welcome message on the login page. We started with an equality affirmation to verify the exact schoolbook of the substance:

Next workweek when the designer decide to re-word the substance to “ Welcome! Let ’ s get this company started! ”, the test will fail. If the value of the test is to make certain there IS a welcome message in the login page, regardless of its specific test, a less fragile approach would be to use a “ contains ” assertion:

As you make assertions in your UI regression exam, reckon what might be tested more efficiently in a lower-level test. The specific text value for a message could be verify in a unit-level regression test which become vary anytime the message text is changed in the code.

UI tests tend to be brittle, a subtle modification can cause failure that we spend clip investigating, only to find it was a unauthentic, unimportant failure. It ’ s a full practice to write UI tests in the least specific way.

Assertions that work in different environments

Ideally, our tests can run in any environs we have, including topically on a developer ’ s machine, in a exam environment, in staging, or production. Use assertions that indorse this whenever potential. We also want to find the bare, most straightforward way to aver that a condition we want to check is true.

For instance, we desire a sanity check at the start of our exam that chit marketing and stigmatize elements to create sure we really are on the login page. We can do a simple averment that the domain name in the URL is right, without include the entire host name: & nbsp; That exam is valid for other environments as long as we use the same naming formula, such as “ test.mabl.com ”.

To make certain our app is serving a secure connector, we prefer a more specific assertion:

Here ’ s another example of supporting multiple environs with a less-specific but still useful averment. We want to end our felicitous path login test by lumber out and verifying the login page charge up. Again, by asserting on the apt part of the URL, we can get accurate feedback no issue where we run the trial, be it dev, test, staging or production:

Other pliant but utile types of assertions

A variation on “ contains ” is “ starts with ”. Our test to login with GitHub clicks the “ GitHub Login ” button and asserts that indeed the GitHub login page has load. We don ’ t require to assert on the full string, which contains values that may modify, so we assert on just the get-go of the URL. The test output prove the entire value, but the assertion is but done against the starting value ofhttps: //github.com/login.

If we only want to make sure a particular button or ikon is on the page, it may be enough to insure that it is exists on the page, without checking for detail such as a text string value. That way our test will keep working if there are stylistic changes that don ’ t affect functionality. We assert its presence.

Optimizing feedback

We ’ ve look at slipway to write assertions that are flexible plenty that they won ’ t identify minor intentional modification such as updated error message text verbalism as regression failures, yet still catch the failure we want to memorize about. And, we ’ ve seen that our tests can easily be indite to run on multiple environs.

Every context is different. If you ’ re writing a exam cause for a something where a particular undetected regression failure would be catastrophic, you ’ ll want a potent, specific assertion.

In our adjacent installment, we ’ ll delve into some common pitfalls to watch for as you craft test averment, such as false positives, double negative, too many expected issue, and other assertion anti-patterns.

Resources

The Way of the Web Tester: A Beginner ’ s Guide to Automating Tests, Jonathan Rasmussen, 2016

Test Automation Patterns Wiki, https: //testautomationpatterns.org