This is the 2nd of a three component serial by Matthew Heusser, software delivery consultant and writer. 

When I start to consider about prove, I consider about it in two broad strokes: new feature testing and release-testing. New lineament testing tries to find problems with something new and specific, while release-testing happens after “ code complete ”, to make sure the whole system works together, that a alteration hither didn ’ t break something thither. Release-testing (which some call regression test) slows down the step of release and delays feedback from our client. Release-testing also increases cycle clip - the time from when we begin act on a feature until it hits production. Over time, as our software become more complex, the amount of try we want to do during release quiz go up.

Meanwhile, teams want to transport more often, to tighten the feedback grommet.

Today I am going to utter about making freeing prove go away - or at least drastically reducing it. It all starts during that tutorial in Spain I wrote aboutlast time.

Two Worlds

The frequence of release for the citizenry in my tutorial was really diverse, but two groups really struck me — the telecom that had a four-month test-release cycle, and the Latvian software squad with the capableness to deploy to production every single day.

That means arrive at the role the morning, seem at automated test runs, and making a determination to deploy. There is a ‘ formula ’ to make this possible. It sounds simple and easy:

• Automate a large figure of checks on every build

• Automate deploy to product

• Continuously monitor traffic and logs for errors

• Build the capability to rollback on failure

That transforms the role of test from make the “ testing we always do ” to looking at the risk for a given freeing, trace it up against several different test strategies, and balancing risk, chance, reward, and time place in release-testing?

The trick is to stop looking at the software as a big box, but alternatively to see it as a set of components. The classic set of components are large pieces of base (the contour of the web host, the connectedness to the database, hunt, login, payment) and the things that sit on top of that - product reviews, comments, static html page, and so on. Develop at least two de-ploy strategy — one for audited and mission-critical systems (essential infrastructure, etc) and another for components and add-ons.

SUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.

We ’ ve been doing it for years in large IT system, where different systems feature different liberation cycle; the trick is to part up existing system, so you can realise and make low-risk change easygoing.

This isn ’ t something I dream up; both Zappos and Etsy have to pass PCI audits for financial services, while Zappos is part of Amazon and publicly traded. Both of these brass feature a sophisticated test-deploy process for parts of the application that touch money, and a simpler process for lower-risk changes.

So cleave off the system into different components that can be tested in isolation. Review the changes (perhaps down to the code level) to consider the impact of the change, and test the appropriate amount.

This can liberate up developers to make many tiny modification per day as long as those changes are low endangerment. Bigger changes along a theme can be batched together to preserve testing time — and might mean we can deploy with still considerably less testing than a ‘ full ’ website retest.

But How Do We Test It?

A few years ago, the ideal vision of getting away from manual, documented test event was a single ‘ test it ’ button compound with a thumbs up or down at the end of an “ automatise tryout run. ”

If the risk is different for each release, and we are uncomfortable with our automation, so we actually want to run different trial for each release — exactly what thinking testers (so, anyone on the team) can do with exploratory testing.

So let the computers furnish some machine-driven checks, all the time. Each cockcrow, maybe every half an hour, we get a story, look at the changes, and decide what is the right thing for this re-lease. That might mean full-time exploratory testing of major features for a day or two, it might be e-mail the team and inquire everyone to spend a half hr testing in product.

This outcome is grown up package testing, varying the test approach to poise risk with cost.

The first step that I talked about today is separating components and developing a scheme that changes the examination effort based on which parts were changed. If the risk is minimum, so deploy it every day. Hey, deploy it every hr.

This recipe is not charming. Companies that try it find engineering challenge. The first build/deploy system they write tends to become hard to maintain over time. Done improper continuous examine creates taxonomical and organizational risk.

It ’ s also a hard sell. So let ’ s talking about ways to change the system to shrink the release-test cycle, deploy more oft, and reduce hazard. The small improvements we make will stand on their own, not jeopardise anyway — and allow us to quit at any clip and announce victory!

A Component Strategy

When a company like etsy.com says that new programmers commit and push code to production the first day, do they genuinely mean modifications to defrayment processing, search, or display for all products? Of course not.

Instead, programmers postdate a well-written set of directions to … wait for it … add the new user to the static HTML ‘ about us ’ page that lists all the employees, along with an image. If this modification generates a bug, that will probably result in an X over an icon the new hire forgot to upload, or possibly, at worst, break a div tag so the page mis-renders.

A bad commit on day one looks like this - not a bungled financial transaction in production.

How much testing should we have for that? Should we retest the whole site?

Let ’ s say we design the push to product so the ‘ push ’ only copies HTML and image files to the webserver. The host is never ‘ down ’, and serves complete pages. After the transposition, the new page appears. Do we really demand to give it the full monty, the week-long burn down of all that is full and right in quiz? Couldn ’ t the developer try it on a local machine, push to stag-ing, try again, and “ just push it? ” Questions on how? More to come.

By Matthew Heusser – matt.heusser @ gmail.com for Sauce Labs

Stay tuned next workweek for the third part of this mini serial! You can postdate Matt on Twitter at@mheusser.