Continuous testing

Recently someone pointed me to aTwitter threadwhich had a connection to anarticleabout “ Continuous Testing ”. I was surprise to read this definition of uninterrupted testing in the clause:

Continuous testing is the operation of executing automated tests as component of the package delivery line to obtain feedback on the business risks associated with a software liberation candidate as rapidly as possible.

Well, sure, that is portion of continuous testing, and an important part. But there issomuch more. & nbsp;

After I read that, and looked around some more, I find there are entire record on uninterrupted testing that use the same narrow definition of “ only the automated regression trial in the deployment pipeline ”. And whoever wrote the Wikipedia debut on it likewise correspond with that narrow view.

Not my definition!

I have a really different idea about what appoint “ continuous try ”. In my psyche, “ agile testing ” is the same thing as “ continuous essay ”.Janet Gregoryand I, along with others in the agile testing community,define agile prove this way:

Collaborative testing practices that occur unendingly, from inception to delivery and beyond, supporting frequent speech of value for our customers. Testing activities focalise on edifice quality into the product, using fast feedback loops to validate our understanding. The practices strengthen and support the mind of whole squad responsibility for character.

For a more specific definition of continuous screen in DevOps, I find Dan Ashby ’ sdefinition, and especially his optic model (which many other authors, including myself, Janet andKatrina Clokierefer to in our books), excogitate what my own high-performing teams have cover over the retiring decades. He doesn ’ t have a short definition, but his model, based on the DevOps space loop, expresses it well:

Dan Ashby, “Continuous Testing in DevOps”, 2016

Dan ’ s situation has another great model for continuous testing near the end of that station, showing that our screen starts with testing new feature ideas, and extends into the many former potential type of testing we might do on a package product. And he ’ s come up with another poser too, you can see it in histweet. 

More perspectives

I asked around on a mates of testing community Slack workspaces to see what other people think of when they learn “ continuous essay ”. I wish whatBenjamin Fellowsdescribed on the Ministry of Testing ’ s testersio.slack.com workspace:

I 've taken it to mean no prove phase and instead guide elements from both dislodge left and shifting flop (prove throughout the development and operations process, from testing initial requirements to using customer metrics to further test after release)

SUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.

Matt Lievertzposted his view on the same yarn, and I find this one worry too:

I ’ m not certain if there ’ s a normal definition, but if I heard it I would suspect it touch to either:

1) the essay component of uninterrupted desegregation

2) uninterrupted monitoring in prod — testing in prod continuously as a functionality-level monitor

The monitoring and observability aspects of continuous testing are definitelyvital. 

Others in this conversation said they could see value in both definitions, but they tended to go with the Dan Ashby perspective. & nbsp;

I blogged about my own perspective of continuous testing, based on both Dan ’ s framework and on Jez Humble and David Farley ’ sUninterrupted Testing book. Peter Zimmerer ’ scomment and the linkup he include are worth a read. He did some research on the “ existent essence of uninterrupted quiz ”, and as a result, he uses two “ profound objectives and characteristics ”, which I wish a lot:

Continuous testing means:

1. Testing over the whole lifecycle by shift left and shift flop, i.e. testing continuously from begin to end

2. Strategic exam mechanization, i.e. continuously reuse & amp; adapt & amp; execute (only the) needed test in an level-headed manner

Other names for uninterrupted examination?

When people use the term “ continuous testing ”, their meaning might be quite different from my own. Should we come up with some other condition that ’ s not being used anywhere else yet? We ’ d love to hear your mind on terminology, please use the Twitter button at the bottom of the page to let us know. & nbsp;

Heather Reid part adiscourse on the same matteron the Ministry of Testing Club. I wishSergio Freire ’ s ideato use a different name for “ uninterrupted testing ”, to make it more clear what we imply:

For me Uninterrupted Testing is more Omni Testing (or Omnipresent Testing), in the sentiency that testing is invariably present during the whole ontogenesis living cycle.

Some people may imagine about Continuous Testing as meaning tests are “ continuously ” running; these trial are assumed to be automatize single (i.e. machine-driven checks).

In my sentiment I would prefer Omni Testing as I see testing as something that is always present, no topic what form of approaching you ’ re following or at what phase of your development living cycle you are.

You use testing to guarantee and search a bunch of quality attributes.

Janet Gregory and I pen about uninterrupted testing framework in our approaching new book,Agile Testing Condensed. Janet came up with a term I really like:“ holistic testing ”. & nbsp;

We learn about a feature that our customers may want, we make and deliver it, and we learn how the client really use it. We use that feedback to determine what to build (or remove) next.

I continue to ask citizenry about their own definition of “ continuous screen ”. I ’ m assay to keep an open mind, there is always more to learn. Whatever we name it, prove is not ever divide from software evolution. It ’ s desegregate with coding and many other activities throughout the countless cringle of package delivery. & nbsp;