How do you know if you are wasting your clip? What exam are effective and which ace are done because “ That ’ s the way we do things ”?

Tracking bug metrics can bring testing hole to light.

You cognize that lineament you worked on, you be so proud of it. It was all bracing and shiny like a red Porsche:

And when it eventually hit the app store, your valuation end up looking like this:

You ask yourself:How did it happen? Why are the customers finding so many bugs? What are we missing?

It sounds like you ’ ve got some holes in your testing!

The Proof is in the Pudding (or Data)

You ’ ve probably heard that the definition of insanity is doing the same thing and expecting different results. Have you ever mat like you are on a ontogeny team that practices this? While change can be hard to accept, it can be even hard to implement. How do you convince citizenry they need to change?

That ’ s flop! It ’ s all about the data. It ’ s moderately hard to argue when the facts are star you in the face. Being able to force valuable metrics from your bug tracking tool is a unproblematic, potent way to hear where alteration is needed.

Sources of Bug Tickets

It ’ s important to understand the different manner bugs are discovered for your product. As we ’ ve noted above, client are decidedly a source. That ’ s one.

Let ’ s looking at a simple development living cycle of a feature and see what else could activate a defect report:

• First, you have your developer, writing the code, and writingunit and integrating tryoutwhich flag issues during continuous integration.

  SUSA automates exploratory testing with persona-driven behavior, catching bugs that scripted automation misses.

• When narration are consummate they might go through apattern audit and a smoke tryoutby the QA team.

• The team is also writing extraautomatize scripts, such as UI tests against mutual workflow to catch succeeding regressions.

• When the feature is complete, extraexploratory testsare run by QA, plus maybe a large radicalBug Bash (see below).

• Finally, the feature is released to theclients.

Make Use of Your Bug Tracking Tool

Ok, so now we know when bug ticket are created. Are you tracking this? Why not?

I worked on a project that had a heavy investment in automated testing — so much so that it took the largest chunk of our budget. Our mantra was Everything Must Be Automated. We were successful — much every test was automated. But it didn ’ t seem to involve the lineament bottom line from our client ’ s perspective.

So, we start perform a deep analysis of our bugs. Luckily we had a mandatory battlefield in our bug ticket for the Issue Source, which identified where in the lifecycle the bugs were found. By running a story showing the percentage of bugs ground per stage, we could well see the most effective source of bug identification. This simple field allowed us to discover that our automation was not supporting its own weight. Why would our heavy investment provide the last-place return?

Well, basically it forced us to re-evaluate our mechanisation scheme. We determined where our tests were rickety designed and implemented new best practices to resolve them. (Stay tuned for another blog about the real values of the metrics analysis.)

Having had the baseline of effect thanks to the reports based on the Issue Source, we were capable to monitor the percent of glitch report and determine whether our new practice were successful.

Is a New Tool Always Successful?

A great side welfare of utilise metrics is the power to determine if a new process works.

I mentioned the Bug Bash as an Issue Source field option. My product development radical is constantly reviewing the up-to-the-minute testing trend to bring new ideas to the team. You might have say a blog theme byAshley Hunsberger about Bug Bashes. This is where you dedicate a twosome of hour for the whole ontogeny radical, including designers, developer, testers, and anyone who desire to join in, to pound on a feature.

We tried out the Bug Bash as an experiment. The first time, it turned out to be the well-nigh successful tool in the QA tool box. Once we implemented it on a regular basis, we be able to use the metrics from the results to mold when it is most useful in the procedure, and what tweaks we might make to keep it successful as a practice.

Know Thyself

Get to cognise your metrics. They are powerful.

Knowledge is Power!

Joe Nolan is the Mobile QA squad lead at Blackboard. He has over 10 years of experience lead multinationally located QA teams, and is the founder of the DC Software QA and Testing Meetup.

Products

Global tools

Set up and integrate

Resources

Company