Sure you see your bug story inJIRA, but how do you actually cognise the stage of quality in your apps and processes? Bug counting metrics are a outstanding commence point, but if you really want to know if your team is producing a calibre app and improving their internal procedure, you need to look to former tools to see how your production is trending.

Know Thyself

If your internal processes and test coverage are miserable, most likely this will translate into a poor app. Get your house in order with the following:

• Start by watching your automatise build and test results, use a tool like

• Use a tool likeDatadogto see what you want to mensurate, and proctor course in build failures based on machine-driven examination results, and analyze how long it takes to deploy. You can use this information to regulate unmediated and indirect impacts to the different teams.

• Determine the crucial things you want to include in your CI and Automated Test coverage. Have a pct goal of reporting to endeavour for.

• Monitor your bug backlog. Is it accumulating per release?

• Keep track of your scheme availability for customers. Use a tool likeCrashlyticsto measure who is using the application, real time data about the app ’ s performance and problems.

• Set performance test baseline, using tools such asJMeterto prove your APIs, andCharlesto test your page load times.

• How are the scrum team doing? Are they improving their ability to produce more story points per dash as they get used to work together and with their features?

You should critique the metrics from each of these to determine if you are treading water or improving. Here is a shot of Datadog in activeness on a alive scheme. The names were changed to protect the innocent:

The App Store

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

The easy and most obvious place to see your app ’ s character is through the app store. Just look at the ratings. But ratings don ’ t always state the story. Review all of the comments. Keep a run tally for thing like confident vs negative comments. Also include whether the commentary are requests for new functionality requirements, or actual bug feedback. Compile metrics based on this info based on liberation edition, and use it to spot trends. Did some features get more negative feedback than others?

App Feedback

Along the like lines as the feedback found in the app shop, my team has a built-in feedback feature for users. This allows users to directly send comments, questions, and supply them to the team. While much of this is used by the client support squad, the QA team reexamine each of those and demonstrate the like type of prosody gleaned from the app memory.

User Forums, Twitter and Reddit

If your company has a exploiter assembly where you can see the conversation, you should take time to supervise them. Are there blogger out there reviewing your product? Follow them. This is where you will find the super users who can truly dig out the issue, from serviceableness to bug, while besides supply useful tips to improve. Make use of the societal media sites like Reddit. It only conduct a couple of users to disseminate a negative post/tweet to expose the quality of the app. Or, you can use a tool likeBrandwatchthat monitors societal feeds and you can get a feel for the proportion of confident to negative feedback.

Comparison to Your Competitors

Think your app is doing well based on the feedback? Good valuation in the app store? Check out your competitors. Not merely can you do a good side by side comparison, but you can likewise survey the comment on their pages to see if people are say ‘ much better/worse then x ’.

Client Support

Your Client Support squad has the best pulse on the app. They are inaugural to be hit with customer feedback, though unluckily it ’ s mostly negative at their point. But they can provide prosody as to the bulk and criticality of calls and issues they receive. They hold their fingers on the pulse of the app.

Bug Bash Feedback

I ’ m saving my favorite for last — bug bash feedback from your interior teams. In Ashley Hunsberger ’ s blog postTesting from a Different Point of View, she discusses bug bash, or as she calls them, “ Exploratory Testing on Steroids. ” Bug bashes are explorative test charters where both members from your scrum squad, plus anyone else (like client support) can join in and run examination during a large-scale grouping test. These tests are outstanding because they show you bugs that all of your other QA processes experience missed, plus call out things like usability. (As a fillip, these teams take early training and visibility to the app!) They are most good when you set up a survey to be completed immediately after the exam. Keep it simple, and use evaluation scales:

• How would you pace the app ’ s quality?

• Do you think customers will like the features?

• Is the app ready for freeing?

• If you be valuation the app in the app store, what would you give it?

• Allow for commentary.

Make this anonymous and required. You may be surprised by the feedback.

Watch the Trends!

Pull all of this information together. Assign rating systems for each to give you a Quality number. Create uncomplicated dashboards to show the trends, and that countenance you to practise down. Use your imagery! I ’ d love to hear what other methods you use to track the character of YOUR apps.Joe Nolan (@ JoeSolobx) is a Roving QA Team Manager with over 12 geezerhood of experience result multi-nationally located QA teams, and is the founder of the DC Software QA and Testing Meetup.