Autonomous Software Testing Explained: A Benchmark for AI-Driven QA

We 've been believe long and hard about software testing and mechanization here at Katalon. We 've comment a shift in the Software Testing industriousness from Test Automation toSelf-directed Software Testing (AUST). By `` autonomous, '' we mean software testing that is completely drive by Artificial Intelligence (AI) and Machine Learning (ML), without any human intervention. & nbsp;

We figured the industry shift from Test Automation to AUST could either be messy — a sort of Wild West approach to product festering — or goal-driven, something to which the entire Software Testing community could aim in a spirit of cooperation. That 's where self-driven auto — or self-reliant vehicles, as the mobility industry wish to telephone them — come in. & nbsp;

The measured advances in autonomous vehicles have inspired us to gather a way to benchmark advancement toward autonomy in the package testing arena. We call the benchmark the Autonomous Software Testing Model (ASTM). & nbsp;

We wanted a way to gauge just where software quiz platforms are in their development of `` self-driving '' software testing — Katalon included. We seem to theSociety of Automotive Engineersfor guidance on build an ASTM benchmark.

Building a software testing benchmark

The benchmark for the maturation of autonomous vehicles that the Society of Automotive Engineers (SAE) created observes that humans will be regard in the evolution of the technology as a matter of course. In other lyric, autonomous vehicle will not dead pop out of a black box overnight. Instead, humans will be involved with developing, grooming, and testing the engineering for some time to come.

The ASTM framework, so, sets out six stages of human-machine desegregation. Zero (0) represent full manual plan and effectuation of testing procedures. The highest level, Six (6), sees full AI/ML designing, implementation, and decision-making of test:

Level 0: Manual Testing

The human performs all activities and actions and makes all decisions concerning examination of the System Under Testing (SUT). & nbsp;

Flat 1: Assisted Test Automation

The human do most actions for testing the SUT with the support of the computer, e.g., automated package tools. The calculator conduct out sure examine actions automatically under the human ’ s control. The human holds the master character and makes all testing decisions. & nbsp;

Level 2: Partial Test Automation

Both the human and the computer perform testing activity and generate possible decision options, but the human yet do most testing decisions. The computer carries out testing activeness based on the decisions made by the human. & nbsp;

Flat 3: Integrated Automated Testing

The computer generates a listing of decision options, selects an option, and performs actions based on this option if the human approves. The human can also select another decision option for the computer to take out. & nbsp;

Flat 4: Intelligent Automated Testing

The calculator generates decision options, ascertain the best option, and performs testing action according to this choice. The human can withal intervene if necessary. & nbsp;

Unwavering 5: Autonomous Testing

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

The calculator has total control of the testing operation for the SUT, including making decision and carrying out all testing actions. The human can not interpose.

The software testing lifecycle

We felt we needed to ground the ASTM still more soundly in the software testing cycle. Defining each stage of software test would enable us to apply the benchmark to software testing platform on the marketplace. We identified the nine activity areas that define software testing as:

1. Test Management (TMN)
2. Test Orchestration (TOR)
3. Test Maintenance (TMA)
4. Test Generation (TGR)
5. Test Optimization (TOP)
6. Test Monitoring (TMO)
7. Test Execution (TEX)
8. Test Management (TMN)
9. Test Evaluation and Reporting (TRE)

Self-reliant software testing model self-assessment

We applied the ASTM benchmark to Katalon. & nbsp; We nock the Katalon functionality on a spider diagram to gauge the stage to which Katalon has develop its autonomous lineament.

In particular, Katalon rates 3rd-level sovereign activity in the areas of Test Orchestration (TOR), Test Maintenance (TMA), Test Monitoring (TMO), and Test Execution (TEX). The benchmark imply that while the can give a list of decision-testing pick for these activities, the system still postulate human approval to act on the alternative; or, alternatively, the human can select another decision option for Katalon to act on. & nbsp;

If you are a Katalon user, where would you place our platform 's grade of self-reliance? & nbsp;

If you would wish to learn more about how can:

• velocity up software deployment
• increase the accuracy of your package prove try
• trim overall software development cost,

then contact us for a demo. & nbsp;

Download our academic paper entitled for a fuller description of the ASTM benchmark. The paper also includes:

• Full descriptions of the software screen lifecycle activity
• Charts that cross-reference ASTM stages with standard testing activities
• A look at how the ASTM benchmark gauges former package testing platforms
   

More to say:

• [Download free ebook]How to shift from manual to automated testing?
• The power of a examination platform
• Rethinking Test Automation to Address Business Challenges