Automated exploratory testing

Automated exploratory testing is a method integrating test automation within the Exploratory Testing Session that enables the testers to save money and to become more productive during test execution. I recently exercised this approach in 2 major clients. In the first case, no automation existed in the specific project so we used a 3rd party tool that can record all tester’s actions during test execution and then analyze the log file afterwards and even use it for future regression test. This method is called Passive Automated Exploratory Testing (Passive AET). On the second case, a KDT approach was used. We used the KDT infrastructure to automate the test execution. This method is called Active Automated Exploratory Testing (Active AET).

Passive Automated Exploratory Testing (Passive AET)

Passive exploratory test automation is a method in which specific tools are installed on the testers PCs that record each one of the testers actions performed during testing. This method can be performed in pair exploratory testing or by a single tester. The execution of the tests is performed in the same way as manual exploratory testing. The only change is that at the end of the test session, an additional task is being scheduled for designing the test results based on the recorded session for reporting purpose as well as making the recorded session reusable. In order to complete the passive automated exploratory approach – video capturing tool is also being used during the session – enabling the tester to save more time and money on the bug reporting stage. Reporting bugs via video files has it benefits regardless to this article's specific issues such as: • A simple and clear step by step view of the scenario until the bug appears • Eliminates 90% of bug irreproducible debates between the developers team and the testers • Enables Old bugs to stay valid despite resource changeover during the project • Replace the need for endless descriptions of system state, • Captures the exact response time of the system at a specific moment – a fantastic way for performance test at a low cost testing tool. By reducing the time and effort dedicated to report a bug – the exploratory testing is now fully automated and powerful way for running, documenting and reporting bugs.

Active Automated Exploratory Testing (Active AET)

Active Automated Exploratory Testing integrates the KDT approach with the Session Execution. The implementation of the Active AET is recommended for pair testing. Usually, the first tester is responsible to create the automated test script while the second tester is responsible for the execution.

The initial activity of the test execution is to execute all existing regression automated test scripts (if this session was already executed in the past). At the end of each session, a closure phase should take place with the following tasks: - The first and second tester should switch rolls so the first tester (who created the automated test scripts) should re-execute all the tests to ensure robustness and reliability. - A definition which one of the automated test scripts should be used for regression tests. The main criteria used are: >>Coverage – does the automated test script simulate a situation that is generic to the application performance? >>Uniqueness – does the automated test script simulate a situation that is less likely to be performed by an exploratory tester during the next test sessions? >>Existence – does the same or similar automated test script already exists in the automated test repository. -For each one of the automated test scripts, high level description should be made (usually the same syntax used for definition of the test log in manual test script) and other characteristics used for definition of the test.

Main benefits of AET

The main benefits of AET compared to manual exploratory testing are: - Analyze Test – AET allows better analyze of the tests - Reproducing bugs – sine in ET tests are not planned in advance, in many cases bugs are found but the tester is unable to reproduce the activities that led to the bug. Using AET ensure better reproduction of the bug. - Enhance Test Coverage – since at the beginning of each session, the existing test automated scripts are being executed, the coverage of the tests is increased. - Save time – when using passive AET, the same tasks formally performed by 2 testers can be performed by one tester while the session duration increases only by 50% and even less. - Providing assurance to the stakeholders – using AET provides assurance to the stakeholders that all activities are being documented in a level that is not less than the level used for scripted testing. For one of my clients, using AET was the only way to convince him to use ET.