Reduction-based methods and metrics for selective regression testing. (c2000)

Abstract

Selective regression testing attempts to choose an appropriate subset of test cases from among a previously run test suite for a software system, based on information about the changes made to the system to create new versions. In this thesis, we address two major problems in selective regression testing: the regression test selection problem and the coverage identification problem. To address the former problem, we propose three reduction-based selective regression testing methods that reduce the number of selected test cases for retesting the modified software by omitting redundant tests from the initial test suite. But, one method, referred to as Modification-Based Reduction version 1 (MBRl), selects a reduced number of test cases based on the modification made and its effects in the software. A second method, referred to as Modification-Based Reduction version 2 (MBR2) improves MBRI by omitting tests that do not reach the modification. A third method, referred to as Precise Reduction (PR), further reduces the number of test cases selected by omitting all non-modification-revealing tests from the initial test suite. To approach the latter selective retesting problem, we suggest two McCabebased regression test selection metrics that could be also extended to address the test selection problem. These metrics are the Reachability regression Test selection McCabe-based metric (RTM) , and dataflow Slices regression Test McCabe-based metric (STM). The suggested metrics help in monitoring testcoverage adequacy, reveal any shortage or redundancy in the test suite, and assist in identifying where additional tests may be required for retesting. We empirically compare MBRl, MBR2, and PR with three reduction and precision-oriented methods on 60 test-problems. The results show that PR selects the least number of test cases most of the time and omits nonmodification- revealing test cases all the time. We illustrate a typical application of our suggested metrics using the 60 test-problems on two coverage-oriented selective regression testing methods.