This invention is related to the field of optimizing test suites in software regression testing; more particularly, the invention presents the systematic 5 approach to minimize and order the test cases in a test suite which in turn reduces the cost and effort requisite in the testing process.
BACKGROUND & PRIOR ART
Testing is an important and critical phase in software development life cycle. It is 10 intended to determine whether the software program/product behaves as expected or not before their release. According to the testing results, debugging is performed by the developers to find out the cause of the errors and consequently remove it. Sometimes, corrections require certain modification/changes in the existing program which may introduce other errors that cause the existing system 15 to stop behaving as expected. So, after modification, there is a need of revalidation i.e. regression testing.
Regression testing involves the creation of new test cases to examine the new or modified parts of the software. Progressively, a test suite growth increases which make it expensive to execute the entire test suite and the addition of new test cases 20 in the existing test suite makes some test cases redundant or obsolete as well. Thus, it is beneficial to optimize the test suite by removing the identified invalid test cases which, in turn, can decrease the cost and effort required for effective regression testing. To optimize the result, various regression testing techniques such as test case selection, minimization, and prioritization techniques have been 25 projected. But, only some of the prior work did utilize the combination of these techniques to optimize the test suite in terms of their size as well as their effectiveness. Combining these techniques in a proper manner with incorporating clustering method may highly improve the test suite effectiveness. While performing optimization, the selection of coverage criteria plays an important 30
3
role. Most of the existing techniques used either one or two of the criteria to generate the representative test suite. The present invention used the multiple criteria with the combination of minimization and prioritization techniques to get an optimal representative test suite. Combining this regression testing techniques rather than using only single testing activity is more useful to produce an optimal 5 result. It would also be advantageous to provide some guidelines and proper roadmap to developers/testers which surely help them in handling different testing situations.
OBJECTIVE OF THE INVENTION
The prime object of this invention is to provide a method and a roadmap to 10 generate an optimal test suite using sequential process with the help of multiple coverage criterion and combination of minimization and prioritization techniques.
BRIEF DESCRIPTION OF DRAWINGS
In order that the advantages of the present invention will be easily understood, a detail description of the invention is discussed below in conjunction with the 15 appended drawings, which, however, should not be considered to limit the scope of the invention to the specific method and system, but are for better explanation and understanding only. The invention feature will be demonstrated with additional specificity and detail with reference to the accompanying drawings, in which: 20
FIG. 1 shows the block diagram or model for test suite optimization. The given steps are in accordance with reference to the present invention.
FIG. 2A-2B illustrates the logical flow diagram of optimizing test suite in accordance with present invention.
DETAIL DESCRIPTION 25
In the following description, various features and a thorough understanding of the present invention are provided herein. This invention provides an apparatus or system to the user to do effective regression testing by optimizing the test suite. A number of test cases are created in the process of testing and after subsequent
4
modification in the software program, additional test cases are generated for regression testing and it results in test suite size increment. The main purpose herein is to produce an optimal test suite with the help of combining one or more regression testing techniques and using multiple coverage criteria as well.
FIG. 1 is a block diagram illustrating one embodiment of a test suite optimizer 5 (100) in accordance with the present invention. The depicted software test suite optimizer (100) includes a test case analyzer module (101), a cluster generation module (102), and the optimizer module (103). Where the optimizer module (103) further comprises of minimization module (104) and prioritization module (105). The various modules of the test suite optimizer (100) cooperate to optimize 10 software regression testing by determining unnecessary test cases via the creation and analysis of a test case similarity matrix. The test case analyzer module (101) takes input as program source code, test cases, coverage information etc. In this model, set of four metrics are used: block coverage, control flow, def-use and data flow coverage to compare the pair of test cases. Selection of these metrics is based 15 on their ability and their relation to the program source code. It is totally dependent on the user’s choice. Based on these metrics, diversity values are generated for each test case pair and by utilizing these diversity values, the test cases are grouped into one or more clusters by the cluster generation module (102). On each cluster, the optimization process is applied according to the user 20 choice. The optimizer module (103) which is comprised of two sub-modules i.e. minimization module (104) and prioritization module (105), responsible for further optimization of test suite by removing the redundant and obsolete test cases as well as by ranking the remaining effective test cases based on their importance (maximum coverage or maximum fault detection capability). The user 25 can choose the order of optimization also. Either they can select minimization first and then prioritization or vice versa and by default former one is selected.
FIGS. 1A-1B illustrates an embodiment logic flow diagram for test case analysis, clustering, and optimization. A test suite containing a number of test cases are designed to validate various features, functions, paths, logic etc. of the software 30 under test (201). Test cases of a test suite are executed on the instrumented
5
software code and the execution profiles or coverage information is gathered (202). With the help of the captured coverage information, the test case coverage matrix is generated for each selected criterion (i.e. branch, path, def-use, data flow etc.) (203). The difference between each pair of test cases is computed with the help of coverage matrix and compared accordingly to identify how much they are 5 similar to each other (204). Similar test cases are grouped into one or more cluster to optimize the process (205) and on each cluster, the combination of the optimization process is performed (206) that is decided by user preferences (207). In an embodiment at decision blocks (208) and (211) a determination is made as to which combination of the optimization process is to be performed. If selection 10 value is chosen as ‘1’, the minimization (209) is performed first and then prioritization (210) will be. However, the reverse is applied for selection value ‘2’ and in the default case (212) the former one will be processed. The result as representative test suite will be stored in optimal test suite storage (213). In an embodiment at decision block (214), a determination is made as to whether any 15 further modification in the software is made by developers or not. If no modification happened, and then stop the optimization process (216) otherwise new test cases are generated and merged with existing one to validate the updated software with new features or corrections (215). Repeat the steps from (202) and so on until no further modifications in the software are made by developers.

CLAIMS
A method for generating an optimal test suite, comprising test case analyzer module to compare and analyze the test cases based on their coverage data, cluster generator module for grouping of similar test cases based on their similarity values, and optimizer module to minimize and 5 prioritize the test cases for effective regression testing.
2. To measure the diversity between pair of test cases, multiple coverage criteria i.e. branch, control flow, def-use, data flow and path coverage is used herein rather than using single coverage criteria.
3. The combination of more than one regression testing techniques i.e. 10 minimization and prioritization are used by optimizer herein to get the minimized and ordered cluster of similar test cases.
4. Clustering process is used to generate more than one group of similar test cases with the help of calculated diversity or similarity values for each pair of test cases, and on each cluster further optimizer is executed. 15
5. The present invention provides a selection feature to choose the order of execution of the minimization and prioritization process, depending upon the user requirements. Either they select minimization first followed by prioritization or vice versa; for minimization enhanced greedy approach is used and prioritization is performed based on maximum unique path 20 coverage information.