Abstract: METHOD AND SYSTEM FOR SELECTION OF REGRESSION TEST CASES FOR VALIDATION OF SOFTWARE APPLICATION Abstract of the Disclosure A method and a system for identifying a set of regression test cases from a plurality of test cases to validate an upgrade of a software application are provided. The invention involves recording run-time interactions between the plurality of functions by executing the plurality of test cases on the software application. Thereafter, functions are identified from the plurality of functions in the upgrade of the software application. Subsequently, a set of regression test cases are identified on the basis of mapping between the run-time interactions and the functions.
BACKGROUND
The invention relates generally to the field of software development lifecycle. More specifically, the invention relates to a method and a system for identifying a set of regression test cases to validate an upgrade of a software application.
Maintenance of a software application typically involves modification in its code to meet the customers' requirements. Examples of the modifications include fixing bugs, adding new functionalities, improving the existing functionalities, and the like. After ncorporating the changes in the code, the software application is validated using regression testing. Regression testing includes selective re-testing of the application or ts components to verify if there are any unintended effects of the modifications to the :ode and whether the software application still meets the specific requirements.
Traditionally, regression testing uses the entire set of testing suite on any jpgrade of a software application. Since this process is time consuming, various egression test selection techniques have been proposed. These approaches use static analysis of the source code. Further, some of the selection techniques use the code nformation analyzed at the granular level of the code statements. The granular level analysis involves the complete list of statements that are executed for a given test case. furthermore, software developers, based on their experience and knowledge of the code change specifications that need to be re-executed, intuitively select regression ests for a large and complex software application requiring thousands of test cases.
One or more of the regression testing techniques mentioned above are costly and time consuming and result in delay in delivering software applications to clients. further, the source code is generally not available for analysis such as upgrades to commercial Off-the-shelf (COTS) components. Furthermore, analyzing at the level of he code statement is complex and time consuming.
In light of the foregoing, there is a need for a method for selecting regression test cases that are fewer in number such that the testing is effective. Further, the method
needs to be independent of the source code of the software application without needing analysis at the level of the code statement.
SUMMARY
An object of the invention is to provide a method and a system for identifying a set of regression test cases from a plurality of test cases.
Another object of the invention is to provide a method and a system for validating an upgrade of a software application.
Yet another object of the invention is to provide a method and a system for effectively identifying one or more functions that have been modified in the upgrade of the software application.
To achieve the objectives mentioned above, the invention provides a method and a system for identifying a set of regression test cases from a plurality of test cases to validate an upgrade of a software application. The software application includes a plurality of components. Each component has a plurality of functions. The method involves recording run-time interactions between the plurality of functions by executing the plurality of test cases on the software application. Thereafter, the functions that have been modified are identified from the plurality of functions in the upgrade of the software application. Subsequently, a set of regression test cases are identified on the basis of mapping between the run-time interactions and the identified functions.
The invention provides a fewer number of regression test cases for the testing of the upgrades of the software application. Using fewer test cases for testing saves cost and time, thereby ensuring faster delivery of the software application to the clients. Further, the method described above is independent of the source code of the software application, since it uses an intermediate language code.
BRIEF DESCRIPTION OF THE DRAWINGS
Various embodiments of the invention will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the invention, wherein like designations denote like elements, and in which:
Figure 1 is a flowchart illustrating a method for identifying a set of regression test cases from a plurality of test cases, in accordance with an embodiment of the invention;
Figure 2 is a flowchart illustrating a method for recording run-time interactions between a plurality of functions in a software application, in accordance with an embodiment of the invention;
Figure 3 is a sample Functional Interaction Graph (FIG), In accordance with an embodiment of the invention;
Figure 4 is a flowchart illustrating a method for identifying one or more functions that have been modified in the upgrade of the software application, in accordance with an embodiment of the invention;
Figure 5 is a flowchart illustrating a method for selecting a set of regression cases on the basis of mapping between run-time interactions and one or more functions, in accordance with an embodiment of the invention;
Figure 6 is a block diagram of a system for identifying a set of regression test cases from a plurality of test cases for validating an upgrade of a software application, in accordance with an embodiment of the invention;
Figure 7 is a block diagram of an interaction recorder for recording run-time interactions between the plurality of components, in accordance with an embodiment of the invention; and
Figure 8 is a block diagram of an impact determiner for identifying one or more functions from the plurality of functions In each of the plurality of components. In accordance with an embodiment of the invention.
DETAILED DESCRIPTION OF DRAWINGS
The Invention provides a method and a system for identifying a set of regression test cases from a plurality of test cases to validate an upgrade of a software application. A software application includes a code written in a computer language to execute a particular task. The examples of a computer language include object-oriented languages such as JAVA™, C++, and the like. The software application includes a plurality of components. Each of the plurality of components includes a plurality of functions. The software application is upgraded periodically to introduce additional features for more applications and adaptability.
Figure 1 is a flowchart illustrating a method for identifying a set of regression test cases from a plurality of test cases, in accordance with an embodiment of the invention. The set of regression test cases are a set of functional test cases to test the software application for errors and abnormal outputs.
At step 102, run-time interactions between each of the plurality of functions of the software application are recorded. The run-time interactions are recorded to capture the dynamic behavior of the software application. The dynamic behavior of a software application is defined as a set of interactions among the components and among the invoked functions. In various embodiments of the invention, the dynamic behavior of the software application is captured by executing a plurality of test cases. The dynamic behavior is captured for each of the test cases. In various embodiments of the invention, the dynamic behavior is represented in the form of a Functional Interaction Graph (FIG), known in the art. The details of how the run-time interactions are recorded are described in conjunction with Figure 2. It should be noted that step 102 is performed only once on the software application.
At step 104, one or more functions that have been modified are identified in the upgrade of the software application. The one or more functions are modified during the upgrade of the software application. The details of the identification of the one or more functions are described in conjunction with Figure 3.
At step 106, a set of regression test cases are selected from the plurality of test cases. The set is selected on the basis of the mapping between the run-time behavior represented by the FIG obtained at step 102 and the one or more functions that have been modified as obtained at step 104. The details of the selection of a set of regression test cases are described in conjunction with Figure 4.
Figure 2 is a flowchart illustrating a method for recording run-time interactions among the plurality of functions in a software application, in accordance with an embodiment of the invention. At step 202, the plurality of test cases is executed on the software application. The plurality of test cases is executed by one or more software programmers. The software programmers analyze and write each of the test cases. Further, the results of the test cases are analyzed by the software programmers. During the testing of the code, defects or failures may be observed. The execution of a test case is either manual or automated. If it is executed manually, the software programmers run the test case on the software application being executed and observe the results. If the execution is automatic, the test scripts written by the software programmers are run by another program on the software application being tested. This is extendable to the test suite level. A suite can have hundreds of test cases.
At step 204, the dynamic behavior of the plurality of components is captured on the basis of the execution of the plurality of test cases. In various embodiments of the invention, a language run-time profiler, already known in the art, is used to capture the dynamic behavior. The language run-time profiler interacts with the software applications written in programming languages such as C#, Visual Basic.NET, C++, and the like, to capture the behavior associated with garbage collection, loading and unloading of the plurality of functions, and just-in-time (JIT) compiler notifications. Three steps are involved in capturing the dynamic behavior. The first step includes enabling the profiler by setting appropriate variables. Thereafter, the profiler is requested by the software application for function invocations generated for a particular software application. Subsequently, the invoked functions are accessed using tracer/profiler API.
At step 206, the captured dynamic behavior Is represented as a Functional Interaction Graph (FIG), known in the art. A sample FIG is shown in Figure 3 that depicts a relationship between the execution of each of the plurality of test cases and one or more functions invoked corresponding to the execution of each of the plurality of test cases. In accordance with Figure 3, a test case T1 invokes the functions N1.C1.F0, N1.C4.F3 and N2.C8.F22 where N1 and N2 are two components from the plurality of components, C1, C4 and Cs are classes of N1 and N2 components. Similarly, Fo, F3 and F22 are functions from the plurality of functions that belong to C1, C4 and C8 classes, respectively. This part of the graph is defined as a sub-graph. In various embodiments of the invention, the FIG is constructed only once for each software application.
Figure 4 is a flowchart illustrating a method for identifying one or more functions that have been modified in the upgrade of the software application, in accordance with an embodiment of the invention.
At step 402, the plurality of components in the software application and the upgrade of the software application are converted into corresponding Intermediate language codes such as Microsoft™ Intermediate Language. In an embodiment of the invention, the binary codes of the software application and its upgrade that is in the form of executables (.EXE) and dynamic link libraries (DLL) are converted into the intermediate language code.
At step 404, one or more parameters are extracted from the corresponding intermediate language codes of the software application and the upgrade of the software application. The one or more parameters include one or more functions, class objects and class hierarchies.
At step 406, syntactic analysis is performed on the intermediate language codes of the software application and the upgrade of the software application. In an embodiment of the invention, Cecil ™ library is used to extract the one or more parameters. These corresponding parameters extracted from the intermediate language codes of the software application and the upgrade of the software application are compared with each other. One or more functions that have been modified in the
upgrade of the software application are recorded on the basis of the comparison. For example, if it is detected that a constructor of a class is modified, then each function in the class is recorded as modified. Further, along with syntactic analysis, semantic analysis is performed for the identification of the broader effects of the upgrade of the software application.
At step 408, semantic analysis is performed on the intermediate language codes. The semantic analysis includes identification of one or more functions that get invoked due to various concepts of object-oriented programming. The various concepts include polymorphism, dynamic binding and exception features. The types of modifications taking place in the semantic analysis include modifications due to dynamic binding, modifications due to inheritance tree, modifications due to exception handling and modifications due to implementation of exceptions.
For example, following is an example of a modification due to dynamic binding:
The code present in the software application:
In the example above, Class B inherits from Class A and a virtual function calc() is executed in the software application and its upgrade. However, the function calc() in Class A is modified in the upgrade of the software application. This modified function affects the execution of function fund () of Class D, since the argument passed to that function may also be an object type A due to the inheritance property. Therefore, in this case, both functions A::calc() and D::func1() are recorded as modified functions.
Similarly, the following is an example of a modification due to inheritance tree:
The code present in the software application:
In the example above, Class A does not Inherit from System.Array in the upgrade of the software application and hence the execution of each function in Class A is influenced. Therefore, each function in Class A is recorded as modified. Similarly, the following is an example of a modification due to exception handling: The code present in the software application:
In the example above, in the upgrade of the software application, the function func1() does not handle exceptions whereas function func2() handles exceptions e1 and e2 with a change in its implementation while handling an exception of type e1. In such cases, both the functions func1() and func2() are recorded as modified functions.
Similarly, the following is an example of a modification taking place due to implementation of exceptions:
In the example above, when a class is extended with exceptions and changes are made to those class functions or fields, the functions in the child class that inherited the exception functions are recorded as modified functions.
At step 410, the one or more functions in the software application that correspond to the one or more functions obtained at step 406 and step 408 are collated.
Figure 5 is a flowchart illustrating a method for selecting a set of regression cases on the basis of mapping between run-time interactions and the one or more functions, in accordance with an embodiment of the invention.
At step 502, affected sub-graphs of the FIG are identified on the basis of the mapping between the FIG and the one or more functions that have been modified. The identified sub-graphs include a set of test cases and the one or more functions corresponding to the set of test cases. Subsequently, at step 504, one or more test
cases are identified as the set of regression test cases on the basis of the affected sub¬graphs.
Figure 6 is a block diagram of system 600 configured to identify a set of regression test cases from a plurality of test cases to validate an upgrade of a software application, in accordance with an embodiment of the invention. System 600 includes an interaction recorder 602, an impact determiner 604 and a test case selector 606.
Interaction recorder 602 is configured to capture the dynamic behavior of the plurality of components. The capturing of the dynamic behavior is based on the execution of the plurality of test cases. In various embodiments of the invention, the dynamic behavior of the software application is captured by executing a plurality of test cases. The dynamic behavior is represented in the form of an FIG. The details of interaction recorder 602 are described in conjunction with Figure 7.
Impact determiner 604 is configured for the identification of one or more functions that have been modified in the upgrade of the software application. In various embodiments of the invention, the source code of the software application is not available for the understanding of the logic and control flow. Therefore, the upgrade of the software application is converted into an intermediate language code for the identification of the one or more functions. The details of impact determiner 604 are described in conjunction with Figure 8. The one or more functions are used for the selection of the set of regression test cases.
Test case selector 606 is configured for the selection of a set of regression test cases from the plurality of test cases. The set is selected on the basis of the mapping between the run-time behavior represented as an FIG and the one or more functions that have been modified. One or more sub-graphs of the FIG are identified on the basis of the mapping between the FIG and the one or more functions. Subsequently, a set of regression test cases is identified, based on the affected sub-graphs.
Figure 7 is a block diagram of interaction recorder 602 configured to record run¬time interactions between the plurality of components of the software application, in
accordance with an embodiment of the invention. Interaction recorder 602 includes a test case execution module 702 and a capturing module 704. Capturing module 704 includes a language run-time profiler 706.
Test case execution module 702 is configured to execute the plurality of test cases on the software application. Based on the execution of the plurality of test cases, capturing module 704 is configured to capture the dynamic behavior of the plurality of components. Test case execution module 702 includes language run-time profiler 706 having a language run-time profiler, which is known in the art. Language run-time profiler 706 interacts with software applications written in programming languages such as C#, Visual Basic.NET, and C++, to capture the behavior associated with garbage collection, loading and unloading of the plurality of functions, and just-in-time (JIT) compiler notifications. Subsequently, test case execution module 702 represents the captured dynamic behavior as an FIG. The details of the FIG are described in conjunction with Figure 2 and Figure 3.
Figure 8 is a block diagram of impact determiner 604 configured to identify the one or more functions from the plurality of functions in each of the plurality of components, in accordance with an embodiment of the invention. Impact determiner 604 includes disassembling module 802, syntactic analysis module 804 and semantic analysis module 806.
Disassembling module 802 configured to convert the plurality of components in the software application and the upgrade of the software application into corresponding intermediate language codes such as MicrosoftTM Intermediate Language. In various embodiments of the invention, the respective binary codes of the software application and the upgrade of the software application in the form of executables (.EXE) and dynamic link libraries (DLL) are converted into the intermediate language code. Further, disassembling module 802 extracts one or more parameters from the corresponding intermediate language codes. The one or more parameters include functions, class objects and class hierarchies from the plurality of components. The intermediate language codes are used to perform syntactic and semantic analyses.
Syntactic analysis module 804 is configured to perform syntactic analysis on the intermediate language codes to identify one or more functions that have been modified in the upgrade of the software application. During syntactic analysis, the modifications at the syntactic level are analyzed. In various embodiments of the invention, syntactic level changes include change in the syntax due to textual differences among corresponding line statenients of the code of the upgrade of the software application. Various parameters are extracted from the intermediate language code. The parameters extracted include namespaces, nested namespaces, types and classes and their attributes, nested classes, methods, constructors, class fields, constants and function codes.
Semantic analysis module 806 is configured to perform semantic analysis on the intermediate language codes to identify one or more functions that have been modified in the upgrade of the software application. The semantic analysis includes identification of one or more functions that get affected because of the changed functions due to various concepts of object-oriented programming. The various concepts include polymorphism, dynamic binding and exception features. The types of modifications taking place in the semantic analysis include modifications due to dynamic binding, modifications due to inheritance tree, modifications due to exception handling and modifications due to implementation of exceptions.
The invention described above provides a fewer number of regression test cases for testing of the upgrades of the software application. Using fewer test cases for testing saves cost and time, thereby reducing delays in delivering the software application to the clients. Further, the method described above is independent of using the source code of the software application.
In various embodiments of the invention, the modules of system 600 may be implemented in the form of software, hardware, firmware, or combinations thereof.
The system for identifying a set of regression test cases as described in the present invention or any of its components may be embodied in the form of a computer system. Typical examples of a computer system include a general-purpose computer, a
programmed microprocessor, a micro-controller, a peripheral integrated circuit element, and other devices or arrangements of devices that are capable of implementing the steps that constitute the method of the present invention.
The computer system comprises a computer, an input device, a display unit and the Internet. The computer also comprises a microprocessor, which is connected to a communication bus. The computer also includes a memory, which may be a Random Access Memory (RAM) and Read Only Memory (ROM). Further, the computer system comprises a storage device, which can be a hard disk drive or a removable storage drive such as a floppy disk drive, an optical disk drive, etc. The storage device can also be other similar means for loading computer programs or other instructions into the computer system. The computer system includes a communication unit, which enables the computer to connect to other databases and the Internet through an I/O interface. The communication unit enables the transfer and reception of data from other databases. The communication unit may include a modem, an Ethernet card or any similar device that enables the computer system to connect to databases and networks such as LAN, MAN, WAN and the Internet. The computer system facilitates inputs from a user through an input device that is accessible to the system through an I/O interface.
The computer system executes a set of instructions that is stored in one or more storage elements, to process input data. The storage elements may hold data or other information, as desired, and may be in the form of an information source or a physical memory element present in the processing machine.
The set of instructions may include various commands that instruct the processing machine to perform specific tasks such as the steps that constitute the method of the present invention. The set of instructions may be in the form of a software program. Further, the software may be in the form of a collection of separate programs, a program module with a larger program, or a portion of a program module, as in the present invention. The software may also include modular programming in the form of object-oriented programming. Processing of input data by the processing machine may
be in response to users commands, the results of previous processing, or a request made by another processing machine.
While the preferred embodiments of the invention have been illustrated and described, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the invention, as described in the claims.
We Claim:
1. A method for identifying a set of regression test cases from a plurality of test
cases for validating an upgrade of a software application, the software
application comprising a plurality of components, each component of the plurality
of components comprising a plurality of functions, the method comprising :
a. recording run-time interactions between the plurality of functions in each of
plurality of components, the run-time interactions being recorded by
executing the plurality of test cases on the software application;
b. identifying one or more functions from the plurality of functions in each of
the plurality of components, wherein the one or more functions have been
modified during upgrading of the software application; and
c. selecting the set of regression test cases based on mapping between the
run-time interactions and the one or more functions.
2. The method according to claim 1, wherein the run-time interactions are recorded by a language run-time profiler.
3. The method according to claim 1, wherein the run-time interactions are recorded as a Functional Interaction Graph (FIG), the FIG being used to represent the dynamic behavior of the plurality of functions.
4. The method according to claim 1, wherein identifying the one or more functions comprises converting the plurality of components into an intermediate language code.
5. The method according to claim 1, wherein identifying the one or more functions comprises syntactic analysis of the intermediate language code.
6. The method according to claim 1, wherein Identifying the one or more functions comprises semantic analysis of the intermediate language code.
7. A system for identifying a set of regression test cases from a plurality of test cases for validating an upgrade of a software application, the software application comprising a plurality of components, each component of the plurality of components comprising a plurality of functions, the system comprising:
a. an interaction recorder configured for recording run-time interactions
between the plurality of components;
b. an impact determiner configured for identifying one or more functions from
the plurality of functions in each of the plurality of components, the one or
more functions being modified during upgrading of the software
application; and
c. a test case selector configured for selecting a set of regression test cases
based on mapping between the run-time interactions and the one or more
functions.
8. The system according to claim 7, wherein the interaction recorder comprises a test case execution module configured for executing the plurality of test cases on the software application.
9. The system according to claim 7, wherein the interaction recorder comprises a capturing module configured for capturing dynamic behavior of the plurality of functions in execution of the plurality of test cases on the software application.
10. The system according to claim 9, wherein the capturing module comprises a
language run-time profiler configured for generating a Functional Interaction
Graph (FIG) to capture the dynamic behavior. 11 .The system according to claim 7, wherein the impact determiner comprises a
disassembling module configured for converting the plurality of components into
an intermediate language code.
12. The system according to claim 7, wherein the impact determiner comprises a syntactic analysis module configured for syntactic analysis of the intermediate language code.
13. The system according to claim 7, wherein the impact determiner comprises a semantic analysis module configured for semantic analysis of the intermediate language code.
14. A computer program product for use with a computer, the computer program product comprising a computer usable medium having a computer readable program code embodied,therein for identifying a set of regression test cases from a plurality of test cases for validating an upgrade of a software application, the software application comprising a plurality of components, each component of the plurality of components comprising a plurality of functions, the computer readable program code performing:
a. recording run-time interactions between the plurality of functions in each of plurality of components, the run-time interactions being recorded by executing the plurality of test cases on the software application;
b. identifying one or more functions from the plurality of functions in each of
the plurality of components, wherein the one or more functions have been
modified during upgrading of the software application; and
c. selecting the set of regression test cases based on mapping between the
run-time interactions and the one or more functions.
15. The computer program product according to claim 14, wherein the run-time
interactions are recorded by a language run-time profiler.
16. The computer program product according to claim 14, wherein the run-time
interactions are recorded as a Functional Interaction Graph (FIG), the FIG being
used to represent the dynamic behavior of the plurality of functions.
17. The computer program product according to claim 14, wherein the computer
readable code for identifying the one or more functions performs converting the
plurality of components into an intermediate language code.
18. The computer program product according to claim 14, wherein the computer
readable code for identifying the one or more functions performs syntactic
analysis of the intermediate language code.
19.The computer program product according to claim 14, wherein the computer readable code for identifying the one or more functions performs semantic analysis of the intermediate language code.
| # | Name | Date |
|---|---|---|
| 1 | 1565-che-2008 correspondance others 03-08-2009.pdf | 2009-08-03 |
| 2 | 1565-che-2008 assignment 03-08-2009.pdf | 2009-08-03 |
| 3 | 1565-che-2008 form-5.pdf | 2011-09-03 |
| 4 | 1565-che-2008 form-3.pdf | 2011-09-03 |
| 5 | 1565-che-2008 form-1.pdf | 2011-09-03 |
| 6 | 1565-che-2008 drawings.pdf | 2011-09-03 |
| 7 | 1565-che-2008 description (complete).pdf | 2011-09-03 |
| 8 | 1565-che-2008 correspondence-others.pdf | 2011-09-03 |
| 8 | 1565-che-2008 form-5.pdf | 2011-09-03 |
| 9 | 1565-che-2008 claims.pdf | 2011-09-03 |
| 10 | 1565-che-2008 abstract.pdf | 2011-09-03 |