FORM 2
THE PATENTS ACT  1970
(39 of 1970)
&
THE PATENT RULES  2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

TITLE OF INVENTION:
“A SYSTEM AND METHOD FOR AUTOMATING BUILD VERIFICATION TESTING”

Applicant
TATA Consultancy Services Limited
A company Incorporated in India under The Companies Act  1956
Having address:
Nirmal Building  9th Floor 
Nariman Point  Mumbai 400021 
Maharashtra  India

The following specification particularly describes the invention and the manner in which it is performed.

FIELD OF THE INVENTION

The present invention relates to the field of software testing. More particularly  the invention relates to a system and method of automating build verification testing.
BACKGROUND OF THE INVENTION

Functional testing is a type of black box testing that bases its test cases on specifications of a software component under test. Functions are tested by feeding them an input and examining an output  and internal program structure is rarely considered during the testing process. Functional testing mainly involves identification of functions that the software is expected to perform  creation of input data based on the function""s specifications  determination of output based on the function""s specifications  execution of the test case and comparison of actual and expected outputs. However it is evident from the process that functional testing involves late identification of user interface (UI) defects. Moreover  when deployed in multiple environments it leads to introduction of multiple failure points to be tested. The time lost on validation of UI is very critical as the same can be utilized more effectively on validating system performance for business needs.
The build verification test [BVT] also known as smoke testing is usually a predefined short set of tests  which helps to streamline the main functionality of the software. It is performed prior to the delivery of the project to the testing team and accordingly any build that fails in the build verification test is rejected  and testing continues on the previous build. Build verification testing [BVT] is important because it let the developers know right away if there is any critical problem with the build  thereby avoiding delay in the testing process. Build failures interrupt unattended automation script runs thereby reducing the effectiveness of the testing automation process. Smoke tests or BVT can either be performed manually or using an automated tool. When automated tools are used  the tests are often initiated by the same process that generates the build itself.
Repeatable scenarios for testing would be the ideal candidates for automation. Automation of this process would bring down the time of testing the build for every release. Also once the automation is in place; it could even be used by the developers to test the build before handing it to the testing team.
For a manual tester  it becomes taxing to execute the tests on daily builds which would incur more effort  time and cost leading to inefficient utilization of resources. To address this specific challenge automation of BVT is necessitated which would assist the testers to ease out the process of testing the daily or frequently occurring builds. In a scenario where there are multiple releases per year and verification is done manually it might consume days to complete the initial testing for each release. Automation of BVT is therefore foreseen as an urgent solution to bring down the testing time and increase the testing quality by reducing the manual effort.
Generally  while automating any application automation tool identifies the objects in the application with specific properties. User can include some properties according to his choice in order to make the tool identify the application. Once user selects the different properties of the objects  BVT would take these properties as inputs and generates the values for these properties using its feature. Once the object properties for a particular build are captured  the BVT scripts are tested for the subsequent builds thereby ensuring that the object properties values are tested and checking that build is stable and could be tested for functional/regression testing phases.
In the state of art  automation of testing has been done by generating scripts in the backend by capturing the event of the device. The inherent problem associated with this approach is that slight change in the properties of the GUI makes it necessary to repeat all the test scenarios that have been previously tested. Moreover  identification of different objects and verification of the property values of the objects has been ignored for this.
In the light of foregoing problem  there exists a need for an automation framework for build verification testing that shall work on the granular level to capture object properties for sufficing the need for a more effective and efficient sanity/smoke testing for the build.

OBJECTIVES OF THE INVENTION

The principle object of the present invention is to utilize object properties of the GUI for the purpose of automating build verification testing.

Another object of the invention is without having to change the code of the object properties  it can be used for testing thus reusability is ensured.
Another object of the invention is to analyze object level properties of the components to gain one time understanding and eventually reducing the testing effort for the further project.
Another object of the invention is to reduce testing time with less chance of human error as the process is automated.

Another object of the invention is to leverage the features of QTP in terms of object property comparison.

Another significant object of the invention is to reduce the multiple failure points introduced in a heterogeneous environment during deployment of the build.

It is another object of the invention to avoid slipping of the UI defects to the customer end.

Another object of the invention is to find the critical bugs in the application before the functional testing is initiated thus saving a lot of resources and eventually time.

Yet another object of the invention is to check and verify the stability of the build.

SUMMARY

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter  nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

Additional features and advantages of the invention will be set forth in the description which follows  and in part will be obvious from the description  or may be learned by the practice of the invention. These and other features of the present invention will become more fully apparent from the following description  or may be learned by the practice of the invention as set forth hereinafter.

In one of the preferred embodiments of the present invention  a method and system for automating build verification testing using object properties of the components within the user interface has been provided. BVT is an automation framework built on top of QTP which is used to test the object properties of the GUI based systems where there are multiple builds changing quite frequently. It would compare the object properties of the new build vs the previous build and creates a report that gives the information regarding the various changes to the object properties of the build and verify its stability.

One embodiment of the invention contemplates a system comprising of an input module for retrieving plurality of object associated properties for each component of a user interface  a comparing module for mapping the component properties with those stored in a repository; and a reporting module to illustrate the stability of the build based result sheet. The system includes a memory device coupled with a processor to assist the comparing module and provide customizable object property list for generating the result sheet. The system also comprises of an auxiliary module to facilitate additional checks for easy extensions.

The system aims to automate the build verification testing process by utilizing property list of each component within a user interface by using automated scripts thus eliminating the need for sanity and user acceptance testing.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary  as well as the following detailed description of preferred embodiments  is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention  there is shown in the drawings example constructions of the invention; however  the invention is not limited to the specific system and method disclosed in the drawings:

Figure 1 depicts a process flow of the method for verifying the stability of the build in accordance with one of the embodiments of the present invention.
Figure 2 describes the process steps involved in the actual testing of the build in accordance with one disclosed embodiment of the present invention.
Figure 3 shows the high level architecture of the system involved in verifying the stability of build in accordance with one of the preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Some embodiments of this invention  illustrating all its features  will now be discussed in detail. The words "comprising " "having " "containing " and "including " and other forms thereof  are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items  or meant to be limited to only the listed item or items.

It must also be noted that as used herein and in the appended claims  the singular forms "a " "an " and "the" include plural references unless the context clearly dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present invention  the preferred  systems and methods are now described.

The present invention describes a system and method to make the build stable before functional testing. The present invention aims to automate the build verification testing process by utilizing property list of each component within a user interface. The system mainly comprises of a memory module coupled to a processor  a scripting unit to retrieve property list of the components within the UI  a comparing module which maps all the object properties of the components within the UI with the object property list in the repository  an auxiliary module for navigating through the different screens of the application to capture the object properties and which goes through the flow of the sanity/smoke test cases and a reporting module to generate reports illustrating the stability of the build so that it can be handed over to the testing team for comprehensive testing of other functional aspects.

In an embodiment the input module 300 consists of a test build 202 to be verified for stability. The test build 202 comprises of various user interface 201 screens consisting of multiple objects within the user interface. Validation of the properties of each of the objects leads to verifying the stability of the build.

The functionality of the scripting unit 302 is to retrieve the property list for each of the components within the user interface 201. Once the property lists are captured  the scripting unit 302 then captures the object property values for each of the object property lists from the user interface. The scripting unit 302 is mapped to the object repository file of QTP thereby ensuring that the object properties and their values are taken with respect to the QTP tool for any GUI based application.

In a preferred embodiment the memory module 308 coupled to the processor executes a series of instructions to assist the comparing module 304 for performing special purpose computing task and eventually aid the build verification process.

The comparing module 304 maps the object properties retrieved from the scripting unit 302 to the object properties present in the repository 208. The comparing module 304 has the various objects properties and their values stored in it. Basically  for the GUI based application system the object properties and their values are captured and stored in a spreadsheet. Again  for the new build these scripts takes the values from the new GUI based application build and compares it with the existing values and generates a report that gives the information regarding the instability  if any   within the build with respect to the changes to the object properties of the user interface.

In another embodiment the auxiliary module 312  provides a facility to perform additional checks for easy extensions. If any additional object properties that needs to be validated for example  color of the link  color of the text box  then these are validated as part of the auxiliary module. The auxiliary module has the feature of providing the flow between the various user interface screens. Usage of descriptive programming languages like XML and the like assists the developer in collaboration.

The report generation module 314 generates reports in the form of HTML format to verify the stability of the build and highlight the changes between the previous build and new build.

Referring to figure 1  the diagram 100 shows us a process flow of the method for verifying the stability of the build. The first step in the process 104 includes retrieving of the object properties 203 of the components within the user interface 201.The object properties 203 of the components refer to but are not limited to transparency  size  alignment  and the like. The second step 106 involves comparison between the retrieved properties in the first step and the property list present in the repository 208. The final step 108 involves generation of reports regarding the stability of the build based on the results obtained in the second step 106. The reports help the developers and testers analyze the critical faults within the build that need to be corrected at priority before functional testing of the build.

Now referring to figure 2  the diagram 200 describes the various steps involved in the actual testing of the build. The test build 202 is used as an input for build verification. The build is provided to the driver script 204 which then extracts the object properties and using the comparing modules maps the properties with the scenario sheet 206 and the object properties present in the repository 208. Based on mapping  the report 210 is generated to verify the stability of the build. If the build is stable it is forwarded to the testing team for functional testing. Thus the need for sanity testing and user acceptance testing is eliminated.

Further referring to figure 3  the diagram 300 shows all the various components involved in the process of verifying the stability of the build. The input module 300 consists of a test build 202 to be verified for stability. The test build 202 comprises of various user interface 201 screens consisting of multiple objects within the user interface. Validation of the properties of each of the objects leads to verifying the stability of the build. The functionality of the scripting unit 302 is to retrieve the property list for each of the components within the user interface 201. In a preferred embodiment the memory module 308 coupled to the processor executes a series of instructions to assist the comparing module 304. The comparing module 304 maps the object properties retrieved from the scripting unit 302 to the object properties present in the repository 208. In another embodiment the auxiliary module 312  provides a facility to perform additional checks for easy extensions. The report generation module 314 generates reports in the form of HTML reports to verify the stability of the build.

In another aspect of the invention scripts that can be readily used for testing are made available thus automating the build verification testing process. Furthermore  the same scripts can be customized and can be used for testing purposes thus eliminating the need to redesign the scripts for the new scenario leading to increase in resource utilization.

The advantages associated with the mentioned process include:
• Enabling faster deployment of stable systems into the environment thus reduced testing time and better resource utilization
• Moreover  scripting retrieves property lists for each screen and prepares expected user friendly reports generation making it easy to understand the defects in the build
• BVT framework provides as extensibility  in which developer can add their own user defined functions.
• BVT framework helps to deploy the stable application in the environment.
• BVT framework uses both descriptive programming and object repository in .xml format to identify the objects in the AUT. As BVT uses object repository in .xml format helps the developer to collaborate closely with the framework; even the developer can construct the BVT test with limited testing tool knowledge
• BVT mainly does the UI validation  thus BVT framework covers more than 80% of UI validation

WORKING EXAMPLE

Let us consider a test case file as shown in the below table 1. Test cases are customized in the excel sheet  it contains the flow of the execution. Test data like expected property values and the data required application flow are contained in the same sheet  both these can be customized by the user. It consists of various fields of object properties that need to be checked in the build. All these file supplies input to the driver script  driver script is developed in QTP. This driver script gets the input from object repositories and test case excel sheets. As shown in the table 1  on first page for paragraph 3 there is user defined height width and size for object ‘username’ which is specified in the last column of the row. Similarly on page 2 there is user defined height and width. According to the execution flow in the test case sheet  the driver script will interact with the AUT (Application Under Test) and perform the operations to generate results in the HTML format .Further now table 2 illustrates the object sheet to be used for report generation and table 3 shows the HTML report generated using the test cases and object sheet..

S.no Para 1 Para 2 Para 3 Para 4
Page 1 Open web http://new tours.com
Type input username mercury
checkprop username User defined Height  width  size 20;30;900
checkprop password default
Type input password mercury
Click and wait loginbtn 30000
Select and wait List form port London
Stop
Page 2 Open web http://google.com
Check Prop googletext User defined Height  width 
Type input googletext mylife
Click Google btn

Table 1: Test case file

Google Browser {“Google”}
Google Browser {“Google”}.Page{“Google”}
Q Browser {“Google”}. Page{“Google”}.WebEdit {“q”}
Google search Browser {“Google”}.WebButton{“Google Search”}

Table 2: Objects Sheet

Object Name: username
Property Name Expected Value Actual Value Result
Height 20 22 Failed
width 30 86 Failed
size 900 10 Failed

Table 3: Report Sheet

WE CLAIM:
1. A computer implemented method to automate a build verification testing  the method comprising:
retrieving at least one property associated with each component of a user interface via a scripting language;
mapping the retrieved properties against a plurality of object property list stored in a repository; and
generating at least one validation report for the user interface to verify the stability of the build.
2. The method of claim 1  wherein the property list refers to the plurality of object properties associated with components present within the user interface.

3. The method of claim 1  wherein the user interface includes plurality of components those need to be identified for testing.

4. The method of claim 1  wherein the repository includes plurality of property lists for each components within the user interface.

5. The method of claim 1  wherein the object property result sheet includes but is not limited to the screen name  object name  object property value  match count  mismatch count and the like.

6. The method of claim 1  wherein the validation report is a HTML report to verify the stability of the build.

7. The method of claim 1  wherein the scripting language refers to but is not limited to usage of JavaScript  ASP  JSP  PHP  Perl  and the like.

8. The method of claim 1  further provides a customizable object property list for generating an object property result sheet.
9. The method of claim 1  further facilitates additional checks for easy extensions to validate additional object properties.

10. An automated system for build verification testing  executed by a processor comprising:
an input module adapted to retrieve a plurality of property list for each component within an user interface via a scripting language;
a comparing module configured to map component properties to an object property list stored in a repository;
a memory device coupled with the processor to store and provide customizable object property list for generating an object property result sheet; and
a reporting module configured to generate validation reports based on the object property result sheet and the object property list of the repository for the user interface to verify stability of the build.

11. The system of claim 10  wherein the property list refers to the plurality of object properties associated with components present within the user interface.
12. The system of claim 10  wherein the user interface includes plurality of components those need to be identified for testing.

13. The system of claim 10  wherein the scripting language refers to but is not limited to usage of JavaScript  ASP  JSP  PHP  Perl  and the like.

14. The system of claim 10  wherein the repository includes plurality of property lists for each components within the user interface.

15. The system of claim 10  wherein the object property result sheet includes but is not limited to the screen name  object name  object property value  match count  mismatch count and the like.

16. The system of claim 10  wherein the reporting module generates end result in a HTML report to verify the stability of the build.
17. The system of claim 10  further comprising of an auxiliary module adapted to facilitate additional checks for easy extensions to validate additional object properties.