Applicant(s) – Tata Elxsi Limited  ITPB Road  Whitefield  Bangalore – 560048 India
Inventor(s) –
(a) Harwinder Jutla  c/o Tata Elxsi Limited  ITPB Road  Whitefield  Bangalore – 560048 India

Title of the Invention Automating GUI testing using checksum method on the graphics frame buffer
Background:
Testing is of course  a fundamental part of the development cycle of devices. Traditionally  and to varying degrees today also  this testing is carried out manually. The industry is spending considerable resources (manpower and monetary) in building test frameworks which allow testing activities to be automated as much as possible.
The complexity of devices and their user interfaces is increasing. More functionality is being crammed into devices and hence the complexity of testing increases. Therefore  automation is seen as sound investment because it yields many advantages – repeatability  less human error prone  increase in test coverage  better stress testing and so on.
In this increased complexity of user interfaces  the challenges to test the graphical user interfaces increase also. Irrespective of the size of the display (several lines of LED displays  small LCD screens as on mobile phones  or larger screens as in televisions)  the use cases have increased substantially over the years due to increased functionality and hence the challenges to test the user interface on these devices have also increased.
Further  in the same way that device functionality testing is being automated  there is a need to increase automating the testing of graphical user interfaces also. In attempting to do so  it presents its own challenges and the industry has adopted various mechanisms to try and overcome these challenges. Some of these methods deployed are less successful than others but there is clearly room to improve automating the user interface on screen based devices.
Object
An opportunity therefore exists to increase the reliability of the level of automation deployed in the industry today. As a result  this method set out in this document achieves this on a number of different fronts and hence has the capacity to substantially improve user interface testing reliability without increased capital or investment.
Provided testing is considered as part of the design process and is built and designed in from the start  this method will successfully allow better user interface testing.
Summary of Invention
In existing systems where GUI testing is automated  there are 2 methods adopted to determine if the image displayed on the Device Under Test (DUT) is free from errors. These are:
1. Using a camera  capture the image shown on the DUT and then use software on Test PC to process the image  determine the area to be tested and then compare the pixels of the captured image with that of the saved golden image.
Disadvantage with this method is that the captured image has to be good enough resolution to be able to segregate the area of interest to a pixel level and hence the images are potentially large. The storage requirements are also large since an image will be stored for every test run and if you are saving historical data  then storage increases substantially. Further  if OCR is performed on the image  this can be unreliable and give false readings. In order to overcome false results  the same test has to performed multiple times and the best cases are taken as the results
2. Used less often  the image of interest is sent over a communication link to the Test PC where it compared with the saved image
Disadvantage with this system is that the images transferred are potentially large and hence the communication link needs to be fast. This is not always possible as some smaller devices may not have a fast external link. Further  storage requirements are again large.
This invention puts forward a method by which these disadvantages can be eliminated. Instead of OCR or large image transfers  a checksum is performed on the frame buffer of the images being displayed and under test.
Detailed Description
The method proposed here is that instead of transferring the image from DUT  a CRC is performed of the graphics area of interest. The graphics data is always available in the frame buffer of the DUT. Hence  only a small amount of data needs to be transferred to the Test PC. This CRC can be compared with the golden CRC for that image to determine if there are errors in the image.
This method provides pixel accuracy.
It is possible to use CRC32 as a standard for all DUTs. CRC32 will cover a large enough resolution to cover all devices today as well as foreseeable future. It is important to use a method such that CRC values do not collide i.e. two different images should not generate the same checksum.
A CRC32 checksum method will cater for data length of 2^32-1= 429467295 bits. If we assume devices have a maximum color depth of 32 bits  then CRC32 will cater for devices with a maximum resolution of 11K x 11K for 1:1 aspect ratio and for a 16:9 (DTVs)  this would allow for a resolution of 15446 x 8688. This is more than sufficient for foreseeable future.
It is possible to adopt CRC24 or CRC16 for devices with smaller graphics screens and hence the system can be dynamic depending on the need and performance available in the DUT.
This method has the following advantages:
• Pixel accuracy
• Only small amounts of data (32 bits for CRC32) need to be transmitted back to the Test PC and hence low bandwidth communication links can be used.
• It is much easier to test only segments of the screen without losing accuracy and determinability.
Description of the System
A typical test setup scenario for a DUT (Device Under Test) is shown in Fig 1.

DUT
This is the graphical device under test. This can be any device such as mobile phones  tablets  STBs  DTVs  and automotive clusters. The only requirement from the DTU is that it must have a communication port (RS232  Ethernet etc.) available to communicate with the PC and access to the DUT software must be available so that the Interface Software Module can be written/modified and integrated into the device.
Interface Software Module
This is the module which is responsible for servicing all the functionality required by the Automated Test Application Software at the DUT end. This small software module resides in the DUT and is responsible for the following:
• Parsing commands from the test PC
• Calculate and send back the CRC of the graphics area in the DUT as requested by the test PC. The area will be specified in terms of x1 y1 and x2 y2.
• Insert simulated either hard keys or soft keys (Touch  infra-red  key presses etc.) into the main software stack of the DUT as if the user had pressed these keys. The key type is sent by the test PC.
Test Application Software on a PC
This PC runs the main test software suite. This is outside the scope of this invention. The user can use this to script the actions he wants to perform to carry out tests. The data sent back from the DUT is stored in the local PC storage with the test run. The fail/pass will be determined by comparing the returned CRC data with the stored golden reference.
Claims
The following claims are made with respect to the above defined invention:
1. Automated testing is pixel accurate
2. The system is fully deterministic.
3. Due to the nature of the invention  DUT testing can be carried out efficiently and quickly over low bandwidth communication links available on the DUT like RS232
4. Using the enhancement that the Interface Software module will provide  it can be used to simulate every type of user input (touch screen  Infra-red  key presses  swipes etc.)  the coverage of automated testing can be significantly increased
5. With low end devices with slower processors and lower screen resolutions  the method can be adapted to work efficiently even with such devices so that CRC calculations do not become processor bandwidth intensive.
6. By adding extra functionality implementation in the Interface Software Module in the DUT  time sensitive graphics images can also be tested as well as animated GUIs – for example  parts of the user interface which appear for a short time and then disappear as part of the implemented functionality.
Abstract
Automating GUI testing is a challenging task. Complexity of GUI testing increases when multiple languages are supported by the device. The most common method of automating GUI testing today is a combination of graphics image capture & comparison and a level of OCR. Reliability of such methods can lead to false results.
The method discussed here eliminates these inaccuracies and thereby potentially reducing cost and increasing reliability & robustness of the testing methods.
This method provides substantial flexibility whereby only areas of interest on the screen can be tested. This method is only suitable for a static image which is the case for majority of GUIs. This method describes how a CRC value of the graphics image under test can be used instead of image comparisons for automating GUI testing as well as increasing the coverage of automated testing by a resident test software module within the device under test.