Claims:We Claim:

1. A processor-implemented method for evaluation of scripts of a load testing application, the method comprising:
receiving, via one or more hardware processors, one or more scripts generated by the load testing application;
identifying, via the one or more hardware processors, in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script; and
systematically mapping, via the one or more hardware processors, the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and ter-minate transaction statements and a set of non-matching transaction statements, the set of non-matching transaction statements comprising one or more initiate transaction state-ments and one or more initiate transaction statements, wherein at least a count of state-ments in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application.
2. The method as claimed in claim 1, wherein systematically mapping the plurality of initiate transaction statements with the plurality of terminate transaction statements of the script in the iterative manner comprises:
comparing transaction names in last-occurred termination-transaction statement with last-occurred initiate-transaction statement in a plurality of iterations; and
incrementing a counter on determination of a mapping between the last-occurred termination-transaction statement with last-occurred initiate-transaction state-ment.
3. The method as claimed in claim 1, further comprising applying one or more rules associ-ated with one or more quality testing parameters to the one or more scripts for evaluation of the one or scripts.
4. The method as claimed in claim 3, wherein the one or more quality testing parameters comprises files, dynamic values, MAX_HTML parameter, web_reg_find function, one or more transactions and functions placed within the one or more transactions associated with the script.
5. The method as claimed in claim 1, further comprising determining format of the one or more scripts prior to applying the one or more rules to the one or more scripts.
6. The method as claimed in claim 1, wherein the load testing application comprises Load-Runner® application.
7. The method as claimed in claim 1, wherein a terminate-transaction statement of the plu-rality of terminate-transaction statements comprises one of a lr_end_transaction(“trn_name”,LR_AUTO), lr_end_transaction(“trn_name”,LR_PASS) and lr_end_transaction(“trn_name”,LR_FAIL), and wherein an initiate-transaction state-ment of the plurality of terminate-transaction statements comprises lr_start_transaction(“trn_name”).
8. A system executed by a computing device for evaluation of scripts of a load testing application, the system comprising, the system comprising:
one or more memories; and
one or more hardware processors, the one or more memories coupled to the one or more hardware processors, wherein the one or more hardware processors are capable of executing programmed instructions stored in the one or more memories to:
receive one or more scripts generated by the load testing application;
identify in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script; and
systematically map the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements, the set of non-matching transaction statements comprising one or more initiate transaction statements and one or more initiate transaction statements, wherein at least a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application.
9. The system as claimed in claim 8, wherein to systematically map the plurality of initiate transaction statements with the plurality of terminate transaction statements of the script in the iterative manner, said one or more processors are further configured by the instruc-tions to:
compare transaction names in last-occurred termination-transaction statement with last-occurred initiate-transaction statement in a plurality of iterations; and
increment a counter on determination of a mapping between the last-occurred termination-transaction statement with last-occurred initiate-transaction statement.
10. The system as claimed in claim 8, wherein said one or more processors are further con-figured by the instructions to apply one or more rules associated with one or more quality testing parameters to the one or more scripts for evaluation of the one or scripts.
11. The system as claimed in claim 10, wherein the one or more quality testing parameters comprises files, dynamic values, MAX_HTML parameter, web_reg_find function, one or more transactions and functions placed within the one or more transactions associated with the script.
12. The system as claimed in claim 8, wherein said one or more processors are further con-figured by the instructions to determine format of the one or more scripts prior to applying the one or more rules to the one or more scripts.
13. The system as claimed in claim 8, wherein the load testing application comprises Load-Runner® application.
14. The system as claimed in claim 8, wherein a terminate-transaction statement of the plural-ity of terminate-transaction statements comprises one of a lr_end_transaction(“trn_name”,LR_AUTO), lr_end_transaction(“trn_name”,LR_PASS) and lr_end_transaction(“trn_name”,LR_FAIL), and wherein an initiate-transaction state-ment of the plurality of terminate-transaction statements comprises lr_start_transaction(“trn_name”).
, Description:TECHNICAL FIELD
[001] The present subject matter relates, in general, to load testing and, in particular, to providing method and systems for evaluation of scripts generated from a load testing application.
BACKGROUND
[002] A load testing application is utilized for performance evaluation of software applications. For example, load testing is intended to determine responsiveness, throughput, reliability, and/or scalability of a system or a software application under a given workload. Load testing facilitates in identifying bottlenecks in the system and collecting other performance-related data to help stakeholders make informed decisions related to the overall quality of the application being tested.
[003] The load testing application creates a virtual workload on a system under test (SUT) to measure the response time of the SUT under said workload. Typically, the load testing applications include a script recorder that records communications on the software application when the software application is accessed by customers. The recorded scripts are replayed multiple times, and the metrics generated during said replaying is utilized for performance evaluation of the SUT. In order to suitably evaluate the performance of the SUT, the scripts generated by the load testing application have to be evaluated.
SUMMARY
[004] The following presents a simplified summary of some embodiments of the disclosure in order to provide a basic understanding of the embodiments. This summary is not an extensive overview of the embodiments. It is not intended to identify key/critical elements of the embodiments or to delineate the scope of the embodiments. Its sole purpose is to present some embodiments in a simplified form as a prelude to the more detailed description that is presented below.
[005] In view of the foregoing, an embodiment herein provides a method and system for evaluation of scripts of a load testing application. In one aspect, a processor-implemented method for evaluation of scripts of a load testing application is provided. The method includes receiving, via one or more hardware processors, one or more scripts generated by the load testing application. Further, the method includes identifying, via the one or more hardware processors, in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script. Furthermore, the method includes systematically mapping, via the one or more hardware processors, the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements. The set of non-matching transaction statements includes one or more initiate transaction statements and one or more initiate transaction statements. At least a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application.
[006] In another aspect, processor-implemented system for evaluation of scripts of a load testing application is provided. The system includes a hardware having one or more memories and one or more hardware processors coupled to the one or more memories to effect a computing device. The one or more hardware processors are capable of executing programmed instructions stored in the one or more memories to receive one or more scripts generated by the load testing application. Further, the one or more hardware processors are capable of executing programmed instructions to identify in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script. Furthermore, the one or more hardware processors are capable of executing programmed instructions to systematically map, via the one or more hardware processors, the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements. The set of non-matching transaction statements includes one or more initiate transaction statements and one or more initiate transaction statements. At least a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application
[007] In yet another aspect, a non-transitory computer-readable medium having embodied thereon a computer program for executing a method for evaluation of scripts of a load testing application, is provided. The method includes receiving one or more scripts generated by the load testing application. Further, the method includes identifying in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script. Furthermore, the method includes systematically mapping the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements. The set of non-matching transaction statements includes one or more initiate transaction statements and one or more initiate transaction statements. At least a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application.
BRIEF DESCRIPTION OF THE FIGURES
[008] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and modules.
[009] FIG. 1 illustrates a network environment implementing a system for evaluation of scripts of a load testing application, according to an embodiment of present disclosure;
[0010] FIG. 2 illustrates a block diagram of a system for evaluation of scripts of a load testing application in accordance with an example embodiment;
[0011] FIG. 3 illustrates a flow diagram for a method for evaluation of scripts of a load testing application, in accordance with an example embodiment; and
[0012] FIGS. 4A-4C illustrates an example user interface (UI) for displaying report associated with evaluation of scripts of a load testing application, in accordance with an example embodiment.
[0013] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems and devices embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
DETAILED DESCRIPTION
[0014] Performance of software applications can be tested by measuring various attributes of software applications, such as, responsiveness, throughput, reliability, agility, and so on, under various workload conditions. Said workload conditions are created or simulated by load testing applications. An example of such a load testing application LoadRunner®. A load testing application includes a script recorder. The script recorder is configured to record communications on the software application when the software application is accessed by customers and/or users. Such communications are recorded in form of scripts. The load recorder may open a browser in which load tester personnel can enter a website that is to be recorded. In certain instances, the load tester personnel may have an option to select a file to record in.
[0015] The load testing application may include files, namely, vuser_init, action, and vuser_end. The files, vuser_init and vuser_end, correspond to initiation and ending files respectively. The vuser_init and vuser_end files include recordings of scripts of the software application/website browser. In a load test, where usually files are called multiple times, the “vuser_init” and “vuser_end” files can be called only once. The action files include recordings of the software application/website browser. The load testing personnel (hereinafter referred to as a tester) can add multiple action files for recording. For example, the tester may perform certain actions in a web-browser. Once the tester’s actions are completed, the tester may stop the recording, and load testing application creates a script which emulates the actions performed by the tester. The script recorder can then generate multiple scripts related to the script by imitating said script. Once the scripts are created, the tester can replay the scripts multiple times. The replaying of the script allows in automatically repeating the action performed by the tester. The load testing application includes a load generator that is capable of replaying the recorded scripts, in some instances, with some modifications.
[0016] The replaying of the scripts can be performed for any number of iterations and/or for any duration. Said repetition may cause a load on the server of the software application and/or website. The load testing application captures behaviour of the server under load conditions in form of certain metrics. The load testing application includes components for monitoring said metrics of the software application during the replay of the recorded scripts. Such metrics may include, for example, the central processing unit (CPU), memory, disk IO of the physical servers and response time, throughput of the System Under Test (short as SUT), and so on. Said metrics are analysed, and based on the analysis a load testing report is generated. The load testing report is indicative of the performance of the software application, and can be utilized by developers of the website to improve website’s performance.
[0017] The scripts generated by the load testing application plays an important role for performance evaluation of the software application, since said scripts may be automatically played multiple times and/or for multiple different durations to thereby load the server. It is important that the scripts are in compliance of various quality assessment standards. There may be numerous gaps in recording of scripts that may lead to incorrect performance evaluation of the load testing application.
[0018] The present subject matter describes systems and methods to evaluate the standards of the scripts of the load testing application. In an implementation, said system may include hardware and software that may be collectively configured to host an Information Technology (IT) application for performing various functions pertaining to performance evaluation of the load testing application. In the foregoing discussion, said IT application for evaluation of the scripts of the load testing application shall be termed as ‘scripts evaluation application’ or a ‘scripts evaluation system’.
[0019] In an embodiment, the disclosed scripts evaluation system for a load testing application may take, as input, the scripts generated by the load testing application, to evaluate the standards of the scripts. The scripts evaluation system may then analyse the scripts based on a plurality of rules associated with one or more performance parameters of the scripts. The performance parameters may include the standards of the scripts. Examples of said performance parameters may include, but are not limited to, files, transactions, dynamic values, MAX_HTML parameter, placement of web_reg_find before web_url, and functions within transactions. The details of said performance parameters and the plurality of rules associated with the performance parameters are described further in detail with reference to FIGS. 1 through 4C.
[0020] FIG. 1 illustrates a network environment 100 implementing a scripts evaluation system 102 for a load testing application, according to an embodiment of present disclosure. The network environment 100 may be understood as a public or a private networking system. As shown, the network environment 100 may include the scripts evaluation system 102 coupled to a load testing application server 104 over a network 106 through one or more communication links 108a, 108b.
[0021] The network 106 may be understood as a network, including personal computers, laptops, various servers and other computing devices. The communication links 108a, 108b between the scripts evaluation system 102 and the load testing application server 104 are enabled through a desired form of communication, for example, via dial-up modem connections, cable links, and digital subscriber lines (DSL), wireless or satellite links, or any other suitable form of communication. Further, the network 106 may be a wireless network, a wired network, or a combination thereof. The network 106 can also be an individual network or a collection of many such individual networks, interconnected with each other and functioning as a single large network, e.g., the Internet or an intranet. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and such. The network 106 may either be a dedicated network or a shared network, which represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), etc., to communicate with each other. Further, the network 106 may include network devices, such as network switches, hubs, routers, and Host Bus Adapters (HBAs) for providing a link between the scripts evaluation system 102 and the load testing application server 104. The network devices within the network 106 may interact with the scripts evaluation system 102 and the load testing application server 104 through the communication links 108a, 108b.
[0022] The scripts evaluation system 102 may be embodied in a computing device 112. Examples of the computing device 112 may include, but are not limited to, a desktop personal computer (PC), a notebook, a laptop, a portable computer, a smart phone, a tablet, and the like. The load testing application server 104 may be one, or combination of one or more, storage server or network server. In an embodiment, the scripts evaluation system 102 may display a screen of the scripts evaluation system 102 and allow the users to input data.
[0023] Although, it is shown that the scripts evaluation system 102 is external to the load testing application/system, in some embodiments, the scripts evaluation system 102 may be embodied in the load testing server or any other computing device embodying the load testing system. The scripts evaluation system 102 and functionalities of various components thereof are described further with reference to FIG. 2.
[0024] FIG. 2 illustrates a block diagram of a system 200 for evaluation of scripts of a load testing application in accordance with an example embodiment. The system 200 may be an example of the scripts evaluation system 102 (FIG. 1). Here, the system 200 facilitates in evaluation of the standards of the scripts generated from a load testing application, and hence the system 200 may also be referred to as script evaluation system. Hereinafter, the terms system 200 and script evaluation system 200 may be used interchangeably throughout the description.
[0025] In an example embodiment, the system 200 may be embodied in, or is in direct communication with a computing device, for example the computing device 112 (FIG. 1). The system 200 includes or is otherwise in communication with one or more hardware processors such as a processor 202, at least one memory such as a memory 204, and a communication interface 206. The processor 202, memory 204, and the communication interface 206 may be coupled by a system bus such as a system bus 208 or a similar mechanism.
[0026] The at least one memory such as the memory 204, may store instructions, any number of pieces of information, and data, used by a computer system, for example the system 200 to implement the functions of the system 200. The memory 204 may include for example, volatile memory and/or non-volatile memory. Examples of volatile memory may include, but are not limited to volatile random access memory (RAM). The non-volatile memory may additionally or alternatively comprise an electrically erasable programmable read only memory (EEPROM), flash memory, hard drive, or the like. Some examples of the volatile memory includes, but are not limited to, random access memory, dynamic random access memory, static random access memory, and the like. Some example of the non-volatile memory includes, but are not limited to, hard disks, magnetic tapes, optical disks, programmable read only memory, erasable programmable read only memory, electrically erasable programmable read only memory, flash memory, and the like. The memory 204 may be configured to store information, data, applications, instructions or the like for enabling the system 200 to carry out various functions in accordance with various example embodiments. Additionally or alternatively, the memory 204 may be configured to store instructions which when executed by the processor 202 causes the system 200 to behave in a manner as described in various embodiments.
[0027] The memory 204 also includes module(s) 210 and a data repository 230. The module(s) 210 include, for example, a reception module 212, a transaction identification module 214, a mapping module 216, and other module(s) 218. The other modules 218 may include programs or coded instructions that supplement applications or functions performed by the performance evaluation system 200. The data repository 230 may include scripts data 232, rule(s) data 234, and other data 236. Further, the other data 238 amongst other things, may serve as a repository for storing data that is processed, received, or generated as a result of the execution of one or more modules in the module(s) 210.
[0028] Although the data repository 230 is shown internal to the scripts evaluation system 200, it will be noted that, in alternate embodiments, the data 230 can also be implemented external to the scripts evaluation system 200, where the data 230 may be stored within a database communicatively coupled to the scripts evaluation system 200. The data contained within such external database may be periodically updated. For example, new data may be added into the database and/or existing data may be modified and/or non-useful data may be deleted from the database. In one example, the data may be stored in an external system. In another embodiment, the data stored in the data repository may be distributed between the scripts evaluation system 200 and the external database.
[0029] The hardware processor such as the processor 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that facilitates in evaluation of scripts of the load testing application. Further, the processor 202 may comprise a multi-core architecture. Among other capabilities, the processor 202 is configured to fetch and execute computer-readable instructions or modules stored in the memory 204. The processor 202 may include circuitry implementing, among others, audio and logic functions associated with the communication. For example, the processor 202 may include, but are not limited to, one or more digital signal processors (DSPs), one or more microprocessor, one or more special-purpose computer chips, one or more field-programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), one or more computer(s), various analog to digital converters, digital to analog converters, and/or other support circuits. The processor 202 thus may also include the functionality to encode messages and/or data or information. The processor 202 may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor 202. Further, the processor 202 may include functionality to execute one or more software programs, which may be stored in the memory 204 or otherwise accessible to the processor 202.
[0030] The communication interface 206 is configured to facilitate communication between the load testing application server 104 and the system 200 (or the computing device 112 embodying the system 200). The communication interface 206 may be in form of a wireless connection or a wired connection. Examples of wireless communication interface 206 may include, but are not limited to, IEEE 802.11 (Wifi), BLUETOOTH®, or a wide-area wireless connection. Example of wired communication interface 206 includes, but is not limited to Ethernet.
[0031] In an example embodiment, a user interface 240 may be in communication with the processor 202. Examples of the user interface 240 include but are not limited to, input interface and/or output user interface. The input interface is configured to receive an indication of a user input. The output user interface provides an audible, visual, mechanical or other output and/or feedback to the user. In this regard, for example, the processor 202 may comprise user interface circuitry configured to control at least some functions of one or more elements of the user interface 240, such as, for example, a speaker, ringer, microphone, display, and/or the like. The processor 202 and/or user interface circuitry comprising the processor 202 may be configured to control one or more functions of one or more elements of the user interface 240 through computer program instructions, for example, software and/or firmware, stored on a memory, for example, the memory 204, and/or the like, accessible to the processor 202.
[0032] In an embodiment, the scripts evaluation system 200 is implemented for evaluation of the scripts generated from the load testing applications. According to an implementation, for the purpose of said evaluation, the system 200 may receive one or more scripts recorded from the load testing system. In an embodiment, the reception module 212 may receive the scripts from the load testing application server. The recorded scripts may be in same format or different formats. For example, the recorded scripts can be in formats such “.c”, “.java”, and so on.
[0033] The recorded scripts may include transactions. The transactions are the timers that can be applied to any action performed by a user (or a tester) on the website. For example, when a request to access a particular page or a website is made (for example, by providing a web address of a website in a browser), the user actions performed while making the request may be referred to as a transaction. The transactions are utilized for determining time taken (or response time) of a particular request. A script may include a plurality of initiate-transaction statements and a plurality of terminate-transaction statements. An initiate-transaction statement, for example, lr_start_transaction is inserted before beginning of a request/transaction, which is to be analysed. In particular, the lr_start_transaction is utilized to check the response time of a particular request, for example, a request to load a web page. The initiate transaction, for example, lr_start_transaction and the terminate-transactions, for example, lr_end_transaction are inserted before and after the request, respectively.
[0034] In a scenario where the transactions are not closed properly, the load testing application may not make accurate calculations of the time taken by particular user-actions. Additionally, to get an accurate reading on the time taken, the tester may create nested transactions, which allows the tester to get specific timings. Hence, it is necessary to make sure that the nesting of the transactions is done properly. In an embodiment, for evaluation of the scripts of the load testing application, the system 200 may be caused to verify whether an initiate-transaction is ended (or terminated) and/or vice-verse. Additionally, the system 200 determines whether not the transactions are nested properly.
[0035] In an embodiment, to determine proper closing and/or nesting of the transaction in the scripts of the load testing application, the system 200 may be caused to identify a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions in each of the scripts. In an embodiment, the transaction identification module 214 identifies the plurality of initiate-transaction statements and the plurality of terminate-transaction statements in the script. In an embodiment, the transaction identification module 214 may identify the initiate-transaction statements and the terminate-transaction statements based on the names of the transactions. In an embodiment, each lr_start_transaction has a corresponding lr_end_transaction. For example, for a transaction named trns_1, Lr_start_transaction("trns_1") may denote an initiate-transaction statement, and a lr_end_transaction("trns_1,"LR_AUTO") may denote a terminate-transaction statement. Herein, it may be noted that, the name of the transaction in both the initiate-transaction statement and the terminate-transactions statement is "trns_1". Hence, a pair of transactions having same initiate and terminate statements always has the same name.
[0036] In an embodiment, the system 200 is caused to systematically map the plurality of initiate-transaction statements with the plurality of terminate-transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements. In an embodiment, the mapping module 216 is configured to systematically map the initiate-transaction statements with the terminate-transaction statements. In an embodiment, systematically mapping the initiate-transaction statements and the terminate-transaction statements includes comparing the names of the initiate-transaction statement and the terminate-transaction statements. Herein, the names of transactions in the initiate and terminate transactions statements may be matched in an order from an innermost transaction towards outermost transaction in the script so as to identify pairs of matching transactions. An example of systematically matching the initiate-transaction statements and terminate-transaction statements is described below. Considering an example script given below:
lr_start_transaction ("trns_1");
lr_start_transaction ("trns_2");
lr_start_transaction ("trns_3");
lr_end_transaction ("trns_3,"LR_AUTO");
lr_end_transaction ("trns_2,"LR_AUTO");
lr_end_transaction ("trns_1,"LR_AUTO")
[0037] In the above example, the system 200 may identify the inner most transaction "trns_3". The mapping module 216 may further determine whether first occurrence of terminate-transaction after a last occurrence of initiate-transaction in the script have same name. For example, in the present embodiment, the first occurrence of the terminate-transaction lr_end_transaction("trns_3,"LR_AUTO") is after the last occurrence of initiate-transaction lr_start_transaction("trns_3") corresponding to the transaction "trns_3". Upon determination, that the first occurrence of terminate-transaction after the last occurrence of initiate-transaction in the script for the innermost transaction have the same transaction name, the mapping module 216 may further identify whether a subsequent transaction which is next to the innermost transaction follows the rule of “whether a first occurrence of terminate-transaction after the last occurrence of initiate-transaction in the script have the same name”. In the above example, the subsequent transaction is “trans_2”. The first occurrence of the terminate-transaction is “lr_end_transaction ("trns_2,"LR_AUTO")” and the last occurrence of the initiate-transaction is “lr_start_transaction ("trns_2")”. As is seen, the transaction name in the first occurrence of the terminate-transaction is same as the transaction name in the last occurrence of the initiate-transaction, thereby indicating matching of the initiate-transaction statements and terminate-transaction statements.
[0038] The systematic mapping of the initiate-transactions and the terminate-transactions is repeated iteratively until the outermost initiate transaction has been reached. As is noted, in the above example the outermost initiate transaction is the “lr_start_transaction (“trns_1”)”. In an embodiment, the systematic mapping of the initiate-transactions and the terminate-transactions facilitates in determining whether the transactions are nested properly. For example, in case the first occurrence of the terminate-transaction is not determined to be same as the transaction name in the last occurrence of the initiate-transaction, it can be implied that said transaction is not nested properly. In an embodiment, the system 200 may be caused to display the name of transaction that is determined to be nested improperly. In an embodiment, the user interface 240 of the system 200 may display the transactions that may not be nested properly.
[0039] Herein, it will be understood that for the scripts in “.c” format, the initiate-transaction statements and the terminate-transaction statements includes “lr_start_transaction” and “lr_end_transaction” respectively, and for the scripts in “.java” format, then the initiate-transaction statements and the terminate-transaction statements includes “lr.start.transaction” and “lr.end.transaction”. In an embodiment, the system 200 may be caused to automatically determine the format of the scripts input to the system 200, and in accordance with determination of the format, the system may be caused to evaluate the scripts.
[0040] In an embodiment, the mapping module 216 is further caused to determine whether the initiate-transactions are closed properly by matching the initiate transactions with the terminate transactions. The mapping module 216 is caused to identify the terminate-transaction statements corresponding to each of the initiate-transaction statements. For example, the mapping module 216 may identify whether or not corresponding to the initiate transaction "trns_3", a terminate transaction "trns_3"exist. Alternatively or additionally, the mapping module 216 may also be caused to determine whether or not corresponding to each of the terminate transactions, the initiate transactions exist. If it is determined that corresponding to an initiate transaction, a terminate transaction does not exist, the user interface 240 may be caused to display said transaction as not being ended. Such transaction corresponding to which an initiate transaction (in case, said transaction is a terminate-transaction) or a terminate transaction (in case, said transaction is an initiate-transaction) is not existing, may be categorized as non-matching transactions. Similarly, the transactions corresponding to which an initiate transaction (in case, said transaction is a terminate-transaction) or a terminate transaction (in case, said transaction is an initiate-transaction) is existing, may be categorized as matching transactions.
[0041] In an embodiment, the mapping module 216 may include a counter for determining the number of matching transactions, as well as non-matching transactions. For example, when the mapping module 216 determines that corresponding to an initiate-transaction, a terminate-transaction is existing, the mapping module 216 may increment value of the counter by one. Likewise the mapping module 216 may be caused to determine a set of non-matching transaction statements based on the matching of the names of the transactions. Herein, a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application. For example, more the number of non-matching transactions statements, lower is the quality of the scripts generated by the load testing application. In an embodiment, the number of non-matching transactions statements being less than a predetermined threshold may be indicative of unacceptable quality of the scripts.
[0042] In an embodiment, the system 200 is further capable of applying one or more rules associated with one or more quality testing parameters to the one or more scripts for evaluation of the scripts generated from the load testing application. In an embodiment, the one or more of rules may be associated with a plurality of performance parameters of the load testing application scripts. The plurality of performance parameters may include files, dynamic values, MAX_HTML parameter, placement of web_reg_find, and functions within the transactions. The one or more rules may be stored in the rules data 234 of the memory 204.
[0043] As described above, the scripts of the load testing application includes transac-tions having initiate-transaction statement and terminate-transaction statements. The initiate-transaction statements and terminate-transaction statements may be present in the files, such Vuser_init” file and “vuser_end” files. During a load test, it may be required to log-in and log-out multiple time. In an embodiment, the rules corresponding to the performance param-eter “files” may include determination whether or not the files “Vuser_init” and “vuser_end” include code for log-in and log-out. The code in said files includes user-actions corresponding to log-in and log-out that are to be performed only once during the replay of the script. The rules corresponding to the files enables in verifying whether or not said files contain codes for the one-time user-actions, thereby helping the testing personnel to decide whether or not the user-actions need to be placed in the files. For example, if the files “Vuser_init” and “vuser_end” do not contain any code, the system 200 may provide an indication to the tester to add a code, for example, log-in code under “vuser_init” and logout under “vuser_end”, so that said functions can be performed only once during the load test. If code is added to the files “Vuser_init” and “vuser_end”, the need to login every time and logout every time can be eliminated, thereby providing more accurate results for the load testing.
[0044] Additionally or alternatively, the rules corresponding to the “files” may include determination whether the file vuser_init includes a script summary. The script summary associated with scripts may include script metadata such as script title, script description, script owner, data requirements, dependencies, date of creation, protocol, script review and change history. The availability of the script summary ensures availability of said metadata for later references and modifications in the scripts, there by facilitating in maintaining consistency of the scripts, in case any changes or modifications are made to the scripts.
[0045] In an embodiment, the system 200 may further be caused to apply one or more rules associated with dynamic values for evaluation of scripts of the load testing application. Herein, the dynamic values may be associated with communication session of the software application and/or the website, and may include various values. Examples of such values may include, but are not limited to, session authentication value, and so on. For example, considering a scenario wherein when a user logs in to a website, the user may utilize log-in credentials, that can be entered into text boxes on the webpage of the website. When the user clicks the "log-in" button, data corresponding to the log-in credentials is sent to the server. Upon validating that the log-in credentials, the sever sends values to the browser. These values could be session IDs or extra information, identifying the user. These values are not constant and change depending on the users which may be trying to log-in at the website. Since the values are changing, said values may be considered as dynamic values, and are to be captured so that the values can be passed on to subsequent requests (during replay of scripts).
[0046] In another example scenario, consider a shopping website. While browsing through the website, a "Laptops" category is selected, then a value corresponding to said category is sent to the server, and in response the server may send back data that is relevant to the "Laptops" category. The values received from the server may be different for different laptops (for instance, the values may correspond to the dimensions, configurations, laptop company, prices, and so on of different laptops). On selection of a subcategory corresponding to the category laptop, the server may send new values again, and hence said values can be considered as dynamic values, and are to be captured.
[0047] In yet another example scenario, consider an online payment portal that provides a list or dropdown of all the banks for the payment. A bank from the list may be selected for payment. On selecting the bank, the bank name may be sent to the server, and in response, the server may return certain values or identifiers based on the bank selected. The values sent by the server may differ for different banks. Hence said values can be considered as dynamic values and are to be captured.
[0048] Herein, various examples have been provided for explanation of the term ‘dynamic values’, however, the above described examples should not be considered as limiting to the present disclosure. In various embodiments and scenarios, the dynamic values may be associated with various applications, including the ones which are not described herein.
[0049] The dynamic values may be utilized for determining presence of correlation in the values that are received from the server of the website. The correlation of the values is to be carried out since actions carried out in a real life situation may give different or new values from the server. Since, the load testing application replays a recorded set of user-actions, it is expected at the load testing application that the response received is same as the one received at the time of recording the script. It may however not be possible since the values from the server may change. The one or more rules associated with determination of dynamic values in the script ensures that correlations are present in the script.
[0050] The presence of correlation may be determined based on following four functions:
a) “web_reg_save_param_ex”
b) “web_reg_save_param_regexp”
c) “web_reg_save_param_xpath”
d) “web_reg_save_param”
[0051] Any one of the above four functions can be used to correlate values. A script developer (or a script testing professional) working on the scripts can use said functions interchangeably, even in a single script. The system 200 may be caused to determine the functions from amongst the above defined four functions that are used in the script for correlating the values. The system may further be caused to determine whether said functions are placed at the proper points in the script. For example, the correlation functions are to be called before the "web_url" function is called. The web_url function may perform actual user action. The correlation function must be called before the "web_url" function so that the value can be captured from the previous "web_url" function call and then passed to the following/subsequent “web_url” function call.
[0052] In some scenarios, when newer versions of the load testing applications are released, the usage of the load testing application can be increased because of inclusion of various additional features to the updated versions. In such scenarios, the correlation functions may also change or be deprecated. In an embodiment, the system 200 is caused to determine whether the correlation functions are deprecated in a current version of the load testing application. In an embodiment, upon such determination, the system 200 is caused to provide an indication that the said functions are deprecated. Additionally or alternatively, the system 200 may be caused to update the functions with current definitions thereof. Additionally or alternatively, the system may be caused to determine whether various naming conventions for functions are followed in compliance with the naming convention of the updated version of the load testing application.
[0053] In certain scenarios, one or more values captured during the recording of scripts may be longer than an allowable default limit of value length of the load testing application. In such a scenario, the load testing application may tend to ignore an extra length of the value and may only capture the default values, thereby causing errors in the script during replay since a part of the value of the script have been omitted. In an embodiment, the system may further be caused to apply one or more rules associated with a maximum length parameter for performance evaluation of the load testing application. In the present embodiment, the system 200 may be caused to apply the maximum length parameter so that a required size can be set and all values are captured properly. Examples of said parameter may be MAX_HTML parameter. In an embodiment, a SET_MAX_HTML function is called and a value is passed into the method. For Example, SET_MAX_HTML (1024). The example represents a function call to the method and 1024 represents the value to which the maximum size is set. The system 200 is configured to check whether or not said function call is made in the script.
[0054] In an embodiment, the system 200 may further be caused to apply one or more rules associated with web_reg_find parameter for evaluation of scripts of the load testing application. During the replay of the recorded scripts, the load testing application sends the captured requests for accessing web page without using the browser. To send the captured requests, the load testing application utilizes the URL (“web_url”) of the webpage/software application that has been navigated. In order to verify that the response that is received is the correct one, testers places a “web_reg_find” function which finds a particular text in the response received from the server of the webpage. If the text is found then it means that the expected page has been received. In an embodiment, for this to happen, the system places the “web_reg_find” function before “web_url”, so that before LR sends out the request, it knows, what it need to check in the response.
[0055] Various methods placed within the transactions may include web_reg_find, web_url, and web_submit. The methods, namely, “web_url”and “web_submit” are utilized by the load testing application for sending a request to the server. If said functions are placed outside the transactions, then the time taken to send the request and receive the response from the server cannot be obtained, thereby leading to wrong reporting in a performance test report generated by the load testing application. In an embodiment, the system 200 may further be caused to apply one or more rules associated with functions placed within transactions for evaluation scripts of the load testing application. In an embodiment, the system 200 may search for the methods, web_reg_find, web_url, and web_submit to determine whether said functions are present inside a transaction. In an embodiment, the system 200 may determine whether the methods, namely web_reg_find, web_url, and web_submit, are in-between a pair of transactions, i.e. a start transaction and an end transaction of a script. If said methods are placed outside the pair of transactions, then, the system 200 may provide a recommendation that the methods in such case must be placed inside the transactions.
[0056] In an embodiment, the system 200 may include additional rules for evaluating the performance of the scripts of the scripts. Such additional rules may be related to various quality testing parameters such as think time, checkpoints, Messages, Cookies and cache and comments. Herein, think time refers to the time during which a user of a software application awaiting between perfoming actions. The system 200 may be cause to determine presence of think time in the scripts, and further verify that think time is less than a threshold value of the think time. In case, the script format is in “.c”, the system 200 may be caused to utilize the statements, “lr_thinktime ()” and “.java takes “lr.thinktime ()”, for evaluation of the script based on the think time.
[0057] The quality testing parameter namely checkpoint enables the system to perform a search for presence for “Text/Image checks”. In case, the script format is in “.c”, the system 200 may be caused to utilize the statement “web_reg_find” for performing the search. Alternatively, in case, the script format is in “.java”, the system 200 may be caused to utilize the statement the system 200 may be caused to utilize the statement “web.reg.find” for performing the search. The quality testing parameter, namely messages enables the system 200 to determine whether the web_Set_Max_HTML paramter is set. The rule for the quality testing parameter namely Cookies and cache determines whether cookies are disabled, and ensures Cache cleanup is used. In case, the script format is in “.c”, the system 200 may be caused to utilize “web_add_cookie” for cookies and “web_cache_cleanup” for clearing the cache. Alternatively, in case, the script format is in “.java”, the system 200 may be caused to utilize “web.add.cookie” for cookies and “web.cache.cleanup” for clearing the cache. The rule for the quality testing parameter comments searches whether or not comments are inserted in the scripts.
[0058] In an embodiment, the scripts received at the system may be in different formats. For example, various scripts may be in .C or .Java format. The system may be configured to accept the scripts of different formats and evaluate standards of said scripts. In an embodiment, the system is capable of accepting multiple scripts and evaluating said multiple scripts in parallel.
[0059] The system further includes a reports generation module. The report generation module # is configured to generate reports for evaluation of scripts. An example UI and a report generated on the UI are described further with reference to FIG. 4C.
[0060] FIG. 3 is a flow diagram depicting an example method 300 for evaluation of scripts of a load testing application, in accordance with an example embodiment. The method 300 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types. The method 300 may also be practiced in a distributed computing environment where functions are performed by remote processing devices that are linked through a communication network. The order in which the method 300 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 300, or an alternative method. Furthermore, the method 300 can be implemented in any suitable hardware, software, firmware, or combination thereof.
[0061] In an embodiment, the method 300 depicted in the flow chart may be executed by a system, for example, the system 200 of FIG. 2. In an example embodiment, the system 200 may be embodied in a computing device, for example, the computing device 112 (FIG. 1).
[0062] At 302, the method includes receiving one or more scripts generated by the load testing application. In an embodiment, the load testing application may be a LoadRunner® application. In an embodiment, the one or more scripts may be received at a hardware including one or more processors and one or more memory devices effecting a computing device, for example the computing device 112.
[0063] At 304, the method 300 includes identifying, in each script of the one or more scripts, a plurality of initiate-transaction statements and a plurality of terminate-transaction statements associated with one or more transactions of the script. At 306, the method 300 include systematically mapping the plurality of initiate transaction statements with the plurality of terminate transaction statements in an iterative manner, to identify at least one of a set of matching pairs of initiate and terminate transaction statements and a set of non-matching transaction statements. In an embodiment, the set of non-matching transaction statements includes one or more initiate transaction statements and one or more initiate transaction statements. In an embodiment, at least a count of statements in the set of non-matching transaction statements is indicative of quality assessment of the scripts generated from the load testing application.
[0064] In various embodiment, the method may further include applying one or more rules associated with one or more quality testing parameters to said scripts for performance evaluation of the load testing application. Herein, the quality testing parameters for evaluation of scripts may include, but are not limited to, files, think time, dynamic values, checkpoints, messages, MAX_HTML parameter web_reg_find function, Cookies and cache, Functions placed within transactions, and comments. The rules associated with each of said quality testing parameters are described with reference to FIG. 2, and thus for the brevity of description, are not described herein again.
[0065] In an embodiment, the system may be caused to grade the scripts for an easier understanding of compliance to standards. For example, each rule of the rules may be assigned a weightage. In an embodiment, the weightage assigned to each of the rules may be equal. In another embodiment, the weightage assigned to each of the rules may vary depending on sensitivity of compliance of said rule. For example, the compliance of rules for determining whether the transactions are nested properly, determination of matching and non-matching transactions, placement of web_reg_find is placed before web_url, determination whether functions are placed or not inside the transactions, and cache check and cookies check) may be more sensitive than compliance of rest of the rules for evaluation of the scripts. In case the script is determined to be in compliance of a rule, a corresponding weightage may be assigned to the script, and in case, the script is determined to be non-compliant to a rule, and then the weightage associated with said rule may not be added to an aggregate weightage of the script. Upon performing a check for all the rules on the script, the system 200 may compute the aggregated weightage for grading the script. In an embodiment, the system may be caused to compute a ratio of the number of rules compliant and the total number of rules to determine the grade assigned to the script.
[0066] FIGS. 4A-4C illustrates an example user interface (UI) for evaluation of scripts of a load testing application, in accordance with an example embodiment. In an embodiment, the UI may be implemented by a system for evaluation of scripts of a load testing application. For example, the UI may be implemented by the system 200 (FIG. 2), and the UI may be an example of the UI 240.
[0067] Referring now to FIG. 4A, a window 402 of the UI is displayed to a user of the system for allowing the user to access the system 200. The window 402 may include a browsing sub-window 404 for allowing the user to select the scripts that are to be evaluated. In an embodiment, the user may select multiple scripts for evaluation by browsing said scripts through the sub-window 404. In an embodiment, the multiple scripts selected by the user may be in different formats, for example, “.c” format, “.java” format, and so on.
[0068] Referring to Fig. 4B, a window 410 of the UI 400 is displayed. The window 410 is shown to include a screen displaying the parsing of the selected scripts for evaluation of said scripts. FIG. 4C illustrates a reports screen 420 that may be generated based on the evaluation of the selected scripts. As illustrated in FIG. 4C, the reports screen 420 may illustrate the compliance checks corresponding to various rules for the selected scripts. Additionally or alternatively reports screen 420 may illustrate a grade assigned to the scripts based on the compliance check passed by the scripts.
[0069] Various embodiments provide method and system for evaluation of scripts generated from the load testing application. The disclosed system is capable of intelligently apply rules associated with quality assessment parameters of scripts. The disclosed system is further configured to process a batch of scripts together and generate a quality assessment report for the same.
[0070] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
[0071] It is, however to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
[0072] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[0073] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.