DESC:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
SYSTEM AND METHOD FOR DETERMINING SERVER RESPONSE TIME FOR A USER INITIATED WEB PAGE REQUEST

Applicant
TATA Consultancy Services Limited
A Company Incorporated in India under The Companies Act, 1956
Having address:
Nirmal Building, 9th Floor,
Nariman Point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The present subject matter described herein, in general, relates to a system to determine a server response time, and more particularly to the system for determining a server response time for a user initiated web page request.

BACKGROUND
The dependency on internet and web applications has drastically increased a load over the servers. The servers are supposed to supply data and are used as an interface to pull out relevant information as per a user’s requirement. The user while requesting for a web page is generally not aware how the data is being pulled and thus expects every time a quick reply to his web page request. To analyze, how quickly server serves the user request, most common monitoring done. By analyzing the server response time, problems may be identified and measures could be taken to maintain this quick retrieval with respect to the web page request to meets the user’s satisfaction index.
Large number of solutions has been proposed to determine and analyze the server response time. Many tools are present in market today, out of which some are commercial, some are open source and some are research oriented. Most of the tools require execution of additional scripts either at client side or server side. Although, the solutions dedicated towards estimation of the server response time at client end are good in accuracy, but such solutions are dependent on other components like network connectivity which are not governed by web-server and may not be helpful in gaining deep insight into current status of server and measuring web-server performance alone. Further, methods and tools used to determine server response time at server end are complex in nature and consume extra resources which may compete with live systems and hinder their performance. Moreover, certain tools when used for determining the server response time at server end require input in a particular format providing session information and therefore extra time and resources are required to provide the input in that particular format.
Further, in many cases, it is a common practice for a system administrator just to collect bare minimum information in certain formats like CLF (Common Log Format) to reduce the logging stress on the live system. However, in such cases the absence of sessions’ pertinent data from the access logs due to which deriving cluster information just from the access-log data becomes a challenge.
In all it is a challenge to address this process of determining the server response time in a simpler manner and by using minimum information. Therefore, there is need to develop such a solution which is capable of determining the server response time in an efficient manner without exhausting too much of resources.
SUMMARY
This summary is provided to introduce aspects related to system and method for determining the server response time for a user initiated web page and the aspects are further described below in the detailed description. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.
In one implementation, the present invention provides a system to determine a server response time for a user initiated web page request. The system comprises of a data storing means configured to store access log data so collected with respect to the user initiated web page request and a processor coupled to a memory and to execute a plurality of modules. The processor is configured to sort and analyze the access log data by means of a programmed set of instructions to determine the server response time. The memory comprises of plurality of modules. The memory comprises of an analytical module configured to perform an analysis of the access log data and filter the access log data by removing unwanted access log data and adds a filtered access log data into input parameters. The memory further comprises of a cluster generating module configured to generate clusters by applying a clustering technique over the input parameters, wherein each cluster represents a group of web components, the web components which are part of a single web-page request initiated by a web user. The cluster generating module is configured to assigns a membership vector to each web-component; the membership vector represents a probability of the web-component belonging to a cluster of a parent web page request appearing in the access log data. Further the cluster generating module, use the membership vector so assigned to determine cluster information .The memory further comprises of a time calculation module configured to use the cluster information so generated, to calculate the server response time in terms of timestamps difference within each cluster.
The present invention also provides a method to determine server response time for a user initiated web page request. The method comprises of storing access log data so collected with respect to the user initiated web page requests and processing the access log data to sort and analyze the access log data to determine the server response time. The processing further comprises of performing an analysis of the access log data and filter the access log data by removing unwanted access log data and adds a filtered access log data into input parameters, generating a cluster by applying a clustering technique over the input parameters, wherein each cluster represents a group of web components, the web components which are part of a single web page request. The clustering technique assigns a membership vector to each web-component request from the access log data. The membership vector represents a probability of the web-component belonging to a parent web page request appearing in the access log data. The method further comprises of using the membership vector so assigned to determine cluster information. The cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.
BRIEF DESCRIPTION OF THE DRAWINGS
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to refer like features and components.
Figure 1 illustrates a network implementation of a system for determining a server response time is shown, in accordance with an embodiment of the present subject matter.
Figure 2 illustrates the system, in accordance with an embodiment of the present subject matter.
Figure 3 illustrates a server response time calculation in accordance with an exemplary embodiment of the invention.
Figure 4 illustrates a server response time calculation in accordance with another exemplary embodiment of the invention.
Figure 5 illustrates a server response time calculation in accordance with another exemplary embodiment of the invention.
Figure 6 illustrates a method for determining the server response time, in accordance with another embodiment of the present subject matter.

DETAILED DESCRIPTION
System and method for determining a server response time for a user initiated web page request are disclosed. An access log data in CLF (Common Log Format) is collected with respect to each request initiated by a user for accessing a web page. The access log data is further preprocessed by performing an analysis over the access log data. The access log data after performing the preprocessing analysis is then used for generating clusters by applying a clustering technique. Later, cluster information is used to calculate the server response time in terms of timestamp difference within each cluster.
While aspects of described system and method for determining a server response time may be implemented in any number of different computing systems, environments, and/or configurations, the embodiments are described in the context of the following exemplary system.
Referring now to Figure 1, a network implementation of a system for determining a server response time is illustrated, in accordance with an embodiment of the present subject matter. In one embodiment, the system collects an access log data in CLF (Common Log Format) and processes and filters the access log data to further generate clusters. The cluster information is later used for determining the server response time.
Although the present subject matter is explained considering that the system 102 is implemented as an application on a server, it may be understood that the system 102 may also be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, and the like. It will be understood that the system 102 may be accessed by multiple users through one or more user devices 104-1, 104-2…104-N, collectively referred to as user 104 hereinafter. Examples of the user devices 104 may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, and a workstation. The user devices 104 are communicatively coupled to the media system 102 through a network 106.
In one implementation, the network 106 may be a wireless network, a wired network or a combination thereof. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network 106 may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use Hypertext Transfer Protocol (HTTP), to communicate with one another. Further the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
Referring now to Figure 2, the system 102 is illustrated in accordance with an embodiment of the present subject matter. In one embodiment, the system 102 may include at least one processor 202, an input/output (I/O) interface 204, and a memory 206. The at least one processor 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processor 202 is configured to fetch and execute computer-readable instructions stored in the memory 206.
The I/O interface 204 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface 204 may allow the media system 102 to interact with a user directly or through the client devices 104. Further, the I/O interface 204 may enable the media system 102 to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface 204 can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface 204 may include one or more ports for connecting a number of devices to one another or to another server.
The memory 206 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory 206 may include modules 208 and data 210.
The modules 208 include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. In one implementation, the modules 208 may include an analytical means, a cluster generating module, a time calculation module and other modules 218. The other modules 218 may include programs or coded instructions that supplement applications and functions of the system 102.
The data 210, amongst other things, serves as a repository for storing data processed, received, and generated by one or more of the modules 208. The data 210 may also include an access log data in CLF (Common Log Format), and other data 130. The other data 130 may include data generated as a result of the execution of one or more modules in the other module 218.
Referring to Figure 2, details of the system 102 is illustrated, in accordance with an embodiment of the present subject matter. In one implementation, in order to determine the server response time a data storing means (226) is configured to store an access log data. The access log data is in CLF (Common Log Format) without any session information and is collected with respect to a user initiated web page. The system 102 comprises of a processor (202), a memory (206). The memory stores plurality of modules (208). The plurality of modules (208) further includes an analytical module (212), a cluster generating module (214) and a time calculation module (216).
Each http request initiated by a web-user gets logged at server in form of the access-log data. The World Wide Web Consortium (W3C) maintains a standard format - Common Log Format (CLF) for the access-log data. The data storing means (226) collects the access log data and store the access log data for further processing.

Below mentioned is an example of entries in CLF format and Table 1 explains fields for the CLF format:

127.0.0.1 - bob [16/Oct/2012:13:55:36 +0530] "GET /index.html http/1.1" 200 2326
127.0.0.1 - bob [16/Oct/2012:13:55:36 +0530] "GET /logo.gif http/1.1" 200 323
127.0.0.1 - bob [16/Oct/2012:13:55:36 +0530] "GET /counter.jpg http/1.1" 200 23
Table 1
Value Description
127.0.01

-

Bob
[16/Oct/2012:13:55:36 +0530]

"GET /index.html http/1.1"

200

2326
IP address or host name of http client that made the request
The remote log-name of client
The username with which client has authenticated (if known)
Date, Time, and zone of the request.
Request line containing http request method, requested resource, and http protocol version
http status code
Size of the response object excluding the header returned to client, measured in bytes

Before determining the server response time, the system (102) first identifies a set of requests belonging to the web-page request of the user, as one web-page is constructed by large number of independent components (alternatively called web-resources) and one user request for a single web-page results into multiple http request entries in server access-log, each corresponding to the constituting components of that page (e.g. logo.gif and counter.jpg are part of index.html). This identification is very useful in determining the server response time.
The system (102) further comprises of the processor (202) coupled to the memory (206) to execute plurality of modules (218) for determining the server response time. The processor (202) uses programmed set of instructions for determining the server response time.
Before processing the data for server time determination, a step of pre-processing is performed by the processor (202) by executing the analytical module (212). The analytical module (212) performs an analysis of the access log data by performing data pre-processing tasks.
Data pre-processing tasks include data filtering, transformation, grouping and sorting. During these pre-processing steps, one or more impertinent entries are filtered out from access-log data. The log entries with erroneous status code 4xx and 5xx are excluded from analysis. As well as the entries with http requests other than get, put and post are filtered out.
The access-log data is then converted into a Dataset X€ { xij |1 = i = N; 1 = j = V} containing N number of data-points (http requests) and V number of measurements (variables) for each data-point. Here, two non-numerical data fields are used- information pertaining to client IP and web-component or web-resource requested (URI request); and one numerical attribute that is time at which the request received by the server (time-stamp).
In the next step, the analysis module (212) groups all entries coming from same IP, preserving order appeared in access-logs and sort in ascending order of total number of parent web-page requests coming from same IP.
The analysis module (212) further subdivides data by specifying upper limit on transaction duration. If difference in timestamp entries for two successive http requests coming from one IP is greater than this pre-specified limit, the algorithm (set of programmed instructions) marks the first entry as end of one logical session the second entry as beginning of new and divides data into two subsets for further analysis. The value for this parameter is recommended as twice the value for maximum time permitted by the SLA.
Finally, the analysis module (212), for each non-numerical variable j, prepares a master list of all possible values (J1, J2, …..Jv(j)) and unit-normalizes each numerical variable as shown in (1). Here Jmin and Jmax are minimum and maximum values for the numerical variable j in a logical session identified as above respectively.

The processer (202) further executes the cluster generating module (214) to generate clusters by applying a cluster technique over the input parameters. The input parameters further includes filtered access log data as created by the analytical module (212).
The cluster generating module (214) uses a variant of traditional FCM algorithm to accommodate both numerical as well as non-numerical variables. The FCM algorithm is an iterative algorithm and it requires five input parameters: Dataset X representing pre-processed entries from access-log data; number of clusters C sought (an integer); fuzziness factor ø (a real value 1 = ø < 8); tolerance limit € (a small positive real value); and maximum number of iterations max_itr (a large positive integer).
Initialization phase of this algorithm involves generation of one random membership M € {mik | 1= i = N’ 1 = k = C} matrix such that 0 = mik = 1 and . Here mik represents membership value of ith data-point in kth cluster. Instead of generating membership matrix completely in random fashion, the matrix may be initialized as per frequency of each requested resource appearing in log data, which will help in achieving faster convergence of the following iterative steps.
1) From the membership matrix M, module (204) calculates cluster centers matrix
CC € {ckj |1= k = C 1 = j = V} as follows:
a) For numerical variables: Cluster center is calculated as a fixed point as per (2).
……. (2)
(b) For non-numerical variables: Cluster center is calculated as an influence vector given by (3)

Here ? kjl represents influence value of a particular value Jl of variable j, while determining the cluster center ckj such that:
0 = ?kjl (?l € [1,v(j)]) = 1 and ?l=1v(j) ? kjl = 1.The influence value ? kjl is calculated as per (4).

2) Based on the matrix CC, module (214) calculates the dissimilarity matrix D €{ dik | 1= i = N; 1 = k= C}as per (5) where Wj represents weight assigned to variable j.
d_ik=?_(j ? non-numerical)^ ¦?W_j*(?_(l=1,x_ij?J_l)^(v(j))¦?_kjl )^2 + ? ?_(j ? numerical)^ ¦?W_j*(x_ij-c_kj )^2 ?
3) Based on this dissimilarity matrix D, module (214) re-computes the membership matrix M as per (6):

These steps are repeated by the processor until algorithm achieves convergences (|Mitr – Mitr-1| = €) or maximum number of iteration is reached. At the end of iterative phase, each entry from access-log data will have a membership vector representing probability of this web-component belonging to each parent web-page call appearing in access-log.
Based on the membership matrix generated at the end of clustering phase, each http request is tagged for a web-component to a parent web-page call having the highest membership value for that web-page cluster. At this point, a validation check is performed that– no two http requests for a particular web-component can get tagged to the same parent page. In multiuser scenario, it is often possible that concurrent users accessing the same web-page virtually at the same time. Due to which, access-log will have multiple entries for the same parent page and its constituent components having almost same timestamp. In such cases, algorithm may give same probability to multiple requests for a particular web-component. So here first request is assigned to the cluster having the highest probability, second request to the cluster having second highest probability and so on.
The processor (202) further executes the time calculation module (216) to use the membership vector so assigned to determine cluster information. Once the cluster information is obtained, each web-page request cluster may be observed and then estimate response time as difference between timestamps of the earliest and the latest entries for that particular cluster given by (7):

The present invention may be well understood by way of below mentioned working example.
The proposed algorithm is validated by the system (102) and processor (202) of the system (102) in two steps: first step was to validate the page tagging algorithm and second was to assess accuracy of response time estimation. The access-log data from three different sources is collected by the data storing means (226). Two datasets were obtained from two different project teams maintaining live applications in production setup and third access-log dataset was generated by conducting an in-house experiment. For this experiment three applications were hosted on a common apache tomcat web-server. Three applications deployed were: a) JPetStoreApp: It is a Pet Store application based completely on open source freeware (struts and others); b) NxGCel: It is a telecom reporting application on mobile usage; and c) VINS: Vehicle Insurance System is an in-house application.
Although the clustering algorithm is designed for CLF format, the system (102) is customized to capture Referrer – URL containing link (parent web-page) to resource (child component) being requested. This additional field is used only for validation purpose and has not been used in clustering phase described above. All experiments were conducted using following values for input parameters: number of clusters C as number of page calls, fuzziness factor ø as 1.5, tolerance limit € as 10-6 and maximum iterations of 100. Table II describes accuracy of the clustering algorithm.
Below is disclosed a table for validation for clustering:
Dataset 1 Dataset 2 Dataset 3( In-house Exp)
Accuracy 98.66% 98.62% 98.34%

To validate response-time estimation, another set of experiments were conducted by the system (102). For this purpose, JPetStoreApp (web-application running on Apache Tomcat web-server) and Grinder (an open source load testing framework) are used. To vary degree of concurrency system (102) has used 30 users having an inter-arrival time (IAT) of 30, 20 and 10 seconds. Figure 3, 4 and 5 shows an average response-time of eight web-pages of JPetStoreApp application in load-testing experiments. Figure 3, 4 and 5 also compares these results with estimated average response time by analyzing access-log data generated during load testing runs. Table III lists absolute error in response time estimation compared to value measured by Grinder.

Table III: Absolute Errors
30s IAT 20s IAT 10s IAT
Page 1 0.67% 11.97% 19.41%
Page 2 11.15% 9.23% 2.65%
Page 3 14.20% 7.21% 10.57%
Page 4 15.61% 3.90% 1.82%
Page 5 7.15% 10.38% 16.86%
Page 6 16.74% 0.27% 10.70%
Page 7 4.20% 4.42% 0.43%
Page 8 3.64% 7.25% 12.82%

This is further to be noted that Grinder reports end-to-end response time. It represents cumulative value for response time in all three phases of one http request: connection phase, server processing phase and network transmission phase, whereas proposed algorithm estimates only server response time. With the assumption that network delay is almost negligible in LAN, time measured by Grinder equals to server response time. From table III one can ascertain that upper bound on error in response time estimation is about 20%.
Referring now to Figure 6, a method 600 for determining the server response time is shown, in accordance with an embodiment of the present subject matter. The method 600 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types. The method 600 may also be practiced in a distributed computing environment where functions are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, computer executable instructions may be located in both local and remote computer storage media, including memory storage devices.
The order in which the method 600 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 600 or alternate methods. Additionally, individual blocks may be deleted from the method 600 without departing from the spirit and scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method 600 may be considered to be implemented in the above described media system 102.
At block 602, access log data so collected with respect to a user initiated web page requests is stored.
At block 604, an analysis of the access log is performed. The access log data is filtered by removing unwanted access log data and adds a filtered access log data into input parameters.
At block 606, clusters are generated by applying a clustering technique over the input parameters. Each cluster so generated represents a group of web components, the web components which are a part of a single web page request. The clustering technique assigns a membership vector to each web component from the access log data. The membership vector represents a probability of the web-component belonging to a parent web page request appearing in the access log data.
At block 608, the membership vector so assigned is used to determine cluster information. The cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.
Although implementations for methods and systems for determining the server response time for a user initiated web page request have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations for determining the server response time.
,CLAIMS:WE CLAIM:
1. A system to determine a server response time for a user initiated web page request, the system comprising:
a data storing means configured to store access log data so collected with respect to the user initiated web page request;
a processor coupled to a memory, configured to sort and analyze the access log data by means of a programmed set of instructions to determine the server response time, the processor comprising:
an analytical module configured to perform an analysis of the access log data and filter the access log data by removing unwanted access log data and adds a filtered access log data into input parameters;
a cluster generating module configured to generate clusters by applying a clustering technique over the input parameters, wherein each cluster represents a group of web components, the web components that are part of a single web-page request initiated by a web user, the cluster generating module is configured to assign a membership vector to each web-component, where the membership vector represents a probability of the web-component belonging to a cluster of a parent web page request appearing in the access log data;
a time calculation module configured to further use the membership vector so assigned to determine a cluster information, the cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.
2. The system of claim 1, wherein the access log data so collected is in a Common Log Format without any session information.
3. The system of claim 1, wherein the access log data further comprises numerical and non-numerical access log data.
4. The system of claim 1, the analytical module removes access log data with an erroneous status code as well as the requests having the categories other than GET, PUT and POST, the analytical module further unit-normalizes numerical variable and prepares a master list for all non-numerical variables after filtering, sorting and grouping is performed.
5. The system of claim 1, wherein the input parameters further comprises number of clusters, fuzziness factor, tolerance limit and maximum number of iterations.
6. The system of claim 1, wherein the cluster generating module applies a variant of fuzzy c-mean technique to generate the clusters.
7. The system of claim 1, wherein the time calculation module use the membership vector so assigned to determine a cluster information, the cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.
8. A method to determine server response time for a user initiated web page request, the method comprising:
storing access log data so collected with respect to the user initiated web page requests;
processing the access log data to sort and analyze the access log data to determine the server response time, the processing further comprising:
performing an analysis of the access log data and filter the access log data and filter the access log data by removing unwanted access log data and adds a filtered access log data into input parameters;
generating a cluster by applying a clustering technique over the input parameters, wherein each cluster represents a group of web components, the web components that are part of a single web-page request initiated by a web user, the generating a cluster further comprises to assign a membership vector to each web-component, where the membership vector represents a probability of the web-component belonging to a cluster of a parent web page request appearing in the access log data;
using the membership vector so assigned to determine a cluster information, the cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.
9. The method of claim 8, wherein the access log data further comprises numerical and non-numerical access log data.
10. The method of claim 8, wherein the access log data so collected is in a Common Log Format without any session information.
11. The method of claim 8, wherein the analysis is performed to further remove access log data with an erroneous status code as well as the requests having the categories other than GET, PUT and POST, the analysis further unit normalizes the numeric variable and prepares master lists for all non-numerical variables, after the filtering, sorting and grouping is performed.
12. The method of claim 8, wherein the input parameters further comprises number of clusters, fuzziness factor, tolerance limit and maximum number of iterations.
13. The method of claim 8, wherein the clusters are generated by applying a variant of fuzzy c-mean technique to generate the clusters.
14. The method of claim 8, wherein the membership vector so assigned is used to determine cluster information, the cluster information is further used to calculate the server response time in terms of timestamps difference within each cluster.