Claims:
1. A processor implemented method for determining service dependency based on network packets traces, comprising:
(a) obtaining, a plurality of network packets traces (202) corresponding to a first logical component and a second logical component;
(b) converting, by a instances conversion module (204), the plurality of network packets traces (202) into a plurality of service instance logs;
(c) preprocessing, by a class determination module (206), the plurality of service instance logs to obtain a plurality of service classes associated with timing information, wherein a parameter value in the first logical component and in the second logical component are masked;
(d) computing, by a reduction module (208A), an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component; and
(e) determining, by an analysis module (208B), a service dependency (210) between the second logical component and the first logical component based on the computed assignment matrix.

2. The processor implemented method of claim 1, wherein the plurality of service instance logs comprises at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component.

3. The processor implemented method of claim 1, wherein the first logical component corresponds to an HTTP request, and wherein the second logical component corresponds to an SQL Query.

4. The processor implemented method of claim 1, wherein the service instance logs are captured for a predefined time interval.

5. The processor implemented method of claim 4, wherein the predefined time interval is determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.

6. A system (100) to determine service dependency based on network packets traces, wherein the system comprising:
a memory (102) storing instructions;
one or more communication interfaces (106); and
one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to:
(a) obtain, a plurality of network packets traces (202) corresponding to a first logical component and a second logical component;
(b) convert, by a instances conversion module (204), the plurality of network packets traces (202) into a plurality of service instance logs;
(c) preprocess, by a class determination module (206), the plurality of service instance logs to obtain a plurality of service classes associated with timing information, wherein a parameter value in the first logical component and in the second logical component are masked;
(d) compute, by a reduction module (208A), an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component; and
(e) determine, by an analysis module (208B), a service dependency (210) between the second logical component and the first logical component based on the computed assignment matrix.

7. The system of claim 6, wherein the plurality of service instance logs comprises at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component.

8. The system of claim 6, wherein the first logical component corresponds to an HTTP request, and wherein the second logical component corresponds to an SQL query.

9. The system of claim 6, wherein the service instance logs are captured for a predefined time interval.

10. The system of claim 9, wherein the predefined time interval is determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.
, Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:

SYSTEM AND METHOD FOR DETERMINATION OF SERVICE DEPENDENCY BASED ON NETWORK PACKETS TRACES

Applicant

Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The disclosure herein generally relates to data processing, and, more particularly, to system and method for determination of service dependency based on network packets traces.

BACKGROUND
In an industrial scenarios, when performance issues arise in production it can be very difficult to drill down cause of response time delays because the application developers may not be around to assist the operations staff. Root-cause analysis of the slow responses for web based applications, requires determining service dependencies. In general, base service (for e.g. SQL service) is service on which dependent service (for e.g. HTTP requests) depends. For example, an HTTP application server makes SQL query calls to the database server in order to complete its response. In case of a micro service based architecture, there will be calls to many other downstream services. These service instances are recorded in the network packet logs (tcpdump) with the variables, and can be converted to the service class by replacing the variables with constants. There are no effective method/process which aims at passively determining the service class dependencies. There are work reported, dealing with the correlating the instance of HTTP and SQL and they are intrusive methods. This is needed in DevOps environment to know the dependency of services since issues have to be fixed quickly and code may not be available. Technological gaps are there which are as follows: a) significantly larger file/log size are required, b) complexity involved in determination of relation, and c) intrusive and costly solution.
Online applications are constructed tier-wise with a server in tier n taking services from a server in tier n+1. When it comes to debugging end user response time issues, it is essential to know which tier contributes to response time delays. It is required to understand the exact nature of dependencies amongst the tiers. For example in the case of web applications, it is essential to know which SQL queries are executed by a given http request in a web/application server. Often, this information is not available to operational staff maintaining production systems. Commercial tools with high licensing costs work by instrumenting the code, aid in analysis, but their impact on performance cannot be ignored.
There are some typical approach which use stepwise linear regression to determine service class dependencies. It uses resource utilization measures to construct a workload to utilization model of depended server and uses the results to derive a workload-to-workload model that finally learns service dependencies. These approaches require some information from the system servers and are intrusive. These algorithms however attempt to correlate service instances and not service classes. If the algorithms are tweaked to consider nodes as service classes it will help find service classes, but they will take lot more computations.

SUMMARY
Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one aspect, processor implemented method for determining service dependency based on network packets traces is provided. The method includes (a) obtaining, a plurality of network packet traces corresponding to a first logical component and a second logical component; (b) converting, by a instances conversion module, the plurality of network packets traces into a plurality of service instance logs; (c) preprocessing, by a class determination module, the plurality of service instance logs to obtain a plurality of service classes associated with timing information, wherein a parameter value in the first logical component and in the second logical component are masked; (d) computing, by a reduction module, an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component; and (e) determining, by an analysis module, a service dependency between the second logical component and the first logical component based on the computed assignment matrix.
In an embodiment, the plurality of service instance logs may include at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component. In an embodiment, the first logical component may corresponds to an HTTP request, and the second logical component may corresponds to an SQL Query. In an embodiment, the service instance logs may be captured for a predefined time interval. In an embodiment, the predefined time interval may be determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.
In another aspect, there is provided a processor implemented system to determine service dependency based on network packets traces. The system comprises: a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to: (a) obtain, a plurality of network packets traces corresponding to a first logical component and a second logical component; (b) convert, by a instances conversion module, the plurality of network packets traces into a plurality of service instance logs; (c) preprocess, by a class determination module, the plurality of service instance logs to obtain a plurality of service classes associated with timing information, wherein a parameter value in the first logical component and in the second logical component are masked; (d) compute, by a reduction module, an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component; and (e) determine, by an analysis module, a service dependency between the second logical component and the first logical component based on the computed assignment matrix.
In an embodiment, the plurality of service instance logs may include at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component. In an embodiment, the first logical component may corresponds to an HTTP request, and the second logical component may corresponds to an SQL Query. In an embodiment, the service instance logs may be captured for a predefined time interval. In an embodiment, the predefined time interval may be determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.
In yet another aspect, there are provided one or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors causes (a) obtaining, a plurality of network packets traces corresponding to a first logical component and a second logical component; (b) converting, by a instances conversion module, the plurality of network packets traces into a plurality of service instance logs; (c) preprocessing, by a class determination module, the plurality of service instance logs to obtain a plurality of service classes associated with timing information, wherein a parameter value in the first logical component and in the second logical component are masked; (d) computing, by a reduction module, an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component; and (e) determining, by an analysis module, a service dependency between the second logical component and the first logical component based on the computed assignment matrix.
In an embodiment, the plurality of service instance logs may include at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component. In an embodiment, the first logical component may corresponds to an HTTP request, and the second logical component may corresponds to an SQL Query. In an embodiment, the service instance logs may be captured for a predefined time interval. In an embodiment, the predefined time interval may be determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
FIG. 1 illustrates a block diagram of a system for determination of service dependency based on a plurality of network packets traces according to embodiments of the present disclosure.
FIG. 2 is an exemplary system for determination of service dependency based on a plurality of network packets traces according to embodiments of the present disclosure.
FIG. 3 is a flow diagram illustrating a method of determining service dependency based on a plurality of network packets traces according to embodiments of the present disclosure.
FIG. 4 is an exemplary graphical representation illustrating a deriving probability of an SQL to be present during HTTP request according to embodiments of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS
Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope and spirit being indicated by the following claims.
The embodiments of the present disclosure describes a method finding out web service dependency from limited TCP packets (e.g. in case of web applications, to know which SQL queries are executed by a given HTTP request in a web/application server). The method determines cause of service in a (n+1)th tier for given service in a nth tier. In an embodiment, the input may be a limited network packets traces which are captured using tcpdump from tracing two services. Further the tcpdump logs are converted into instances of class whose service dependency has to be determined. Thereafter, service instance logs are converted into service class along with timing information and other relevant information. An assignment matrix is created which states a number of web services active while the corresponding SQL query was active using the obtained information. The service dependency is determined by generating a relation between the classes based on the output of assignment matrix.
Referring now to the drawings, and more particularly to FIG. 1 through 4, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
FIG. 1 illustrates a block diagram of a system 100 for determination of service dependency based on a plurality of network packets traces according to embodiments of the present disclosure. In an embodiment, the system 100 includes one or more processors 104, communication interface device(s) or input/output (I/O) interface(s) 106, and one or more data storage devices or memory 102 operatively coupled to the one or more processors 104. The memory 102 comprises a database 108. The one or more processors 104 that are hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
The I/O interface device(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server.
The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes.
The database 108 may store information but are not limited to, a plurality of parameters, wherein the parameters are specific to an entity (e.g., user, device, machine, equipment, and the like). Further, the database 108 stores information pertaining to inputs fed to the system 100 and/or outputs generated by the system (e.g., at each stage), specific to the methodology described herein. More specifically, the database 108 stores information being processed at each step of the proposed methodology.
FIG. 2 is an exemplary system 100 for determination of web service dependency based on a plurality of network packets traces according to embodiments of the present disclosure. In an embodiment, the system 100 for determination of web service dependency may be based on limited network packets traces is provided. The system 100 includes a plurality of network packets 202, a processor 104, and a service dependency 210. The processor 104 further includes an instances conversion module 204, a class determination module 206, and a reduction and analysis module 208. The network packets traces 202 corresponding to a plurality of logical components are considered. In an embodiment, a first logical component 202A corresponds to n-tier (e.g. a HTTP server or a web server). In another embodiment, a second logical component 202B corresponds to (n+1) tier (e.g., a SQL server).
In an exemplary embodiment, the network packets traces 202 (e.g. captured using at least one of tcpdump) from tracing two services, includes three phases. The network packets traces i.e. the tcpdumps are converted into service or function instance logs by the instances conversion module 204. The output of the instances conversion module 204 is instances of a class whose service dependency is to be determined. For example, the data is available in files named HTTP Request Log and SQL Log as shown in below Table 1 and Table 2 respectively:
Table 1
Start time End
time HTTP Request Instance Name Class Name
ST1 ET1 /home/desk.jsp?id=75 IN1 CN1
ST2 ET2 /home/desk.jsp?id=154 IN2 CN1
ST3 ET3 /home/display.html?tx=pay IN3 CN2

Table 2
Start time End time SQL Query Instance Name Class Name
ST11 ET1 Select rno from table1 where uid=45 INQ1 CNQ1
ST12 ET2 Select rno from table1 where uid=78 INQ2 CNQ1
ST13 ET13 Select * from table2 INQ3 CNQ2

In an embodiment, the service instance logs are converted into service class (i.e. preprocessing) by the class determination module 206. In an embodiment, the parameter values in each SQL-query are masked which converts them from instance to class. Similarly, the process is applied for the HTTP service. In an embodiment, in SQL query, INSERT, SEARCH, DELETE and UPDATE calls are considered. The output instances are converted to the classes and with the corresponding timing information, along with other relevant information.
In third phase, the service dependency 210 is determined by the reduction and analysis module 208. In an embodiment, presence of the HTTP class during each of SQL-query. Based on this, decide if the HTTP class service is dependent on the SQL query class. Mathematically, problem can be defined as determining matrix NHQ[K, I] be defined such that shown in equation 1.
????QL [??]= ?_(i=1)^I¦?(NHQ[k,i]×Nhttp[i]) ? (Equation 1)
where ????????[??] is number of instances of SQL of class k, ??h??????[??] is the number of instance of http requests of class i. ??HQ[??,i] is the number of calls made by any instance of HTTP request class i to SQL class k. ??HQ[??,i] represents the service class dependencies i.e. the unknown which are to be determined.
The third phase includes two sub-phases, first is reduction phase and the analysis phase. For example, the subscript i and c indicates instances and classes, respectively.

Table 3
HTTP request class 1 HTTP request class 2 … HTTP request class I
SQL class 1 ??????[??,??] ??????[??,??] … ??????[??,??]
SQL class 2 ??????[??,??] ??????[??,??] … ??????[??,??]
… … … … …
SQL class K ??????[??,??] ??????[??,??] … ??????[??,??]

For example, table 5 illustrates an assignment matrix of classes of unique queries and a unique URL as shown below:
Table 5
HTTP request 1 HTTP request 2 HTTP request 3 HTTP request 4
???? ???? ? ????
???????????? c1 t11 t12 ?
? ? ? ? ?
???????????? ci ti1 ti2 ?
? ? ? ? ?
???????????? cm tm1 tm2 ?

In an embodiment, the reduction phase is to summarize number active HTTP request for all SQL queries. For each SQL query instance from the input packets traces (as in Table 2), all the present HTTP requests are identified. An assignment matrix for corresponding query and HTTP class is incremented by one. This is regardless of how many times the given HTTP-request is present during the SQL query in context. In an embodiment, observation is each SQL-query in response to one of HTTP-request of a class. If it is not incremented only once (i.e. more than 1), it cause an error. For example, table 6 depicts a sample output i.e. an assignment matrix obtained after execution of reduction phase. In addition, count of the instances each of the HTTP class and save the counts CNTH array (i.e. Count for HTTP request) for that HTTP request class (??1, ??2,... , ????). Similarly, count of the instances each of SQL class and save the CNTQ array (i.e. Count for SQL) for SQL class (c1,c2, …, cm).
Table 6
HTTP requests Instances of classes (DB) HTTP request 1 HTTP request 2 HTTP request3 HTTP request4
Query (http request) 250 495 253 245
Query1 500 500 15 5 9
Query2 495 25 495 9 12
Query3 503 250 12 253 25
Query4 740 7 84 244 490

For example, pseudo Code for the Reduction sub phase as shown below:
1 for each QueryEntry Qi in SQLlog
2 CNTQ[Qc]+=1
3 CHQ[] =[0,0,0,.. 0];// Initialize all http request to 0
4 for each HttpRequest Hi in the HTTPRequestLog
5 if (TQs > THs && TQe< THe && CHQ[Hc]==0)
6 CHQ[Hc]=1;
7 AM[Qc,Hc]+=1
8 End if
9 End for
10 End for
11 for each HttpRequest Hi in the HttpRequestLog
12 CNTH[Hc]+=1
13 End
In an embodiment, the analysis phase is to determine the dependency of particular query for the http request. The analysis phase generates NHQ matrix as defined in table 3. In an embodiment, ? is an input parameter and 0= ?<1. Tuning of the ? can be based on packet loss observed in recording logs. If there is no packet loss than ?=0, to accommodate packet loss ? can be increased. Initialize the NHQ matrix to all zero as illustrated in line 1 of pseudo code of analysis phase. Update the each cell of matrix such that is set to nearest integer ratio of assignment matrix AM[k,i] to CNTH[i] if they are within user defined limits ?.
For example, pseudo Code for the analysis sub phase is as shown below:
1 Initialize NHQ [K, I] = zeros[K][I]
2 for each row k in AM
3 for each column i in row k
4 // corresponding to each http request
5 Rfloat =AM [k, i]/CNTH(i)
6 R= Roundoff(Rfloat)
7 if (abs((R-Rfloat)/R)=?)
8 NHQ(k,i) = R
9 End for
10 End
In an embodiment, calculation of NHQ values is illustrated as follows. For e.g. 500 (=AM (1, 1)) is divided by 250 (= CNTH (1)), obtained 2, which put in NHQ(1,1). The analysis phase is executed on data thus obtained from reduction phase. An sample of NHQ matrix obtained is illustrated in Table 7. This means that per invocation, an HTTP request of class 1 makes 2 calls to SQL query of class 1 and 1 call to SQL query of class 3. Similarly an HTTP request of class 2 makes one request of SQL query of class 2 per invocation.
Table 7
HTTP requests HTTP request 1 HTTP request 2 HTTP request3 HTTP request 4
Query1 2 0 0 0
Query2 0 1 0 0
Query3 1 0 1 0
Query4 0 0 1 2

The property that ????Q[??,??] which is (NSQL[k] )/(Nhttp[i] ) must be close to an integer is being exploited (i.e. round-off threshold) to determine service class dependencies. In an embodiment, observation is that, each HTTP service class depends on integral number of request to SQL service class.
In an embodiment, for each occurrence of the query, an HTTP request are present. The HTTP-request is defined to be present during given SQL query, if the HTTP-request started before start of SQL query and if HTTP-request completed post completion of the SQL query. The present system utilizes the following scenarios:
a) if HTTP-request is responsible for the SQL-query, than that HTTP-request always be present during the SQL-query.
b) if HTTP-request is not responsible for the SQL-query than that HTTP-request may or may not be present during the execution of the SQL-query.
The system identifies which HTTP-request which are consistently present during given class of SQL-request and then quantify them. For example, depending on the fact that query duration is in between the corresponding HTTP start time and end time i.e. service are synchronous. In an embodiment, there can be multiple HTTP requests during a particular query. For example, considering the above mentioned scenarios, the system determine the service dependency based on below mentioned predefined rules:
a) Network traces from different servers, should be time synchronized. Otherwise, the time difference should be included in system.
b) Service request calls from (tier n) to (tier n+1) are synchronous RPC calls.
c) A service request class on tier n make the same set of calls on tier n+1 at all times (instances) i.e. the given HTTP-request can be same set of SQL-request.
The present disclosure is essentially comparing-counting statistics of HTTP requests and SQL queries. In an embodiment, the packets dumps used for input is captured for some minimum time duration. The minimum duration for capturing packets helps to minimize the possibility of misclassification of service dependencies is determined as below:
For example, considering a simple case where there are 2 concurrent HTTP requests (i.e. H1 and H2 respectively) that are independent. Only H1 is the cause of a SQL query Q1 and H2 does not cause any SQL queries. This implies that whenever Q1, is present there may be H1, but H2 can be absent or present. Therefore, though H2 does not call Q1, but H2 and Q1 can appear correlated in time in the packets dump. This is possible especially in cases where very few instances of H2 and Q1 are present in the capture. To minimize this possibility, it is desirable to extend the duration of the packets dump log, hence at least one occurrence of H2 and Q1 is captured where they do not overlap. In an embodiment, instances where Q1 and H2 should not overlap.
Let TH denote the average response per unit time for H2 and TQ that for Q1, X is the throughput. In an embodiment, dependency of these variables on the accuracy of the service dependency results. Let Q1 and H2 be considered as independently occurring pulses with a width equal to their average response times and frequency equal to their respective throughputs.
In an embodiment, Q1 is found to overlap completely within H2 for N instances. This means that the duration of packets capture should last for at least 1 more than N instances to avoid misclassifications. Considering pn be user specified probability of Q1 overlapping with H2 for N consecutive instances. The relationship between pn and N is captured in equation 2,
??= lnpn/ln((T_H-T_Q)/((1/X)-T_H )) … (Equation 2)
This means for a given probability pn of misclassification (due to overlap) specified by the user, the number of instances of H2 and Q1 should occur N times i.e., the duration of the packets capture should last for at least (N +1) instances for correct classification.
In an embodiment, following steps explains a method to determine minimum packets capture duration given user specified pn as input. Assuming, that TQ, TH and corresponding throughputs are known apriori for all HTTP and SQL classes (based on earlier monitoring logs, which could even include packets capture logs):
a) Set X to the minimum throughput observed across all the HTTP request classes and SQL query classes. Further, assuming that lesser the number of samples for a class, higher the chances of overlap for that class. Set TQ to be the average response time for that class.
b) Determine the HTTP class which has a throughput closest to X. Set TH to be the average response time for that class.
c) Calculate N as per equation 2.
d) Calculate the required duration Tcap of the packets capture as (N + 1)/X to allow for one more instance of combined occurrence (but not overlapped) of Q1 and H2 in the packets capture log.
e) Scale Tcap if network packets losses are expected. For example T*(1+?)/(1- ?).
If the input packets capture dump duration is less than Tcap then flag an alert to the user indicating that there could be misclassifications of service class dependencies.
For example, considering an HTTP class labeled H and a SQL class labeled Q with the nearly the same throughput. These classes are independent by design i.e. H does not depend on Q, but appears correlated in the packets capture logs. Let TH be the average response time for H and TQ be the average response time for Q. If the throughputs of H and Q are not close enough then the possibility of a misclassification is remote and considering throughput as X. For a misclassification to occur one instance of H and one instance of Q should be present in every time period with duration T = 1/X.
Within the time window of T, the occurrence of an instance of H can be represented as a pulse of width TH. Similarly, the occurrence of an instance of Q can be represented as a pulse of width TQ. Another condition for misclassification is that the pulse TQ should be contained completely within the pulse TH.
In order to derive the probability of misclassification it is essential to understand the overall sample space and the event space corresponding to misclassification. For this divide time into infinitesimally small parts ???. Based on this time period T, TH and TQ can be expressed as integral multiples of ???. According to the first condition for misclassification, exactly one HTTP request (represented by pulse TH) and exactly one SQL query (represented by pulse TQ) from time t=0 to t=T. Hence, the TH pulse can start anywhere in between t=0 to t=T-TH. This means that the number of ways in which the TH pulse can appear within period T is N?? = (T – ????)/ ???. This expression ensures that it excludes cases where the TH pulse overlaps outside of the period T.
Similarly, the TQ pulse can start anywhere in between t=0 to t=T-TQ. The number of ways in which the TH pulse can appear within period T is ???? = (T – ????)/ ???. The sample space is defined as the combination of ways in which the TH and TQ pulses can appear within the period T. This is essentially the product of NH and NQ
???????????? ??????????= (??-????) / ?t * (??-????) / ?t … (equation 4)
To derive the expression for the event space by considering the number of ways in which the pulse TQ can be completely contained within the TH pulse i.e. for a given instance of TH pulse within T. The TQ pulse can be present in number of ways defined by NHQ = (TH – TQ)/ ???. This excludes cases where the TQ pulse overlaps outside of the TH pulse. The TH pulse can of course start in NH ways as earlier. The event space is a combination of these events thus defined as a product of NHQ and NH. Thus
?????????? ??????????= (????-????) / ?t * (??-????) / ?t… (equation 4)
Deriving the probability of a single misclassification p as
??=?????????? ??????????/???????????? ?????????? … (equation 5)
Probability p is can now be obtained using equation 3, 4, and 5 as indicated by the equation 6.
??=????-????/ ??-???? … (equation 6)
Given that ??=1/?? implies that there will be one instance of H and Q every T units of time. Rewriting the equation 6 in term of X gives equation 7.
??=????-???? / 1/X -???? … (equation 7)
Misclassification during a particular period of time T is independent of misclassification in the next or following time periods and is equal to p for each time period. The probability pn that the misclassification occurs for N consecutive time periods is given by
???? = ???? … (equation 8)
The term pn could be the overall misclassification error specified by the user based on past experience. To estimate N and based on that determine the minimum packets capture duration.
FIG. 3 is a flow diagram illustrating a method of determining service dependency based on the plurality of network packets traces according to embodiments of the present disclosure. In an embodiment, the system 100 comprises one or more data storage devices or the memory 102 operatively coupled to the one or more hardware processors 104 and is configured to store instructions for execution of steps of the method by the one or more processors 104. The flow diagram depicted is better understood by way of following explanation/description.
The steps of the method of the present disclosure will now be explained with reference to the components of the system 200 as depicted in FIG. 2. In an embodiment of the present disclosure, at step 302, the one or more hardware processors 104, obtain, a plurality of network packets traces (202) corresponding to a first logical component and a second logical component. In an embodiment of the present disclosure, at step 304, the one or more hardware processors 104, convert, by an instances conversion module (204), the plurality of network packets traces (202) into a plurality of service instance logs. In an embodiment of the present disclosure, at step 306, the one or more hardware processors 104, preprocess, by a class determination module (206), the plurality of service instance logs to obtain a plurality of service classes associated with timing information. In an embodiment, a parameter value in the first logical component and in the second logical component are masked. In an embodiment of the present disclosure, at step 308, the one or more hardware processors 104, compute, by a reduction module (208A), an assignment matrix based on the plurality of service classes associated with first logical component and the second logical component. In an embodiment of the present disclosure, at step 310, the one or more hardware processors 104, determine, by an analysis module (208B), a service dependency (210) between the second logical component and the first logical component based on the computed assignment matrix.
In an embodiment, the plurality of service instance logs may include at least one of (i) a first start time and a first end time corresponding to a first logical component, and (ii) a second start time and a second end time corresponding to a second logical component. In an embodiment, the first logical component may corresponds to an HTTP request, and the second logical component may corresponds to an SQL Query. In an embodiment, the service instance logs may be captured for a predefined time interval. In an embodiment, the predefined time interval may be determined based on average response per unit time for a minimum occurrence of a service instance and average response per unit time for a minimum occurrence of a query instance, an associated throughput thereof.
FIG. 4 is an exemplary graphical representation illustrating a deriving probability of an SQL to be present during HTTP request according to embodiments of the present disclosure.
The embodiments of the present disclosure is a robust, simple, lightweight and non-intrusive method to learn service class dependencies for an n-tier application. The minimum packets capture duration for correct service dependency determination. The embodiments of the present disclosure can be used in following ways, i.e. (a) Understand service in tier n+1 responsible for service in tier n. Production staff can immediately inform corresponding service development team about it, and (b) Fine grained performance modeling of an n-tier application for more accurate performance predictions in the presence of performance or to develop models which can help in capacity planning. This, along with other information will help better resource planning for various workloads.
The embodiments of the present disclosure helps in learning service dependencies from the data obtained in the form of network packets dumps. These network packets are recorded (using tcpdump) and preprocessed to instances. Instances were converted to the class. The algorithm has displayed the 100 percent accuracy in classifying the relationship of class of request from tier n (HTTP request) to that tier n+1 (SQL server).
There are many web applications hosted on a powerful server with multiple functionality. Each functionality can be implemented in the multiple web pages, and each web page may (e.g. http) none, single or multiple dependence (e.g. queries). These service dependence determination is not trivial until having corresponding code for it. The embodiments of the present disclosure aims at determining the service dependence based on network packets.
The embodiments of the present disclosure helps to find all the SQL queries executed in the context of a processing a particular http request. While HTTP-SQL is one type of example addressed in the present disclosure, and can be extended in application i.e., to be as generic in determining service dependencies of a tier n on tier n+1. Hence, methodology presented in the present disclosure work for any other application apart from HTTP-SQL based web applications. This assumes importance with emergence of micro-service based applications.
The embodiments of the present disclosure have no intrusion when comes to the use of network packets traces. Rather than run these tools on the servers themselves, in order to be truly non-intrusive these tools could run on servers, which are receiving mirrored packets passing through the application servers. This is possible through simple procedure called switch port mirroring, which is used in present disclosure. Network packets traces do not contain this handle like thread-id. Hence, there is a need to have an alternate method to determine which SQL queries are called during the processing of a given http request i.e., to determine the service dependencies of a given class of http requests on the database request. The present disclosure presents a new approach to make this association, using network packets traces as an input. The embodiments of the present disclosure is least intrusive - no information needs to be collected from the servers. It only needs network packets traces, which can be non-intrusively collected by means such as switch port mirroring.
The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope and spirit of disclosed embodiments being indicated by the following claims.