Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
SYSTEMS AND METHODS FOR IDENTIFYING SOLUTIONS FOR ERRORS IN LOG FILES OBTAINED FROM EXECUTION ENVIRONMENTS

Applicant:
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th Floor,
Nariman Point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The disclosure herein generally relates to analysis of log files from execution environments, and, more particularly, to systems and methods for identifying solutions for errors in log files obtained from execution environments.

BACKGROUND
In the DevOps process, testing teams use thousands of automated testcases to test a product in a regression testing running daily. The testing suite of such number of testcases generates thousands of log files per run, in various distributed environment with different formats (log4j, Server logs, Process logs, etc.). Every day, several testers consolidate the errors that occurred in that run by manually scavenging the log files. The respective developers/subject matter experts (SMEs) also had to manually scan the log files apart from the testing report to provide solutions to the errors/defects. The solutions depend on SMEs and the impact of solution in many cases lead to rework of the defect/errors and high degree of heterogeneity. This leads to huge bottleneck in automation of DevOps process leading to loss of productivity and agility.

SUMMARY
Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
For example, in one aspect, there is provided a processor implemented method for identifying solutions for errors in log files obtained from execution environments. The method comprises obtaining, via one or more hardware processors, a plurality of log files from at least one of a first execution environment, and a second execution environment, wherein the plurality of log files correspond to one or more file formats; pre-processing, via the one or more hardware processors, the plurality of log files to obtain a plurality of pre-processed log files comprising one or more unique error sentences; tokenizing, via the one or more hardware processors, the one or more unique error sentences by using a Natural Language Processing (NLP) library; removing, via the one or more hardware processors, stop words comprised in the one or more unique error sentences; calculating, via the one or more hardware processors, a frequency of each word comprised in the one or more unique error sentences; calculating, by using a word vector model via the one or more hardware processors, a word weight for each word comprised in the one or more unique error sentences based on the calculated frequency; generating, via the one or more hardware processors, one or more error sentence vectors for the one or more unique error sentences using the calculated word weight for each word; computing, via the one or more hardware processors, a cosine similarity between the one or more unique error sentence vectors of the plurality of log files and a plurality of error sentence vectors comprised in a pre-defined database, by using a modified Smooth Inverse Frequency (mSIF) model to obtain a set of matched cosine similarities; performing, via the one or more hardware processors, a comparison of each cosine similarity comprised in the set of matched cosine similarities with a pre-defined threshold value; and identifying, via the one or more hardware processors, one or more solutions to the one or more unique error sentences based on the comparison.
In an embodiment, the first execution environment and the second execution environment are different from each other.
In an embodiment, the one or more of solutions are identified for the one or more unique error sentences when the cosine similarity is greater than or equal to the pre-defined threshold value.
In an embodiment, the one or more unique error sentence vectors are generated based on an equation: v(s)=a/((a+p(w)))*v(w), and wherein v is average number of words present in each sentence, v(w) is a vector representation of a word, p(w) is number of times a word has occurred by the total number of words, a is a smoothing hyper-parameter calculated based on a first pre-defined variable a, and a second pre-defined variable z, a is (a) computed based on a sum of (i) p(w) and (ii) v and total number of words in the pre-defined database or (b) empirically determined.
In an embodiment, the cosine similarity is computed as a simple mean of projection weight vectors of the one or more unique error sentences and the one or more error sentence vectors for each sentence vector comprised in the pre-defined database.
In an embodiment, the modified Smooth Inverse Frequency (mSIF) model is obtained by: removing stops words from the plurality of error sentences by removing one or more influential non-relevant words comprised in the plurality of error sentences; calculating a frequency of each word comprised in the plurality of error sentences; calculating, by using a word vector model, a word weight for each word comprised in the plurality of error sentences based on the calculated frequency; and generating one or more error sentence vectors for the plurality of error sentences using the calculated word weight for each word to obtain the modified Smooth Inverse Frequency (mSIF) model.
In another aspect, there is provided a processor implemented system for identifying solutions for errors in log files obtained from execution environments. The system comprises: a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to: obtain a plurality of log files from at least one of a first execution environment, and a second execution environment, wherein the plurality of log files correspond to one or more file formats; pre-process the plurality of log files to obtain a plurality of pre-processed log files comprising one or more unique error sentences; tokenize the one or more unique error sentences by using a Natural Language Processing (NLP) library; remove stop words comprised in the one or more unique error sentences; calculate a frequency of each word comprised in the one or more unique error sentences; calculating, by using a word vector model via the one or more hardware processors, a word weight for each word comprised in the one or more unique error sentences based on the calculated frequency; generate one or more error sentence vectors for the one or more unique error sentences using the calculated word weight for each word; compute a cosine similarity between the one or more unique error sentence vectors of the plurality of log files and a plurality of error sentence vectors comprised in a pre-defined database, by using a modified Smooth Inverse Frequency (mSIF) model to obtain a set of matched cosine similarities; perform a comparison of each cosine similarity comprised in the set of matched cosine similarities with a pre-defined threshold value; and identify one or more solutions to the one or more unique error sentences based on the comparison.
In an embodiment, the first execution environment and the second execution environment are different from each other.
In an embodiment, the one or more of solutions are identified for the one or more unique error sentences when the cosine similarity is greater than or equal to the pre-defined threshold value.
In an embodiment, the one or more unique error sentence vectors are generated based on an equation: v(s)=a/((a+p(w)))*v(w), and wherein v is average number of words present in each sentence, v(w) is a vector representation of a word, p(w) is number of times a word has occurred by the total number of words, a is a smoothing hyper-parameter calculated based on a first pre-defined variable a, and a second pre-defined variable z, a is (a) computed based on a sum of (i) p(w) and (ii) v and total number of words in the pre-defined database or (b) empirically determined.
In an embodiment, the cosine similarity is computed as a simple mean of projection weight vectors of the one or more unique error sentences and the one or more error sentence vectors for each sentence vector comprised in the pre-defined database.
In an embodiment, the modified Smooth Inverse Frequency (mSIF) model is obtained by: removing stops words from the plurality of error sentences by removing one or more influential non-relevant words comprised in the plurality of error sentences; calculating a frequency of each word comprised in the plurality of error sentences; calculating, by using a word vector model, a word weight for each word comprised in the plurality of error sentences based on the calculated frequency; and generating one or more error sentence vectors for the plurality of error sentences using the calculated word weight for each word to obtain the modified Smooth Inverse Frequency (mSIF) model.
In yet another aspect, there are provided one or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause identifying solutions for errors in log files obtained from execution environments by obtaining a plurality of log files from at least one of a first execution environment, and a second execution environment, wherein the plurality of log files correspond to one or more file formats; pre-processing the plurality of log files to obtain a plurality of pre-processed log files comprising one or more unique error sentences; tokenizing the one or more unique error sentences by using a Natural Language Processing (NLP) library; removing stop words comprised in the one or more unique error sentences; calculating a frequency of each word comprised in the one or more unique error sentences; calculating, by using a word vector model, a word weight for each word comprised in the one or more unique error sentences based on the calculated frequency; generating one or more error sentence vectors for the one or more unique error sentences using the calculated word weight for each word; computing a cosine similarity between the one or more unique error sentence vectors of the plurality of log files and a plurality of error sentence vectors comprised in a pre-defined database, by using a modified Smooth Inverse Frequency (mSIF) model to obtain a set of matched cosine similarities; performing a comparison of each cosine similarity comprised in the set of matched cosine similarities with a pre-defined threshold value; and identifying one or more solutions to the one or more unique error sentences based on the comparison.
In an embodiment, the first execution environment and the second execution environment are different from each other.
In an embodiment, the one or more of solutions are identified for the one or more unique error sentences when the cosine similarity is greater than or equal to the pre-defined threshold value.
In an embodiment, the one or more unique error sentence vectors are generated based on an equation: v(s)=a/((a+p(w)))*v(w), and wherein v is average number of words present in each sentence, v(w) is a vector representation of a word, p(w) is number of times a word has occurred by the total number of words, a is a smoothing hyper-parameter calculated based on a first pre-defined variable a, and a second pre-defined variable z, a is (a) computed based on a sum of (i) p(w) and (ii) v and total number of words in the pre-defined database or (b) empirically determined.
In an embodiment, the cosine similarity is computed as a simple mean of projection weight vectors of the one or more unique error sentences and the one or more error sentence vectors for each sentence vector comprised in the pre-defined database.
In an embodiment, the modified Smooth Inverse Frequency (mSIF) model is obtained by: removing stops words from the plurality of error sentences by removing one or more influential non-relevant words comprised in the plurality of error sentences; calculating a frequency of each word comprised in the plurality of error sentences; calculating, by using a word vector model, a word weight for each word comprised in the plurality of error sentences based on the calculated frequency; and generating one or more error sentence vectors for the plurality of error sentences using the calculated word weight for each word to obtain the modified Smooth Inverse Frequency (mSIF) model.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
FIG. 1 depicts an exemplary system for identifying solutions for errors in log files obtained from execution environments, in accordance with an embodiment of the present disclosure.
FIG. 2 depicts an exemplary flow chart illustrating a method for identifying solutions for errors in log files obtained from execution environments, using the system of FIG. 1, in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION OF EMBODIMENTS
Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.
In the DevOps process, testing teams use thousands of automated testcases to test a product in a regression testing running daily, which generate thousands of log files per run, in various distributed environment with different formats. Errors are consolidated and subject matter experts (SMEs) have to manually scan the log files apart from the testing report to provide solutions to the errors/defects. The solutions depend on SMEs and the impact of solution in many cases lead to rework of the defect/errors and high degree of heterogeneity. This leads to huge bottleneck in automation of DevOps process leading to loss of productivity and agility.
Embodiments of the present disclosure address the challenges by providing systems and methods that auto capture the log files of different formats (log4j, Jenkins, Maven, HTML, etc.) efficiently in a scalable, extendible, and plug-able way. The system then mines and parses the log files based on given identifiers to standardize and de-duplicate to create unique error records with detail description including cause, position, module, timestamp, etc. The system predicts the solutions leveraging the knowledge base by using natural language processing (NLP) techniques (MSiF or mSIF) which is faster and accurate from the known algorithms. The system stores the processed log files in a structured log files platform so that any customizable report can be generated. The system and method as implemented by the present disclosure bring automated, intelligent, homogeneous integrated log analysis solution in every stage of distributed or non-distributed DevOps environment.
Referring now to the drawings, and more particularly to FIG. 1 through 2, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
FIG. 1 depicts an exemplary system for identifying solutions for errors in log files obtained from execution environments, in accordance with an embodiment of the present disclosure. In an embodiment, the system 100 includes one or more hardware processors 104, communication interface device(s) or input/output (I/O) interface(s) 106 (also referred as interface(s)), and one or more data storage devices or memory 102 operatively coupled to the one or more hardware processors 104. The one or more processors 104 may be one or more software processing components and/or hardware processors. In an embodiment, the hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) is/are configured to fetch and execute computer-readable instructions stored in the memory. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices (e.g., smartphones, tablet phones, mobile communication devices, and the like), workstations, mainframe computers, servers, a network cloud, and the like.
The I/O interface device(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface device(s) can include one or more ports for connecting a number of devices to one another or to another server.
The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic-random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, a database 108 is comprised in the memory 102, wherein the database 108 comprises information such as a plurality of log files, execution environments, a plurality of pre-processed log files, unique error records (also referred as unique error sentences and interchangeably used herein), corresponding unique error sentence vectors (also referred as vectors and interchangeably used herein), cosine similarities computed for vectors, pre-defined threshold, one or more solutions for the unique errors, and the like. The database 108 further comprises a modified Smooth Inverse Frequency (mSIF/MSiF) model, other pre-processing techniques, and the like. The memory 102 further comprises (or may further comprise) information pertaining to input(s)/output(s) of each step performed by the systems and methods of the present disclosure. In other words, input(s) fed at each step and output(s) generated at each step are comprised in the memory 102 and can be utilized in further processing and analysis.
FIG. 2 depicts an exemplary flow chart illustrating a method for identifying solutions for errors in log files obtained from execution environments, using the system 100 of FIG. 1, in accordance with an embodiment of the present disclosure. In an embodiment, the system(s) 100 comprises one or more data storage devices or the memory 102 operatively coupled to the one or more hardware processors 104 and is configured to store instructions for execution of steps of the method by the one or more processors 104. The steps of the method of the present disclosure will now be explained with reference to components of the system 100 of FIG. 1, and the flow diagram as depicted in FIG. 2.
At step 202 of the method of the present disclosure, the one or more hardware processors 104 obtain a plurality of log files from at least one of a first execution environment and a second execution environment. In an embodiment, the plurality of log files corresponds to one or more file formats. The first execution environment and the second execution environment are different from each other, in one example embodiment. Executable files can be generated in two types of environments:
Distributed execution environment: Application is deployed on multiple machines, but the testing is done for all the instances simultaneously. The logs (or log files) are generated for different cases on different machine but belong to single testing execution cycle. Multiple processes of one instance of testing cycle can run in parallel on multiple machines simultaneously. For example, One Machine can have windows® operating system installation and some specific features of product, whereas other machine is Linux® operating system and has different product features.
Non-distributed execution environment: The application is deployed on single machine and is tested also on the same machine. The logs generated in such cases are present in that single execution environment.
Below are few examples of log files obtained from various execution environments. Such exemplary log files shall not be construed as limiting the scope of the present disclosure.
Type of Logfile – Log4j, Format type - .log file
Information contained in the log files:
15-Oct-2020 18:29:11 398809 ERROR - Mask Mplus:110 - Process failed for execution id:22902. Schema name and catalog name is null for mplussource
15-Oct-2020 18:29:13 401490 DEBUG -
com.abc.mplus.beans.exception.DSException: Schema name and catalog name is null for mplussource
at com.abc.mplus.services.common.maskerutils.MplusMaskerUtility.populateModel(MplusMaskerUtility.java:927)
at com.abc.mplus.services.common.maskerutils.MplusMaskerUtility.addMplusbase(MplusMaskerUtility.java:826)
at com.abc.mplus.services.common.maskerutils.MplusMaskerUtility.addMplusBaseToProject(MplusMaskerUtility.java:532)
at com.abc.mplus.services.mask.MaskMplusServiceProvider.setupServiceSetting(MaskMplusServiceProvider.java:652)
at com.abc.mplus.services.common.maskerutils.MplusMaskerUtility.init(MplusMaskerUtility.java:327)
at com.abc.mplus.services.mask.MaskMplusServiceProvider.processInputs(MaskMplusServiceProvider.java:431)
at com.abc.mplus.services.common.framework.ServiceProvider$ServiceRunner.run(ServiceProvider.java:189).
…
Caused by: java.io.FileNotFoundException: D:\Privacy\PrivacyNew\plugins\techniques\Randomization\mplusset\LDs1_752693576_STRING.ds (The system cannot find the file specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at java.io.FileReader.(FileReader.java:58)
at com.abc.masketeer.server.utils.ExcelMplusMaskingReportGenerator.getTechParamsProperty(ExcelMplusMaskingReportGenerator.java:1518)
... 56 more
15-Oct-2020 18:29:13 401723 ERROR - Mask Mplus:110 - Process failed for execution id:7112638.
Type of Logfile – Jenkins, Format type - .log file
Information contained in the log files:
Started by upstream project "[8mha:////4Fk2OkR/87VpLVsfCp82JJzY5FaT8lQzhwdkNkU9+Wy7AAAAqB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyxEgYZ/az8JH2XxJLEgJzS4viAzIJUn8y81PiwzNRyfQDT/GYC0QAAAA==[0mMplusPlus_PipeLine_View" build number [8mha:////4MMJ2vahRk8ESXF0wJqWqDkyLSRwrKsfGfyFKnBeFsZkAAAArB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyWEgYF/az8JH2XxJLEgJzS4viAzIJUn8y81PiwzNRyfUMDcyMAh2gsVtUAAAA=[0m1072
originally caused by:
Started by timer
Building in workspace C:\Users\172049\.jenkins\workspace\MplusPlus_Main
Checking out a fresh workspace because C:\Users\172049\.jenkins\workspace\MplusPlus_Main\Maven\mplusplus_internal_jars doesn't exist
Cleaning local Directory Maven\mplusplus_internal_jars
Checking out svn://10.23.156.60/MplusPlus/Source/1_0/1_0/Branch/Branch_MplusPlus_2_5/mplusplus_internal_jars at revision '2020-07-18T16:33:36.648 +0530' --quiet
ERROR: Failed to check out svn://10.23.156.60/MplusPlus/Source/1_0/1_0/Branch/Branch_MplusPlus_2_5/mplusplus_internal_jars
org.tmatesoft.svn.core.SVNException: svn: E210003: No route to host: connect
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:91)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:75)
at org.tmatesoft.svn.core.internal.io.svn.SVNPlainConnector.open(SVNPlainConnector.java:65)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:79)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1282)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.getLatestRevision(SVNRepositoryImpl.java:173)
at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getRevisionNumber(SVNBasicDelegate.java:480)
at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getLocations(SVNBasicDelegate.java:833)
at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:527)
at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doCheckout(SVNUpdateClient16.java:875)
at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:66)
at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:18)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at hudson.scm.subversion.CheckoutUpdater$SubversionUpdateTask.perform(CheckoutUpdater.java:130)
…
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
Type of Logfile –Testing logs, Format type - .html file
Information contained in the log files:
!DOCTYPE html>