FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)
Title of invention:
A METHOD AND SYSTEM FOR DYNAMIC MODELING OF DATA PROCESSING RULES IN DISTRIBUTING COMPUTING PLATFORMS
Applicant:
TATA Consultancy Services Limited
A company Incorporated in India under The Companies Act, 1956
Having address:
Nirmal Building, 9th Floor,
Nariman Point, Mumbai 400021,
Maharashtra, India
The following specification describes the invention and the manner in which it is to be performed.

FIELD OF THE INVENTION
The present invention relates generally to a field of large data processing in a distributed computing platform. More specifically, it relates to a method and system for dynamically modeling the rules required for large data processing in a distributed computing platform.
BACKGROUND OF THE INVENTION
With the advent of Information technology platforms, there has been significant increase in the amount of data that is being processed and/or analyzed in order to execute several internet based applications. The variety of data available across data warehouses, data marts, content management systems, file systems, streaming data etc need to be analyzed/processed to extract information. For example, the amount of data involved in several social networking applications may be in the range of several terabytes and petabytes. Such large amount of data processing is generally termed as big data processing. In order to execute the applications in a desired manner, efficient processing of such big data is essential. The data processing may involve the steps of extraction, normalization or transformation followed by loading on the target resource.
The data processing of such big data is managed and control by several binding rules that when applied, performs the task of data validation, data transformations and data loading etc. In general, since the data handled itself is huge, the amount of processing required should be vast and need to meet the desired objective of the data processing. Therefore, conventionally, the tasks required to be performed is distributed amongst one or more computing platforms in a distributed computing platform. Though, such arrangement of division of tasks have been proposed in the

existing network management techniques, there exists the problem of system overloading, degradation and load balancing when processing of big data is involved. The existing systems and methods are not capable of handling processing of large set of data typically of the range in several terabytes and petabytes. Further, data that is to be processed may be received from both structured and unstructured resources. The existing systems are adapted to process the data received from structured resources and fails to process the data from unstructured resources.
Additionally, the existing systems and methods of data processing lack efficient management and parallel execution of tasks by individual nodes in the distributed computing platform. For instance, in a specific scenario of data processing, there may be a set of nodes busy in performing the designated tasks bounded by the rules while another set of nodes being idle without performing any task or have completed the allocated tasks. In such scenario, it is apparent to distribute the task amongst the nodes that are idle so that efficiency of the entire processing is enhanced. However, the existing systems and methods lack in dynamic modification of rules and their deployment thereof in order to process the data processing jobs parallel. Therefore, an awkward situation arises, wherein few of the systems are overloaded while others enjoying the freedom of idleness resulting in the degradation of the distributing platform and ultimately affecting the data processing capabilities, typically data the processing of big data.
Thus, there is a need in the art for a system and method for dynamic modeling of data processing rules and deployment thereof on data processing nodes in the network facilitating parallel processing of jobs to avoid degradation, imbalance and overload of the network. Further, there is a need in the art for the system and method that enables processing of data received from structured and unstructured resources.

OBJECTS OF THE INVENTION
The primary object of a present invention is to provide a system and method enabling a web-based interface for dynamic modeling of rules distributed on one or more slave nodes of a distributed computing platform.
Another object of the invention is to provide a system and method that enables defining the rule-logic for each of the rules using a common rule language module.
Yet another object of the invention is to provide a system and method for real-time deployment of the defined rule-logic on one or more slave nodes using a common rule run-time module.
Yet another object of the invention is to enable a system and method for creating a an in-memory cache in the individual slave nodes that caches details of the rules to be implemented for data/job processing as per defined logic using said common rule run-time module.
Still another object of the invention is to enable a system and method for dynamic modeling of the rules and their implementation, instantiation and deployment thereof using a common modeling environment.
SUMMARY OF THE INVENTION:
Before the present systems and methods, enablement are described, it is to be understood that this application is not limited to the particular systems, and methodologies described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosures. It is also to be understood that the terminology used in the description is for the purpose of describing the particular

versions or embodiments only, and is not intended to limit the scope of the present application.
In one embodiment, the present invention discloses a system and method for dynamic modeling of data processing rules in distributed computing platforms. The system and method of the present invention proposes a Rule Injection Framework for real-time management and deployment of rules required for data processing/analysis in distributed computing platforms.
In one embodiment, according to the method of the present invention, rules are configured on big data retrieved from structured and unstructured resources. The configuration of rules involves rule sequencing and rule defining for each of the rules. Each of these configured rules is decoupled and externalized into a rule repository for distributing the installation of these rules on the slave nodes of the distributing platform. Such externalization is enabled by defining the externalized rule logic across the slave nodes. In this embodiment, the method further comprises step of creating a rule context to access the rules from the rule repository. As a result of such context creation, an in-memory cache storing the rule details for the created rule context is implemented in each of the slave nodes. The rule details are cached in the memory across all slave nodes for the faster access by map and reduce tasks during run-time.
In one embodiment, the system of the present invention comprises a platform specific decoupling configured to decouple the rules across one or nodes. An externalizing module is configured to externalize rule logic of the decoupled rules. The system comprises a common rule language module configured to define the externalized rule logic across one or more nodes. In accordance with the system of the invention, a common rule run time module is provided that is configured to deploy the defined externalized rule logic across the one or more nodes. Further, the

system comprises an extreme data controller for modeling the rules across one or more nodes in the distributed computing platforms handling big data.
BRIEF DESCRIPTION OF DRAWINGS
The foregoing summary, as well as the following detailed description of embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the present document example constructions of the invention; however, the invention is not limited to the specific methods, systems and frameworks disclosed in the document and the drawings:
Figure 1 is a system block diagram (100) illustrating a rule injection framework comprising plurality of system components collectively performing the dynamic rules modeling and implementation thereof in big data processing according to an exemplary embodiment of the invention.
Figure2 illustrates various technical components integrated in to the rule injection framework facilitating rule modeling, instantiation and deployment thereof on one or more nodes according to an exemplary embodiment of the invention.
Figure 3 illustrates a flow diagram depicting processor implemented steps according to an exemplary embodiment of the invention.
DETAILED DESCRIPTION:
The description has been presented with reference to an exemplary embodiment of the invention. Persons skilled in the art and technology to which this invention pertains will appreciate that alterations and changes in the described method and

system of operation can be practiced without meaningfully departing from the principle spirit and scope of this invention.
In one embodiment of the present invention, a system and method of dynamic rule modeling and real-time deployment of individual server nodes is disclosed. The invention proposes a Rule Injection Framework (R1F) that enables real-time management and instrumentation of data processing rules to perform data processing jobs by a cluster of nodes in Big Data environment. In one embodiment, the Rule Injection framework is adapted to decouple and externalize various types of rules involved in big data processing/analytics.
In one embodiment, the framework enables defining externalized rule logic across one or more cluster nodes in the big data platform. The externalized logic is pushed to cluster nodes in a specific format (XML or RON) before the data processing job is started. An in memory representation is created in each cluster node using the data received. The data processing/analytics job accesses the in-memory representation of the rule during execution.
In one embodiment, the rule injection framework comprises a common modeling environment that enables a web-based interface for modeling and implementing the data processing rules over a cluster of nodes in the distributed computing platform. The rule injection framework comprises a common rule language module capable of defining the rule decoupling and rule externalized logic for instrumentation of the rules across several slave nodes. A common run time module is configured to deploy the externalized rule logic across the nodes wherein an in-memory cache of the rules is created across the nodes that enables these nodes to access the cached in-memory representation of rules during execution of the data processing job. The rule modeling that includes rule sequence defining, workflow definition and rule type definition is dynamically modified in real-time using the common modeling

environment and accordingly deployed in real-time on the nodes during execution of the data processing jobs. Various embodiments of the invention will now be described in detail referring to appended figures.
Referring to figure 1 is a rule injection framework (100) comprising plurality of system components collectively performing dynamic rules modeling and implementation thereof in big data processing according to an exemplary embodiment of the invention. As illustrated the rule injection framework (100) comprises a common modeling environment (101), hereinafter referred to as CME, a workflow server electronically coupled with said CME and a cluster of nodes (105) communicating with the CME (105) through the workflow server (103). In an exemplary embodiment, the CME comprises of modules configured to dynamically model the data processing rules over the cluster of nodes (105) and implementation thereof.
As illustrated in figure 1, the CME (105) further comprises a common rule language module (107) hereinafter referred to as a CRL module, a workflow definition module (111), and a rule repository (115). In an exemplary embodiment, the CME (105) is adapted for configuring the rules on the data received from both structured and unstructured resources. In an exemplary embodiment, structured data sources include data received from the sources such as relational databases while unstructured resources includes data received from resources such as email, documents and social media data. In an exemplary embodiment, the CRL module (107) is configured to decouple and externalize the configured rules over the cluster of nodes (103).The CRL module is adapted to define rules in XML, define the sequence of rules for data processing jobs, and define the rule type. More particularly, the definition of rules indicates the tasks that need to be performed by one or more nodes in the cluster of nodes (103). In an exemplary embodiment, the externalized logic is implemented by

the CRL module (107) using two different XML languages including Data Transformation Language (DTL) and Text Query Language (TQL).
In an exemplary embodiment, the Data Transformation Language (DTL) is XML based language adapted to externalize the rule logic from map reduce programs for data received from structured sources such as relational databases. The Text Query Language (TQL) is configured for defining the rule logic for externalization of rules for data received from unstructured resources such as email contents, text document and social media like facebook, twitter and blog etc. For example, the rule defined to retrieve the email ids from the text file by the Text Query Language (TQL) is as follows:

Similarly, an example of rule definition used to externalize rules across cluster of nodes in distributed computing platform using the Data Transformation Language (DTL) is as follows:


In an exemplary embodiment, the rules defined and its relevant metadata is then stored in the rule repository (115). The CME (101) is adapted to dynamically model the rule definition using a web-based interface (107) as shown in figure 1. In an exemplary embodiment, the CME modifies the rule definition, rule type by using said web-based interface (107). In an exemplary embodiment, the workflow definition module (111) is configured for defining the XML-based workflow depicting the flow of data from one or modes in the distributed computing platform during processing of jobs. The CRL module therefore defines the logic of externalization of rules across one or more nodes in the distributed computing platform (cluster of nodes (105) shown in figure 1).
As illustrated in figure 1, the distributed computing platform (105) comprises a master node (121) and several slave nodes/ data nodes (123-1, 123-2... 123-N). The master node is configured to manage the data processing jobs and the file management. More particularly, the master node (121) is adapted to distribute jobs for processing among individual data nodes (123-1, 123-2... 123-N) and management of file systems while used and shared during processing of the jobs in the distributed platforms. Each of these data nodes (123-1, 123-2... 123-N) further comprises common rule run-time modules (113-1, 113-2...113-N), hereinafter referred to as CRR modules that enables deployment of rules on said data nodes.
In an exemplary embodiment, each data node in the distributed computing platform (105) is capable of executing a map-reduce program that facilitates data processing of jobs designated to the individual data nodes. In an exemplary embodiment, the map-reduce program is a software program written in object-oriented language, preferably using JAVA language enabled to execute jobs on the individual nodes. In an exemplary embodiment, the rule injection framework creates a real-time linking of the rules and the associated metadata thereof stored in the rule repository (115)

with the map-reduce programs when the job processing is initialized by one or more slave nodes in the distributed computing platform (103).
In an exemplary embodiment, the common rule run-time module (CRR) module (113) is configured to deploy the rules from the rule repository (115) across one or more nodes in the cluster (103).The externalized logic built by the CRL module (109) is pushed to the cluster nodes (103) in a specific format (XML or RON) before the data processing of the jobs is started. An in memory representation is created in each cluster node using the data received. The data processing/analytics job accesses the in-memory representation of the rule during execution. In an exemplary embodiment, the common rule run-time (CRR) modules (113-1,1-13-2...113-N) are configured to create said in-memory representation which further enables the access of the representation to the slave nodes during run-time. The rule parameters are injected into the map-reduce jobs executed on one or more data nodes and hence the rules are deployed on the nodes for implementation thereof enabling parallel data processing of jobs such as data extract, data validate, data transform and data load etc. This process is generally termed as rule instrumentation which can be implemented by following two different approaches according to an exemplary embodiment of the invention.
In an exemplary embodiment, in the first approach, the map-reduce program in each of the individual data nodes (123) triggers a REST call to a Rule Repository (115) to retrieve the rule details in an object format. A library part of rule injection framework interprets Rule Object and creates an in-memory representation of the rule in each data node. The framework facilitates accessing these in-memory rule representations during map reduce program execution. More particularly, the externalized logic is pushed to cluster nodes in a specific format (XML or RON) before the data processing job is started. An in memory representation is created in each cluster node using the data received. The data processing/analytics job accesses

the in-memory representation of the rule during execution. In an exemplary embodiment, the common rule run-time (CRR) modules (113-1, 113-2...113-N) are configured to create said in-memory representation which further enables the access of the representation to the slave nodes during run-time. The rule parameters are injected into the map-reduce jobs executed on the data nodes through REST service invocation that sends Rule Object in RON data format. In an exemplary embodiment, in the second approach, the rule parameters and the externalization logic defined by the CRL module (109) using DTL/TQL are modeled through the web interface (107) and injected into the workflow XML (111). The workflow XML (111) is pushed to the workflow server (103) for execution. The individual slave nodes then execute the map-reduce programs thereby enabling data processing under the control of workflow management (117) in the workflow server (103). The workflow monitoring module (119) is configured to monitor the deployed workflow in the cluster (105). More specifically, the monitoring module (119) is adapted to identify the failure nodes and replace them with other nodes, if any.
In an exemplary embodiment, the instrumentation of rules by means of the above disclosed approaches enables the deployment of rules in the map-reduce programs to initiate the data processing jobs in compliance with the prescribed rules injected by means of REST service or from the defined workflow XML through Workflow server. The creation of rule context and in-memory cache representation enables the slave nodes (123-1, 123-2...123-N) to retrieve information about the rules in advance before execution of jobs. Further, these nodes access the rule details from the cache during execution and hence enable the execution of map-reduce program to execute to facilitate data job processing. Alternatively, in another case, wherein rules logic is captured through workflow XML, the workflow management pushes the logic to the individual data node to execute the map-reduce program for facilitating data job processing.

Referring to figure 2 is a block diagram (200) illustrating multiple technical components of the rule injection framework (100) according to an exemplary embodiment of the invention. As illustrated in figure 2, the rule injection framework (100) comprises several technical components including a rule object (202), a rule executor (204), an expression evaluator (206), an interface object (208), a data parser (210), a rule extractor (212), an interface extractor (214), an expression constructor (216), a RON parser (218), a RON proxy (220), a rule context builder (222) and a rule context (224). These technical components collectively along with the set of modules illustrated in figure 1 implements the method of rule externalization and deployment thereof for big data analytics and/or processing in accordance with one embodiment of the invention. In an embodiment, these components may be hosted by one or more modules shown in figure 1. In another embodiment, each of these components may act as a standalone component electronically coupled with one or more modules as shown in figure 1 to perform the desired task.
In an exemplary embodiment, the Data Transformation Language (DTL) and Text Query Language (DQL) are the two XML based languages configured to externalize the rule logic from map reduce programs. The DTL and TQL are XML based languages. The XML schema definition based validation and JAXB objects based parsing of these languages is done by the language parser. The Expression Constructor (216) component is used to create one or more Java Unified Expression Evaluator (JUEL) compatible expressions. It is generated out of the language parser output. The expression generated is executed using the Dynamic Expression Evaluator (206). This component is built on top of JUEL library. The Interface Object (208) acts as an Input/output component enabling input/output data definition. The Interface Extractor (214) is a component that parses and extracts the interface definition from the data processing flow. The RON Proxy (220) is responsible for creating REST URLs to retrieve rule definition in RON format for a given map-reduce job. The Rule Object (204) is an object that represents the data processing

expression details and input/output data definition called interface. The Rule Executer (204) is a component that evaluates the rule based on the rule object, interface object and input data.
In an exemplary embodiment, the RON Parser (218) is utilized for parsing the Rule Objects to create in memory representation of the rule. The rule context builder (222) is configured to build the rule context based on the output of the RON Parser (218). The Rule Context module (224) is used to get access to rules from the map reduce program. The Data Parser (210) is used to parse the data stored in HDFS based on the interface specification given in the modeling environment. The Rule Extractor (212) is configured for extracting the rules from the flow definition. Finally, the map reduce program (226) is configured to execute the job processing task based on the retrieved rule details.
Figure 3 illustrates a flow diagram depicting processor implemented steps according to an exemplary embodiment of the invention.
At step 301, configuration of rules on structured and unstructured data sources takes place.
At step 303, defining of the rules configured based on XML languages such as data transformation language (DTL) and text query language (TQL) is done.
At step 305, the externalization and decoupling logic for the rules is defined using the XML languages (DTL/TQL) and stored in a rule repository.
At step 307, a rule-context for accessing the rules from the rule repository is created using REST style and a custom rule object notation called RON (Rule Object Notation) over HTTP or XML over HTTP.

At step 309, an in-memory cache representation about the rule details is created in the data node that is accessible to map-reduce program for job processing.
The framework enables dynamic modeling of the rules and hence real-time modification of rule configuration, rule definition, externalized rule logic definition and rule instrumentation to deploy rules thereof on several data nodes of the cluster takes place dynamically. Such rule modeling is implemented by means of a web-based interface coupled with the common modeling environment of the framework. Thus, the steps (301, 303, 305, 307 and 309) as discussed above are to be iteratively performed by the system under the directions of the processor when the rules are modeled by the user monitoring the real-time deployment of rules using the web-based interface.
The methodology and techniques described with respect to the exemplary embodiments can be performed using a machine or other computing device within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies discussed above. The machine may comprise a mobile phone, a tablet PC, a laptop computer, a smart-phone, a PDA or any mobile machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
The machine may include a processor (e.g., a central processing module), a memory which communicates with each other via a bus. The memory stores the instructions when executed, may cause the processor of the machine to instruct one or more modules illustrated in figure 1 to perform any one or more of the methodologies discussed above.

Claims:
1. A method for dynamic modeling of a rule and context sensitive interpretation thereof over multiple platforms in a distributed computing environment, characterized by a web-based interface configured for externalizing rules and allowing a user to access the rules distributedly, the method comprising steps of:
a. configuring the rules for XML definition, rule sequencing, rule type and
combinations thereof by a rule modeler and storing each configured rule
into a rule repository;
b. decoupling the rules stored into the repository for distributedly installing
rules across all slave nodes;
c. defining the externalized rule logic across one or more slave nodes using
a common rule language module;
d. executing parallely said decoupled rules in each slave node, wherein
before starting the data processing job (during runtime):
I. creating a rule context for accessing the rules from the rule
repository, II. creating an in-memory cache of the retrieved rules and commonly used data within and across the jobs across all processor nodes; and
e. modifying said rules dynamically and deploying to runtime cluster using
said web based interface.
2. A method as claimed in claim 1, wherein a stable performance for increasingly higher quantum of data processing load is obtained by an memory distributed cache implementation with in context of platform specific programming model like map reduce executing parallely across multiple nodes.

3. The method as claimed in claim 1, wherein the distributed platform can be selected from group that includes hadoop and NoSQL databases like mongoDB.
4. The method as claimed in claim 1, wherein the rules are capable of working for big data sets.
5. The method of claim 1, wherein said rule type is selected from a group consisting of a validation rule, a transformation rule, a computing rule or combinations thereof.
6. The method of claim 1, wherein said externalized logic is defined by said common rule language module either in data transformation language (DTL) or a text query language (TQL).
7. The method as claimed in claim 6, wherein the Data transformation language (DTL) or a Text query language (TQL) are XML/RON (Rule Object Notation) based languages.
8. The method of claim 1, wherein said rule context is created by invocation of REST style service using a custom rule object notation called RON (Rule Object Notation) over HTTP or XML over HTTP.
9. A rule modeling framework for dynamic modeling of a rule and context sensitive interpretation thereof over multiple database platforms in a distributed computing environment, characterized by a web-based interface configured for externalizing rules and allowing a user to access the rules that is getting executed distributedly, the framework comprises of:
a. a platform specific module written in Java or Map-reduce configured to
decouple the rules across one or nodes;
b. a common rule language module configured to define the externalized
rule logic across one or more nodes;
c. a common rule run time module configured to deploy the defined
externalized rule logic across the one or more nodes by:

i) creating an in-memory cache of the rules across the one or more
nodes upon defining and deploying the rule logic; ii) enabling said one or more nodes to access the cached in-memory representation of rules during execution of the data processing job. and d. a common modeling environment configured to dynamically model the rules and deployment thereof using said web-based interface.
10. The framework as claimed in claim 9, wherein said common language module further comprising XML/RON (Rule Object Notation) based language components including a Data transformation language (DTL), a Text query language (TQL) or combinations thereof.
11. The framework as claimed in claim 9, further comprising an Expression Constructor component to create one or more Java Unified Expression Evaluator (JUEL) compatible expressions.
12. The framework as claimed in claim 11, further comprising a Dynamic Expression Evaluator component configured to evaluate said created JUEL compatible expressions.
13. The framework as claimed in claim 9, further comprising an Interface Object
component configured to enable input/output data definition in said framework.
14. The framework as claimed in claim 9, further comprising an Interface Extractor component that parses and extracts the interface definition from the data processing flow.
15. The framework as claimed in claim 9, further comprising a RON Proxy (220) component adapted for creating REST URLs to retrieve rule definition in RON format for a given map-reduce job.
16. The framework as claimed in claim 9, further comprising a Rule Object component that represents the data processing expression details and input/output data definition.

17. The framework as claimed in claim 9, further comprising a Rule Executer
component that evaluates the rule based on the rule object, interface object and input data.
18. The framework as claimed in claim 9, further comprising a RON Parser component configured for parsing the Rule Objects to create in memory representation of the rule.
19. The framework as claimed in claim 18, further comprising a rule context builder configured to build the rule context based on the output of said RON Parser.
20. The framework as claimed in claim 9, further comprising a Rule Context module enabled to get access to rules from the map reduce program.
21. The framework as claimed in claim 9, further comprising a Data Parser to
parse the data stored in the cluster based on the interface specification given in the modeling environment.
22. The framework as claimed in claim 9, further comprising a Rule Extractor
configured for extracting the rules from the flow definition.