Description:FIELD OF INVENTION

The present invention relates to the workflow orchestrator during model inference. More particularly, the present invention relates to the system and method of workflow orchestrator as a feature for decomposing and optimizing query workflows during model inference by introducing a highly innovative intermediate layer.

BACKGROUND

Large language models (LLMs) and similar inferences of systems have rapidly gained traction across diverse applications, includes customer support, scientific research, and decision-making processes. Their ability to generate coherent and contextually relevant outputs from input queries has made them indispensable tools in various domains. However, these systems face critical limitations that hinder their efficiency and effectiveness in handling complex queries. While these systems are adept at generates outputs based on input queries, challenges remain in decomposes complex queries into manageable tasks, ensures workflows utilize both deterministic systems (e.g., APIs, databases) and generative content effectively optimizing the workflow to achieve accuracy and efficiency.

PRIOR ART
Prior attempts to address these issues are noteworthy but incomplete. For instance, EP2893446A1 describes a system and method for workflow orchestration for use with a cloud computing environment. Cloud environments, such as Oracle Public Cloud (OPC), provide a suite of applications, middleware, and database offerings that can be delivered to tenants in a self-service, elastically scalable, and secure manner. In accordance with an embodiment, the cloud environment can include a Platform as a Service (PaaS) environment, which provides a variety of services such as virtual assembly creation. A workflow orchestrator can be used to orchestrate operations between the cloud environment and the PaaS environment, e.g., by receiving a request from a tenant automation system, and coordinating the provisioning and deployment of virtual assemblies or applications. A customer can interact with the PaaS environment, e.g., to request a service, deploy to the service, or monitor the service.

Similarly, CN110321413B introduces a computer-implemented session system framework for performing tasks associated with client requests. A session application running on the hardware processor provides an application workflow orchestration, receives the client requests and sends one or more application requests based on the application workflow orchestration. A session system running on the hardware processor provides session workflow orchestration, which receives one or more application requests. The session application and the session system develop a dialog context and store the dialog context in the memory device. Session applications and session systems develop a dialog context by invoking at least one microservice to perform tasks associated with one or more application requests. The session application generates a response to the client request based on the developed dialog context.

OBJECTS OF THE INVENTION

The primary object of the present invention is to provide a system and method for efficiently compressing context data into predefined memory blocks while preserving the semantic meaning and structural relationships inherent within the data.

Another object of the present invention is to provide a system and method for compressing context data that ensures that data remains contextually relevant and accessible, even within constrained memory environments.

Another object of the invention is to provide a system and method for compressing context data that uses a dynamic data compression approach that adapts to varying memory capacities, allowing for optimal storage, retrieval, and querying of large datasets without compromising data integrity.

A further object of the invention is to provide a system and method for compressing context data that facilitates the efficient categorization, encoding, and storage of data entities and their interrelationships, ensuring that the data retains its contextual fidelity and can be effectively analysed and processed within limited memory constraints.

Yet another object of the invention is to provide a system and method for compressing context data that overcomes the limitations of traditional data compression techniques, thereby providing a scalable, adaptable solution for modern data processing systems, particularly in environments where memory resources are limited.

SUMMARY
Before the present invention is described, it is to be understood that the present invention is not limited to specific methodologies and materials described, as these may vary as per the person skilled in the art. It is also to be understood that the terminology used in the description is for the purpose of describing the particular embodiments only and is not intended to limit the scope of the present invention.
The disclosed invention presents a system and method for orchestrating workflows during model inference, particularly for large language models (LLMs) or similar models. The innovation introduces an intermediate orchestration layer positioned before the inferencing layer. This layer is responsible for dynamically decomposing complex input queries into manageable blocks, retrieving relevant attributes and intents, and refining the scope through iterative questioning. It constructs an optimal workflow by blending both deterministic systems (such as APIs and databases) and generative approaches. The system utilizes algorithm to ensure efficient pathfinding, thus optimizing the inference process for more accurate and resource-efficient outcomes. These challenges include the need for effective decomposition of queries, integration of both deterministic and generative elements, and optimizes workflows for both accuracy and efficiency. The proposed workflow orchestrator addresses these issues by enables adaptive workflows capable of handles diverse query complexities and constraints. This description serves as a general overview and is not intended to restrict the scope of the invention, which is adaptable to various methodologies and materials as recognized by those skilled in the art.

BRIEF DESCRIPTION OF DRAWINGS

The present invention, together with further objects and advantages thereof, is more particularly described in conjunction with the accompanying drawings in which:

Fig. 1. illustrates an overview of system and method of workflows orchestrator

DETAILED DESCRIPTION OF THE INVENTION:

Before the present invention is described, it is to be understood that this invention is not limited to methodologies described, as these may vary as per the person skilled in the art. It is also to be understood that the terminology used in the description is for the purpose of describing the particular embodiments only and is not intended to limit the scope of the present invention. Throughout this specification, the word “comprise”, or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps. The use of the expression “at least” or “at least one” suggests the use of one or more elements or ingredients or quantities, as the use may be in the embodiment of the invention to achieve one or more of the desired objects or results. Various embodiments of the present invention are described below. It is, however, noted that the present invention is not limited to these embodiments, but rather the intention is that modifications that are apparent are also included.
The present invention provides a system and method for system and method of workflow orchestrator as a feature for decomposing and optimizing query workflows during model inference by introducing a highly innovative intermediate layer. The present invention is directed to achieve the objectives of workflows orchestrator to decompose complex user queries due to ambiguity, incomplete information, or multi-step task requirements into the blocks.
The invention provides a system comprising an orchestrator layer designed to interface between the user query input and the underlying inferencing model layer. This orchestrator layer is responsible for executing a series of systematic and optimized functions to process complex user queries into actionable outputs. The orchestrator layer acts as a mediator between user input and the underlying LLMs, enabling a structured approach to query processing and ensuring that the query is divided into logically consistent blocks for further processing. The layer also optimizes the workflow by ensuring that tasks are executed in the most efficient order, based on predefined attributes and user-specific criteria. This layer is responsible for decomposing complex user queries into smaller, manageable components, ensuring that the query's intent is preserved while optimizing the execution path.
The present invention discloses a system and method for orchestrating workflows, providing a structured orchestrator layer to handle complex user queries before they are processed by inferencing models like large language models (LLMs). This orchestrator layer, positioned between the user input and the inferencing model, enhances query processing through the following modules-
• Attribute Extraction module: Extracts predefined attributes such as intent, actors, actions, and context from the query by leveraging underlying models.
• Task Decomposition module: Splits attributes and intents into smaller blocks, each representing a unit of work.
• Probe System Interaction module: Routes these blocks through a probe system that combines LLM inferencing and deterministic layers (APIs, databases, etc.) to propose paths for the intent.
• Iterative Questioning module: Refines the scope of the query through iterative questioning to resolve ambiguities.
• Workflow Optimization module: Constructs an optimized workflow using algorithm to determine the most efficient path for task completion.
The detailed process of the workflow orchestrator is as follows:
Step 1: Attribute Extraction module
The first step focuses on understanding the user’s query by extracting structured attributes. This process ensures that the system has a comprehensive understanding of the task at hand before proceeding.
Intent: The system identifies the user's primary goal. This could be anything from solving a problem, gathering information, or executing a specific task.
Actors: It identifies all the entities involved in the task. These could include the user, external tools, or APIs.
Actions: The specific steps or operations implied by the query are extracted. These could range from querying a database to generating content.
Context: Any additional details, such as constraints, preferences, or situational factors, are captured to ensure the task aligns with the user's needs.
Output: A structured representation of the query’s attributes, providing a foundation for subsequent steps.
Step 2: Task Decomposition module
Once the attributes are extracted, the system decomposes the query into smaller, manageable blocks, each representing a self-contained unit of work. This decomposition ensures that the query is broken down into logically structured tasks that can be handled independently, yet remain connected to the original intent. Dependencies between tasks are identified to maintain proper sequencing and logical flow.
Segmentation: The system divides the query into manageable units, ensuring each block represents a single task or logical step.
Dependency Analysis: It identifies dependencies between tasks to maintain proper sequencing.
Validation: The system ensures that the decomposition preserves the original intent and that no critical elements are omitted.
Output: A set of smaller, logically ordered tasks that form the basis for further exploration.
Step 3: Probe System Interaction
This step involves exploring potential solutions for each decomposed task. The system employs a hybrid approach to combine creativity with factual accuracy.
LLM Inferencing: Leveraging large language models to brainstorm and generate potential solutions or ideas for open-ended tasks.
Deterministic Layers: APIs, databases, and predefined algorithms are used to validate the solutions or retrieve factual data.
Iterative Probing: For ambiguous blocks, the system probes further, asking clarifying questions or conducting additional data retrieval to ensure accuracy.
Output: A list of potential solutions, combining generative insights and grounded factual data for each task.
Step 4: Iterative Questioning module
This step refines the task scope by resolving ambiguities and narrowing objectives. The system iterates through a cycle of targeted questioning and refinement until all tasks are clear and actionable.
Targeted Questions: The system identifies ambiguities or gaps in information and generates precise questions to resolve them.
Feedback Loop: It integrates user feedback or retrieved information to refine the task definition.
Repetition: This process continues until each block is well-defined and free of ambiguity.
Output: Fully clarified tasks with a clear scope, ready for workflow construction.
Step 5: Workflow Construction module
The orchestrator now integrates the refined tasks into a cohesive workflow, ensuring logical sequencing and optimization.
Incorporation of Attributes: Context, actors, actions, and additional details are integrated into the workflow to provide a comprehensive roadmap.
Algorithmic Optimization: Using algorithm, the system determines the most efficient path to complete the tasks while minimizing resource use and time.
Error Handling: The workflow includes fallback mechanisms to handle potential errors or exceptions during execution.
Output: An optimized, detailed workflow designed to achieve the user’s goal effectively and efficiently.
Step 6: Dynamic Execution module
In the final step, the orchestrator executes the workflow dynamically, allowing for adjustments based on real-time results and feedback.
Adaptive Execution: The system monitors progress and adapts the workflow based on intermediate outputs or changing conditions.
Integration of Layers: A balance is maintained between deterministic operations (predefined rules and algorithms) and generative flexibility (creative problem-solving).
Performance Monitoring: The system continuously evaluates the workflow’s efficiency and accuracy, making improvements as needed.
Output: The task is completed successfully with a balance of precision and adaptability, ensuring optimal outcomes for the user’s query.
According to the embodiment of the present invention, the proposed workflow orchestrator system introduces a novel and efficient approach to managing complex queries during model inference. By integrating deterministic and generative approaches and leveraging advanced algorithms, the system ensures precise, efficient, and value-driven outputs. It provides the following advantages:
1.Enhanced Efficiency: The orchestrator layer streamlines workflows by dynamically decomposing and optimizing tasks.
2.Contextual Precision: Iterative questioning and attribute extraction ensure clarity and relevance.
3.Hybrid Approach: Combines deterministic systems with generative models for balanced outputs.
4.Optimized Workflows: Uses algorithm for efficient pathfinding.
5.Dynamic Adaptability: Allows real-time adjustments to workflows based on intermediate results.
While considerable emphasis has been placed herein on the specific elements of the preferred embodiment, it will be appreciated that many alterations can be made and that many modifications can be made in preferred embodiment without departing from the principles of the invention. These and other changes in the preferred embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.
, Claims:We claim,
1. A system and method of workflow orchestrator during model inference comprising: characterised in that,
the system comprises of orchestrator layer designed to interface between the user query input and the underlying inferencing model layer and the orchestrator layer comprises of attribute extraction module, task decomposition module, probe system interaction module, iterative questioning module and workflow optimization module;
wherein the method of workflow orchestrator consists of attribute extraction module that extracts predefined attributes such as intent, actors, actions, and context from the query by leveraging underlying models; task decomposition module that splits attributes and intents into smaller blocks, each representing a unit of work; probe system interaction module that routes these blocks through a probe system that combines LLM inferencing and deterministic layers such as APIs, databases, to propose paths for the intent; iterative questioning module that refines the scope of the query through iterative questioning to resolve ambiguities; and workflow optimization module that constructs an optimized workflow using algorithm to determine the most efficient path for task completion.
2. The system and method as claimed in claim 1, wherein in the attribute extraction module, the system first identifies the user’s query that is selected from solving a problem, gathering information, or executing a specific task, the actors such as user, external tools or APIs identify all the entities involved in the user query, then the specific steps or operations implied by the query are extracted, any additional details, such as constraints, preferences, or situational factors, are captured to ensure the task aligns with the user's needs and a structured representation of the query’s attributes, providing a foundation for subsequent steps is provided as output of this module.

3. The system and method as claimed in claim 1, wherein in the task decomposition module, once the attributes are extracted, the system decomposes the query into smaller, manageable blocks, each representing a self-contained unit of work to ensure that the query is broken down into logically structured tasks that can be handled independently, yet remain connected to the original intent and dependencies between tasks are identified to maintain proper sequencing and logical flow, after that the system divides the query into manageable units, ensuring each block represents a single task or logical step and it identifies dependencies between tasks to maintain proper sequencing to ensure that the decomposition preserves the original intent and that no critical elements are omitted and as output a set of smaller, logically ordered tasks that form the basis for further exploration is obtained.

4. The system and method as claimed in claim 1, wherein in the probe system interaction module, large language models are used to brainstorm and generate potential solutions or ideas for open-ended tasks, then APIs, databases, and predefined algorithms are used to validate the solutions or retrieve factual data and for ambiguous blocks, the system probes further, asking clarifying questions or conducting additional data retrieval to ensure accuracy and a list of potential solutions, combining generative insights and grounded factual data for each task are given as output.

5. The system and method as claimed in claim 1, wherein in the iterative questioning, the system identifies ambiguities or gaps in information and generates precise questions to resolve them and it integrates user feedback or retrieved information to refine the task definition and this process continues until each block is well-defined and free of ambiguity and fully clarified tasks with a clear scope, ready for workflow construction are given as output.

6. The system and method as claimed in claim 1, wherein in the workflow construction module the context, actors, actions, and additional details are integrated into the workflow to provide a comprehensive roadmap and using algorithm, the system determines the most efficient path to complete the tasks while minimizing resource use and time and the workflow includes fallback mechanisms to handle potential errors or exceptions during execution and as output an optimized, detailed workflow is designed to achieve the user’s goal effectively and efficiently.

7. The system and method as claimed in claim 1, wherein in the dynamic execution module, the system monitors progress and adapts the workflow based on intermediate outputs or changing conditions and a balance is maintained between deterministic operations such as predefined rules and algorithms and generative flexibility such as creative problem-solving and the system continuously evaluates the workflow’s efficiency and accuracy, making improvements as needed and the task is completed successfully with a balance of precision and adaptability, ensuring optimal outcomes for the user’s query.