Abstract: The present invention describes a system and method for mining of test cases using graph-based repository knowledge and large language models. The system comprises of an input unit, a processing unit configured to execute instructions, a memory storing invoice data, rate cards, and usage logs; and output unit, wherein the processing unit further comprises of input and change understanding layer, repository knowledge graph, impact analysis engine, test case mining layer, output and feedback layer. The system converts user-specified code changes into semantic use cases using large language models. The said dynamic graph-based engine that identifies impact areas in the code repository by multi-hop traversal. The mining layer retrieves relevant test cases using a hybrid of embeddings, tags, and LLM reranking. The system uses LLMs for description generation and for intelligent ranking, multi-hop graph traversal captures deep relationships and reduces regression testing time and improves confidence in change verification.
Description:FIELD OF INVENTION
The present invention relates to the field of computer-implemented mechanisms with minimal human intervention. More particularly, the invention relates to identifying relevant test cases for a given code change or use case using a combination of graph-based repository knowledge modelling and intelligent semantic matching powered by large language models (LLMs).
BACKGROUND
The workflow of computer-implemented mechanisms with minimal human intervention constitutes interactions between a user and a system, commonly referred to as use cases wherein a user interacts with a system to achieve a specific goal. For example, a use case for an e-commerce website may be “User places an order.”
From these user cases, test cases are derived which provide the specific steps needed to validate whether a system is capable of performing the intended use case. Test cases are detailed, step-by-step instructions used to verify specific functionalities within computer-implemented mechanisms with minimal human intervention. Test cases are created during the verification or testing phase and are crucial for identifying defects and ensuring that the outputs align with user requirements.
When the underlying code is modified, whether to implement new functionalities, correct defects, enhance system efficiency, or address other performance-related issues, it becomes necessary to make corresponding adjustments to test cases to ensure accurate testing after the modifications. This involves identifying impacted areas, selecting relevant tests, and updating or generating new test cases to cover the changes. Thus, effective test case management is central to reflect the evolving codebase to implement new features, fix bugs, improve performance, or address other issues within the system.
The current testing mechanisms exhibit significant inefficiencies in selecting appropriate test cases in response to use case or a change in code change. Regression test suites, which are a collection of test cases designed to ensure that updates or new code changes do not break existing functionality, are large in scale and thus pose the risk of considerable time and resource burden. Moreover, prevailing test selection methodologies depend largely on static analysis techniques or manual tagging processes.
To overcome the gaps in existing systems for managing code changes and performing corresponding validation, there is a need for a novel system that dynamically determines the impacted functional areas, thereby reducing regression testing time, improving resource efficiency, and enhancing confidence in change verification.
Prior Arts:
US11822533B2 discloses a computer-implemented method for managing updates to a knowledge graph with incoming change requests. The method comprises creating a snapshot of the knowledge graph, building sub-regions of the knowledge graph, classifying each incoming change to the knowledge graph so that it relates to one of the sub-regions, creating at least one change layer, wherein each of the at least one change layer relates to one of the sub-regions, wherein each of the at least one change layer comprises a portion of the incoming change requests, closing the at least one change layer upon reaching a predetermined threshold value for the at least one change layer, and integrating the at least one change layer into the knowledge graph, thereby generating a updated knowledge graph.
US11921764B2 introduces a device which receives, in near-real time, unstructured data associated with an application or a system, and extracts textual data from the unstructured data, and generates parsed textual data which processes the textual data to identify topical data and groups the topical data into categories which is used to generate a knowledge graph which is further stored in a data structure. The knowledge graph enables the device to provide answers to questions associated with the application or the system.
US20230153527A1 discloses a system for infusing knowledge graphs and language models (LM) for natural language sentence pair tasks to receive a sentence indicative of a premise and a sentence indicative of a hypothesis. It further extracts LM embeddings for the corresponding sentence, generates a common-sense knowledge graph for corresponding sentence and nodes are derived from the common-sense knowledge graph, which are assigned importance scores, and apply an aggregation function to generate pooled values. The LM embeddings, graph embeddings and pooled values to generate concatenated data and classify the concatenation data to indicate a relationship between the natural language inference pair.
Whereas the aforementioned prior arts disclose systems for managing and integrating code changes into a knowledge graph, employing knowledge graphs and language models for natural language inference tasks, and extracting textual information from unstructured data, they do not focus on validation or test case selection. Nor do they construct a code repository knowledge graph to identify impacted functional areas or apply large language models and embeddings to semantically interpret code changes and match them with relevant test cases. Furthermore, the existing systems do not suggest the mining and ranking of test cases by leveraging graph-based repository knowledge combined with semantic scoring techniques. Therefore, there exists a need for a comprehensive mechanism that enables the identification of the impact of updates or code changes, facilitates semantic understanding of such changes, and supports the verification, analysis, and updating of test cases, while operating with minimal human intervention, thereby decreasing regression testing time, improving resource efficiency, and enhancing the reliability of change verification processes.
DEFINITIONS:
The expression “system” used hereinafter in this specification refers to an ecosystem comprising, but is not limited to a system with a user, input and output devices, processing unit, plurality of mobile devices, a mobile device-based application to identify dependencies and relationships between diverse businesses, a visualization platform, and output; and is extended to computing systems like mobile, laptops, computers, PCs, etc.
The expression “input unit” used hereinafter in this specification refers to, but is not limited to, mobile, laptops, computers, PCs, keyboards, mouse, pen drives or drives.
The expression “output unit” used hereinafter in this specification refers to, but is not limited to, an onboard output device, a user interface (UI), a display kit, a local display, a screen, a dashboard, or a visualization platform enabling the user to visualize, observe or analyse any data or scores provided by the system.
The expression “processing unit” refers to, but is not limited to, a processor of at least one computing device that optimizes the system.
OBJECTS OF THE INVENTION
The primary object of the invention is to provide a system and method for identifying relevant test cases for a given use case or code change.
Another object of the invention is to offer a less time-consuming and an inexpensive system and method for mining of test cases using a combination of graph-based repository knowledge modelling and intelligent semantic matching.
Yet another object of the invention is to dynamically determine impact areas from a use case or code change to eliminate inefficiencies following the changes.
Yet another object of the invention is to enable human feedback loops and continuous learning.
SUMMARY
Before the present invention is described, it is to be understood that the present invention is not limited to specific methodologies and materials described, as these may vary as per the person skilled in the art. It is also to be understood that the terminology used in the description is for the purpose of describing the particular embodiments only and is not intended to limit the scope of the present invention.
The present invention describes a system and method for mining of test cases using graph-based repository knowledge and large language models. The system comprises of an input unit, a processing unit configured to execute instructions, a memory storing invoice data, rate cards, and usage logs; and output unit, wherein the processing unit further comprises of input and change understanding layer, repository knowledge graph, impact analysis engine, test case mining layer, output and feedback layer. The system converts user-specified code changes into semantic use cases using large language models. The said dynamic graph-based engine that identifies impact areas in the code repository by multi-hop traversal. The mining layer retrieves relevant test cases using a hybrid of embeddings, tags, and LLM reranking. The system uses LLMs for description generation and for intelligent ranking, multi-hop graph traversal captures deep relationships and reduces regression testing time and improves confidence in change verification.
BRIEF DESCRIPTION OF DRAWINGS
A complete understanding of the present invention may be made by reference to the following detailed description which is to be taken in conjugation with the accompanying drawing. The accompanying drawing, which is incorporated into and constitutes a part of the specification, illustrates one or more embodiments of the present invention and, together with the detailed description, it serves to explain the principles and implementations of the invention.
FIG. 1 illustrates the workflow of the method of the present invention;
FIG. 2 illustrates the functions of the repository knowledge graph.
DETAILED DESCRIPTION OF INVENTION:
Before the present invention is described, it is to be understood that this invention is not limited to methodologies described, as these may vary as per the person skilled in the art. It is also to be understood that the terminology used in the description is for the purpose of describing the particular embodiments only and is not intended to limit the scope of the present invention. Throughout this specification, the word “comprise”, or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps. The use of the expression “at least” or “at least one” suggests the use of one or more elements or ingredients or quantities, as the use may be in the embodiment of the invention to achieve one or more of the desired objects or results. Various embodiments of the present invention are described below. It is, however, noted that the present invention is not limited to these embodiments, but rather the intention is that modifications that are apparent are also included.
The present invention describes the system and method for mining of test cases using graph-based repository knowledge and large language models. The system helps in identifying relevant test cases for a given code change or use case using a combination of graph-based repository knowledge modelling and intelligent semantic matching powered by large language models (LLMs). This invention dynamically determines impact areas from a change, traverses a repository knowledge graph, and retrieves relevant test cases using a hybrid algorithm combining embeddings, tagging, and LLM-based scoring.
According to the embodiment of the present invention, as described in FIG. 1, the system comprises of an input unit, a processing unit configured to execute instructions, a memory storing invoice data, rate cards, and usage logs; and output unit, wherein the processing unit further comprises of Input and Change Understanding Layer, Repository Knowledge Graph, Impact Analysis Engine, Test Case Mining Layer, Output and Feedback Layer.
According to the embodiment of the present invention, the different modules of the system function in the following manner:
1. Input and Change Understanding Layer
This layer inputs Code differences, Commit messages, PR descriptions and natural language use case inputs. It parses the change using a change parser, uses an LLM (e.g., GPT-4) to extract functional intent and generate a use case summary and annotates impacted components and modules with metadata.
2. Repository Knowledge Graph
The module maintains a directed property graph where nodes represent: functions, files, modules, test cases and edges represent relationships: calls, modifies, belongs_to, verified_by, depends_on as described in FIG. 2.
3. Impact Analysis Engine
This engine given an entry point (functions, modules, or files from the change) and performs a multi-hop traversal with relevance scoring. This engine identifies impact areas in the code repository by multi-hop traversal.
Multi-hop Graph Traversal Algorithm:
def multi_hop_relevance(start_nodes, graph, max_hops=3, decay=0.85):
visited = {}
queue = [(node, 1.0) for node in start_nodes]
for hop in range(max_hops):
next_queue = []
for node, score in queue:
if node in visited and visited[node] >= score:
continue
visited[node] = score
for neighbor in graph.get_neighbors(node):
edge_weight = graph.get_edge_weight(node, neighbor)
next_queue.append((neighbor, score * edge_weight * decay))
queue = next_queue
return {k: v for k, v in visited.items() if 'test_case' not in k}
4. Test Case Mining Layer:
In this layer, each test case is stored as a document. The layer retrieves relevant test cases using a hybrid of embeddings, tags, and LLM reranking.
{
"name": "test_payment_gateway_timeout",
"code": "...",
"tags": ["payments", "timeout", "UI"],
"linked_functions": ["retry_payment", "process_gateway"],
"description": "Verifies retry logic when the payment gateway times out."
}
Matching Technique:
• Embedding-Based Matching: Compute embeddings of use case + impacted nodes and test case descriptions using e5-large-v2 or CodeBERT.
• Tag-Based Filtering: Match semantic tags from impacted graph to test case tags.
• LLM Reranking: Use an LLM to score top N candidates.
Hybrid Relevance Scorer:
def hybrid_score(embedding_score, tag_match_score, llm_score):
return 0.4 * embedding_score + 0.3 * tag_match_score + 0.3 * llm_score
5. Output and Feedback Layer
In this layer, users can provide feedback on test case quality, which is stored for retraining and tuning the scoring model. A feedback-driven system continuously improves relevance scoring using supervised or reinforcement learning
Output format:
[
{
"test_name": "test_gateway_timeout",
"relevance_score": 0.91,
"reason": "Test validates retry logic modified in this change.",
"tags": ["payments", "timeout"]
}
]
Example Use Case
A developer modifies retry_payment() to handle a new gateway error. The system parses the change, identifies impact in the Payments module, and retrieves three test cases, of which one directly validates retry logic under timeout conditions, and another indirectly validates the fallback path. The top test case is marked as highly relevant with a score of 0.91.
Advantages :
• Combines structural repository knowledge with semantic understanding.
• Uses LLMs not only for description generation but also for intelligent ranking.
• Multi-hop graph traversal captures deep relationships.
• Reduces regression testing time and improves confidence in change verification
While considerable emphasis has been placed herein on the specific elements of the preferred embodiment, it will be appreciated that many alterations can be made and that many modifications can be made in preferred embodiment without departing from the principles of the invention. These and other changes in the preferred embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.
, Claims:We claim,
1. A system and method for mining of test cases using graph-based repository knowledge and large language models
characterised in that
the system comprises of an input unit, a processing unit configured to execute instructions, a memory storing invoice data, rate cards, and usage logs; and output unit, wherein the processing unit further comprises of input and change understanding layer, repository knowledge graph, impact analysis engine, test case mining layer, output and feedback layer;
the said system converts user-specified code changes into semantic use cases using large language models;
the said dynamic graph-based engine that identifies impact areas in the code repository by multi-hop traversal;
the mining layer retrieves relevant test cases using a hybrid of embeddings, tags, and LLM reranking;
the method to store and represent test cases in a structured document format enriched with semantic metadata.
2. The system and method as claimed in claim 1, wherein the dynamically determines impact areas from a change, traverses a repository knowledge graph, and retrieves relevant test cases using a hybrid algorithm combining embeddings, tagging, and LLM-based scoring.
3. The system and method as claimed in claim 1, wherein the feedback-driven system that continuously improves relevance scoring using supervised or reinforcement learning.
4. The system and method as claimed in claim 1, wherein the input layer inputs code differences, commit messages, PR descriptions and natural language use case inputs and it parses the change using a change parser, uses an LLM to extract functional intent and generate a use case summary and annotates impacted components and modules with metadata.
5. The system and method as claimed in claim 1, wherein the Repository Knowledge Graph maintains a directed property graph where nodes represent functions, files, modules, test cases and edges represent relationships.
6. The system and method as claimed in claim 1, wherein the system combines structural repository knowledge with semantic understanding, uses LLMs for description generation and for intelligent ranking, multi-hop graph traversal captures deep relationships and reduces regression testing time and improves confidence in change verification.
| # | Name | Date |
|---|---|---|
| 1 | 202521036849-STATEMENT OF UNDERTAKING (FORM 3) [16-04-2025(online)].pdf | 2025-04-16 |
| 2 | 202521036849-POWER OF AUTHORITY [16-04-2025(online)].pdf | 2025-04-16 |
| 3 | 202521036849-FORM 1 [16-04-2025(online)].pdf | 2025-04-16 |
| 4 | 202521036849-FIGURE OF ABSTRACT [16-04-2025(online)].pdf | 2025-04-16 |
| 5 | 202521036849-DRAWINGS [16-04-2025(online)].pdf | 2025-04-16 |
| 6 | 202521036849-DECLARATION OF INVENTORSHIP (FORM 5) [16-04-2025(online)].pdf | 2025-04-16 |
| 7 | 202521036849-COMPLETE SPECIFICATION [16-04-2025(online)].pdf | 2025-04-16 |
| 8 | 202521036849-FORM-9 [26-09-2025(online)].pdf | 2025-09-26 |
| 9 | 202521036849-FORM 18 [01-10-2025(online)].pdf | 2025-10-01 |
| 10 | Abstract.jpg | 2025-10-08 |