1. A computer-implemented method for evaluating long-form descriptive answers using an automated evaluation system (100), comprising: a. a automated evaluation system (100), an input comprising a question (102), a gold reference answer (104), and a candidate answer (106) submitted for evaluation; b. a semantic retrieval module (110), semantic representations of the question (102) and the candidate answer (106), and accessing a dense vector index (108) storing embedded representations of reference answers; c. a retrieval query (112) based on the semantic representations of the question (102) and the candidate answer (106), and retrieving a plurality of reference answers (114) from the dense vector index (108) based on similarity measures; d. a retrieved reference answers (114) and the candidate answer (106) to a rubric-guided evaluation module (114) configured to evaluate the candidate answer according to predefined evaluation criteria including coverage (114), accuracy (114), faithfulness (114), and clarity (114); e. determining evaluation outputs based on the rubric-guided evaluation, the evaluation outputs comprising a predicted numerical score (116) and a predicted categorical label (118) corresponding to the evaluated candidate answer; and f. a textual critique (120) referencing the retrieved reference answers (114) to provide an evidence-grounded explanation of the evaluation outputs.

2. The system (100) as claimed in claim 1, wherein the semantic representations of the question (102), the gold reference answer (104), and the candidate answer (106) are generated using a sentence embedding model.

3. The system (100) as claimed in claim 1, wherein the dense vector index (108) is a vector similarity search database configured to store embedded representations of reference answers and perform semantic similarity search. The system (100) as claimed in claim 1, wherein a retrieval query (112) is generated based on a combined semantic representation of the question (102) and the candidate answer (106) for retrieving relevant reference answers (114).

4. The system (100) as claimed in claim 1, wherein the semantic retrieval module (110) retrieves a plurality of top-ranked reference answers forming the retrieved reference set (114) used for evaluation.

5. The system (100) as claimed in claim 1, wherein the rubric-guided evaluation module (114) evaluates the candidate answer (106) based on a plurality of rubric-based criteria including coverage, accuracy, faithfulness, and clarity (114).

6. The system (100) as claimed in claim 1, wherein the rubric-guided evaluation module (114) determines rubric-based subscores corresponding to each of the evaluation criteria.

7. The system (100) as claimed in claim 1, wherein the predicted categorical label (118) corresponds to predefined assessment categories including masterful, competent, emerging, and inadequate.

8. The system (100) as claimed in claim 1, wherein the textual critique (120) includes an explanation referencing the retrieved reference answers (114) to justify the predicted numerical score (116) and the predicted categorical label (118).

9. The system (100) as claimed in claim 1, wherein the predicted numerical score (116) corresponds to a predefined scoring range used for automated evaluation of descriptive answers.

6. DATE AND SIGNATURE Dated this on 07th March, 2026 Signature Mr. Srinivas Maddipati (IN/PA 3124) Agent for applicant

Description:DESCRIPTION
Technical Field of the Invention

The present invention relates to the field of artificial intelligence and natural language processing, particularly in relation to automated educational assessment systems. The invention provides a retrieval-augmented framework designed to evaluate descriptive or long-form textual responses. The system utilizes semantic embedding, vector similarity retrieval, rubric-based scoring, and large language model-driven critique generation to enable reliable and interpretable automated grading. The invention is particularly useful in digital learning platforms, online examination systems, and intelligent tutoring environments.

Background of the Invention

Automated evaluation of descriptive or long-form answers has become increasingly important in modern educational environments due to the rapid growth of digital learning platforms, online examinations, and large-scale assessment systems. In many academic environments, subjects such as science, humanities, and technical disciplines require students to provide written explanations, which are traditionally assessed by human evaluators. However, manual evaluation of such responses is often time-consuming, resource-intensive, and difficult to scale when dealing with a large number of students. As a result, various automated assessment systems have been developed to assist educators in performing evaluation tasks in a more efficient and scalable manner.

Early automated assessment systems predominantly relied on rule-based mechanisms, keyword matching techniques, or statistical regression models for evaluating student responses. In such systems, the evaluation was typically performed based on the presence of predefined keywords, syntactic similarity measures, or trained scoring functions. However, these approaches were limited in their ability to capture the deeper semantic meaning of descriptive answers. Consequently, the systems often produced inaccurate or inconsistent evaluation results, particularly in situations where students used alternative wording or provided conceptually correct explanations that did not exactly correspond to the expected keywords.

With the advancement of natural language processing technologies, particularly the development of large language models (LLMs), automated grading systems have significantly evolved to incorporate contextual understanding and reasoning capabilities. Such systems are capable of analyzing complex textual responses and generating evaluation outcomes based on predefined prompts and scoring instructions. However, despite these advancements, many existing approaches primarily depend on the internal reasoning capabilities of the language models without relying on external authoritative references. As a consequence, such systems may generate unsupported explanations, inconsistent scoring outcomes, and limited transparency in the overall evaluation process.

Existing automated evaluation techniques often fail to incorporate structured grading rubrics similar to those employed by human evaluators during manual assessment. In conventional academic evaluation, human examiners typically consider multiple assessment dimensions, including conceptual coverage, factual accuracy, clarity of explanation, and logical consistency. However, many automated systems generate only a single overall score or rely on simplified evaluation metrics that do not adequately capture these multidimensional grading criteria and also lack of evidence-supported explanations in such systems makes it difficult for educators and learners to understand the basis on which the evaluation decisions are derived.

Certain recent approaches have utilized retrieval-based techniques and retrieval-augmented generation methods in natural language processing applications. However, these approaches are mainly designed for tasks such as question answering, document retrieval, or content generation rather than for evaluating descriptive answers. Moreover, existing systems rarely combine semantic retrieval, rubric-based scoring, and critique generation within a single framework for answer evaluation. Therefore, there remains a need for an automated evaluation system capable of providing reliable, interpretable, and evidence-based assessment of long-form responses.

Accordingly, the present invention provides a retrieval-augmented framework for evaluating long-form answers in AI-based educational assessment systems. The framework integrates semantic retrieval of reference answers, rubric-based multi-dimensional scoring, and automated critique generation to produce evidence-supported evaluation results. By combining retrieval techniques with large language model-based reasoning, the invention enables more reliable and interpretable automated grading aligned with human assessment practices.

Objects of the Invention

The primary object of the present invention is to provide a retrieval-augmented system for automated evaluation of long-form descriptive answers, enabling reliable and interpretable grading outcomes.

Another object of the present invention is to integrate semantic retrieval with large language model-based evaluation to improve reliability in automated grading.

Another object of the invention is to provide a rubric-guided scoring mechanism based on coverage, accuracy, faithfulness, and clarity.

Another object of the invention is to generate structured critiques linked to retrieved evidence.

Another object of the invention is to provide scalable and consistent automated grading framework with improved classification accuracy and reducing misclassification of borderline categories.

Another object of the present invention is to enhance interpretability of automated grading decisions through explainable AI-based evaluation mechanisms.

Another object of the present invention is to provide an evidence-grounded evaluation mechanism that minimizes unsupported or hallucinated justifications during answer assessment.

Brief Summary of the Invention

The present invention provides a retrieval-augmented framework for interpretable evaluation of long-form descriptive answers in AI-based educational assessment systems. The framework integrates semantic retrieval, rubric-based evaluation, and critique generation to produce reliable and transparent automated grading. By utilizing external reference answers during evaluation, the system enhances consistency between automated scoring and human assessment practices.

In one embodiment, the system is configured to receive a question and a corresponding candidate answer submitted by a learner. The candidate answer along with a plurality of reference answers is processed through a semantic embedding module that converts the textual content into vector representations. The generated embeddings are stored and indexed using a vector similarity search mechanism, thereby enabling retrieval of semantically relevant reference answers corresponding to the candidate response.

The retrieved reference answers are provided to an evaluation module along with a structured scoring rubric. The evaluation module analyzes the candidate answer with respect to the references and rubric criteria and generates outputs including a score, grade label, rubric-based sub scores, and a structured critique.

The system provides an evidence-grounding mechanism that ensures the generated critiques are supported by the retrieved reference materials. This approach reduces unsupported reasoning and improves the reliability of automated grading. Consequently, the framework enhances scoring consistency, interpretability, and scalability in automated evaluation systems.

Brief Description of Drawings

The invention will be further understood from the following detailed description of a preferred embodiment taken in conjunction with an appended drawing, in which:

Figure 1 Overview of the QA-J Evaluation Pipeline. The framework combines retrieval-based grounding and rubric-oriented critique generation to evaluate candidate answers with both quantitative scores and qualitative reasoning.

REFERENCE NUMERALS
100 Automated answer evaluation system / QA-J evaluation pipeline
102 Question input
104 Gold reference answer
106 Candidate answer
108 Dense vector index / vector database
110 Semantic retrieval module
112 Retrieval query generation module
114 Retrieved reference answers
115 Predicted numerical score output
116 Predicted categorical label output
130 Textual critique generation module

Detailed Description of the Invention

It is to be understood that the present disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. In addition, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

The use of “including”, “comprising” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items. Further, the use of terms “first”, “second”, and “third”, and the like, herein do not denote any order, quantity, or importance, but rather are used to distinguish one element from another.

According to an exemplary embodiment of the present invention, a computer-implemented framework for evaluating long-form descriptive answers in an automated educational assessment system (100) is disclosed. The system (100) is configured to receive input data comprising a question (102), a gold reference answer (104), and a candidate answer (106) submitted for evaluation. The question (102) represents a prompt provided to a learner, the gold reference answer (104) represents authoritative information corresponding to the question, and the candidate answer (106) represents the response submitted by the user for assessment.

In accordance with an exemplary embodiment of the present invention, wherein it generates a retrieval query (112) using the semantic representations derived from the question (102) and the candidate answer (106) and the retrieval query (112) is used to perform a similarity search within the dense vector index (108) in order to identify a plurality of retrieved reference answers (114) that are semantically relevant to the candidate answer (106).

In accordance with an exemplary embodiment of the present invention, wherein the retrieved reference answers (114) provide contextual information that serves as supporting evidence during the evaluation of the candidate answer. The retrieved references assist the evaluation system in comparing the content of the candidate response with authoritative or expert information. By utilizing such references during assessment, the evaluation module performs an evidence-based analysis, thereby improving evaluation accuracy, minimizing unsupported reasoning, and enhancing the reliability of automated grading outcomes.

In accordance with an exemplary embodiment of the present invention, wherein the rubric-guided evaluation module (114) configured to evaluate the candidate answer (106) based on predefined evaluation criteria. The rubric-guided evaluation module (114) performs a multi-dimensional assessment using evaluation parameters including coverage (114), accuracy, faithfulness, and clarity.

In accordance with an exemplary embodiment of the present invention, wherein the coverage parameter (114) evaluates the extent to which the candidate answer addresses the key concepts of the question, the accuracy parameter (114) determines the correctness of the information provided, the faithfulness parameter (114) assesses consistency between the candidate answer and the retrieved reference answers (114), and the clarity parameter (114) evaluates the coherence and readability of the response.

In accordance with an exemplary embodiment of the present invention, wherein the evaluation outputs based on the assessment performed by the rubric-guided evaluation module (114). The outputs include a predicted numerical score (116) representing a quantitative evaluation of the candidate answer and a predicted categorical label (118) corresponding to predefined evaluation categories. The categorical label (118) may represent qualitative performance levels associated with the candidate answer.

In accordance with an exemplary embodiment of the present invention, wherein the textual critique (120) providing qualitative reasoning for the evaluation decision and the retrieved reference answers (114) to ensure that the explanation is evidence-grounded and aligned with the retrieved authoritative information.

In accordance with an exemplary embodiment of the present invention, wherein the integration of semantic retrieval, rubric-guided evaluation, and structured critique generation, the present invention enables interpretable, reliable, and scalable automated evaluation of long-form descriptive answers in educational assessment systems.

In references to figures, Figure 1 illustrates an QA-J evaluation pipeline implemented in an automated answer evaluation system (100). The system (100) receives input data comprising a question (102), a gold reference answer (104), and a candidate answer (106) submitted for evaluation. These inputs are processed by a semantic retrieval module (110) which interacts with a dense vector index (108) containing embedded representations of reference answers. The semantic retrieval module (110) generates a retrieval query (112) based on the question (102) and the candidate answer (106) to retrieve a plurality of reference answers (114) relevant to the candidate response. The retrieved references (114) are provided to a rubric-guided evaluation module configured to assess the candidate answer according to predefined evaluation parameters including coverage, accuracy, faithfulness, and clarity. Based on the evaluation, the system generates output results comprising a predicted numerical score (116), a predicted categorical label (118) corresponding to predefined assessment levels, and a textual critique (120) explaining the evaluation outcome with reference to the retrieved information, thereby enabling interpretable and evidence-based automated assessment of descriptive
Advantages of the Invention

The reduced hallucinated explanations in automated grading are achieved by incorporating a semantic retrieval mechanism configured to retrieve relevant reference answers prior to evaluation, thereby ensuring that the assessment and generated critique are grounded in retrieved evidence and improving the reliability and credibility of the evaluation process.

The improved alignment between automated evaluation and human grading standards is achieved through the proposed framework, wherein experimental validation demonstrates approximately 14% improvement in Pearson correlation and 13.6% improvement in Spearman correlation with human scores, thereby enhancing the accuracy of automated grading outcomes.

The improvement in classification accuracy of answer quality categories is achieved by the proposed system, resulting in approximately 12% enhancement in classification performance and enabling more reliable categorization of responses into qualitative labels.

The reduction in misclassification of borderline answers, particularly within intermediate evaluation categories, is achieved by the proposed framework, thereby providing more stable evaluation outcomes for responses that fall near category boundaries and provides structured rubric-based evaluation, wherein answers are assessed across multiple rubric dimensions including coverage, accuracy, faithfulness, and clarity, thereby improving fairness and transparency compared to conventional scoring systems.

The generation of structured textual critiques linked to supporting evidence is achieved by the system, wherein the generated explanations reference retrieved materials, thereby enabling clear understanding of the reasoning behind evaluation decisions.
The scalable automated assessment framework capable of handling large datasets of descriptive answers across multiple domains, thereby making the system suitable for large-scale educational environments including online learning platforms and competitive examination systems.
, Claims:CLAIMS
I/We Claim:
1. A computer-implemented method for evaluating long-form descriptive answers using an automated evaluation system (100), comprising:
a. a automated evaluation system (100), an input comprising a question (102), a gold reference answer (104), and a candidate answer (106) submitted for evaluation;
b. a semantic retrieval module (110), semantic representations of the question (102) and the candidate answer (106), and accessing a dense vector index (108) storing embedded representations of reference answers;
c. a retrieval query (112) based on the semantic representations of the question (102) and the candidate answer (106), and retrieving a plurality of reference answers (114) from the dense vector index (108) based on similarity measures;
d. a retrieved reference answers (114) and the candidate answer (106) to a rubric-guided evaluation module (114) configured to evaluate the candidate answer according to predefined evaluation criteria including coverage (114), accuracy (114), faithfulness (114), and clarity (114);
e. determining evaluation outputs based on the rubric-guided evaluation, the evaluation outputs comprising a predicted numerical score (116) and a predicted categorical label (118) corresponding to the evaluated candidate answer; and
f. a textual critique (120) referencing the retrieved reference answers (114) to provide an evidence-grounded explanation of the evaluation outputs.

2. The system (100) as claimed in claim 1, wherein the semantic representations of the question (102), the gold reference answer (104), and the candidate answer (106) are generated using a sentence embedding model.
3. The system (100) as claimed in claim 1, wherein the dense vector index (108) is a vector similarity search database configured to store embedded representations of reference answers and perform semantic similarity search.

The system (100) as claimed in claim 1, wherein a retrieval query (112) is generated based on a combined semantic representation of the question (102) and the candidate answer (106) for retrieving relevant reference answers (114).

4. The system (100) as claimed in claim 1, wherein the semantic retrieval module (110) retrieves a plurality of top-ranked reference answers forming the retrieved reference set (114) used for evaluation.

5. The system (100) as claimed in claim 1, wherein the rubric-guided evaluation module (114) evaluates the candidate answer (106) based on a plurality of rubric-based criteria including coverage, accuracy, faithfulness, and clarity (114).

6. The system (100) as claimed in claim 1, wherein the rubric-guided evaluation module (114) determines rubric-based subscores corresponding to each of the evaluation criteria.

7. The system (100) as claimed in claim 1, wherein the predicted categorical label (118) corresponds to predefined assessment categories including masterful, competent, emerging, and inadequate.

8. The system (100) as claimed in claim 1, wherein the textual critique (120) includes an explanation referencing the retrieved reference answers (114) to justify the predicted numerical score (116) and the predicted categorical label (118).

9. The system (100) as claimed in claim 1, wherein the predicted numerical score (116) corresponds to a predefined scoring range used for automated evaluation of descriptive answers.

6. DATE AND SIGNATURE

Dated this on 07th March, 2026
Signature

Mr. Srinivas Maddipati
(IN/PA 3124)
Agent for applicant