Description:BACKGROUND
Field of Invention
[001] Embodiments of the present invention generally relate to an evaluating system and particularly to a content evaluation system and a method thereof.
Description of Related Art
[002] In field of computer science engineering, there lies a stream of natural language processing also known as NLP. NLP facilitates a machine to receive and interpret natural language, natural language being a language spoken by humans (such as English, German, French, and so forth). This facility of providing natural language to machines omits usage of machine-based syntax. For example, to print a star on an output screen, all compilers require a very specific set of semantics and syntax. However, by the usage of NLP, a user commands the machine to print the star, and irrespective of the compiler and machine environment, the machine will print the star on the output screen.
[003] Moreover, the same principle is used for generating text using the machine based on prompts. However, several times the machine fails to distinguish differences between words used in prompts. Additionally, many existing solutions heavily depend on word-level text embedding, posing challenges in capturing nuanced sentence meanings. Existing approaches falter when confronted with scenarios where word order, contextual understanding, and multiple-word meanings are pivotal for accurate evaluation. Consider the example of “Bob killed Tom” and “Tom killed Bob”: tokenization obscures word order, hindering traditional embeddings from discerning distinct meanings. Similarly, words with multiple meanings, dependent on adjacent words, pose challenges for standard embeddings, evident in phrases like “Reserve Bank of India” versus “A River bank.”
[004] Furthermore, an often overlooked aspect is susceptibility to adversarial responses—crafted to deceive scoring systems. Issues include repeated sentences, irrelevant responses, and using the prompt as a reply. Addressing this requires adversarial training, exposing the model to misleading examples during training to enhance its ability to identify and handle deceptive responses.
[001] US20200020243A1 discloses a ‘No-ground truth short answer scoring’. However, the above-mentioned art is slow in operation and complex to set up.
[002] US11790227B1 discloses a ‘Systems and methods for neural content scoring’. However, the above-mentioned art operates a set of nonlinear operations that applies to a plurality of representation vectors in a neural network to generate a single vector output.
[003] There is thus a need for an improved and advanced content system that can administer the aforementioned limitations in a more efficient manner.
SUMMARY
[004] Embodiments in accordance with the present invention provide a content-evaluation system. The system comprising: an input unit adapted to collect an essay corpus from a user. The system further comprising: a processor in communication with the input unit. The processor is configured to: receive the collected essay corpus from the user; and perform a text embedding on the collected essay corpus. The text embedding captures a semantic meaning of sentences in the collected essay corpus; apply a padding to the collected essay corpus. The padding is applied to adjust a length of each sentence in the collected essay corpus; determine a coherence score for the collected essay corpus using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model; determine a content score for the collected essay corpus using a Rubric-Based Automatic Short Answer Scoring System (RBASS) model; and evaluate the collected essay corpus by calculating a final score for the collected essay corpus by combining and averaging the coherence score and the content score.
[005] Embodiments in accordance with the present invention further provide a method for evaluating content using a content evaluation system. The method comprising steps of: receiving a collected essay corpus from a user; performing a text embedding on the collected essay corpus; applying a padding to the collected essay corpus. The padding is applied to adjust a length of each sentence in the collected essay corpus. The method further comprising steps of: determining a coherence score for the collected essay corpus using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model; determining a content score for the collected essay corpus using a Rubric-Based Automatic Short Answer Scoring System (RBASS) model; and evaluating the collected essay corpus by calculating a final score for the collected essay corpus by combining and averaging the coherence score and the content score.
[006] Embodiments of the present invention may provide a number of advantages depending on their particular configuration. First, embodiments of the present application may provide a content evaluation system.
[007] Next, embodiments of the present application may provide a content evaluation system that conducts a sentence level encoding to capture sentence-to-sentence connectivity and sequence.
[008] Next, embodiments of the present application may provide a content evaluation system that handles adversarial responses, such as irrelevant responses, repeated sentences, and so forth.
[009] Next, embodiments of the present application may provide a content evaluation system that is fully automated and robust.
[0010] Next, embodiments of the present application may provide a content evaluation system that reduces dependence on human input.
[0011] Next, embodiments of the present application may provide a content evaluation system that adeptly handles adversarial responses.
[0012] Next, embodiments of the present application may provide a content evaluation system that accommodates polysemous words.
[0013] Next, embodiments of the present application may provide a content evaluation system that demonstrates effectiveness across diverse domains.
[0014] These and other advantages will be apparent from the present application of the embodiments described herein.
[0015] The preceding is a simplified summary to provide an understanding of some embodiments of the present invention. This summary is neither an extensive nor exhaustive overview of the present invention and its various embodiments. The summary presents selected concepts of the embodiments of the present invention in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other embodiments of the present invention are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The above and still further features and advantages of embodiments of the present invention will become apparent upon consideration of the following detailed description of embodiments thereof, especially when taken in conjunction with the accompanying drawings, and wherein:
[0017] FIG. 1A illustrates a block diagram of a content evaluation system, according to an embodiment of the present invention;
[0018] FIG. 1B illustrates a determination of a coherence score using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model, according to an embodiment of the present invention;
[0019] FIG. 2 illustrates a block diagram of a processor of the content evaluation system, according to an embodiment of the present invention;
[0020] FIG. 3A illustrates an evaluation of a final score, according to an embodiment of the present invention;
[0021] FIG. 3B illustrates a graph depicting a result comparison of the content evaluation system, according to an embodiment of the present invention; and
[0022] FIG. 4 depicts a flowchart of a method for evaluating content using the content evaluation system, according to an embodiment of the present invention.
[0023] The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word "may" is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include”, “including”, and “includes” mean including but not limited to. To facilitate understanding, like reference numerals have been used, where possible, to designate like elements common to the figures. Optional portions of the figures may be illustrated using dashed or dotted lines, unless the context of usage indicates otherwise.
DETAILED DESCRIPTION
[0024] The following description includes the preferred best mode of one embodiment of the present invention. It will be clear from this description of the invention that the invention is not limited to these illustrated embodiments but that the invention also includes a variety of modifications and embodiments thereto. Therefore, the present description should be seen as illustrative and not limiting. While the invention is susceptible to various modifications and alternative constructions, it should be understood, that there is no intention to limit the invention to the specific form disclosed, but, on the contrary, the invention is to cover all modifications, alternative constructions, and equivalents falling within the scope of the invention as defined in the claims.
[0025] In any embodiment described herein, the open-ended terms "comprising", "comprises”, and the like (which are synonymous with "including", "having” and "characterized by") may be replaced by the respective partially closed phrases "consisting essentially of", “consists essentially of", and the like or the respective closed phrases "consisting of", "consists of”, the like.
[0026] As used herein, the singular forms “a”, “an”, and “the” designate both the singular and the plural, unless expressly stated to designate the singular only.
[0027] FIG. 1A illustrates a block diagram of a content evaluation system 100 (hereinafter referred to as the system 100), according to an embodiment of the present invention. In an embodiment of the present invention, the system 100 may be configured to evaluate a score of a textual corpus. The textual corpus may be provided by a user, in an embodiment of the present invention. In another embodiment of the present invention, the textual corpus may be self-generated by the system 100 and may further be evaluated for provision of the score. According to embodiments of the present invention, the textual corpus may be, but not limited to, a story, a play, an article, a technical documentation, an article, a debate, and so forth. In a preferred embodiment of the present invention, the textual corpus may be an essay corpus. Embodiments of the present invention are intended to include or otherwise cover any type of the textual corpus, including known, related art, and/or later developed technologies.
[0028] According to embodiments of the present invention, the system 100 may comprise an input unit 102, a processor 104, a coherence score 106, a content score 108, and a final score 110.
[0029] In an embodiment of the present invention, the input unit 102 may be a device utilized by the user to provide and/or input the essay corpus to the system 100. The input unit 102 may further be configured to enable the user to respond to an already provided essay corpus to the system 100, in an embodiment of the present invention. In an embodiment of the present invention, the input unit 102 may further be configured to display the final score 110 evaluated corresponding to the provided essay corpus.
[0030] The input unit 102 may be, but not limited to, a personal computer, a consumer device, and alike. Embodiments of the present invention are intended to include or otherwise cover any type of the input unit 102 including known, related art, and/or later developed technologies.
[0031] In an embodiment of the present invention, the personal computer may be, but not limited to, a desktop, a server, a laptop, and alike. Embodiments of the present invention are intended to include or otherwise cover any type of the personal computer including known, related art, and/or later developed technologies.
[0032] Further, in an embodiment of the present invention, the consumer device may be, but not limited to, a tablet, a mobile phone, a notebook, a netbook, a smartphone, a wearable device, and so forth. Embodiments of the present invention are intended to include or otherwise cover any type of the consumer device including known, related art, and/or later developed technologies.
[0033] According to an embodiment of the present invention, the input unit 102 may comprise software applications such as, but not limited to, a grammar application, a dictionary application, and alike. In a preferred embodiment of the present invention, the input unit 102 may comprise a computer application (not shown) which may be a computer-readable program installed in the input unit 102 for executing functions associated with the system 100.
[0034] In an embodiment of the present invention, the processor 104 may be connected to the input unit 102 for collecting the essay corpus provided by the user. The processor 104 may be configured to determine the coherence score 106 and the content score 108 of the collected essay corpus, in an embodiment of the present invention. In an embodiment of the present invention, the processor 104 may further evaluate the final score 110 for the collected essay corpus by combining and averaging the coherence score 106 and the content score 108.
[0035] The processor 104 may further be configured to execute computer-executable instructions to generate an output relating to the system 100. According to embodiments of the present invention, the processor 104 may be, but not limited to, a Programmable Logic Control (PLC) unit, a microprocessor, a development board, and so forth. Embodiments of the present invention are intended to include or otherwise cover any type of the processor 104 including known, related art, and/or later developed technologies. In an embodiment of the present invention, the processor 104 may further be explained in conjunction with FIG. 2.
[0036] FIG. 1B illustrates a determination of the coherence score 106 using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model, according to an embodiment of the present invention.
[0037] In an embodiment of the present invention, the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may be a sequence-to-sequence model designed to evaluate coherence in the collected essay corpus and/or in the responses for the already collected essay corpus.
[0038] In an embodiment of the present invention, the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may tokenize the collected essay corpus into sentence vectors using sentence Bidirectional Encoder Representations from Transformers (BERT). The Bidirectional Encoder Representations from Transformers (BERT) may be a pre-trained natural language processing model that may understand a context by considering both preceding and succeeding words in a sentence, enhancing grasp of words, their meanings, and relationships among the words.
[0039] In an embodiment of the present invention, the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may convert the sentence vectors into 3-dimensional vectors, such that the 3-dimensional vectors may be conducive for training on a neural network.
[0040] In an embodiment of the present invention, the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may consist of five stacked layers of Long Short-Term Memory (LSTM) units. Each of the stacked layers of Long Short-Term Memory (LSTM) units may be equipped with an input, an output, and a context gate, in an embodiment of the present invention. In an embodiment of the present invention, the Long Short-Term Memory (LSTM) units may be a type of recurrent neural network (RNN) architecture designed to address the vanishing gradient problem, allowing for more effective capture and retention of long-term dependencies in sequential data.
[0041] In an embodiment of the present invention, a first layer of a Long Short-Term Memory (LSTM) unit may be a Long Short-Term Memory (LSTM) input layer with a variability of 128*94. In an embodiment of the present invention, a second layer of the Long Short-Term Memory (LSTM) unit may be a Long Short-Term Memory (LSTM) 1 layer with the variability of 1*300. In an embodiment of the present invention, a third layer of the Long Short-Term Memory (LSTM) unit may be a Long Short-Term Memory (LSTM) 2 layer with the variability of 1*200. In an embodiment of the present invention, a fourth layer of the Long Short-Term Memory (LSTM) unit may be a Long Short-Term Memory (LSTM) 3 layer with the variability of 1*100. In an embodiment of the present invention, a fifth layer of a Long Short-Term Memory (LSTM) unit may be a dense layer with the variability of 64*1.
[0042] In an embodiment of the present invention, the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may be trained using a Root Mean Square Propagation (RMSProp) optimizer to minimize a mean square error. The Root Mean Square Propagation (RMSProp) may be an optimization algorithm used in training the neural network. The Root Mean Square Propagation (RMSProp) may adopt a learning rate for each parameter individually by dividing the learning rate for a weight by the square root of the mean of the squared gradients for that weight. The Root Mean Square Propagation (RMSProp) may mitigate the issues of oscillating or diminishing learning rates encountered in optimization problems.
[0043] In an embodiment of the present invention, a dropout rate of the Coherence-Based Automatic Short Answer Scoring System (CBASS) model may be 0.5. The dropout rate may be incorporated to address potential overfitting, while the initial learning rate may be 0.001, and a rectified linear unit (ReLU) may serve as an activation function.
[0044] FIG. 2 illustrates a block diagram of the processor 104 of the system 100, according to an embodiment of the present invention. The processor 104 may comprise the computer-executable instructions in form of programming modules such as a collection module 200, an embedding module 202, a scoring module 204, and an evaluation module 206.
[0045] In an embodiment of the present invention, the collection module 200 may be configured to receive the collected essay corpus from the user using the input unit 102. According to embodiments of the present invention, the user may provide the essay corpus using means such as, but not limited to, a typing means, a vocal input means, a character recognition means, and so forth. Embodiments of the present invention are intended to include or otherwise cover any input means for providing the essay corpus to the system 100, including known, related art, and/or later developed technologies.
[0046] The collection module 200 may further be configured to receive the responses for the already collected essay corpus from other users, in an embodiment of the present invention. In an embodiment of the present invention, the collection module 200 may further be configured to transmit the collected essay corpus to the embedding module 202.
[0047] In an embodiment of the present invention, the embedding module 202 may be activated upon receipt of the collected essay corpus from the collection module 200. The embedding module 202 may be configured to perform a text embedding on the collected essay corpus, in an embodiment of the present invention. In an embodiment of the present invention, the text embedding may capture a semantic meaning of sentences in the collected essay corpus. In a preferred embodiment of the present invention, the text embedding performed on the collected essay corpus may be a sentence-level text embedding.
[0048] In another embodiment of the present invention, the embedding module 202 may be configured to apply a padding to the collected essay corpus. The padding may be applied to adjust a length of each sentence in the collected essay corpus, in an embodiment of the present invention. In an embodiment of the present invention, the padding may be performed to ensure uniformity in the length of each sentence.
[0049] In a preferred embodiment of the present invention, a size of the padding may be 96*128. In another embodiment of the present invention, the size of the padding may be 64*64. In another embodiment of the present invention, the size of the padding may be 128*128. Further, the size of the padding may be consistent in input dimensions, in an embodiment of the present invention. Embodiments of the present invention are intended to include or otherwise cover any size of the padding including known, related art, and/or later developed technologies.
[0050] Upon performing the text embedding and application of the padding, the collected essay corpus may be transmitted to the scoring module 204.
[0051] In an embodiment of the present invention, the scoring module 204 may be activated upon receipt of the embedded and padded collected essay corpus from the embedding module 202. The scoring module 204 may be configured to determine the coherence score 106 for the collected essay corpus using the Coherence-Based Automatic Short Answer Scoring System (CBASS) model, in an embodiment of the present invention.
[0052] In another embodiment of the present invention, the scoring module 204 may be configured to determine the content score 108 for the collected essay corpus using a Rubric-Based Automatic Short Answer Scoring System (RBASS) model.
[0053] In an embodiment of the present invention, the Rubric-Based Automatic Short Answer Scoring System (RBASS) model may embed the collected essay corpus into vectors using a Sentence_BERT technique. The Sentence_BERT technique may be designed to capture nuanced semantic representations of the collected essay corpus, in an embodiment of the present invention.
[0054] In another embodiment of the present invention, the collected essay corpus supplied to the Rubric-Based Automatic Short Answer Scoring System (RBASS) model for determination of the content score 108 may carry a maximum mark of 5 and six centroids including 0.
[0055] In an embodiment of the present invention, the Rubric-Based Automatic Short Answer Scoring System (RBASS) model may randomly select default centroids and calculate cosine similarities between each response and every centroid.
[0056] In an embodiment of the present invention, based on these calculated similarities, responses may then be assigned to their nearest centroid lists. After this assignment, the Rubric-Based Automatic Short Answer Scoring System (RBASS) model may proceed to recalculate centroids. This iterative process of assigning responses to centroids and recalculating continues until no observable change exists between the new and old centroids. The convergence of centroids may signify the completion of the clustering process.
[0057] In an embodiment of the present invention, the Rubric-Based Automatic Short Answer Scoring System (RBASS) model may leverage the cosine similarity metric and iterative centroid assignment to cluster responses around representative points, enabling the determination of content scores based on proximity. By randomly initializing centroids and iteratively refinements, the Rubric-Based Automatic Short Answer Scoring System (RBASS) model may aim to robustly categorize responses, contributing to an automated system capable of assessing the content relevance of both expert and student answers in the context of specific prompts.
[0058] In an embodiment of the present invention, the scoring module 204 may further transmit the determined coherence score 106 and the determined content score 108 for the collected essay corpus to the evaluation module 206.
[0059] In an embodiment of the present invention, the evaluation module 206 may be activated upon receipt of the coherence score 106 and the content score 108 for the collected essay corpus from the scoring module 204. The evaluation module 206 may be configured to evaluate the collected essay corpus, in an embodiment of the present invention. In an embodiment of the present invention, the collected essay corpus may be evaluated by calculating the final score 110. The final score 110 may be calculated by combining and averaging the coherence score 106 and the content score 108, in an embodiment of the present invention.
[0060] In another embodiment of the present invention, the evaluation module 206 may be configured to compare the final score 110 with a benchmark score of the corresponding essay corpus. Upon comparison, if the final score 110 is greater than the benchmark score, then the evaluation module 206 may validate the collected essay corpus. Else, the evaluation module 206 may invalidate the collected essay corpus, and may prompt the user to provide another and/or improve the essay corpus.
[0061] In yet another embodiment of the present invention, the evaluation module 206 may be configured to grade the collected essay corpus based on the final score 110 against comparison with benchmark ranges. In an exemplary embodiment, if the final score 110 may lie in a first benchmark range, then the evaluation module 206 may grade the collected essay corpus as ‘EXCELLENT’. In another exemplary embodiment, if the final score 110 may lie in a second benchmark range, then the evaluation module 206 may grade the collected essay corpus as ‘VERY GOOD’. In yet another exemplary embodiment, if the final score 110 may lie in a last benchmark range, then the evaluation module 206 may grade the collected essay corpus as ‘FAIL’.
[0062] FIG. 3A illustrates an evaluation of the final score 110, according to an embodiment of the present invention. In an exemplary embodiment of the present invention, a response 300 may be collected from the user. The collected response 300 may further be converted into a dense vector 302. Further, the coherence score 106, and the content score 108 may be determined for the collected response 300 using the Coherence-Based Automatic Short Answer Scoring System (CBASS) model, and the Rubric-Based Automatic Short Answer Scoring System (RBASS) model respectively. Later, the final score 110 for the collected response 300 may be calculated by combining and averaging the coherence score 106 and the content score 108.
[0063] FIG. 3B illustrates a graph 304 depicting a result comparison of the system 100, according to an embodiment of the present invention. In an embodiment of the present invention, an x-axis of the graph 304 may depict the response 300 collected by the system. The y-axis of the graph may depict the final score 110 calculated by the system 100 for each response 300, in an embodiment of the present invention.
[0064] In an embodiment of the present invention, the response 300 in a first iteration collected by the system 100 may have an actual score of 9. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 9.1. However, other known models may calculate a score of 7.8 for the same corresponding response 300.
[0065] In an embodiment of the present invention, the response 300 in a second iteration collected by the system 100 may have the actual score of 2. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 2.1. However, other known models may calculate the score of 1.5 for the same corresponding response 300.
[0066] In an embodiment of the present invention, the response 300 in a third iteration collected by the system 100 may have the actual score of 7. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 7. However, other known models may calculate the score of 5.2 for the same corresponding response 300.
[0067] In an embodiment of the present invention, the response 300 in a fourth iteration collected by the system 100 may have the actual score of 0. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 0.02. However, other known models may calculate the score of 2.1 for the same corresponding response 300.
[0068] In an embodiment of the present invention, the response 300 in a fifth iteration collected by the system 100 may have the actual score of 0. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 0.62. However, other known models may calculate the score of 2.91 for the same corresponding response 300.
[0069] In an embodiment of the present invention, the response 300 in a sixth iteration collected by the system 100 may have the actual score of 5. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 4.91. However, other known models may calculate the score of 7.7 for the same corresponding response 300.
[0070] In an embodiment of the present invention, the response 300 in a seventh iteration collected by the system 100 may have the actual score of 0. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 0. However, other known models may calculate the score of 0.5 for the same corresponding response 300.
[0071] In an embodiment of the present invention, the response 300 in an eighth iteration collected by the system 100 may have the actual score of 6. The final score 110 of the corresponding response 300 as calculated by the system 100 may be 5.89. However, other known models may calculate the score of 7.3 for the same corresponding response 300.
[0072] FIG. 4 depicts a flowchart of a method 400 for evaluating content using the system 100, according to an embodiment of the present invention.
[0073] At step 402, the system 100 may receive the collected essay corpus from the user.
[0074] At step 404, the system 100 may perform the text embedding on the collected essay corpus.
[0075] At step 406, the system 100 may apply the padding to the collected essay corpus.
[0076] At step 408, the system 100 may determine the coherence score 106 for the collected essay corpus using the Coherence-Based Automatic Short Answer Scoring System (CBASS) model.
[0077] At step 410, the system 100 may determine the content score 108 for the collected essay corpus using the Rubric-Based Automatic Short Answer Scoring System (RBASS) model.
[0078] At step 412, the system 100 may evaluate the collected essay corpus by calculating the final score 110 for the collected essay corpus by combining and averaging the coherence score 106 and the content score 108.
[0079] While the invention has been described in connection with what is presently considered to be the most practical and various embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.
[0080] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined in the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements within substantial differences from the literal languages of the claims. , Claims:CLAIMS
I/We Claim:
1. A content evaluation system (100), the system (100) comprising:
an input unit (102) adapted to collect an essay corpus from a user; and
a processor (104) in communication with the input unit (102), characterized in that the processor (104) is configured to:
receive the collected essay corpus from the user;
perform a text embedding on the collected essay corpus, wherein the text embedding captures a semantic meaning of sentences in the collected essay corpus;
apply a padding to the collected essay corpus; wherein the padding is applied to adjust a length of each sentence in the collected essay corpus;
determine a coherence score (106) using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model;
determine a content score (108) using a Rubric-Based Automatic Short Answer Scoring System (RBASS) model; and
evaluate the content by calculating a final score (110) for the collected essay corpus by combining and averaging the coherence score (106) and the content score (108).
2. The system (100) as claimed in claim 1, wherein the text embedding performed on the collected essay corpus is a sentence-level text embedding.
3. The system (100) as claimed in claim 1, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model tokenizes the collected essay corpus into sentence vectors using a sentence Bidirectional Encoder Representations from Transformers (BERT).
4. The system (100) as claimed in claim 1, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model consists of five stacked layers of Long Short-Term Memory (LSTM) units wherein each of the stacked layers is equipped with an input, an output, and a context gate.
5. The system (100) as claimed in claim 1, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model is trained using a Root Mean Square Propagation (RMSProp) optimizer to minimize a mean square error.
6. A method (400) for evaluating content using a content evaluation system (100), the method (400) characterised by steps of:
receiving a collected essay corpus from a user;
performing a text embedding on the collected essay corpus;
applying a padding to the collected essay corpus; wherein the padding is applied to adjust a length of each sentence in the collected essay corpus;
determining a coherence score (106) using a Coherence-Based Automatic Short Answer Scoring System (CBASS) model;
determining a content score (108) using a Rubric-Based Automatic Short Answer Scoring System (RBASS) model; and
evaluating the content by calculating a final score (110) for the collected essay corpus by combining and averaging the coherence score (106) and the content score (108).
7. The method (400) as claimed in claim 6, wherein the text embedding performed on the collected essay corpus is a sentence-level text embedding.
8. The method (400) as claimed in claim 6, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model tokenizes the collected essay corpus into sentence vectors using a sentence Bidirectional Encoder Representations from Transformers (BERT).
9. The method (400) as claimed in claim 6, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model consists of five stacked layers of Long Short-Term Memory (LSTM) units, wherein each of the stacked layer is equipped with an input, an output, and a context gate.
10. The method (400) as claimed in claim 6, wherein the Coherence-Based Automatic Short Answer Scoring System (CBASS) model is trained using a Root Mean Square Propagation (RMSProp) optimizer to minimize a mean square error.
Date: February 20, 2024
Place: Noida

Dr. Keerti Gupta
Agent for the Applicant
(IN/PA-1529)