FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION (See Section 10 and Rule 13)
Title of invention:
METHOD AND SYSTEM FOR SENTENCE CLASSIFICATION USING DILATION AND CAPSULE BASED NEURAL NETWORK
Applicant
Tata Consultancy Services Limited A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India
Preamble to the description:
The following specification particularly describes the invention and the manner in which it is to be performed.

CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY
[001] The present application is a patent of addition of Indian Patent Application No. 201921025909, filed on 28 June 2019, the entire content of which is hereby incorporated herein by way of reference.
TECHNICAL FIELD [002] The disclosure herein generally relates to text classification, and, more particularly, to a method and system for sentence classification using dilation and capsule based neural network.
BACKGROUND [003] Natural Language Processing (NLP) and Artificial Intelligence (AI) provide opportunities in various domains and the possible use cases are endless. However, every domain has a specific requirement that poses a challenge to mere application of NLP and AI. For instance, patent attorneys are required to review technical notes and patent documents critically. There is a need for the patent attorneys to have several discussions with inventors to ascertain eligibility of the subject matter under consideration for say, patent protection. Such a discussion is an extremely critical and a time consuming task. There is also a dependence on domain expertise to evaluate the subject matter under consideration. Absence of a dataset for training a classifier for application to a patent domain also makes sentence classification a challenge.
SUMMARY
[004] Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
[005] In an aspect, there is provided a processor implemented method comprising the steps of: receiving, via one or more hardware processors configured to employ a capsule based neural network, a sentence to be classified into one or more pre-defined classes, wherein the sentence belongs to a technical note for a

given domain and comprises a plurality of words, wherein one or more words from the plurality of words are Out-of-Vocabulary (OOV) words indicative of words that relate to the given domain and remaining words are in-vocabulary words; mapping, by an embedding layer in the capsule based neural network, each word from the plurality of words to a real valued fixed length embedding vector containing lexical and semantic representation of the plurality of words; encoding, by a feature extraction layer in the capsule based neural network, using Bidirectional Long Short-Term Memory (BiLSTM), one or more high-level features of the embedding vector associated with each word from the plurality of words in the sentence into context vectors; performing, a dilation operation, by a dilation layer in the capsule based neural network, with an empirically defined dilation rate on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words; convoluting, by a primary capsule layer in the capsule based neural network, each of the context vectors having lower weights assigned to the OOV words, using a shared window (��) and the dilation rate to obtain corresponding parent capsules, wherein the shared window is , wherein � is a capsule dimension convoluting with the context vectors and is number of BiLSTM units; convoluting, by a convolutional
capsule layer in the capsule based neural network, each of the parent capsules to obtain child capsules, wherein each of the child capsules corresponds to at least one parent capsule; and wherein a coupling strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm; and computing, a probability of the received sentence belonging to the one or more pre-defined classes using a fully connected layer and a softmax layer in the capsule based neural network based on the coupling strength between the parent capsules and the child capsules.
[006] In another aspect, there is provided a system comprising: one or more data storage devices operatively coupled to one or more hardware processors and configured to store instructions configured for execution via the one or more

hardware processors, wherein the one or more hardware processors are configured to employ a capsule based neural network and perform: receiving a sentence to be classified into one or more pre-defined classes, wherein the sentence belongs to a technical note for a given domain and comprises a plurality of words, wherein one or more words from the plurality of words are Out-of-Vocabulary (OOV) words indicative of words that relate to the given domain and remaining words are in-vocabulary words; mapping, by an embedding layer in the capsule based neural network, each word from the plurality of words to a real valued fixed length embedding vector containing lexical and semantic representation of the plurality of words; encoding, by a feature extraction layer in the capsule based neural network, using Bidirectional Long Short-Term Memory (BiLSTM), one or more high-level features of the embedding vector associated with each word from the plurality of words in the sentence into context vectors; performing, a dilation operation, by a dilation layer in the capsule based neural network, with an empirically defined dilation rate on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words; convoluting, by a primary capsule layer in the capsule based neural network, each of the context vectors having lower weights assigned to the OOV words, using a shared window (��) and the dilation rate to obtain corresponding parent capsules, wherein the shared window is , wherein � is a capsule
dimension convoluting with the context vectors and is number of BiLSTM
units; convoluting, by a convolutional capsule layer in the capsule based neural network, each of the parent capsules (pi) to obtain child capsules, wherein each of the child capsules corresponds to at least one parent capsule (pi); and wherein a coupling strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm; and computing, a probability of the received sentence belonging to the one or more pre-defined classes using a fully connected layer and a softmax layer

in the capsule based neural network based on the coupling strength between the parent capsules and the child capsules.
[007] In yet another aspect, there is provided a computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: receive via one or more hardware processors configured to employ a capsule based neural network, a sentence to be classified into one or more pre-defined classes, wherein the sentence belongs to a technical note for a given domain and comprises a plurality of words, wherein one or more words from the plurality of words are Out-of-Vocabulary (OOV) words indicative of words that relate to the given domain and remaining words are in-vocabulary words; map, by an embedding layer in the capsule based neural network, each word from the plurality of words to a real valued fixed length embedding vector containing lexical and semantic representation of the plurality of words; encode, by a feature extraction layer in the capsule based neural network, using Bidirectional Long Short-Term Memory (BiLSTM), one or more high-level features of the embedding vector associated with each word from the plurality of words in the sentence into context vectors; perform, a dilation operation, by a dilation layer in the capsule based neural network, with an empirically defined dilation rate on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words; convolute, by a primary capsule layer in the capsule based neural network, each of the context vectors having lower weights assigned to the OOV words, using a shared window and the dilation rate to obtain corresponding parent
capsules, wherein the shared window is , wherein is a capsule
dimension convoluting with the context vectors and is number of BiLSTM
units; convolute, by a convolutional capsule layer in the capsule based neural network, each of the parent capsules to obtain child capsules, wherein each of the child capsules corresponds to at least one parent capsule; and wherein a coupling

strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm; and compute, a probability of the received sentence belonging to the one or more pre-defined classes using a fully connected layer and a softmax layer in the capsule based neural network based on the coupling strength between the parent capsules and the child capsules.
[008] In an embodiment, the one or more pre-defined classes include (i) Context indicative of background information of the technical note; (ii) Objective indicative of a problem addressed by an author in the technical note; (iii) Motivation indicative of an incentive for the author of the technical note to work on the problem addressed; (iv) Current Situation indicative of state-of-the-art approaches to solve the problem addressed; (v) New Observation indicative of a basis for a solution to the problem addressed by the author in the technical note; (vi) Solution indicative of a solution to the problem addressed by the author in the technical note; and (vii) Technical Advancement indicative of a competency of the solution to the problem addressed by the author in the technical note.
[009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS [010] The accompanying drawings, which are incorporated in and
constitute a part of this disclosure, illustrate exemplary embodiments and, together
with the description, serve to explain the disclosed principles:
[011] FIG.1 illustrates an exemplary block diagram of a system for
sentence classification using dilation and capsule based neural network, in
accordance with some embodiments of the present disclosure.

[012] FIG. 2A is a functional block diagram of the architecture of the capsule based neural network of FIG.1 according to some embodiments of the present disclosure.
[013] FIG.2B is a functional block diagram illustrating the dilation operation performed by the capsule based neural network of FIG.1 according to some embodiments of the present disclosure.
[014] FIG. 3A through FIG.3C illustrate an exemplary flow diagram of a computer implemented method for sentence classification using dilation and capsule based neural network, in accordance with some embodiments of the present disclosure
[015] FIG. 4 illustrates an exemplary sentence classification performed by the system of FIG.1 in accordance with some embodiments of the present disclosure.
DETAILED DESCRIPTION OF EMBODIMENTS [016] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope being indicated by the following claims.
[017] Sequence Labeling is one of the prominent tasks in Natural Language Processing (NLP). The Applicant has addressed sequence labeling task particularly in a dialogue setting in Indian patent application no. 201921025909, filed on 28 June 2019. In a dialogue setting context is arranged hierarchically and requires the model to remember something from each of the previous utterances in a dialogue.

[018] In the present disclosure, the Applicant has addressed the problem of sentence classification. Early classification approaches involved models like term frequency–inverse document frequency (tf-idf). These approaches involved use of manually crafted features followed by use of a classification model such as Support Vector Machine (SVM). After the emergence of state-of-the-art methodologies for embedding words into a fixed length vector, this form of representation became a norm in NLP. Convolutional Neural Networks (CNNs) based approaches are known for squeezing out distinctive phrases from text and hence are suitable for text classification. However, one needs to be careful in selecting the window size of the convoluting kernels, larger the window size, larger is the parameter space, which makes them difficult to train. On the other hand, smaller window sizes may lead to loss of important information. CNNs, typically involves convoluting over the input data first and then performing a down-sampling method to extract only high-level features. Traditionally, Maxpool as a down-sampling method has been used as it selects only the most relevant feature from the input but loses temporal information, if present. Recurrent Neural Network (RNN) based approaches use the recurrent structure to capture long term dependencies in texts and hence introduce less noise than CNNs. Yet, RNNs are known to be biased towards the extreme ends of the input and hence may lose information in case of large documents, where an important feature may appear anywhere not only at extremes. A combination of RNN and CNN have been used to improve the results of the task by using each other’s pros and cons. Attention has also been used along with RNN with a view to focus on specific words for classification, which has resulted in better classification accuracy. Recently, Capsule Networks have been seen performing well on some of the text classification problems. The present disclosure models the capsule layer and dilated Bidirectional Long Short-Term Memory (BiLSTM) to successfully learn to focus (using capsule) or ignore (using dilation) specific types of words or phrases.
[019] Although the Applicant has addressed the problem of sentence classification particularly for a use case that is relevant to patent attorneys. It may be noted here by those skilled in the art that although the description hereinafter

refers to pre-defined classes that have been curated from the perspective of subject matter eligibility assessment of a technical note in a given domain, the pre-defined classes may be customized to suit the use case under consideration and the embodiments explained for the classification task may be applied suitably to other use cases as well. Particularly, the present disclosure does not require a hierarchical neural network. Also, the document encoding layer, the conditional random fields (CRF) layer, and the multiple BiLSTM layers provided in the model disclosed in the Applicant’s Indian patent application no. 201921025909 are not required in the architecture provided in the present disclosure. Furthermore, rather than the context that was critical in the Indian patent application no. 201921025909, the Applicant, accuracy in the present disclosure, directs attention to lowering weights associated with Out-of-Vocabulary (OOV) words to improve the classification.
[020] In the context of the present disclosure, the OOV words are indicative of words that relate to the given domain while the remaining words are referred to as in-vocabulary words. Referring to the use case of assessing subject matter eligibility based on a technical note in a given domain, typically a patent attorney is required to cull out some key ingredients such as ‘context’, ‘motivation’, etc. for the invention disclosed in the technical note. Identifying these ingredients requires a careful manual reading and verification process in which both the patent attorney and the inventor have to confer with each other to reach a mutual agreement on key contributions and the novelty of the invention. The patent attorney then walks through a decision making work-flow, e.g., guidelines for patent eligibility provided by patent offices in various jurisdictions and uses prior experience to conclude whether the subject matter of the technical note is patent eligible.
[021] A sentence classifier that can elicit information in the form of key ingredients from the technical note (e.g. a research paper), may aid ease in understanding of the invention disclosed in the technical note from a patent eligibility perspective and reduce legal costs. A large organization that typically generates a huge Intellectual Property (IP) need not rely on patent attorneys’ time for culling out the key ingredients. The present disclosure provides an automated

system and method that classifies sentences / statements from the technical note into pre-defined classes (or ingredients, used interchangeably in the context of the present disclosure) to help reduce time spent in patent attorney-inventor interactions, expedite evaluation of subject matter eligibility and save legal costs.
[022] State-of-the-art lacks training data for sentence classification directed to the use case of subject matter eligibility. The present disclosure provides a taxonomy of the ingredients based on abstract and introduction part of research papers and annotated the sentences within them to create the training dataset. Since research papers are often provided as input to patent attorneys for drafting patent specifications, they serve as an input to the system of the present disclosure. It may be noted that novel work may belong to any domain e.g. life sciences, material science, data science, etc. which forms the basis for the presence of OOV words in a technical note. Presence of OOV words make sentence classification a challenge. Further, sentences tend to be longer than average sentences and require treatment of long term dependencies in the input sentence. As a result of these challenges, state-of-the-art-approaches for sentence classification did not perform well on the dataset generated. A taxonomy for creating a novel training dataset and use of dilation operation in a capsule based neural network is disclosed in the present disclosure to deal with OOV words and long term dependencies. Capsule layers have traditionally been used with CNNs for various image-based learning task and for text applications. Applicant’s Indian patent application no. 201921025909 provided a system and method using capsule layers in text-based learning task. The present disclosure employs an RNN with dilation for capturing the long term dependencies and then the capsule layer is used to bolster semantic processing required for ingredient identification.
[023] Referring now to the drawings, and more particularly to FIG. 1 through FIG.4, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.

[024] FIG.1 illustrates an exemplary block diagram of a system 100 for sentence classification using dilation and capsule based neural network, in accordance with some embodiments of the present disclosure. In an embodiment, the system 100 includes one or more processors 104, communication interface device(s) or input/output (I/O) interface(s) 106, and one or more data storage devices or memory 102 operatively coupled to the one or more processors 104. The one or more processors 104 that are hardware processors can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, graphics controllers, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) are configured to fetch and execute computer-readable instructions stored in the memory. In the context of the present disclosure, the expressions ‘processors’ and ‘hardware processors’ may be used interchangeably. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
[025] I/O interface(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface(s) can include one or more ports for connecting a number of devices to one another or to another server.
[026] The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, one or more modules (not shown) of the system 100 can be stored in the memory 102.

[027] FIG. 2A is a functional block diagram of the architecture of the capsule based neural network of FIG.1 and FIG.2B is a functional block diagram illustrating the dilation operation performed by the capsule based neural network of FIG.1 according to some embodiments of the present disclosure. FIG.3A through FIG.3C illustrate an exemplary flow diagram of a computer implemented method 300 for sentence classification using dilation and capsule based neural network, in accordance with some embodiments of the present disclosure. In an embodiment, the system 100 includes one or more data storage devices or memory 102 operatively coupled to the one or more processors 104 and is configured to store instructions configured for execution of steps of the method 300 by the one or more processors 104.
[028] The system 100 and method 300 the present disclosure enables mapping of statements or sentences in a research article or a technical note (used interchangeably in the context of the present disclosure) to one of the pre-defined classes or ingredients. The problem is formalized as a sentence classification problem, wherein given a received sentence and a label corresponding to the received sentence where Xi is a sequence of words, i.e.
and represents an associated class. For the sentences Xi
against which class labels are not available, the class ci is predicted such that , where C is a set of all the ingredients cj.
[029] FIG. 4 illustrates an exemplary sentence classification performed by the system of FIG.1 in accordance with some embodiments of the present disclosure. The ingredients or the pre-defined classes of the present disclosure will be explained with reference to FIG.4. In an embodiment, one or more pre-defined classes include (i) Context indicative of background information of the technical note; (ii) Objective indicative of a problem addressed by an author in the technical note; (iii) Motivation indicative of an incentive for the author of the technical note to work on the problem addressed; (iv) Current Situation indicative of state-of-the-art approaches to solve the problem addressed; (v) New Observation indicative of a basis for a solution to the problem addressed by the author in the technical note; (vi) Solution indicative of a solution to the problem addressed by the author in the

technical note; and (vii) Technical Advancement indicative of a competency of the solution to the problem addressed by the author in the technical note.
[030] Context or Background: Sentences that provide background information or context of the invention are marked in this category. The reader gets to know about the area of invention and a general introduction to some of the important keywords of the related domain, e.g., the first sentence of FIG.4, indicates that the invention is related to warranty of products by manufacturing companies.
[031] Objective (OBJ): This class of sentences describes inventors focus of cardinal importance, i.e., what is the exact problem that the inventors solve through the invention in the technical note. For example, the second sentence of FIG.4 indicates that the key focus is to predict number of failures for every part of a product.
[032] Motivation (MOT): Every research is supposed to benefit the research or social community. The incentive to let the researcher devote their time and resources on their work are described in Motivation (MOT) section of the article. For example, the third sentence in FIG.4 is indicative of financial impact of the predictions being the key motivation of the invention.
[033] Current Situation and its Problems (CSP): Normally, various shortcomings and flaws of the current state-of-art are studied and an improvement/alternative approach is proposed. Sentences that describe the state-of-the-art approaches, and their shortcomings, are categorized in this class. For example, the fourth sentence in FIG.4 indicates that currently Weibull distribution on history data is used and it does not take into account the real condition of the product.
[034] New Observation (NO): In order to achieve the goal as given in ‘OBJ’, and solve the problems reported in ‘CSP’ inventors observe the process and the technology used very carefully and pick an observation that becomes the basis of their novel solutions. Sentences that present the details of such observations are categorized in the ‘NO’ category. For example, in FIG.4, inventors observe that data from service records can be used to take into account the operating condition of the product when trying to predict the number of part failures.

[035] Solution: Sentences that present an overview of the proposed solution are assumed to be in this category. For example, the second last sentence in FIG.4 proposes a solution to the problem described in ‘CSP’.
[036] Technical Advancement: The competency of the novel solution is usually portrayed quantitatively, e.g., measuring accuracy or computational efficiency. Such statements belong to the class Technical Advancement (TA). The last sentence in FIG.4 is indicative of the class TA.
[037] Organization (Org): Sentences of this type often occur in the research articles. For example, a paragraph indicating how the research paper is organized is typically provided at the end of the introduction paragraph. These statements are not necessary for assessing subject matter eligibility, but several other classes may be defined depending on the use case under consideration.
[038] In accordance with the present disclosure, certain traits or cues were used to identify the ingredients. For e.g. phrases like is called, was introduced, known as, etc. may be used to identify the class Objective. Phrases like is an important task, we need to, etc. may be used to identify the class Motivation. Phrases like are done using, currently is performed, has limitation, drawbacks of may be used to identify the class Current Situation and its Problems. Phrases such as we propose, we used, performed experiments may be used to identify the class Solution and phrases like outperform, experiments, show that, achieve better may be used to identify the class Technical Advancement. These words/phrases and their synonyms are the in-vocabulary words that play an important role in classifying the sentences. The OOV words are thus de-prioritized in the method 300 of the current disclosure.
[039] The steps of the method 300 will now be explained in detail with reference to the components of the system 100 of FIG.1 and the functional block diagrams of FIG.2A and FIG.2B. Although process steps, method steps, techniques or the like may be described in a sequential order, such processes, methods and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes

described herein may be performed in any order practical. Further, some steps may be performed simultaneously.
[040] In an embodiment of the present disclosure, the one or more processors 104 configured to employ a capsule based neural network comprising an (i) embedding layer, (ii) a feature extraction layer, (iii) a dilation layer, (iv) a primary capsule layer, (v) a convolutional capsule layer and a (vi) fully connected layer and a softmax layer. In accordance with the present disclosure, the one or more hardware processors are configured by the instructions stored in the one or more data storage devices or memory 102 operatively coupled to the one or more processors 104 for execution of steps of the method 300 by the one or more processors 104 as explained hereinafter.
[041] Accordingly, in an embodiment, the one or more hardware processors 104 are further configured to receive, at step 302, a sentence to be classified into the one or more pre-defined classes, wherein the sentence belongs to a technical note for a given domain and comprises a plurality of words [w1,w2,… .,wN], wherein one or more words from the plurality of words are the OOV words indicative of words that relate to the given domain and remaining words are the in-vocabulary words.
[042] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the embedding layer are configured to map, at step 304, each word from the plurality of words to a real valued fixed length embedding vector vi containing lexical and semantic representation of the plurality of words. The embedding layer is typically represented by a ‘weight matrix’ � ∈
is the vector dimension and |�| is the vocabulary size (number of unique words in a training dataset). Each column � of the weight matrix corresponds to a column vector word in the vocabulary.
[043] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the feature extraction layer are configured to encode, at step 306, one or more high-level features of the embedding vectors vi associated with each word from the plurality of words in the sentence into context vectors ci. The high-level features, in an embodiment, may be word count, average

word density, part-of-speech (POS), and the like. In an embodiment, low-level features may be the words themselves. The embedding vectors vi are processed and the high-level features are encoded into a vector ssen. The BiLSTMs are used for this and context vectors ci is obtained wherein context vectors
for a word wi, where are right and left contexts and dsen is
number of BiLSTM units. Finally for all the N words, N = [c1,c2, …cN] ∈ , where N is the maximum number of words fed to the BiLSTMs at each time step and dsem is the number of BiLSTMs used to extract a summary. Use of BiLSTMs in place of CNNs eschews the problem of choosing a suitable window size and reduces possible noise to capture smoothened context.
[044] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the dilation layer are configured to perform a dilation operation, at step 308, with an empirically defined dilation rate dr on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words.
[045] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the primary capsule layer are configured to perform convoluting, at step 310, of each of the context vectors having lower weights assigned to the OOV words, using a shared window (Wb) and the dilation rate dr to obtain corresponding parent capsules (pi), wherein the shared window is wherein � is a capsule dimension convoluting with the context vectors (ci), and dsen is number of BiLSTM units. The shared window (Wb) convolutes not only adjacent context vectors (ci,ci+1,…) but also with distant context vectors (ci+dr), skipping dr vectors in between.
[046] Consequently, in spite of using RNNs, which are known to be biased for extremes, in accordance with the present disclosure, the system 100 is able to focus on words between the sentence as shown in FIG.2B. Dilation refers to introducing holes in a matrix. As shown in FIG.2B, say once the embedding vectors

are passed through the BiLSTM, hidden vectors Hl,H2, ...Hm, where m is the number of words in a sentence are obtained. Hl,H2, ...Hm are convoluted using holes or gaps so that some vectors (corresponding to un-needed words, OOV words in this case) can be skipped. Suppose all the vectors are arranged in a matrix Hl, H2, - Hm and there is another matrix which has some holes in it, by using the matrix with holes, vectors corresponding to words 1, 3 and 5 can be convoluted (or mixed) vectors as shown in FIG.2B. Since dilation allows to skip certain words or phrases (which could be verbiage or noise), it is possible to attend to words or phrases which also are at a distance (of dilation rate) rather than only the adjacent words or phrases and skip noise. By skipping these noise, an additional step of removing stop words could be averted and the important words/phrases were learned automatically by the capsule based neural network. As a result of using the dilated BiLSTMs, the in-vocabulary words or phrases (like our objective, methodology includes etc., that are required for the classification task were learnt by the capsule based neural network.
[047] For each context vectors , the shared window with holes
convoluting with vectors in with stride of one to get a capsule
, where g is a non-linear squash function to shrink small vectors to around 0 and larger to around 1. Larger the size of capsules, larger is the probability of presence of instantiated parameters they represent and b is the bias vector. After the convolution operation, a capsule feature map stacked
with a total -dimensional capsules representing contextual capsules.
[048] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the convolutional capsule layer are configured to perform convoluting, at step 312, each of the parent capsules , to obtain child capsules, wherein each of the child capsules corresponds to at least one parent capsule ; and wherein a coupling strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm. By using a shared transformation matrix, the parent-child coupling may be represented by where is the child

capsule and Ws is shared weight between capsules i and j. Finally, the coupling strength is determined by the routing algorithm to produce a parent feature map.
[049] In an embodiment of the present disclosure, the one or more hardware processors 104 constituting the fully connected layer and the softmax layer are configured to compute, at step 314, a probability of the received sentence belonging to the one or more pre-defined classes based on the coupling strength between the parent capsules and the child capsules. The fully connected layer or a flattening layer flattens the child capsules into a list of capsules which is then multiplied by a transformation matrix WFC followed by the routing algorithm to compute the probability.
EXPERIMENTS AND OBSERVATIONS
[050] Dataset: A new dataset comprising approximately 9000+ sentences from about 400+ research papers has been created for future research. These sentences were annotated to indicate the pre-defined class it belongs to. The dataset was then divided in the ratio 90-10-10 for training, development and testing respectively.
[051] To compare the performance of the system and method of the present disclosure on the different types of sentences, following publicly available datasets were used:
a) 20 Newsgroups: The 20 newsgroups dataset is a collection of newsgroup
documents. The bydate variant of the dataset consisting of classes comp, politics,
rec, and religion with standard split was used. The 20Newsgroups dataset was
chosen for testing performance of dilation operation on BiLSTMs to capture long
term dependencies in large documents.
b) AG’s News: AG’s News is also a collection of news articles consisting of 108K training sentences, 12K development sentences and 7.6K testing sentences. The average length of sentences in AG’s News corpus is almost same as the length of sentences in the dataset created as part of this disclosure by the Applicant.
c) TREC: TREC dataset was introduced for question categorization task with 6 labels, 5.4K training, 0.5K development and 0.5K testing sentences. The average

length of statements in TREC is 10 words which makes it a suitable candidate for evaluating performance of the system and method of the present disclosure on short texts.
[052] Training details: For training the model of the present disclosure, BiLSTMs were used for capturing semantic relationship between the text components. During hyperparameter tuning, the number of LSTM units, dsen were varied between {128-512} with a step size of 64. Number of capsules, C varied between {16-20} with their dimension d varied between {16-20}, routings, r were tested within range {3-7}. The maximum input sentence length was kept to 100 and word embedding of dimension 300 were obtained from Glove by J. Penning et al. in the Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). Dropout values were adjusted as described by N.Srivasta et al. in The Journal of Machine Learning Research. All the values were tuned on the development sentences.
[053] Experiments: Baseline Methods: For the performance comparison of the system and method of the present disclosure, some of the baseline models that were implemented and tested include Tf-idf, vanilla CNNs, Hybrid Channel CNNs, vanilla BiLSTMs, LSTM with skip connections, RCNN, CNN-LSTM, Feed Forward Attention Networks and Capsules with CNN. Classification accuracy has been used as a metric for evaluation. A comparison of the model of the present invention with the baseline approaches is proved in Table I below. Table I

Models Test Accuracy
Tf-idf 47.6
CNN 62.42
Hybrid Channel CNN 66.69
BiLSTM 73.47
LSTM with skip connections 71.57
RCNN 74.28
CNN-LSTM 77.6

Feed Forward Attention Networks 74.57
Capsules with CNN 71.3
Model of the present disclosure 80.0
[054] From the Table I it may be noted that the model of the present disclosure enabled by the system 100 and method 300 described herein above outperforms all the other models by a good margin beating the second best model CNN-LSTM by approximately 3%. It may also be noted that CNNs are outperformed by all models using LSTMs as a feature extractor corroborating the fact that LSTMs are able to handle long sentences. Furthermore, performing dilation in accordance with the present disclosure helped capture long term dependencies from the input sentence leading to an improvement of about 3% when compared to the second best model.
[055] Error analysis on the dataset of the present disclosure: A few misclassified samples from the testing sentences of each class were selected for analyzing the model weights assigned to each word. To understand the model weights assigned to input words, LIME provided by M.T.Ribeiro et al. in the Proceedings of the 22nd AC SIGKDD international conference on knowledge discovery and data mining was used on each of the misclassified sentences to get a better understanding. LIME uses a local interpretable model to approximate the model in question and tries to create certain explanations of input data by performing some perturbations on input data to understand the relationship between input and output data. The output of LIME may be interpreted as weights assigned to the words where positive weights are colored in green and words with negative weights in red. LIME also provides an explanation for probabilities assigned to each class based on the weights assigned to each word. It provides an explanation for each class by assigning the positive weights (green) to the words which play a major role in assigning higher probabilities to the current class. For example, as shown below, the same sentence is explained twice to highlight the words with higher (green) and lower (red) weights for two classes (correct and predicted). For the sake of explanation, the green text is represented by bold text and the red text is represented by underlined text. However, it may be noted that using LIME, there

will be different shades of green and red texts displayed depending on the weights assigned which is not possible to be illustrated in the patent specification.
[056] y = Current Situation (probability 0.664) problem with this pre-determined number of personas and predetermined persona characteristics is that these vary depending on domain/geography/size of the enterprise and require domain expertise (subject matter experts-sme) to formulate the persona characteristics and persona numbers generalization is not easy across domain/geography/size etc our focus in this work is to learn from users profile, usage characteristics, style of work, behavior, etc. and arrive at the best persona definitions and optimum number of personas to be cost efficient.
[057] y = Objective (probability 0.034) problem with this pre-determined number of personas and predetermined persona characteristics is that these vary depending on domain/geography/size of the enterprise and require domain expertise (subject matter experts-sme) to formulate the persona characteristics and persona numbers generalization is not easy across domain/geography/size etc our focus in this work is to learn from users profile, usage characteristics, style of work, behavior, etc. and arrive at the best persona definitions and optimum number of personas to be cost efficient.
[058] It may be noted from the above observations that Objective was misclassified as Current Situation. After analysis, it was noted that the sentence contains both information, i.e. about the problem associated with the current situation (hence the positive weights for words like problem, not, etc.) and the objective of the authors (hence the positive weights for words like our, focus, etc.).
[059] y = Current Situation (probability 0.028) in our earlier work[10], we proposed an indirect approach of estimating bp via the r and c parameters of 2-element windkesel model using ppg features.
[060] y = Solution (probability 0.330) in our earlier work[10], we proposed an indirect approach of estimating bp via the r and c parameters of 2-element windkesel model using ppg features.
[061] It may be noted from the above observations that Current Situation was misclassified as Solution. It may be clearly seen that the statement describes a

solution related to the author’s previous proposed solution. The author has clearly described a solution but since it has been proposed earlier, it was categorized as Current Situation.
[062] Based on several misclassified sentences, it was noted that the model of the present disclosure gets confused when there is a presence of multiple sources of information pertaining to different classes. However, a comparison of the performance of the model of the present disclosure with publicly available datasets like 20 Newsgroups, AG’s News and TREC dataset shows that the model of the present disclosure achieves state-of-the-art results on two of the published results. It also achieves a competitive score on TREC dataset where numbers are reported in one of their CNN non-static implementation. Table II below shows performance on publicly available datasets. Table II:

Dataset Reported Test Accuracy Model of the present disclosure
20 Newsgroups 96.69 96.74
AG’s News 92.6 93.4
TREC 93.6 92.8
[063] Thus, in the present disclosure, the Applicant provides a capsule based neural network architecture with dilated BiLSTM to automatically classify statements into one of the pre-defined classes (ingredients) that may be further used for various applications. In one embodiment, the pre-defined classes explained here relate to subject matter eligibility assessment. The efficacy of the capsules with dilated BiLSTM layers on the dataset created by the Applicant has also been illustrated along with that on three publicly available datasets.
[064] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do

not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
[065] It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
[066] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[067] The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily

defined herein for the convenience of the description. Alternative boundaries can
be defined so long as the specified functions and relationships thereof are
appropriately performed. Alternatives (including equivalents, extensions,
variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
[068] Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
[069] It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.

We Claim:
1. A processor implemented method (300) comprising the steps of:
receiving, via one or more hardware processors configured to employ a capsule based neural network, a sentence to be classified into one or more pre-defined classes, wherein the sentence belongs to a technical note for a given domain and comprises a plurality of words, wherein one or more words from the plurality of words are Out-of-Vocabulary (OOV) words indicative of words that relate to the given domain and remaining words are in-vocabulary words (302);
mapping, by an embedding layer in the capsule based neural network, each word from the plurality of words to a real valued fixed length embedding vector containing lexical and semantic representation of the plurality of words (304);
encoding, by a feature extraction layer in the capsule based neural network, using Bidirectional Long Short-Term Memory (BiLSTM), one or more high-level features of the embedding vector associated with each word from the plurality of words in the sentence into context vectors (306);
performing, a dilation operation, by a dilation layer in the capsule based neural network, with an empirically defined dilation rate on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words (308);
convoluting, by a primary capsule layer in the capsule based neural network, each of the context vectors having lower weights assigned to the OOV words, using a shared window (Wb) and the dilation rate dr to obtain corresponding parent capsules, wherein the shared window is Wb ∈

wherein d is a capsule dimension convoluting with the context vectors and dsen is number of BiLSTM units (310);
convoluting, by a convolutional capsule layer in the capsule based neural network, each of the parent capsules (pi) to obtain child capsules, wherein each of the child capsules corresponds to at least one parent capsule (pi); and wherein a coupling strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm (312); and
computing, a probability of the received sentence belonging to the one or more pre-defined classes using a fully connected layer and a softmax layer in the capsule based neural network based on the coupling strength between the parent capsules and the child capsules (314).
2. The processor implemented method of claim 1, wherein the one or more pre-defined classes include (i) Context indicative of background information of the technical note; (ii) Objective indicative of a problem addressed by an author in the technical note; (iii) Motivation indicative of an incentive for the author of the technical note to work on the problem addressed; (iv) Current Situation indicative of state-of-the-art approaches to solve the problem addressed; (v) New Observation indicative of a basis for a solution to the problem addressed by the author in the technical note; (vi) Solution indicative of a solution to the problem addressed by the author in the technical note; and (vii) Technical Advancement indicative of a competency of the solution to the problem addressed by the author in the technical note.
3. A system (100) comprising:
one or more data storage devices (102) operatively coupled to one or more hardware processors (104) and configured to store instructions

configured for execution via the one or more hardware processors, wherein the one or more hardware processors are configured to employ a capsule based neural network and perform:
receiving a sentence to be classified into one or more pre-defined classes, wherein the sentence belongs to a technical note for a given domain and comprises a plurality of words, wherein one or more words from the plurality of words are Out-of-Vocabulary (OO V) words indicative of words that relate to the given domain and remaining words are in-vocabulary words;
mapping, by an embedding layer in the capsule based neural network, each word from the plurality of words to a real valued fixed length embedding vector containing lexical and semantic representation of the plurality of words;
encoding, by a feature extraction layer in the capsule based neural network, using Bidirectional Long Short-Term Memory (BiLSTM), one or more high-level features of the embedding vector associated with each word from the plurality of words in the sentence into context vectors;
performing, a dilation operation, by a dilation layer in the capsule based neural network, with an empirically defined dilation rate on each of the context vectors to lower weights associated with the context vectors corresponding to the OOV words identified by back propagation, wherein the lower weights are in comparison to weights assigned by the capsule based neural network to the in-vocabulary words in the plurality of words;
convoluting, by a primary capsule layer in the capsule based neural network, each of the context vectors having lower weights assigned to the OOV words, using a shared window (Wb) and the dilation rate dr to obtain corresponding parent capsules, wherein the shared window is Wb ∈ wherein � is a capsule dimension convoluting with the context vectors and dsen is number of BiLSTM units;
convoluting, by a convolutional capsule layer in the capsule based neural network, each of the parent capsules (pi) to obtain child capsules,

wherein each of the child capsules corresponds to at least one parent capsule (pi); and wherein a coupling strength between a parent capsule from the primary capsule layer and a child capsule from the convolutional capsule layer is determined by a routing algorithm; and
computing, a probability of the received sentence belonging to the one or more pre-defined classes using a fully connected layer and a softmax layer in the capsule based neural network based on the coupling strength between the parent capsules and the child capsules.
4. The system of claim 3, wherein the one or more pre-defined classes include
(i) Context indicative of background information of the technical note; (ii) Objective indicative of a problem addressed by an author in the technical note; (iii) Motivation indicative of an incentive for the author of the technical note to work on the problem addressed; (iv) Current Situation indicative of state-of-the-art approaches to solve the problem addressed; (v) New Observation indicative of a basis for a solution to the problem addressed by the author in the technical note; (vi) Solution indicative of a solution to the problem addressed by the author in the technical note; and (vii) Technical Advancement indicative of a competency of the solution to the problem addressed by the author in the technical note.