DESCRIPTION

TITLE

Natural Language Understanding System

FIELD OF INVENTION

This invention relates to natural language understanding and interpretation by a computer system. It relates in general to tools and methods for computational linguistics. In particular, the present invention relates to a framework and system to achieve natural language understanding in machines.

BACKGROUND

Natural language understanding (NLU) applications utilise computing machinery to produce actionable information by processing source texts written in natural language. Typically, an NLU application would process text, written in a natural language, and in conjunction with a stored dataset of domain knowledge, generate actionable information. Few applications of general NLU systems include machine translation, question answering, and automated summarization. Domain-specific examples of NLU applications include medical diagnosis systems, quantitative trading algorithms, and web search, amongst others.

One early attempt at building an NLU application in the broad domain of common sense reasoning was undertaken by the CYC project (Lenat et al, 1989). The goal of the CYC project was to construct a knowledge base of common sense facts that would enable an NLU system to parse, as the source text a typical desk encyclopaedia, into actionable knowledge. The CYC project hasn’t met its stated goals. Many recent techniques and approaches for implementing NLU systems, focus on either restricting the domain of the problem space or utilizing automatic or statistical means to derive asserted or non-asserted relations. However, in these conventional techniques, the actionable information produced by such systems is significantly lacking in accuracy and completeness, compared to information capable of being produced by human processing.

One approach to implementing practical NLU applications is to restrict the domain of the problem. This may involve applying restrictions in the scope of the source text or of the output in order to simplify the types of information that are produced and processing techniques required. For example, U.S. Pat. No. 5,721,938, entitled "Method and Device for Parsing and Analyzing Natural Language Sentences and Text", teaches a method for parsing natural language source texts that categorizes words as either noun or verb units. The method is designed for the domain of grammar checker applications, and is not suitable for implementation of other broader NLU applications.

Another approach to implementing practical NLU applications relies on generating output information that is short of full understanding by employing approximate methods. For example, a conventional system for translating a source text into another natural language that generates the literal translation of the source text will commonly produce resultant translations that are erroneous or approximate.

Most of the NLU systems utilize statistical methods to approximate understanding of the source text when complete understanding is not achievable. For example, U.S. Pat. No. 5,752,052, entitled "Method and System for Bootstrapping Statistical Processing into a Rule-based Natural Language Parser", discloses a method of modifying a rule-based natural language parser using summary statistics generated from a source text. The summary statistics are compiled from a corpus of text that is similar in syntactic properties to the source text in order to estimate the likelihoods that candidate rules should be applied. Using these statistics to implement a rule-based parser thereby results in output that can be erroneous or approximate.

Neural network-based Natural Language Processing/Understanding systems are being extensively discussed. These systems generally convert the input text into a vector domain and use the concept of Transformers and attention networks to improve the contextual understanding of the input. These are purely statistical methods and lack the capacity to understand the input in the context as elaborated below. Few of the popular offerings in this domain are BERT, GPT-3, Roberta. These systems have a wide application and coverage. However they are unable to perform basic mathematical, relational and temporal analysis on the input text. These systems lack reasoning capabilities, are unable to perform cause and effect analysis, lack explain-ability, partial re-teachability, and require large training datasets. Data dependence inadvertently brings along hidden biases in such systems.

The Natural Language Processing/Understanding Technologies have made focused progress. However there remains a huge gap between the way a machine and a human process natural language. This gap has put a limitation on applicability of available Natural Language Processing/Understanding technologies to common problems which require natural language understanding. Some of the cases where current technologies fall short are enumerated below: -
• Specific domain level expertise (like finance or medical) is required to understand the natural language inputs.
• Large training datasets are not available which is a prerequisite for all statistical systems.
• Explain-ability is a prerequisite in critical, high stake and high-risk applications like finance, disaster recovery and other sensitive decisions. A subject matter expert should be able to look under the hood and decipher the reasons for a particular system decision.
• Application of common sense (or general knowledge) and reasoning is expected from systems. However, it falls short of expectations.
• Mathematical reasoning and capabilities are essential capabilities.
• Identification of complex relation between entities are important to achieve understanding as also carrying forward the context across a passage.
• Assigning cause and effect based on common worldly knowledge or power of reasoning are critical features expected from a text processing system.

Therefore, what is desired is a general-purpose, accurate, and complete method for natural language understanding that is capable of delivering actionable information that is suitable to be used in a broad range of applications.

References:-
U.S. Patent Documents
Patent Number Date Inventor
5721938
February 1998 Stuckey
5752052
May 1998 Richardson
9110883
August 2015 Ghannam
2003/0233224
December 2003 Marchisio
2004/0220796
November 2004 Parkinson
2006/0217963
September 2006 Masuichi
2007/0016398
January 2007 Buchholz
10,503,769
December 2019 Ghannam
9934465
April 2018 Hunt
10474647
Nov 2019 Sweeney
5794050
Aug 1998 Kathleen Dahlgren
7840400
November 2010 Ofer Lavi

OBJECTIVE

It has been proposed above that there is a pressing need to have a general-purpose, accurate, and complete method for natural language understanding capable of delivering actionable information that is suitable to be used in a broad range of information applications. The current statistical and rule-based systems fall short of the desired results as has been discussed in the background section.

The principal object of this invention relates to natural language understanding and interpretation by a computer system. AUI Systems has developed a solution framework based on proprietary technology which provides meaning interpretation and a robust universal knowledge system. It combines linguistics, general and domain knowledge, common sense and reasoning through subsystems to achieve Natural Language Understanding in machines.
Before we elaborate the subsystems forming the framework, it would be prudent to qualify ‘Understanding’ in machines. Information input is a necessary though not sufficient ingredient to claim ‘Understanding’. Information can be received from various sources. However for the sake of this claim we shall restrict to natural language inputs (language as used by humans to communicate with each other).
It may be claimed that the receiver of information understood a natural language input if the following is achieved: -
• Receiver identifies the participants (Nouns or agents), qualifications (conditions or constraints) and actions(verbs) performed by agents.
• The cause, effect and consequence of all the actions, being performed in the natural language input information, are mapped.
• A coherent story (sequence of actions and corresponding state change of participant entities) of the input information is mapped.

Let us examine the usage of some critical terminologies pertinent to this claim.

1. Entity. An entity is any physical, non-physical, tangible, or non-tangible concept. Entities are the primary participants in information sharing.
2. Property. Each entity can have many properties associated with it. These properties define some aspect of the entity. They can be simple attributes or its relationship with other entities.
3. Entity-State: It is the set of all the properties an entity has at a given point of time.
4. Activity. An activity is the action which an entity or multiple entities perform.
5. Activity-Role. These are important participants involved in an Activity, state of that Activity or the relation of the Activity to other Entities/Activities.
6. Role-Performing-Entity/ Role-Performing-Activity. This is the Entity/Activity which will be taking a particular Activity-Role in an activity.
7. Activity-State. Activity-state is the set of all roles associated with the activity at a given point in time.

A system may be qualified to have understood a natural language input when it achieves the following abilities: -
1. Uniquely identify all the entities in the input.
2. Establishes the state of all entities.
3. Identifies the activities performed by the entities.
4. Derives the change of state in the entities due to the activities performed by the entities.
5. Establishes causes of the activities being performed and also the subsequent effects.
6. Performs Entity State Tracking by transferring the state of entities from one activity to other in a time sequence.
7. Builds a comprehensive, logical and Continuous narrative based on the input.
8. Uses the above listed capabilities in all inputs and reasoning tasks.

Natural Language Understanding System

The fundamental approach used in this invention is that the understanding is a process of mapping the previously unknown input (natural language Input) onto a known scalable model by using the previously mapped knowledge of the Known-world template. Each word in the input adds certain meaning or context to the overall Understanding.

As the principal objective of the invention, AUI has developed a proprietary framework consisting of the following subsystems to achieve understanding in a computer systems.
• Knowledge Base
• NLU Engine
Please refer Fig 1 for block diagram.

Knowledge Base. Another object of the invention is to model, capture and make available the worldly knowledge, also referred to as common or general knowledge, to the system to achieve understanding. Please refer Fig 2 for block diagram.

The Knowledge base is designed for the following: -
• Capture knowledge to process a particular Language construct.
• Capture meaning of Entities, Actions and Properties.
• Capture the meaning of Activities and build templates resembling real world.
• Help convert Natural Language Input into Semantic Data output.
• Help perform inference on the semantic data to derive entity state and activity state.
• To be universal and not limited to a language or domain.
• Unique identification of worldly concepts and activities.
• Flexible and extendable.
• Domain independent knowledge ingestion.
• Capable of capturing common sense knowledge.

The knowledge base has three fundamental components: -
• Symbol Index. This index stores all the symbols that are used in the system. This also gives a unique identity to every symbol. The Symbol Index adds certain basic meaning to each symbol and helps in converting the tokens (generated by the system in the first step) to symbols. Please refer Annexure 1.
• Taxonomy. This is the relationship which classifies all the symbols into an interlinked hierarchy. Each node in the Taxonomy groups a set of symbols into a homogenous group which share one or more common properties. The Taxonomy has been designed to be flexible and provide multiple classifications to the same symbol when required. The Taxonomy servers as the basic tool for generalization of knowledge. Please refer Annexure 2.
• Meaning Store. This captures all the rules and facts that are required to perform natural language understanding. AUI has developed specific knowledge rules required for each specific task. Knowledge Rules have specific pattern which helps the system to perform Entity Identification, Relation Identification, Activity Identification and Infer the Entity and Activity State. Each of these can be extended by adding more Knowledge rules to cover wider input. Hierarchical Taxonomy is used to compress the number of rules required in each case. Please refer to the Generalization and Knowledge Compression explanation in the Annexure 12. These rules are as follows: -
o The Entity-State Extraction Rules. These rules help to identify the entity-statewhen an Entity has a property qualification by an adjective or combined with other entities directly or through prepositions. These help to build complex Entities from simple entities by building the Entity relations with other entities or attributes.
? Adjective-Noun rules. These rules define the entity-state of an entity when it is qualified by an adjective. Please refer Annexure 3.
? Noun-Noun rules. These rules define the entity relation of entities when two Nouns are used one after the other in a sentence. Please refer Annexure 4.
? Noun-Preposition-Noun rules. These rules define the Entity relation of the entities when two Nouns are used with a preposition in between. Please refer Annexure 5.
o Activity-State Extraction Rules: - These rules are used to build the Activities that are performed in a sentence. These rules will help identify the unique action performed and all the roles of the different entities that are involved in the activity. They also help to build the relation between multiple Activities that are initiated in the input.
? Verb-Subject-Object rules. These rules uniquely define the entities that are used along with a verb and help to infer the Entity-State and Activity-State of all the related Entities once this verb is performed. Please refer Annexure 6.
? Activity-Preposition/Conjunction-Activity rules. When multiple verbs are used in a sentence, there is a unique way these verbs influence each other. These rules capture the relation between the extracted activities and their parent verbs and the inter-relation between various activities. Please refer Annexure 7.
? Adverb-Verb rules. The Adverb Verb rules captures the change in the activity-state and entity-state when an adverb interacts with a verb in a sentence. Please refer Annexure 8.
o Entity-State-Inferencing rules. These rule captures the effect of one entity-state on to the other entities and their state(s). It also identifies the rules for transferring the Entity-State from one sentence to another. Please refer Annexure 9.
o Activity-State-Inferencing rules. The information of all the roles in an activity generally does not come in a single sentence. These rules specify the methodology to complete the activity-state by inferencing the roles from other preceding sentences and from the associated entity state. Please refer Annexure 10.
o Commonsense Rules. These are the most high level rules which help to derive a high level inference about a set of activities and entities in a given context. Please refer Annexure 11.

NLU Engine

Another object of the invention is to create a NLU Engine in general and create English NLU Engine in particular. The NLU Engine performs of a series of transformation operations on the Natural Language Input (NLI) to finally produce an output (Semantic data) which represents the Understanding of the input. These set of operations use the Knowledge Base which is described in the previous section.
The following paragraph is used to explain the NLU Engine working. The example sentence has been selected to highlight all the different processes that are involved: -

Aman withdrew 500 dollars from the bank. He went and bought a box of chocolates for 100 dollars from the gift shop to celebrate Anya’s success. She had been adjudged as the student of the year unanimously. He called and congratulated her.

The NLU Engine performs the following series of Operations on the NLI to achieve the Understanding. Please refer Fig 3 for block diagram.

• Tokenization. The process of tokenization involves breaking the input text into individual sentences and then breaking the individual sentences into tokens. A token will be sequence of characters. A token can be an individual character, number, word, or punctuation mark. This can be achieved with any statistical modelling or by using a set of rules.
o Output of Sentence Tokenization. The input paragraph is divided into the following three sentences: -
? Aman withdrew 500 dollars from the bank.
? He went and bought a box of chocolates for 100 dollars from the gift shop to celebrate Anya’s success.
? She had been adjudged as the student of the year unanimously.
? He called and congratulated her

o Output of Word Tokenisation. For the sake of clarity, each token is placed in a cell of the table.

? Sentence 1.
Aman Withdrew 500 Dollars From The bank

? Sentence 2
He Went And Bought A Box
Of chocolates For 100 Dollars From
The gift Shop To Celebrate Anya
‘ S success

? Sentence 3
She Had been adjudged as the
Student Of The year unanimously

? Sentence 4
He Called And Congratulated her

• Symbolization. The process of Symbolization involves mapping an individual token or a group of consecutive tokens to one or more than one Symbols. These symbols have unique identity and are predefined and created in the knowledge-world template. This process also involves converting numbers, dates and other pattern-based tokens like SSN number, Pin code to symbols. This process uses the Symbol Index and the Taxonomy to add the first level of Semantic meaning to the input. The output of this process is as shown below. The number/s show the unique symbol ID of each/ group of tokens: -

? Sentence 1.
Aman [2] Withdrew: [4] 500
[36] Dollars
[5] From
[6] The
[7] Bank
[8,9]
PS: the number 500 identified as a number and tagged as [36]

? Sentence 2
He
[10] Went
[12] And
[13] Bought
[15] A
[16] Box
[17]
Of
[18] Chocolates
[20] For
[21] 100
[36] Dollars
[5] From
[6]
The
[7] gift
[23,42] Shop
[22] To
[24] Celebrate
[25] Success
[27]
Of
[18] Anya
[26]
PS: ‘Anya’s success’ has been transformed to ‘success of Anya’

? Sentence 3
She
[28] Had
[30] Been
[31] Adjudged
[33] As
[34] The
[7]
Student of the year
[35] Unanimously
[43]
PS: More than one token can be mapped to a single symbol

? Sentence 4
He
[10] called
[38] and
[13] congratulated
[41] her
[39]

• Symbol Sense Disambiguation. The process of Symbol Sense Disambiguation involves removing multiple symbols which may have been assigned to a token or a sequence of tokens in the previous step. The is carried out in two steps: -
o Syntactic Symbol Sense Disambiguation. A set of syntactic Symbol Sense Disambiguation rules have been built which have been formulated by analyzing the Language grammar rules. This step basically helps in removing multiple senses that have a different Part of Speech (PoS).
? Sentence 2:
He
[10] Went
[12] And
[13] Bought
[15] A
[16] Box
[17]
Of
[18] Chocolates
[20] For
[21] 100
[36] Dollars
[5] From
[6]
The
[7] gift
[23, 42] Shop
[22] To
[24] Celebrate
[25] Success
[27]
Of
[18] Anya
[26]
PS: The Token Gift had two symbols 23 and 42. By using the Syntactic rules, the Verb symbol (23) has been removed by using the rule ‘Article Verb/noun’ implies ‘article noun’

o Semantic Symbol Sense Disambiguation. This uses the Entity-State Extraction Rules and the Verb-Subject-Object rules to identify the correct Symbol which is appropriate in the current context and perform disambiguation.
? Sentence 1.
Aman [2] Withdrew: [4] 500
[36] Dollars
[5] From
[6] The
[7] Bank
[8,9]
PS: The token Bank had two symbols 8 and 9. By using the Verb-Subject-Object rule ‘Human withdraw money from financial institutions’, the symbol 9 (verb) has been removed

• Creation of Unambiguous Sentence. An Unambiguous Sentence is a sentence in which each word or set of words will have only one symbol. For every word or word sequence with multiple symbols associated, a separate Unambiguous Sentence is created. Then only one unambiguous sentence is selected for further processing. In the above examples, all three sentences will have only one unique symbol sentences. Hence four Unambiguous sentences will be formed as below: -
? Sentence 1.
Aman [2] Withdrew: [4] 500
[36] Dollars
[5] From
[6] The
[7] Bank
[8]

? Sentence 2
He
[10] Went
[12] And
[13] Bought
[15] A
[16] Box
[17]
Of
[18] Chocolates
[20] For
[21] 100
[36] Dollars
[5] From
[6]
The
[7] Gift
[42] Shop
[22] To
[24] Celebrate
[25] Success
[27]
Of
[18] Anya
[26]

? Sentence 3
She
[28] Had
[30] Been
[31] Adjudged
[33] As
[34] The
[7]
Student of the year
[35] Unanimously
[43]

? Sentence 4
He
[10] called
[38] and
[13] congratulated
[41] her
[39]

• Identification of Entities and Logical Symbol Groups. Each of the Noun type symbols are converted into entities. Further the Entity-State Extraction Rules are applied to these entities to build the relation between the entities. The Logical Symbol Groups consists of logically grouped Entities by applying the Entity-State Extraction Rules. The application of Entity-State Extraction Rules also results in identification of the main entity in a set of symbols that are acting like one complex logical entity and the remaining entities will have property relation built with the Main entity. The Logical Symbol Group will also include the associated Preposition and Negation if any. The output after this step has been performed is as below: -

? Sentence 1.
Aman Withdrew Dollars
• Has value=500 From The Bank
PS:
1. Each cell is showing one Logical Symbol Group. The symbols ‘from’, ‘the’ and ‘bank’ has been combined to form one Logical Symbol Group.
2. ‘500 dollars’ has been applied the Adjective-Noun rule ‘NUMBER NOUN’ implies ‘NOUN has value = NUMBER’ which results in ‘dollar has value=500’

? Sentence 2
He Went And Bought Box
• Has content=chocolates
For Dollars
• Has value=100 From the gift shop
Shop
• Has offering=gift to
Celebrate Success
• Is of-entity=Anya
PS:
1. ‘box of chocolates’ has been applied the Noun-Preposition-Noun rule ‘CONTAINER of NOUN’ which implies ‘CONTAINER has content = NOUN’ which results in Box has content=chocolates’
2. Similar transformation has been applied on ‘success of Anya’ and ‘100 dollars’
3. ‘Gift shop’ has been applied with the Noun-Noun Rule and results in the relation ‘Shop has offering=gift’
? Sentence 3
She Had Been Adjudged
as the Student of the year Unanimously
PS: The Logical Symbol Group has been formed with ‘as’, ‘the’ and ‘Student of the year’.

? Sentence 4
He called and congratulated her

• Creation of Simple Sentence. A Sentence may have one or more verbs. A simple sentence is a sentence with one verb and its associated subjects and objects. The Engine divides the Full sentence (also called as Composite sentence) into simple sentences. It also builds edges between the simple sentences which captures the linking Preposition/ conjunction. In the above example only the second sentence has two verbs. The first and the third sentence goes unchanged. The second sentence gets split into simple sentences. The details of simple sentences formed along with the simple sentence ID is as follows: -
? Sentence 1. (S1-SS1)
Aman Withdrew Dollars
• Has value=500 From The Bank
PS: No Breaking of the sentence as only one verb is present

? Sentence 2 gets broken into three simple sentences as below: -
? Sentence 2 Simple sentence 1 (S2-SS1)
He Went

? The edge between the two simple sentences.
• S2-SS1-> and-> S2-SS2

? Sentence 2 Simple sentence 2 (S2-SS2)
Bought Box
• Has content=chocolates for Dollars
• Has value=100
From the gift shop
Shop
• Has offering=gift

? The edge between the two simple sentences.
• S2-SS2-> to-> S2-SS3

? Sentence 2 Simple sentence 3 (S2-SS3)
Celebrate Success
• Is of-entity=Anya

? Sentence 3 (S3-SS1)
She Had Been Adjudged
As the Student of the year Unanimously

? Sentence 4 gets broken into two simple sentences

? Sentence 4 Simple sentence 1(S4-SS1)
He called

? The edge between the two simple sentences.
• S4-SS1-> and-> S4-SS2

? Sentence 4 Simple sentence 2(S4-SS2)
congratulated her

• Processing of Actions. The verb in each of the simple sentences is processed using the Verb-Subject-Object rules. The following set of actions are performed in this step: -
o Subject Transfer. Some of the simple sentences may not have a subject. This step identifies a suitable subject in the preceding simple sentence of the same composite sentence and transfers it to the current simple sentence. In the example, the two simple sentence S2-SS2 and S2-SS3 does not have any subjects. The Engine applies the Verb-Subject-Object rules to identify suitable subjects for these to Simple sentences.

? Sentence 2 Simple sentence 2 (S2-SS2)
He Bought Box
• Has content=chocolates for Dollars
• Has value=100
From the shop
PS: Subject transfer from S2SS1

? Sentence 2 Simple sentence 3 (S2-SS3)
He Celebrate Success
• Is of-entity=Anya
PS: Subject transfer from S2SS1

? Sentence 4 Simple sentence 2(S4-SS2)
He congratulated her

o Passive Sentence to active sentence conversion. If the simple sentence is in a passive form, the sentence is converted to an active form by transferring the subject to a suitable object position by inserting the required preposition. This step uses the passive sentence identification rules and the Verb-Subject-Object rules to transfer the subject to the object. Sentence 3 here is in the passive form. This sentence will be transformed to an Active form as follows: -

? Sentence 3 (S3-SS1)
Unknown Subject Adjudged She (her) As the Student of the year Unanimously
PS: This transformation also uses the Verb-Subject-Object rule ‘Human/organization adjudge human/organization as title’. The subject of the passive sentence gets transformed as the primary object of the verb

o Object Transfer. Some of the simple sentences will not have objects. This step is required because some objects are combined with multiple verbs in a sentence or placed at a different location in the composite sentence. The object transfer also uses the knowledge in the Verb-Subject-Object rules to perform the transfer of only suitable objects.

? Sentence 4 Simple sentence 1(S4-SS1)
He called her

o Activity and Entity State Inferencing. By the application of the Verb-Subject-Object rules, the following inferencing is performed: -
? Identification of Activity
? Identification of Entities which takes a suitable role in the activity.
? Inferencing the Entity state.
Each of the Simple sentence above is applied the corresponding Verb-Subject-Object rules. This results in creation of the following Activities and Entities state:-

Simple Sentence Activities Entity State
S1-SS1 Withdraw
• Performer : Aman
• Amount: 500 Dollars
• Source: Bank Aman
• Has possession=500 dollars
Bank
• Had possession=500 dollars
S2-SS1 Change-Of-Location
• Performer: He
S2-SS2 Sell-Item
• Buyer: He
• Item: Box of chocolates
• Cost: 100 dollar
• Location: Gift shop He
• Had possession=100 dollars
• Has possession= box of chocolates
Shop
• Has offering=gift
S2-SS3 Celebration
• Performer: He
• Reason: Success
S3-SS1 Facilitation
• Awardee: She
• Title: Student of the year
S4-SS1 Communicate
• Performer: He
• Performee: Her
S4-SS2 Congratulate
• Performer: He
• Performee: Her
PS: The Role Performee basically means the Primary Object on which the action is performed

o Adverb Processing. The Adverbs primarily effect the way an action is performed. Hence, these adverbs result in either adding a role in the activity or sometimes changes the entity-state of the involved entities. In the example above, the Sentence 3 has one adverb. By applying the Adverb-Verb rule, the engine adds the following role to the identified activity: -

S3-SS1 Facilitation
• Awardee: Anya
• Title: Student of the year
• Decision-type: unanimously

o Pronoun Replacement. The pronouns in the simple sentences are also replaced with the suitable entities from the Entities in the same sentences/ entities which were created in the previous sentences. The Pronoun replacement logic also uses the Verb-Subject-Object rules for identification of the correct pronouns. The State of each of the Activities and Entities after pronoun replacement change as follows: -

Simple Sentence Activities Entity State
S1-SS1 Withdraw
• Performer: Aman
• Amount: 500 Dollars
• Source: Bank Aman
• Has possession=500 dollars
Bank
• Had possession=500 dollars
S2-SS1 Change-Of-Location
• Performer: Aman
S2-SS2 Buy-Item
• Buyer: Aman
• Item: Box of chocolates
• Cost: 100 dollars
• Location: Gift Shop Aman
• Had possession=100 dollars
• Has possession= box of chocolates
S2-SS3 Celebration
• Performer: Aman
• Reason: Success
S3-SS1 Facilitation
• Awardee: Anya
• Title: Student of the year
• Decision-type: unanimously
S4-SS1 Communicate
• Performer: Aman
• Performee: Anya
S4-SS2 Congratulate
• Performer: Aman
• Performee: Anya

• Creation of Relation between Activities. Once all the simple sentences of a composite sentence are processed, the edge information available between the simple sentences are used to build the relation between the Activities created in each sentence by applying the Activity-Preposition/Conjunction-Activity rules.
In the example, S2-SS2 results in creation of the Activity Buy-Item and S2-SS3 results in creation of the activity Celebration. These two Simple sentences are connected by the edge S2SS2->to->S2SS3. The Activity-Preposition/Conjunction-Activity rule ‘ACTIVITY to ACTIVITY’ which results in the inference of the Role =Reason in the activity Buy-Item. The final Activity will look as follows: -

S2-SS2 Buy-Item
• Buyer: Aman
• Item: Box of chocolates
• Cost: 100 dollars
• Location: Gift Shop
• Reason: Celebration

• Inference on Entity State. The inference on entity state is performed using the Entity-State Inferencing rules. The following steps are performed: -
o Updating the final entity state based on the entity-state before performing the current action and the state after performing the current action

Entity State Before Action (Transferred from previous Sentence) Entity State after Action Entity State after Inference
Aman
• Has possession=500 dollars
Aman
• Had possession=100 dollars
Aman
• Has possession=400 dollars

o Updating the entity-stateof entities based on the change of entity state of related Entities/Activities which have changed due to the current action.

Entity State of an Entity Entity State of the Related Entity Entity State after Inference of the related Entity
Aman
• Has Location=gift shop
Aman
• Has possession=chocolates
chocolates
• Has location=Gift shop

o The updated entity state is transferred to the next sentence based on the entity state transfer rules that are part of the Entity-State Inferencing rules. Some of the examples of entity-state transfer is as below: -

From To State Transfer
S1SS1 S2SS1 Aman
• Has possession=500 dollars
S2SS1 S2SS2 Aman
• Has possession=500 dollars

• Activity Role completion. Sometimes, the information of some of the entities which are part of an Activity come across from other earlier sentences. The Activity-State Inferencing rules are applied on the activities using the entity state of all the entities inferred so far. This results in identification of more roles in the activities thereby enriching the activities. In the example above, the activity-state of two activities result in inferring the destination role of one of the activities: -

Activities Before Role Completion Change-Of-Location
• Performer: Aman
Buy-Item
• Buyer: Aman
• Item: Box of chocolates
• Cost: 100 dollars
• Location: Gift Shop
Activities after Role Completion Change-Of-Location
• Performer: Aman
• Destination: Gift Shop
Buy-Item
• Buyer: Aman
• Item: Box of chocolates
• Cost: 100 dollars
• Location: Shop

• Common Sense and Context based Reasoning. Common sense and context-based reasoning uses the commonsense rules to derive context/domain specific inferences. The step also applies the context specific rules to derive context specific activity-state and entity-state. In the above example, we can apply the commonsense rule that if a person withdraw money from bank, he/she should have a bank account with sufficient balance. Hence applying this commonsense rule, the system derives the following fact: -

Aman
• Has asset= Bank account

The Final Output (also referred to as Semantic Data)

The final output of the Natural Language Understanding system consists of: -
• Semantic logical entities.
• Mapped relations between various entities.
• Final inferred state of all entities.
• All semantic activities which correspond to the input information.
• The identified roles of semantic activities.
• Semantic activities and the entities’ state inferred on a timeline interlinking all the activities and the entities.

The final understanding of the example paragraph above will be as below: -

Sentence Activities Entities and Entity State
Aman withdrew 500 dollars from the bank. Withdraw
• Performer: Aman
• Amount: 500 Dollars
• Source: Bank Aman
• Has possession=500 dollars
• Has asset= Bank account

Bank
• Had possession=500 dollars

He went and bought a box of chocolates for 100 dollars from the gift shop to celebrate Anya’s success. Change-Of-Location
• Performer: Aman
• Destination: Gift Shop

Buy-Item
• Buyer: Aman
• Item: Box of chocolates
• Cost: 100 dollars
• Location: Gift Shop
• Reason: Celebration

Celebration
• Performer: Aman
• Reason: Success Aman
• Has possession=400 dollars
• Has possession= box of chocolates
• Has Location=Gift shop

Box
• Has content=chocolates
• Has location = Gift shop

Success
• Is of-entity=Anya

Shop

Anya

She had been adjudged as the student of the year unanimously. Facilitation
• Awardee: Anya
• Title: Student of the year
• Decision-type: unanimously Anya
• Has Award=Student of the year

Aman
• Has possession=400 dollars
• Has possession= box of chocolates

He called and congratulated her.
Communicate
• Performer: Aman
• Performee: Anya

Congratulate
• Performer: Aman
• Performee: Anya Anya
• Has Award=Student of the year

Aman
• Has possession=400 dollars
• Has possession= box of chocolates
PS: Only relevant entities and their states is shown in the table

This final output of the NLU Engine will be the understood state representing the natural language input. This output which is also called the semantic output is in the form of interlinked graph of activities and entities with their enriched activity-state and entity-state. This output can be further consumed by any application as explained below.
• Semantic data can be stored as the equivalent of the NLI and used to retrieve, analyze, and visualize the NLI
• Relevant information can be extracted for any specific business needs.
• It can be used to classify the NLI
• It can be used as input for a conversational system.

SUMMARY

The AUI Natural Language Understanding process takes the unstructured Natural Language input in the form of text and produces a semantic output fit to be easily utilized by prevalent information systems with complete explain-ability. This output can be consumed in any of the Natural language-based applications for classification, information extraction, conversations and any other analytical tasks.
This invention implements natural language understanding capability in computer systems. The system analyzes a natural-language input text and through a series of processing cycles, transforms the source text into semantic data consisting of temporal referenced entities, their interrelations mapped, semantic activities identified, roles and actors mapped to identified semantic activities, entity states inferred, logic, reasoning and mathematics applied, and a story built. This output of semantic data is then utilized in various systems to utilize the unstructured textual inputs with accuracy and accountability. The system output is fully explainable. The applications involve all systems which consume natural language inputs to provide query answering, summarization, information extraction and conversational interactions.

THE CLAIMS

We claim: -

1. A framework and inherent method of interpreting and understanding natural language textual inputs using a system comprising of the Knowledge Base and Natural Language Understanding (NLU) Engine sub systems. The steps involve converting the language dependent textual input into language agnostic semantic data, application of common and mathematical analysis through multiple inferencing cycles to achieve the final output of temporally mapped, inter-related, semantic logical entities with their final state duly inferred using common-sense reasoning.

2. The design and method of claim 1 wherein the “Knowledge Base” models, captures and makes available the worldly knowledge (also referred to as common or general knowledge) and language processing rules to the system.

3. The design and methods of claim 1 wherein the “NLU Engine” with inputs from “Knowledge Base” converts the language dependent textual input into language agnostic “Semantic Data”.

Annexure 1
SYMBOL INDEX
The Symbol Index gives a unique ID to every Sense in the Knowledge Base. The important information stored for each sense is as follows: -
• Unique ID. This ID is an unambiguous identity for the sense in the entire Knowledge Base.
• Sense Name. This is the Human readable word/words which represent the sense.
• Parts of Speech Tag. This gives the part of speech of the sense.
• Tense. The tense of the sense if POS=Verb
• Plurality. Defines the Plurality of a sense if the POS=noun
• Lemma. Points to the Sense which the Lemme of this sense
• Common noun type for proper noun. Used to identify the Proper noun type
• Remarks. Human readable description of the Symbol

The following Table is a sample of some of the senses that are used in the example in the document.

Unique ID Sense Name Parts of Speech Tag Tense Plurality Lemma Common noun type for proper noun Remarks
1 human Noun
2 Aman Proper Noun Human
3 Withdraw Verb Present
4 withdrew Verb Past Withdraw
5 dollars Noun plural dollar
6 from Preposition
7 the Article
8 bank Noun Financial institution
9 bank Noun River bank
10 He Pronoun
11 Go Verb Present
12 went Verb Past go
13 and Conjunction
14 buy Verb Present
15 bought Verb Past buy
16 a Article
17 box Noun
18 Of Preposition
19 chocolate Noun
20 chocolates Noun plural chocolate
21 for Preposition
22 shop Noun Financial institution
23 Gift Verb Present Action of giving a gift
24 To Preposition
25 celebrate Verb Present
26 Anya Proper Noun human
27 success Noun
28 she Pronoun
29 has Verb Present
30 had Verb Past has
31 been Verb Present
32 adjudge Verb Present
33 adjudged Verb Past adjudge
34 as Preposition
35 student of the year Noun
36 _NUMBER Noun
37 call Verb Present
38 called Verb Past call
39 her Pronoun
40 congratulate Verb Present
41 congratulated Verb Past congratulate
42 gift Noun
43 unanimously Adverb

Annexure 2

TAXONOMY

Every Sense in the Knowledge Base is organized in a hierarchical tree structure. The hierarchy starts with a notional node called Knowledge Root. The first level of children of the Knowledge Root are the POS tag nodes. Then each of the sense is populated into its tree. Each sense can also have multiple parents. Each child inherits all the properties and rules defined for the parent.
A sample Taxonomy if as below: -

Annexure 3

Adjective-Noun rules

The Adjective-Noun Rules can be defined for any Adjective (both leaf and in-between nodes in the Hierarchy of Adjectives). The Adjective-Noun Rules will be of the following Format: -
Rule: Number Noun
Inference: Noun has Value = Number
The methodology of application: -
Whenever a pattern with a number followed by any noun comes in the Natural Language Input, the Rule will match. This will result in application of the inference corresponding to this rule. For Example: -
In the input “Aman withdrew 500 dollars from the bank”, 500 dollars matches the rule (500 is number, dollar is a Noun). Hence the resulting Inferences will be
• Creation of an Entity called dollars
• Adding of the property “Value” in the Entity dollars resulting in
dollars has Value = 500

Annexure 4

NOUN-NOUN RULES

The Noun-Noun Rules can be defined for any Noun (both leaf and in-between nodes in the Hierarchy of Nouns). The Noun-Noun Rules will be of the following Format: -
Rule: Offering Business-Activity-space
Inference: Business-Activity-space has Offering = Offering
The methodology of application: -
Whenever a pattern with Offering type followed by any Business-Activity-space comes in the Natural Language Input, the Rule will match. This will result in application of the Inference corresponding to this rule. For Example: -
In the input “from the gift shop”, gift shop matches the rule (Gift is an offering, shop is a Business-Activity-space). Hence the resulting Inferences will be
• Creation of Entities Gift and Shop
• Adding of the property “Offering” in the Entity Shop as Shop has Offering = Gift

Annexure 5

NOUN-PREPOSITION-NOUN RULES

The Noun-Preposition-Noun Rules can be defined for any Preposition when a pair of Nouns come in-between a preposition in a Sentence that has a logical meaning (both leaf and in-between nodes in the Hierarchy of Nouns). The Noun-Preposition-Noun Rules will be of the following Format: -
Rule: CONTAINER of NOUN
Inference: CONTAINER has content = NOUN
The methodology of application: -
Whenever a pattern with any Container type symbol is followed by the preposition ‘Of’ followed by any NOUN comes in the Natural Language Input, the Rule will match. This will result in application of the Inference corresponding to this rule. For Example: -
In the input “He bought a box of chocolates”, box of chocolates matches the rule (Box is a Container, Chocolates is noun). Hence the resulting Inferences will be
• Creation of Entities Box and Chocolates
• Adding of the property “content” in the Entity Basket as Box has content = chocolates

Annexure 6

VERB-SUBJECT-OBJECT RULES

The Verb-Subject-Object Rules can be defined for any verb. Any Sentence with a Verb will have the following format: -
[Subject] [Verb] [Direct Object] {Preposition Noun-Phrase}
The {Preposition Noun-Phrase} can come zero or more than zero number of times in the sentence. The Verb-Subject-Object Rules will be of the following Format: -
Rule: [Human] [Buy] [Noun] {for Currency} {from Business-Activity-space}
Inference:
• Activity: Buy-Item
• Role: Buyer, Role-Performing-Entity/ Role-Performing-Activity: Human (Subject)
• Role: Location, Role-Performing-Entity/ Role-Performing-Activity: Business-Activity-space (3rd object)
• Role: Item, Role-Performing-Entity/ Role-Performing-Activity: Noun (Direct object which is also the first object)
• Role: Cost, Role-Performing-Entity/ Role-Performing-Activity: Currency (Which is the 2nd object)
The methodology of application: -
Whenever a pattern with
• Any Human as the subject
• Followed by verb ‘Buy’ (or any other tense of buy)
• Followed by the direct object of type Noun (optional)
• Followed by preposition ‘for’ and any currency (optional)
• Followed by Preposition ‘from’ and any Business-Activity-space (optional)
comes in the Natural Language Input, the Rule will match. This rule is matched even if zero or more than zero objects match. This will result in application of the Inference corresponding to this rule. For Example: -
In the input “He bought a Box of Chocolates for 100 Dollars”, The matches are as follows: -
• Subject: He is a human, hence matches the subject
• Verb: ‘Bought’ is the past tense of ‘Buy’ hence match
• Direct Object (first Object): Box is a Noun. Matches the direct object
• 2nd Object: ‘for’ and ‘Dollar’ matches the 2nd object
• 3rd Object: Shop matches the third object
The Inference will be applied as follows: -
• Creation of Activity Buy-Item
• Adding Role-Performing-Entity He to the role “Buyer”.
• Adding Role-Performing-Entity Box to the role “Item”.
• Adding Role-Performing-Entity Rs to the role “Cost”.
• Adding Role-Performing-Entity Shop to the role “Location”.
Note: Box will have Chocolates as part of its Entity-State and Dollar will have 500 as part of its entity state
Annexure 7

ACTIVITY-PREPOSITION/CONJUNCTION-ACTIVITY RULES

The Activity-Preposition/Conjunction-Activity rules can be defined for any Preposition/ Conjunction. This is for handling the relation between two activities which are generated by processing a complex sentence with more than one verb.
For Example, Aman bought chocolates to celebrate Anya’s Success. This sentence is broken into two simple sentences with one verb in each simple sentence like below: -
• Aman bought chocolates
• to celebrate Anya’s Success
These two simple sentences are connected by the preposition ‘to’. The first simple sentence will generate the Activity ‘Buy-item’ and the 2nd simple sentence will generate the Activity ‘Celebrate’ with their corresponding roles and role performing agents as below: -
Buy-Item
• Buyer: Aman
• Item: chocolates

Celebration
• Performer: Aman
• Reason: Success

The Activity-Preposition/Conjunction-Activity Rules will be of the following Format: -
Rule: Activity1 to Activity2
Inference: Activity1
Role= Reason, Role-performing-Activity = Activity2
The methodology of application: -
Here Buy-item is of type Activity and Celebration is of type Activity which are interconnected by the preposition ‘to’. Hence the rule matches and the state of activity Buy-item changes as follows: -
Buy-Item
• Buyer: Aman
• Item: chocolates
• Purpose: Celebration (The Celebration will be referring the complete Activity Celebration with all its identified roles)

Annexure 8

ADVERB-VERB RULES

The Adverb-Verb rules can be defined for any Adverb when it accompanies a Verb in a sentence. The adverb may be adjacent to the verb or somewhere in the simple sentence. The Adverb-Verb rules will be of the following Format: -

Rule: decision-type Verb
Inference: Verb has decision-type = decision-type

The methodology of application: -
Whenever an adverb is encountered in a Simple sentence, the adverb is paired with the Verb and if the adverb and the verb match the rule, the corresponding inference is applied. Later, when the verb is converted into an activity, the inference is further transferred to the Activity. For Example: -
In the input “She had been adjudged as the student of the year unanimously”, unanimously is a decision-type and adjudge is a verb and hence matches the rule. Hence the resulting Inferences will be
Facilitation
• Awardee: Anya
• Title: Student of the year
• Decision-type: unanimously
Annexure 9

ENTITY-STATE INFERENCING RULES

There are three kinds of Entity-State Inferencing rules. They are as follows: -
1. Intra Entity-State Inferencing. These rules are made to carry out inference between the properties of the same entity.
Rule: Entity had Possession currency1 AND Entity has Possession currency2 (The same entity has two possession properties with value of type Currency)
Inference: Entity has Possession currency2 – currenct1 (perform the subtraction operation on the value of the currency)
Example: Aman had possession = 100 Dollars and Aman has possession = 500 Dollars
Result: Aman has possession = 400 Dollars
2. Inter Entity-State Inferencing. These rules are made to carry out inference between the properties of the different entities which are connected by some property.
Rule: Entity1 has Possession Entity2 AND Entity1 has Location Entity3
Inference: Entity2 has Location Entity 3
Example: Aman has possession = Chocolates and Aman has Location = Shop
Result: Chocolates has Location = Shop
3. Property Transfer between Sentence for the same entity. These Rules are used to transfer a property of an Entity from one sentence to the other sentence. Primarily, properties are defined as single valued and multi valued properties. When an Entity gets a property in one sentence which is single valued, the property is transferred to the same entity in the next sentence only if the same property change has not occurred in the next sentence. If the property is multi valued, and if the same property gets changed in two sentences, both properties are kept with the entity.
Example 1: Multi Valued Properties
Sentence 1: Aman bought an apple.
State: Aman has possession = apple.
Sentence 2: Aman bought an orange.
State: Aman has possession = orange.
State after Property Transfer
Aman
has possession = apple
has possession = orange
Example 2: Single Valued Properties
Sentence 1: Aman went to Paris.
State: Aman has location = Paris.
Sentence 2: Aman flew to London.
State: Aman has location = London.
State after Property Transfer
Aman
has location = London
Annexure 10

ACTIVITY-STATE INFERENCING RULES

The Activity-State Inferencing rules helps to complete the roles of an activity based on other related activities and entities.
For Example, we have
“Aman went and bought chocolates from the shop” which results in Activities

Activity: Change-of-location
• Performer: Aman

Activity: Buy-item
• Buyer: Aman
• Item: chocolates
• Location: Shop

“Aman bought chocolates from the shop” results in an entity state
Aman
• has location = Shop
We can build a Role completion rule as follows: -
Rule: Activity has role=performer and Role-performing-Entity=Human &&
This Human has Location=X
Inference: Activity has role= Destination and Role-performing-Entity=X
Applying this rule, the above ‘Movement’ Activity will look like this after the Inference

Change-of-location
• Performer: Aman
• Destination: Shop

Annexure 11

COMMONSENSE RULES

Common Sense Rules can be built for the following things: -
• Inferencing of Entity States/Activity states which are otherwise hidden based on other Activity-states and/or Entity States
Example:
Rule:
Activity= Withdraw
Roles
• Performer=Human
• Source: Bank
Inference: (Entity State)
Human
• Has asset= Bank account

• Creation of New Activity based on other Activity-states and/or Entity States
Rule:
Activity = Describe
Roles
• Performer: Customer Service Agent
• Listener: Customer
• Topic: Product Feature
• Context: Outbound Call
•
Inference: (Create New Activity)
Create Activity: Agent-Selling-Attempt
Roles:
• Offering: Product
• Performer: Customer Service Agent

• Creation of New Entity based on other Activity-states and/or Entity States
Let’s say the sentence ‘Aman shot the elephant from his phone’ results in an activity as follows: -
Take-Photo
• Performer: Aman
• Target: Elephant
• Equipment: phone

So, we can build the rule:
Rule:
Take-Photo
• Performer: Aman
• Target: Elephant
• Equipment: phone

Inference: (Create New Entity)
Create Entity: Photo

Annexure 12

GENERALIZATION AND KNOWLEDGE COMPRESSION

Generalization implies the capability to extend knowledge to a broader set. When the available knowledge is generalized, it results in compression of knowledge. The following explains how our knowledge base structure uses Taxonomy to achieve the same.

The Taxonomy creates a Hierarchy of concepts. Each node in the Hierarchy qualifies a set of concepts to have a certain property.
Let us say we have to create an Adjective-Noun Rule to capture the words Tall Building.

The rule will be
“[Tall] [Building]” => Building has Height=tall

Looking at the building, this rule can be applied to all structures, We create a Node in the Hierarchy called Physical-Structures and add Building, Bridge, Tower, etc. to this node. Now the Rule can be generalized to
“[Tall] [Physical-Structures]” => Physical-Structures has Height=tall

Further we can keep creating higher level nodes including more concepts and may be reach a Node Physical-Entity to have all Concepts which are Physical and modify the rule to
“[Tall] [Physical-Entity]” => Physical-Entity has Height=tall

If we try to include non-Physical entity, then the Inference may not be suitable like in “Tall claim”.
On the Adjective side, we can group all adjectives which are indicative of Height and may be create a node called Height-indicating-Adjectives and further Generalize the rule to the following
“[Height-indicating-Adjectives] [Physical-Entity]”
=> Physical-Entity has Height= Height-indicating-Adjectives

Hence, we can generalize a rule to cover more use cases by building the rule at a higher level in the hierarchy and also reduce the number of required rules.

This Rule will now match all the following instances when it comes in Natural Language: -
• Tall Building
• Short person
• Tall Chair
• Short animal
And will not match the following: -
• Tall idea
• Short speech
• Long road

A sample Taxonomy can be as below: -

DIAGRAMS

Figure 1 : Block Diagram

Natural Language Understanding System

Figure 2 : Block Diagram

Knowledge Base

Figure 3 : Block Diagram

NLU Engine

,CLAIMS:We claim: -

1. A framework and inherent method of interpreting and understanding natural language textual inputs using a system comprising of the Knowledge Base and Natural Language Understanding (NLU) Engine sub systems. The steps involve converting the language dependent textual input into language agnostic semantic data, application of common and mathematical analysis through multiple inferencing cycles to achieve the final output of temporally mapped, inter-related, semantic logical entities with their final state duly inferred using common-sense reasoning.

2. The design and method of claim 1 wherein the “Knowledge Base” models, captures and makes available the worldly knowledge (also referred to as common or general knowledge) and language processing rules to the system.

3. The design and methods of claim 1 wherein the “NLU Engine” with inputs from “Knowledge Base” converts the language dependent textual input into language agnostic “Semantic Data”.