Abstract: The present invention relates to a system and a method that facilitates phonetic detection and fat finger analysis leveraging the noble approach of good content data (product description, titles, etc) and noisy content data (like search queries, customer feedback chats which contains clean tokens along with misspelled words and typos). The present disclosure also provides for a system and method for understanding how Indic languages (high variance in localised dialects) adds noise to the system thus, respecting “no one size fits all” different from languages spoken in western regions (low variance in localised dialects) makes system to robust enough to make clear understanding of concept irrespective of finger typos or spell errors.
DESC:FIELD OF INVENTION
[0001] The embodiments of the present disclosure generally relate to spell correction systems. More particularly, the present disclosure relates to a system and method for leveraging both phonetic and fat finger character transition analysis for incorporating nuances to spelling correction systems in the Indic context. The disclosure also showcases how regional i.e., geographical and spoken local language phenomenon cause the spell errors and how these insights are used to build Indic aware spell correction systems.
BACKGROUND OF THE INVENTION
[0002] The following description of related art is intended to provide background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section be used only to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of prior art.
[0003] Natural language processing and its understanding has made a great impact in solving real life problem. It is very important for any system to understand how humans interpret language forms, correct spell error and their connections with lexical, syntactic and semantic stages to extract valuable nugget for word context understanding irrespective of any language forms whether Indic or any foreign language.
[0004] However, most of the current systems are limited to word level correction instead of knowledge aware concepts level corrections. Most search engines fail to search results even there is a minute delta difference or noise in a word that is being search. A fat finger error is a human error caused by pressing the wrong key when using a computer to input data. Fat finger errors are often harmless but can sometimes have a significant market impact. This makes search system to lack semantic information as well as regional language understanding. The correct phonetic mapping of words while retrieving information from search engines can make a huge impact search relevance and personalisation of results and can improve efficiency by multiple folds. As, Indic dialects have very high variance which make it clear that “No one size fits all”, multiple systems are required to handle those linguistic variances in an efficient way. While, most of western countries have low variance as language spoken in those regions is not that diverse and phonetic.
[0005] There is therefore a need in the art to provide an Indic aware system that can handle high linguistic variance in an efficient way.
OBJECTS OF THE PRESENT DISCLOSURE
[0006] Some of the objects of the present disclosure, which at least one embodiment herein satisfies are as listed herein below.
[0007] It is an object of the present disclosure to provide a personalised Search Results based on localised dialects.
[0008] It is an object of the present disclosure to provide a better understanding chat-bot application.
[0009] It is an object of the present disclosure to provide an auto-spell correction / suggestion system for discovery of unique words.
SUMMARY
[0010] This section is provided to introduce certain objects and aspects of the present disclosure in a simplified form that are further described below in the detailed description. This summary is not intended to identify the key features or the scope of the claimed subject matter.
[0011] In an aspect, the present disclosure provides for a spell correction system. The system may include a processor that executes a set of executable instructions that may be stored in a memory, upon which execution, the processor may cause the system to receive a word input from a keypad associated with the computing device and receive a batch of words from a knowledgebase associated with a centralized server, the batch of words pertaining to a similar set of words to the received word input. The system may further extract, by an artificial intelligence (AI) engine, a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of the word input with a predefined phonetic index. In an embodiment, the AI engine may be associated with the processor. The system may further extract, by the AI engine, a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of each word having the predefined phonetic index. The system may then compare, by a retrofitting engine, the first set of attributes with the second set of attributes of each said word in the received batch of words. In an embodiment, the retrofitting engine may be associated with the processor. Based on the compared first set of attributes with the second the second set of attributes of each word, the system may then generate and map, by the retrofitting engine, a score to each compared first set of attributes with the second the second set of attributes of each word, wherein the score is generated based on the second set of attributes of each word having the highest similarity in the predefined phonetic index with the first set of attributes; and, further, generate, by the retrofitting engine, based on the mapped first set of attributes with the second the second set of attributes of each said word, a response to the received input word.
[0012] In an embodiment, the word input may be a word of any or a combination of a predefined language, a mixture of one or more languages, a mixture of one or more dialects with the predefined language, based on an accent of a native place associated with the one or more languages or the one or more dialects or a combination thereof. Further, in an embodiment, the processor may be configured to consider each word from the batch of words one at a time for providing the response.
[0013] In an embodiment, the response comprises any or a combination of correct spelling of the received input word and a detected phonetic word associated with one or more languages.
[0014] In an embodiment, the processor may be configured to determine, by the retrofitting engine, one or more global phonetic words from the knowledgebase to enhance phonetic vocabulary based on the score generated for each second set of attributes.
[0015] In an embodiment, the processor may be configured to inject, by the retrofitting engine, the one or more global phonetic words to one or more global knowledge graphs to drive any search engines or associated auto-correction systems.
[0016] In an embodiment, the second set of attributes may include a plurality of phonetic variants for the word input received, the plurality of phonetic variants further including a mixture of good context words, clean golden word set such as product catalogue, description, titles along with noisy content data such as localised phonetically similar sounding words from search, chat data, typos from fat finger syndrome.
[0017] In an embodiment, the processor may be further configured to train, by the AI engine, a mapping of the first set of attributes with the second set of attributes of each word of the bath of words for auto-spell correction or detection of phonetic words in the context of language understanding and extraction of information from the detected phonetic words.
[0018] In an embodiment, the processor may be further configured to update the knowledgebase with latest phonetic knowledge to leverage spell correction-based information retrieval.
[0019] In an embodiment, the processor may be further configured to pre-process the input word, wherein the pre-process comprises tokenisation, stop-word removal and n-gram phrase extraction.
[0020] In an embodiment, the processor may be further configured to normalize one or more high scores from the generated scores associated with each second set of attributes with edit distance scores to remove any repeated character pairs phonetic noise.
[0021] In an embodiment, the processor may be further configured to compute a set of character-to-character transition instructions from the first set of attributes, the set of character-to-character transition instructions comprising transition probability from a corrected character to a misspelled character associated with the input word.
[0022] In an aspect, the present disclosure provides for a spell correction method. The method may include the steps of receiving, by a processor, a word input from a keypad associated with the computing device (104), wherein the processor executes a set of executable instructions that are stored in a memory and receiving, by the processor, a batch of words from a knowledgebase associated with a centralized server, the batch of words pertaining to a similar set of words to the received word input. The method may further include the steps of extracting, by an artificial intelligence (AI) engine associated with the processor, a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of words with a predefined phonetic index, and extracting, by the AI engine, a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of said word having the predefined phonetic index. The method may further include the step of comparing, by a retrofitting engine associated with the processor, the first set of attributes with the second set of attributes of each said word in the received batch of words. Based on the compared first set of attributes with the second the second set of attributes of each word, the method may include generating and mapping, by the retrofitting engine, a score to each compared first set of attributes with the second the second set of attributes of each word. The score may be generated based on the second set of attributes of each word having the highest similarity in the predefined phonetic index with the first set of attributes. Furthermore, the method may include the step of generating, by the retrofitting engine, based on the mapped first set of attributes with the second the second set of attributes of each said word, a response to the received input word.
BRIEF DESCRIPTION OF DRAWINGS
[0023] The accompanying drawings, which are incorporated herein, and constitute a part of this invention, illustrate exemplary embodiments of the disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present invention. Some drawings may indicate the components using block diagrams and may not represent the internal circuitry of each component. It will be appreciated by those skilled in the art that invention of such drawings includes the invention of electrical components, electronic components or circuitry commonly used to implement such components.
[0024] FIG. 1 illustrates an exemplary network architecture (100) in which or with which the system of the present disclosure can be implemented, in accordance with an embodiment of the present disclosure.
[0025] FIG. 2 illustrates an exemplary representation (200) of system (102) or a centralized server (112), in accordance with an embodiment of the present disclosure.
[0026] FIG. 3 illustrates an exemplary high level system architecture (300) in accordance with an embodiment of the present disclosure.
[0027] FIG. 4A-4E illustrate exemplary representations (400) of the proposed implementations, in accordance with an embodiment of the present disclosure.
[0028] FIGs. 5A-5C illustrate exemplary representations of analysis on similar sounding n-character pairs, in accordance with an embodiment of the present disclosure.
[0029] FIG. 6 illustrates an exemplary computer system in which or with which embodiments of the present invention can be utilized in accordance with embodiments of the present disclosure.
[0030] The foregoing shall be more apparent from the following more detailed description of the invention.
BRIEF DESCRIPTION OF INVENTION
[0031] In the following description, for the purposes of explanation, various specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. It will be apparent, however, that embodiments of the present disclosure may be practiced without these specific details. Several features described hereafter can each be used independently of one another or with any combination of other features. An individual feature may not address all of the problems discussed above or might address only some of the problems discussed above. Some of the problems discussed above might not be fully addressed by any of the features described herein.
[0032] Embodiments of the present invention may be provided as a computer program product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. The machine-readable medium may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).
[0033] The present invention provides solution to the above-mentioned problem in the art by providing a system and a method for training neural networks on local data (domain-specific) and using these embeddings to 'retrofit’ on global pre-trained embeddings (domain-agnostic) to generate personalised auto spell-correction systems. Particularly, the system and method may provide improved efficiency on Indic transliterated data across in NLP tasks, such as spell checker, auto-spell correction and suggestion on multiple Indic languages.
[0034] Referring to FIG. 1 that illustrates an exemplary network architecture (100) in which or with which system (102) of the present disclosure can be implemented, in accordance with an embodiment of the present disclosure. As illustrated, the exemplary architecture (100) includes a system (102) equipped with an artificial intelligence (AI) engine (214) for phonetic and fat finger character transition analysis. The input can be received from a plurality of users (110-1, 110-2,…. 110-n) (hereinafter interchangeably referred as user or client; and collectively referred to as users 110). Each user may be associated with at least one computing device (104-1, 104-2,…. 104-n) (hereinafter interchangeably referred as a smart computing device or audio device; and collectively referred to as 104). The users (110) may interact with the system (102) by using their respective computing device (104). The computing device (104) and the system (102) may communicate with each other over a network (106). The system (102) may be associated with a centralized server (112). Examples of the computing devices (104) can include, but are not limited to, a computing device (104) associated with healthcare and hospital-based assets, education sector, customer-based firms, a smart phone, a portable computer, a personal digital assistant, a handheld phone and the like.
[0035] Further, the network (106) can be a wireless network, a wired network, a cloud or a combination thereof that can be implemented as one of the different types of networks, such as Intranet, BLUETOOTH, MQTT Broker cloud, Local Area Network (LAN), Wide Area Network (WAN), Internet, and the like. Further, the network 106 can either be a dedicated network or a shared network. The shared network can represent an association of the different types of networks that can use variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like. In an exemplary embodiment, the network 104 can be anHC-05 Bluetooth module which is an easy to use Bluetooth SPP (Serial Port Protocol) module, designed for transparent wireless serial connection setup.
[0036] According to various embodiments of the present disclosure, the system (102) can provide for an Artificial Intelligence (AI) based phonetic detection and fat finger analysis by using signal processing analytics, particularly for providing input services in at least one or more languages and dialects. In an illustrative embodiment, the phonetic processing AI techniques can include, but not limited to, a Language Processing Algorithm and can be any or a combination of machine learning (referred to as ML hereinafter), deep learning (referred to as DL hereinafter), and natural language processing using concepts of neural network techniques. The technique and other data or phonetic model involved in the use of the technique can be accessed from a database in the server.
[0037] In an aspect, the system (102) can receive a word input from a keypad associated with the computing device (104). In an embodiment, the system (102) can receive a word input from a keypad associated with the computing device (104) and also receive a batch of words from a knowledgebase associated with the centralized server (112), the batch of words pertaining to a similar set of words to the received word input.
[0038] In an embodiment, the word input may be but not limited to a word of any or a combination of a predefined language, or a mixture of one or more languages with the predefined language, a mixture of one or more dialects with the predefined language based on an accent of a native place associated with the one or more languages. The word input may be common to one or more geographical regions based on a similar phonetic index. For example, cities such as Guwahati-Shillong, Ahmedabad-Rajkot, Mumbai-Pune-Aurangabad, Chennai-Coimbatore-Trichy, Dehradun-Calcutta-Bengal may give strong evidence of being phonetically similar due to regional dialect spelt at geographical proximity places.
[0039] In an embodiment, the system (102) may further extract, by the AI engine (214), a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of words with a predefined phonetic index. The system can further extract, by the AI engine (214), a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of said word having the predefined phonetic index. The system may then compare with the help of a retrofitting engine (216), the first set of attributes with the second set of attributes of each word in the received batch of words.
[0040] In an embodiment, the system with the help of the retrofitting engine (216), can generate and map a score to each compared first set of attributes with the second the second set of attributes of each word based on the compared first set of attributes with the second the second set of attributes of each word. In an embodiment, the score may be generated based on the second set of attributes of each word having the highest similarity in the predefined phonetic index with the first set of attributes.
[0041] In an embodiment, the system (102), equipped with the retrofitting engine (216) can generate a response to the received input word based on the mapped first set of attributes with the second the second set of attributes of each word.
[0042] In an exemplary embodiment, the second set of attributes may include all possible phonetic variants for a given word. The possible phonetic variants may include a mixture of good context words for example, clean golden word set such as product catalogue, description, titles and the like along with noisy content data such as localised phonetically similar sounding words from search, chat data, typos from fat finger syndrome and the like.
[0043] The AI engine (214) may be further configured to train how to spell correction or detection of phonetic words in the context of language understanding and the information which could be extracted from them. The AI engine (214) may further be configured to update with latest phonetic knowledge used to leverage spell correction-based information retrieval.
[0044] In an exemplary embodiment, the word input may pass through layers of pre-processing steps from tokenisation, stop-word removal and n-gram phrase extraction and the like. The scores generated by the retrofitting engine may be based on NER detection and TF-IDF scores but not limited to the like.
[0045] In an exemplary embodiment, the high score values from the generated scores may be then normalised with edit distance scores to remove any repeated character pairs phonetic noise.
[0046] In an embodiment, the similar phonetic words extracted may be used to compute char-to-char transition matrix where each cell represent transition probability from corrected character to misspelled character, for example ‘Y’ to ‘I’, where ‘Y’ being correct character and ‘I’ being misspelled word.
[0047] In an embodiment, the system (102) for may include one or more processors coupled with a memory, wherein the memory may store instructions which when executed by the one or more processors may cause the system to perform the extraction, determination and generation of steps as described hereinabove.
[0048] FIG. 2 illustrates an exemplary representation (200) of system (102), in accordance with an embodiment of the present disclosure. In an aspect, the system (102) may comprise one or more processor(s) (202). The one or more processor(s) (202) may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuitries, and/or any devices that process data based on operational instructions. Among other capabilities, the one or more processor(s) (202) may be configured to fetch and execute computer-readable instructions stored in a memory (204) of the system (102). The memory (204) may be configured to store one or more computer-readable instructions or routines in a non-transitory computer readable storage medium, which may be fetched and executed to create or share data packets over a network service. The memory (204) may comprise any non-transitory storage device including, for example, volatile memory such as RAM, or non-volatile memory such as EPROM, flash memory, and the like.
[0049] In an embodiment, the system (102) may include an interface(s) 206. The interface(s) 206 may comprise a variety of interfaces, for example, interfaces for data input and output devices, referred to as I/O devices, storage devices, and the like. The interface(s) 206 may facilitate communication of the system (102). The interface(s) 206 may also provide a communication pathway for one or more components of the system (102). Examples of such components include, but are not limited to, processing engine(s) 208 and a database 210.
[0050] The processing engine(s) (208) may be implemented as a combination of hardware and programming (for example, programmable instructions) to implement one or more functionalities of the processing engine(s) (208). In examples described herein, such combinations of hardware and programming may be implemented in several different ways. For example, the programming for the processing engine(s) (208) may be processor executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the processing engine(s) (208) may comprise a processing resource (for example, one or more processors), to execute such instructions. In the present examples, the machine-readable storage medium may store instructions that, when executed by the processing resource, implement the processing engine(s) (208). In such examples, the system (102) may comprise the machine-readable storage medium storing the instructions and the processing resource to execute the instructions, or the machine-readable storage medium may be separate but accessible to the system (102) and the processing resource. In other examples, the processing engine(s) (208) may be implemented by electronic circuitry.
[0051] The processing engine (208) may include one or more engines selected from any of a data acquisition engine (212), AI engine (214), retrofitting engine (216) and other units (218).
[0052] The data acquisition engine (212) may be configured to receive a word input from a keypad associated with the computing device and receive a batch of words from a knowledgebase associated with the centralized server (112), the batch of words pertaining to a similar set of words to the received word input.
[0053] The artificial intelligence (AI) engine (214) may be configured to extract a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of the word input with a predefined phonetic index. The AI engine, may further extract a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of each word having the predefined phonetic index.
[0054] The retrofitting engine (216) may be configured to compare the first set of attributes with the second set of attributes of each word in the received batch of words. Based on the compared first set of attributes with the second the second set of attributes of each word, the retrofitting engine (216) may then generate and map a score to each compared first set of attributes with the second the second set of attributes of each word. The retrofitting engine (216) may be further configured to generate a response to the received input word based on the mapped first set of attributes with the second the second set of attributes of each said word.
[0055] FIG. 3 illustrates an exemplary high level system architecture (300) in accordance with an embodiment of the present disclosure. As illustrated, in an aspect, the high-level system architecture (interchangeably referred to as the phonetic variant model training pipeline 300 or simply as pipeline 300) may extract all possible phonetic variants for a given word. The dataset contains mixture of good context words (302) such as clean golden word set such as product catalogue, description, titles and the like along with Noisy content data (312) containing localised phonetically similar sounding words from search, chat data and the like. The dataset may then pass through layers of pre-processing steps from tokenisation, stop-word removal and n-gram phrase extraction (304) based on NER detection and TF-IDF scores.
[0056] The pre-processed data may be then passed to model training step (306) where words having high phonetic index and extracted using Soundex, Double meta-phoneme and the like may get more attention for high similarity score. The high score values are normalised with edit distance scores (to remove any “repeated character pairs” phonetic noise to generate a candidate set (308). The similar phonetic words extracted are used to compute char-to-char transition matrix (324) where each cell represent transition probability from corrected character to misspelled character. The localised embeddings are finally retrofitted by the retrofitting model (318) with global phonetic words (310) to enhance phonetic vocabulary which is injected to global knowledge graphs (316) used to drive any search engines / auto-correction systems. This is a cyclic process and keeps on updating with latest phonetic knowledge used to leverage spell correction-based information retrieval.
[0057] For example, similar sounding words such as Shirt, Short and Skirt have high phonetic similarity and minimal edit distance score (edit_distance (shirt, short) = 1 should make the embeddings closer to each other. The Retrofitting Model (318) may make the embeddings understand the concepts and thus separate the words, which may be similar sounding but have different meanings in the same domain such as ‘Fashion’.
[0058] In an exemplary embodiment, the character transitions of uni-character and bi-character pairs for calculating cross regional MSE scores such as characters with n > 2, generally add noise to spell correction and should be neglected.
[0059] Based on character transition matrices of size n x n (where, n = 26 (alphabets) + 26*26 (alphabet pairs) = 702 ) of a given region (which could be city / state etc.), Mean Squared Error (MSE) is calculated for pair of regions using the equation below,
…………………(1)
[0060] FIG. 4A-4D illustrate exemplary representations (400) of the proposed implementations, in accordance with an embodiment of the present disclosure. As illustrated in FIG. 4A, each represents MSE score of phonetic transitions calculated using equation (1) between pair of cities, where darker shade (0.0 <= MSE <= 0.4) conveys high phonetic similarity between pair of cities and vice versa.
[0061] In an exemplary embodiment, character transitions probability matrices for phonetic transliterations, fat finger typos are shown below:
[0062] The character transitions probability matrices like for phonetic transliterations, fat finger typos for cross city, state and regional levels (North, South, East, West, Central regions of India) shows that Indic phonetic heat maps (plotted over any pair of cities) have smooth distribution probabilities due to dialects spoken in those regions while fat finger heat maps are pixelated distribution due to common human typos occurred irrespective of his/her demographics or geographic regions. For example, as shown in FIGs. 4B and 4C, some of the highlighted cities such as Guwahati-Shillong, Ahmedabad-Rajkot, Mumbai-Pune-Aurangabad, Chennai-Coimbatore-Trichy, Dehradun-Calcutta-Bengal may give strong evidence of being phonetically similar due to regional dialect spelt at geographical proximity places.
[0063] Bangalore and Vijayawada and Indore are some cities which are phonetically very different from other cities and are highlighted by its specific dialects spoken at these locations. Similarly, cities like Bhubaneswar (CENTRAL) v/s Coimbatore (SOUTH) have high MSE scores due to location specific phonetics.
[0064] Delhi - Chandigarh - Ambala are very closer to each other and thus have least MSE scores (high similarity) representing a similar relationship of spelt dialects due to similar language spoken in those regions.
[0065] Cities like Ahmedabad - Vadodara - Rajkot have minimal difference in MSE scores, representing a similar relationship of spelt dialects due to phonetic similarity.
[0066] As illustrated in FIGs. 4D and 4E, there is mostly uniform and low MSE score (0.0 - 0.01) for all cross city comparisons. This actually makes sense as Fat Finger Typos are not location specific. Thus, we can conclude that there isn’t much Variance in typos due to Fat Finger Syndrome. As highlighted in FIGs. 4D and 4E, MSE for Fat-Finger (0.0 - 0.01) while MSE for Phonetic (0.0-2.0) which shows that upper limit (0.01) for fat-finger is almost equal to lower limit of phonetic transitions (~ 0.0). Thus, fat-finger transitions can be powered by a single national-level matrix.
[0067] As illustrated, in FIGs. 4D and 4E, Some of the highlighted regions such as Ahmedabad - Rajkot - Vadodara , Delhi-Ambala-Chandigarh, Gwalior-Indore , Chennai-coimbatore - Madurai - Trichy, Hyderabad - Vijayawada - South Karnataka are phonetically similar as well as neighbouring states, which shows the flavour speech spoken at those neighbouring regions. There are some city pairs like (Ahmedabad , Kochi) , (Ajmer , Kochi) , (Rajkot , Madurai) which have very high MSE scores. And thus, they are clearly separated from other due to different geographical parts of India.
• FIGs. 5A-5C illustrate exemplary representations of analysis on similar sounding n-character pairs, in accordance with an embodiment of the present disclosure. As illustrated in FIG. 5A, analysis on similar sounding n-character pairs and its correction such as ‘F' most commonly replaced with ‘PH’
• ‘K’ most commonly replaced with ‘C’ or ‘X’
• ‘I’ most commonly replaced with ‘Y’ or ‘EE’
[0068] The above are some frequent replacements which could be found in Indic search results. In FIG. 5A, ‘index’ refers to correct word mapping to its most frequent ‘misspelled word’ (where ‘I’ is replaced with ‘Y’) along with the frequency of occurrence of respective index and misspelled words in grocery domain. FIG. B also shows uni-char ‘S’ replaced with a bi-char ‘SH’ in fashion domain. (where ‘Socks’ is frequently searched as ‘Shocks’.)
[0069] FIG. 5B illustrates an exhaustive indic versus western region character to character analysis that gives clear indication that Indic spell errors are prone to multiple dialects spoken in various parts of country while foreign errors are generally occurred due to finger slippage (Fat Finger Syndrome) while typing. While FIG. 5C illustrates a probability transition matrix showing possible typos when searching for ‘ADIDAS’, where ‘D’ is frequently replaced with ’S’ (which can also be found adjacent to ‘D’ in Keyboard too.). Phonetically similar sound words have character transition probabilities which are domain agnostic in nature, i.e., irrespective of any domain such as search engines for retail, fashion, grocery, electronics and etc, have almost similar transition matrices which make phonetic spell correction a powerful and impactful tool in any possible domain.
[0070] FIG. 6 illustrates an exemplary computer system in which or with which embodiments of the present invention can be utilized in accordance with embodiments of the present disclosure. As shown in FIG. 6, computer system 600 can include an external storage device 610, a bus 620, a main memory 630, a read only memory 640, a mass storage device 650, communication port 660, and a processor 670. A person skilled in the art will appreciate that the computer system may include more than one processor and communication ports. Examples of processor 670 include, but are not limited to, an Intel® Itanium® or Itanium 2 processor(s), or AMD® Opteron® or Athlon MP® processor(s), Motorola® lines of processors, FortiSOC™ system on chip processors or other future processors. Processor 670 may include various modules associated with embodiments of the present invention. Communication port 660 can be any of an RS-232 port for use with a modem based dialup connection, a 10/100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. Communication port 660 may be chosen depending on a network, such a Local Area Network (LAN), Wide Area Network (WAN), or any network to which computer system connects. Memory 630 can be Random Access Memory (RAM), or any other dynamic storage device commonly known in the art. Read-only memory 640 can be any static storage device(s) e.g., but not limited to, a Programmable Read Only Memory (PROM) chips for storing static information e.g., start-up or BIOS instructions for processor 670. Mass storage 650 may be any current or future mass storage solution, which can be used to store information and/or instructions. Exemplary mass storage solutions include, but are not limited to, Parallel Advanced Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and/or Firewire interfaces), e.g. those available from Seagate (e.g., the Seagate Barracuda 7102 family) or Hitachi (e.g., the Hitachi Deskstar 7K1000), one or more optical discs, Redundant Array of Independent Disks (RAID) storage, e.g. an array of disks (e.g., SATA arrays), available from various vendors including Dot Hill Systems Corp., LaCie, Nexsan Technologies, Inc. and Enhance Technology, Inc.
[0071] Bus 620 communicatively couples with processor(s) 670 with the other memory, storage and communication blocks. Bus 620 can be, e.g., a Peripheral Component Interconnect (PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), USB or the like, for connecting expansion cards, drives and other subsystems as well as other buses, such a front side bus (FSB), which connects processor 670 to software system.
[0072] Optionally, operator and administrative interfaces, e.g., a display, keyboard, and a cursor control device, may also be coupled to bus 620 to support direct operator interaction with a computer system. Other operator and administrative interfaces can be provided through network connections connected through communication port 660. The external storage device 610 can be any kind of external hard-drives, floppy drives, IOMEGA® Zip Drives, Compact Disc - Read Only Memory (CD-ROM), Compact Disc-Re-Writable (CD-RW), Digital Video Disk-Read Only Memory (DVD-ROM). Components described above are meant only to exemplify various possibilities. In no way should the aforementioned exemplary computer system limit the scope of the present disclosure.
[0073] Thus, the present disclosure provides a unique and inventive solution for efficiently leveraging the noble approach of good content data (product description, titles, etc) and noisy content data (like search queries, customer feedback chats which contains clean tokens along with misspelled words and typos). The present disclosure also provides for a system and method for understanding how Indic languages (high variance in localised dialects) adds noise to the system thus, respecting “no one size fits all” different from languages spoken in western regions (low variance in localised dialects) makes system to robust enough to make clear understanding of concept irrespective of finger typos or spell errors.
[0074] While considerable emphasis has been placed herein on the preferred embodiments, it will be appreciated that many embodiments can be made and that many changes can be made in the preferred embodiments without departing from the principles of the invention. These and other changes in the preferred embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter to be implemented merely as illustrative of the invention and not as limitation.
ADVANTAGES OF THE PRESENT DISCLOSURE
[0075] The present disclosure provides a personalised Search Results based on localised dialects.
[0076] The present disclosure provides a better understanding chat-bot application.
[0077] The present disclosure provides an auto-spell correction / suggestion system for discovery of unique words.
,CLAIMS:1. A spell correction system, said system comprising:
a processor (202) that executes a set of executable instructions that are stored in a memory (204), upon which execution, the processor (202) causes the system to:
receive a word input from a keypad associated with the computing device (104);
receive a batch of words from a knowledgebase associated with a centralized server (112) operatively coupled to the computing device (104), the batch of words pertaining to a similar set of words to the received word input;
extract, by an artificial intelligence (AI) engine (214), a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of words with a predefined phonetic index, wherein the AI engine (214) is associated with the processor;
extract, by the AI engine (214), a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of said word having the predefined phonetic index;
compare, by a retrofitting engine (216), the first set of attributes with the second set of attributes of each said word in the received batch of words, wherein the retrofitting engine (216) is associated with the processor;
based on the compared first set of attributes with the second the second set of attributes of each said word, generate and map, by the retrofitting engine (216), a score to each compared first set of attributes with the second the second set of attributes of each said word, wherein
the score is generated based on the second set of attributes of each said word having the highest similarity in the predefined phonetic index with the first set of attributes; and,
generate, by the retrofitting engine (216), based on the mapped first set of attributes with the second the second set of attributes of each said word, a response to the received input word.
2. The system as claimed in claim 1, wherein the word input is a word of any or a combination of a predefined language, a mixture of one or more languages with the predefined language, a mixture of one or more dialects with the predefined language based on an accent of a native place associated with the one or more languages or the one or more dialects or a combination thereof, and wherein the processor is configured to consider each said word from the batch of words one at a time for providing the response.
3. The system as claimed in claim 1, wherein the response comprises any or a combination of correct spelling of the received input word and a detected phonetic word associated with one or more languages.
4. The system as claimed in claim 1, wherein the processor is configured to determine, by the retrofitting engine, one or more global phonetic words from the knowledgebase to enhance phonetic vocabulary based on the score generated for each second set of attributes.
5. The system as claimed in claim 3, wherein the processor is configured to inject, by the retrofitting engine, the one or more global phonetic words to one or more global knowledge graphs to drive any search engines or associated auto-correction systems.
6. The system as claimed in claim 3, wherein the second set of attributes comprises a plurality of phonetic variants for the input word received, wherein the plurality of phonetic variants further comprises a mixture of good context words, clean golden word set such as product catalogue, description, titles along with noisy content data such as localised phonetically similar sounding words from search, chat data, typos from fat finger syndrome.
7. The system as claimed in claim 3, wherein the processor is further configured to train, by the AI engine (214), a mapping of the first set of attributes with the second set of attributes of each said word of the bath of words for auto-spell correction or detection of phonetic words in the context of language understanding and extraction of information from the detected phonetic words.
8. The system as claimed in claim 7, wherein the processor is further configured to update the knowledgebase with latest phonetic knowledge to leverage spell correction-based information retrieval.
9. The system as claimed in claim 7, wherein the processor is further configured to pre-process the input word, wherein the pre-process comprises tokenisation, stop-word removal and n-gram phrase extraction.
10. The system as claimed in claim 7, wherein the processor is further configured to normalize one or more high scores from the generated scores associated with each second set of attributes with edit distance scores to remove any repeated character pairs phonetic noise.
11. The system as claimed in claim 7, wherein the processor is further configured to compute a set of character-to-character transition instructions from the first set of attributes, the set of character-to-character transition instructions comprising transition probability from a corrected character to a misspelled character associated with the input word.
12. A spell correction method, said method comprising:
receiving, by a processor (202), a word input from a keypad associated with the computing device (104), wherein the processor executes a set of executable instructions that are stored in a memory (204);
receiving, by the processor (202), a batch of words from a knowledgebase associated with a centralized server (112), the batch of words pertaining to a similar set of words to the received word input;
extracting, by an artificial intelligence (AI) engine (214), a first set of attributes from the received word input, the first set of attributes pertaining to a first set of characters of words with a predefined phonetic index, wherein the AI engine (214) is associated with the processor;
extracting, by the AI engine (214), a second set of attributes from each word in the received batch of words, the second set of attributes pertaining to a second set of characters of said word having the predefined phonetic index;
comparing, by a retrofitting engine (216), the first set of attributes with the second set of attributes of each said word in the received batch of words, wherein the retrofitting engine (216) is associated with the processor;
based on the compared first set of attributes with the second the second set of attributes of each said word, generating and mapping, by the retrofitting engine (216), a score to each compared first set of attributes with the second the second set of attributes of each said word, wherein the score is generated based on the second set of attributes of each said word having the highest similarity in the predefined phonetic index with the first set of attributes; and,
generating, by the retrofitting engine (216), based on the mapped first set of attributes with the second the second set of attributes of each said word, a response to the received input word.
13. The method as claimed in claim 12, wherein the word input is a word of any or a combination of a predefined language, a mixture of one or more languages with the predefined language, a mixture of one or more dialects with the predefined language based on an accent of a native place associated with the one or more languages or the one or more dialects or a combination thereof, and wherein the method further comprises the step of:
considering, by the processor (202), each said word from the batch of words one at a time for providing the response.
14. The method as claimed in claim 12, wherein the response comprises any or a combination of correct spelling of the received input word and a detected phonetic word associated with one or more languages.
15. The method as claimed in claim 12, wherein the method further comprises the step of:
determining, by the retrofitting engine (216), one or more global phonetic words from the knowledgebase to enhance phonetic vocabulary based on the score generated for each second set of attributes.
16. The method as claimed in claim 15, wherein the method further comprises the step of:
injecting, by the retrofitting engine, the one or more global phonetic words to one or more global knowledge graphs to drive any search engines or associated auto-correction methods.
17. The method as claimed in claim 12, wherein the second set of attributes comprises a plurality of phonetic variants for the input word received, wherein the plurality of phonetic variants further comprises a mixture of good context words, clean golden word set such as product catalogue, description, titles along with noisy content data such as localised phonetically similar sounding words from search, chat data, typos from fat finger syndrome.
18. The method as claimed in claim 12, wherein the method further comprises the step of:
training, by the AI engine (214), a mapping of the first set of attributes with the second set of attributes of each said word of the bath of words for auto-spell correction or detection of phonetic words in the context of language understanding and extraction of information from the detected phonetic words.
19. The method as claimed in claim 12, wherein the method further comprises the step of:
Updating, by the processor, the knowledgebase with latest phonetic knowledge to leverage spell correction-based information retrieval.
20. The method as claimed in claim 12, wherein the method further comprises the step of:
pre-processing the input word, wherein the pre-processing comprises tokenisation, stop-word removal and n-gram phrase extraction.
| # | Name | Date |
|---|---|---|
| 1 | 202121039439-STATEMENT OF UNDERTAKING (FORM 3) [31-08-2021(online)].pdf | 2021-08-31 |
| 2 | 202121039439-PROVISIONAL SPECIFICATION [31-08-2021(online)].pdf | 2021-08-31 |
| 3 | 202121039439-FORM 1 [31-08-2021(online)].pdf | 2021-08-31 |
| 4 | 202121039439-DRAWINGS [31-08-2021(online)].pdf | 2021-08-31 |
| 5 | 202121039439-DECLARATION OF INVENTORSHIP (FORM 5) [31-08-2021(online)].pdf | 2021-08-31 |
| 6 | 202121039439-FORM-26 [19-10-2021(online)].pdf | 2021-10-19 |
| 7 | 202121039439-Proof of Right [11-11-2021(online)].pdf | 2021-11-11 |
| 8 | 202121039439-ENDORSEMENT BY INVENTORS [25-08-2022(online)].pdf | 2022-08-25 |
| 9 | 202121039439-DRAWING [25-08-2022(online)].pdf | 2022-08-25 |
| 10 | 202121039439-CORRESPONDENCE-OTHERS [25-08-2022(online)].pdf | 2022-08-25 |
| 11 | 202121039439-COMPLETE SPECIFICATION [25-08-2022(online)].pdf | 2022-08-25 |
| 12 | 202121039439-FORM 18 [27-08-2022(online)].pdf | 2022-08-27 |
| 13 | Abstract1.jpg | 2022-09-12 |
| 14 | 202121039439-FER.pdf | 2024-01-25 |
| 15 | 202121039439-FER_SER_REPLY [11-04-2024(online)].pdf | 2024-04-11 |
| 16 | 202121039439-CORRESPONDENCE [11-04-2024(online)].pdf | 2024-04-11 |
| 17 | 202121039439-CLAIMS [11-04-2024(online)].pdf | 2024-04-11 |
| 18 | 202121039439-ABSTRACT [11-04-2024(online)].pdf | 2024-04-11 |
| 19 | 202121039439-FORM-8 [09-11-2024(online)].pdf | 2024-11-09 |
| 20 | 202121039439-FORM-26 [28-02-2025(online)].pdf | 2025-02-28 |
| 21 | 202121039439-US(14)-HearingNotice-(HearingDate-10-09-2025).pdf | 2025-08-18 |
| 22 | 202121039439-Correspondence to notify the Controller [04-09-2025(online)].pdf | 2025-09-04 |
| 23 | 202121039439-Written submissions and relevant documents [25-09-2025(online)].pdf | 2025-09-25 |
| 24 | 202121039439-FORM-26 [25-09-2025(online)].pdf | 2025-09-25 |
| 25 | 202121039439-Annexure [25-09-2025(online)].pdf | 2025-09-25 |
| 26 | 202121039439-US(14)-HearingNotice-(HearingDate-15-12-2025).pdf | 2025-11-19 |
| 1 | 9439E_18-01-2024.pdf |