Description:TECHNICAL FIELD

[0001] The present invention relates to the field of Translation Systems, and more particularly, the present invention relates to the automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations.

BACKGROUND ART
[0002] The following discussion of the background of the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was published, known, or part of the common general knowledge in any jurisdiction as of the application’s priority date. The details provided herein the background if belongs to any publication is taken only as a reference for describing the problems, in general terminologies or principles or both of science and technology in the associated prior art.
[0003] Without communication, human existence would be severely limited. Sign language is the language used by those who are hard of hearing. Deaf people rely on sign language as their main means of communication. Visual perception is the primary means of interpretation and translation. Sign language relies on the shape, placement, and motion of the hands as its fundamental characteristics. Many people who come into touch with the deaf, including parents of deaf children, instructors of deaf students, and others who are hard of hearing, use sign language.
[0004] The worldwide prevalence of hearing loss is estimated at 466 million, with 34 million of those people being teens. Around the globe, people utilize a wide variety of sign languages. When compared to spoken languages, sign language is very uncommon. Indian Sign Language (ISL) is the name given to India's sign language. Developed countries have an extremely low number of schools catering to deaf students. People with hearing loss have a high unemployment rate in industrialized countries. The letters of the English alphabet are represented in Indian Sign Language (ISL) via the use of fingerspelling. The two-handed method is used in ISL, although it may also be done one-handed.
[0005] A vital component of learning sign language is fingerspelling, even if it is used less often in informal signing. Signing may be quite challenging for someone new to the language unless they have access to a trained tutor. Sign language may be a challenging language for beginners to learn on their own, without the guidance of an experienced practitioner. There is a severe lack of human resources in this field. India has extremely few interpreters, with about 250 available.
[0006] This equates to around one for every 20284 deaf people in the country. The high expense that parents face when trying to educate their deaf children, difficulties in understanding spoken and written language, and instructors' lack of sign language expertise are all factors that make it difficult for deaf children to learn sign language. Sign language, like many spoken languages, has several regional varieties. For the deaf community, this is a major communication obstacle.
[0007] In light of the foregoing, there is a need for Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations that overcomes problems prevalent in the prior art associated with the traditionally available method or system, of the above-mentioned inventions that can be used with the presented disclosed technique with or without modification.
[0008] B. R. Reddy et al. have proposed the design and implementation of a model that translates native Indian regional language (Tamil) into text and then matches it to sign language animations using a video animation collection has been suggested and is to be implemented by B. R. Reddy et al. Two deep learning models, Bi LSTM and LSTM (Long Short-Term Memory), as well as the Google API, are used to transform the voice into text. Following this, the text is converted to sign language by use of the Indian Sign Language (ISL) datase t. The suggested models were able to get 45% accuracy with LSTM, 65% accuracy with Bi-LSTM, and 95% accuracy with Google API, in that order.
[0009] Sharma et al. have presented a system that takes text and audio as input and compares it to the films stored in the authors' database. If they are a good match, it proceeds through tokenization and lemmatization to produce signs that correspond to each other according to the norms of Indian Sign Language grammar. Natural language processing powers the system's backbone features including tokenization, parsing, lemmatization, and part-of-speech tagging1. To decipher the user's intended statement using Indian Sign Language, the interpreter described in the article use hybrid CNN models to identify various sign motions. No matter the circumstances, the system can attain an accuracy of 80 –95%.
[0010] All publications herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies, and the definition of that term in the reference does not apply.

OBJECTS OF THE INVENTION

[0011] The principal object of the present invention is to overcome the disadvantages of the prior art by providing Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations.
[0012] An object of the present invention is to provide Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations, wherein the system has received validation from the hearing-impaired individuals of deaf schools in Patiala and specialists in ISL.
[0013] Another object of the present invention is to provide Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations that offers a valuable resource for teachers in Punjab’s deaf schools, parents, and relatives of the deaf community, aiding them in learning sign language.
[0014] Yet another object of the present invention is to provide Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations, wherein the system helps to disseminate information to deaf people in public utility p laces like railways, banks, hospitals, etc.
[0015] The foregoing and other objects of the present invention will become readily apparent upon further review of the following detailed description of the embodiments as illustrated in the accompanying drawings.

SUMMARY OF THE INVENTION
[0016] The present invention relates to Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations.
[0017] Sign language is a vital communication bridge for the hearing-impaired community, enabling them to express their thoughts and emotions. The realm of computer science has made commendable progress in deciphering Indian Sign Language (ISL), leading to the development of various text-to-ISL translation systems. However, each system comes with its unique set of advantages and limitations. Our research presents a pioneering approach to translating Punjabi text into ISL, achieving an impressive accuracy of 83% for three 3,000 elementary sentences and 70% for five hundred complex and compound sentences. The bilingual dictionary for Punjabi text to Indian Sign Language contains approximately 7000 words. This innovative system has received validation from the hearing-impaired individuals of deaf schools in Patiala and specialists in ISL. It offers a valuable resource for teachers in Punjab’s deaf schools, parents, and relatives of the deaf community, aiding them in learning sign language. This system will help to disseminate information to deaf people in public utility p laces like railways, banks, hospitals, etc.
[0018] The objectives of the present invention are: 1. To understand Indian Sign Language (ISL) and its Grammar. 2. To coin various ISL grammar rules for Punjabi text. 3. To develop a bilingual dictionary for Punjabi text to the Indian Sign Language translation system. 4. To integrate various Punjabi NLP tools like Stemmer, Part of Speech tagger, Wordnet, Punjabi to English transliterator, etc needed for developing the translation system. An effort will be to use already existing tools and modify them as per requirements. 5. To develop an algorithm for translating Punjabi text to Indian Sign Language using synthetic animations.
[0019] While the invention has been described and shown with reference to the preferred embodiment, it will be apparent that variations might be possible that would fall within the scope of the present invention.

BRIEF DESCRIPTION OF DRAWINGS
[0020] So that the manner in which the above-recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may have been referred by embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
[0021] These and other features, benefits, and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
[0022] Figure 1: Architecture of the Developed System;
[0023] Figure 2: Web Application View of Punjabi Indian Sign Language Dictionary using Synthetic Animations;
[0024] Figure 3: Web Application View of Stemmer used in the Developed System;
[0025] Figure 4: Web application view of the Punjabi text to Indian Sign Language converter;
[0026] Figure 5: Punjabi text to Punjabi Indian Sign Language text conversion after applying ISL grammar rules; and
[0027] Figure 6: Web Application view of the Part of Speech Tagger used in the developed system.

DETAILED DESCRIPTION OF THE INVENTION
[0028] While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and the detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim.
[0029] As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein are solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers, or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles, and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
[0030] In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element, or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
[0031] The present invention is described hereinafter by various embodiments with reference to the accompanying drawing, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, several materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
[0032] The present invention relates to Automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations. The translation system boasts a user-friendly interface, ensuring seamless interaction for users solely through internet connectivity, eliminating the need for any specialized hardware or software. • The advantages of the translation system are multifold. Firstly, it operates with remarkable speed, providing swift and efficient translation services. Furthermore, its capability to translate text in real-time sets it apart, catering to the immediacy often required in communication. • A noteworthy application of the system is witnessed in Punjab, where school teachers and parents of deaf children utilize it to learn sign language. This initiative not only fosters inclusivity but also contributes significantly to the educational support system for the deaf community. • Expanding its utility, the system can be seamlessly integrated into public spaces, enhancing accessibility for the deaf community. This broad implementation aligns with our commitment to making communication more inclusive and accommodating for everyone. • An essential feature of our system lies in the utilization of SiGML files to digitally represent signs. Notably lightweight, these files surpass the bulk of human videos, facilitating easy sharing over the internet. This streamlined sharing process ensures widespread accessibility, ultimately contributing to a more connected and inclusive society.
[0033] Our research methodology for translating Punjabi text into Indian Sign Language (ISL) is divided into five stages: Stage 1 Pre-processing: The initial stage involves cleaning the input sentence by removing extra spaces and non-Punjabi characters, except numerals. Stage 2: Part of Speech (POS) Tagging: In this stage, a POS tagger system is employed to assign a POS tag to each word in the input sentence. The tags encompass various parts of speech such as nouns, pronouns, verbs, adjectives, conjunctions, and their respective subcategories. An existing POS tagger for Punjabi is utilized in our translation system. (see Figure 6) Stage 3 Elimination: ISL sentences are composed only of meaningful words, while certain words like linking verbs, suffixes, and articles are typically omitted. After applying Punjabi grammar rules, the ISL sentence retains all words from the original sentence but in a differ ent order. Words not required in sign languages, such as articles, conjunctions, prepositions, modals, possessive endings, interjections, and certain adverbs, are identified and eliminated. (see Figure 5) Stage 4 Stemming: Stemming, the process of generating morphological variants of a root/base word, is carried out in this stage. ISL uses root words in sentences, devoid of suffixes, gerunds, or inflections. If a word in the ISL sentence is not a root word, it is converted into one using a Punjabi stemmer. (see Figure 3) Stage 5 Retrieval of the SiGML File: The output from the fourth stage serves as the input for this stage. The SiGML file corresponding to the filtered Punjabi text is retrieved from the database (see Figure 2). If the SiGML file for a Punjabi word is unavailable in the bilingual dictionary (Punjabi to Indian Sing Language Dictionary), synonyms of the word are sourced from the synonyms database . If the newly replaced word also does not exist in the dictionary, the Punjabi-to English transliteration is used to find English equivalents for the Punjabi word, and the avatar performs fingerspelling for the resulting transliteration word.
[0034] Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the 5 embodiments shown along with the accompanying drawings but is to be providing the broadest scope consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention and appended claims.
, Claims:We Claim:

1) An automation translation system for Punjabi text to Indian sign language (ISL) using synthetic animations, the system uses a translation method that comprises five stages: Stage 1 Pre-processing; Stage 2: Part of Speech (POS) Tagging; Stage 3 Elimination; Stage 4 Stemming; and Stage 5: Retrieval of the SiGML File.

2) The system as claimed in claim 1, wherein the pre-processing involves cleaning the input sentence by removing extra spaces and non-Punjabi characters, except numerals.

3) The system as claimed in claim 1, wherein the Part of Speech (POS) Tagging stage involves a POS tagger system is employed to assign a POS tag to each word in the input sentence.

4) The system as claimed in claim 3, wherein the tags encompass various parts of speech such as nouns, pronouns, verbs, adjectives, conjunctions, and their respective subcategories.

5) The system as claimed in claim 1, wherein the ISL sentences are composed only of meaningful words, while certain words like linking verbs, suffixes, and articles are typically omitted and After applying Punjabi grammar rules, the ISL sentence retains all words from the original sentence but in a differ ent order.

6) The system as claimed in claim 1, wherein the process of generating morphological variants of a root/base word, is carried out in this stage.

7) The system as claimed in claim 1, wherein the ISL uses root words in sentences, devoid of suffixes, gerunds, or inflections.

8) The system as claimed in claim 1, wherein if a word in the ISL sentence is not a root word, it is converted into one using a Punjabi stemmer; wherein the output from the fourth stage serves as the input for this stage.

9) The system as claimed in claim 1, wherein if the SiGML file for a Punjabi word is unavailable in the bilingual dictionary (Punjabi to Indian Sing Language Dictionary), synonyms of the word are sourced from the synonyms database; wherein if the newly replaced word also does not exist in the dictionary, the Punjabi-to English transliteration is used to find English equivalents for the Punjabi word, and the avatar performs fingerspelling for the resulting transliteration word.