FORM 2
THE PATENTS ACT, 1970
(39 of 1970) THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10; rule 13)
TITLE OF THE INVENTION
“INBUILT MULTILINGUAL VOICE ASSISTANT SYSTEM WITH AUTOMOBILE DIAGNOSTIC ALERTS AND FEEDBACKS AND
METHOD THEREOF”
APPLICANT
TATA MOTORS LIMITED
Bombay House, 24 Homi Mody Street,
Hutatma Chowk, Mumbai 400 001,
Maharashtra, India; an Indian Company.
PREAMBLE TO THE DESCRIPTION
The following specification particularly describes the invention and the manner in
which it is to be performed

PRIORITY DETAILS
The Present application claims priority from an Indian Provisional application filed on 31st January 2022 filed at The Indian Patent Office.
TECHNICAL FIELD
Present disclosure, in general, relates to the field of automobiles. Particularly, but not exclusively, the present disclosure relates to an inbuilt multilingual voice assistant system with automobile diagnostic alerts and feedbacks and method thereof.
BACKGROUND OF THE INVENTION
Voice assistant system in the conventional vehicle system is implemented for better user experience which is further connected to a user or plurality of users via a communicable device. The system aids the user to communicate with communicable device using voice and also responds to the voice commands of the user. A majority of entities involved in manufacturing and sale of consumer electronics products are merging the products with voice activated systems as a user interface which have generated better outputs than conventional interactive systems involving touch based panel or button/switch operated methods.
One of the major concerns in an automotive industry is how to make the vehicle safe for their users. With many advanced driver-assistance systems features coming into picture, Vehicles are becoming advanced in terms of safety and awareness features for drivers. Drivers are getting alerts on instrument cluster, using infotainment system and engaging with multiple interfaces in vehicle, which can be distracting.
The voice assistant systems are providing better outputs because they are helping the users to interact with the systems verbally in comparison with the conventional method of action base inputs where touch based interaction is involve. A plurality of voice assistant manufacturers have developed their own voice assistants having

their unique names. While the voice assistant system provide the assistance through server linked system wherein the server is enabled to assist the system to help in processing of the voice assistant systems remotely. The server contains the repository of modules which help in the proper working of the voice assistant system. This limitation of internet connectivity or the network connectivity causes a major drawback for voice assistant system which especially implemented in automobile as they travel in remote places where the internet signals are not reachable and communication through internet is very difficult. Thus, such there is a requirement of a voice assistant system within automobiles, which can overcome such long-standing problems of reliability on server-based systems.
Furthermore, vehicle controls has to be operated physically. There is no audio for vehicle level warnings & alerts. This data may be available on Instrument cluster.
Furthermore, the conventional voice assistant systems operate on English language only or in one language only. The conventional voice assistant system cannot communicate a sentence which involves the mixing of two or more languages in a sentence. This is a long standing requirement as many citizens of country which has a diverse set of languages communicate with the mixing of at least two languages in one sentence. Many users understand properly clearly when communication is provide by the voice assistant system in English language mixed with the local language. The mixing of at least two languages provide better user experience as it helps the user to understand a sentence properly and accurately.
The present disclosure is directed to overcome one or more limitations stated above or any other limitations associated with the conventional configuration of voice assistant systems for an automobile.

SUMMARY OF THE DISCLOSURE
One or more shortcomings of the conventional system or method are overcome, and additional advantages are provided through the provision of the method as claimed in the present disclosure.
Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed disclosure.
In one non-limiting embodiment of the disclosure, Disclosed is a personal multilingual voice assistant system in a vehicle, which can provide assistance to a user or a passenger for assisting in a plurality of activities. The assistance is provided essentially for the following, but not limited to:
• Changing the parameter within a vehicle comprising sound levels, HVAC parameters and the like.
• Providing alerts and solution to the user to for indicating any failure of subsystem of the vehicle and how to solve the same.
• Providing alerts to the user for any situations which can be risky for the user.
Furthermore, the personal multilingual voice assistant system is enabled to provide assistance with the mixing of at least two languages intermixed with each other. The intermixing of the languages may comprise, but is not limited to:
• Hindi and English
• Marathi and English
• Kannada and English
• Punjabi and English
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES
The novel features and characteristic of the disclosure are set forth in the appended claims. The disclosure itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying figures. One or more embodiments are now described, by way of example only, with reference to the accompanying figures wherein like reference numerals represent like elements and in which:
Figure 1 illustrates a structural layout of the elements of a personal voice assistance system (100), in accordance with an embodiment of the present disclosure.
Figure 2 illustrates a hardware (200) of the personal voice assistance system, in accordance with an embodiment of the present disclosure.
The figure depicts embodiments of the disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the system and method for dampening vibrations in a vehicle without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION
The foregoing has broadly outlined the features and technical advantages of the present disclosure in order that the description of the disclosure that follows may be better understood. Additional features and advantages of the disclosure will be described hereinafter which form the subject of the disclosure. It should be appreciated by those skilled in the art that the conception and specific embodiments disclosed may be readily utilized as a basis for modifying or designing other system for carrying out the same purposes of the present disclosure. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the disclosure. The novel features which are believed to be characteristic of the disclosure, as to its organization, together with further

objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present disclosure.
In the present document, the word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or implementation of the present subject matter described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described below. It should be understood, however that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the scope of the disclosure.
The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusions, such that a system that comprises a list of components does not include only those components but may include other components not expressly listed or inherent to such mechanism. In other words, one or more elements in the device or mechanism proceeded by “comprises… a” does not, without more constraints, preclude the existence of other elements or additional elements in the mechanism.
Disclosed is a personal multilingual voice assistant system in a vehicle, which can provide assistance to a user or a passenger for assisting in a plurality of activities. The assistance is provided essentially for the following, but not limited to:
• Changing the parameter within a vehicle comprising sound levels, HVAC parameters and the like.

• Providing alerts and solution to the user to for indicating any failure of subsystem of the vehicle and how to solve the same.
• Providing alerts to the user for any situations which can be risky for the user.
Furthermore, the personal multilingual voice assistant system is enabled to provide assistance with the mixing of at least two languages intermixed with each other. The intermixing of the languages may comprise, but is not limited to:
• Hindi and English
• Marathi and English
• Kannada and English
• Punjabi and English
Referring Fig. 1, a structural layout of the elements of a personal voice assistance system (100), is disclosed is accordance with various embodiments of the present subject matter.
The Vehicle Personal assistant with Voice Recognition Intelligence, which takes the user input in form of audio file and convert it to text, process it and returns the output in various forms like action to be performed or the search result is dictated to the end user. In addition, this proposed system can change the way of interactions between end user and the vehicle. Complete eco-system of Multilingual Voice assistant is designed in such a way that the end user will be able to interact with vehicle better and use services in efficient way.
In-Vehicle Voice assistance system aims to remove the distraction of looking down at our phone or around cabin to access various information and Functions. Voice technology allows the machine to turn the speech signal into text or commands through the process of identification and understanding the intent of a sentence & makes the function of natural communication. Voice technology involves many fields of psychology, linguistics, computer science, signal processing, AI (artificial intelligence) and is also related to the person's body language. The ultimate goal is to achieve natural language communication between a person and machine.

The use of speech or voice technology in a work environment to facilitate a variety of tasks. The main highlight of said system is the fact that it caters to untapped market of Commercial vehicle, which is presently testing the waters with newly equipped infotainment devices. Adding voice technology with multiple options of languages such as commonly used Hindi, English or regional languages like Tamil, Bengali and Marwari will make a great impact on user experience margin. This feature will give the vehicle a unique edge in the competing market.
An intelligent personal assistant can help someone with basic tasks. They often understand natural language and can help with things like creating meeting requests, making phone calls, and sharing the weather forecast. Intelligent personal assistants have access to a large amount of information on a device or online, which enables them to perform simple tasks. Technology is constantly advancing and changing, and the voice assistant market is progressing along with it. Speech or voice technology, in the form of speech recognition, is used in a variety of different environments to facilitate the ease of completing various day-to-day tasks. Voice directed system is one way of performing tasks and adding multiple languages gives it a more user-friendly touch and convenience.
The system comprises of different structural and functional units which completes the whole process. The main aspect of it is to capture the human speech and work upon it to make a meaningful conversation with the user. The basic function is as follows: The audio signals are processed by the computing device to identify the languages being spoken. Given the system is trained for certain number of languages, it will identify the language being spoken by the user with the help of NLP. Once the system identified the languages it then proceeds with determining the response for user in the same language that is detected. At any given time, a user can change the language by simply uttering sentences in a different language. For example, driver in a vehicle saying “Gaana chalao” , the system will detect language as Hindi and will proceed with its response in Hindi i.e. “Aap konsa Gaana

chalana Chahenge”. User at this moment can speak in English or any other language and the system will respond in the changed language. User doesn’t have to utter fixed set of commands, with intelligent system any utterance with distinct unambiguous words can be detected and hence making system more friendly and comfortable for user.
Voice assistant system is embedded in a NXP based I.mx processor topped up with intended applications. User can use a wake up word (in our case it’s “Hey/Hello TATA”) to activate the system and ask it any question related to vehicle like "what is the current coolant temperature?" or even to perform tasks like " Turn on Blower/ turn on Wiper" etc. This speech is recorded and converted to a text file, which is fed to ASR (Automated Speech recognizer). Upon understanding the intent of the sentence, a Corresponding action/response is generated through dialog manager based on the user experience intent defined. Dialog manager is designed to map all user utterances to relevant action or response in order to provide seamless experience to the user. The system is also connected with Multiple ECUs in the vehicle and give information based on the data provided by these ECUs. Once a proper response is decided the TTS (Text to speech) handler will announce the result via speaker. This product becomes handy once the user gets to interact with it and explore many possibilities.
The main highlight of Voice assistance system is the fact that it caters to untapped market of Commercial vehicle which is presently testing the waters with newly equipped infotainment devices. Adding voice technology with multiple options of languages such as commonly used Hindi, English or regional languages like Tamil, Bengali and Marwari etc. will make a great impact on user experience & vehicle safety. This feature will give the vehicle a unique edge in the competing market.
To induce a wake up call for the system, the user may use “Hey Tata” as the wake word in our commercial vehicle’s system.

Wake words rely on a special algorithm that is always listening for a particular word or phrase so that a phone, smart speaker, or something else can begin communicating with a server to do its job. Wake words need to be long enough to be distinct, easy for a human to speak, and simple for a machine to recognize.
Voice assistants don’t really understand what one is saying - they just listen for their wake word and then begin communicating with a server to complete a task. NLP (Natural language processing) is a form of artificial intelligence that helps technology interpret human language.
Referring Fig. 2, a hardware (200) of the personal voice assistance system, in accordance with an embodiment of the present disclosure.
The hardware 200 may comprise a pigtail cutout, a USB and cutout, a PTT button on PCB, a mic and speaker connected over the cover of the hardware 200.
The unique features may comprise the following:
• Provided Voice Interface as one alternative to overcome the driver’s distractions problems.
• Simple Single interface with user friendly Dialogs design and implementation.
• Wake up Work or Push to talk button to awake the system.
• Understanding multiple regional languages and responding as per the language detected by the system say Hindi, Bengali or Tamil etc.
• Integration with In-Vehicle ECUs to control Cabin Functions.
• Vehicle level information like Coolant temperature, vehicle speed limit etc.
• Auto language changes as per user voice command.
The advancement further comprise Voice technology will provide ultimate breakthrough to drivers on their Long- haul journeys, to technicians performing vehicle diagnosis and to fleet owner for accessing complete know how of the vehicle by just asking about it.

The interactive, multilingual, auto language setting as per voice command and smart assistant will enhance the customer's overall experience and engagement with the vehicle & safety.
Apart from the technology itself, the Design of Dialog Manager, user dialogs,
system’s responses, error & exception handling, all this constitutes to a very subjective area
in this development. It depends/varies from person to person on how to make a difference
in creating an exceptional user experience. With many possible use cases & sentence syntax
one can go to lengths in order to define unique action/responses from the system
Equivalents
With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
It will be understood by those within the art that, in general, terms used herein, are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding the description may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to

imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should typically be interpreted to mean "at least one" or "one or more"); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B."
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not

intended to be limiting, with the true scope and spirit being indicated in the description.
Referral Numerals:

Description Referral numerals
Personal Voice Assistance System 100
Physical Hardware 200

WE CLAIM:
1. A multi-lingual voice assistance system (100) for an automobile, the system (100) comprising:
an input-output interface (101);
one or more microphones (102) and speakers (103);
a memory (104) comprising a data pertaining to a plurality of language and a plurality of modules which enable a processor to establish a two way communication, learn and develop multi-lingual speeches;
the processor (105) coupled with the input-output interface and the one or more microphone and speakers, further capable of executing one or more steps by accessing the modules and data from the memory, the steps comprising;
monitoring, via at least one microphone, one or more multi-lingual speeches of the one or more users within the vehicle;
detecting, via at least one microphone, a multi-lingual activation speech request by the one or more users within the monitored speeches and conversation for initiation of voice assistance;
activating the components of the system (100) by supplying power and establishing a communicative connection with one or more vehicle sub – units (106);
analyzing the multi-lingual speeches by one or more users by natural language processing to detect one or more commands present in the speeches of the one or more users to perform a function by one or more vehicle sub – units;
producing a conformity by way of speech of the commands providing by the users in a multi-lingual speech output;
establishing a multi-lingual communication between the one or more users to formalize a signal to be sent to the required sub-unit of the vehicle for the complete execution of the command.

2. The multi-lingual voice assistance system (100) of claim 1, wherein the system (100) is enabled to monitor the speeches of predefined users by the way of voice recognition.
3. The multi-lingual voice assistance system (100) of claim 1, wherein one or more microphone is enabled to monitor multiple speeches by the multiple users.
4. The multi-lingual voice assistance system (100) of claim 1, wherein the activation speech is pre-defined by the user through input-output interface and stored in the memory.
5. The multi-lingual voice assistance system (100) of claim 4, wherein the activation speech is defined by the implementation of the multi-lingual voice assistance.
6. The multi-lingual voice assistance system (100) of claim 1, wherein detection of the speech in multiple languages is enabled by accessing the data of the multiple languages stored in the memory.
7. The multi-lingual voice assistance system (100) of claim 1, wherein auto tuning is enabled by the mixing of at least two languages by using natural language programming and machine learning methodology.
8. The multi-lingual voice assistance system (100) of claim 7, wherein artificial intelligence is implement in updating the language data, developing the speech mixing of the two languages with updated data.
9. The multi-lingual voice assistance system (100) of claim 7, wherein the processor is enabled to develop new mixed language speeches from the monitored and analyzed speeches of the user.

10. A method of provision of a multi-lingual voice assistance to one or more users in a vehicle, the method comprising:
monitoring, via a processor, one or more multi-lingual speeches of the one or more users within the vehicle;
detecting, via the processor, a multi-lingual activation speech request by the one or more users within the monitored speeches and conversation for initiation of voice assistance;
activating, via the processor, a plurality of components of a multi-lingual voice assistance system (100) by supplying power and establishing a communicative connection with one or more vehicle sub – units;
analyzing, via the processor, the multi-lingual speeches by one or more users by natural language processing to detect one or more commands present in the speeches of the one or more users to perform a function by one or more vehicle sub – units;
producing, via the processor, a conformity by way of speech of the commands providing by the users in a multi-lingual output;
establishing, via the processor, a multi-lingual communication between the one or more users to formalize a signal to be sent to the required sub-unit of the vehicle for the complete execution of the command.