DESC:FORM-2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10 and rule 13)

Title: A SYSTEM AND METHOD FOR SCENARIO GENERATION FOR AUTONOMOUS VEHICLE

APPLICANT DETAILS:
(a) NAME: SIMDAAS AUTONOMY PRIVATE LIMITED
(b) NATIONALITY: Indian
(c) ADDRESS: H NO 528 NANKARI BAGIA PRADHAN GATE IIT KANPUR, IIT,
KALYANPUR, KANPUR NAGAR, UTTAR PRADESH

PREAMBLE TO THE DESCRIPTION:
The following specification (particularly) describes the nature of the invention (and the manner in which it is to be performed):
A SYSTEM AND METHOD FOR SCENARIO GENERATION FOR AUTONOMOUS VEHICLE
FIELD OF THE INVENTION:
The present invention relates to the field of autonomous vehicles. Specifically, the present invention provides a system and method for scenario generation for autonomous vehicles using Large Language Models (LLM).

BACKGROUND OF THE INVENTION:
The following background discussion includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
The critical problem faced by autonomous vehicles is regarding edge case detection. In the existing methods, conceptualizing and creating scenarios, including edge case scenarios for training autonomous vehicles is a protracted and labor-intensive process. Whereas the alternative of automating it using LLMs is highly discouraged due to its immense error rates. There are several prior art which tried to solve the above mentioned problem for example, US10345811B2 discloses method and apparatus for evaluating driving performance under a plurality of driving scenarios and conditions. More specifically, the application teaches a method and apparatus for testing a driving scenario repetitively while altering a parametric variation, such as fog level, in order to evaluate driving system performance under changing conditions.
Further, US20200409369A1 discloses autonomous vehicle service assignment simulation using predefined scenarios. In particular, a computing system comprising one or more computing devices can obtain data associated with a simulated autonomous vehicle within a simulation environment based at least in part on the predefined scenario. The computing system can initiate a simulation of the predefined scenario using the data associated with the simulated autonomous vehicle to perform the predefined scenario within the simulation environment. The computing system can receive one or more simulated events to attempt to complete the predefined scenario. The computing system can determine whether the autonomous vehicle has successfully completed the predefined scenario.
In another document, US20210286924A1 discloses an autonomous vehicle which is processed to generate augmented data. The augmented data describes an actor in an environment of the autonomous vehicle, the actor having an associated actor type and an actor motion behavior characteristic. The augmented data may be varied to create different sets of augmented data. The sets of augmented data can be used to create one or more simulation scenarios that in turn are used to produce machine learning models to control the operation of autonomous vehicles.
The drawback in the existing method lies with the substantial time required to conceive and generate these scenarios, both in terms of simulation and real-life application.
The present invention provides a method and system to overcome this challenge by significantly enhancing the efficiency and diversity of the entire process, while generating similar quality of precise, relevant and valid scenarios.

OBJECT/S OF THE INVENTION:
An object of the present invention is to provide a system and method for scenario generation for autonomous vehicles.
Another object of the present invention is to provide a system and method for scenario generation for autonomous vehicles using LLM.
Another object of the present invention is to provide a system and method for scenario generation for autonomous vehicles using LLM by prompting techniques, iterative validation procedures, and context-aware refinement strategies.
Another object of the present invention is to provide a method that optimizes the parameters that define a scenario, i.e., static and dynamic actors, using prompts generated with the help of LLM and predefined prompts that are relevant and useful in each phase of the pipeline.
Another object of the present invention is to provide a system and method for autonomously generating additional edge cases, expanding the range of scenarios beyond the limitations of human ideation while tackling the erroneous outputs of LLM.

SUMMARY OF THE INVENTION:
In an aspect the present invention provides a system for scenario generation for autonomous driving simulation comprising:
a) a repository unit configured to generate a diverse set of textual descriptions for various driving scenarios;
b) a LLM unit is configured to receive queries from the repository, where the queries comprise the generated textual description with a pre-defined prompt structure for both a static and a dynamic element for scenario development;
c) a python unit is configured to receive a python dictionary (intermediate output) from the LLM unit;
wherein LLM unit is configured to make changes to information stored in the python unit based on the new information and the said information is converted into an OpenDRIVE and an OpenSCENARIO files.
In an embodiment, the static element comprises a road network, a roadside objects and a crossing.
In an embodiment, the dynamic elements comprise vehicles, pedestrians and other moving objects such as animals.

In an embodiment, the repository unit combines the generated description with a pre-defined prompt structure to create specific queries for the LLM unit.
In an aspect the present invention provides a method for scenario generation for autonomous vehicle comprising:
a) generating a diverse set of textual descriptions for various driving scenarios by a repository;
b) receiving queries from the repository by a LLM unit, where the queries comprise the generated textual description with a pre-defined prompt structure for both a static and a dynamic element for scenario development; and
c) receiving python dictionary (intermediate output) from the LLM unit;
wherein LLM unit is configured to make changes to information stored in the python unit based on the new information and the said information is converted into an OpenDRIVE and an OpenSCENARIO files.

DETAILED DESCRIPTION OF DRAWINGS:
The advantages and features of the present disclosure will become better understood with reference to the following detailed description and claims taken in conjunction with the accompanying drawing, in which:
Fig. 1 illustrates the flow chart of the present invention.

DETAILED DESCRIPTION OF PRESENT INVENTION:
The foregoing descriptions of specific embodiments of the present disclosure have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiment was chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.
The terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item.
The terms “having”, “comprising”, “including”, and variations thereof signify the presence of a component.
The present invention provides a system and method for scenario generation for autonomous vehicles.
In an embodiment, the present invention provides a system and method for scenario generation for autonomous vehicles using LLM.
In an embodiment, the present invention provides a system and method for scenario generation for autonomous vehicles using LLM by prompting techniques, iterative validation procedures, and context-aware refinement strategies.
In an embodiment, the present invention provides a method that optimizes the parameters that define a scenario, i.e., static and dynamic actors, using prompts generated with the help of LLM and predefined prompts that are relevant and useful in each phase of the pipeline.
In an embodiment, the present invention provides a system and method for autonomously generating additional edge cases, expanding the range of scenarios beyond the limitations of human ideation while tackling the erroneous outputs of LLM.
In another embodiment, the present invention system and method generate validated scenario descriptions for autonomous driving simulations. These descriptions employ validated data and develop rules and methods to accurately represent both the scenario itself and its environment. The present invention leverages the extensive knowledge and creative capabilities of Large Language Models (LLMs) to produce a diverse range of potential scenarios. These methods and systems of the present invention ensure that the generated scenarios adhere to established data validation protocols, conform to pre-defined scenario rules and constraints and maintain internal consistency and avoid logical inconsistencies.
In another embodiment, the present invention system and method mitigate the limitations of LLMs and enables the generation of highly accurate and comprehensive driving simulations for autonomous vehicle development.
In another embodiment, the present invention essential process begins with generating a diverse set of textual descriptions for various driving scenarios using a developed repository. For each scenario, the system employs a unique approach. The system combines the generated description with a pre-defined prompt structure to create specific queries for the LLM. These queries, tailored for both static and dynamic elements, aim to extract crucial information needed for scenario development. Static elements encompass the road network, roadside objects, crossings, and more. Predefined prompts like "number of roads" or "lane width" efficiently gather this data. For dynamic elements like vehicles and pedestrians, the system leverages the LLM's creativity. Using prompts like "relevant information about vehicles," the LLM identifies and extracts pertinent details. Further, all gathered data, both static and dynamic, is meticulously stored in a dedicated database for further processing.
In an embodiment, the next essential process step of the present invention is validating and refining extracted data. The present invention applies two-step validation to ensure the extracted data reflects the intended scenario. The method utilizes a technique similar to prompt creation. By combining LLM-generated prompts with predefined queries the system assesses the preciseness of the data. Any discrepancies between the extracted information and the original scenario description or established constraints trigger data modifications to ensure accuracy.
Thereafter, the system of the present invention performs a comprehensive scenario validation. The system applies the hybrid prompting technique, it queries the LLM to verify if the extracted data paints a consistent picture with the textual description, then checks for any inconsistencies and makes final modifications before incorporating it into the scenario description file.
In an embodiment, the process of scenario generation for an autonomous vehicle in the present invention includes steps of generating a diverse set of textual descriptions for various driving scenarios by a repository unit. Thereafter the repository unit queries the LLM unit for the generated textual description with a pre-defined prompt structure for both a static and a dynamic element for scenario development. After the queries is completed the repository unit receives python dictionary (intermediate output) from the LLM unit.
In an embodiment, in the present invention the LLM unit is configured to make changes to information stored in the python unit based on the new information and the said information is converted into an OpenDRIVE and an OpenSCENARIO files.
In an embodiment, in the present invention the repository unit applies a hybrid prompting technique by querying the LLM unit to verify if the extracted data paints a consistent picture with the textual description.
In an embodiment, in the present invention repository unit in conjunction with LLM unit ensures context-aware refinement by factoring in both the original scenario description and predefined constraints, resulting in descriptions that are not only accurate but also relevant to the specific scenario.
In another embodiment, the present invention system and method blends the creativity of LLM and methods to foster efficient and reliable data extraction, minimizing errors and inconsistencies. Furthermore, the iterative validation process, encompassing both data preciseness and scenario consistency checks, elevates the overall accuracy and internal logic of the generated descriptions. The method of the present invention also ensures context-aware refinement by factoring in both the original scenario description and predefined constraints, resulting in descriptions that are not only accurate but also relevant to the specific scenario.
As shown in the figure 1, the workflow of the present invention is disclosed. The system and method of the present invention provides the option to the user for selecting the field in which user want to generate scenario both for the static as well as the dynamic scenario. In response to the user query, LLM unit generates description for N such scenario, thereafter the system selects the scenario from it which is not yet processed and based on the scenario the LLM unit generates the prompts and ready with the answer of the road scenario for the generated prompt. The system selects the prompt which has not been used yet and LLM unit uses that prompt to generate the python dictionary and based on the python dictionary the LLM unit make changes to information stored in the python dictionary based on the new information. The final information is converted into an OpenDRIVE and an OpenSCENARIO files and saved in the repository unit. The said OpenDRIVE and the OpenSCENARIO files are more accurate to be used in the autonomous vehicle driving scenarios.
In another embodiment, the present invention system and method autonomously generate precise, relevant and contextually valid scenarios, including edge case scenarios that are essential in the effective training of autonomous vehicles. The method not only performs these tasks but also autonomously generates additional edge cases, expanding the range of scenarios beyond the limitations of human ideation while tackling the erroneous outputs of LLMs. This streamlined approach enhances efficiency and introduces novel scenarios that might not have been envisioned through traditional methods.
In an embodiment, the invention has diverse application areas, including but not limited to:
1. Creating typical scenarios for autonomous vehicles.
2. Generating edge-case scenarios for autonomous vehicles.
3. Developing regular scenarios for robots.
4. Formulating edge-case scenarios for robots.
5. Crafting typical scenarios for drones.
6. Crafting edge-case scenarios for drones.
7. Utilize LLMs to generate dynamic and challenging scenarios for video games, enhancing the realism and unpredictability of in-game environments.
8. Develop realistic and diverse scenarios for training emergency responders, such as firefighters, paramedics, and law enforcement, using LLMs to simulate complex and evolving situations.
9. Generate realistic flight scenarios for pilot training simulations, incorporating a wide range of challenging conditions and unexpected events.
10. Create lifelike medical scenarios for training healthcare professionals, allowing them to practice and enhance their skills in various medical situations.
11. Simulate complex supply chain and logistics scenarios, optimizing decision-making processes and testing the resilience of logistics networks.
12. Develop scenarios to simulate cyber threats and attacks, allowing cybersecurity professionals to train in a realistic environment and improve their response capabilities.
13. Use LLMs to generate diverse traffic and city planning scenarios, helping urban planners and traffic management systems prepare for various situations and challenges.
14. Simulate agricultural scenarios to optimize crop planning, resource allocation, and decision-making for precision farming practices.
15. Generate scenarios to simulate the impact of natural disasters, aiding in disaster preparedness and response training for emergency services.
16. Simulate complex scenarios for space exploration missions, including spacecraft navigation, communication challenges, and unexpected events.
17. Develop realistic military training scenarios, incorporating diverse terrain, enemy behaviours, and tactical challenges using LLM-generated simulations.
18. Create scenarios to simulate various customer interactions and retail situations, allowing businesses to train employees in customer service and problem-solving.
These application areas showcase the versatility of the invention across a spectrum of autonomous systems including but not limited to vehicles, robots, and drones, both in routine and challenging scenarios.
,CLAIMS:We Claim:
1. A method for scenario generation for an autonomous driving simulation comprising:
a) generating a diverse set of textual descriptions for various driving scenarios by a repository unit;
b) receiving queries from the repository by a LLM unit, where the queries comprise the generated textual description with a pre-defined prompt structure for both a static and a dynamic element for scenario development;
c) receiving python dictionary (intermediate output) from the LLM unit;
wherein LLM unit is configured to make changes to information stored in the python unit based on the new information and the said information is converted into an OpenDRIVE and an OpenSCENARIO files.
2. The method for scenario generation for an autonomous vehicle as claimed in claim 1, wherein a static element comprises a road network, a roadside objects and a crossing.
3. The method for scenario generation for an autonomous vehicle as claimed in claim 1, wherein dynamic elements comprise vehicles, pedestrians and other moving objects such as animals.
4. The method for scenario generation for an autonomous vehicle as claimed in claim 1, wherein the repository unit combines the generated description with a pre-defined prompt structure to create specific queries for the LLM unit.
5. The method for scenario generation for an autonomous vehicle as claimed in claim 1, wherein the repository unit applies a hybrid prompting technique by querying the LLM unit to verify if the extracted data paints a consistent picture with the textual description.
6. The method for scenario generation for an autonomous vehicle as claimed in claim 1, wherein the repository unit in conjunction with LLM unit ensures context-aware refinement by factoring in both the original scenario description and predefined constraints, resulting in descriptions that are not only accurate but also relevant to the specific scenario.
7. A system for scenario generation for an autonomous driving simulation comprising:
a) a repository unit configured to generate a diverse set of textual descriptions for various driving scenarios;
b) a LLM unit is configured to receive queries from the repository, where the queries comprise the generated textual description with a pre-defined prompt structure for both a static and a dynamic element for scenario development; and
c) a python unit is configured to receive a python dictionary (intermediate output) from the LLM unit;
wherein LLM unit is configured to make changes to information stored in the python unit based on the new information and the said information is converted into an OpenDRIVE and an OpenSCENARIO files.