Description:Complete Specification:

The following specification describes and ascertains the nature of this invention and the manner in which it is to be performed.
Field of the invention
[0001] This invention is related to a method of detecting a hazardous object on road during autonomous driving and a control unit thereof.

Background of the invention

[0002] Road scenes environment is continuously developing with new highways, roads, city streets, and expressways. Small obstacles such as debris, water spots and other factors further compound the challenges in safe driving. In autonomous driving or assisted driving applications, the detection of lost cargo items on the road, such as unclaimed and potentially suspicious packages, injured persons, or animals, etc. is an essential application. Note that such objects are static and small is size unlike moving vehicles; but nevertheless, are important to avoid collision as they pose a safety issue. One of the challenges in computer vision is that the accuracy of the verification is dependent on extrinsic factors such as lighting, pose, distance from the camera etc. If the image is not captured with proper lighting, pose and working distance, then it will lead to false positives and false negatives. In addition, we find that the outliers captured using camera are not sufficient (in different conditions etc.)

[0003] A US patent 2010063649 discloses an intelligent driving assistant system applied to a handheld device. The invention can detect more than one safety mode including lane departure detection, lost-cargo detection, detecting front-object under driving condition, detecting side-object under driving condition and detecting back-object under driving condition, also the invention can mention alarm according to the detection results from different modules. Finally the invention can store the real-time image according to the detection results from different modules, and then transfer the related information to other places for real-time notice with matching the information from the GPS system and the digitized map.

Brief description of the accompanying drawings
[0004] Figure 1 illustrates a control unit for detecting at least one hazardous object on the road, in accordance with an embodiment of the invention; and
[0005] Figure 2 illustrates a flowchart of a method for detecting at least one hazardous object on the road, in accordance with the present invention.

Detailed description of the embodiments
[0006] Figure 1 illustrates a control unit for detecting at least one hazardous object in accordance with one embodiment of the invention. The control unit 10 receives a video sequence 12 having multiple images 14 as an input from an image capturing unit 16. The control unit 10 transmits the received images to an intelligent learning module 18 after forming a real-world data set. The control unit 10 processes the received images 14 and initializes at least one image formed by a text prompt using a stable diffusion module in the control unit 10. The received images 14 are unlabeled. Then the control unit 10 generates a domain adapted synthetic data from the stable diffusion module and compares at least one feature of the processed images with a real- time test data for detecting the hazardous object using the intelligent learning module 18 in the control unit 10. The intelligence module 18 is a self -supervised learning module 18 which is developed and trained from unlabeled dataset and does not require supervision.

[0007] According to one embodiment of the invention, the image capturing unit 16 is a camera, an imaging sensor, and the like. The image capturing unit 16 is positioned at a specific position on the vehicle for taking a video sequence in the real time. The captured video sequence comprises multiple images/frames 14 of the road that the vehicle is travelling. The control unit 10 receives the captured images and an image processing and object detection unit 16 present in the control unit 10 processes the received images for identifying an area of interest. Each captured image has an area of interest. According to one embodiment of the invention, the control unit 10 is present in the vehicle and according to another embodiment of the invention, the control unit 10 is operated from a cloud-computing source.

[0008] The control unit 10 builds the intelligence learning module 18, wherein the type of intelligence learning module 18 is chosen from the group of intelligence modules comprising an artificial intelligence module, a deep learning module, a machine learning module, a self-supervised learning module and the like. The control unit 10 develops a neural network /intelligence network by using training data that is inputted into the intelligence learning module 18 (for example an AI module) during the calibration process for training the intelligence learning module 18 in identifying the correct objects on the road in the real time. The pre-data that is loaded in the intelligence learning module 18 are captured based on the manual markings on the road, where the vehicle will be travelling. The intelligent learning module 18 is trained with a pre-loaded data that comprises the real-time data set and the generated synthetic data set. The initializing process generates a similar image based on the received text prompt. I.e., for instance, if the received text prompt is a word “dustbin”, then the control unit 10 generates an image similar or closer version of the dustbin, using at least one methodology.

[0009] The image processing and object detection unit 22 present in the control unit 10 identifies an area of interest in each of the received images 14 and identifies the annotated images in each of the received multiple images 14. For detecting at least one hazardous object on the road, the control unit 10 extracts at least one feature from an area of interest in each of the received multiple images 14. The at least one feature is chosen from a group of features comprising a pattern, a color contract, texture, outline, etc.

[0010] Figure 2 illustrates a flowchart of a method of detecting at least one hazardous object on the road according to the present invention. In step S1, a video sequence (12) having multiple images (14) as an input from an image capturing unit (16) is received to a control unit (10). In step S2,the received images (14) are transmitted to an intelligent learning module 18 after forming a real-world data set. In step S3, the received images are processed and at least one image formed by a text prompt is initialized using a stable diffusion module 17 in the control unit 10.
[0011] In step S4, a domain adapted synthetic data is generated from the stable diffusion module 17. In step S5, least one feature of the processed images 14 are compared with a real- time test data for detecting the hazardous object using the intelligent learning module 18 in the control unit 10.

[0012] The method is explained in detail. The camera or image capturing unit 16 is positioned on the vehicle in a specific location to capture the surroundings of the path/road on which the vehicle is travelling. With the captured multiple images 14 , the control unit 10 forms a real-world data set. This data set is transmitted and accessed by the control unit 10 for processing the received images further. Simultaneously, based on a text prompts (for at least one anomaly), the control unit 10 initializes the process of generating a similar image using at least one generative AI model 17, wherein one such diffusion model is a stable diffusion model. The control unit 10 generates the synthetic data set that is domain adapted using the diffusion model 17. The synthetic data set comprises the data that is generated from the text prompts via the stable diffusion models. The synthetic data set is domain adapted in order to simulate real-world image settings, before transferring it to the intelligent learning module 18. Earlier to this, the control unit 10 initializes the process of creating images from generative AI using the text prompts.

[0013] Simultaneously, the intelligent learning module 18 continuously receives the images 14 from the real time of the surroundings where the vehicle is travelling. The current/real-time images that are received from the surroundings are compared with the images 14 that are pre-loaded from the synthetic data set and the real-world data set. Based on the comparison, the control unit 10 detects the hazardous object that is lying on the road where the vehicle is travelling.

[0014] With the above-mentioned method, any damage or accident to the vehicle can be avoided during an autonomous driving mode. The detection of the hazardous objects on the road is effectively done with the above methodology. It provides a low-cost effective solution.

[0015] It should be understood that embodiments explained in the description above are only illustrative and do not limit the scope of this invention. Many such embodiments and other modifications and changes in the embodiment explained in the description are envisaged. The scope of the invention is only limited by the scope of the claims.
, Claims:We Claim:

1. A control unit (10) for detecting at least one hazardous object on the road, said control unit (10) adapted to:
- receive a video sequence (12) having multiple images (14) as an input from an image capturing unit (16);
characterized in that:
- transmit said received images (14) to an artificial intelligent learning module (18) after forming a real-world data set;
- process and initialize said received images (14) and at least one image formed by a text prompt using a stable diffusion module (17) in said control unit (10);
- generate a domain adapted synthetic data from said stable diffusion module (17);
- compare at least one feature of said processed images (14) with a real- time test data for detecting said hazardous object using said intelligent learning module (18)in said control unit (10).

2. The control unit (10) as claimed in claim 1, wherein said intelligent learning module (18) is trained with a pre-loaded data that comprises said real-time data set and the generated synthetic data set.

3. The control unit (10) as claimed in claim 1, wherein said initialize process generates a different image initialized on said received text prompt.

4. The control unit (10) as claimed in claim 1, wherein at least one hazardous object on the road is detected using at least one feature extracted from an area of interest in each of said received multiple images (14).

5. The control unit (10) as claimed in claim 4, wherein said at least one feature is chosen from a group of features comprising a pattern, a color contract, texture, outline, etc.

6. The control unit (10) as claimed in claim 1, wherein said real-time test data is collected by said control unit(10) during an operational mode of said vehicle.

7. A method of detecting at least one hazardous object on the road, said method comprising :
- receiving a video sequence (12) having multiple images (14) as an input from an image capturing unit (16) to a control unit (10);
characterized in that:
- transmitting said received images (14) to an intelligent learning module (18) after forming a real-world data set;
- processing and initializing said received images (14) and at least one image formed by a text prompt using a stable diffusion module (17) in said control unit (10);
- generating domain adapted synthetic data from said stable diffusion module (17);
- comparing at least one feature of said processed images with a real- time test data for detecting said hazardous object using said intelligent learning module (18) in said control unit (10).