Description:Complete Specification:

The following specification describes and ascertains the nature of this invention and the manner in which it is to be performed.
Field of the invention
[0001] This invention is related to a control unit for detecting at least one hazardous object on the road. The invention is exclusively used in the autonomous driving application.

Background of the invention

[0002] In an autonomous driving vehicle, identification of the objects correctly plays a vital role. There will be lot of scene images captured and stored (say autonomous driving scene images). These scene images contain a variety of objects like vehicles, bicycles, pedestrians which are termed as valid class of objects. Along with these valid objects, there will be presence of some static hazardous objects like lost cargo, boxes, wooden frames, debris, broken vehicle parts, broken tires / wheels, stones, animal corpses (out-of-schema objects) lying on the road and are also captured in these scene images. This category also includes potholes and open manholes. These types of objects pose a threat to autonomous vehicles as they hinder the actual drivable region on road. The present invention provides a solution to the problem of finding and separating these hazardous objects from valid object classes.

[0003] A US patent 2010063649 discloses an intelligent driving assistant system applied to a handheld device. The invention can detect more than one safety mode including lane departure detection, lost-cargo detection, detecting front-object under driving condition, detecting side-object under driving condition and detecting back-object under driving condition, also the invention can mention alarm according to the detection results from different modules. Finally the invention can store the real-time image according to the detection results from different modules, and then transfer the related information to other places for real-time notice with matching the information from the GPS system and the digitized map.
 
Brief description of the accompanying drawings
[0004] Figure 1 illustrates a control unit for detecting at least one hazardous object on the road, in accordance with an embodiment of the invention; and
[0005] Figure 2 illustrates a flowchart of a method for detecting at least one hazardous object on the road, in accordance with the present invention.

Detailed description of the embodiments
[0006] Figure 1 illustrates a control unit for detecting at least one hazardous object in accordance with one embodiment of the invention. The control unit 10 receives a video sequence 12 having multiple images 14 as an input from an image capturing unit 16 and identifies annotated images in the received multiple images 14 for forming a set of pristine quality patches. The control unit 10 builds and trains an intelligence module 18 for forming a set of semseg patches from the received multiple images 14 and compares the set of the pristine quality patches and the set of semseg patches by the intelligence module 18 for identifying at least one hazardous object on the road.

[0007] According to one embodiment of the invention, the image capturing unit 16 is a camera, an imaging sensor and the like. The image capturing unit 16 is positioned in the vehicle for taking a video sequence in the real time. The captured video sequence comprises multiple images/frames 14 of the road that the vehicle is travelling. The control unit 10 receives the captured images and an image processing unit 16 present in the control unit 10 processes the received images for identifying an area of interest. Each captured image has an area of interest. According to one embodiment of the invention, the control unit 10 is present in the vehicle and according to another embodiment of the invention, the control unit 10 is operated from a cloud-computing source.

[0008] The intelligence module 18 contains two sets of image patches. One is pristine image patch set and other is semseg patch set. The pristine quality patch set comprises the annotated images that are done manually, which include painted road markings, lane markings and the like.
[0009] The semseg patch set comprises hazardous objects (i. e, non-pristine quality patches). The semseg set of patches are obtained from scene images by using a pre-trained deep neural network trained for the task of semantic segmentation. The hazardous objects present on the road (mostly on sides, near road / lane markings) and these are not labeled (for training deep neural network), the semantic segmentation model labels them as road / lane markings. The pristine quality image patches also contains road / lane markings, and these are obtained by using ground truth annotations which are called as annotated images.

[0010] Figure 2 illustrates a flow chart of a method for detecting at least one hazardous object on the road. The method comprises following steps. In step S1, a video sequence 12 having multiple images 14 is received as an input from an image capturing unit 16. In step S2, annotated images are identified in the received multiple images for forming a set of pristine quality patches. In step S3, an intelligence module 18 has been build and trained for forming a set of semseg patches from the received multiple images 14. In step S4, the set of pristine quality patches and the set of semseg patches are compared for identifying at least one hazardous object on the road.

[0011] The method is explained in detail. The method disclosed in the present invention is used in many applications, one application being an autonomous driving. If the objects (which are road signs/traffic signs/or anything related to the vehicle driving conditions) are not identified properly, it will become a serious hazard. In order to overcome that, the present invention provides the solution in correctly identifying the objects on the road. The image capturing unit 16 captures a video sequence 12 of the road in the real time and the captured video sequence comprises multiple images 14. The images will have different types of objects laying on the road. For instance, road sign boards, lane markings, traffic signal lights and the like. In addition to that, there will be other objects (like cardboards, dead animals and the like). The control unit 10 has to effectively identify the correct objects during the real time for the safe movement of the vehicle.

[0012] The control unit 10 builds the intelligence module 18, wherein the type of intelligence module 18 is chosen from the group of intelligence modules comprising an artificial intelligence module, a deep learning module, a machine learning module and the like. The control unit 10 develops a neural network /intelligence network by using pre-data that is inputted into the intelligence module 18 during the calibration process for training the intelligence module 18 in identifying the correct objects on the road in the real time. The pre-data that is loaded in the intelligence module 18 comprises the pristine quality patches that are captured based on the manual markings on the road, where the vehicle will be travelling.

[0013] The image processing unit 22 present in the control unit 10 identifies an area of interest in each of the received images and identifies the annotated images in each of the received multiple images 14 for forming the set of pristine quality patches. As mentioned above, the control unit 10 forms a set of semseg patches from the received multiple images using the intelligence module 18. The set of semseg patches are formed using a patch extraction detection technique. The semseg patches are extracted after identifying the area of interest in each of the captured multiple images.

[0014] For detecting at least one hazardous object on the road, the control unit 10 at least one feature extracted from an area of interest in each of the received multiple images 14. The at least one feature is chosen from a group of features comprising a pattern, a color contract. For instance, based on the feature extracted, the control unit 10 segregates the semseg patch from the pristine quality patch. The control unit 10 uses an outlier detection module 20 for detecting at least one hazardous object upon comparing the set of semseg patches with the pristine quality patches using an outlier detection module 20. The outlier detection module 20 is made as an integral part of the intelligence module 18 for correct identification of the at least one hazardous object on the road.

[0015] Each patch present in a database of the intelligence module 18 (both semseg patches and pristine quality patches) is represented using a feature vector obtained from a pre-trained deep neural network. This deep neural network is trained for feature extraction. The control unit 10 takes the image patches, defining a pretext task for the model to execute. The features are then supplied to an outlier detection module 20. This module will first model the distribution of pristine quality patches and this distribution is termed as inlier distribution. Later, each patch from the set of semseg patches is scored against this inlier distribution based on similarity and are labeled as either inliers or outliers. This way, all the hazardous objects present in the set of semseg patches are labeled as outliers from the rest of pure road marking patches.

[0016] With the above-mentioned method, any damage or accident to the vehicle can be avoided during an autonomous driving mode. The detection of the hazardous objects on the road is effectively done with the above methodology. It provides a low-cost effective solution.

[0017] It should be understood that embodiments explained in the description above are only illustrative and do not limit the scope of this invention. Many such embodiments and other modifications and changes in the embodiment explained in the description are envisaged. The scope of the invention is only limited by the scope of the claims.
, Claims:We Claim:

1. A control unit (10) for detecting at least one hazardous object on the road, said control unit (10) adapted to:
- receive a video sequence (12) having multiple images (14) as an input from an image capturing unit (16);
characterized in that:
- identify annotated images in said received multiple images (14) for forming a set of pristine quality patches;
- build and train an intelligence module (18) for forming a set of semseg patches from said received multiple images;
- compare said set of said pristine quality patches and said set of semseg patches by said intelligence module (18) for identifying at least one hazardous object on said road.

2. The control unit (10) as claimed in claim 1, wherein said intelligence module (18) is trained with a pre-loaded data that comprises said annotated images.

3. The control unit (10) as claimed in claim 1, wherein said set of semseg patches are formed using a patch extraction detection technique.

4. The control unit (10) as claimed in claim 1, wherein at least one hazardous object on the road is detected using at least one feature extracted from an area of interest in each of said received multiple images (14).

5. The control unit (10) as claimed in claim 4, wherein said at least one feature is chosen from a group of features comprising a pattern, a color contract.

6. The control unit (10) as claimed in claim 1, wherein said set of semseg patches is extracted from area of interest each of said multiple images (14).

7. The control unit (10) as claimed in claim 1, wherein said control unit (10) detects at least one hazardous object upon comparing said set of semseg patches with said pristine quality patches using an outlier detection module (20).

8. A method for detecting at least one hazardous object on the road, said method comprising:
- receiving a video sequence having multiple images (14) as an input from an image capturing unit (16) of a control unit (10);
characterized in that:
- identifying annotated images in said received multiple images (14) for forming a set of pristine quality patches;
- building and training an intelligence module (18) for forming a set of semseg patches from said received multiple images (14);
- comparing said set of said pristine quality patches and said set of semseg patches for identifying at least one hazardous object on said road.