Description:Complete Specification:

The following specification describes and ascertains the nature of this invention and the manner in which it is to be performed.
Field of the invention
[0001] This invention is related to a control unit for detecting at least one marking on a road and a method thereof. The invention is exclusively used in the autonomous driving application.

Background of the invention

[0002] Road markings refer to the signs drawn on the surface of the road. These differ from traffic signs erected by the side or on top of roads. Road markings are as important to detect as traffic signs for navigation systems and driver assistive devices. While many road markings provide information that is redundant to traffic signs (e.g., speed limits), other information, such as arrows for turn-only lanes, are exclusively provided by road markings.

[0003] A US patent 9401028 discloses a method and system for video-based road departure warning for a vehicle on a road, is provided. Road departure warning involves receiving an image of a road in front of the vehicle from a video imager, and detecting one or more road markings in the image corresponding to markings on the road. Then, analyzing the characteristics of an image region beyond the detected markings to determine a rating for drivability of the road corresponding to said image region, and detecting the lateral offset of the vehicle relative to the markings on the road based on the detected road markings. A warning signal is generated as function of said lateral offset and said rating.
 
Brief description of the accompanying drawings
[0004] Figure 1 illustrates a control unit for detecting at least one marking on a road, in accordance with an embodiment of the invention; and
[0005] Figure 2 illustrates a flowchart of a method for detecting at least one marking on a road, in accordance with the present invention.

Detailed description of the embodiments
[0006] Figure 1 illustrates a control unit for detecting at least one marking on a road in accordance with one embodiment of the invention. The control unit 10 comprises an image capturing unit 13, an image processing unit 16, a blob detector module 15. The control unit 10 receives a video sequence 12having multiple images 14 of the road as an input, from the image capturing unit 13. The control unit 10 extracts a road region in at least one of the images 14 using a free space detection technique in the image processing unit 16. The control unit 10 applies a blob detector technique to get an area of interest in the road region of the at least one image 14 using the blob detector module 15. The control unit 10 detects multiple patches 22 in the region of interest and segregating unwanted patches and desired patches. The control unit 10 separates a foreground and a background in the multiple patches 22 using at least one clustering technique and checking for at least one condition to be met in the multiple patches 22. The control unit 10 extracts at least one feature from each of the desired patches 22 using an intelligence module 18 and detecting the at least one road marking, upon comparison with a corresponding pre-defined feature.

[0007] Further the components involved in detecting the marking 20 on the road is explained in detail. According to one embodiment of the invention, the blob detector module 15 is made as an integral part of the image processing unit 16. The image capturing unit 13 is chosen from a group of capturing units comprising a camera, an image capturing sensing element and the like. The image capturing unit 13 is positioned on the vehicle 11, that is travelling on the road, where the markings 20 need to be detected. The video sequence 12 includes multiple images 14 of the road where the vehicle 11 is travelling.

[0008] Figure 2 illustrates a flow chart of a method for detecting at least one marking on a road in accordance with the present invention. In step S1, a video sequence 12 is inputted to a control unit 10 that have multiple images 14 of the road, from an image capturing unit 13. In step S2, a road region is extracted in at least one of the images 14 using a free space detection technique. In step S3, a blob detector technique is applied to get an area of interest in the road region of at least one image. In step S4, multiple patches 22 are detected in the area of interest and segregating unwanted patches and desired patches. In step S5, a foreground and a background in the multiple patches 22 is separated using at least one clustering technique and checking for at least one condition to be met in said multiple patches 22. In step S6, at least one feature is extracted from each of the desired patches 22 and detecting the at least one road marking 20, upon comparison with a corresponding pre-defined feature.

[0009] The captured multiple images 14 are transferred to the image processing unit 16 wherein, the captured images 14 are sharpened for minimizing blurred arrow marks present and other artefacts. From each of the captured images 14, the region of interest is detected using a detector module 15, wherein one such the detector module 15 is a Maximally Stable Extremal Region (MSER) detector module. It is to be understood that the detector module can be any other detector module, but not limited to the above mentioned one. The Maximally Stable Extremal Region (MSER) detector 15 is employed by the control unit 10 to get initial probable candidates of road arrow marks in each of the image 14.

[0010] The control unit 10 uses a non-maximum suppression technique for segregated the unwanted patches and the desired patches from the area of interest which is identified in each of the multiple images 14. Applying non-maximum suppression technique on all the images 13 filters out all the unwanted patches. The unwanted patches involves the overlapping patches and the patches that comprises lane markings or any other patch that has data which is not related to the markings. All those patches are referred as unwanted patches, and these are filtered out using the non-maximum suppression technique. The control unit further removes the patches that comprises high random foreground shapes. With the removal of the unwanted patches, the false positive detections can be avoided.

[0011] After the separation of the unwanted patches and the desired patches, the control unit 10 process the multiple images 14 in the image processing unit 16, for separating /removing the foreground and the background region using any one of the clustering techniques, one such technique is k-means technique. Here in the k-means technique, k is the number of clusters to be removed from the image, wherein the number of clusters is 2 (foreground and background).

[0012] The control unit 10 checks for the for the at least one condition, wherein at least one condition is chosen from the following conditions comprising, check for a size of an object in a number of user defined images 14, size of the object being constant in every of the user defined images 14. The control unit 10 checks for the following conditions to be met in all the desired patches that are taken out from each of the multiple captured images 14. The user defines number of frames to be scrutinized in order to identify the object. I.e.., the control unit 10 checks for the presence of the object in all the user defined images. For example, if the user defines the number of frames/images 14 to be scrutinized as 5, then that number is provided as an input to the control unit 10. The control unit 10 checks for the presence of the object in the before 5 and after 5 frames or the control unit 10 checks the frames taken, that are present before and after the image /frame that comprises the desired patch.

[0013] The control unit 10 checks for the foreground surrounded by the road region in the “n” user defined frames and the size of the object in all the user defined frames/images 14. The control unit 10 further checks for the increase in the object size in a constant manner and the control unit 10 also verifies for the object shape to be constant in all the user defined frames/images 14. The control unit 10 further checks for the size of the patches 22, if found the size of the patch 22 to be small, then that patch 22 is not considered for further processing.

[0014] For validating the patches 22, the control unit 10 extracts features using an intelligence module 18. The intelligence module 18 is chosen from a group of intelligence modules comprising an artificial intelligence module, a deep learning module, a machine learning module and the like. The intelligence module 18 is pre-trained with the data, wherein the data being the multiple images 14 of the road having lane markings, road signs, traffic signs and the like. The intelligence module 18 is pre-trained with the above data during the calibration process and the features from each of the image is extracted and is stored in a memory 17.

[0015] In the real-time, the control unit 10 to validate the patch 22 having the desired printed shape (for example an arrow mark), extracts multiple features. The features are chosen from a group of features related to a color, an orientation, at least one corner in each of the desired patches 22. In addition to that, the control unit 10 extracts a set of features comprising dominant colors present in each of the desired patch 22, distribution of pixels orientation in each of the patch 22 and corners / key-points in each of the desired patch 22.

[0016] The control unit 10 verifies whether a specific shape is present in each of the desired patch 22 based on the dominant colors present in that desired patch 22 once the desired patch 22 satisfies the above-mentioned conditions. Ideally two dominant colors are found in specified shape present in the patch 22, which are a foreground color and a background road color. The control unit 10 uses the feature of the orientation of all the pixels present in the desired patch 22. The background / road pixels will have similar orientation since it is a plane region and foreground pixels (specially boundary pixels) orientation will give a specified shape information. And the control unit 10 verifies the number of key-points or corners, wherein the number of corner points will be more in case of random foreground shape. In another scenario, the control unit 10 verifies a presence of a specified shape (or structure) in each of the desired patch. For example, in case of arrow marks, the control unit 10 searches for arrow tail and head presence in every desired patch 22. Thus, the control unit 10 detects the marking 20 of the road using the above methodology.

[0017] The above-mentioned methodology is providing a low-cost effective solution in eliminating the overlapping patches. The above method is applied in the autonomous driving application. The above method provides a solution in detecting the correct road markings (apart from the lane markings) in order to provide a smooth ride to the user when the autonomous driving mode is activated.

[0018] It should be understood that embodiments explained in the description above are only illustrative and do not limit the scope of this invention. Many such embodiments and other modifications and changes in the embodiment explained in the description are envisaged. The scope of the invention is only limited by the scope of the claims.
, Claims:We Claim:

1. The control unit (10) as claimed in claim 1, wherein said control unit (10) comprises an image capturing unit (13), an image processing unit (16), a blob detector module (15),
said control unit adapted to:
- input a video sequence (12) having multiple images (14) of said road, from said image capturing unit (13);
- extract a road region in at least one said image (14) using a free space detection technique in said image processing unit (16);
- apply a blob detector technique to get an area of interest in said road region of said at least one image using said blob detector module (15);
characterized in that:
- detect multiple patches (22) in said region of interest and segregating unwanted patches and desired patches;
- separate a foreground and a background in said multiple patches (22) using at least one clustering technique and checking for at least one condition to be met in said multiple patches;
- extract at least one feature from each of said desired patches using an intelligence module (18) and detecting said at least one road marking, upon comparison with a corresponding pre-defined feature.

2. The control unit (10) as claimed in claim 1, wherein said image processing unit (16) adapted to sharpen said at least one image for minimizing blurred arrow marks present in said image (13).

3. The control unit (10) as claimed in claim 1, wherein said region of interest in detected using said detector module (15), wherein one such said detector module (15) is a Maximally Stable Extremal Region (MSER) detector module.

4. The control unit (10) as claimed in claim 1, wherein said unwanted patches and said desired patches are segregated using a non-maximum suppression technique.

5. The control unit (10) as claimed in claim 1, wherein said at least one condition is chosen from the following conditions comprising, check for a size of an object in a number of user defined images (14), size of said object being constant in every said user defined image (14).

6. The control unit (10) as claimed in claim 1, wherein said features are chosen from a group of features related to a color, an orientation, at least one corner in each of said desired patches (22).

7. The control unit (10) as claimed in claim 1, wherein control unit (10) adapted to verify presence of an arrowhead and the tail in said desired patches (22).

8. A method of detecting at least one marking (20) on a road by a control unit (10), said method comprising:

- inputting a video sequence (12) having multiple images (14) of said road to said control unit (10), from an image capturing unit (13);
- extracting a road region in at least one said image using a free space detection technique;
- applying a blob detector technique to get an area of interest in said road region of at least one image using a blob detector module (15);
- detecting multiple patches (22) in said area of interest and segregating unwanted patches and desired patches;
- separating a foreground and a background in said multiple patches (22) using at least one clustering technique and checking for at least one condition to be met in said multiple patches (22);
- extracting at least one feature from each of said desired patches (22) and detecting said at least one road marking (20), upon comparison with a corresponding pre-defined feature