Claims:1. A method for detecting a moving object from a video captured using a motion camera, the method comprising a processor implemented steps of:

receiving the video captured from the motion camera as an input;
applying a lane detection algorithm on the video to estimate an area present in a lane in front of the motion camera;
identifying a plurality of corner points within the estimated area, wherein the plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area;
calculating a plurality of flow vectors using the plurality of corner points;
clustering the plurality of flow vectors based on their magnitude, angle and position (MAP) depending on their respective threshold values;
clustering an output of MAP clustering based on the shear of the plurality of flow vectors;
selecting the clusters out of the output of the MAP clustering and the shear based clustering which are present within the estimated area based on a predefined criteria; and
segmenting selected clusters to detect a boundary of the moving object using a color segmentation based model.

2. The method of claim 1, wherein the predefined criteria comprises a base of the cluster to lie inside an area of the lane and a width of the cluster is within a width of the lane.

3. The method of claim 1, wherein the step of segmenting results in identification of a bounding box bounding the moving object.

4. The method of claim 1 further comprising the step of validating the detection of the object using clustering based on the object.

5. The method of claim 1 further comprising the step of tracking the detection of the object based on at least one of:
a normalized template matching,
a flow vector magnitude and phase based tracking, or
a watershed based tracking.

6. The method of claim 1, wherein the step of preprocessing further comprises the step of enhancement of an image captured using the motion camera.

7. The method of claim 1, wherein the threshold value for magnitude, angle and position is set adaptively based on their average value.

8. The method of claim 1, wherein the calculation of the plurality of flow vectors is performed using a pyramidal optical flow.

9. The method of claim 1, wherein the colour segmentation based model is a watershed model

10. The method of claim 1, wherein the motion camera is installed on a vehicle travelling on a road.

11. The method of claim 1 further comprising the step of detecting a second object overtaking the moving object, wherein the second object is entering from a bottom right corner of a bottom left corner.

12. The method of claim 1 further comprising the detection of the moving object during night as follows:

providing an infrared filter for the motion camera during night;
providing infrared LEDs to illuminate the area in front of the motion camera;
classifying a plurality of divergent and non- divergent points in front of the motion camera with respect to chaos and angle;
filtering out the plurality of classified non-divergent points based on their density;
creating an adaptive region of interest (ROI) based on the density of the plurality of non-divergent points;
extracting high density region within the region of interest;
selecting two consecutive frames of detection for voting to follow tracking within the high density region; and
tracking the moving object using an adaptive template matching based on the intensity.

13. The method of claim 1, wherein the moving object is either classified or non-classified.

14. A system for detecting a moving object from a video captured using a motion camera, the system comprises:

an input module for providing the video captured from the motion camera as an input;
a memory; and
a processor in communication with the memory, wherein the processor further comprising:
an area estimation module for applying a lane detection algorithm on the video to estimate an area present in a lane in front of the motion camera;
a corner points identification module for identifying a plurality of corner points within the estimated area, wherein the plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area;
a flow vectors calculation module for calculating a plurality of flow vectors using the plurality of corner points;
a clustering module for
clustering the plurality of flow vectors based on their magnitude, angle and position (MAP) depending on their respective threshold values, and
clustering an output of MAP clustering based on the shear of the plurality of flow vectors;
a selection module for selecting the clusters out of the output of the MAP clustering and the shear based clustering which are present within the estimated area based on a predefined criteria; and
a segmentation module for segmenting selected clusters to detect a boundary of the moving object using a color segmentation based model.
, Description:FORM 2

(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION

(See Section 10 and Rule 13)

Title of invention:

METHOD AND SYSTEM FOR DETECTING MOVING OBJECTS FROM A VIDEO CAPTURED USING A MOTION CAMERA

Applicant

Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the embodiments and the manner in which it is to be performed.

TECHNICAL FIELD

[001] The embodiments herein generally relates to the field of video processing, and, more particularly, to a method and system for detecting moving objects from a video captured using a motion camera.

BACKGROUND

[002] Moving object detection plays an important role in various fields such as in the field of autonomous vehicles, intelligent robots, video monitoring, activity recognition, tracking, surveillance, security and the like. A number of techniques have been developed to detect motion within the field of view of a still video camera. However, image-based moving object detection technique deployed on the still camera usually cannot provide dynamic support. On the other hand, for movable camera surveillance, the movement of the camera will cause the entire image change and the compensation to the error caused by the camera movement makes it difficult to use a single image-based technique to effectively detect moving objects.
[003] When the common methods which are used for fixed camera are applied to a moving observer such as an autonomous vehicle, the produced image moves as the observer moves and the whole background also changes, so that it becomes difficult to stably detect the moving object from the background.
[004] Various video based applications has also been used like ADAS (advanced driver assistance system) in autonomous vehicles, the video to be processed in captured through a camera in motion. E.g., in ADAS, the front facing/ road facing camera is setup at the back of the rear view mirror. Hence with respect to movement of the vehicle, the camera also moves and with respect to camera the entire scene and hence all the objects present in a scene are in motion. The first challenge in the existing method of vehicle detection kind of application is need for extensive training. The current object detection algorithms (both vehicle and pedestrian detection) work based on features like HoG and classifier like support vector machine (SVM) or Deep Learing. Both of the aforementioned approaches requires a lot of offline data set both for positive (the intended object) and negative. The features (like HoG and Haar) are extracted from the offline dataset to train classifier like SVM. This training involves a lot of text where the mentions of fields actually occur.
[005] Practically, if a video is captured from a camera fit in a moving object (like vehicle) all the objects in the scene will be apparently moving. As our vision system (dorsal pathways), the moving objects are more salient and their location is important than their class.

SUMMARY

[006] The following presents a simplified summary of some embodiments of the disclosure in order to provide a basic understanding of the embodiments. This summary is not an extensive overview of the embodiments. It is not intended to identify key/critical elements of the embodiments or to delineate the scope of the embodiments. Its sole purpose is to present some embodiments in a simplified form as a prelude to the more detailed description that is presented below.
[007] In view of the foregoing, an embodiment herein provides a system for detecting a moving object from a video captured using a motion camera, the system comprises an input module, a memory and a processor. The input module provides the video captured from the motion camera as an input. The processor further comprises an area estimation module, a corner points identification module, a flow vectors calculation module, a clustering module, a selection module and a segmentation module for detection and localization. The area estimation module applies a lane detection algorithm on the video to estimate an area present in a lane in front of the motion camera. The corner points identification module identifies a plurality of corner points within the estimated area, wherein the plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area. The flow vectors calculation module calculates a plurality of flow vectors using the plurality of corner points. The clustering module clusters the plurality of flow vectors based on their magnitude, angle and position (MAP) depending on their respective threshold values, and clusters an output of MAP clustering based on the shear of the plurality of flow vectors. The selection module selects the clusters out of the output of the MAP clustering and the shear based clustering which are present within the estimated area based on a predefined criteria. The segmentation module segments selected clusters to detect a boundary of the moving object using a color segmentation based model.
[008] In another aspect the embodiment here provides a method for detecting a moving object from a video captured using a motion camera. Initially, the video captured from the motion camera is received as an input. A lane detection algorithm is then applied on the video to estimate an area present in a lane in front of the motion camera. In the next step a plurality of corner points are identified within the estimated area, wherein the plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area. In the next step, a plurality of flow vectors using the plurality of corner points are calculated. The plurality of flow vectors are then clustered based on their magnitude, angle and position (MAP) depending on their respective threshold values. Followed by the clustering an output of MAP clustering based on the shear of the plurality of flow vectors. In the next step, the clusters are selected out of the output of the MAP clustering and the shear based clustering which are present within the estimated area based on a predefined criteria. And finally the selected clusters are segmented to detect a boundary of the moving object using a color segmentation based model.
[009] It should be appreciated by those skilled in the art that any block diagram herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computing device or processor, whether or not such computing device or processor is explicitly shown.

BRIEF DESCRIPTION OF THE DRAWINGS

[010] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[011] Fig. 1 illustrates a block diagram of a system for detecting a moving object from a video captured using a motion camera according to an embodiment of the present disclosure;
[012] Fig. 2A-2B is a flowchart illustrating the steps involved in detecting a moving object from a video captured using a motion camera according to an embodiment of the present disclosure; and
[013] Fig. 3A-3B is a flowchart illustrating the steps involved in detecting the moving object from a video captured using the motion camera during the night according to an embodiment of the disclosure.

DETAILED DESCRIPTION

[014] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[015] Referring now to the drawings, and more particularly to Fig. 1 through Fig. 3, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
[016] According to an embodiment of the disclosure, a system 100 for detecting and localizing a moving object from a video captured using a motion camera 102 is shown in Fig. 1. The system 100 uses the capability of understanding the pixel movement to segment which all pixels belong to a specific object and then classify the object as obstacles or vehicles based on its placement on the area in front of the motion camera.
[017] When a video is captured from the camera fit in a moving object (like vehicle), all the objects in the scene will be apparently moving. As our vision system (dorsal pathways), the moving objects are more salient and their location is important than their class. If we move forward, we can see the static objects are moving as diverged from a center of divergence which are considered as the vanishing point. The potential feature points (like corners) in the scene move far from the vanishing point radially as the frame number is increased i.e., as the camera moves forward. The proposed invention would track this particular property of divergence of a scene and indicate outliers who are not following the said property. The outliers are potential member to be globally moving objects. The moving objects are segregated from the scene in multi-level hierarchical Fuzzy C-Means clustering on phase and magnitude of flow vectors.
[018] According to an embodiment of the disclosure, the system 100 further comprises an input module 104, a memory 106 and a processor 108 as shown in the block diagram of Fig. 1. The processor 108 works in communication with the memory 106. The processor 108 further comprises a plurality of modules. The plurality of modules accesses the set of algorithms stored in the memory 106 to perform a specific task. The processor 108 further comprises an area estimation module 110, a corner points identification module 112, a flow vectors calculation module 114, a clustering module 116, a selection module 118 and a segmentation module 120.
[019] According to an embodiment of the disclosure the input module 104 is configured to provide an input to the processor 108. The input module 104 is configured to provide the video captured from the motion camera 102. The use any type of motion camera 102 available in the art is well within the scope of this disclosure. In an example the input module 104 can be display screen which can be used to display the output. The input module 104 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite.
[020] According to an embodiment of the disclosure, the system 100 comprises the area estimation module 110. The area estimation module 110 is configured to estimate the area present in front of the motion camera 102. If the motion camera 102 is installed on a moving vehicle, then the area estimation module 110 estimates the area of the lane in which the vehicle is travelling. The area is estimated using a lane detection algorithm. The use of any other algorithm is well within the scope of this disclosure. In the lane detection algorithm, lane forms an important role in segmenting objects of interest. Currently, the offline CSV file is read to obtain the area information. The CSV file contains the right lane and left lane pixel coordinates, using which two equations are generated that represents the path and width of the area in front of the camera. These equations were chosen such that the width data could be manipulated to give an estimation of multi-lane data.
[021] According to an embodiment of the disclosure, the system 100 further comprises a preprocessor 122. The pre-processor 122 is configured to enhance the delivered output from the area estimation module 110. The preprocessing may comprises three steps - image enhancement, visible area detection and adaptive RoI based on gradient change.
[022] The image enhancement involves normal sharpening method which involve subtracting the Gaussian blurred image from the original image. This is done so as to improve the visibility of corners for identifying better corners to track.
[023] To identify the visible area, first the area of interest is limited to the estimated region of five on-going. It is noticed that, no matter what the color of the area is, the area seems to follow a gradual gradient change. So the standard deviation of the image is taken (each pixel with respect to the neighboring pixels) and biggest blob is found with the lowest deviation. The adaptive region of interest (RoI) is calculated based on the sudden vertical gradient change which is usually noticeable when the surface changes. If the camera is installed on the vehicle, the gradient change is expected mostly at the base of the vehicle.
[024] According to an embodiment of the disclosure, the system 100 also comprises the corner points identification module 112. The corner points identification module 110 identifies a plurality of corner points within the estimated area, wherein the plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area. In order to segment the moving object, it is necessary to understand how the pixels are moving, but calculating the flow for each and every pixel is computationally complex. So probable pixels are chosen that can help in better detection (pixels that are expected to be a part of moving object which could be tracked). The pre-processed is provided to the corner points identification module 112. The plurality corner point pixels are chosen within the adaptive regions of interest who's base lie inside the estimated area, assuming that these regions are probable moving object.
[025] According to an embodiment of the disclosure, the system 100 also comprises the flow vectors calculation module 114. The flow vectors calculation module 114 calculates a plurality of flow vectors using the plurality of corner points. The plurality of flow vectors are calculated only on corner points to improve the performance. By dense optical flow it is impossible to achieve real-time performance of the system. The plurality of flow vectors are found to track the flow of the moving object. A use pyramidal optical flow process proposed by Lucus-Kanade is used for finding the plurality of flow vectors. The history of this flow is maintained to understand the flow with better consistency. All the points that doesn’t have proper flow or that are closely packed are removed from the whole history.
[026] According to an embodiment of the disclosure the system 100 further comprises the clustering module 116. The clustering module 116 sequentially performs a magnitude, angle and position (MAP) based clustering followed by a shear based clustering. Initially, the plurality of flow vectors are clustered based on their magnitude, angle and position (MAP) depending on their respective threshold values. The threshold for each parameter is set adaptively based on their average value. In case of position, the threshold is set based on the average of the shortest distance of all point in the cluster to their corresponding closest point within the cluster. The angle and magnitude threshold have been set based on a tolerance from their average values. Any point outside the cluster that lies within these threshold is added to the cluster.
[027] In an example, when the motion camera is on a base vehicle, then the moving object such as other vehicles moving in same direction to the base vehicle are detected through phase criteria of flow vector. The diverging flows from vanishing point are background and vehicles moving in same direction to the base vehicle in relatively higher speed executes convergent or chaos flow angle. While, the vehicles moving in opposite direction to the base vehicle shows same phase property that of the background but the non-linear magnitude property of background would treat the vehicle coming in opposite direction as anomaly the expected non-linearity of flow-magnitude.
[028] Moreover, the output of MAP based clustering is further clustered based on the shear of the plurality of flow vectors. The output clusters from the MAP clustering are over-clustered and there maybe multiple cluster which are part of the same object. To combine all the clusters that lie on the same objects shear based clustering was used. For the moving object, all the points in a planar object (moving or non-moving) will move uniformly keeping relatively a parallel spacing along the flow. This property is utilized in-order to further cluster the clustered flow vectors into clusters that lie on the same plain.
[029] According to an embodiment of the disclosure, the system 100 further comprise the selection module 118. The selection module 118 selects the clusters out of the output of the MAP clustering and the shear based clustering which are present within the estimated area based on a predefined criteria. The predefined criteria comprises a base of the cluster to lie inside an area of the lane and a width of the cluster is within a width of the lane
[030] According to an embodiment of the disclosure, the system 100 also comprises the segmentation module 120 for detection and localization of the moving object. The segmentation module 120 is configured to segment the selected clusters to detect a boundary of the moving object using a color segmentation based model like watershed based segmentation model. The use of any other model is well within the scope of this disclosure. The watershed based segmentation is done so as to understand the exact boundary of the object which further helps us to identify two things – (1) To identify the accurate bounding box, (2) To eliminate false based on the shape of the object. To avoid over segmentation, marker based watershed segmentation is utilized. The marker, in-turn, is created by drawing a cross in the middle of the in-lane clusters which contribute to the detections and all the remaining points contributing to the background. After the marker is made, the marker is passed to the watershed algorithm to get all the segments. The segment that contributes to the background is drawn in black and the remaining in white so that simple contour extraction can extract the detections.
[031] According to an embodiment of the disclosure, the system 100 further comprises a validation module 124. The validation module 124 is configured to validate the presence of the detected boundary of the moving object. The validation is performed using an object based clustering. The boundary of the object is correlated with the flow points to understand which all flow points are part of the moving object.
[032] There are multiple ways to validate the detection. In an example, the output contours from the watershed is analyzed based on the amount of expansion made with respect to the shear based cluster or the object based cluster. In another example, the aspect ratio based filter is used to avoid false with absurd width to height ratio. In yet another example, the choice of bounding box, when there is overlap in detection, is made by understanding the visibility of the 4 edges of the detection. If no edges are found the detection is eliminated, or if both the overlapping detections are showing edges, priority is categorized based on the position and length of the edge with respect to the detection bounding box.
[033] According to an embodiment of the disclosure, the system 100 can also be configured to perform tracking of the moving object. The tracking can be performed using one of the following three methods. The method is chosen based on the required accuracy and performance:
1. Normalized template matching: This algorithm uses normal template matching but the template and the search image is re-sized to a specific height or width by maintaining the aspect ratio. This is done so to reduce the computation as the search window and the template size is reduced and the matrix convolution is in-turn reduced. This is fast but there are misses in the detections.
2. Optical flow based tracking: This is used to adaptively change the size and template of the detection based on the movement of the object. As the object moves, the flow points in the moving object either comes closer or moves further depending on if the moving object is moving away from the camera or moving towards the camera respectively. So the amount of shrink or extension is calculated to adaptively change the size of the detection.
3. Watershed based tracking: In this tracking, template matching, followed by marker based watershed segmentation, is deployed to get the tracked position, with the accurate bounding box. This is the tracking which has optimal detection and performance.
[034] According to an embodiment of the disclosure, the system 100 is also configured to detect a moving object if the object is overtaking the camera. This can be helped with the example of vehicle which is explained in detail in the later part of the disclosure.
[035] In operation, a flowchart 200 illustrating the steps of detecting a moving object from a video captured using the motion camera 102 is shown in Fig. 2A-2B. Initially, at step 202, the captured video from the motion camera 102 is provided as an input to the processor 108 using the input module 104. In the next step 204, a lane detection algorithm is applied on the video to estimate an area present in a lane in front of the motion camera 102. The use of any other algorithm is well within the scope of this disclosure. In the next step 206, a plurality of corner points are identified within the estimated area. The plurality of corner points are chosen within an adaptive region of interest whose base lie inside the estimated area. In the next step 208, a plurality of flow vectors are calculating using the plurality of corner points.
[036] In the next step 210, the plurality of flow vectors are clustered based on their magnitude, angle and position (MAP) depending on their respective threshold values. The threshold for each parameter is set adaptively based on their average value. In case of position, the threshold is set based on the average of the shortest distance of all point in the cluster to their corresponding closest point within the cluster. The angle and magnitude threshold have been set based on a tolerance from their average values. Any point outside the cluster that lies within these threshold is added to the cluster. At step 212, the output of MAP clustering is further clustered based on the shear of the plurality of flow vectors. In the next step 214, the clusters out of the output of the MAP clustering and the shear based clustering are selected which are present within the estimated area based on a predefined criteria. And finally at step 216, the selected clusters are segmented to detect a boundary of the moving object using a color segmentation based model.
[037] According to an embodiment of the disclosure, the system 100 is also configured to detect the presence of the moving object during the night time. For the purpose of detection of the moving object during night, infrared (IR) LEDs 126 are used. The IR LEDs 126 are used to illuminate the scene which gives better visibility during night. A flowchart 300 illustrating the steps of detecting of the moving object from a video captured using the motion camera 102 during night is shown in Fig. 3A-3B. The present example has been explained with the help of a camera and IR LEDs fit around each of a plurality of IR sensors 130 on a moving vehicle. Though it should be appreciated that the same set up can also be provided at any other location where moving object detection is required during the night.
[038] Initially at step 302, an infrared filter 128 is provided for the motion camera 102 during night. At the next step 304, the infrared LEDs 126 are provided to illuminate the area in front of the motion camera 120. In the next step 306, a plurality of divergent and non- divergent points are classified in front of the motion camera with respect to chaos and angle. It was checked if the tracked moving points are moving in a chaotic manner (random) or if it’s moving towards the vanishing point. The chaotic behavior is basically because the moving vehicle would be having a relative velocity very near to zero or changing velocity throughout frames (sometimes relative velocity zero, sometimes higher than ego and sometimes lower. That is quite impossible for background), which means that tracked corner points are almost stationary with respect to the camera.
[039] In the next step 308, the plurality of classified non-divergent points are filtered out based on their density. The classified non-divergent points may sometimes be scattered and there is a possibility that there is wrong calculation of non-divergent points. It is clear that for a moving vehicle, there will be a greater density of non-divergent points with respect to the false calculations. So the non-divergent points are filtered based on density of non-divergent point cluster (position based cluster).
[040] In the next step 310, an adaptive region of interest (ROI) is created based on the density of the plurality of non-divergent points. In the next step 312, high density region is extracted within the region of interest. Initially, (i) Contrast stretching is applied adaptively with respect to Base Y - Coordinate of the RoI. Illumination of the vehicle adaptively changes with respect to the distance in IR. Closer vehicle are brightly illuminated and distant vehicle are less illuminated. In a 2D plane, farther an object are located in upper region of the image (w.r.t the camera alignment). The threshold for contrast stretching is made adaptive for an RoI such that if the RoI is towards a faraway vehicle, more contrast stretch is imposed and vise-versa. After contrast stretching, even if the far away vehicles are properly illuminated by processing. The image is binarized for further processing. Later, a vertical histogram is created for white pixel count and determine threshold with respect to the maximum value. Further, the histogram is binarized with respect to the threshold. The rising edge, falling edge and the width of the pulse is taken into consideration to identify the expected region of the vehicle. The height is determined by the non-divergent points within the expected region. If there is no non-divergent points, the detection is excluded. Further, the next probable detection region is determined w.r.t the position of the current detection. The same process is repeated till the white pixel density of the next probable detection region is less than a predefined tolerance (e.g., 7.5%).
[041] In the next step, 314, two consecutive frames of detection are selected for voting to follow tracking within the high density region. And finally at step 316, the moving object is tracked using an adaptive template matching based on the intensity. The moving object is tracked as follows: Initially, normal template matching is done to track the next position. Followed by the contrast stretching over the entire image w.r.t the base Y – Coordinate of the tracked vehicle bounding box. Further, the template w.r.t the vertical histogram of white pixels within/around the tracked bounding region is shrunk or enlarged (widthwise). As the vehicle was moving away from the scene, the size also is shrunk. The bounding box is adaptively changed to fit exactly around the vehicle. For this purpose, the vertical white pixel histogram is utilized again. If the detected vehicle is decelerating, the tracked bounding box may not cover the whole vehicle as the size of the vehicle becomes bigger when it comes close to the camera. Here the bounding box is enlarged such that it covers the entire blob of the vehicle. The template is now updated. Since the new template is enclosing the vehicle with precision, the correlation for the template matching would be better. Further, if this detection does not satisfy the aspect ratio condition (3:2), then the image is rotated and the same process is used to adaptive change the actual height of the template. This process is done so that if the detection is a two wheeler, the height is properly fixed. This is also helpful in case of fast moving vehicle, where, at some point of time, the height also needs to be shrunk for better visualization. Finally, the detected rectangle is remapped to the actual orientation and displayed.
[042] According to an embodiment of the disclosure, the system 100 for detecting the moving object from a video captured using the motion camera 102 can also be explained with the help of an example. In the example, the camera is installed on a vehicle and the vehicle is moving. Thus the system 100 can be used to detect other vehicles which are moving in front of the vehicle.
[043] In an example the system 100 can also be used to detect the overtaking vehicle, i.e. when one vehicle overtakes a host vehicle on which the camera is installed. In case of autonomous vehicles, it is important to detect the overtaking vehicle as early as possible to avoid any kind of accident. The present disclosure provides a hypothesis of presence of passing vehicle in detection stage and confirms its presence using proposed generic I^2F^2C framework. The detection stage involves two levels of preprocessing to determine the entry of overtaking vehicle.
[044] Overtaking vehicles can be present only on the bottom-left and/ or bottom-right corners of the scene and hence a quantitative amount (e.g., 10% -15% of bottom-left) of the left and right side of the image is considered to be RoI. The quantitative amount is based on region growing from two corners of the frame on a test video containing different kinds of overtaking vehicles. Small steps are grown and check for the enclosure of the overtaking vehicle to choose the RoI for a given image size. Once the overtaking vehicle enters the scene and anomaly is detected, it takes a maximum of 3 frames to confirm its presence. Essentially it was checked whether the vehicle remains enclosed in the chosen RoI until the confirmation stage is processed and the RoI was fixed. This in turn provides perfect enclosure of overtaking vehicle for that particular image size and camera alignment. The entire processing is then carried out only in adaptively selected RoI from the entire image where there is a possibility of the overtaking vehicle entry.
Mathematical model for detection confirmation
[045] The detection confirmation stage is essentially a validation of the detected white dense blob to be an overtaking vehicle wherein the hypothesis of the presence of overtaking vehicle as predicted by anomaly detection stage is confirmed by the generic I^2F^2C framework. Histogram of Optical flow is primarily used for vehicle confirmation. Harris corner detector was used to determine suitable points to calculate the sparse optical flow in the selected RoI for over at-least three frames. By increasing the number of frames for judgment the confidence of confirming the presence of overtaking vehicle was improved but it has an inverse relationship with performance. Next the angle flow correspondence of corner points over three frames is estimated to plot the flow histogram. If the vehicle is overtaking from left then majority of the corner points should move towards the vanishing point and they should exhibit angle values between the fixed thresholds and will fall into the same bin in the histogram. The same holds good for vehicle moving from the right. So if it's an overtaking vehicle then the angle histogram will have its majority in bins that indicates the flow towards vanishing point be it from left or right. For left passing vehicle it could be observed that majority of points lie in regions of 120-270 degrees and for right passing vehicle it lies in the region of 270-360. In some cases it was found to lie in 0-40 bins owing to the circularity of histogram bin division. Because of circularity possessed by Histogram bin division the way in which the arrangement of bins considered varies.
[046] The flow histogram can also be treated as PDF (Probability Density Function) where the bin number is the random variable. In agreement with the aforementioned rationale, the PDF would be concentrated in the bins specified resulting sudden rise in the slope of the corresponding CDF (cumulative distribution function). For left car the change of change of CDF is very evident unlike to the right car as the occupancy of bins of interest is not concentrated as far as linear arrangement of bins is concerned. As the bin arrangement is not inherently linear but circular with periodicity 9, the PDF and CDF are modeled in the framework of MODULE9 arithmetic. The MOD9 arithmetic can shift the CDF from F_X(x) to F_Y(y) where y is a new random variable representing new bin position (binPos_{new}) shifting the old bin positions x binPos_{old} circularly as depicted for angular shift (shift_{ang}). This operation would rearrange the bins for right region ensuring the region of occurrences of right overtaking car flow phases to be neighbors (e.g., x=0 would be converted to y=8 if shift_{angle} = 1 as MOD9(-1) = 8. This circular rotation enables the CDF slope to be evident for both right and left corner of the scene for confirmation of passing vehicle detection.
[047] It is to be noted that the background in the right region usually is of the form of left vehicle and the background in the left takes form of the right vehicle template because of the nature of divergence of background which forces itself out of the frame as the vehicle moves. The angles exhibited by the background while moving out of frame is similar to the angles exhibited by vehicle regions in the other end of the frame. It has also been observed that with different shift_{angle} also the pattern of CDF is maintained for presence of cars in the RoI.
[048] Once the entry of passing vehicle is detected the next task is to mark the vehicle boundary. To mark the vehicle initially the corner points coordinates are sorted with respect to rows and columns and determined the minimum and maximum column value (x_{min} and x_{max}, respectively) along with minimum and maximum row value (y_{min} and y_{max}, respectively). The (x_{min}, y_{min}) duple is the starting co-ordinate of the bounding rectangle with height y_{max}- y_{min} and width x_{max}- x_{min}. This rectangle is used to indicate the location and entry of passing vehicle.
[049] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
[050] The embodiments of present disclosure herein provides a method and system for detecting a moving object from a video captured using a motion camera.
[051] It is, however to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.
[052] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
[053] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
[054] A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
[055] Input/output (I/O) devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
[056] A representative hardware environment for practicing the embodiments may include a hardware configuration of an information handling/computer system in accordance with the embodiments herein. The system herein comprises at least one processor or central processing unit (CPU). The CPUs are interconnected via system bus to various devices such as a random access memory (RAM), read-only memory (ROM), and an input/output (I/O) adapter. The I/O adapter can connect to peripheral devices, such as disk units and tape drives, or other program storage devices that are readable by the system. The system can read the inventive instructions on the program storage devices and follow these instructions to execute the methodology of the embodiments herein.
[057] The system further includes a user interface adapter that connects a keyboard, mouse, speaker, microphone, and/or other user interface devices such as a touch screen device (not shown) to the bus to gather user input. Additionally, a communication adapter connects the bus to a data processing network, and a display adapter connects the bus to a display device which may be embodied as an output device such as a monitor, printer, or transmitter, for example.
[058] The preceding description has been presented with reference to various embodiments. Persons having ordinary skill in the art and technology to which this application pertains will appreciate that alterations and changes in the described structures and methods of operation can be practiced without meaningfully departing from the principle, spirit and scope.