Field of Invention
The present invention relates to navigations systems, and more particularly to, vision based vehicle detection and tracking.
Background of the Invention
One of the major challenges of this generation of road transport vehicles is to increase the safety of the passengers and the pedestrians. To this end, automated road navigation systems provide various levels of assistance to automobile drivers to increase the safety and reduce the driving effort. There are various kinds of active and passive road navigation systems which determine the type, location and relative velocity of obstacles in a vehicle's path or vicinity. Various systems have been developed which use active techniques such as radar, lasers, or ultrasonic transceivers to gather information about a vehicle's surroundings for automated navigation systems. For example, various adaptive cruise control methods use Doppler radar, lasers or ultrasonic transceivers to determine the distance from a host vehicle to the leading vehicle traveling along a road. These systems are classified as "active" systems because they typically emit some form of energy and detect the reflected energy or other predefined energy emitters.
Passive systems, on the other hand, typically detect energy without first emitting a signal, i.e., by viewing reflected or transmitted light. Optical detection systems typically perform extensive data processing roughly emulating human vision processing in order to extract useful information from incoming optical signal. Automated navigation systems using optical systems

must solve a variety of processing tasks to extract information from input data, interpret the information and trigger an event such as a vehicle control input or warning signal to an operator or other downstream receivers.
US Patent No. 6999004 discloses a system and method for vehicle detection and tracking in tunnels. The method comprises the steps of capturing a plurality of image frames viewing at least one traffic lane; extracting at least one feature from the plurality of image frames; detecting at least one object indicative of a vehicle from the extracted feature; and tracking the detected vehicle over time to determine the detected vehicle's velocity. The system comprising at least one image capture device for capturing a plurality of image frames viewing at least one traffic lane; and a processor adapted for extracting at least one feature from the plurality of image frames, detecting at least one object indicative of a vehicle from the extracted feature, and tracking the detected vehicle over time to determine the detected vehicle's velocity.
US Patent No. 6611229 discloses a vehicle tracking system, wherein with respect to vehicles owned by members on which communication units containing GPS receivers are mounted, the vehicle tracking system specifies a previously-registered member and the vehicle owned by the previously-registered member based upon a request of the previously-registered member for providing positional information of a vehicle owned by the previously-registered member, and the vehicle tracking system executes a polling operation of positional information to the vehicle owned by the previously-registered member; retrieves an existence position of the vehicle on a map from a map database based upon positional information transmitted from the vehicle owned by the previously-registered member; displays the existence position of

the vehicle by superimposing on the map; and provides the existence position superimposed on the map as vehicle positional information data to the previously-registered member.
US Patent No. 5459460 discloses a collision warning system mounted on a vehicle to issue an alarm when the vehicle approaches near an obstacle running in front of the vehicle. The system detects a speed (Vf) of the vehicle; a distance (R) between the vehicle and the obstacle; a speed (Va) of the obstacle using both the speed (Vf) of the vehicle and the distance (R). The collision warning system issues an alarm to a driver if the following relationship exists between the vehicle and the obstacle. In this system laser beams are used to compute the range and thereby the speed.
However, such active systems are susceptible to interference from similar active systems operating nearby. Emitted energy can be reflected and scattered by surrounding objects thereby introducing errors into the navigation systems. Active systems also consume substantial electrical power and are not well suited for adaptation to existing infrastructure, such as the national highway system. GPS based navigation systems comprise a variety of interdependent data acquisition and processing tasks to provide various levels of output. However, these elaborate systems are very expensive and complicated to install.
Hence, there is a need for a system and method for accident reduction and driver assistance which works in real time. The disclosed system and methods should provide a low cost solution while combining high performance. The disclosed system should also be versatile enough to be able to combine vehicle detection and tracking with a variety of applications such as adaptive cruise

control, obstacle detection and avoidance, multi vehicle tracking, lane detection and so on a single platform.
Objects and Summary of the Invention
The present invention has the objective to provide a vision based system and method for efficient target detection and tracking.
It is also an objective of the instant invention to provide a system and method for target detection and tracking that is cost effective and computationally faster.
It is still another objective of the instant invention to provide an automated system and method that can be combined with a plurality of road navigation systems to facilitate better driving experience.
To achieve the aforesaid objectives the instant invention provides a method for detection and tracking of leading target comprising the steps of:
- capturing a plurality of image frames in at least one scene in front of a host system;
- processing the image frames to determine one or more image characteristics;
- extracting one or more edges by determining one or more connected components of at least one target in an intensity plane of each image frame;
- clustering of the edges using a predefined distance;

- eliminating one or more outliers based on predefined and adaptive
thresholds;
forming a bounding box around at least one cluster;
- qualifying a cluster as the target in consecutive image frames; and
- tracking the target by tracking the cluster features in consecutive frames.
The instant invention further provides a system for detection and tracking of a leading target comprising:
- means for image acquisition;
- means for processing coupled to the means for image acquisition for processing one or more acquired images to extract at least one image characteristic in the corresponding intensity plane for each image frame; and
- storage device coupled to the means for processing for storing predefined, acquired and processed information.
Brief Description of Drawings
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.
FIG. 1 illustrates components of an exemplary system of the instant invention for target detection and tracking

FIG. 2 is a flow diagram illustrating an exemplary method of implementation of the instant invention for target detection and tracking wherein the target is a vehicle.
FIG. 3 is a flow diagram illustrating the image processing and clustering phase of an exemplary method of implementation of the instant invention.
FIG. 4 is a flow diagram illustrating the target detection and target tracking phase of an exemplary method of implementation of the instant invention.
Detailed Description of Drawings
A system and method for target detection and tracking is described. The system and methods are not intended to be restricted to any particular form or arrangement, or any specific embodiment, or any specific use, disclosed herein, since the same may be modified in various particulars or relations without departing from the spirit or scope of the claimed invention hereinabove shown and described of which the apparatus or method shown is intended only for illustration and disclosure of an operative embodiment and not to show all of the various forms or modifications in which this invention might be embodied or operated.
A forward looking detector mounted on a vehicle can capture a plurality of video streams and pass it to an electronic control unit for further processing. The forward looking detector can be an image acquisition device such as a forward facing camera mounted on the top or behind the wind shield of the passenger compartment of a vehicle. The electronic control unit can comprise a

means for processing to process the acquired video streams according to the method of the present invention. The disclosed method detects a target based on the edges in the Luminance (Y) plane of the acquired image. In accordance with the dynamic feature recognition aspect of the present invention, the image processor can convert the RGB frame to Y plane. The connected components can be extracted by scanning the edges in the region of interest (ROI) of the image. Subsequendy, the extracted connected components can be clustered. Single link clustering using taxicab distance can be used. While clustering the centroid of the connected components can be used as the representative. Two level clustering can be performed to eliminate outliers based on fixed and/or adaptive distance thresholds. To draw an exact bounding box fitting around each cluster, vertical profiling over only the qualifying clusters and computing the limits can be performed. A particular cluster can be qualified as a target if the centroid of the cluster lies within certain vicinity and the cluster bounding box's width to height ratio and size varies under predefined thresholds for specified number of consecutive frames. In addition the same features can be checked in the frames to determine a cluster to be the target to be tracked. Once a target is identified, it can be determined if it is "closing in" or "moving away" based on the information extracted from a plurality of consecutive frames.
The techniques described herein may be used in many different operating environments and systems. It should be appreciated that components of an implementation according to the invention can be implemented on specially programmed computing hardware, application specific integrated circuitry, digital signal processors, software or firmware running on general purpose or application specific devices or various combinations of the foregoing. An

exemplary implementation of the disclosed method of the present invention is discussed in the following section with respect to the accompanying figures.
Fig. 1 illustrates components of an exemplary system 100 for target detection and tracking according to an exemplary implementation of the present invention. According to an embodiment,' exemplary system 100 can include a forward looking detector 102 coupled to a electronic control unit 104, driver vehicle interface 106, one or more network interfaces 108 and one or more other interfaces 110. Forward looking detector 102 can be an image acquisition device such as a forward facing camera. The use of a single low cost forward looking detector 102 results in an affordable system which is also simple to install. Further, being a passive system, it is not affected on account of interference by active systems that may be present in the vicinity of the host vehicle. Forward looking detector 102 can be fixed to the host vehicle and can move in one or more directions. Forward facing detector is coupled to electronic control unit 104 for further processing of the image data acquired by the camera during vehicle detection and tracking. The processed results can be made available by the electronic control unit 104 to the user through one or more driver vehicle interfaces 106. Driver vehicle interface can include audible, visual and tactile indicators to present relevant information to the user. The driver vehicle interface 106 can provide the user with system status, warnings, operational and diagnostic messages generated by the electronic control unit 104 in a plurality of formats such as graphs and text formats for a better driving experience.
To this end, electronic control unit 104 can include processing engine 112 coupled to memory 114 and vehicle control and interface unit 116. Processing

engine 112 can be a single processing entity or a plurality of processing entities comprising multiple computing units. In some embodiments processing engine 112 can be a digital signal processor or an image processor configured to perform the method of the present invention. Memory 114 can include, for example, volatile memory (e.g., RAM) and non volatile memory (e.g., ROM, flash memory, etc.). Memory 114 can store the acquired images and their relevant data, operational instructions for at least a plurality of predefined conditions, wamings, messages and other relevant data for use during detection and processing of information. Memory 114 can also store the results of the processing to be used by one or more relevant systems in the vehicle such as automatic cruise control and collision warning system. The operational instructions and relevant data stored in memory 114 can be executed by the processing engine 112. Vehicle control and interface unit 116 can facilitate interfacing and control of vehicle components such as controlling the engine speed or shifting automatic transmission based on the results obtained by processing the acquired vehicle data.
Network interfaces 108 and other interfaces 110 can facilitate interaction between the functional components of system 100 and one or more systems exiting within the vehicle or over a vehicle network. One or more network interfaces 108 can be used with optional in vehicle data network for data communication to data recording or diagnostic devices. One or more other interfaces 110 can facilitate interaction with one or more additional navigation systems such as adaptive cruise control, obstacle detection and avoidance, multi vehicle tracking and lane detection. The detection and tracking information can be utilized by the adaptive/cruise control to provide warning signal or stimulate other operations which would adjust a vehicle's speed to maintain relative

separation between a leading vehicle and the host vehicle. The method and the system of the present invention can be easily adapted to track multiple vehicles and thus provide information regarding traffic in other lanes which may be useful for any number of navigation or collision avoidance tasks.
System 100 can be implemented as a standalone system or as an additional feature in a more extensive navigation system including lane departure warning, headway indication and warning, and cut-in warning and lane change assist features and so on. On initialization/starting the vehicle, the disclosed system 100 can perform a power-up self test. System status and other relevant information can be provided to the user through visible and/or audible indicators coupled with the driver vehicle interface 106. Forward looking detector 102 can capture video data and pass the video output signal to electronic control unit 104 for further processing. Video output signal can be connected to the input of processing engine 112 adapted to accept the video signals. Video frames from the forward facing detector are then sampled, captured and stored in memory 114. The digitized image in the form of pixel information can then be stored, manipulated and otherwise processed in accordance with the capabilities of the vision system. The digitized images are accessed form the memory 114 and processed by the processing engine 112 according to the operational instructions stored in memory 114. Output information can be further sent to the driver vehicle interface 106 for triggering output actions such as display and audible alarm outputs. The result of the processing of acquired video information by the processing engine 112 can be utilized by an optional vehicle network through network interface 108 for data communication to data recording or diagnostic devices. Output information can also be sent through one or more other interfaces 110 to other relevant

systems in the host vehicle such as automatic cruise control and collision warning system for aiding in better navigation.
Exemplary method for target detection and tracking using a monocular detector is described with reference to description of figures 2-4. Processes 200, 300 and 400 are illustrated as a collection of blocks in a logical flow graph, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer instructions that, when executed by one or more processors, perform the recited operations. The order in which the process is described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the process, or an alternate process. Additionally, individual blocks may be deleted from the process without departing from the spirit and scope of the subject matter described herein. For discussion purposes, the processes 200, 300 and 400 are described with reference to the implementations of Fig 1.
Fig. 2 illustrates a flow diagram illustrating an exemplary method of implementation of the instant invention practiced in vehicle detection and tracking. The method of the present invention discloses a mono-vision based solution to detect and track at least one leading vehicle.
At block 202, a forward facing detector 102 installed on a host vehicle, can capture video images of a target scene and pass the video output signal to electronic control unit 104 for further processing. Acquired video frame sets can be sampled, captured and stored in memory 114.

At block 204, the acquired video output signal can be connected to the input of processing engine 112 adapted to accept the video signals. The digitized image in the form of pixel information can then be stored, manipulated and otherwise processed in accordance with the capabilities of the vision system. The digitized images can be accessed form the memory 114 and processed by the processing engine 112 according to the operational instructions stored in memory 114.
At block 206, the disclosed method can detect a leading vehicle based on the edges in the Y plane of the acquired image. The RGB input can be converted to Y-Plane containing the luminance information of the image. The connected components can be extracted by scanning the edges in the ROI of the image. The noisy connected components can be eliminated based on the length. Further, the remaining connected components can be clustered. Single link clustering using taxicab distance can be used. While clustering, the centriod of the connected components can be used as the representative. Two level clustering can be performed to eliminate outliers. Further, to draw exact bounding box fitting around the bigger cluster representing the target, we can perform vertical profiling over only the qualifying connected components in that cluster and compute the limits. The cluster information such as centroid of the cluster, shape and size of the cluster bounding box in predefined number of consecutive frames is used to qualify the cluster as a target vehicle.
At block 208, tracking of a detected vehicle is performed based on the movement of cluster centroid, and variations of shape and size of the cluster bounding box in consecutive frames. Further, it may be determined if the vehicle is closing in or moving away based on the movement of the bounding box in consecutive frames.

Figure 3 is a flow diagram illustrating the acquired video frame processing phase of an exemplary method of implementation of the instant invention practiced in vehicle detection and tracking. The disclosed method involves detection of relevant edges, clustering of edges, elimination of outliers, and subsequent formation of a bounding box of appropriate size to positively detect a vehicle.
At block 302, relevant edges in the Y-plane of the image can be detected. The RGB frame can be converted to Y plane containing the luminance information for the image as the human eye is more sensitive to luminance information as compared to color. Subsequently, we can apply histogram equalization. Histogram equalization ensures uniform distribution of intensities. On this, sobel edge detection operator can be applied. The sobel edge detection operator can determine the gradient of the image intensity at each point The result therefore shows how "abruptiy" or "smoothly" the image changes at that point, and therefore how likely it is that that part of the image represents an edge, as well as how that edge is likely to be oriented. The image pixels at which the response is above a predefined threshold can be retained to obtain the desired edges.
At block 304, Clustering of the detected edge connected components can be performed. The disclosed method of the present invention can use single link clustering using taxicab distance. While clustering, the centriod of the connected components can be used as the representative. The process of clustering of edges can be performed in at least two phases to eliminate

outliers. In the first phase of clustering, the outliers can be eliminated based on a predefined fixed distance threshold, i.e, if the distance between the components and the centroid of the cluster is greater than the predefined threshold value then such components can be eliminated from the cluster. In this first phase, for each edge connected component a cluster is identified. The cluster for an edge connected component is formed by including the edge connected components which lie within the specified distance threshold. The distance between components refers to the distance between centroids of those components. In this scenario it can be observed that a component may get clustered to more than one clusters based on its distance value with respective component. Once this is done a cluster is identified as potential cluster if the number of components is greater than three and also has maximum number of pixels constituting the components in it when compared to the clusters of all other components. Thus a single cluster for a frame is identified. Further, in the second phase, a new cluster centroid can be computed by considering the components in the cluster identified in the first phase. Here, the outliers can be eliminated based on the adaptive threshold, i.e., the threshold can be computed based on the information of the detected vehicle in the previous frames.
At block 306, the extreme coordinates of the connected components in the cluster can form the extreme boundaries of the bounding box to positively indicate the detected leading vehicle. In order to increase the preciseness of the left and right boundary of the bounding box, we can compute the vertical profile of the pixels representing the connected components in the cluster representing positively the vehicle. The bottom and top boundary of the bounding box can be identified by checking the extreme ends of the connected components. If necessary, further horizontal profiling can be done to refine the

bottom and top boundary of the bounding box. In the disclosed method of the present invention, profiling can act as a fine tuner to identify the proper left, right, top and bottom bounds of the cluster. Thus, fine tuning of the bounding box around the cluster qualified as the vehicle can be performed. This increases the accuracy of the computations involved in the process.
Figure 4 is a flow diagram illustrating the target detection and tracking phase of an exemplary method of implementation of the instant invention practiced in vehicle detection and tracking. Generally, the tracking phase is performed through the maximization of the correlation between the portion of the image contained into the bounding box of the previous frame and the new frame. Specifically, the detected edge movements in consecutive frames can be used for tracking the target vehicle.
At block 402, the relevant information from predefined number of frames is verified. The information includes the centroid of the cluster, size and shape of the cluster bounding box.. The shape of the cluster can be regarded as the width to height ratio of the cluster bounding box. Initially a cluster can be
qualified to be the detected vehicle by determining if the centroid of the cluster
>
lies within a certain vicinity and the shape and size of the cluster bounding box varies within specified limits for a predefined number of consecutive frames.
At block 404, once the relevant cluster representative of the vehicle is identified positively, the information associated with the vehicle cluster is tracked in the consecutive frames. The cluster parameters of the incoming video frame i.e. the bounding box size, shape and centroid is compared with the averages of that of the specified number of previous frames in which the vehicle was

identified. If the target vehicle is not identified in recent specitied number frames, then the clusters in recent specified number of frames are checked for qualifying a new target vehicle as explained in block 402.
At block 406, the relative direction of motion of the target leading vehicle can be determined. To determine whether the target vehicle is "closing in" or "moving away" from the host vehicle, the change in the position of the bottom line of the bounding box with respect to those of the previous frames can be tracked and relative distance can be determined. By knowing the precise calibration of the camera used, absolute values of distance and size may be determined.
At block 408, the information acquired and processed by the system of the present invention can be fed to other relevant systems in the vehicle or coupled over a network. The information processed can be utilized to trigger one or more indicators such as visual or audible alarms through the driver vehicle interface. Further, the information derived by processing the data acquired by the forward looking detector can be further fed to a plurality of associated systems like automatic cruise control, automatic parking system, and so on for facilitating a better driving experience.
The embodiments described above and illustrated in the figures are presented by way of example only and are not intended as a limitation upon the concepts and principles of the present invention. It is envisioned, that the present invention disclosed herein is applicable to a wide variety of machine vision systems to sort a plurality of objects including vehicles, of a particular quality, from all objects in an image, according to width, alignment, and other criteria

described herein. It should be apparent that additional processing components, i.e., shadow elimination components, absolute distance and speed computation components, may be appended to the system components described herein without departing from the scope of the invention.
As such, it will be appreciated by one having ordinary skill in the art that various changes in the elements and their configuration and arrangement are possible without departing from the spirit and scope of the present invention as set forth in the appended claims. It will readily be appreciated by those skilled in the art that the present invention is not limited to the specific embodiments shown herein. Thus variations may be made within the scope and spirit of the accompanying claims without sacrificing the principal advantages of the invention.

We claim:
1. A method for detection and tracking of leading target comprising the steps of:
- capturing a plurality of image frames in at least one scene in front of a host system;
- processing the image frames to determine one or more image characteristics;
- extracting one or more edges by determining one or more connected components of at least one target in an intensity plane of each image frame;
- clustering of the edges using a predefined distance;

- eliminating one or more outliers based on predefined and adaptive thresholds;
- forming a bounding box around at least one cluster;
- qualifying a cluster as the target in consecutive image frames; and
- tracking the target by tracking the cluster features in consecutive frames.

2. The method as claimed in claim 1, wherein the step of tracking comprises the step of determining the direction of motion based on the lower bounds of the bounding box of the target in consecutive video frames.
3. The method as claimed in claim 1, wherein the step of detecting edges comprises the steps of:

- enhancing contrast characteristics of the image for better detection of features;
- determining the gradient of image intensity at each point; and
- retaining image pixels with intensity greater than a predefined threshold value.
4. The method as claimed in claim 1, wherein the step of clustering of the edges comprises the steps of:
- extracting edge connected components based on the edge connectivity;
- determining pixel count of the edge connected components and removing the edge connected components with pixel count less than a predefined threshold;
- performing single link clustering using a predefined distance value; and

- identifying a cluster having a predefined number of connected
components and maximum number of pixels in the connected
components in the cluster and determining the centroid for the cluster.
5. The method as claimed in claim 3, wherein the step of eliminating one or
more outliers comprises the steps of:
- re-clustering image components and defining new centroid for the identified cluster; and
- eliminating outliers based on the adaptive distance threshold during re clustering.
6. The method as claimed in claim 1, wherein the step of forming the
bounding box comprises the steps of:
- determining extreme coordinates of the connected components in the cluster representing the target;
- determining vertical profile of pixels of edge connected components of the cluster; and
- performing horizontal profiling of the pixels of the connected components in the cluster for fme tuning the top and bottom bounds of the bounding box.
7. A system for detection and tracking of a leading target comprising
- means for image acquisition;
- means for processing coupled to the means for image acquisition for processing one or more acquired images to extract at least one image characteristic in the corresponding intensity plane for each image frame; and

- storage device coupled to the means for processing for storing
predefined, acquired and processed information.
8. The system as claimed in claim 6, wherein the means for processing
is configured to:
- enhance contrast characteristics of image pixels for better detection of features in an acquired image;
- transmit information to one or more systems communicatively coupled to the system.
9. The system as claimed in claim 6, wherein the storage device stores
at least operational instructions for the system, warnings, messages,
system status information and output graphics and sound
information.
10. The system as claimed in claim 6, wherein the system comprises one or more interfaces for communicatively coupling the system to one or more visual, sound and tactile indicators, one or more networks and one or more navigation systems.
11. A system for detection and tracking of a leading target substantially as herein described with reference to and as illustrated by the accompanying drawings.
12. A method for vehicle detection and tracking substantially as herein described with reference to and as illustrated by the accompanying drawings.

13. A computer program product for detection and tracking of a leading target, comprising one or more computer readable media configured to perform the method as claimed in any of the claims 1-6.