Claims:We Claim:

1. A method for occlusion detection during ground based object tracking, the method comprising:
obtaining a reference template (frame) including size, shape and orientation of tracking object captured by analog camera, wherein the obtained reference template is stored in a memory;
tracking a size, shape and orientation of an object (successive frame) continuously for occlusion detection, wherein the tracking is achieved by analog camera in video frame and store the same in memory;
computing a FFT for the reference frame and the successive frame to determine the cross power spectrum;
calculating cross correlation of the computed cross power spectrum of the reference frame and the successive frame to retrieve a cross correlation function value ‘C’, wherein ‘C’ is a threshold value; and
measuring the retrieved cross correlation function value in order to determine one or more parameters including change in object orientation, object size and object shape.

2. The method of claim 1, wherein if the cross correlation function value of ‘C’ is in the range of C1 to C0 indicates the change in object orientation and object size, then the detection of tracking is identified as normal tracking of target.

3. The method of claim 1, wherein if the cross correlation function value of ‘C’ is in the range of C2 to C1 indicate the change in orientation, size and shape of the target being tracked.

4. The method of claim 1, wherein if the cross correlation function value of ‘C’ is less than or equal to C1 and greater than C2 indicates that the reference template stored in memory block during tracking phase is updated with new template.

5. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than or equal to C3 and less than C2 indicates that the detection of tracking is consider as occlusion detection or the target being tracked is considered to be under occlusion.

6. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than and equal to C3 and less than C2 indicates that the memory track has to be enabled or trajectory (error signal) prior to occlusion detection will be used for tracking.

7. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than and equal to C3 and less than C2 indicates that the detection of tracking mode is selected as a memory track mode.

8. The method of claim 1, wherein if the cross correlation function value of ‘C’ is less than C3 indicates that the detection of tracking is considered to be complete track lost or target is completely occluded and the object tracking mode is unlocked and manual tracking is initiated. , Description: FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“A method for occlusion detection during ground based object tracking”
By
BHARAT ELECTRONICS LIMITED
Nationality: Indian
M/s. Bharat Electronics Limited, Corporate Office, Outer Ring Road, Nagavara, Bangalore-560045, Karnataka, India

The following specification particularly describes the invention and the manner in which it is to be performed.

Field of the invention
The present invention mainly relates to occlusion detection and more particularly to a method for occlusion detection using correlation coefficient during ground based object tracking.
Background of the invention
Object tracking and detection is well known in the art which is a process of tracking an object. Usually, the tracking is performed in the context of higher-level applications that require the location and/or shape of the object in every frame. It has a variety of uses, some of which are human-computing device interaction, security and surveillance, video communication and compression, advance driving assistance system, scene understanding, autonomous navigation, augmented reality, traffic control, medical imaging and video editing, etc. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of the object, size, and the scene, non-rigid object structures, object-to-object and object-to-scene occlusions, and camera motion, etc.
Many challenges still exist while detecting an object such as illusion, low visibility, cast shadows and most importantly occlusions of object. Occlusions occur under two categories, firstly its, self-occlusion which means that, from a certain viewpoint, one part of an object is occluded by another part. Secondly, its inter-object occlusion which means when two objects being tracked occludes each other. One will review various occlusion handling methods that involved single and multiple cameras according to their application.
Presently, mathematical modelling is used for and measuring of occlusion detection in a ground objects tracking and air target tracking. For example, US5602760A titled “Image based detection and tracking system and processing method employing clutter measurements and signal to clutter ratio” describes a method for measuring and quantifying clutter, and target detection and tracking systems that employs wavelet-based clutter quantification to generate a clutter number and a signal-to-clutter ratio derived therefrom to achieve improved target detection performance. The method processes video signals representative of an image scene containing a target and background clutter to provide for more accurate tracking of the target by a tracker(s). The method comprises processing the video signals to compute a wavelet clutter number, processing the video signals to compute a signal-to clutter ratio using the wavelet clutter number, and generating a pointer to a lookup table that sets parameters and selects the tracker that is to be used to track the target based upon the computed signal-to clutter ratio.
Further, there are many methods available for clutter measurement in image based object detection and tracking such as Global or background only measures such as image standard deviation, entropy or edge per unit area, Target dependent measure that incorporates a priori target information which becomes the basis of measure, because clutter is then relative term, Two dimensional wavelet based clutter measurement and signal to clutter ratios in image based detection and tracking system.
Therefore there is a need in the art with the method for occlusion detection using correlation coefficient as a measure for ground based object tracking and to solve the above mentioned limitations.
Summary of the Invention
An aspect of the present invention is to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below.
Accordingly, in one aspect of the present invention relates to a method for occlusion detection during ground based object tracking, the method comprising: obtaining a reference template (frame) including size, shape and orientation of tracking object captured by analog camera, wherein the obtained reference template is stored in a memory; tracking a size, shape and orientation of an object (successive frame) continuously for occlusion detection, wherein the tracking is achieved by analog camera in video image frame and store the same in memory; computing a FFT for the reference frame and the successive frame to determine the cross power spectrum; calculating cross correlation of the computed cross power spectrum of the reference frame and the successive frame to retrieve a cross correlation function value ‘C’, wherein ‘C’ is a threshold value; and measuring the retrieved cross correlation function value in order to determine one or more parameters including change in object orientation, object size and object shape.
Other aspects, advantages, and salient features of the invention will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the invention.
Brief description of the drawings
The above and other aspects, features, and advantages of certain exemplary embodiments of the present invention will be more apparent from the following description taken in conjunction with the accompanying drawings in which:
Figure 1 shows the block diagram of occlusion detection and cross correlation function according to one embodiment of the present invention.
Figure 2 shows the object under tracking according to one embodiment of the present invention.
Figure 3 shows the flowchart of an object tracking under an occluded condition according to one embodiment of the present invention.
Persons skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and may have not been drawn to scale. For example, the dimensions of some of the elements in the figure may be exaggerated relative to other elements to help to improve understanding of various exemplary embodiments of the present disclosure. Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.
Detailed description of the invention
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the invention as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention are provided for illustration purpose only and not for the purpose of limiting the invention as defined by the appended claims and their equivalents.
It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.
By the term “substantially” it is meant that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations and other factors known to those of skill in the art, may occur in amounts that do not preclude the effect the characteristic is intended to provide.
Figs. 1 through 3, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way that would limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged communications system. The terms used to describe various embodiments are exemplary. It should be understood that these are provided to merely aid the understanding of the description, and that their use and definitions, in no way limit the scope of the invention. Terms first, second, and the like are used to differentiate between objects having the same terminology and are in no way intended to represent a chronological order, unless where explicitly stated otherwise. A set is defined as a non-empty set including at least one element.
The present invention describes a methodology for ground based object tracking where the background modelling is complex. Further, this invention is pertaining to measure for occlusion detection while ground objet is being tracked, using video camera and pan tilt system. Image processing based ground object tracking comprises of two analog video cameras one for day operation CCD (charge coupled device) and another for day and night operation TI (Thermal Imager) camera. Video camera image frames are in PAL (Phase Alteration by Line) format at the rate of 25 frames per second. Video cameras are mounted on two axis stabilized pan and tilt system controlled by processing block. Pan and tilt system controlled by any computing platform generate pan tilt command at the rate of frame rate. The video camera can be controlled by user in terms of change in field of view, zoom level and focus through any standard interface. Analog cameras are connected to video ADC (Analog to Digital Converter) and output of video analog to digital converter is connected to video processing device like (FPGA) Field Programmable Gated Array. Camera field of view, zoom and focus must be adjusted in such a way that an object being tracked will be seen on display.
The area occupied by an object in video image frame can be within minimum 8x8 pixels to maximum 256x256 pixels. The number of pixels or window size associated to object size is stored in memory interfaced to processing device. Number of pixels as similar to object window size is selected in successive frame and can be stored in another memory. In this invention a cross correlation function between two reference object and shifted object in next successive frame is calculated. As long as line of sight is maintain between object and sensor an object will be tracked smoothly. If size, orientation or shape of an object is changed during tracking, normalized cross correlation function value is also changed. Once orientation, size shape of object under tracking is changed, value of normalized cross correlation function starts reducing from maximum value. Percentage reduction in the value of cross correlation during tracking phase is used as measure to detect change in object size and occlusion. During object tracking a threshold of cross correlation is used as a measure for performance of track quality, if threshold is falls below calculated value, object under track is being unlocked. Fast Fourier Transform (FFT) based Phase correlation method is used for object tracking in a complex background. Cross power spectrum between reference frame and next successive frame in that object is shifted is calculated.
Cross correlation function is estimated using Fast Fourier Transform (FFT) based Phase correlation method. Cross correlation coefficient is computed between two successive frames by finding 2-D FFT of reference frame and complex conjugate of FFT of successive frames and estimating IFFT (Inverse Fourier transform). Calculated cross power spectrum value between two successive frames multiplied by 100 to upscale cross correlation value. Range of calculated maximum value of cross correlation function is used as a measure for occlusion detection, reference template updating, object track performance, object track unlock and track break. The change in object size, shape and orientation which are under tracked with respect to reference frame is detected using cross correlation function as a measure.
Figure 1 shows the block diagram of occlusion detection and cross correlation function according to one embodiment of the present invention.
The figure shows a block diagram for measure for occlusion detection in a ground based object tracking. The object tracking comprises of two analog video cameras one for day time CCD (Charge coupled device) and other one for night time tracking TI (Thermal Imager) camera. Video camera gives out image frames in PAL (Phase Alteration by Line) format at the rate of 25 frames per second. Video cameras are mounted on two axis stabilized pan and tilt system. The pan and tilt system controlled by any computing platform generate pan tilt command at the rate of 20 ms through serial communication port. The video camera can be controlled by user in terms of change in field of view, zoom level and focus through serial port command interface. Analog cameras are connected to video ADC (Analog to Digital Converter) and output of video analog to digital converter is connected to video processing device like (FPGA) Field Programmable Gated Array. Camera field of view, zoom and focus must be adjusted in such a way that an object being tracked will be seen on display. The area occupied by an object in video image frame can be within minimum 8x8 pixels to maximum 256x256 pixels. The number of pixels or window size associated to object size is stored in memory interfaced to processing device. Number of pixels as similar to object window size is selected in successive frame and can be stored in another memory. As long as line of sight is maintain between object and sensor object will be tracked smoothly. If the size, orientation and shape of the object are changed during object tracking the value of normalized cross correlation function will be changed. The value of normalized cross correlation function starts reducing from maximum value. Percentage reduction in the value of cross correlation during tracking phase is used as measure to estimate change in object size or occlusion detection. During object tracking a threshold Magnitude of cross correlation is used as a measure for performance of track quality, is fall below some threshold level object under track is being unlocked
The present invention uses a processor 10 which is used for finding correlation coefficient using FFT based phase correlation function. The value of cross correlation function is used as a measure for occlusion detection and change in object size shape and orientation of object being tracked. During ground object tracking a reference image template of size selected by operator is stored in a memory 12. Object being tracked, window size in successive frame is selected in reference frame by operator is stored in memory 14.
The memory block 14 stores image from next successive frame to find cross correlation function to determine object shift from two successive frames. Processor 10 performs the 2-D FFT in an object window in a reference frame and object window in successive frame of size selected by operator. The cross correlation function estimated in processor block 10 is used as a measure for occlusion detection and change in target size shape and target orientation. A comparator block 16 used to find the range of cross correlation function to use as a measure to select a template updating, memory tracking and target unlocking.
In one embodiment the present invention relates to a method for occlusion detection during ground based object tracking, the method comprising: obtaining a reference template (frame) including size, shape and orientation of tracking object captured by analog camera, wherein the obtained reference template is stored in a memory; tracking a size, shape and orientation of an object (successive frame) continuously for occlusion detection, wherein the tracking is achieved by analog camera in video image frame and store the same in memory; computing a FFT for the reference frame and the successive frame to determine the cross power spectrum; calculating cross correlation of the computed cross power spectrum of the reference frame and the successive frame to retrieve a cross correlation function value ‘C’, wherein ‘C’ is a threshold value; and measuring the retrieved cross correlation function value in order to determine one or more parameters including change in object orientation, object size and object shape.
The Cross power spectrum is calculated by using reference frame in memory block 12 and successive frame in memory block 14. The Cross power spectrum is calculated by finding FFT (Fast Fourier Transform) and complex conjugate of FFT of next successive frame and multiplied both by element wise and divides by mod value of multiplied output. The value of correlation coefficient (CR) is calculated by taking IFFT (Inverse Fourier transform) of cross power spectrum calculated. The Value of cross correlation coefficient is used as a reference measure and stored in a memory buffer at input frame rate. The cross correlation coefficient value will be multiplied by factor 100 to upscale the value. The dynamic range of cross correlation coefficient will fall in the range of -100 to 100 after multiplying by 100. The performance of ground target tracking is measured, using the calculated value of cross correlation function on the scale of 0 to 100 as minus sign indicates the direction. The value of cross correlation coefficient is used as a measure for change in object orientation, object size and object shape.
Figure 2 shows the object under tracking according to one embodiment of the present invention.
The figure shows the object under tracking. The original object size embedded in a window 22 is selected by operator as a reference template. The size, shape and orientation of object under tracked 24 being changed because of maneuvering and change in distance from object to sensor. Cross correlation of object orientation and size shape change 24 is calculated and used as a measure for reference template updating. Cross correlation between reference template and object under occlusion 26 is calculated and used as a measure for tracking. The occlusion 26 comes in front of object during tracking is detected by cross correlation function value. Object under tracking 28 in figure 2 is covered completely by occlusion is being detected by reduction in value of cross correlation function. The cross correlation between object reference template 22 and completely occluded object 26 is calculated and used as a measure to decide object can be tracked or unlocked.
Figure 3 shows the flowchart of an object tracking under an occluded condition according to one embodiment of the present invention.
The complete flow chart for measure based occlusion detection is depicted in Figure 3. Initially, an analog camera is connected to Analog to digital converter IC (Video ADC) to digitized analog video. Analog camera is mounted on a two axis stabilized platform pan platform may move either in elevation or azimuth direction based upon target movement. The digitized video is converted into sequence of images using processing device like Field Programmable Gated Array (FPGA) to find object position shift in next successive frame. Object position shift in successive frames are calculated at video frame rate. The object being tracked may have in video frame captured by analog camera. A window size encapsulating object being tracked is selected based upon object size and shape display on video frame. Image data within window selected by operator is called as reference template stored in a memory which is connected to processing device. Fast Fourier Transform (FFT) based Phase correlation method can be used for object tracking in a complex background. Fast Fourier Transform (FFT) of reference template is calculated and real and imaginary part is stored in separate memories attached with processing device. Same size of object window and object position being tracked is selected in reference frame as well as successive frame to estimate the object shift in next frame. Complex conjugate of Fourier transform in a fixed window centred on centre of image frame is calculated for every successive frame being tracked. Fourier transforms of reference frame or object template is multiplied element wise with complex conjugate of Fourier transform of successive frame. Absolute value or mod of the multiplied value of reference frame and successive frame is calculated. Cross power spectrum between reference frame and successive frame in which object is shifted computed. A correlation coefficient between reference frame and successive frame is calculated after performing inverse transform this method is already well known and being used. Normalized cross correlation by applying inverse Fourier transform is calculated. Value of cross correlation function is varying between -1 to 1. As long as the object being tracked is visible by camera the normalized cross correlation is between -1 to 1. Sign of correlation coefficient is denoting the direction of object being moved.
Moreover, this invention is related to find the situation where occlusion comes in front of object or any obstacle between camera and object. In this innovation a measure to find change in object size and change in object orientation, occlusion detection for ground target tracking is proposed. The value of cross correlation function is multiplied with constant value 100 to scale up the normalized cross correlation value from 0 to 100. Value of normalized cross correlation is used as a measure to determine change in object orientation and change in object size or target is completely occluded. The comparator 16 finds the range of cross correlation function after multiplying by 100 with calculated cross correlation value. The intermediate value of cross correlation coefficient is denoted by C0, C1, C2, C3 and C and ranged from 0 to 100.
The change in object orientation and object size is reflected in change in the range of C0 to C3, where C0 is 100. The highest value of cross correlation function in a range of C1 to C0 is considered to be normal tracking of target. The value of cross correlation function in a range of C2 to C1 is indicating the change in orientation, size and shape of target being tracked. The reference template stored in memory block 12 during tracking phase is updated with new template if the value of cross correlation function is less than or equal to C1 and more than C2. The Magnitude of cross correlation function is greater than or equal to C3 and less than C2 is considered to be occlusion detection or the target being tracked is considered to be under occlusion. The magnitude of cross correlation function is less than C2 and more than and equal to C3 than memory track is enabled or trajectory (error signal) prior to occlusion detection is being used for tracking. If the cross correlation function value of ‘C’ is greater than and equal to C3 and less than C2 indicates that the detection of tracking mode is selected as a memory track mode. If the cross correlation function value of ‘C’ is less than C3 indicates that the detection of tracking is considered to be complete track lost or target is completely occluded and the object tracking mode is unlocked and manual tracking is initiated. The memory track will be continue till four seconds to re-acquire the target without updating reference template in memory block 12. The value of cross correlation function will be checked at rate of image frame rate or 40 mili second.
Those skilled in this technology can make various alterations and modifications without departing from the scope and spirit of the invention. Therefore, the scope of the invention shall be defined and protected by the following claims and their equivalents.
FIGS. 1-3 are merely representational and are not drawn to scale. Certain portions thereof may be exaggerated, while others may be minimized. FIGS. 1-3 illustrate various embodiments of the invention that can be understood and appropriately carried out by those of ordinary skill in the art.
In the foregoing detailed description of embodiments of the invention, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description of embodiments of the invention, with each claim standing on its own as a separate embodiment.
It is understood that the above description is intended to be illustrative, and not restrictive. It is intended to cover all alternatives, modifications and equivalents as may be included within the spirit and scope of the invention as defined in the appended claims. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein,” respectively.

We Claim:

1. A method for occlusion detection during ground based object tracking, the method comprising:
obtaining a reference template (frame) including size, shape and orientation of tracking object captured by analog camera, wherein the obtained reference template is stored in a memory;
tracking a size, shape and orientation of an object (successive frame) continuously for occlusion detection, wherein the tracking is achieved by analog camera in video frame and store the same in memory;
computing a FFT for the reference frame and the successive frame to determine the cross power spectrum;
calculating cross correlation of the computed cross power spectrum of the reference frame and the successive frame to retrieve a cross correlation function value ‘C’, wherein ‘C’ is a threshold value; and
measuring the retrieved cross correlation function value in order to determine one or more parameters including change in object orientation, object size and object shape.

2. The method of claim 1, wherein if the cross correlation function value of ‘C’ is in the range of C1 to C0 indicates the change in object orientation and object size, then the detection of tracking is identified as normal tracking of target.

3. The method of claim 1, wherein if the cross correlation function value of ‘C’ is in the range of C2 to C1 indicate the change in orientation, size and shape of the target being tracked.

4. The method of claim 1, wherein if the cross correlation function value of ‘C’ is less than or equal to C1 and greater than C2 indicates that the reference template stored in memory block during tracking phase is updated with new template.

5. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than or equal to C3 and less than C2 indicates that the detection of tracking is consider as occlusion detection or the target being tracked is considered to be under occlusion.

6. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than and equal to C3 and less than C2 indicates that the memory track has to be enabled or trajectory (error signal) prior to occlusion detection will be used for tracking.

7. The method of claim 1, wherein if the cross correlation function value of ‘C’ is greater than and equal to C3 and less than C2 indicates that the detection of tracking mode is selected as a memory track mode.

8. The method of claim 1, wherein if the cross correlation function value of ‘C’ is less than C3 indicates that the detection of tracking is considered to be complete track lost or target is completely occluded and the object tracking mode is unlocked and manual tracking is initiated.

Abstract
A method for occlusion detection during ground based object tracking

The present invention mainly relates to occlusion detection and more particularly to a method for occlusion detection using correlation coefficient during ground based object tracking, the method comprising: obtaining a reference template (frame) including size, shape and orientation of tracking object captured by analog camera, wherein the obtained reference template is stored in a memory; tracking a size, shape and orientation of an object (successive frame) continuously for occlusion detection, wherein the tracking is achieved by analog camera in video image frame and store the same in memory; computing a FFT for the reference frame and the successive frame to determine the cross power spectrum; calculating cross correlation of the computed cross power spectrum of the reference frame and the successive frame to retrieve a cross correlation function value ‘C’, wherein ‘C’ is a threshold value; and measuring the retrieved cross correlation function value in order to determine one or more parameters including change in object orientation, object size and object shape.
Figure 1 (for publication)