CLIAMS:We Claim:
1. A method for panoramic view generation in real time by an image processing device, the method comprising:
receiving, by a video processor of the image processing device, an input video signal, comprising a plurality of frames from an image sensor, wherein each frame of the plurality of frames comprises one or more columns;
receiving, by the video processor, one or more input parameters comprising at least count of panoramic strips, Field of View (FOV) and Revolution Per Minute (RPM) associated with a pan-tilt system coupled with the image sensor;
estimating, by the video processor, one or more first new columns of each frame that overlap with the successive frame based on the received input parameters;
determining, by the video processor, one or more second new columns of each frame from among the one or more first new columns that overlap with the successive frame based on at least one of minimum distance and minimum cost calculated between the first new columns of the successive frames; and
generating, by the video processor, a panoramic view of the input video signal by concatenating the one or more second new columns between the successive frames.

2. The method as claimed in claim 1, wherein receiving the input video signal comprising the steps of receiving an analog input video signal and converting the received analog input video signal into corresponding digital video signal.

3. The method as claimed in claim 1, wherein determining the one or more second new columns comprising:
dividing each of the one or more first new columns into a plurality of blocks, each block having a predetermined number of pixels;
calculating at least one of an Euclidean distance and minimum cost involved between each of the plurality of blocks; and
identifying the one or more first new columns having minimum Euclidean distance and minimum cost as the one or more second new columns that overlap between two successive frames.

4. The method as claimed in claim 1, further comprising:
dividing the generated panoramic view of the input video signal into a plurality of panoramic strips;
determining difference between the successive panoramic strips to identify one or more valid moving objects in each panoramic strip based on a predetermined panoramic threshold value; and
generating an object detected panoramic view of the input video signal based on the determined difference, aspect ratio and size of the one or more valid moving objects.

5. A system for panoramic view generation and automatic object detection in real time, the system comprising:
an image sensor capable of capturing an input video signal, the input video signal comprising a plurality of frames and each frame comprising one or more columns; and
a video processor coupled with image sensor and configured to:
receive one or more input parameters comprising at least number of panoramic strips, Field of View (FOV) and Revolution Per Minute (RPM) associated with a pan-tilt system coupled with the image sensor;
estimate one or more first new columns of each frame that overlap with the successive frame based on the received input parameters;
determine one or more second new columns of each frame from the one or more first new columns of that frame overlapping with the successive frame based on at least one of minimum distance and minimum cost calculated between the first new columns of the successive frames; and
generate a panoramic view of the input video signal by concatenating the one or more second new columns that overlap between the successive frames.

6. The system as claimed in claim 5, wherein the video processor is configured to determine one or more second new columns by the steps of:
dividing each of the one or more first new columns into a plurality of blocks, each block having a predetermined number of pixels;
calculating at least one of an Euclidean distance and minimum cost involved between the plurality of blocks; and
identifying the one or more first new columns having minimum Euclidean distance and minimum cost as the one or more second new columns that overlap between two successive frames.

7. The system as claimed in claim 5, wherein the video processor is further configured to:
divide the generated panoramic view of the input video signal into one or more panoramic strips;
determine difference between the successive panoramic strips to identify one or more valid moving objects in each panoramic strip based on a predetermined panoramic threshold value; and
generate an object detected panoramic view of the input video signal based on the determined difference, aspect ratio and size of the one or more valid objects.

8. The system as claimed in claim 5, further comprises a video decoder, coupled with the image sensor, capable of receiving an analog input video signal and converting the received analog input video signal into corresponding digital input video signal.

9. The system as claimed in claim 5, further comprises one or more storage devices coupled with the video processor, configured to store the one or more panoramic strips, the one or more valid objects and the predetermined panoramic threshold value.

10. The system as claimed in claim 5, further comprising a display device for displaying at least the object detected panoramic view, live video display, zoomed video display, RPM, FOV, the number of panoramic strips and so on.
,TagSPECI:FIELD OF THE PRESENT DISCLOSURE
The present disclosure primarily relates to image processing, more particularly, to a method and system for panoramic view generation and automatic object detection in real time.
BACKGROUND
Generally, conventional photographic systems that create panoramic images are available. These systems use lens and mirror assembly to capture the light striking at any point in space coming from 360 degrees that converges on a center point from image having low resolution and have distortion. Few conventional systems generate panoramic view using multiple videos from multiple sensors. Few other systems generate panoramic view in non-real time using image processing techniques. Some of the conventional systems are described below.
US20110102542A1 titled “System and Method for Panoramic Image Stitching” describes a method for capturing images of a target, such as test tube, that is presented in front of an optical imager and rotated 360 degrees. The image capturing system captures series of images which are then stitched together to form a panoramic image of the target. The panoramic image may then be processed to decode any information placed on the target, capture other information placed on the target, or obtain information about the target. Since the camera is stationary at one point hence this method cannot be used to display 360 degree view of any place/location.

US6426774B1 titled “Panoramic Camera” describes a camera device virtually all of the light that converges on a point in space is captured. In this light striking at any point in space is captured if it comes from any direction, 360 degrees around the point and from angles 50 degrees or more above and below the horizon. The panoramic image is recorded as a two dimensional annular image. However, these conventional systems generate panorama based on offline data which requires more memory and use multiple sensors to capture the images.
Further, these conventional systems generate panoramic views based on mechanical inputs from a mechanical device that meets with day-to-day wear and tear, hence may lead to incorrect image overlap estimation and therefore inaccurate panoramic view generation. Few other conventional systems fail to automatically detect moving objects during the panoramic view generation avoiding false alarms of object detection.
Therefore, there is a need for a method and system for panoramic view generation and automatic object detection in real time that is independent of image sensors and that overcome the above mentioned difficulties or problems. Consequently, those skilled in the art will appreciate the present disclosure that provides many advantages and overcomes all the above and other limitations.
SUMMARY

The shortcomings of the prior art are overcome and additional advantages are provided through the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed disclosure.

Accordingly, the present disclosure relates to a method of panoramic view generation and automatic object detection in real time. The method comprises the step of receiving an input video signal, comprising a plurality of frames, wherein each frame of the plurality of frames comprises one or more columns. The method further comprises the step of receiving one or more input parameters comprising at least count of panoramic strips, Field of View (FOV) and Revolution Per Minute (RPM) associated with a pan-tilt system and estimating one or more first new columns of each frame that overlap with the successive frame based on the received input parameters. Upon estimating the first columns, one or more second new columns of each frame from among the one or more first new columns that overlap with the successive frame based on at least one of minimum distance and minimum cost calculated between the first new columns of the successive frames is determined. Based on the determination of the second new columns, a panoramic view of the input video signal is generated by concatenating the one or more second new columns between the successive frames.
Further, the present disclosure relates to a system for panoramic view generation and automatic object detection in real time. The system comprises an image sensor capable of capturing an input video signal, the input video signal comprising a plurality of frames and each frame comprising one or more columns. The system further comprises a video processor coupled with image sensor. The video processor receives one or more input parameters comprising at least number of panoramic strips, Field of View (FOV) and Revolution Per Minute (RPM) associated with a pan-tilt system coupled with the image sensor. The video processor estimates one or more first new columns of each frame that overlap with the successive frame based on the received input parameters and determines one or more second new columns of each frame from the one or more first new columns of that frame overlapping with the successive frame based on at least one of minimum distance and minimum cost calculated between the first new columns of the successive frames. Based on the determination of the second new columns, the video processor generates a panoramic view of the input video signal by concatenating the one or more second new columns that overlap between the successive frames.

The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

The features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding is that these drawings depict only several embodiments in accordance with the disclosure and are therefore, not to be considered limiting of its scope. The disclosure will be described with additional specificity and detail through use of the accompanying drawings.

Figure 1 illustrates an exemplary system for panoramic view generation and automatic object detection in real time in accordance with an embodiment of the present disclosure;

Figure 2 illustrates an exemplary block diagram of a real time panoramic view generation and object detection device in accordance with an embodiment of the present disclosure;

Figure 3a illustrates perspective view of image sensor’s FOV and image overlap in accordance with an embodiment of the present disclosure.

Figure 3b illustrates an exemplary image overlapping between successive frames in accordance with an embodiment of the present disclosure;

Figure 4 illustrates exemplary object detection in real time from successive panorama in accordance with an embodiment of the present disclosure;

Figure 5 illustrates a flowchart for real time panoramic view generation and automatic object detection in accordance with an embodiment of the present disclosure;

Figure 6 is a block diagram of an exemplary computing system for implementing embodiments consistent with the present disclosure;

The figures depict embodiments of the disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION

While the present disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the present disclosure to the particular forms disclosed, but on the contrary, the present disclosure is to cover all modifications, equivalents, and alternative falling within the scope of the present disclosure.

The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by “comprises… a” does not, without more constraints, preclude the existence of other elements or additional elements in the system or apparatus.

In the following detailed description of the embodiments of the present disclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the present disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.

The present disclosure relates to a method and system for panoramic view generation and object detection in real time. In one embodiment, an input video signal is received and processed to generate digitized form of the input video signal. One or more input parameters obtained during capturing of the input video signal by a pan-tilt system is also received. Based on the input parameters, the number of new columns that overlap between successive frames of the input video signal is determined. The exact number of columns that overlap between the successive frames is further determined by calculating Euclidean distance and minimum cost using Block Truncated Coding (BTC) and Mean Absolute Differing (MAD) techniques. Upon calculating, the panoramic views are generated and presence of moving objects in the generated panoramic views is detected based on size, aspect ratio and positon of detected objects to avoid false alarms.

Figure 1 illustrates an exemplary system 100 for panoramic view generation and automatic object detection in real time in accordance with an embodiment of the present disclosure.

As illustrated, the system 100 hosts one or more panoramic images for display to end user. The system 100 comprises an image processing device or a real time panoramic view generation and object detection device 102 (hereinafter referred to as device 102) and a pan-tilt system 104 coupled with the device 102 and at least one image sensor for example, image sensor 106A and image sensor 106B (collectively referred to as image sensor 106). Only one image sensor is enabled at a time during day and night. In one aspect, the image sensor 106A captures the video signals during day time and the image sensor 106B captures the video signals during night. The image sensor 106A may be for example, a Thermal Imager (TI) camera and the image sensor 106B may be, for example a Charge Coupled Device (CCD) camera used for capturing the input video signals. The captured video signals are transmitted to the device 102 via a communication channel 108. The communication channel 108 transmits the captured video signals as Phase Alternating Line (PAL) video signals to the device 102.

The system 100 further comprises a user interface 110 that enables transmission of commands or instructions from the end user, for example commands to capture the video signal, to adjust the position of lens and mirror assembly in the pan-tilt system 104, to display live video, to display zoomed video or to display of sector and so on. The user interface 110 may be, for example a Man-Machine or a graphical user Interface that receives input or commands from the end user through port RS422.

The system 100 further comprises a display device 112 for displaying the generated panoramic views in one or more panoramic strips 114-1, 114-2 (collectively referred to as 114), live video 116, zoomed video 118, parameters like RPM 120, FOV 122 and count of panoramic strips 124 and so on. The display device 112 displays the complete panoramic view in various strips due to limitation in the screen resolution 126 of the display device 112. The display device 112 receives the PAL panoramic views generated by the device 102 via the communication channel 108 for displaying.

The device 102 comprises at least a video processor 128 to process the input video signal received from the image sensors 106A and 106B and generate PAL panoramic views of the input video signal. In one embodiment, the device 102 may be a panoramic view generation and object detection device 102 as illustrated in Figure 2. The device 102 comprises the video processor 128, a video decoder 202, a video digital-to-analog converter 204 and one or more storage devices including a dual port RAM (DPRAM) 206 and Bank of Synchronous Burst Synchronous RAM (SBSRAM) 208 coupled with the video processor 128.

The video decoder 202 receives the input analog PAL video signal and converts the received input analog PAL signal into a corresponding digital video signal. In one embodiment, the video decoder 202 samples the received input analog PAL signal at a predetermined rate for example, 27 MHZ and in a predetermined bit format, for example 16 bit format (8 bit luminance and 8 bit chrominance component). The digitized video signal in the predetermined bit format is sent from the video decoder 202 to the video processor 128 for further processing.

The video processor 128 pre-process the received digitized video signal for example, the video processor 128 converts the video signal in 16 bit format into corresponding colored video signal having 24 bit color format(8 bit- Red, 8 bit – Green and 8 bit- Blue). The video processor 128 determines overlap estimation between the successive frames of the colored video signal. The video signal may comprise one or more frames and each frame may comprise one or more columns of pixels. The video processor 128 determines the overlapping of successive frames and also determines the number of new columns based on the overlapping. In one embodiment, the video processor 128 determines the number of new columns per frame based on one or more input parameters including at least Revolution per Minute (RPM) 120, Field of View (FOV) 122, Frame period, degree of revolution, and number of pixels within the Field of View 122 and so on. Frame period may be the time required for scanning a frame, a typical frame period may be 20ms. The input parameters like FOV and RPM are provided by the pan-tilt system 104 to the video processor 128 via the user interface 110. Based on the values of the input parameters received, the video processor 128 determines the number of new columns per frame. In one example, the number of new columns per frame 304 as illustrated in Figure 3a is determined according to the below equation (1):

New columns per frame = f (Frame Period, FOV) ……...…(1)

where f (.) relates FOV and frame period to the number of new columns.

The number of columns per frame thus determined is stored in internal memory of the video processor 128 at a predetermined rate for example, 20ms. Total number of frames and the total number of pixels in the input video signal is determined by the video processor 128 based on RPM value 120. The video processor 128 determines the total number of frames in the input video signal as illustrated below in equation (2):

Total number of frames = (60/RPM) * 25 frames ………(2)

Based on the determined total number of frames, the input video signal is divided accordingly. Further, the video processor 128 divides each frame into one or more columns or pixels based on the calculation of the total number of pixels or columns as illustrated below in equation (3):

Total number of pixels = Total frames X New columns per frame ………..….(3)

Upon determining the number of new columns, the number of frames and number of columns per frame, the video processor 128 determines the correlation or overlap between two successive frames. In one embodiment, the video processor 128 determines the image overlap 302 or the exact shift between two successive frames using Block Truncation Coding (BTC) and Mean Absolute Differencing (MAD) based techniques as illustrated in Figures 3a and 3b. The video processor 128 retrieves a predetermined area or number of columns of the successive frames and correlates the retrieved frames to determine the overlap. In one embodiment, the video processor 128 retrieves the first or last predetermined number of columns of previous 308 or recent frame 312 stored in the internal memory of the video processor 128. For example, the last 3 columns of the previous frame 308 and first 3 columns of the recent frame 312 is retrieved and each retrieved column is divided into a plurality of blocks, each block having a predetermined number ‘n X n’ of pixels, for example, 8 X 8 pixels. Upon determining the blocks, the video processor 128 determines the exact shift between the successive frames by sliding the recent frame 306 over the previous frame 308 so as to overlap the first 3 columns of the recent frame 306 with the last 3 columns of the previous frame 308.

For each block in the sliding columns, the video processor 128 determines the Euclidean distance and minimum cost using BTC and MAD techniques. For example, the video processor 128 select two luminance values to represent each pixel in the block. These values are chosen such that the sample mean and standard deviation of the reconstructed block are identical to those of the original block. Let K be the number of pixels of an n x n block K = n2 and x1, x2, x3, …, xk be the intensity values of the pixels in a block of the original frame. The first two sample moments m1and m2 are given by equations (4) and (5).
……………. (4)

…….(5)
Sample standard deviation s is given by
……..(6)
In one embodiment, the video processor 128 determines the matching of one macro block with another based on the output of a cost function. For example, the video processor 128 determines the macro block that results in the least cost is the one that matches the closest to current block. In another embodiment, known cost functions like Mean Absolute Difference (MAD) and Mean Squared Error (MSE) may be used to determine the matching of the blocks.

Based on the determined Euclidean distance and minimum cost, the video processor 128 determines the correlation between the successive frames, for example correlation 312 between recent frame 306 and the previous frame 308, the correlation 314 between recent frame 306 and next frame 316. Based on the determination of correlations or the exact column shift, the frames can be concatenated to generate panoramic views in real time. The total number of non-overlapping pixels from successive frames is stored in the DPRAM 206. Live video 116 received from the image sensor 106 is sub-sampled and stored in the DPRAM 206 along with generated panoramic view and zoomed video 118. The video processor 128 also stores the one or more panoramic views or panoramas in the SBSRAM 208 that may be retrieved for object detection.

The video processor 128 determines that the total number of pixels as calculated in equation (3) is always greater than the screen resolution 126 and therefore, the panoramic view is divided into a plurality of strips for example, the panoramic view is divided into two strips, strip 1 having field of view 0 -180 degree and strip 2 having field of view 181- 360 degree. The count of panoramic strips 124 can also be received as input from the user via the user interface 110. Based on the determined total number of pixels and the received input value of the count of panoramic strips 124, the total number of pixels per strip may be calculated as illustrated in below equation (7):

Pixels per strip = Total number of pixels / Number of strips ………..…(7)

Based on the number of pixels per strip determined as above, the panoramic view thus generated using BTC and MAD techniques is divided into the plurality of panoramic strips 114. In one example, the number of pixels per strip 402 may be 701 pixels and the number of panoramic strips is 2. As illustrated in Figure 4, the generated first panoramic view 400 is divided into two panoramic strips 114-1 and 114-2. The panoramic strip 114-1 corresponds to the panoramic view of 0 -180 degree of revolution and the panoramic strip 114-2 corresponds to the panoramic view of 181-360 degree of revolution. . Further, the next panorama or second panoramic view 406 stored in the SBSRAM 208 is retrieved and divided into a plurality of strips 114. One or more objects in the panoramic strip 114-1 of the first panorama 400 are identified and the old location is determined. For example, the object 404 located at location 408 denoted as X1, Y1 coordinates is determined. Further, the video processor 128 determines the new location of the object 404 in the second panorama 406, for example 410 denoted as X2, Y2 and determine the spatial difference between the new and old locations by subtracting the panoramas. During the subtraction, the pixels with intensity exceeding predetermined threshold intensity are considered to be valid objects. The mean and maximum value of all the subtracted data is calculated as mentioned in below equations (8) and (9).
…………… (8)
Wherein xi indicates the subtracted pixel intensities, and N denotes no. of the pixels in the subtracted Panorama.
…(9)
The predetermined threshold value can be determined based on mean and maximum value as mentioned in equation (10) below:
………………(10)
The video processor 128 also determines the size of the objects by counting connected pixel horizontally and vertically or taking projection on horizontal and vertical plane. The video processor 128 determines stationary and moving objects based on the change in the location coordinates of the objects in different panoramas. If there is no change in the location coordinates, the size and aspect ratio, the video processor 128 removes such objects from the panoramas, thus determining only valid or moving objects and avoiding false alarms on the identification of objects.

Figure 5 illustrates a flowchart for a method of real time panoramic view generation and automatic object detection in accordance with an embodiment of the present disclosure.

As illustrated in Figure 5, the method 500 comprises one or more blocks implemented by the video processor 128 for reducing functional ambiguity from an image. The method 500 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions or implement particular abstract data types.
The order in which the method 500 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 500. Additionally, individual blocks may be deleted from the method 500 without departing from the spirit and scope of the subject matter described herein. Furthermore, the method 500 can be implemented in any suitable hardware, software, firmware, or combination thereof.
At block 502, receive input and process input video signal. In one embodiment, the video decoder 202 receives the input analog PAL video signal and converts the received input analog PAL signal into a corresponding digital video signal. In one embodiment, the video decoder 202 samples the received input analog PAL signal at a predetermined rate for example, 27 MHZ and in a predetermined bit format, for example 16 bit format (8 bit luminance and 8 bit chrominance component). The digitized video signal in the predetermined bit format is sent from the video decoder 202 to the video processor 128 for further processing. The video processor 128 pre-process the received digitized video signal for example, the video processor 128 converts the video signal in 16 bit format into corresponding colored video signal having 24 bit color format(8 bit- Red, 8 bit – Green and 8 bit- Blue).

At block 504, receive input parameters from pan-tilt system. The input parameters like FOV and pan-tilt (RPM) are provided by the pan-tilt system 104 to the video processor 128 via the user interface 110. In one embodiment, the one or more input parameters include at least Revolution per Minute (RPM) 120, Field of View (FOV) 122, Frame period, degree of revolution, and number of pixels within the Field of View and so on. Frame period may be the time required for scanning a frame, a typical frame period may be 20ms.

At block 506, determine overlap estimation using BTC and MAD techniques. In one embodiment the video processor 128 determines the number of new columns per frame based on the values to the input parameters received. The number of columns per frame thus determined is stored in internal memory of the video processor 128 at a predetermined rate. Total number of frames and the total number of pixels in the input video signal is determined by the video processor 128 based on RPM value 120. The video processor 128 determines the total number of frames in the input video signal. Based on the determined total number of frames, the input video signal is divided accordingly. Further, the video processor 128 divides each frame into one or more columns or pixels based on the calculation of the total number of pixels or columns. Upon determining the number of new columns, the number of frames and number of columns per frame, the video processor 128 determines the correlation or overlap between two successive frames is determined. In one embodiment, the video processor 128 determines the image overlap 302 or the exact shift between two successive frames using Block Truncation Coding (BTC) and Mean Absolute Differencing (MAD) based techniques.

The video processor 128 retrieves a predetermined area or number of columns of the successive frames and correlates the retrieved frames to determine the overlap. In one embodiment, the video processor 128 retrieves the first or last predetermined number of columns of previous 308 or recent frame 312 stored in the internal memory of the video processor 128. For example, the last 3 columns of the previous frame 308 and first 3 columns of the recent frame 312 is retrieved and each retrieved column is divided into a plurality of blocks, each block having a predetermined number ‘n X n’ of pixels, for example, 8 X 8 pixels. Upon determining the blocks, the video processor 128 determines the exact shift between the successive frames by sliding the recent frame 306 over the previous frame 308 so as to overlap the first 3 columns of the recent frame 306 with the last 3 columns of the previous frame 308.

For each block in the sliding columns, the video processor 128 determines the Euclidean distance and minimum cost using BTC and MAD techniques. For example, the video processor 128 select two luminance values to represent each pixel in the block. These values are chosen such that the sample mean and standard deviation of the reconstructed block are identical to those of the original block. Let K be the number of pixels of an n x n block K = n2 and x1, x2, x3, …, xk be the intensity values of the pixels in a block of the original frame. The first two sample moments m1and m2 and the Sample standard deviation s is determined.

The video processor 128 determines the matching of one macro block with another based on the output of a cost function. For example, the video processor 128 determines the macro block that results in the least cost is the one that matches the closest to current block. In another embodiment, known cost functions like Mean Absolute Difference (MAD) and Mean Squared Error (MSE) may be used to determine the matching of the blocks. Based on the determined Euclidean distance and minimum cost, the video processor 128 determines the correlation between the successive frames, for example correlation 312 between recent frame 306 and the previous frame 308, the correlation 314 between recent frame 306 and next frame 316. Based on the determination of correlations or the exact column shift, the frames can be concatenated to generate panoramic views in real time.

At block 508, perform object detection. In one embodiment, the video processor 128 determines that the total number of pixels is always greater than the screen resolution 126 and therefore, the panoramic view is divided into a plurality of strips for example, strips 0 -180 and 181- 360. The count of panoramic strips 124 can also be received as input from the user via the user interface 110. Based on the determined total number of pixels and the received input value of the count of panoramic strips 124, the total number of pixels per strip is determined.

Based on the number of pixels per strip determined, the panoramic view thus generated using BTC and MAD techniques is divided into the plurality of panoramic strips 114. In one example, the number of pixels per strip 402 may be 701 pixels and the number of panoramic strips is 2. As illustrated in Figure 4, the generated first panoramic view 400 is divided into two panoramic strips 114-1 and 114-2. The panoramic strip 114-1 corresponds to the panoramic view of 0 -180 degree of revolution and the panoramic strip 114-2 corresponds to the panoramic view of 181-360 degree of revolution. . Further, the next panorama or second panoramic view 406 stored in the SBSRAM 208 is retrieved and divided into a plurality of strips 114. One or more objects in the panoramic strip 114-1 of the first panorama 400 are identified and the old location is determined. For example, the object 404 located at location 408 denoted as X1, Y1 coordinates is determined. Further, the video processor 128 determines the new location of the object 404 in the second panorama 406, for example 410 denoted as X2, Y2 and determine the spatial difference between the new and old locations by subtracting the panoramas. During the subtraction, the pixels with intensity exceeding predetermined threshold intensity are considered to be valid objects. The mean and maximum value of all the subtracted data is calculated and the predetermined threshold value can be determined based on mean and maximum values.

In one embodiment, the video processor 128 also determines the size of the objects by counting connected pixel horizontally and vertically or taking projection on horizontal and vertical plane. The video processor 128 determines stationary and moving objects based on the change in the location coordinates of the objects in different panoramas. If there is no change in the location coordinates, the size and aspect ratio, the video processor 128 removes such objects from the panoramas, thus determining only valid or moving objects and avoiding false alarms on the identification of objects.

At block 510, display panoramic view with detected objects. In one embodiment, the display device 112 displays the generated panoramic views in one or more panoramic strips 114-1, 114-2, the live video 116, the zoomed video 118, the parameters like RPM 120, FOV 122 and the count of panoramic strips 124 and so on. The display device 112 displays the complete panoramic view in various strips due to limitation in the screen resolution 126 of the display device 112. The display device 112 receives the PAL panoramic views generated by the device 102 via the communication channel 108 for displaying.

Figure 6 is a block diagram of an exemplary computing system for implementing embodiments consistent with the present disclosure.
Variations of computing system 601 may be used for implementing all the computing systems that may be utilized to implement the features of the present disclosure. Computing system 601 may comprise a central processing unit (“CPU” or “processor”) 602. Processor 602 may comprise at least one data processor for executing program components for executing user- or system-generated requests. The processor may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. The processor 602 may include a microprocessor, such as AMD Athlon, Duron or Opteron, ARM’s application, embedded or secure processors, IBM PowerPC, Intel’s Core, Itanium, Xeon, Celeron or other line of processors, etc. The processor 602 may be implemented using mainframe, distributed processor, multi-core, parallel, grid, or other architectures. Some embodiments may utilize embedded technologies like application-specific integrated circuits (ASICs), digital signal processors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.

Processor 602 may be disposed in communication with one or more input/output (I/O) devices via I/O interface 603. The I/O interface 603 may employ communication protocols/methods such as, without limitation, audio, analog, digital, monoaural, RCA, stereo, IEEE-1394, serial bus, universal serial bus (USB), infrared, PS/2, BNC, coaxial, component, composite, digital visual interface (DVI), high-definition multimedia interface (HDMI), RF antennas, S-Video, VGA, IEEE 802.n /b/g/n/x, Bluetooth, cellular (e.g., code-division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), WiMax, or the like), etc.

Using the I/O interface 603, the computing system 601 may communicate with one or more I/O devices. For example, the input device 604 may be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touchpad, trackball, sensor (e.g., accelerometer, light sensor, GPS, gyroscope, proximity sensor, or the like), stylus, scanner, storage device, transceiver, video device/source, visors, etc. Output device 605 may be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, or the like), audio speaker, etc. In some embodiments, a transceiver 606 may be disposed in connection with the processor 602. The transceiver may facilitate various types of wireless transmission or reception. For example, the transceiver may include an antenna operatively connected to a transceiver chip (e.g., Texas Instruments WiLink WL1283, Broadcom BCM4750IUB8, Infineon Technologies X-Gold 618-PMB9800, or the like), providing IEEE 802.11a/b/g/n, Bluetooth, FM, global positioning system (GPS), 2G/3G HSDPA/HSUPA communications, etc.

In some embodiments, the processor 602 may be disposed in communication with a communication network 608 via a network interface 607. The network interface 607 may communicate with the communication network 608. The network interface 607 may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/40/400 Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE 802.11a/b/g/n/x, etc. The communication network 608 may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface 607 and the communication network 608, the computing system 601 may communicate with devices 609, 610, and 611. These devices may include, without limitation, personal computer(s), server(s), fax machines, printers, scanners, various mobile devices such as cellular telephones, smartphones (e.g., Apple iPhone, Blackberry, Android-based phones, etc.), tablet computers, eBook readers (Amazon Kindle, Nook, etc.), laptop computers, notebooks, gaming consoles (Microsoft Xbox, Nintendo DS, Sony PlayStation, etc.), or the like. In some embodiments, the computer system 601 may itself embody one or more of these devices.

In some embodiments, the processor 602 may be disposed in communication with one or more memory devices (e.g., RAM 613, ROM 6Error! Reference source not found.14, etc.) via a storage interface 612. The storage interface may connect to memory devices including, without limitation, memory drives, removable disc drives, etc., employing connection protocols such as serial advanced technology attachment (SATA), integrated drive electronics (IDE), IEEE-1394, universal serial bus (USB), fiber channel, small computer systems interface (SCSI), etc. The memory drives may further include a drum, magnetic disc drive, magneto-optical drive, optical drive, redundant array of independent discs (RAID), solid-state memory devices, solid-state drives, etc.

The memory 615 may store a collection of program or database components, including, without limitation, an operating system 6Error! Reference source not found.16, user interface application 6Error! Reference source not found.17, web browser 618, mail server 619, mail client 620, user/application data 621 (e.g., any data variables or data records discussed in this disclosure), etc. The operating system 616 may facilitate resource management and operation of the computing system 601. Examples of operating systems include, without limitation, Apple Macintosh OS X, UNIX, Unix-like system distributions (e.g., Berkeley Software Distribution (BSD), FreeBSD, NetBSD, OpenBSD, etc.), Linux distributions (e.g., Red Hat, Ubuntu, Kubuntu, etc.), IBM OS/2, Microsoft Windows (XP, Vista/7/8, etc.), Apple iOS, Google Android, Blackberry OS, or the like. User interface 617 may facilitate display, execution, interaction, manipulation, or operation of program components through textual or graphical facilities. For example, user interfaces may provide computer interaction interface elements on a display system operatively connected to the computing system 601, such as cursors, icons, check boxes, menus, scrollers, windows, widgets, etc. Graphical user interfaces (GUIs) may be employed, including, without limitation, Apple Macintosh operating systems’ Aqua, IBM OS/2, Microsoft Windows (e.g., Aero, Metro, etc.), Unix X-Windows, web interface libraries (e.g., ActiveX, Java, Javascript, AJAX, HTML, Adobe Flash, etc.), or the like.

In some embodiments, the computing system 601 may implement a web browser 618 stored program component. The web browser may be a hypertext viewing application, such as Microsoft Internet Explorer, Google Chrome, Mozilla Firefox, Apple Safari, etc. Secure web browsing may be provided using HTTPS (secure hypertext transport protocol), secure sockets layer (SSL), Transport Layer Security (TLS), etc. Web browsers may utilize facilities such as AJAX, DHTML, Adobe Flash, JavaScript, Java, application programming interfaces (APIs), etc. In some embodiments, the computer system 301 may implement a mail server 619 stored program component. The mail server may be an Internet mail server such as Microsoft Exchange, or the like. The mail server may utilize facilities such as ASP, ActiveX, ANSI C++/C#, Microsoft .NET, CGI scripts, Java, JavaScript, PERL, PHP, Python, WebObjects, etc. The mail server may utilize communication protocols such as internet message access protocol (IMAP), messaging application programming interface (MAPI), Microsoft Exchange, post office protocol (POP), simple mail transfer protocol (SMTP), or the like. In some embodiments, the computing system 601 may implement a mail client 620 stored program component. The mail client may be a mail viewing application, such as Apple Mail, Microsoft Entourage, Microsoft Outlook, Mozilla Thunderbird, etc.

In some embodiments, computing system 601 may store user/application data 621, such as the data, variables, records, etc. as described in this disclosure. Such databases may be implemented as fault-tolerant, relational, scalable, secure databases such as Oracle or Sybase. Alternatively, such databases may be implemented using standardized data structures, such as an array, hash, linked list, struct, structured text file (e.g., XML), table, or as object-oriented databases (e.g., using ObjectStore, Poet, Zope, etc.). Such databases may be consolidated or distributed, sometimes among the various computer systems discussed above in this disclosure. It is to be understood that the structure and operation of the any computer or database component may be combined, consolidated, or distributed in any working combination.

As described above, the modules, amongst other things, include routines, programs, objects, components, and data structures, which perform particular tasks or implement particular abstract data types. The modules may also be implemented as, signal processor(s), state machine(s), logic circuitries, and/or any other device or component that manipulate signals based on operational instructions. Further, the modules can be implemented by one or more hardware components, by computer-readable instructions executed by a processing unit, or by a combination thereof.

The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments. Also, the words "comprising," "having," "containing," and "including," and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., are non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope and spirit of disclosed embodiments being indicated by the following claims.

ADVANTAGES OF THE PRESENT DISCLOSURE
In one embodiment, the present disclosure generates the panoramic view generation in less computation time. Further, the panoramic views thus generated are not influenced by the mechanical wear and tear factors of the pan-tilt system used for capturing the images in 360degree view.

Further, the present disclosure reduces the number of false alarms that are generated during object detection by comparing the successive panoramas based on size, aspect ratio and so on.

Reference numerals used in the present disclosure
100 System
102 Real time panoramic view generation and object detection device
104 Pan-Tilt system
106A,106B Image sensors
108 Communication channel
110 User interface
112 Display device
114 Panoramic strips
116 Live video
118 Zoomed vide
120 Revolution Per Minute (RPM)
122 Field Of View (FOV)
124 Count of Panoramic strips
126 Screen resolution
128 Video processor
202 Video decoder
204 Video digital-to-analog converter (DAC)
206 Dual port RAM (DPRAM)
208 Bank of Synchronous Burst Synchronous RAM (SBSRAM)
302 Image overlap
304 New columns
306 Recent frame
308 Previous frame
310 First 3 columns of the previous frame
312 Correlation between previous and recent frame
314 Correlation between recent and next frame
316 Next frame
400 First panorama
402 Pixels per strip
404 Object
406 Second panorama
408 Old coordinates X1, Y1 of object
410 New coordinates X2, Y2 of object

The foregoing detailed description has described only a few of the many possible implementations of the present disclosure. While considerable emphasis has been placed herein on the particular features of this present disclosure, it will be appreciated that various modifications can be made, and that many changes can be made in the preferred embodiments without departing from the principles of the present disclosure. These and other modifications in the nature of the present disclosure or the preferred embodiments will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the present disclosure and not as a limitation.