FORM 2
THE PATENTS ACT,
1970 (39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION (See section 10, rule 13)
“SYSTEM FOR GENERATING A REAL-TIME SURROUND VIEW OF A VEHICLE AND METHOD THEREOF”
MINDA CORPORATION LIMITED., an Indian company of E-5/2, Chakan Industrial Area, Phase - III, M.I.D.C, Tal khed, Pune-410501,
Maharashtra, India
The following specification particularly describes the invention and the manner in which it is to be performed.

TECHNICAL FIELD [001] The present disclosure generally relates to real-time monitoring of automotive vehicle parking systems. More particularly, but not exclusively, the present disclosure relates to generating a real-time surround view of a vehicle to detect intruders in the vehicle surroundings.
BACKGROUND [002] Vehicles of modern times embed complex electronic systems and sensors to improve user or driver safety and convenience. For example, in recent years, cameras are increasingly used for driver assistance and vehicle safety. The cameras are either used alone or in combination with other sensors (e.g., ultrasonic, radar, lidar, etc.). A fused sensor system, which merges data from multiple sensors, can be used for various applications such as traffic light and traffic sign recognition, object detection and classification, tracking of objects, parking systems, and other similar applications.
[003] Nowadays, camera systems are gaining popularity especially in parking systems for automotive applications. The camera systems use one or more cameras either alone or in combination with different sensors (ultrasonic, radar, etc.) for facilitating the driver in getting a good understanding of vehicle surrounding while parking the vehicle.
[004] Surround view monitoring is a technique that is developed to assist drivers in parking vehicles more easily by providing surround views of the vehicles e.g., side view, virtual bird’s-eye view (top view) from above the vehicle, split views, and other such complimentary views. Surround view monitoring systems help drivers to visually confirm vehicle’s position relative to lines around parking spaces and adjacent objects, allowing the drivers to maneuver the vehicles into parking spaces with more ease. Conventionally, the surround view monitoring systems typically include an arrangement inclusive of four cameras mounted on four sides of the vehicles, which create a virtual composite 360° bird’s-eye view of the vehicles and their surroundings. Such systems provide exterior view of the vehicles to aid the drivers in maneuvering the vehicles to park. The systems may further provide an alert to the drivers about any obstacle which may not be immediately visible to the drivers.
[005] Conventional parking systems consist of multiple cameras mounted on the vehicles to acquire images of surroundings to aid in parking the vehicles. Target area to be detected is

captured and a homographic matrix is created. The homographic matrix processes image information of the target area to obtain detection results. Additionally, the conventional parking systems comprise multiple ultrasonic sensors mounted on the vehicles. Such systems (having multiple cameras and multiple sensors) are not optimal because processing of data from multiple sensors and cameras is complex. Additionally, installing multiple cameras also incurs complex image processing.
[006] There is no efficient, cost-effective, and convenient solution which can assist the driver in parking the vehicle, especially at the time of dynamic insertion of an object in the vicinity of the vehicle during reverse parking. Thus, there exists a need for a technology that solves the above-mentioned problems and overcomes the disadvantages or difficulties of existing surround view monitoring systems and/or techniques associated therewith. Particularly, there exists a need for techniques that facilitate cost effective, less complex managed surround view monitoring system capable of effectively detecting dynamic object in vehicle surrounding to assist the drivers in parking the vehicles.
[007] The information disclosed in this background of the disclosure section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
SUMMARY [008] One or more shortcomings discussed above are overcome, and additional advantages are provided by the present disclosure. The present disclosure provides a solution to the above-identified problems by providing a cost effective, less complex, and highly accurate system for generating real-time surround view of a vehicle that uses a single rear facing camera mounted at the rear end of the vehicle and two pairs of proximity sensors mounted on sides of the vehicle. The proposed technique counters the usage of multiple cameras and counters the cost and complexity issues faced with existing surround view monitoring systems. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the disclosure.

[009] An objective of the present disclosure is to provide an affordable, less complex, and highly accurate system for generating real-time surround view of a vehicle to detect dynamic intruders in vehicle surrounding. Another objective of the present disclosure is to create a graphical overlaid bird eye’s view using only a single rear end camera and two pairs of proximity sensors. Yet another objective of the present disclosure is to provide a driver assistance system that may detect presence of a static or dynamic intruder in the vehicle surrounding and generate an alert when the intruder approaches the vehicle.
[0010] The above stated objects as well as other objects, features, and advantages of the present disclosure will become clear to those skilled in the art upon review of the following description, the attached drawings, and the appended claims. According to an aspect of the present disclosure, methods and apparatus/systems are provided for generating real-time and highly accurate surround view of a vehicle in a cost-effective manner.
[0011] In one non-limiting embodiment of the present disclosure, a system for generating real-time surround view of a vehicle is disclosed. The system includes at least one camera mounted on a rear outer side of the vehicle and configured to continuously capture images based on movement of the vehicle in a reverse direction, a memory configured to store the images captured by the at least one camera, a display device, at least two pairs of proximity sensors comprising a first pair of proximity sensors mounted at predefined locations on an outer right-hand side of the vehicle relative to a driver, and a second pair of proximity sensors mounted at predefined locations on an outer left-hand side of the vehicle relative to the driver.
[0012] In one non-limiting embodiment of the present disclosure, two proximity sensors of each pair are mounted such that Field of Views (FoVs) of the two proximity sensors overlap with each other. The system further comprises at least one processor communicatively coupled with the at least one camera, the memory, the display device, and the at least two pairs of proximity sensors, wherein based on movement of the vehicle in the reverse parking trajectory, the at least one processor is configured to stitch at least one real-time image captured by the at least one camera with at least one image stored in the memory to generate a stitched image depicting a surround view of the vehicle corresponding to a current location of the vehicle, display the surround view on the display device, and upon detecting at least one intruder/object within the overlapping FoVs of any pair of proximity sensors generate a real-time surround

view of the vehicle by overlaying a graphical representation of the at least one intruder/object on the surround view of the vehicle displayed on the display device.
[0013] In one non-limiting embodiment of the present disclosure, the at least one processor is further configured to generate an audio-visual alert when a distance between the at least one intruder/object and the vehicle is less than a predefined threshold distance. In one non-limiting embodiment of the present disclosure, each proximity sensor is an ultrasonic sensor, and the at least one intruder/object is a dynamic intruder/object which was not captured by the at least one camera.
[0014] In one non-limiting embodiment of the present disclosure, the two proximity sensors of each pair are mounted at an orientation of 45o with respect to an outer surface of the vehicle such that the FoVs of the two proximity sensors are orthogonal with each other.
[0015] In one non-limiting embodiment of the present disclosure, to generate a stitched image from a first input image and a second input image, the processor (202) is configured to apply a vertical convolution mask over the first and second input images to reduce a number of magnitude values in the first and second input images; compare the reduced number of magnitude values in the first image with the reduced number of magnitude values in the second image to identify a first set of key points for the first image and a second set of key points for the second image; compare the first set of key points with the second set of key points to generate a list of matched feature vectors based on a maximum threshold value and a minimum threshold value; estimate a homographic matrix using the matched feature vectors; and generate the stitched image for the first and second images by applying a wrapping transformation using the homographic matrix.
[0016] In one non-limiting embodiment of the present disclosure, a method of generating a real-time surround view of a vehicle is disclosed. The method includes mounting at least one camera on a rear outer side of the vehicle, mounting at least two pairs of proximity sensors comprising a first pair of proximity sensors mounted at predefined locations on an outer right-hand side of the vehicle relative to a driver and a second pair of proximity sensors mounted at predefined locations on an outer left-hand side of the vehicle relative to the driver. Two proximity sensors of each pair are mounted such that Field of Views (FoVs) of the two

proximity sensors overlaps with each other. The method further comprises continuously capturing images based on movement of the vehicle in a reverse direction using the at least one camera and storing the captured images in a memory. The method further comprises stitching at least one real-time image captured by the at least one camera with at least one image stored in the memory for generating a stitched image depicting a surround view of the vehicle corresponding to a current location of the vehicle, and displaying the surround view on a display device. The method further comprises upon detecting at least one intruder/object within the overlapping FoVs of any pair of proximity sensors, generating a real-time surround view of the vehicle by overlaying a graphical representation of the at least one intruder/object on the surround view of the vehicle displayed on the display device.
[0017] In one non-limiting embodiment of the present disclosure, the method includes generating an audio-visual alert when a distance between the at least one intruder and the vehicle is less than a predefined threshold distance. In one non-limiting embodiment of the present disclosure, each proximity sensor is an ultrasonic sensor, and the at least one intruder is a dynamic intruder/object which was not captured by the at least one camera. In one non-limiting embodiment of the present disclosure, the two proximity sensors of each pair are mounted at an orientation of 45o with respect to an outer surface of the vehicle such that the FoVs of the two proximity sensors are orthogonal with each other.
[0018] In one non-limiting embodiment of the present disclosure, stitching a first input image and a second input image comprises applying a vertical convolution mask over the first and second input images to reduce a number of magnitude values in the first and second input images; comparing the reduced number of magnitude values in the first image with the reduced number of magnitude values in the second image to identify a first set of key points for the first image and a second set of key points for the second image; comparing the first set of key points with the second set of key points to generate a list of matched feature vectors based on a maximum threshold value and a minimum threshold value; estimating a homographic matrix using the matched feature vectors; and generating the stitched image for the first and second images by applying a wrapping transformation using the homographic matrix.
[0019] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above,

further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.
BRIEF DESCRIPTION OF DRAWINGS [0020] Further aspects and advantages of the present disclosure will be readily understood from the following detailed description with reference to the accompanying drawings. Reference numerals have been used to refer to identical or functionally similar elements. The figures together with a detailed description below, are incorporated in and form part of the specification, and serve to further illustrate the embodiments and explain various principles and advantages, in accordance with the present disclosure wherein:
[0021] Figure 1A illustrates an exemplary environment or system 100 where the proposed techniques of generating real-time surround view of a vehicle may be implemented, in accordance with some embodiments of the present disclosure.
[0022] Figure 1B shows an exemplary illustration of the system 100 of Figure 1A depicting Field of Views (FoV) of a camera and proximity sensors, in accordance with some embodiments of the present disclosure.
[0023] Figure 2 illustrates an exemplary block diagram 200 of the system 100, in accordance with some embodiments of the present disclosure.
[0024] Figure 3 illustrates an illustration 300 of positions of the vehicle 110 at different instances during reverse parking as created by the system 100, in accordance with some embodiments of the present disclosure.
[0025] Figure 4 shows an illustration 400 of stitching of images, in accordance with some embodiments of the present disclosure.
[0026] Figure 5A shows an illustration 500-1 of the vehicle obstructed by a dynamic intruder in a reverse parking trajectory, in accordance with some embodiments of the present disclosure.
[0027] Figure 5B shows an illustration 500-2 of the vehicle parked at the final position using

the system 100 by avoiding the intruder 590, in accordance with some embodiments of the present disclosure.
[0028] Figure 6 illustrates a flow diagram representing an exemplary method 600 of generating a real-time surround view of a vehicle, in accordance with an embodiment of the present disclosure.
[0029] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and executed by a computer or processor, whether or not such computer or processor is explicitly shown.
DETAILED DESCRIPTION [0030] The terms “comprises”, “comprising”, “includes” or any other variations thereof, are intended to cover a non-exclusive inclusions, such that a setup, device that comprises a list of components that does not include only those components but may include other components not expressly listed or inherent to such setup or device. In other words, one or more elements in a system or apparatus proceeded by “comprises… a” does not, without more constraints, preclude the existence of other elements or additional elements in the system or apparatus or device. It could be noted with respect to the present disclosure that the terms like “system for generating a real-time surround view of a vehicle and method thereof”, “The system” refers to the same system which is used in the present disclosure.
[0031] In the present document, the word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or implementation of the present subject matter described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0032] While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however, that it is not intended to limit the

disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the scope of the disclosure.
[0033] The terms “user” and “driver” may be interchangeably used throughout the description. The terms “managed around view” “around view” “surround view” “surrounding view” and “bird’s eye view” may be interchangeably used throughout the description. The terms “object” “obstacle” and “intruder” may be interchangeably used throughout the description.
[0034] In the following detailed description of the embodiments of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.
[0035] Referring now to Figure 1A, which illustrates an exemplary environment or system 100 where the proposed techniques of generating a real-time surround view of a vehicle 100 may be implemented, in accordance with some embodiments of the present disclosure. The system 100 may include at least one camera 130 mounted on a rear outer side of the vehicle 110 and at least two pairs of proximity sensors 120-1L, 120-3L and 120-2R, 120-4R mounted on outer sides of the vehicle.
[0036] In one non-limiting embodiment, the camera 130 may be mounted on rear outer side of the vehicle 110 (e.g., on a tailgate, a rear number plate holder, or a rear bumper of the vehicle 110). The camera 130 starts working or turns ON as soon as the reverse gear of the vehicle 110 is engaged. The camera 130 may continuously capture images of the rear side surround of the vehicle 110 as soon as the reverse gear of the vehicle 110 is engaged or as the vehicle 110 starts moving in a reverse direction (e.g., in a reverse angular trajectory path) or changes its orientation. The camera 130 may be a charged coupled device (CCD), a complementary metal oxide semiconductor (CMOS) camera, a night vision camera, but not limited thereto. The camera 130 may comprise a fish-eye lens or another type of wide-angle lens. An exemplary field of view (FOV) 160 of the rear camera is shown in Figure 1B.

[0037] In one non-limiting embodiment, the at least two pairs of proximity sensors 120-1L, 120-3L and 120-2R, 120-4R as shown in Figure 1A may include a first pair of proximity sensors 120-2R, 120-4R and a second pair of proximity sensors 120-1L, 120-3L. In an exemplary embodiment, one proximity sensor 120-2R of the first pair of proximity sensors may be mounted at a predefined location in a front right-hand outer side of the vehicle 110 relative to a driver and the other proximity sensor 120-4R of the first pair of proximity sensor may be mounted at a predefined location in a rear right-hand outer side of the vehicle 110 relative to the driver (as shown in Figure 1A). Similarly, one proximity sensor 120-1L of the second pair of proximity sensors is mounted at a predefined location in a front left-hand outer side of the vehicle 110 relative to the driver and the other proximity sensor 120-3L in the second pair of proximity sensor is mounted on a predefined location in a rear left-hand outer side of the vehicle 110 relative to the driver (as shown in Figure 1A). The two proximity sensors 120-2R, 120-4R of the first pair of proximity sensors are mounted in the right-hand outer side of the vehicle 110 in such a way that Field of Views (FoVs) of the two proximity sensors 120-2R, 120-4R overlap with each other. Similarly, the two proximity sensors 120-1L, 120-3L in the first pair of proximity sensors are mounted in the left-hand outer side of the vehicle 110 such a way that the FoVs of the two proximity sensors 120-1L, 120-3L overlap with each other (as shown in Figure 1B).
[0038] In an exemplary embodiment, two proximity sensors of each pair are mounted at an orientation of 45o with respect to an outer surface of the vehicle 110 such that the FoVs of the two proximity sensors are orthogonal with each other and there is no crosstalk or any other interference. Specifically, the proximity sensors of the first pair 120-2R, 120-4R are mounted at an orientation of 45o with the outer right surface of the vehicle 110 and their FoVs are orthogonal (perpendicular) with each other. Similarly, the proximity sensors of the second pair 120-1L, 120-3L are mounted at an orientation of 45o with the outer left surface of the vehicle 110 and their FoVs are orthogonal (perpendicular) with each other. As illustrated in Figure 1B, the placement and the orientation of the first and second pair of sensors create intersecting areas 140, 150 respectively wide enough to capture any static or dynamic intruder in the vicinity of the vehicle 110 both in the right-hand side and the left-hand side of the vehicle 110.
[0039] In one non-limiting embodiment, the proximity sensors 120-2R, 120-1L mounted in

the front end of the vehicle 110 are fitted on the right side and the left side of a front bumper respectively or on Outside Rear View Mirrors (ORVM), but not limited thereto. In another non-limiting embodiment, the proximity sensors 120-4R, 120-3L mounted in the rear end of the vehicle 110 are fitted on the right side and the left side of a rear bumper respectively. In a non-limiting embodiment, the proximity sensors may include ultrasonic sensors or electromagnetic sensors, but not limited thereto. In general, an ultrasonic sensor may accurately measure distance of an object/obstacle from the vehicle 110 using ultrasonic sound waves. The ultrasonic sensor requires a transmitter and a receiver. The transmitter sends an ultrasonic pulse which travels through the air and if there is an obstacle or object, the pulse bounces back to the sensor and is received by the receiver. By calculating the travel time, the distance of the obstacle from the vehicle can be calculated. In an embodiment, the ultrasonic sensor may comprise transceivers where the transmitter and receiver functions are integrated into a single unit.
[0040] In the exemplary environment presented in Figure 1B, the vehicle is shown as a four-wheeler vehicle. However, it must be appreciated that the techniques of the present disclosure are equally applicable for any vehicle including two wheeler vehicles, three-wheeler vehicles, or multi-wheeler vehicles. In the present disclosure, the proximity sensors 120-1L, 120-3L, 120-2R, 120-4R may be collectively referred as proximity sensors 120.
[0041] Referring now to Figure 2 that shows an exemplary block diagram 200 of a monitoring system (which may be same as the system 100) for generating a real-time surround view of a vehicle 100 as illustrated in Figure 1A. As shown in Figure 2, the system 100 may include a vehicle control sub-system 220 that may include at least one processor 202, a memory 204, a transceiver 206, and an interface 208. The processor 202 may be communicatively coupled to the memory 204, the interface 208, and to the transceiver 206. The memory 204 may store the images (static images) captured by the at least one camera 130. In an exemplary embodiment the memory 204 may store consecutive dynamic images captured by the camera 130 for a stipulated duration of time. The accumulation of dynamic images takes a static form once they get stored inside the memory 204. The processor 202 (more generally, the vehicle control sub-system 220) may receive data from the camera 130 and the proximity sensors 120 for generating the surround view of the vehicle surrounding.
[0042] In one non-limiting embodiment, the memory 204 may comprise various datasets and

commands related to operation of the vehicle 110. The memory 204 may also store necessary commands needed for execution of various operations of the system 100 for generating the real¬time surround view of the vehicle 110. The memory 204 may include a Random-Access Memory (RAM) unit and/or a non-volatile memory unit such as a Read Only Memory (ROM), optical disc drive, magnetic disc drive, flash memory, Electrically Erasable Read Only Memory (EEPROM), a memory space on a server or cloud and so forth. For the sake of illustration, it is assumed here that the memory is a non-volatile memory. The processor 202 may process or perform various operations of the system 100 for generating real-time surround view of the vehicle 110. Examples of the processor may include, but not restricted to, a general-purpose processor, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), microprocessors, microcomputers, micro¬controllers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions.
[0043] In one non-limiting embodiment, the system 100 may also include at least one display device 212 and other units 214. The display device 212 and the other units 214 may be communicatively coupled to the processor 202. The display device 212 may be a Light Emitting Diode (LED) display, a liquid-crystal display (LCD), or a touch screen display. The display device 212 may be used for displaying the real-time surround view of the vehicle 110 and for other purposes as well such as, but not limited to, displaying alert/warning messages, audio/video player, navigation etc. The other units 214 may include a speaker or audio alert unit, haptic alert unit, wheel speed sensors, not limiting thereto. The interfaces 208 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, an input device-output device (I/O) interface, a network interface and the like. The I/O interfaces may allow the vehicle 110 to interact with other computing systems/devices directly or through other devices. The network interface may allow the vehicle 110 to interact with one or more devices either directly or via a network.
[0044] Referring now to Figure 3 that shows an illustration 300 of the positions of the vehicle 110 at different instances during reverse parking as created by the system 100 for generating the real-time surround view of the vehicle 110, in accordance with some embodiments of the present disclosure. In a non-limiting embodiment, the system 100 for generating real-time surround view of the vehicle 110 activates the camera 130 when the vehicle 110 is put in reverse

gear and starts moving in the reverse angular trajectory path. The camera 130 starts capturing dynamic images of the vehicle surround and stores the captured images in the memory 204, which gets converted into static form for a stipulated amount of time. Image A, image B and image C as illustrated in Figure 3 shows images captured by the camera 130 at different instances along the vehicle angular trajectory path, while doing reverse parking of the vehicle 110. In an exemplary embodiment when the vehicle 110 starts moving in the reverse direction, the camera 130 gets activated and captures an image A of vehicle surround and stores it in the memory 204. The camera 130 continuously captures images B and C as the vehicle moves in reverse parking trajectory or changes its orientation. The images A-C are stored in the memory 204. Since the viewing direction of camera is different from images A to C, combining the images A to C captured may give a wide-angle view of the vehicle surrounding, where the C image depicts the live/latest dynamic image.
[0045] In a non-limiting embodiment, the at least one processor 202 may process the images stored in the memory 204 within a stipulated time to create a combined or stitched bird’s eye view image. The combined image may be relayed over a human machine interface (HMI) for displaying on the display device 212. The stitched image is a temporarily formed image which may be replaced by another stitched image after the passage of a stipulated amount of time. In an embodiment, the processor 204 keeps on changing the stitched images on the display device 212 such that a time difference between two consecutive stitched images is small enough to be viewed by a human eye, so as to give appearance of a video. The images are stored and retrieved at an interval so as to fit into a frame, where a frame is set in motion at a rate of 30 frames/second.
[0046] The pairs of proximity sensors 120-1L and 120-3L on the left hand side and 120-2R and 120-4R on the right hand side form the angular FoVs 340-1, 350-1, 340-2, 350-2, 340-3, 350-3 as depicted which are intersecting at the terminal point, this forming a conical view as shown in the Figure 3. These conical views which are the FoVs 340-1, 350-1, 340-2, 350-2, 340-3, 350-3 of the proximity sensors are integrated or graphically overlayed on visual views of the rear camera so as to generate an enhanced surround view. This enhanced surround view helps to capture and detect the unidentified dynamic objects/intruders accurately nearby the vehicle trajectory while parking. The object/intruder’s accurate position is depicted/graphically

on the display device 212 overlaid on the stitched view created with the help rear viewing camera 130 and historic images with a combination of an audio-visual alert.
[0047] Referring now to Figure 4 that shows an illustration 400 of stitching of images, the images to be stitched may be static images or dynamic images or a combination of both. Consider that as the vehicle moves in reverse direction, the camera captures images 472, 474, and 476 at instances A, B, and C respectively, where images 472 and 474 are separated by a stitching plane 482 and images 474 and 476 are separated by a stitching plane 484. For generating a stitched image from two input images, the processor 202 implements an image stitching technique. In one non-limiting embodiment, as a first step the image stitching technique includes feature detection where key points are detected, and local invariant descriptors are extracted from the two input images. In a non-limiting embodiment, the two input images include a static image 474 and a dynamic image 476. The two images 474, 476 are stitched along the stitching plane 484. In a non-limiting embodiment, to stitch the two images 474, 476, a vertical convolution mask is applied to reduce the number of magnitude values in the two input images 474, 476. In an exemplary embodiment the vertical convolution mask is a 9X1 mask. The reduced number of magnitude values in the the two input images 474, 476 are compared to identify a first set of key points in the static image 474 and a second set of key points in the dynamic image 476. The key points that are less similar are chosen to be the key points of an image. In various embodiments, the first set of key points and the second set of key points are compared to compute a list of matched feature vectors which include a maximum threshold and a minimum threshold. The threshold is computed as the summation of mean of absolute difference (histogram difference between two consecutive pixel intensities) and standard deviation of the absolute difference. The threshold is computed as:
t= σadh + µadh,……(1)
where µadh= mean absolute difference and, σadh= standard deviation of absolute difference. Each pixel value of the image representing intensity value is compared to the specified threshold. This divides all the pixels of the input images into two groups namely pixels having intensity value lower than threshold and pixels having intensity value greater than threshold. In a non-limiting embodiment, pixels that have intensities lower than the threshold are discarded to saves time and reduce computational complexity.

[0048] The image stitching technique further includes estimating a homographic matrix using the matched feature vectors. In various embodiments, the homographic matrix allows the images to be shifted from one view to another view of the same scene by multiplying the homographic matrix with key points in one view to find their corresponding locations in another view. In an exemplary embodiment, Random sample consensus (RANSAC) technique is used to estimate the homographic matrix from the matched feature vectors. Further, the image stitching technique includes applying a wrapping transformation using the homographic matrix to generate a stitched image depicting a surround view of the vehicle 110 corresponding to a current location of the vehicle 110. In a non-limiting embodiment, the surround view of the vehicle 110 corresponding to the current location of the vehicle 110 is displayed on the display device 212.
[0049] Since the stitched image is formed by using images stored in the memory 204 and/or using dynamic real-time images captured by the camera 130, it may be possible to capture true real time surrounding of the vehicle 110 by means of sensor fusion of the four ultrasonic sensors overlapped FoVs with the static/dynamic images captured by the rear camera 130. For example, if an object appears near adjacent side of the vehicle 110 in the reverse parking turning radius path, the image corresponding to the adjacent sides of the vehicle has already been captured historically without any object/dynamic intruder by the rear-view camera 130 and a stitched (Static + Dynamic images) image shall be generated by the processor 202 and displayed on the display 212, this is integrated with the graphically overlaid views generated with ultrasonic sensors to detect any unidentified dynamic objects/intruders accurately adjacent/nearby the reverse parking curvature path.
[0050] Referring now to Figure 5A that shows an illustration 500-1 of the vehicle obstructed by a dynamic intruder 590 in a reverse parking trajectory, where the dynamic intruder 590 was not captured by the camera 130 earlier. The proximity sensors detect the presence of the dynamic intruder 590 when the intruder falls within their overlapping FOVs. When the proximity sensors detect the intruder within the overlapping FoVs 140 or 150 of the pair of proximity sensors 120-2R, 120-4R, 120-1L, 120-3L (in Figure 5A, the intruder 590 may be detected within the overlapping FOV 140 of the first pair of proximity sensors 120-2R, 120-4R), the processor 202 receives the distance information of the object/intruder from the surface of the vehicle 110 and creates a bird’s eye 2D metrics for the accurate identification of the

intruder. A distance calibrated grid is formed out of the intersecting region of FoV. The processor 202 determines the (x, y) co-ordinates of the object/intruder as well as perpendicular distance of the object/intruder from the surface of the vehicle 110 within the grid. In various embodiments, the processor 202 generates a real-time surround view of the vehicle 110 by overlaying the graphical representation of the object/intruder on the surround view or instant stitched image of the vehicle 110 displayed on the display device 212.
[0051] In one non-limiting embodiment, at least one colored indication (red, yellow, or green coloured indication) may be displayed on the display device 212 depending on the distance between the object/intruder 590 and the vehicle 110. In one embodiment, red coloured indication may indicate that the intruder is very close to the vehicle 110 and there are high chances that the vehicle 110 will collide with the intruder while moving in reverse direction, green coloured indication indicates that the intruder is significantly away from the vehicle 110 and chances of collision between the intruder and the vehicle 110 are almost zero, yellow coloured indication indicates that the intruder is close to the vehicle 110 and there may be some chances of collision between the intruder and the vehicle so the driver should carefully moving the vehicle 110 in reverse direction. Figure 5B shows an illustration 500-2 of the vehicle 110 parked at the final position using the system 100 by avoiding the intruder 590 with the help of generated real-time surround view of the vehicle 110.
[0052] It may be noted that the alerts generated by the managed surround view monitoring system 100 may be in the form of audio alerts, visual alerts, audio-visual alerts, haptic alerts, message displayed on the display 212, warning chime, or any combination thereof. This way the present disclosure provides a cost optimal and resource efficient solution for detecting presence of objects (specifically dynamic objects) while parking the vehicle 110.
[0053] Referring now to Figure 6 that shows a flow diagram depicting a method 600 to generate a real-time surround view of the vehicle 110. The various operations of the method 600 may be performed by the system 100 for generating a real-time surround view of the vehicle 110 and in particular, by the processor 202 in conjunction with various components of the system 100.
[0054] The method 600 may comprise, at block 602, mounting at least one camera 130 on a

rear outer side of the vehicle 110 such that the camera 130 is configured to continuously capture images based on movement of the vehicle 110 in a reverse direction.
[0055] The method 600 may comprise, at block 604, mounting at least two pairs of proximity sensors 120 comprising a first pair of proximity sensors 120-2R, 120-4R mounted at predefined locations on an outer right-hand side of the vehicle 110 relative to a driver and a second pair of proximity sensors 120-1L, 120-3L mounted at predefined locations on an outer left-hand side of the vehicle 110 relative to the driver. In an exemplary embodiment two proximity sensors of each pair are mounted such that the FoVs of the two proximity sensors overlap with each other. In a non- limiting embodiment, the two sensors of each pair (120-2R, 120-4R) or (120-1L, 120-3L) are placed such that they make an orientation of 45o with the surface of the vehicle 110 and their FoVs are orthogonal (perpendicular) with each other. The two sensors of the first and second pairs (120-2R, 120-4R) and (120-1L, 120-3L) create intersecting areas 140, 150 respectively wide enough to capture any intruder in the intersecting areas 140, 150 i.e., in the vicinity of the vehicle 110 both in the right-hand side and the left-hand side of the vehicle 110.
[0056] The method 600 may comprise, at block 606, continuously capturing images based on movement of the vehicle 110 in a reverse direction using the at least one camera 130. For example, when the gear of the vehicle 110 is engaged in reverse direction, the processor 202 may receive a signal indicating the application of the reverse gear and may activate the camera 130 mounted at the rear of the vehicle 110 for continuously capturing the images in during reverse movement of the vehicle 110.
[0057] The method 600 may comprise, at block 608, storing the captured images in a memory 204. For example, the processor 202 may store the received images in the memory 204 for a stipulated duration of time or for a stipulated distance movement of the vehicle. The accumulation of the images takes a static form once they get stored inside the memory 204.
[0058] The method 600 may comprise, at block 610, stitching at least one real-time image captured by the camera 130 with at least one image stored in the memory 204 for generating a stitched image depicting a surround view of the vehicle 110 corresponding to a current location of the vehicle 110. In one non-limiting embodiment, stitching a first input image and a second input image may comprises applying a vertical convolution mask over the first and second input

images to reduce a number of magnitude values in the first and second input images; comparing the reduced number of magnitude values in the first image with the reduced number of magnitude values in the second image to identify a first set of key points for the first image and a second set of key points for the second image; comparing the first set of key points with the second set of key points to generate a list of matched feature vectors based on a maximum threshold value and a minimum threshold value; estimating a homographic matrix using the matched feature vectors; and generating the stitched image for the first and second images by applying a wrapping transformation using the homographic matrix.
[0059] The method 600 may comprise, at block 612, displaying the surround view on a display device 212. For example, the processor 202 may project the stitched image on the display 212. Further, the processor 204 keeps on changing the stitched images on the display 212 such that a time difference between two consecutive stitched images is small enough to be viewed by a human eye, so as to give appearance of a video.
[0060] The method 600 may comprise, at block 614, upon detecting at least one intruder within the overlapping FoVs of any pair of proximity sensors, generating a real-time surround view of the vehicle 110 by overlaying a graphical representation of the at least one intruder on the surround view of the vehicle 110 displayed on the display device 212.
[0061] In one non-limiting embodiment, the method 600 may comprise generating an alert when a distance between the at least one intruder 590 and the vehicle 110 is less than a predefined threshold distance. In one embodiment, each proximity sensor is an ultrasonic sensor and the at least one intruder 590 is a dynamic intruder which was not captured by the at least one camera 130.
[0062] The method 600 is merely provided for exemplary purposes, and embodiments are intended to include or otherwise cover any methods or procedures for generating surround view of a vehicle surrounding using the techniques described in the present disclosure. The above method 600 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform specific functions or implement specific abstract data types.

[0063] The various blocks of the method 600 shown in Figure 6 have been arranged in a generally sequential manner for ease of explanation. However, it is to be understood that this arrangement is merely exemplary, and it should be recognized that the processing associated with method 600 (and the blocks shown in Figure 6) can occur in a different order (for example, where at least some of the processing associated with the blocks is performed in parallel and/or in an event-driven manner). Additionally, individual blocks may be deleted from the methods without departing from the scope of the subject matter described herein. Furthermore, the methods can be implemented in any suitable hardware, software, firmware, or combination thereof.
[0064] The various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and/or software component(s) and/or module(s). Generally, where there are operations illustrated in Figures, those operations may have corresponding counterpart means-plus-function components. It may be noted here that the subject matter of some or all embodiments described with reference to Figures 1-5 may be relevant for the methods and the same is not repeated for the sake of brevity. In a non-limiting embodiment of the present disclosure, one or more non-transitory computer-readable media may be utilized for implementing the embodiments consistent with the present disclosure.
[0065] As used herein, a phrase referring to “at least one” or “one or more” of a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c. The terms “a”, “an” and “the” mean “one or more”, unless expressly specified otherwise.
[0066] Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the embodiments of the present invention are intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the appended claims.

WE CLAIM:
1. A system (100) for generating real-time surround view of a vehicle (110), comprising:
at least one camera (130) mounted on a rear outer side of the vehicle (110) and configured to continuously capture images based on movement of the vehicle (110) in a reverse direction;
a memory (204) configured to store the images captured by the at least one camera (130);
a display device (212);
at least two pairs of proximity sensors comprising:
a first pair of proximity sensors (120-2R, 120-4R) mounted at predefined
locations on an outer right-hand side of the vehicle (110) relative to a driver, and a second pair of proximity sensors (120-1L, 120-3L) mounted at predefined
locations on an outer left-hand side of the vehicle (110) relative to the driver,
wherein two proximity sensors of each pair (120-2R, 120-4R), (120-1L, 120-3L) are mounted such that Field of Views (FoVs) of the two proximity sensors
overlap with each other; and
at least one processor (202) communicatively coupled with the at least one camera (130), the memory (204), the display device (212), and the at least two pairs of proximity sensors (120-2R, 120-4R), (120-1L, 120-3L), wherein based on movement of the vehicle (110) in the reverse direction, the at least one processor (202) is configured to:
stitch at least one real-time image captured by the at least one camera (130)
with at least one image stored in the memory (204) to generate a stitched image
depicting a surround view of the vehicle (110) corresponding to a current location
of the vehicle (110);
display the surround view on the display device (212); and
upon detecting at least one intruder (590) within the overlapping FoVs (140,
150) of any pair of proximity sensors, generate a real-time surround view of the
vehicle (110) by overlaying a graphical representation of the at least one intruder
(590) on the surround view of the vehicle (110) displayed on the display device
(212).

2. The system (100) as claimed in claim 1, wherein the at least one processor (202) is further configured to generate an alert when a distance between the at least one intruder (590) and the vehicle (110) is less than a predefined threshold distance.
3. The system (100) as claimed in claim 1, wherein each proximity sensor is an ultrasonic sensor, and wherein the at least one intruder (590) is a dynamic intruder which was not captured by the at least one camera (130).
4. The system (100) as claimed in claim 1, wherein the two proximity sensors of each pair (120-1L, 120-3L), (120-2R, 120-4R) are mounted at an orientation of 45o with respect to an outer surface of the vehicle (110) such that the FoVs of the two proximity sensors are orthogonal with each other.
5. The system (100) as claimed in claim 1, wherein to generate a stitched image from a first input image and a second input image, the processor (202) is configured to:
apply a vertical convolution mask over the first and second input images to reduce a number of magnitude values in the first and second input images;
compare the reduced number of magnitude values in the first image with the reduced number of magnitude values in the second image to identify a first set of key points for the first image and a second set of key points for the second image;
compare the first set of key points with the second set of key points to generate a list of matched feature vectors based on a maximum threshold value and a minimum threshold value;
estimate a homographic matrix using the matched feature vectors; and
generate the stitched image for the first and second images by applying a wrapping transformation using the homographic matrix.
6. A method (600) of generating a real-time surround view of a vehicle (110), the
method (600) comprising:
mounting (602) at least one camera (130) on a rear outer side of the vehicle (110);
mounting (604) at least two pairs of proximity sensors comprising a first pair of proximity sensors (120-2R, 120-4R) mounted at predefined locations on an outer right-hand side of the vehicle (110) relative to a driver and a second pair of proximity sensors

(120-1L, 120-3L) mounted at predefined locations on an outer left-hand side of the vehicle (110) relative to the driver, and wherein two proximity sensors of each pair (120-2R, 120-4R), (120-1L, 120-3L) are mounted such that Field of Views (FoVs) of the two proximity sensors overlaps with each other;
continuously capturing (606) images based on movement of the vehicle (110) in a reverse direction using the at least one camera (130);
storing (608) the captured images in a memory (204);
stitching (610) at least one real-time image captured by the at least one camera (130) with at least one image stored in the memory (204) for generating a stitched image depicting a surround view of the vehicle corresponding to a current location of the vehicle (110);
displaying (612) the surround view on a display device (212); and
upon detecting at least one intruder (590) within the overlapping FoVs (140, 150) of any pair of proximity sensors, generating (614) a real-time surround view of the vehicle (110) by overlaying a graphical representation of the at least one intruder (590) on the surround view of the vehicle (110) displayed on the display device (121).
7. The method (600) as claimed in claim 6, further comprising:
generating an audio-visual alert when a distance between the at least one intruder (590) and the vehicle (110) is less than a predefined threshold distance.
8. The method (600) as claimed in claim 6, wherein each proximity sensor is an ultrasonic sensor, and wherein the at least one intruder (590) is a dynamic intruder which was not captured by the at least one camera (130).
9. The method (600) as claimed in claim 6, wherein the two proximity sensors of each pair (120-1L, 120-3L), (120-2R, 120-4R) are mounted at an orientation of 45o with respect to an outer surface of the vehicle (110) such that the FoVs of the two proximity sensors are orthogonal with each other.
10. The method (600) as claimed in claim 6, wherein stitching a first input image and a second input image comprises:
applying a vertical convolution mask over the first and second input images to

reduce a number of magnitude values in the first and second input images;
comparing the reduced number of magnitude values in the first image with the reduced number of magnitude values in the second image to identify a first set of key points for the first image and a second set of key points for the second image;
comparing the first set of key points with the second set of key points to generate a list of matched feature vectors based on a maximum threshold value and a minimum threshold value;
estimating a homographic matrix using the matched feature vectors; and
generating the stitched image for the first and second images by applying a wrapping transformation using the homographic matrix.