Abstract: A pedestrian detection system implemented in a vehicle is disclosed. The system comprises an initialization module, which receives and defines a Region of Interest (ROI) for a set of image frames pertaining to field of view of a vehicle driver, wherein the ROI is defined based on resolution of said each image frame and a region defined in the field of view; a scanning window selection module, which determines a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame; a feature extraction module, which extracts Integral of oriented gradients (IHOG) features from each scanning window; and a pedestrian detection module, which detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
DESC:
FIELD OF DISCLOSURE
[0001] The present disclosure relates to the field of image processing. More particularly, the present disclosure relates to a system and method for pedestrian detection and collision warning.
BACKGROUND OF THE DISCLOSURE
[0002] The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
[0003] Pedestrians are vulnerable participants in a transportation system when accidents happen, especially in urban areas. The first global road safety assessment of World Health Organization (WHO) report shows that traffic accidents are one of the major causes of deaths and injuries around the world. 41% to 75% of road traffic fatal accidents are involving pedestrians, and possibility of pedestrians suffering death due to road accidents is four times compared with that of vehicle occupants. These fatalities mainly occur due to mistake of the pedestrian or the driver. It is important to prevent such accidents and ensure safety of people. With increase in the number of pedestrian fatalities on roads, significance and importance of pedestrian detection solutions is increasing. Pedestrian detection solutions play a vital role in avoiding these accidents by alerting the driver well in advance. Advanced driver assistance systems (ADAS) in automobiles implement pedestrian detection solutions to prevent accidents on roads.
[0004] Detecting pedestrian(s) in an image is a challenging task in the field of object detection. Sensors play an important role when it comes to detecting obstacles in surroundings. Commonly used sensors for the vehicles are LIDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), ultrasound and camera. As compared to other sensors, vision-based systems are gaining significant importance due to their lower cost and advantages as compared to other sensors.
[0005] Different techniques of pedestrian detection are currently being implemented in vehicles. One such technique is motion based detection. In this method, motion is detected from successive frames captured by a camera. Presence of a pedestrian is confirmed on basis of prominent motion detection. However, motion based detection has certain limitations. In actual practice, motion based detection triggers multiple false signals due to capturing non-pedestrian objects too. Further, changes in illumination also affect overall detection. Hence, motion based detection is more suitable in surveillance field where mostly the background is stable.
[0006] In some other existing methods of pedestrian detection, an entire image is scanned at various scales, thereby making the process extremely slow. Saliency based method uses 2D features such as gradient, color, intensity, edge etc., to extract object segments. Since the method is highly dependent on the selected features, human detection is not much efficient. Stereo-based foreground segmentation is one way to eliminate background.
[0007] For most of the existing techniques, one of the major assumptions is that the pedestrians possess a vertical structure at a specific depth.Some of the existing techniques are: v-disparity representation to find vertical and horizontal planes to extract candidate ROIs (regions of interest), stereo-based plane fitting to find different planes, disparity map analysis with Pedestrian Size Constraint (PSC) to extract better ROIs, and multimodal stereo methods that make use of different spectrums like visual and thermal infrared, etc.
[0008] In other methods of pedestrian detection, vision based detection technique is used. Generally, there are two main approaches for vision based pedestrian detection, the whole approach and the part based approach. In whole body detection, a pedestrian is detected as a whole object; whereas in part based approach the detection process is concentrated on parts like head, torso arms, legs, etc. The general process for detection constitutes of pre-processing, foreground segmentation, object classification and tracking. Pre- processing includes exposure correction, dynamic ranging, noise removal, etc. to provide a better input/image for further processing. Foreground segmentation extracts possible candidate ROI by eliminating background and sky regions. This restricts the search to ROI thereby, reducing processing time and false positives.
[0009] The major challenge is the development of reliable on-board pedestrian detection systems as pedestrians have different postures, clothing and shapes. Another challenge to be addressed while detecting pedestrians is the constantly changing illumination, especially during day time. This directly affects the image quality. If the image quality is poor, then information cannot be retrieved. Most of the image processing algorithms are invariant(do not change according to) to illumination changes. Time complexity is more for the stereo-based algorithms and the detection rate is very less in non-textured regions. Additionally, pedestrian detection during day time is difficult, requires complex systems and has less accuracy.
[00010] Thus, there is need for a day time pedestrian detection system which is robust, accurate, fast, efficient and simple. Also, there is a need for a pedestrian detection system which can detect pedestrians wearing any kind of attire. Additionally, there is a need for a pedestrian detection system which can handle the constantly changing day time illumination and is able to detect pedestrians accurately.
OBJECTS OF THE INVENTION
[00011] Some of the objects of the present disclosure, which at least one embodiment herein satisfies are as listed herein below.
[00012] It is an object of the present disclosure to provide a system and method for pedestrian detection that accurately detects a pedestrian and providesa warning accordingly.
[00013] Yet another object of the present disclosure is to provide a system and method for pedestrian detection that detects pedestrians irrespective of their attire.
[00014] Yet another object of the present disclosure is to provide a system and method that detects a pedestrian in varied illuminations of day time.
[00015] Yet another object of the present disclosure is to provide a system and method that detects a pedestrian in all orientations.
[00016] Still another object of the present disclosure is to provide a robust, economic and simple system and method that accurately detects a pedestrian.
SUMMARY
[00017] This summary is provided to introduce simplified concepts of a system and method for pedestrian detection, which are further described below in the Detailed Description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended for use in determining/limiting the scope of the claimed subject matter.
[00018] The present disclosure relates to the field of image processing. More particularly, the present disclosure relates to a system and method for pedestrian detection and collision warning.
[00019] In an aspect, present disclosure elaborates upon a pedestrian detection system implemented in a vehicle, the system comprising: a non-transitory storage device having embodied therein one or more routines operable to detect a pedestrian; and one or more processors coupled to the non-transitory storage device and operable to execute the one or more routines, wherein the one or more routines include: an initialization module, which when executed by the one or more processors, receives one or more image frames and defines a Region of Interest (ROI) for a set of image frames selected from the received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of the each image frame and further based on a region defined in the field of view of the vehicle driver; a scanning window selection module, which when executed by the one or more processors, determines a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame; a feature extraction module, which when executed by the one or more processors, extracts Integral Histogram of oriented gradients (IHOG) features from each scanning window; and a pedestrian detection module, which when executed by the one or more processors, detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
[00020] In an embodiment, the region in the field of view of the vehicle driver may be any of a far region, a middle region or a near region and wherein the initialization module defines the ROI from High Definition (HD) resolution image for the far region, Video Graphics Array (VGA) resolution image for the middle region, and Quarter VGA (QVGA) resolution image for the near region.
[00021] In an embodiment, the feature extraction module extracts IHOG features of each scanning window comprising a plurality of cells such that size of each of the plurality of cells for each scanning window is adjusted to make feature vector length same for each of the plurality of scanning windows.
[00022] In another embodiment, the pedestrian detection module is coupled with a Non-maximal suppression (NMS) module that is configured to provide an output of a single bounding box around the detected pedestrian.
[00023] In yet another embodiment, detection of the pedestrian is independent of speed of the vehicle and distance between the pedestrian and the vehicle.
[00024] In an embodiment, the pedestrian detection module generates an alert on detection of the pedestrian.
[00025] In another embodiment, the system further comprises a tracking module to track the detected bounding box of the pedestrian based on extracted IHOG features of the set of image frames.
[00026] In yet another embodiment, the system further comprises a collision determination module to determine expected time for collision between the vehicle and the pedestrian based on detection of the pedestrian.
[00027] In an embodiment, the scanning window selection module is configured to toggle scanning of the ROI of an image frame of the set of image frames.
[00028] In another embodiment, at least one classifier of the two or more classifiers facilitate detection of the pedestrian using a training image database, the training image database being created based on: cropping at least one training image frame of the one or more image frames using silhouette information of the pedestrian; resizing each of the cropped images to size of a nearest scanning window; removing undesirable structures from a classifier database of the at least one classifier; collecting false positives from the at least one classifier; and adding the false positives to the classifier database of a succeeding classifier.
[00029] In an aspect, present disclosure elaborates upon a method for pedestrian detection comprising: receiving, by one or more processors, one or more image frames and defining a Region of Interest (ROI) for a set of image frames selected from the received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of the each image frame and further based on a region defined in the field of view of the vehicle driver; determining, by the one or more processors, a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame; extracting, by the one or more processors, Integral Histogram of oriented gradients (IHOG) features from each scanning window; and detecting, by the one or more processors, the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
[00030] Various objects, features, aspects and advantages of the present disclosure will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like features.
[00031] Within the scope of this application it is expressly envisaged that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. Features described in connection with one embodiment are applicable to all embodiments, unless such features are incompatible.
BRIEF DESCRIPTION OF DRAWINGS
[00032] The accompanying drawings are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. The diagrams are for illustration only, which thus is not a limitation of the present disclosure, and wherein:
[00033] FIG. 1 illustrates architecture of a pedestrian detection system to illustrate its overall working in accordance with an embodiment of the present disclosure.
[00034] FIG.2A illustrates exemplary functional modules of a pedestrian detection system in accordance with an embodiment of the present disclosure.
[00035] FIG.2B illustrates exemplary ROI for far, middle and near region in accordance with an exemplary embodiment of the present disclosure.
[00036] FIG.2C illustrates an exemplary block diagram of initialization module in accordance with an embodiment of the present disclosure.
[00037] FIG.2D illustrates an exemplary block diagram for implementation of feature extraction in accordance with an embodiment of the present disclosure.
[00038] FIG.2E illustrates an exemplary block diagram of classifier architecture utilized for pedestrian detection in accordance with an embodiment of the present disclosure.
[00039] FIG.2F illustrates an exemplary output of SVM classifier in accordance with an embodiment of the present disclosure.
[00040] FIG.2G illustrates an exemplary output of non-maximal suppression (NMS) module in accordance with an embodiment of the present disclosure.
[00041] FIG.2H illustrates an exemplary block diagram for implementation of tracking module in accordance with an embodiment of the present disclosure.
[00042] FIG.2I illustrates an exemplary block diagram for distance to collision calculation for determining time to collision in accordance with an embodiment of the present disclosure.
[00043] FIG.2J illustrates pedestrian collision warning sample frame output of different stages in accordance with an exemplary embodiment of the present disclosure.
[00044] FIG. 3 illustrates a method of working of the proposed system in accordance with an embodiment of the present disclosure.
[00045] FIG. 4 illustrates overall working of the proposed system in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION
[00046] The following is a detailed description of embodiments of the disclosure depicted in the accompanying drawings. The embodiments are in such detail as to clearly communicate the disclosure. However, the amount of detail offered is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.
[00047] In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details.
[00048] Embodiments of the present invention include various steps, which will be described below. The steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor programmed with the instructions to perform the steps. Alternatively, steps may be performed by a combination of hardware, software, and firmware and/or by human operators.
[00049] Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the present invention may involve one or more computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the invention could be accomplished by modules, routines, subroutines, or subparts of a computer program product.
[00050] If the specification states a component or feature “may”, “can”, “could”, or “might” be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
[00051] As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
[00052] Exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. These exemplary embodiments are provided only for illustrative purposes and so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those of ordinary skill in the art. The invention disclosed may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Various modifications will be readily apparent to persons skilled in the art. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Moreover, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Also, the terminology and phraseology used is for the purpose of describing exemplary embodiments and should not be considered limiting. Thus, the present invention is to be accorded the widest scope encompassing numerous alternatives, modifications and equivalents consistent with the principles and features disclosed. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail so as not to unnecessarily obscure the present invention.
[00053] Thus, for example, it will be appreciated by those of ordinary skill in the art that the diagrams, schematics, illustrations, and the like represent conceptual views or processes illustrating systems and methods embodying this invention. The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing associated software. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the entity implementing this invention. Those of ordinary skill in the art further understand that the exemplary hardware, software, processes, methods, and/or operating systems described herein are for illustrative purposes and, thus, are not intended to be limited to any particular named element.
[00054] Embodiments of the present invention may be provided as a computer program product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. The term “machine-readable storage medium” or “computer-readable storage medium” includes, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).A machine-readable medium may include a non-transitory medium in which data may be stored and that does not include carrier waves and/or transitory electronic signals propagating wirelessly or over wired connections. Examples of a non-transitory medium may include, but are not limited to, a magnetic disk or tape, optical storage media such as compact disk (CD) or digital versatile disk (DVD), flash memory, memory or memory devices. A computer-program product may include code and/or machine-executable instructions that may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
[00055] Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a machine-readable medium. A processor(s) may perform the necessary tasks.
[00056] Systems depicted in some of the figures may be provided in various configurations. In some embodiments, the systems may be configured as a distributed system where one or more components of the system are distributed across one or more networks in a cloud computing system.
[00057] Each of the appended claims defines a separate invention, which for infringement purposes is recognized as including equivalents to the various elements or limitations specified in the claims. Depending on the context, all references below to the "invention" may in some cases refer to certain specific embodiments only. In other cases it will be recognized that references to the "invention" will refer to subject matter recited in one or more, but not necessarily all, of the claims.
[00058] All methods described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.
[00059] Various terms as used herein are shown below. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.
[00060] The present disclosure relates to the field of image processing. More particularly, the present disclosure relates to a system and method for pedestrian detection and collision warning.
[00061] In an aspect, present disclosure elaborates upon a pedestrian detection system implemented in a vehicle, the system comprising: a non-transitory storage device having embodied therein one or more routines operable to detect a pedestrian; and one or more processors coupled to the non-transitory storage device and operable to execute the one or more routines, wherein the one or more routines include: an initialization module, which when executed by the one or more processors, receives one or more image frames and defines a Region of Interest (ROI) for a set of image frames selected from the received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of the each image frame and further based on a region defined in the field of view of the vehicle driver; a scanning window selection module, which when executed by the one or more processors, determines a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame; a feature extraction module, which when executed by the one or more processors, extracts Integral Histogram of oriented gradients (IHOG) features from each scanning window; and a pedestrian detection module, which when executed by the one or more processors, detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
[00062] In an embodiment, the region in the field of view of the vehicle driver may be any of a far region, a middle region or a near region and wherein the initialization module defines the ROI from High Definition (HD) resolution image for the far region, Video Graphics Array (VGA) resolution image for the middle region, and Quarter VGA (QVGA) resolution image for the near region.
[00063] In an embodiment, the feature extraction module extracts IHOG features ofeach scanning window comprising a plurality of cells such that size of each of the plurality of cells for each scanning window is adjusted to make feature vector length same for each of the plurality of scanning windows.
[00064] In another embodiment, the pedestrian detection module is coupled with a Non-maximal suppression (NMS) module that is configured to provide an output of a single bounding box around the detected pedestrian.
[00065] In yet another embodiment, detection of the pedestrian is independent of speed of the vehicle and distance between the pedestrian and the vehicle.
[00066] In an embodiment, the pedestrian detection module generates an alert on detection of the pedestrian.
[00067] In another embodiment, the system further comprises a tracking module to track the detected bounding box of the pedestrian based on extracted IHOG features of the set of image frames.
[00068] In yet another embodiment, the system further comprises a collision determination module to determine expected time for collision between the vehicle and the pedestrian based on detection of the pedestrian.
[00069] In an embodiment, the scanning window selection module is configured to toggle scanning of the ROI of an image frame of the set of image frames.
[00070] In another embodiment, at least one classifier of the two or more classifiers facilitate detection of the pedestrian using a training image database, the training image database being created based on: cropping at least one training image frame of the one or more image frames using silhouette information of the pedestrian; resizing each of the cropped images to size of a nearest scanning window; removing undesirable structures from a classifier database of the at least one classifier; collecting false positives from the at least one classifier; and adding the false positives to the classifier database of a succeeding classifier.
[00071] In an aspect, present disclosure elaborates upon a method for pedestrian detection comprising: receiving, by one or more processors, one or more image frames and defining a Region of Interest (ROI) for a set of image frames selected from the received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of the each image frame and further based on a region defined in the field of view of the vehicle driver; determining, by the one or more processors, a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame; extracting, by the one or more processors, Integral Histogram of oriented gradients (IHOG) features from each scanning window; and detecting, by the one or more processors, the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
[00072] According to an aspect of the present disclosure, a pedestrian detection system (interchangeably referred to as the proposed system, hereinafter) enables detection of a pedestrian and provides a collision warning to a user (driver of a vehicle in which the proposed system is implemented, for example). Many a times, sudden movements in front of the vehicle pose a serious risk to both the pedestrians as well as the driver, along with others such as other occupants of the vehicle and the vehicle itself. In such a situation, the proposed system aids the driver in knowing about movements of various pedestrians well in advance by providing timely warnings, thereby leading to avoidance of accidents by the driver.
[00073] In one embodiment, an on board forward facing camera captures scene ahead of the vehicle. Further, image processing is performed on capturedimages to detect pedestrians in various actions, like standing, running, walking and across the roads, etc. and in all other orientations. The proposed system is able to detect pedestrians wearing any type of clothing/apparel/attire. Hence, an output is accordingly provided upon detection of one or more pedestrians.
[00074] FIG. 1 illustrates architecture of a pedestrian detection system to illustrate its overall working in accordance with an embodiment of the present disclosure.
[00075] As shown in FIG. 1, a pedestrian detection system 100 (referred to as the system 100, hereinafter) receives one or more image frames 102 as input. The image frames are captured using a camera or an image sensor that is preferably placed in rear-view mirror enclosure assembly of a vehicle such that the image frames pertain to field of view of a vehicle driver. Further, the system 100 selects a set of image frames from the received image frames 102 and defines a Region of Interest (ROI) for each image frame. For defining the ROI, the system 100 considers factors such as resolution of the image frame and a region defined in the field of view of the vehicle driver. The region defined in the field of view of the vehicle driver may be any of a far region, a middle region or a near region. In an embodiment, the system 100 defines the ROI from High Definition (HD) resolution image for the far region, Video Graphics Array (VGA) resolution image for the middle region, and Quarter VGA (QVGA) resolution image for the near region.
[00076] Thereafter, the system 100 determines a plurality of scanning windows in the ROI of each image frame as shown at 106. Size of each scanning window is computed based on presumed height of a pedestrian (that is to be detected) in the image frame. In order to enhance efficiency in detection of the pedestrian, the system 100 utilizes a technique to toggle scanning of the ROI of an image frame of the set of image frames.
[00077] In an embodiment, the system 100 extracts Integral Histogram of oriented gradients (IHOG) features from each scanning window as shown at 108.For effective feature extraction, each scanning window comprises a plurality of cells such that size of each of the plurality of cells for each scanning window is adjusted to make feature vector length same for each of the plurality of scanning windows.
[00078] In an embodiment, the system 100 detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers, as shown at 110. To facilitate detection by the classifiers, a training image database is created based on cropping training images of the one or more image frames using silhouette information of the pedestrian, resizing each of the cropped images to size of a nearest scanning window, removing undesirable structures from a classifier database ofat least one classifier of the two or more classifiers, collecting false positives from the at least one classifier, and adding the false positives to the classifier database of a succeeding classifier.
[00079] In an embodiment, the system 100 provides an output of a single bounding box around the detected pedestrian (as shown at 112) and generates an alert on detection of said pedestrian. Those skilled in the art would appreciate that various implementations of the system 100 enable detection of the pedestrian that is independent of speed of the vehicle and distance between the pedestrian and the vehicle.
[00080] FIG.2A illustrates exemplary functional modules of a pedestrian detection system in accordance with an embodiment of the present disclosure.
[00081] In an aspect, the system for pedestrian detection (indicated as system 100 herein) may comprise one or more processor(s) 202. The one or more processor(s) 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuitries, and/or any devices that manipulate data based on operational instructions. Among other capabilities, the one or more processor(s) 202 are configured to fetch and execute computer-readable instructions stored in a memory 204 of the system 100 proposed. The memory 204 may store one or more computer-readable instructions or routines, which may be fetched and executed to create or share the data units over a network service. The memory 204 may comprise any non-transitory storage device including, for example, volatile memory such as RAM, or non-volatile memory such as EPROM, flash memory, and the like.
[00082] The system 100 may also comprise an interface(s) 206. The interface(s) 206 may comprise a variety of interfaces, for example, interfaces for data input and output devices, referred to as I/O devices, storage devices, and the like. The interface(s) 206 may facilitate communication of the system 100 with various devices coupled to the system 100. The interface(s) 206 may also provide a communication pathway for one or more components of the system 100. Examples of such components include, but are not limited to, processing engine(s) 208 and data 228.
[00083] The processing engine(s) 208 may be implemented as a combination of hardware and programming (for example, programmable instructions) to implement one or more functionalities of the processing engine(s) 208. In examples described herein, such combinations of hardware and programming may be implemented in several different ways. For example, the programming for the processing engine(s) 208 may be processor executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the processing engine(s) 208 may comprise a processing resource (for example, one or more processors), to execute such instructions. In the present examples, the machine-readable storage medium may store instructions that, when executed by the processing resource, implement the processing engine(s) 208. In such examples, the system 100 may comprise the machine-readable storage medium storing the instructions and the processing resource to execute the instructions, or the machine-readable storage medium may be separate but accessible to system 100 and the processing resource. In other examples, the processing engine(s) 208 may be implemented by electronic circuitry.
[00084] The data 228 may comprise data that is either stored or generated as a result of functionalities implemented by any of the components of the processing engine(s) 208. For instance, data to be used by training image database 226 as described hereunder may be stored in data 228, besides other data.
[00085] In an exemplary embodiment, the engine(s) 208 may comprise an initialization module 210, a scanning window selection module 212, a feature extraction module 214, a pedestrian detection module 216, a non-maximal suppression module 218, a tracking module 220, a collision determination module 222, other modules 224.
[00086] It would be appreciated that modules being described are only exemplary modules and any other module or sub-module may be included as part of the system 100. These modules too may be merged or divided into super-modules or sub-modules as may be configured.
Initialization Module 210
[00087] In an aspect, initialization module 210 receives one or more image frames pertaining to field of view of a vehicle driver and selects a set of image frames from the received image frames. Further, the initialization module 210 defines a Region of Interest (ROI) for each image frame. The ROI is defined based on factors such as resolution of each image frame and region defined in field of view of the vehicle driver;
[00088] In an embodiment, the region in the field of view of the vehicle driver may be any of a far region, a middle region or a near region. The initialization module 210 defines the ROI from High Definition (HD) resolution image for the far region, Video Graphics Array (VGA) resolution image for the middle region, and Quarter VGA (QVGA) resolution image for the near region. According to an exemplary implementation, definition of ROI by the initialization module 210 in various regions based on resolution of the image is illustrated in FIG. 2B.
[00089] FIG.2C illustrates an exemplary block diagram of the initialization module 210 in accordance with an embodiment of the present disclosure. The initialization module 210 receives input information comprising an input image frame 242, camera parameters 244 and region of interest (ROI) information 246. Based on the input information, the initialization module 210 initializes different parameters including ROI parameters 248, scanning window parameters 250, IHOG (Integral Histogram of Oriented Gradient) parameters 252, SVM (Support vector machine) or classifier parameters 254 and tracker parameters 256. As would be appreciated by the one skilled in the art, properly setting the ROI parameters 248 is most important for pedestrian detection by the system 100 as ROI parameters248 are set so as to guarantee the localization of the pedestrian (i.e. determination of location of the pedestrian in an image).
Scanning Window Selection Module 212
[00090] In an aspect, scanning windows selection module 212 determines a plurality of scanning windows in the ROI of each of the set of image frames. The size of each scanning window is computed based on presumed height of a pedestrian in the image frame. In an embodiment, scanning window selection module 212 receives relevant inputs from the initialization module 210. Scanning windows play an integral role in object classification as objects (that is, the pedestrians) are localized for identification in an image. The scanning window splits depending on cell window size and distance where scanning is being performed, which aids in reducing the computation time. Also, consecutive scan cells may be reused for the same purpose.
[00091] In an embodiment, each ROI is scanned for detecting the pedestrian by considering different window sizes for scanning each of the ROI. For instance, for pedestrian detection in far region, ten windows may be considered in the ROI defined in the HD resolution image; for pedestrian detection in middle region, eleven windows may be considered in the ROI defined in the VGA resolution image; and for pedestrian detection in near region, seven windows may be considered in the ROI defined in the QVGA resolution image. Some exemplary windows sizes used for far, middle and near region pedestrian detection are listed in table 1 hereunder.
Windows for Far region pedestrian detection Windows for Middle region pedestrian detection Windows for Near region pedestrian detection
32x64 40x80 40x80
40x80 48x96 48x96
48x96 56x112 56x112
56x112 64x128 64x128
64x128 72x144 72x144
72x144 80x160 80x160
88x176 88x176 88x176
96x192 96x192
104x208 104x208
112x224 112x224
120x240
Table 1: Window sizes used for far, middle and near region pedestrian detection
[00092] Various embodiments of the present disclosure improve range and quality of detection, for example, the range of detection may be extended upto50 meters. In an embodiment, the window size for scanning the ROIisselected according to presumed height of the pedestrian. Height of the pedestrian in the frame for the given real world distance is calculated using following equation:
Pedestrian Height in Frame= Head y location in frame- Foot y location in frame
where,
Head y location in frame =Fy + (((CH-PH)*cos (theta)) + (RD*sin (theta)))/ ((RD*cos (theta)) –
((CH- PH)*-sin (theta)) * (focal length/pixel size)
Foot y location in frame=Fy + (((CH)*cos (theta)) + (RD*sin (theta)))/ ((RD*cos (theta))-
((CH)*-sin (theta)) *(focal length/pixel size)
Where:
CH: Camera Height in meter
RD: Real world distance of pedestrian from camera in meter
PH: Pedestrian Height Real World (1.8 meter approximately)
Fy: Frame Height in meter
theta: pitch angle
[00093] Once the pedestrian height is obtained, the window is selected by following criteria:
Window Height=Pedestrian Height in Frame *100/80*(nearest multiple of 16)
[00094] Once the window sizes are obtained, the row in which the scanning is required to be performed (for an example range of 35m to 50m) is obtained by trial and error method. For instance, for each window, scanning is performed in the given row, one cell shift up and one cell shift down.
[00095] FIG.2D illustrates an exemplary block diagram for implementation of feature extraction in accordance with an embodiment of the present disclosure. As illustrated, in an embodiment, the scanning window selection module 212 is operatively coupled with a cell size estimation module 262 such that output of the scanning window selection module 212 is provided as an input to the cell size estimation module 262. Each scanning window comprises plurality of cells, the cell size estimation module 262 resizes the cell sizes to make feature vector length for all the scanning windows same. Those skilled in the art would appreciate that the technique of resizing the cell sizes aids in increasing the computation speed and decreasing the complexity.
[00096] According to an embodiment of the invention, different size scanning windows are used for far, middle and near regions. The cell size estimation module 262 estimates cell size for each scanning window to make the IHOG feature of same length for all window sizes. The output of the cell size estimation module 262 is fed to the feature extraction module 214.
Feature Extraction Module 214
[00097] In an aspect, feature extraction module 214 extracts Integral Histogram of oriented gradients (IHOG) features from each scanning window, where each scanning window comprises plurality of cells such that size of each of the plurality of cells for each scanning window is adjusted by the cell size estimation module 262to make feature vector length same for each of the plurality of scanning windows.
[00098] As is known, HOG (Histogram of Gradients), Haar and LBP (Local Binary Pattern) are the most common feature extraction techniques used for object detection. Out of these known techniques, HOG feature extraction technique is most popular for detecting pedestrians. Thus, in an embodiment, the feature extraction module 214 utilizes HOG feature extraction technique to detect the pedestrian. HOG is an edge oriented histogram based on orientation of gradient in localized regions called cells. Therefore, it is easy to express rough shape of an object and HOG is robust to variations in geometry and illumination changes. Resizing of the cell sizes to make the feature vector length for all the scanning windows same by the cell size estimation module 262 aids in increasing the computation speed and decreasing the complexity. Further, Integral histogram (IHOG) is used for fast histogram extraction. Using the estimated cells, the IHOG features are extracted by the feature extraction module 214, for all the scanning windows within the ROI.
Pedestrian Detection Module 216
[00099] In an exemplary embodiment, output of feature extraction module 214 is further used for classification and object detection by the pedestrian detection module 216. In an aspect, pedestrian detection module 216 detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers that may be formed as illustrated in FIG. 2E. Further, the pedestrian detection module 216 is coupled with a Non-maximal suppression (NMS) module 218 that is configured to provide an output of a single bounding box around the detected pedestrian. Those skilled in the art would appreciate that detection of the pedestrian as disclosed in the present application is independent of speed of the vehicle and distance between the pedestrian and the vehicle.
[000100] FIG.2E illustrates an exemplary block diagram of classifier architecture utilized for pedestrian detection in accordance with an embodiment of the present disclosure. As illustrated, according to an embodiment, a three-level classifier is used for detection. Two AdaBoost classifiers 274a and 274b are combined with a support vector machine (SVM) classifier 276 for increasing the detection accuracy. Adaboost and SVM are generally used for classifying objects in real time condition. On the other hand, deep learning based approach like Convolutional Neural Network (CNN), You only look once (YOLO) architecture and Single-Stream Temporal Action Proposals (SST) architecture are used to detect and localize object precisely. However, deep learning approaches are not applicable to run in real time condition as the computation time and memory requirement is high. Since the objective of the present disclosure is to develop a pedestrian detection system which will detect pedestrian precisely and run in real time condition, Adaboost and SVM classifiers are used herein. According to an implementation, a cascade of classifier (two AdaBoost and one SVM) is found to be suitable to perform the said task.
[000101] In an aspect, at least one classifier of the two or more classifiers facilitates detection of the pedestrian using training image database 226. The training image database 226 is created by:
• cropping training images of the one or more image frames using silhouette information of the pedestrian
• resizing each of the cropped images to size of a nearest scanning window
• performing database cleanup by removing undesirable structures such as poles, trees, etc. from classifier database of the at least one classifier (of the two or more classifiers); and
• collecting false positives from the at least one classifier; and adding the false positives to classifier database of a succeeding classifier.
[000102] As illustrated in FIG. 2E, in an exemplary embodiment the extracted IHOG feature 272 from each scanning is fed into the first level of Adaboost classifier 274a. The first level Adaboost classifier 274a passes almost all pedestrians and few non-pedestrians. All the positives from first level Adaboost classifier 274a is fed into second level Adaboost classifier 274b. The second level Adaboost classifier 274b rejects few more non-pedestrians. All the positives from second level Adaboost classifier 274b may be further fed into SVM classifier 276.As the images are passing through the Adaboost classifiers prior to SVM classifier, the load on the SVM classifier is reduced. Only those windows which passes SVM classifier, is considered for detection. An exemplary output of the SVM classifier is illustrated in FIG. 2F. The output the classifier is multiple bounding boxes around the pedestrian. The output of SVM classifier 276 is further fed to non-maximal suppression (NMS) module 218.
Non-maximal suppression module 218
[000103] In an aspect, non-maximal suppression module 218 is coupled with pedestrian detection module 216 and is configured to provide an output of a single bounding box around the detected pedestrian.
[000104] According to an embodiment, as shown in FIG. 2E, output of a support vector machine (SVM) classifier 276 is received by non-maximal suppression module 218. As illustrated in FIG. 2F, output of the classifier 276 is multiple bounding boxes around the pedestrian. Non-maximal suppression (NMS) module 218 suppresses all the multiple boxes and draws a single bounding box around the pedestrian out of the multiple detection boxes, based on box's confidence and location, as illustrated in FIG. 2G.
Tracking Module 220
[000105] In an aspect, tracking module 220 tracks the detected bounding box of the pedestrian based on extracted IHOG features of the set of image frames.
[000106] In an embodiment, output of non-maximal suppression module 218 is received by tracking module 220. Different stages of tracking are as illustrated in FIG. 2H. Those skilled in the art would appreciate that, video tracking is the process of locating a moving object (or multiple objects) over time using a camera. The objective of video tracking is to associate target objects in consecutive video frames. The association is especially difficult when the objects are moving fast relative to the frame rate. A tracker is updated with the variables required for its functions. As illustrated in FIG. 2H, the tracking module 220 determines whether the tracking is in idle state, pre –track state, tracking state or cancel state based on number of valid counts.
Collision Determination Module 222
[000107] In an exemplary embodiment, output of the tracking module 220 is further used to calculate the time to collision by collision determination module 222. Collision determination module 222 calculates time taken for collision (TTC) for the detected pedestrian using equation as under:
TTC= Real World Distance of pedestrian from Host/ (Host velocity –Pedestrian velocity)
[000108] The real world distance estimation requires information about camera parameter and detected bounding box.
[000109] FIG. 2I is a block diagram illustrating working of collision determination module 222. As can be understood, Host velocity/Vehicle velocity is much larger than pedestrian velocity. Camera parameters and the detected bounded box may be used to determine real world distance of the pedestrian from the host/vehicle. Thereafter, TTC ( time to collision ) is calculated using following equation:
TTC= Real World Distance of pedestrian from Host/ (Host velocity)
[000110] Once a pedestrian is detected, a warning is provided to the driver that may be based on the TTC.
[000111] FIG. 2J illustrates a pedestrian collision warning sample frame output of different stages. The pedestrian detection warning may be displayed on any display devices known in the art. The warning may be in various forms known in the art, such as, but not limited to, visual, audio, sensation, combination of any, etc.
[000112] In an embodiment, the scanning window selection module 212 implements toggling of image frame scanning to make the image processing faster. Full strip scanning is performed for the HD resolution of first frame. For second frame, full strip scanning is performed for the VGA and QVGA resolution. In addition to this localized scanning is performed for specific region in the HD resolution where there is a detection box in previous frame. For third frame, full strip scanning is performed for HD resolution. Localized scanning is performed in specific areas of VGA and QVGA where there is detection in previous frame. The process is repeated to increase the rate of frame i.e. Frames per Second (FPS).
Other Modules 224
[000113] In an aspect, other modules 224 implement functionalities that supplement applications or functions performed by the system 100 or the processing engine(s) 208.
[000114] Although the proposed system has been elaborated as above to include all the main modules, it is completely possible that actual implementations may include only a part of the proposed modules or a combination of those or a division of those into sub-modules in various combinations across multiple devices that may be operatively coupled with each other, including in the cloud. Further the modules may be configured in any sequence to achieve objectives elaborated. Also, it may be appreciated that proposed system may be configured in a computing device or across a plurality of computing devices operatively connected with each other, wherein the computing devices may be any of a computer, a smart device, an Internet enabled mobile device and the like. Therefore, all possible modifications, implementations and embodiments of where and how the proposed system is configured are well within the scope of the present invention.
[000115] FIG. 3 illustrates a method of working of system proposed in accordance with an exemplary embodiment of the present disclosure.
[000116] In an aspect, the proposed method may be described in general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types. The method can also be practiced in a distributed computing environment where functions are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, computer executable instructions may be located in both local and remote computer storage media, including memory storage devices.
[000117] The order in which the method as described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method or alternate methods. Additionally, individual blocks may be deleted from the method without departing from the spirit and scope of the subject matter described herein. Furthermore, the method may be implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method may be considered to be implemented in the above described system.
[000118] In an aspect, present disclosure elaborates upon a method for pedestrian detection that comprises, at step 302, receiving, by one or more processors, one or more image frames and defining a Region of Interest (ROI) for a set of image frames selected from the received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of the each image frame and further based on a region defined in the field of view of the vehicle driver; and at step 304, determining, by the one or more processors, a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of a pedestrian in the image frame.
[000119] The method further comprises, at step 306, extracting, by the one or more processors, Integral Histogram of oriented gradients (IHOG) features from each scanning window; and at step 308, detecting, by the one or more processors, the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
[000120] FIG. 4 illustrates overall working of the proposed system in accordance with an exemplary embodiment of the present disclosure.
[000121] As illustrated a plurality of image sensors or cameras shown as 402-1, 402-2….402-N (collectively described as camera 402) can be configured in a vehicle to capture image frames of different regions, particularly the front facing region of the vehicle.
[000122] The system 100receives different images from one or more cameras 402. As elaborated above, thereafter the system 100 generates at least one bounding box 404 containing image of a pedestrian detected. Procedures for generation of the at least one bounding box 404 may be as elaborated above.
[000123] In an exemplary embodiment, the bounding box 404 is displayed to driver of the vehicle on a display device implemented in the vehicle in such a manner that the driver can see the pedestrian in a timely manner and take evasive action to protect the pedestrian.
[000124] The system 100 also determines if a collision with the pedestrian is imminent and if so, generates a collision warning shown as 406. Warning 406 may take the form of audio or video signals or a combination of the two to warn the driver of the vehicle well in time.
[000125] As would be appreciated, system 100may be integrated with existing systems and controls of a vehicle to form an advanced driver assistance system (ADAS), or augment an existing ADAS. For instance, signals generated by the system 100 may be sent to engine control unit (ECU) of the vehicle and may help in automatically applying brakes, disabling acceleration and sounding horn of the vehicle. All such steps can aid in avoiding injury to pedestrians, occupants of the vehicle, and the vehicle itself.
[000126] As would be readily appreciated, while primary application for disclosure as elaborated herein is in the automotive domain for pedestrian detection, it may be used in non-automotive domain as well wherein any moving object may be similarly detected.
[000127] Thus, the system and method of the present disclosure provides for a simple, robust and accurate pedestrian detection irrespective of the pedestrian attire, the pedestrian orientation and the changing day time illumination. Additionally, the system and method of the present disclosure uses a three-level classifier for better detection of pedestrian thereby reducing the false positives. The present disclosure also provides a system and method that implements frame toggling for increasing the frames per second (FPS) while detection. With the system and method of the present disclosure, range of detection and quality of detection is improved.
[000128] As elaborated above, the proposed system uses several unique features. For example, the proposed system considers different regions of interest (ROIs) from different resolution- Far, Near and Middle region –and pedestrian detection is independent of speed of vehicle and distance from object/pedestrian leading to an improved detection range. Window size for scanning is selected according to pedestrian height and cell sizes are resized to make the feature vector length same for all the scanning windows, which aids in decreasing the process complexity and increasing the computation speed, thereby making the system highly responsive. Proposed system uses a cascade of classifiers, for instance, three-level of classifiers isused as described above for better pedestrian detection and reduction in false positives. Also, the proposed system uses a unique strategy for training sample collection and annotation that aids build a strong classifier model.
[000129] As used herein, and unless the context dictates otherwise, the term “coupled to” is intended to include both direct coupling (in which two elements that are coupled to each other or in contact each other)and indirect coupling (in which at least one additional element is located between the two elements). Therefore, the terms “coupled to” and “coupled with” are used synonymously. Within the context of this document terms “coupled to” and “coupled with” are also used euphemistically to mean “communicatively coupled with” over a network, where two or more devices are able to exchange data with each other over the network, possibly via one or more intermediary device.
[000130] Moreover, in interpreting both the specification and the claims, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced. Where the specification claims refers to at least one of something selected from the group consisting of A, B, C ….and N, the text should be interpreted as requiring only one element from the group, not A plus N, or B plus N, etc.
[000131] While some embodiments of the present disclosure have been illustrated and described, those are completely exemplary in nature. The disclosure is not limited to the embodiments as elaborated herein only and it would be apparent to those skilled in the art that numerous modifications besides those already described are possible without departing from the inventive concepts herein. All such modifications, changes, variations, substitutions, and equivalents are completely within the scope of the present disclosure. The inventive subject matter, therefore, is not to be restricted except in the spirit of the appended claims.
ADVANTAGES OF THE INVENTION
[000132] The present disclosure provides a system and method for pedestrian detection that accurately detects a pedestrian and provide a warning accordingly.
[000133] The present disclosure provides a system and method for pedestrian detection that detects pedestrians irrespective of their attire.
[000134] The present disclosure provides a system and method that detects a pedestrian in varied illuminations of day time.
[000135] The present disclosure provides a system and method that detects a pedestrian in all orientations.
[000136] The present disclosure provides a robust, economic and simple system and method that accurately detects a pedestrian.
,CLAIMS:
1. A pedestrian detection system implemented in a vehicle, said system comprising:
a non-transitory storage device having embodied therein one or more routines operable to detect a pedestrian; and
one or more processors coupled to the non-transitory storage device and operable to execute the one or more routines, wherein the one or more routines include:
an initialization module, which when executed by the one or more processors, receives one or more image frames and defines a Region of Interest (ROI) for a set of image frames selected from said received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of said each image frame and further based on a region defined in the field of view of the vehicle driver;
a scanning window selection module, which when executed by the one or more processors, determines a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame;
a feature extraction module, which when executed by the one or more processors, extracts Integral Histogram of oriented gradients (IHOG) features from each scanning window; and
a pedestrian detection module, which when executed by the one or more processors, detects the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
2. The pedestrian detection system of claim 1, wherein the region in the field of view of the vehicle driver is any of a far region, a middle region or a near region and wherein, the initialization module defines the ROI from High Definition (HD) resolution image for the far region, Video Graphics Array (VGA) resolution image for the middle region, and Quarter VGA (QVGA) resolution image for the near region.
3. The pedestrian detection system of claim 1, wherein the feature extraction module extracts IHOG features of each scanning window, each scanning window comprising a plurality of cells such that size of each of the plurality of cells for each scanning window is adjusted to make feature vector length same for each of the plurality of scanning windows.
4. The pedestrian detection system of claim 1, wherein the pedestrian detection module is coupled with a Non-maximal suppression (NMS) module that is configured to provide an output of a single bounding box around the detected pedestrian.
5. The pedestrian detection system of claim 1, wherein detection of the pedestrian is independent of speed of the vehicle and distance between the pedestrian and the vehicle.
6. The pedestrian detection system of claim 1, wherein the pedestrian detection module generates an alert on detection of said pedestrian.
7. The pedestrian detection system of claim 4, said system further comprising a tracking module to track the detected bounding box of the pedestrian based on extracted IHOG features of the set of image frames.
8. The pedestrian detection system of claim 1, said system further comprises a collision determination module to determine expected time for collision between the vehicle and the pedestrian based on detection of said pedestrian.
9. The pedestrian detection system of claim 1, wherein the scanning window selection module is configured to toggle scanning of the ROI of an image frame of the set of image frames.
10. The pedestrian detection system of claim 1, wherein at least one classifier of the two or more classifiers facilitates detection of the pedestrian using a training image database, the training image database being created based on:
cropping training images from one or more image frames using silhouette information of the pedestrian;
resizing each of the cropped images to size of a nearest scanning window;
removing undesirable structures from a classifier database of the at least one classifier;
collecting false positives from the at least one classifier; and
adding said false positives to the classifier database of a succeeding classifier.
11. A method for pedestrian detection, said method comprising:
receiving, by one or more processors, one or more image frames and defining a Region of Interest (ROI) for a set of image frames selected from said received one or more image frames, wherein each image frame of the set of image frames pertains to field of view of a vehicle driver, wherein the ROI is defined based on resolution of said each image frame and further based on a region defined in the field of view of the vehicle driver;
determining, by the one or more processors, a plurality of scanning windows in the ROI of each of the set of image frames, wherein size of each scanning window is computed based on presumed height of the pedestrian in the image frame;
extracting, by the one or more processors, Integral Histogram of oriented gradients (IHOG) features from each scanning window; and
detecting, by the one or more processors, the pedestrian based on the extracted IHOG features from each scanning window using a cascade of two or more classifiers.
| # | Name | Date |
|---|---|---|
| 1 | 201721018168-IntimationOfGrant29-12-2023.pdf | 2023-12-29 |
| 1 | 201721018168-RELEVANT DOCUMENTS [02-04-2018(online)].pdf | 2018-04-02 |
| 2 | 201721018168-Changing Name-Nationality-Address For Service [02-04-2018(online)].pdf | 2018-04-02 |
| 2 | 201721018168-PatentCertificate29-12-2023.pdf | 2023-12-29 |
| 3 | 201721018168-DRAWING [02-05-2018(online)].pdf | 2018-05-02 |
| 3 | 201721018168-Annexure [26-12-2023(online)].pdf | 2023-12-26 |
| 4 | 201721018168-Written submissions and relevant documents [26-12-2023(online)].pdf | 2023-12-26 |
| 4 | 201721018168-COMPLETE SPECIFICATION [02-05-2018(online)].pdf | 2018-05-02 |
| 5 | 201721018168-FORM-9 [03-05-2018(online)].pdf | 2018-05-03 |
| 5 | 201721018168-FORM-26 [13-12-2023(online)].pdf | 2023-12-13 |
| 6 | 201721018168-FORM 18 [03-05-2018(online)].pdf | 2018-05-03 |
| 6 | 201721018168-Correspondence to notify the Controller [12-12-2023(online)].pdf | 2023-12-12 |
| 7 | 201721018168-US(14)-HearingNotice-(HearingDate-14-12-2023).pdf | 2023-11-21 |
| 7 | 201721018168-REQUEST FOR CERTIFIED COPY [12-06-2018(online)].pdf | 2018-06-12 |
| 8 | 201721018168-Proof of Right (MANDATORY) [28-06-2018(online)].pdf | 2018-06-28 |
| 8 | 201721018168-FER.pdf | 2021-10-18 |
| 9 | 201721018168-ABSTRACT [04-06-2021(online)].pdf | 2021-06-04 |
| 9 | ABSTRACT1.jpg | 2018-08-11 |
| 10 | 201721018168-CLAIMS [04-06-2021(online)].pdf | 2021-06-04 |
| 10 | 201721018168-ORIGINAL UR 6(1A) FORM 1-020718.pdf | 2018-08-11 |
| 11 | 201721018168-COMPLETE SPECIFICATION [04-06-2021(online)].pdf | 2021-06-04 |
| 11 | 201721018168-ORIGINAL UR 6( 1A) FORM 26-110418.pdf | 2018-08-11 |
| 12 | 201721018168-CORRESPONDENCE [04-06-2021(online)].pdf | 2021-06-04 |
| 12 | 201721018168-Form 5-240517.pdf | 2018-08-11 |
| 13 | 201721018168-FER_SER_REPLY [04-06-2021(online)].pdf | 2021-06-04 |
| 13 | 201721018168-Form 3-240517.pdf | 2018-08-11 |
| 14 | 201721018168-Form 2(Title Page)-240517.pdf | 2018-08-11 |
| 14 | 201721018168-FORM 3 [04-06-2021(online)].pdf | 2021-06-04 |
| 15 | 201721018168-Form 1-240517.pdf | 2018-08-11 |
| 15 | 201721018168-Information under section 8(2) [04-06-2021(online)].pdf | 2021-06-04 |
| 16 | 201721018168-CORRESPONDENCE(IPO)-(CERTIFIED COPY)-(14-6-2018).pdf | 2018-08-11 |
| 16 | 201721018168-OTHERS [04-06-2021(online)].pdf | 2021-06-04 |
| 17 | 201721018168-PETITION UNDER RULE 137 [04-06-2021(online)].pdf | 2021-06-04 |
| 17 | 201721018168-FORM 3 [30-08-2018(online)].pdf | 2018-08-30 |
| 18 | 201721018168-Form 3-100120.pdf | 2020-01-11 |
| 18 | 201721018168-RELEVANT DOCUMENTS [18-11-2019(online)].pdf | 2019-11-18 |
| 19 | 201721018168-FORM 13 [18-11-2019(online)].pdf | 2019-11-18 |
| 20 | 201721018168-Form 3-100120.pdf | 2020-01-11 |
| 20 | 201721018168-RELEVANT DOCUMENTS [18-11-2019(online)].pdf | 2019-11-18 |
| 21 | 201721018168-FORM 3 [30-08-2018(online)].pdf | 2018-08-30 |
| 21 | 201721018168-PETITION UNDER RULE 137 [04-06-2021(online)].pdf | 2021-06-04 |
| 22 | 201721018168-CORRESPONDENCE(IPO)-(CERTIFIED COPY)-(14-6-2018).pdf | 2018-08-11 |
| 22 | 201721018168-OTHERS [04-06-2021(online)].pdf | 2021-06-04 |
| 23 | 201721018168-Form 1-240517.pdf | 2018-08-11 |
| 23 | 201721018168-Information under section 8(2) [04-06-2021(online)].pdf | 2021-06-04 |
| 24 | 201721018168-FORM 3 [04-06-2021(online)].pdf | 2021-06-04 |
| 24 | 201721018168-Form 2(Title Page)-240517.pdf | 2018-08-11 |
| 25 | 201721018168-Form 3-240517.pdf | 2018-08-11 |
| 25 | 201721018168-FER_SER_REPLY [04-06-2021(online)].pdf | 2021-06-04 |
| 26 | 201721018168-CORRESPONDENCE [04-06-2021(online)].pdf | 2021-06-04 |
| 26 | 201721018168-Form 5-240517.pdf | 2018-08-11 |
| 27 | 201721018168-COMPLETE SPECIFICATION [04-06-2021(online)].pdf | 2021-06-04 |
| 27 | 201721018168-ORIGINAL UR 6( 1A) FORM 26-110418.pdf | 2018-08-11 |
| 28 | 201721018168-CLAIMS [04-06-2021(online)].pdf | 2021-06-04 |
| 28 | 201721018168-ORIGINAL UR 6(1A) FORM 1-020718.pdf | 2018-08-11 |
| 29 | 201721018168-ABSTRACT [04-06-2021(online)].pdf | 2021-06-04 |
| 29 | ABSTRACT1.jpg | 2018-08-11 |
| 30 | 201721018168-FER.pdf | 2021-10-18 |
| 30 | 201721018168-Proof of Right (MANDATORY) [28-06-2018(online)].pdf | 2018-06-28 |
| 31 | 201721018168-US(14)-HearingNotice-(HearingDate-14-12-2023).pdf | 2023-11-21 |
| 31 | 201721018168-REQUEST FOR CERTIFIED COPY [12-06-2018(online)].pdf | 2018-06-12 |
| 32 | 201721018168-FORM 18 [03-05-2018(online)].pdf | 2018-05-03 |
| 32 | 201721018168-Correspondence to notify the Controller [12-12-2023(online)].pdf | 2023-12-12 |
| 33 | 201721018168-FORM-9 [03-05-2018(online)].pdf | 2018-05-03 |
| 33 | 201721018168-FORM-26 [13-12-2023(online)].pdf | 2023-12-13 |
| 34 | 201721018168-Written submissions and relevant documents [26-12-2023(online)].pdf | 2023-12-26 |
| 34 | 201721018168-COMPLETE SPECIFICATION [02-05-2018(online)].pdf | 2018-05-02 |
| 35 | 201721018168-DRAWING [02-05-2018(online)].pdf | 2018-05-02 |
| 35 | 201721018168-Annexure [26-12-2023(online)].pdf | 2023-12-26 |
| 36 | 201721018168-PatentCertificate29-12-2023.pdf | 2023-12-29 |
| 36 | 201721018168-Changing Name-Nationality-Address For Service [02-04-2018(online)].pdf | 2018-04-02 |
| 37 | 201721018168-IntimationOfGrant29-12-2023.pdf | 2023-12-29 |
| 37 | 201721018168-RELEVANT DOCUMENTS [02-04-2018(online)].pdf | 2018-04-02 |
| 1 | 2020-12-1412-47-37E_14-12-2020.pdf |
| 1 | AmendedSearchAE_21-10-2021.pdf |
| 2 | 2020-12-1412-47-37E_14-12-2020.pdf |
| 2 | AmendedSearchAE_21-10-2021.pdf |