TECHNICAL FIELD

[0001] The present invention relates to the field of the traffic monitoring system, and more particularly, the present invention relates to a real-time occluded vehicle detection system.

BACKGROUND ART
[0002] The following discussion of the background of the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgment or admission that any of the material referred to was published, known, or part of the common general knowledge in any jurisdiction as of the application’s priority date. The details provided herein the background if belongs to any publication is taken only as a reference for describing the problems, in general terminologies or principles or both of science and technology in the associated prior art.
[0003] As object detection technology matures, it is widely used across a variety of fields, such as image recognition, and has attained its top in the detection of traditional images [1]. Despite this, object detection technologies still need improvements when dealing with some scenarios like the weak, dim, and severely occluded detection of vehicles with complicated backgrounds in infrared images [2]. Current research focuses on small object detection in the infrared spectrum without considering the impacts of complex backgrounds, which are additionally difficult to identify. Innovative research and improvements by the advancement of deep learning techniques have essentially advanced the progress in object detection through Convolutional Neural Networks (CNN). Object detection based on region-based CNN accomplished a great capability [3].
[0004] Therefore, this paper presents the methods of detecting the infrared objects affected by occlusion and change in viewpoint. Images captured by infrared cameras are susceptible to a number of inherent defects, including the distance of the image, response to changes in light, and point of view. Also, the images might be effortlessly influenced by the radiation of the atmosphere and objects occluding the light flow in an insufficient imaging effect [4]–[6]. Furthermore, a significant amount of noise, the smaller contrast, and the fuzzy boundary between the target and the background cause difficulties to detect infrared images compared to normal datasets like MS COCO and ImageNet. In the case of long-range aerial images in infrared, the target size of the image is much smaller as well as the resolution is too lower than the actual image, which typically has a mean pixel size of 30x30 [7].
[0005] There has been a growing interest in small-scale object detection in the object detection field. Multi-scale fusing [8] and feature fusion [9] are the two most common approaches to small-scale objects. Specifically, the difficulties of detecting objects in difficult backgrounds deliberated in this research are compounded by noise and small scales. Together with the obstructions of surrounding trees as well as other issues, this creates a huge disturbance to detecting model features. It is possible, in some cases, that in the background, non-object features can baffle the detector, leading to having false decisions, which would result in a maximum rate of false detection (i.e. low precision). On the other hand, there is a great practical consequence in studying how to increase the rate of accuracy in detection of the detector to allow it to have still high performance in complex environments with severe occlusive targets. This can liberate the human work force from having to recognize large volumes of images, especially in the detection of military targets. Serial methods like YOLO and their improved variants are complex in terms of their network structure and include a greater number of parameters.
[0006] They need high-quality GPUs (Graphic Processing Units) which have the computational power to detect real-time objects [13]. These devices have limited computing power and memory, and they may need to detect objects in real-time in real-world applications for some mobile and embedded devices (for example, autonomous driving, augmented reality, and other smart devices) [14]. When a real-time inference is needed, including on smartphones and embedded video surveillance, the available computing resources are limited, such as low-powered embedded GPUs or even just CPUs with a limited amount of memory [15]. Because of this, real-time object detection on mobile devices and embedded devices remains a big challenge. Many researchers have pro moted lightweight object detection techniques to solve the problem. Compared to conventional methods, lightweight methods employ fewer parameters and simpler network structures [16]. In turn, that means they do not require as much computing power and memory and can detect things faster. In general, they are more suited to being installed on mobile and embedded devices. Even though the detection accuracy of these sensors is low, it is sufficient for the actual requirements. Deep learning-based lightweight object detection methods can be applied in various applications, such as detecting vehicles, pedestrians, and passengers on buses, in agricultural applications, and in the detection of abnormal human behaviour.
[0007] A common problem is the shortage of infrared data with labels in remote sensing. The visible image dataset is huge, whereas the infrared image dataset is relatively small, which complicates the training procedure for detecting infrared objects. The idea is to introduce a mining block of hard negative examples with the YOLOv4 model to solve the high false-positive rate caused by the complex background. A ratio of approximately 1:3 was used for the addition of negative and positive samples to ensure the balance of positive and negative samples during secondary training. As a final point, the accuracy rate in the detection of the modified YOLOv4 network model increased from 89.45 percent to 92.27 percent, proving that the improved model may satisfy the demand. In general, the proposed work makes these major contributions: A technique of secondary transfer learning is recommended to address the problem of limited datasets.
[0008] The available systems are not economical, accurate, and time efficient. Further, the available systems are not user-friendly as these systems take time to respond.
[0009] In light of the foregoing, there is a need for a real-time occluded vehicle detection system that overcomes problems prevalent in the prior art associated with the traditionally available method or system, of the above-mentioned inventions that can be used with the presented disclosed technique with or without modification.
[0010] All publications herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies, and the definition of that term in the reference does not apply.

OBJECTS OF THE INVENTION

[0011] The principal object of the present invention is to overcome the disadvantages of the prior art by providing a real-time occluded vehicle detection system.
[0012] The object of the present invention is to provide a real-time occluded vehicle detection system that proposes propose an improved algorithm for occluded vehicle detection by applying augmentation policy on the YOLO model and computing the class confidence value of each bounding box and intersection over the union.
[0013] Another object of the present invention is to provide a real-time occluded vehicle detection system, wherein the CSPBlock module uses CSPDarknet53-tiny as the backbone of YOLOv4-tiny, which detects 371 frames to improve the accuracy rate in vehicle detection.
[0014] Another object of the present invention is to provide a real-time occluded vehicle detection system that obtains state-of-the-art (SOTA) performance on both the MS COCO dataset and the PASCAL dataset using YOLOv4 and Hard Negative Example Mining (HNEM).
[0015] Another object of the present invention is to provide a real-time occluded vehicle detection system that achieves a better performance in terms of accuracy, precision, recall, and F1 score for occluded vehicle detection.
[0016] The further object of the present invention is to provide a real-time occluded vehicle detection system that proposes a YOLO-based detection model for occluded objects with HNEM and augmentation policy optimization. The hard-positive and hard-negative detection of objects are to be extracted by a feature vector of the boundary with confidence score thresholding for this detection model.
[0017] The foregoing and other objects of the present invention will become readily apparent upon further review of the following detailed description of the embodiments as illustrated in the accompanying drawings.

SUMMARY OF THE INVENTION
[0018] The present invention relates to a real-time occluded vehicle detection system.
[0019] Object control is a fundamental computer vision technology used in various industries, including healthcare, transportation, and medical services. The object detection method is indeed a computerized method of separating and identifying the objects of interest from its image’s background [17]. However, unlike the object identification system, which merely recognizes specific objects, this skill acquired the location information of the concerned objects and perhaps even the location data of many types of objects in a single image. Deep learning algorithms are now being studied in relation to object-detecting technologies [18], [19]. One-stage and two-stage detectors are two forms of deep-learning computer vision algorithms. One-stage detectors are detection frameworks that recognize objects rapidly using the bounding box’s size, with a dimension of the structuring element purposely designed to be helpful in a variety of contexts [20].
[0020] One-stage sensors like the YOLO, SSD, and RetinaNet allow fast identification by simultaneously running the region proposal and object separation even though such systems are based on real-time detection but less accurate and have low efficiency in detection [20]. Two-stage detectors models are sequentially operating the region proposal and object separation by involving the objects of interest (Region Proposal Network). Faster RCNN and R-FCN systems are the two-stage detectors that provide higher accuracy than one stage detectors, but the detection speed in real-time is low [21]. The YOLO (You Only Look Once) model was implemented with the single neural network by dividing a single image into grid cells, with each cell utilizing the attributes of the overall image.
[0021] The YOLO model’s approach for finding the objects of interest from the target image. The learnt channel is partitioned into M*N grid cells inside the model learning process, as well as the confidence score of every cell are analyzed simultaneously by calculating the confidence score of individual objects [22]. Finally, the models detect the objects of interest based on the class’s resulting image for predictions. In research on image classification, S. Jeon et al. developed a real-time roadway-driven lane recognition system based mainly on the YOLO paradigm for highway image classification [23]. The Jetson Nano Developer Kit and CSI camera are used in this system to gather and analyze driving data in various conditions. Faster R-CNN is the improved version of Fast R-CNN, which generates candidate regions by using RPN (Region Proposal Network) [24]–[28].
[0022] While extracting the candidate region, each pixel generates multiple bounding boxes in the feature map in different scales, based on the offset of the corresponding anchors. The feature maps for classification and regression are routed to two networks when selecting candidate regions. The feature maps are sent into two networks for classification and regression when selecting candidate regions. RPN classification is used to distinguish the regression in RPN and predicts the offset coordinates between the anchor box and the ground truth, regardless of whether the corresponding anchor belongs to the foreground or the background.
[0023] While the invention has been described and shown with reference to the preferred embodiment, it will be apparent that variations might be possible that would fall within the scope of the present invention.

BRIEF DESCRIPTION OF DRAWINGS
[0024] So that the manner in which the above-recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may have been referred by embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
[0025] These and other features, benefits, and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
[0026] Figure 1 structure of YOLOv4 Network, in accordance with an exemplary embodiment of the present invention;
[0027] Figure 2 hard Negative Mining Module based on YOLOv4, in accordance with an exemplary embodiment of the present invention;
[0028] Figure 3 working Model for Vehicle Detection due to Occlusion using HNEM, in accordance with an exemplary embodiment of the present invention;
[0029] Figure 4 YOLOv4 Model Process with HNEM Optimizer, in accordance with an exemplary embodiment of the present invention;
[0030] Figure 5 comparison YOLOv4 with Existing Methods, in accordance with an exemplary embodiment of the present invention;
[0031] Figure 6 detection Result based on YOLOv4 C using HNEM Testing Set, in accordance with an exemplary embodiment of the present invention; and
[0032] Figure 7 (a), (b), (c) shows the Results of Vehicle Detection in Different Conditions.

DETAILED DESCRIPTION OF THE INVENTION
[0033] While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and the detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claim.
[0034] As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein are solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers, or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles, and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
[0035] In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element, or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
[0036] The present invention is described hereinafter by various embodiments with reference to the accompanying drawing, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, several materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
[0037] The present invention relates to Real-time occluded vehicle detection system. In this section, the YOLOv4-tiny model combined with Hard Negative Example Mining (HNEM) uses the CSP Block module by using the CSPDarknet-53 as the backbone to improve the detection rate of occluded vehicles. A. Analysis of Network Structure In order to make Yolov4-tiny have a faster rate of object detection, it is designed from the Yolov4 method. Yolov4-tiny can detect objects at a rate of 371 frames per second using the 1080 Ti GPU with an accuracy that meets the demands of real-world applications. Therefore, mobile devices or embedded systems can use this object detection method with considerable ease. Instead of using the CSPDarknet53 network used in the Yolov4 method, this Yolov4-tiny method uses CSPDarknet53-tiny as the backbone network. Instead of using the ResBlock module in the residual network, the CSPBlock module is used by the CSPDarknet53-tiny network in the cross-stage partial network module. All the feature maps are divided by the CSPBlock module into two parts. It merges the two parts by cross-stage residual edges [32]. In this way, the gradient information propagates along with two different network pathways, resulting in an increased correlation difference. The convolutional networks can be more easily learned B. Exploration of Hard Negative Example Mining Despite the heavy occultation by trees in the infrared image, there are many differences between the background and target. That results in many confusing images. Because complex backgrounds will affect detection performance and increase false detections, an example mining module for hard negatives is proposed with the YOLOv4 model at the end [33]. As a starting point, it is necessary to calculate each bonding box’s class confidence of C1 predicted using the classifier and the IOU values among the ground truth labels and the bounding boxes. As a general rule, a bounding box for which C2 < 0 : 4 is an IOU value indicates an FP (False Positive) sample, meaning it is expected to be an object but is, in fact, background information. The higher the class confidence C1 is for these bounding boxes, the harder it would be for the classifier to identify them correctly. Those are the kind of hard-negative examples used in this experiment. In this case, the samples are sorted according to the confidence value C1, thus forming the sample dataset DV as illustrated in figure 2. Three different YOLO Heads are included with the YOLOv4 model. Every head is fed with a different layer of scales. Consider that a scale of 608x608x3 is input. The three layers prior to the heads are 19x19 x1024, 38x38 x512 and 76x76 x256, respectively. C. Occluded Object Framework using Hard Negative Ex ample Mining This paper proposes an algorithm to detect the occluded vehicle based on YOLOv4 and HNEM in real-time.
[0038] This proposed algorithm is executed in three phases: Augmen tation Policy on Model, second Hard Example Mining, and finally, YOLOv4 modeling. The framework based on YOLOv4 occluded vehicle detection model is shown in Fig ure 3. The data was collected for occlusion vehicle detection obtained from the IARA (Intelligent Robotic Autonomous Automobile) and GOPRO. Above mentioned dataset contains a wide range of objects that may be found in highway vehicle dynamics. It also includes high-resolution picture datasets with respect to time, space and climates. The data was utilized to develop a model that detects objects that due to object occlusion in this research. The obtained data were subjected to nine enhancement policies to develop new training examples. Mix Up, Cut Out, Cut Mix, Mosaic, Blurring, Brightness, Contrast, Hue, and Gray Scale were some of the enhancement policies available [34].
[0039] These strategies are then proposed towards the main YOLO learning process and then obtaining the object bounding box regression value of performing the IOU (Intersection over Union) index. The comparisons were made with truth values, and favorable impact by predicting regression value on occluded crosswalk objects detection were permitted through all the gradient-based residual. In the next stage, the hard example mining is done to obtain the hard-positive and hard negative examples after applying the augmentation policy on training data. The rate for false-negative detection and false-positive detection has increased after calculating the confidence score and bound ing box parameters. In order to overcome the problem of increment rate in false-negative detection and false-positive detection, reclassification and bounding box update were performed in the hard mining stage. YOLOv4 modeling is done based on a convolutional base and bounding box regression at the final stage. Therefore, the proposed method achieved a better result compared to the results of existing method [10].
[0040] D. Evaluation of Model Structure Figure 4 shows the entire optimization process for the improved model using the HNEM optimizer. Since YOLOv4 has many more desirable properties than previous versions of YOLO [35]–[37], it has been adopted as the basis for the research. For example, the backbone has a much deeper structure than the previous Darknet-53 that is far more powerful than YOLOv2’s Darknet-19, but ResNet152 and ResNet-101 are significantly more efficient [21]. In addition, YOLOv4 provides the Cross-Stage-Partial, which can be used to improve both the accuracy and the speed of Darknet53 compared to YOLOv3. Keeping the small-scale features in the cross-stage network and the residual part is possible. Thus, the original YOLOv4 will not require a change to the connection relationship.

[0041] TABLE I and TABLE II show the comparison of the versions from the proposed work on the set of images and existing algorithms with the proposed work. The backbone structure, function loss, FPS, and mAP are compared on the Pascal VOC 2007 and the MS COCO. The YOLOv4 is better at detecting objects with higher accuracy and speed. As evidenced in the published liter ature, YOLOv4 models always perform better than YOLO and YOLOv2, the YOLOv4 model is selected for testing, and no comparison experiments are conducted on YOLO or YOLOv2 models. Before the training, the anchor box sizes are calculated based on the K-means cluster method. After the experiments, k D 9 is set and in the result nine different anchor box sizes are observed: (10, 25), (12, 44), (12, 38), (14, 23), (16, 32), (18, 55), (19, 22), (24, 26), (44, 35), and the image pixels size was set to 416 x416. 4. Results and Analysis Table II shows the mAP and average processing time per frame along with AP (Average Precision), FPS and F1 Score for various models and Table III.
[0042] Generally, Average Precision is used as an evaluation metric to detect the objects, but when binary coding is applied to alleviate the effects of imbalanced negative and positive examples, the score of F1 is also combined as a comprehensive rating Maps of validation losses and training for the YOLOv4 model are shown in figure 5, together with secondary transition, one-transfer, and without transfer learning. The objective of this study is to test the promoting performance of transfer learning in the proposed model. In accordance with figure 3, it can be seen that without transfer learning, the model could not converge to a very small value for the parameters, and their convergence speeds are much slower because of the inadequate datasets and the end test results also confirm this. Training losses do not differ much between one transfer and secondary transfer learning.
[0043] However, secondary transfer learning has a faster convergence speed, and also the validation loss is better matched to the training loss than one transfer learning. An example of detection results is shown in figure 6, which pertains to a test set on YOLOv4 C HNEM. There are three boxes on the plot: in the blue box, GT (i.e. ground truth label) is displayed, whereas, in the green box, TP (i.e. true positive sample) was identified by the model, while in the red box, FP (i.e. False Positive) sample was identified by the model. Listed below are the connections between them. Essentially, class confidence is calculated for each bounding box C1 projected by the classifier and the IOU values C2 among each bounding box and the labeled ground truth labels. The bounding box whose C2 < 0:4 IOU value indicates an FP (False Positive) sample, meaning it is projected as an object when it is the background information. This implies that the higher the C1 confidence level is, the harder it will be for the classifier to identify these bounding boxes correctly. These were the examples considered in this experiment to be hard negatives. The data were sorted in descending order for the sample dataset DV using the confidence value C1.

[0044] D = {C i 1 |C i 2 < 0.4,C i 1 > C j 1 , 1 = i, j = N} (1)
[0045] D ' = {C i 1 |C i 2 < 0.4,C i 1 > C j 1 , 1 = i, j = n} (2)
[0046] With the above equations 1 and 2, it is evident that the more red boxes there are, the lesser the precision of the model is, while the fewer green boxes there are, the lesser the recall rate of the model is. Thus, if the target is deeply occluded, the model is less likely to detect it, or some of the deceptive features may identify it in the background with a high degree of confidence, illustrated in the 3rd column of figure 7. The following equations 3, 4, and 5 show the precision, recall, and F1 Score respectively. Precision = T P /T P + FP (3) Recall = T P/ T P + FN = T P GT (4) F1S core = Precision * Recall Precision + Recall (5) In figure 7, a sample of the detection results of the 3 models is illustrated by occluded images of vehicles. It is possible to identify the misclassified objects from the non-improved models with the improved model. Therefore, incorporating the HNEM module in the real complex.
[0047] An infrared aerial image with a highly weak occluded vehicle detection using the YOLOv4 model was analyzed under complicated background conditions. The YOLOv4 model exploits the extra second-order transfer learning method to deal with the insufficient datasets problem. The hard-negative example mining technique simultaneously reduces the large rate of false detection due to the original model’s complex background and occlusion influences.
[0048] The detection and tracking of moving vehicles in real time are one of the most challenging aspects of computer vision. Currently, available algorithms can detect vehicles, but they’re slow and inaccurate. For surveillance in traffic scenarios, high-precision vehicle detection algorithms are necessary. This improves the surveillance system’s ability to manage traffic since it enables the tracking of every vehicle with more accuracy. Using YOLO (You Only Look Once) V4, an improved algorithm for vehicle detection is proposed that increases vehicle detection speed and accuracy. In the end, videos are used to detect vehicles using a deep learning technique called You Only Look Once (YOLO). The proposed work uses the YOLOv4 model and HNEM by applying the data augmentation technique in real-time, which improves the accuracy rate of occluded vehicles. The advantage of this proposed work is to reduce the vehicle detection time compared with the existing methods and the limitation of this work is to achieve the better detection rate for heavy vehicle in changed climate conditions. Finally, our proposed algorithm shows that the precision, recall, and mAP are 94.6%, 95.3% and 91.05%, respectively. Consequently, the proposed method achieves better results than the other existing methods discussed in the manuscript.
[0049] Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the 5 embodiments shown along with the accompanying drawings but is to be providing the broadest scope consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention and appended claims.

We Claim:

1) A real-time occluded vehicle detection system, the system comprises an electronic device having an algorithm for vehicle detection based on YOLO (You Only Look Once) Version 4 through Convolution Neural Network (CNN) and Hard Negative Example Mining (HNEM) dataset in the training process to improve the accuracy of the vehicle detection.
2) The system as claimed in claim 1, wherein the videos are used to detect vehicles using a deep learning technique called You Only Look Once (YOLO).
3) The system as claimed in claim 1, wherein a plurality of parameters such as accuracy, precision, recognition recall, F1, and mAP have been used to measure the proposed algorithm’s performance.
4) The system as claimed in claim 1, wherein test results indicate good real-time performance and high detection accuracy of the proposed algorithm.
5) The system as claimed in claim 1, wherein the vehicle is detected by applying augmentation policy on the YOLO model and computing the class confidence value of each bounding box and intersection over the union.
6) The system as claimed in claim 1, wherein CSPBlock module uses CSPDarknet53-tiny as the backbone of YOLOv4-tiny, which detects 371 frames to improve the accuracy rate in vehicle detection.
7) The system as claimed in claim 1, wherein the system obtains the state-of-the-art (SOTA) performance on both the MS COCO dataset and PASCAL dataset using YOLOv4 and Hard Negative Example Mining (HNEM).
8) The system as claimed in claim 1, wherein the hard-positive and hard negative detection of objects are to be extracted by a feature vector of the boundary with confidence score thresholding for this detection model.