DESC:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
The Patent Rules, 2003
COMPLETE SPECIFICATION
(See sections 10 & rule 13)
1. TITLE OF THE INVENTION
A ROBOT SYSTEM AND METHOD FOR ROAD OBSTRUCTION DETECTION AND DATA COLLECTION
2. APPLICANT (S)
NAME NATIONALITY ADDRESS
DIVYASAMPARK IHUB ROORKEE FOR DEVICES MATERIALS AND TECHNOLOGY FOUNDATION IN Indian Institute of Technology Roorkee, Roorkee-247667, Uttarakhand, India.
3. PREAMBLE TO THE DESCRIPTION
COMPLETE SPECIFICATION

The following specification particularly describes the invention and the manner in which it is to be performed.

FIELD OF INVENTION:
[001] The present invention relates to the field of obstruction detection systems. The present invention in particular relates to a robot system and method for road obstruction detection & data collection.
DESCRIPTION OF THE RELATED ART:
[002] Robotic or autonomous vehicles (sometimes referred to as mobile robotic platforms) generally have a robotic control system that controls the operational systems of the vehicle. In a vehicle that is limited to a transportation function, the operational systems may include steering, braking, transmission, and throttle systems. In October 2005, five autonomous vehicles (of twenty-three finalist vehicles) successfully completed the “Grand Challenge” of the United States Defense Advanced Research Projects Administration (DARPA), a competition requiring fully robotic vehicles to traverse a course covering more than one hundred miles. These vehicles were outfitted with robotic control systems in which a bank of computers controlled the operational systems of the vehicle, such as the steering, braking, transmission, and throttle, subject to autonomous decisions made by programs on board the vehicle in response to sensor input, without human intervention on the course itself.
[003] Robotic control system sensor inputs may include data associated with the vehicle's destination, preprogrammed path information, and detected obstacle information. Based on one or more of data associated with the information above, the vehicle's movements are controlled. Detecting obstacles, however, may provide false data points, incomplete obstacle data, and/or require large amounts of electronic storage capacity and processing power to compute the relevant data points. Additionally, combining obstacle data with trajectory or directional information may result in vehicle movements that may be characterized as jerky or incorrect vehicle movements.
[004] Reference may be made to the following:
[005] Publication No.US 20240416839 relates to the systems and methods for detecting road obstructions in blind spots when a vehicle traverses a road curve.
[006] Publication No. US8020657B2 relates to the systems and methods for obstacle avoidance. In some embodiments, a robotically controlled vehicle capable of operating in one or more modes may be provided. Examples of such modes include teleoperation, waypoint navigation, follow, and manual mode. The vehicle may include an obstacle detection and avoidance system capable of being implemented with one or more of the vehicle modes. A control system may be provided to operate and control the vehicle in the one or more modes. The control system may include a robotic control unit and a vehicle control unit.
[007] Publication No. IN201821047908 relates to method to prevent road accidents in foggy environments the method comprising: determining visibility index in a frontal area of a vehicle; activating object identifier unit to detect presence of any object within a predetermined distance from the vehicle, when the visibility index is lesser than a threshold visibility index; determining co-ordinates of each of the objects within the frontal area; estimating first operational area being a part of the frontal area; sending instructions to at least one flying object that operates within the first operational area, wherein the flying object includes at least one image capturing device to capture images of the objects found within the first operational area; receiving and rendering the plurality of images to display device installed within the vehicle.
[008] Publication No. IN3470/KOLNP/2009 relates to a method and system that allows a user to perform automatic study, layout and verification of a multidimensional space in real lime where the study can be displayed graphically, in 3 -dimensions for example, via a handheld unit allowing the system to guide and/or navigate the user throughout the multidimensional space as the automatic study and/or layout is being performed.
[009] Publication No. KR20200027885 relates to a method and an apparatus for generating a CNN training image data set to detect an obstacle in an autonomous driving situation, and a test method and a test apparatus using the same. The method for generating a CNN training image data set to detect an obstacle in an autonomous driving situation comprises the steps of: obtaining, by a learning apparatus, an original image representing a road driving situation, and a synthesized label generated by using an original label corresponding to the original image and an additional label corresponding to an image of an arbitrary specific object not corresponding to the original image; and allowing, by the learning apparatus, a first CNN module to generate a synthesized image by using the original image and the synthesized label, wherein the synthesized image is generated by synthesizing the original image and the image of the arbitrary specific object corresponding to the additional label.
[010] Publication No. EP1157913 relates to a system for alerting a controller of a track-led vehicle of the presence of an obstacle in a track of the vehicle. At least one sensor including a video camera is mounted on the vehicle such that its orientation is automatically adjusted relative to the vehicle for sensing a field of view of the track in front of the vehicle so as to produce successive video images thereof each representative of a respective section of track ahead of the vehicle, and an obstacle detection device coupled to the video camera processes successive video images produced thereby so as to produce an obstacle detect signal consequent thereto.
[011] Patent No. US8020657 relates to systems and methods for obstacle avoidance. A robotically controlled vehicle capable of operating in one or more modes may be provided. Examples of such modes include teleoperation, waypoint navigation, follow, and manual mode. The vehicle may include an obstacle detection and avoidance system capable of being implemented with one or more of the vehicle modes. A control system may be provided to operate and control the vehicle in the one or more modes. The control system may include a robotic control unit and a vehicle control unit.
[012] Patent No. US11119192 relates to methods, apparatuses, and systems for detecting overhead obstructions along a path segment. One exemplary method includes receiving three-dimensional data collected by a depth sensing device traveling along a path segment, wherein the three-dimensional data comprises point cloud data positioned above a ground plane of the path segment. The method further includes identifying data points of the point cloud data positioned within a corridor positioned above the ground plane.
[013] Publication No. EP3852083 relates to a system and a method for a vehicle for obstacle detection and avoidance on roads or highways. The inventive system comprises a camera system with at least one forward-looking camera and/or side looking camera, at least one LiDAR-device and a communication device for communication between vehicles. A data evaluation device is communicatively connected to the camera system, the LiDAR-device and the communication device.
[014] Publication No. JPH06297379 relates to an over travel detecting device which is installed on the angular rotary shaft of an industrial robot and can mechanically set and detect the turning limit position of the rotary shaft. An over travel detecting device is equipped with a shaped plate member which is installed on the rotary shaft of the wrist part, etc., of a robot machine body, turns according to the turn of the rotary shaft, and sets the turn limit position at a prescribed position from the turn original point position, and a detection element consisting of a limit switch, etc., which generates an over travel signal in the contact with the obstruction walls for setting the turning limit position which are installed at the road edge of the groove passage of the shaping plate member.
[015] Publication No. US2020225673 relates to a method including capturing, by an image sensor disposed on a robot, images of a workspace; obtaining, by a processor of the robot or via the cloud, the captured images; comparing, by the processor of the robot or via the cloud, at least one object from the captured images to objects in an object dictionary; identifying, by the processor of the robot or via the cloud, a class to which the at least one object belongs using an object classification unit; and instructing, by the processor of the robot, the robot to execute at least one action based on the object class identified.
[016] Publication No. WO2016189777 relates to a road surface marking detection device, comprising a road surface marking detection unit which: segments an edge image, which is generated on the basis of a captured image of a road, into left and right blocks at the approximate center of the edge image; extracts, from prescribed pixel rows of the edge image, the pixel rows in which the distances between the pixels which are nearest to the approximate center in both the left and right blocks of the pixel rows are less than or equal to a prescribed value; and determines whether a road surface marking is present on the basis of the percentage of the number of extracted pixel rows with respect to the number of the prescribed pixel rows.
[017] Publication No. US2016009276 relates to a driver warning system automatically detects the existence of in-the-road, passable obstructions and provides timely notification to a driver in proximity of such an in-the-road, passable obstruction prior to it being encountered with sufficient warning time for the driver to avoid the in-the-road, passable obstruction. To detect in-the-road, passable obstructions, the warning unit of a driver whose vehicle hits an in-the-road, passable obstruction will automatically generate a signal that is indicative of the existence of the in-the-road, passable obstruction and its location.
[018] Publication No. JP2007235642 relates to an erroneous detection of an obstruction caused by a detection error of a vehicle speed sensor or a rudder sensor and to detect a cubic object by a single-lens camera with a high accuracy. This system creates a top view of a first image containing a road surface imaged by an on-board camera, and a top view of a second image captured at a different timing from the first image.
[019] Publication No. JP2007223395 relates to an obstruction detection device of a vehicle capable of preventing a malfunction of a brake working means and a seat belt pretensioner and improving the safety of an occupant. This obstruction detection device of the vehicle is provided with a radar device to detect an obstruction in front of the self vehicle and a working equipment control means to control the working equipment of an own vehicle by receiving obstruction detection information from the radar device.
[020] Publication No. JPH09178855 relates to reduce wrong recognition by transforming the position information of an obstacle detected by a radar into the image coordinates of a camera for recognizing a lane, and comparing it with the position information of the lane to judge whether the obstacle is on the same lane as its own vehicle or not. An obstacle detecting means detects an obstacle on the basis of the data from a radar device, and outputs, from the distance and angle between its own vehicle and the obstruction, the position data of the obstacle in a road coordinate system having the vehicle position as origin. A coordinate transformation means receives and transformation it into the image coordinates of an image pickup means.
[021] Publication No. JPH04148883 relates to always correctly detect an obstruction independent of the magnitude of the curvature of the road by setting up at least two antennae for receiving reflected waves from an obstruction at a prescribed interval in the front of a moving body.
[022] Publication No. JPH03135815 relates to detect an obstruction on a road surface accurately by sending an ultrasonic wave in pulse toward the road surface and adjusting an amplifying rate of the input signal on the basis of the mean value for reflecting the input signal level of the reflected wave thereof. An ultrasonic wave sending means 3 is driven by the ultrasonic signal V1 generated from an ultrasonic signal generating means on the basis of the output of a timing command means 1 to send the ultrasonic wave against a road surface intermittently.
[023] Publication No. JPH04118360 relates to detect such an obstacle as a vehicle or the like on a crossing road without fail by detecting the obstacle on the crossing road when an image of laser beams projected on an imaging surface of a video camera has come to nonlinear beam due to an interruption of the laser beams.
[024] Publication No. JPH06328989 relates to detect a forward obstruction and secure the safe traveling by successively detecting a preceding vehicle as an obstruction until the specified period of time is elapsed when the preceding vehicle detected as an obstruction in the range of the advancing road ahead moves to the range of the reserve road. When a vehicle is traveling, the pulse laser beam is emitted forward from an emitting part of a radar head unit in a radar equipment, an obstruction which is present in the road ahead is scanned, and the distance and the angle relative to the obstruction are received by an advancing road/ reserve road setting part.
[025] Publication No. JPH06305383 relates to detect an obstruction even at a curved road and the like by computing a displacement quantity for moving a light projecting lens in either of the right and left directions based on an output signal from a steering angle sensor, and feeding a control signal corresponding to the computed displacement quantity to a driving means.
[026] Publication No. JPH03217315 relates to reliably detect an obstruction on a road surface by transmitting ultrasonic waves in a pulse form manner toward a road surface located obliquely in front and varying the amplification factor of a receiving signal based on an average value at which the receiving signal level of a reflection wave is reflected. Based on a command signal P1 from a timing command means 1, an ultrasonic transmitting means is driven through an ultrasonic signal generating means to intermittently transmit ultrasonic waves Wa-Wc to a road surface located obliquely in front. Reflection waves Wa'-Wc' reflected by the road surface and an obstruction are received by an ultrasonic receiving means.
[027] Publication No. JPH04240584 relates to highly exactly detect an obstruction in at least three directions of the front, the road surface and the rear with the use of a radar. A radar is provided near the center of rotation of a front wheel of a car and a beam direction is varied in accordance with steering while its beam is rotated together with the front wheel.
[028] Publication No. DE19856823 relates to two sensors having differing positional resolution, e.g. a laser radar and a millimeter-wave radar are mounted on the front of the vehicle and are used in combination to detect obstacles on the road surface. If one of the sensors fails, the system operates an alarm or the brakes based on the output of the other sensor. If the laser radar fails, the alarm threshold is increased and the braking threshold is reduced.
[029] Patent No. US9639811 relates to a method for identifying a simulated social media account history. The method may include querying a social media identification information (“social media ID”) to determine whether the account history includes one or more parameters that indicate whether the social media ID is related to an automated entity or a human entity.
[030] Patent No. US10491120 relates to a system that includes a regulator unit. The regulator unit includes first and second phase units whose outputs are coupled to through first and second coupled inductors, respectively, to a power supply node of a circuit block. The first phase unit may be configured to discharge, for a first period of time, the power supply node through the first inductor in response to determining a sense current is greater than a demand current. The operation of the second phase unit may follow that of the first phase unit after a second period of time has elapsed.
[031] Patent No. US10786197 relates to an evaluation method of evaluating a color irregularity site (a blemish) includes a color irregularity site detection step (step S102) of detecting a plurality of color irregularity sites respectively from a first skin image and a second skin image different from the first skin image, a gravity center position calculation step (step S108) of calculating gravity center positional coordinates of the color irregularity sites respectively for the first skin image and the second skin image, and a matching process step (step S110) of matching the plurality of color irregularity sites included in the first skin image with the plurality of color irregularity sites included in the second skin image based on the calculated gravity center positional coordinates of the color irregularity sites.
[032] Patent No. US10134105 relates to a medical signal processing device processes an image signal received in accordance with a result of examining inside of a subject. The image signal includes pixel data groups that are data of respective pixels different from each other and received by the medical signal processing device in parallel.
[033] The article entitled “An obstacle detection and distance measurement method for sloped roads based on VIDAR” by Guoxin Jiang, Yi Xu, Xiaotong Gong, Shanshang Gao, Xiaoqing Sang, Ruoyu Zhu, Liming Wang, Yuqiong Wang; Journal of Robotics; 15 April 2022 talks about the environmental perception systems can provide information on the environment around a vehicle, which is key to active vehicle safety systems. However, these systems underperform in cases of sloped roads. Real-time obstacle detection using monocular vision is a challenging problem in this situation. In this study, an obstacle detection and distance measurement method for sloped roads based on Vision-IMU based detection and range method (VIDAR) is proposed. First, the road images are collected and processed. Then, the road distance and slope information provided by a digital map is input into the VIDAR to detect and eliminate false obstacles (i.e., those for which no height can be calculated). The movement state of the obstacle is determined by tracking its lowest point. Finally, experimental analysis is carried out through simulation and real-vehicle experiments. The results show that the proposed method has higher detection accuracy than YOLO v5s in a sloped road environment and is not susceptible to interference from false obstacles. The most prominent contribution of this research work is to describe a sloped road obstacle detection method, which is capable of detecting all types of obstacles without prior knowledge to meet the needs of real-time and accurate detection of slope road obstacles.
[034] The article entitled “Automated road defect and anomaly detection for traffic safety: a systematic review” by Munish Rathee, Boris Bacic, Maryam Doborjeh; Sensors, 23(12), 5656; 21 March 2023 talks about the development of sensor technology. As enabling factors, computer vision (CV) combined with sensor technology have made progress in applications intended to mitigate high rates of fatalities and the costs of traffic-related injuries. Although past surveys and applications of CV have focused on subareas of road hazards, there is yet to be one comprehensive and evidence-based systematic review that investigates CV applications for Automated Road Defect and Anomaly Detection (ARDAD). To present ARDAD’s state-of-the-art, this systematic review is focused on determining the research gaps, challenges, and future implications from selected papers (N = 116) between 2000 and 2023, relying primarily on Scopus and Litmaps services. The survey presents a selection of artefacts, including the most popular open-access datasets (D = 18), research and technology trends that with reported performance can help accelerate the application of rapidly advancing sensor technology in ARDAD and CV. The produced survey artefacts can assist the scientific community in further improving traffic conditions and safety.
[035] The article entitled “Robust obstacle detection for advanced driver assistance systems using distortions of inverse perspective mapping of a monocular camera” by Charan D. Prakash, Farshad Akhbari, Lina J. Karam; Robotics and Autonomous Systems Volume 114, Pages 172-186; 2 March 2018 talks about novel proposal assessment method based on fusing statistical properties from both the IPM image and the camera image to perform robust outlier elimination and false positive reduction. We also present an annotated obstacle detection dataset derived from various source videos that can serve as a benchmark for the evaluation of future obstacle detection algorithms. The source videos containing diverse illumination and traffic conditions are derived from multiple publicly available datasets.
[036] The article entitled “Proposal of a method for obstacle detection by the use of camera and line laser” by Miki Suetsugu; Kanako Kinoshita; Akihiro Ito; Shiyuan Yang; Seiichi Serikawa; ACIT '19: Proceedings of the 7th ACIS International Conference on Applied Computing and Information Technology; 29 May 2019 talks about a method for obstacle detection using a line laser and a camera. With this method, an obstacle is detected in a wide range. It is possible to measure the size of and the distance to obstacle. In addition, it has the following features compared with the conventional methods. (1) Processing is simpler than camera processing. (2) It is not affected by variations in illumination and sound noise. (3) As the work before a robot starts, only setting the irradiation angle of laser and the distance between laser and robot. Therefore, the preliminary work can be remarkably reduced.
[037] The article entitled “Fast obstacle detection based on multi-sensor information fusion” by Lu, Linli; Ying, Jie; Proceedings Volume 9301, International Symposium on Optoelectronic Technology and Application 2014: Image Processing and Pattern Recognition; November 2014 talks about a method to realize the real-time access to the information of the obstacle in front of the robot and calculating the real size of the obstacle area according to the mechanism of the triangle similarity in process of imaging by fusing datum from a camera and an ultrasonic sensor, which supports the local path planning decision. In the part of image analyzing, the obstacle detection region is limited according to complementary principle. We chose ultrasonic detection range as the region for obstacle detection when the obstacle is relatively near the robot, and the travelling road area in front of the robot is the region for a relatively-long-distance detection. The obstacle detection algorithm is adapted from a powerful background subtraction algorithm ViBe: Visual Background Extractor. We extracted an obstacle free region in front of the robot in the initial frame, this region provided a reference sample set of gray scale value for obstacle detection. Experiments of detecting different obstacles at different distances respectively, give the accuracy of the obstacle detection and the error percentage between the calculated size and the actual size of the detected obstacle. Experimental results show that the detection scheme can effectively detect obstacles in front of the robot and provide size of the obstacle with relatively high dimensional accuracy.
[038] The article entitled “A compact magnetic field-based obstacle detection and avoidance system for miniature spherical robots” by Fang Wu, Akash Vibhute, Gim Song Soh, Kristin L Wood, Shaohui Foong; Sensors (Basel).;17(6):1231; 2017 May 28 talks about a compact magnetic field-based obstacle detection and avoidance system has been developed for miniature spherical robots. It utilizes a passive magnetic field so that the system is both compact and power efficient. The proposed system can detect not only the presence, but also the approaching direction of a ferromagnetic obstacle, therefore, an intelligent avoidance behavior can be generated by adapting the trajectory tracking method with the detection information. Design optimization is conducted to enhance the obstacle detection performance and detailed avoidance strategies are devised.
[039] The article entitled “A soft robot capable of 2D mobility and self-sensing for obstacle detection and avoidance” by Qin, Lei; Tang, Yucheng; Gupta, Ujjaval; Zhu, Jian; Smart Materials and Structures, Volume 27, Issue 4, April 2018 talks about a soft mobile robot capable of high mobility and self-sensing for obstacle detection and avoidance. This robot, consisting of a dielectric elastomer actuator as the robot body and four electroadhesion actuators as the robot feet, can generate 2D mobility, i.e. translations and turning in a 2D plane, by programming the actuation sequence of the robot body and feet. Furthermore, we develop a self-sensing method which models the robot body as a deformable capacitor. By measuring the real-time capacitance of the robot body, the robot can detect an obstacle when the peak capacitance drops suddenly. This sensing method utilizes the robot body itself instead of external sensors to achieve detection of obstacles, which greatly reduces the weight and complexity of the robot system. The 2D mobility and self-sensing capability ensure the success of obstacle detection and avoidance, which paves the way for the development of lightweight and intelligent soft mobile robots.
[040] In order to overcome above listed prior art, the present invention aims to provide a robot system and method for road obstruction detection & data collection.
OBJECTS OF THE INVENTION:
[041] The principal object of the present invention is to provide a robot system and method for road obstruction detection & data collection.
[042] Another object of the present invention is to provide a robot system which gives early alerts and real-time data, enables proactive road maintenance, and reduces cost.
[043] Yet another object of the present invention is to provide integrates machine learning and drone technology based robot system for real-time pothole detection and monitoring.
SUMMARY OF THE INVENTION:
[044] The present invention provides robot system and method for road obstruction detection & data collection. This integrates machine learning and drone technology for real-time pothole detection and monitoring, offering both technical innovation and commercial potential. The system is, optimized for real-time object detection, is embedded on a lightweight drone equipped with a high-performance camera. The drone can capture images or video of road surfaces, process them locally using the model, and wirelessly transmit classification results to a remote system. The system classifies various pothole types, providing immediate, accurate feedback to users.
[045] By using advanced convolutional neural networks (CNNs), the robot detects potholes accurately in real-time through drone-based systems. It builds upon established drone capabilities by adding live image capture, processing, and classification of potholes using a trained model stored locally. This is a scalable, efficient solution for monitoring road conditions in remote areas, improving road maintenance processes.
BREIF DESCRIPTION OF THE INVENTION
[046] It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered for limiting of its scope, for the invention may admit to other equally effective embodiments.
[047] Figure 1 shows block diagram according to the present invention.
[048] Figure 2 shows flowchart according to the present invention.
DETAILED DESCRIPTION OF THE INVENTION:
[049] The present invention provides a robot system and method for road obstruction detection & data collection. This integrates machine learning and drone technology for real-time pothole detection and monitoring, offering both technical innovation and commercial potential. The system is, optimized for real-time object detection, is embedded on a lightweight drone equipped with a high-performance camera. The drone can capture images or video of road surfaces, process them locally using the model, and wirelessly transmit classification results to a remote system. The system classifies various pothole types, providing immediate, accurate feedback to users.
[050] The system processes images or video streams captured by the drone, enabling on-the-spot classification of potholes based on pre-trained data. The results are displayed wirelessly on a remote PC or system, allowing users to monitor the roads without being on-site.
[051] The invention integrates the system with a network, allowing drones to transmit live data to a centralized server for processing. The server-hosted model will analyze real-time data, making the system robust for large-scale or remote road maintenance operations.
[052] This is a cost-effective, scalable solution for infrastructure monitoring. It can be deployed in various industries, including government road maintenance, construction, and even private sectors like logistics and autonomous vehicle navigation. Its lightweight and regulatory-compliant drone enhances portability and flexibility for commercial operations, while real-time feedback supports proactive road management, reducing the need for manual inspections in hard-to-reach areas.
[053] The method of detecting potholes reduces false-positive and false-negative rates, offering more accurate detection than existing systems. Using edge computing for faster processing speeds ensures real-time analysis and categorization of potholes, which improves operational efficiency. The system's drone operates autonomously with a pre-programmed flight path or real-time path planning, ensuring comprehensive road coverage in hard-to-reach or large areas, unlike many of the referenced systems which may require manual intervention. The system can distinguish between different types of road defects which many existing solutions lack. By providing early alerts and real-time data, it enables proactive road maintenance, and reduces cost.
[054] Figure 1 shows the block diagram. The system comprises a drone (1) equipped with a camera for capturing high-resolution images or videos of roads during flight. This acts as the primary data collection unit for pothole detection. A communication module (2) enables real-time data transmission between the UAV and a ground control station or cloud server. The module ensures low-latency communication for live monitoring and processing. The system incorporates YOLOv8 (You Only Look Once, Version 8), a deep learning-based object detection model (3). The architecture processes incoming image frames to identify and localize potholes on roads with high precision and speed. It uses multiple feature maps (P3, P4, P5) for multi-scale detection. Smart Pothole Detection Output (4) highlights the processed data detected potholes on roads. This can be visualized with bounding boxes and alerts for real-world applications like repair scheduling or road safety improvement.
[055] The terms P2, P3, P4, P5 refer to feature maps at different scales (or resolutions) within the Feature Pyramid Network (FPN) structure used in object detection models like YOLOv8.
Name Resolution (relative to input) Use Case Captures Features of...
P2 1/4 of input size Very small objects Fine details, edges
P3 1/8 of input size Small objects Small potholes, cracks
P4 1/16 of input size Medium objects Medium-sized road damage
P5 1/32 of input size Large objects Large potholes or patches
[056] Potholes and road defects come in various shapes and sizes. Using multiple feature maps like P2–P5:
• The model can accurately detect small cracks or early-stage potholes using P3.
• It can detect wider or deeper potholes using P4 and P5.
• This multi-scale detection improves both recall and precision across different road scenarios.
[057] The smart pothole detection system is a solution integrating advanced technologies for real-time identification and localization of potholes on roads. The system begins with a UAV (Unmanned Aerial Vehicle) equipped with a high-resolution camera to capture aerial images or video footage of roads during its flight. The UAV serves as the primary data acquisition platform, ensuring comprehensive road surface coverage. The captured data is transmitted in real-time using a 4G/5G communication module, which provides low-latency, high-speed data transfer to a ground control station or cloud-based processing unit. This ensures seamless communication and enables real-time analysis of the collected data.
[058] At the core of the system lies the YOLOv8 (You Only Look Once, Version 8) object detection architecture, a state-of-the-art deep learning model designed for high-speed and high-accuracy object detection. YOLOv8 employs convolutional layers to process input images, extract meaningful features, and identify potholes in multiple scales using its feature pyramid network (P3, P4, P5). The model’s ability to predict bounding boxes around detected potholes with high precision ensures robust detection, even in challenging conditions such as varying lighting, shadows, or complex road patterns. Once the data is processed, the system generates a detailed output highlighting detected potholes on the road. The results are visualized with instance segmentation. This information can be further utilized for immediate notifications to relevant authorities, creating repair schedules, or updating a road maintenance database.
[059] The automated nature of the system eliminates the need for manual inspection, improving the efficiency and safety of the monitoring process. The integration of UAVs, advanced communication modules, and deep learning models like YOLOv8 offers significant advantages. The system ensures real-time detection with minimal latency, reduces human effort, and achieves high detection accuracy. By enabling early detection and repair of potholes, this system not only enhances road safety but also contributes to cost-effective and timely infrastructure maintenance.
[060] Figure 2 shows the flowchart. The methodology for real-time pothole detection using a YOLOv8 instance segmentation model and drone technology involves several detailed steps designed to ensure accurate, efficient and actionable results. The process begins with data collection, where images of potholes are gathered from various road conditions. These images can be captured manually using cameras. The flowchart with its working is explained below.
[061] Once the data is collected, it undergoes preprocessing to prepare it for training. This involves labeling images with annotations that include instance segmentation to define the locations and exact shapes of potholes. Annotation tools like LabelImg, makesense.ai or Roboflow are commonly used for this purpose. The dataset is then divided into three subsets: training, validation, and testing. Preprocessing also includes techniques like resizing images and applying augmentations to make the dataset more diverse and improve the model's ability to generalize.
[062] With the dataset ready, the next step is training and validation of the YOLOv8 instance segmentation model. YOLOv8, known for its speed and accuracy, is particularly effective for tasks that combine object detection and segmentation, such as pothole detection. During training, the model learns to identify potholes and delineate their boundaries by using annotated images. Transfer learning, where the model is initialized with pre-trained weights, is often employed to achieve faster convergence and better performance, especially when working with limited datasets. Throughout training, the model’s performance is continuously monitored using metrics like mean Average Precision (mAP), precision, recall, and Intersection over Union (IoU) to ensure it is learning effectively. Based on the results, adjustments such as fine-tuning hyperparameters or adding more data can be made to optimize the model. Our proposed model accuracy is 93%.
[063] Once training is complete, the model undergoes testing on unseen data to evaluate its generalization capabilities. This ensures the model is not overfitted and can perform accurately in real-world scenarios. The testing phase verifies that the model can detect potholes and segment them accurately under conditions different from those in the training dataset.
[064] For deployment, drones equipped with high-resolution cameras are used to capture real-time data of road surfaces. These drones fly over roads, capturing video or images at optimal angles and altitudes to ensure clear and detailed data collection. The captured data is then processed in real time using the trained YOLOv8 instance segmentation model. The model analyzes the input to detect potholes, segment their boundaries, and calculate their dimensions and locations.
[065] Once training is complete, the model’s performance is evaluated using the validation and testing sets. Key metrics like:
• IoU (Intersection over Union)
• Precision and recall
• F1-score
• Inference time are assessed to ensure the model’s robustness in various conditions.
[066] Finally, the output is visualized on a screen, showing annotated images or videos with potholes clearly marked and segmented using a 4G/5G module. This provides immediate and actionable insights, allowing authorities or maintenance teams to identify the exact locations and sizes of potholes for repairs. The use of YOLOv8’s instance segmentation ensures not only that potholes are detected but also that their precise shapes and boundaries are identified, making it an ideal solution for accurate road maintenance planning.
[067] This comprehensive methodology leverages the advanced capabilities of YOLOv8 and drone technology to create an efficient, scalable, and highly accurate system for real-time pothole detection, improving road safety and reducing maintenance delays.
[068] Figure 2 flowchart represents the step-by-step working of a real-time pothole detection system using a drone and a YOLOv8 deep learning model. Let’s go through each block in detail. The process begins by initializing the entire pipeline for building and deploying the pothole detection system. In this phase, a large number of road images or videos are collected. These images include both pothole and non-pothole road conditions. The data may be acquired manually using smartphones, cameras, or UAVs flying over roads. The collected data is cleaned and prepared for use. This involves:
• Annotating potholes using tools like LabelImg or Roboflow.
• Resizing images to a standard resolution.
• Augmenting the dataset (e.g., flipping, rotating, changing brightness) to improve generalization.
• Removing noisy or poor-quality data.
[069] The preprocessed dataset is split into three parts:
• Training set: Used to train the YOLOv8 model.
• Validation set: Used to fine-tune hyperparameters and prevent overfitting.
• Testing set: Used for unbiased evaluation after training.
[070] Here, the YOLOv8 model is trained using the training dataset. During training:
• The model learns to detect potholes by adjusting its weights based on the loss function.
• Validation data is used to monitor the model's accuracy, MAP (mean Average Precision), precision, and recall to ensure effective learning.
[071] Once training is complete, the model’s performance is evaluated using the validation and testing sets. Key metrics like:
• IoU (Intersection over Union)
• Precision and recall
• F1-score
• Inference time are assessed to ensure the model’s robustness in various conditions.
[072] The trained YOLOv8 model is now tested on unseen images (from the test set) to confirm its ability to generalize to real-world conditions. This step ensures the model doesn’t overfit to the training data and is reliable.
[073] In this deployment phase, the trained model is embedded on a drone. The drone flies over the target area and captures real-time images or video streams of the road surface.
[074] The real-time images captured by the drone are passed through the YOLOv8 model. The model detects potholes in each frame and localizes them with bounding boxes or instance segmentation.
[075] The results of the detection are displayed on a remote screen, dashboard, or controller interface. This output includes:
• Detected potholes highlighted in the image.
• Potential severity of the damage.
[076] This visualization helps authorities or users to take immediate action like scheduling repairs.
[077] The process concludes, either by completing a flight mission or after processing a batch of images. The system can loop back to repeat the monitoring over time for continuous assessment.
[078] Numerous modifications and adaptations of the system of the present invention will be apparent to those skilled in the art, and thus it is intended by the appended claims to cover all such modifications and adaptations which fall within the true spirit and scope of this invention.
,CLAIMS:WE CLAIM:
1. A robot system and method for road obstruction detection & data collection comprises-
a) a drone (1) equipped with a camera for capturing high-resolution images or videos of roads during flight which acts as the primary data collection unit for pothole detection.
b) communication module (2) enabling real-time data transmission between the UAV and a ground control station or cloud server ensuring low-latency communication for live monitoring and processing.
c) a deep learning-based object detection model (3) processing incoming image frames to identify and localize potholes on roads with high precision and speed using multiple feature maps (P3, P4, P5) for multi-scale detection.
d) Smart pothole detection output (4) highlights the processed data detected potholes on roads which is visualized with bounding boxes and alerts for real-world applications like repair scheduling or road safety improvement.
2. The robot system for road obstruction detection & data collection, as claimed in claim 1, wherein the system begins with a UAV (Unmanned Aerial Vehicle) equipped with a high-resolution camera to capture aerial images or video footage of roads during its flight which serves as the primary data acquisition platform, ensuring comprehensive road surface coverage and captured data is transmitted in real-time using a 4G/5G communication module, which provides low-latency, high-speed data transfer to a ground control station or cloud-based processing unit ensuring seamless communication and enables real-time analysis of the collected data.
3. The method for road obstruction detection & data collection, as claimed in claim 1, wherein the method includes following steps:
a) The process is initiated to set up the pipeline for pothole detection using drone and AI.
b) Images and videos of roads are collected using cameras or drones, covering both pothole and non-pothole conditions.
c) The collected data is cleaned, annotated, resized, and augmented to prepare it for model training.
d) The dataset is divided into three parts—training, validation, and testing—for systematic model development.
e) The YOLOv8 model is trained using the training set to detect and localize potholes based on visual features.
f) The model’s accuracy is evaluated using validation metrics such as precision, recall, MAP, and IoU.
g) The trained model is tested on new, unseen data to ensure it can generalize well to real-world images.
h) The drone is deployed to fly over roads and capture real-time images or video of road surfaces.
i) Captured images are analyzed using the YOLOv8 model, which detects potholes and marks them with bounding boxes.
j) The detection results are shown on a screen or dashboard, helping maintenance teams act quickly.
k) The process concludes after analysis, or the system can be looped for continuous monitoring.