Description:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
The Patent Rules, 2003
COMPLETE SPECIFICATION
(See sections 10 & rule 13)
1. TITLE OF THE INVENTION
A CLOUD-AI-BASED REMOTELY CONTROLLING UNMANNED AERIAL VEHICLES (UAVs) SYSTEM FOR OBJECT/INCIDENT DETECTION
2. APPLICANT (S)
S. No. NAME NATIONALITY ADDRESS
1 NMICPS Technology Innovation Hub On Autonomous Navigation Foundation IN C/o Indian Institute of Technology Hyderabad, Kandi, Sangareddy, Telangana– 502284, India.
2 Indian Institute Of Technology Hyderabad IN Kandi, Sangareddy, Telangana– 502284, India.
3. PREAMBLE TO THE DESCRIPTION
COMPLETE SPECIFICATION

The following specification particularly describes the invention and the manner in which it is to be performed.

FIELD OF INVENTION:
[001] The present invention relates to the field of unmanned aerial vehicles. The present invention in particular relates to a cloud-AI-based remotely controlling unmanned aerial vehicles (UAVs) system for object/incident detection.
DESCRIPTION OF THE RELATED ART:
[002] Traditional UAV operations were constrained by localized control and limited computing capacities. Operators grappled with challenges related to scalability, real-time data processing, and remote accessibility. Control and data access might be limited to local proximity, restricting remote accessibility. Operations may rely on local networks, hindering operations in remote or disconnected areas. It was clear that a leap forward was needed. The need for cloud-based remote operation of UAVs stems from several critical requirements and emerging challenges in the realm of unmanned aerial operations: Operating multiple UAVs concurrently requires scalable infrastructure, easily facilitated by cloud resources. Cloud-AI-based systems allow for dynamically adjusting resources based on operational needs. Facilitates seamless exchange of information among operators, improving coordination and decision-making. Cloud-AI-based architectures enable quick integration of emerging technologies, fostering innovation and continuous improvement in UAV operations. In essence, the shift towards cloud-AI-based remote operation of UAVs addresses the limitations of traditional systems, offering a more scalable, flexible, and efficient solution that meets the demands of modern aerial operations across various industries and applications.
[003] Reference may be made to the following:
[004] IN Publication No. 202141051539 relates to an unmanned aerial vehicle (UAV) can include more than one camera for capturing image data within a field of view that depends in part upon the location and orientation of the UAV under deep learning technique. The invention also a portion of the image data can be processed and analysis on the UAV to locate objects of interest such as moving objects, people, leaving things or cars, and use that information to determine where to fly the drone in order to capture higher quality image data of those or other such objects. The invented technology also identified the objects of interest can be counted and the density, movement, location, and behavior of those objects identified and also this can help to determine occurrences such as traffic congestion or unusual patterns of pedestrian movement, as well as to locate persons, fires, or other such objects. The invention also be analyzed by a remote system or service that has additional resources to provide more accurate results approx. 98%.
[005] IN Publication No. 202141027839 relates to a space ensuring drones based IoT system using Machine Learning interfaces. The present invention includes a drone mounted with a plurality of sensors for quantifying and inspection of the predetermined conditions in a predefined place; a processing unit provided with a machine learning interface for processing the received input from the plurality of sensors; a user device with a display unit and the alarming unit to be activated while any panic situation is measured and evaluated by the processing unit in conjunction with the plurality of sensors.
[006] Publication No. WO2023137285 relates to techniques for flight path optimization include obtaining map information defining mapping coordinates of a geographical region, obtaining historical UAV-obtained flight information related to a first autonomous flight of a UAV in the geographical region based on a first flight plan, storing the map information, the historical UAV obtained-flight information, the first flight plan, and current values of a set of external factors separate from the flight information, and generating a second flight plan for a second autonomous flight of the UAV. The generating may include processing the map information, the historical UAV-obtained flight information, and the current values of the set of external factors and updating the first flight plan based on the processing to generate the second flight plan. The second flight plan may be transmitted to the UAV for a second autonomous flight of the UAV in the geographical region.
[007] Publication No. KR102260094 relates to a cloud-based big data processing system using a UAV network including a plurality of UE, at least one UAV and a cloud server, which obtains an environment variable for a change of an atmospheric channel, calculates a target input data rate or data flow based on the obtained variable to allow a user terminal to receive a communication packet, transmitted through a low-altitude platform and a high-altitude platform, within specified time in a given communication environment, and calculates a target worker node number within specified time even though additional time occurs due to an error in a process of processing a big data task requested by the user terminal based on the calculated input data rate or data flow to guarantee performance of a system even in a streaming environment requiring real-time processing about big data.
[008] Publication No. US2016246297 relates to a cloud-based system for controlling the use of unmanned aerial vehicles (UAVs) is used as the communication path between a pilot and his/her UAV, eliminating the direct communication between pilot and vehicle. The cloud-based UAV control system is configured to include both “control apps” associated with the actual flight of a UAV and “mission-specific apps” that include a set of instructions for a specific mission (i.e., performing energy audit of an industrial complex). The control apps preferably include flight regulations (as provided by the FAA, for example) that are used define “no-fly zones”. Other legitimate government (or non-government) agencies may provide “electric fence” control apps to the cloud-based system, thus preventing UAVs from entering protected areas. The UAVs interacting with the control system are intelligent, able to receive specific mission-based applications from the control system, allowing the UAVs to collect a wide variety of useful information.
[009] Publication No. US2021088337 relates to an internet protocol-based unmanned aerial vehicle (UAV) management system and method is disclosed that includes UAVs including a communication chip, a user computer, a cloud-based service for performing a virtual UAV. The method includes storing a plurality of pre-planned missions, controlling communication between the UAV and the user computer, mapping the UAV to the virtual UAV, assigning a mission to include multiple waypoints, and allocating the task at the multiple waypoints. Dynamic mission planning of the assigned mission is performed to generate planned paths for performing the task. Operation of the UAV is controlled by way of the corresponding virtual UAV including receiving messages and commands for the mission from the user computer, sending control commands to the UAV, receiving data signals from the UAV, and transmitting location and status of performance of the task for the UAV. The cloud-based service performs image processing using the received data signals.
[010] Publication No. CN110568854 relates to a UAV integrated management system based on big data fusion, and the system comprises a cooperative UAV management and control module, a non-cooperative UAV management and control module, and a front-end service module, wherein the cooperative UAV management and control module and the non-cooperative UAV management and control module are respectively connected to the front-end service module, and the front-end service module is connected to a cloud database. The UAV integrated management system based on big data fusion provided by the invention has system functions such as UAV registration and filing management, UAV flight data management, user integrated operation management, business data management, civil aviation domain and weather information of traditional drone management systems, and provides a strong guarantee for low-altitude safety through the mutual data fusion of the cooperative UAV management and control system and the non-cooperative UAV management and control system.
[011] Publication No. CN110595476 relates to a UAV landing navigation method and device based on GPS and image vision fusion, and the method comprises the following steps of placing a ground end horizontally on the ground, and using a wireless link to feed the azimuth angle omega and position coordinates G0 of the ground end back to an airborne end; forming a point cloud by recording the GPS value at the ground end in a recent certain time interval, fitting the contour of the point cloud with a circular curve, and noting that the coordinate value of the center of the fitted circle is G0 and the fit radius is R. The UAV landing navigation method and device based on GPS and image vision fusion provided by the invention create and realize a signal source-level image visual signal and GPS signal fusion, directly output a "GPS-like signal" which is a fusion of GPS beacon values, have stronger module portability, versatility and universality, achieve double calibration and complementary correction with the help of perspective transformation of image identification itself and feedback signals from several inclination sensors, and greatly improve the accuracy of navigation.
[012] Publication No. KR102015388 relates to a system and method for providing a virtual reality space map based on orthophotograph 3D point cloud DB construction using an unmanned aerial vehicle and a 3D lidar scanner for the ground and, more specifically, to a system and method for providing a virtual reality space map based on orthophotograph 3D point cloud DB construction using an unmanned aerial vehicle and a 3D lidar scanner for the ground, wherein after performing a low altitude aerial survey using an ultra-light unmanned aerial vehicle, an orthophotograph map is produced using 3d information obtained through a mobile mapping device or the 3D lidar scanner for the ground, and the produced orthophotograph map is applied to an inefficient current situation surveying work of a poor environment for repeated survey performed by many existing people, thereby efficiently replacing and saving survey personnel and time and providing a new survey solution.
[013] Publication No. CN105227631 relates to a system and a method for generating network cloud-based supports for area, domestic and international UAV system. The system comprises a higher-level server, multiple lower level serves which directly communicate with the higher level server, and a or more control stations which directly communicates with the lower level servers. Each control station is configured for controlling flight operation of the UAV, acquiring flight information and position information of the UAV, and providing updates of flight information and position information from the control station for the higher level server.
[014] Publication No. US2020101974 relates to a device and a method that allow selection of an optimal travel route by creating a route considering a real-time driving environment in an autonomous driving vehicle are disclosed. The method includes receiving at least one of surrounding situation information or road situation information from a cloud; calculating a score about each of a plurality of lanes of a road based on the at least one of the surrounding situation information or the road situation information; and configuring a travel route based on the calculated scores. The device and method may be associated with an AI (artificial intelligence) device, a drone, an UAV (unmanned aerial vehicle), a robot, an AR (augmented reality) device, a VR (virtual reality) device, and a 5G service.
[015] Publication No. US2017045884 relates to a system and method for drone connectivity using cellular networks for out of line-of-sight applications and tracking drones during flight. The present invention may be modular, allowing users to program customizable modes for one or more microprocessors of the system. In one embodiment, a system comprises a drone configured with a processor, a cellular modem and a flight controller, the processor connected to the cellular modem and connected to the flight controller via serial connections; a cloud server; a remote control device, and wherein the system is configured for communication from the remote control device to the cloud server via internet communication, and from the cloud server to the drone via a cellular network in communication with the cloud server and the cellular modem, the system providing a communication network for the delivery and routing of data between the remote control device and the drone.
[016] Publication No. US2023288923 relates to a remote drone control system includes a pilot endpoint system comprising a pilot endpoint and a controller connected to the pilot endpoint. The remote drone control system includes a control endpoint system including a control endpoint, a signal adaptor connected to the control endpoint, and a transmitter connected to the signal adaptor. A drone is arranged to communicate with the transmitter to receive and send drone operating data to the control endpoint system. The drone is also arranged to communicate drone video data to the control endpoint system. A remote bridge including a server is arranged to connect the pilot endpoint and the control endpoint such that data is communicated amongst the pilot endpoint, control endpoint, and drone in real-time.
[017] Publication No. EP3477618 relates to a virtual private drone (VPD) system may include a device manager, one or more stations, one or more drones, and one more computing engines. The device manager keeps a repository of device metadata of all drones and computing engines in each deployment of the drone, and runs as an always-on point of contact on the network, globally reachable by all other drones and computing engines in the system. The station is a control center of the VPD system, and each station obtains its network address during registration with the device manager, and starts to synchronize copies of the device metadata using local communication. The drone includes a flight controller module programmable with a flight plan using a number of navigating points, and/or responding to runtime commands. The computing engine includes hardware and software that perform specialized compute-intensive data processing such as computer vision and artificial intelligence (AI).
[018] The article entitled “Cloud station:” A cloud-based ground control station for drones” by Lyuyang Hu; Omkar Pathak; Zeyu He; Hunkyu Lee; Mina Bedwany; Jace Mica; Peter J. Burke; IEEE Journal on Miniaturization for Air and Space Systems (Volume: 2,); 1 March 2021 talks about the open source cloud-based ground control station that allows remote piloting of drones (aerial-, land-, or sea-based vehicle) from anywhere in the world with just a Web browser. A standard client-server architecture is used for pilot-server and drone-server communication and it is designed to be scalable for the control of millions of drones simultaneously. CloudStation, a proof of concept Web app, is built with Django and hosted on a cloud-based service provider with global availability and scalability. A demonstration from two antipode points (separated around the world) shows the cloud-based command and control of a remote rover.
[019] The article entitled “An innovative cloud-based supervision system for the integration of rpas in urban environments” by Elisa Capello, Matteo Dentis, Giorgio Guglieri, Laura NovaroMascarello, Luca Spanó Cuomo, Elsevier, February 2018 talks about the cloud-based supervision system for remotely piloted aircraft systems (RPAS), which are operating in urban environments. The novelty of this proposed concept is dual: a Cloud-based supervision system focusing on safety and robustness, the definition of technical requirements allowing the RPAS to fly over urban areas, as a possible evolution of drone use in future smart cities. A new concept for the regulatory issues is also proposed, compared with existing worldwide regulations. The Cloud framework is intended to be an automated system for path planning and control of RPAS flying under its coverage, and not limited to conventional remote control as if supervised by a human pilot. Future works will be based on the experimental validation of the proposed concept in an urban area of Turin (Italy).
[020] Traditional UAV operations, when compared to cloud-AI-based UAV operations, might have several drawbacks: Operating multiple UAVs simultaneously can be challenging due to hardware and infrastructure limitations. Onboarding computing capabilities may be limited, restricting real-time and sophisticated data processing. The operation’s success heavily relies on local computing, power and storage, limiting flexibility and accessibility. Retrieving and analysing data from UAVs typically requires physical access or direct download, hindering real-time decision-making. Limited remote access and data sharing capabilities can hinder collaboration among multiple users or teams.
[021] In order to overcome above listed prior art, the present invention aims to provide cloud-AI-based system and method for remote piloting of UAVs in emergency conditions. The system involves using a cloud infrastructure to control and process the multi sensor data for a quick response of UAVs from a remote location.
OBJECTS OF THE INVENTION:
[022] The principal object of the present invention is to provide a cloud-AI-based system and method for remote piloting of drones involves using a cloud infrastructure to control and perform the UAV operations from a remote location.
[023] Another object of the present invention is to provide cloud-based system and method with improved accuracy and speed of object identification and obstacle avoidance.
[024] Still another object of the present invention is to provide a system and method for cloud-based operations facilitate the aggregation of data from multiple UAVs or sources.
[025] Yet another object of the present invention is to provide a system and method that enables collaborative obstacle avoidance and object identification, improving overall system efficiency and accuracy.
[026] Still another object of the present invention is to prioritize robust disaster recovery and redundancy for enhanced resilience.
SUMMARY OF THE INVENTION:
[027] The present invention relates to a cloud-based system and method for remote piloting of drones involves using a cloud infrastructure to control drones from a remote location.
[028] The invention provides cloud based waypoint navigation for UAVs that leverages the power of cloud computing to enhance navigation capabilities. It utilizes cloud resources to calculate optimal flight paths based on waypoints in real-time, considering factors like obstacles, weather, and mission objectives. The visual data (images or video feeds) captured by the UAV's onboard cameras are transmitted to the cloud in real-time or periodically for processing. The methods within the cloud infrastructure identify and isolate human subjects within the imagery or video streams and upon identification, the system might trigger predefined actions, such as sending alerts or notifications to operators.
[029] Thus, the UAVs can operate from anywhere with an internet connection. Cloud platforms facilitates real-time data sharing and collaboration among multiple users or teams. Sophisticated data processing, including real-time analysis, Artificial intelligence (AI)/ machine learning (ML) powered insights, and large scale data storage can be enabled through cloud systems. The system providers offer robust security measures, including encryption and access controls, ensuring data protection and controlled access. Cloud-based architectures prioritize robust disaster recovery and redundancy for enhanced resilience.
BREIF DESCRIPTION OF THE INVENTION
[030] It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered for limiting of its scope, for the invention may admit to other equally effective embodiments.
[031] Fig.1 shows working mechanism of cloud based remotely controlling UAV.
[032] Fig.2 shows block diagram of cloud based remotely operating a UAV.
[033] Fig.3 shows communication between user, server and UAV.
DETAILED DESCRIPTION OF THE INVENTION:
[034] The present invention provides a cloud-based system and method for remote piloting of drones involves using a cloud infrastructure to control drones from a remote location. The drone is equipped with various sensors, GPS, cameras and 4G wireless communication module. These components capture the data and enable communication with the cloud. The cloud serves as a central hub where the data from drones is processed, stored, and managed. It hosts the software applications and algorithms needed for drone control and data analysis. Drones use wireless communication technologies (such as cellular networks or dedicated communication protocols) to transmit real-time data to the cloud and receive commands from the remote pilot. The remote pilot interacts with a user interface provided by the cloud-based method. This interface allows the pilot to monitor the drone's telemetry data (like altitude, speed, battery status) and control its movements, camera angles, and other functionalities. The cloud performs real-time processing of data received from drones. This can include image recognition, object detection, terrain mapping, or other analytical tasks depending on the drone's purpose. The processed information is often made available to the pilot in a user-friendly format. The working mechanism for the proposed invention is shown in Fig. 1. The system comprises flight controller (PX4) (1), Raspberry Pi (2), camera module (3) and wireless communication module (4). The PX4 flight controller (1) and Raspberry Pi (2) are integrated, with the companion computer handling the onboard camera modules (3). An autonomous navigation method for waypoint-based flight is pre-installed in the companion computer, ensuring safe UAV navigation. Using a wireless communication module (4), users can interact with the UAV via a cloud platform. Data processing in the cloud enables person detection from the onboard camera feed (3).
[035] Cloud computing for enabling sophisticated take-off, navigation to desired locations, and way point based operations for unmanned aerial vehicles (UAVs). Cloud-connected systems can remotely initiate and monitor the take-off sequence of UAVs. This includes pre-flight checks, ensuring proper functionality of sensors and systems before take-off. The UAV integrated with sensors, companion computer, and flight controller. Based on the sensors data, onboard companion computer processes the data and sent to the flight controller. The flight control provides the information about basic operations like pre-flight checks, functionality of sensors etc. With the help of internet connectivity, the user get know the system behavior. During flight, cloud-AI-based navigation systems can continuously update the UAV's route in response to real-time data. With the help of GPS sensor data and internet connectivity the position of UAV can continuously tracked. Cloud computing allows for the efficient management of waypoints. UAVs receive updated waypoint instructions in real-time, enabling dynamic adjustments to the flight path based on mission objectives or changing circumstances. Cloud computing enables the storage and analysis of flight data, including telemetry, sensor data, and mission logs. This data is used for performance evaluation, improving future missions, and refining navigation method. For every flight the mission data is stored on the onboard flight controller. After every mission, the data was transferred to edge server and saved for future mission as well the analysis on previous mission flights (Fig 2).
[036] Thus the drones are operated from anywhere with an internet connection, enabling applications in various industries without geographical constraints. Utilizing cloud computing resources allows for complex data analysis, which is challenging to perform on the drone itself due to hardware limitations. By utilizing cloud resources, intensive computational tasks related to object identification and obstacle avoidance can be handled remotely, freeing up the onboard processing power of the UAV. This allows for more real-time decision-making. Access to cloud computing enables the use of more sophisticated algorithms and models, which might be resource-intensive for onboard systems. This improves the accuracy and speed of object identification and obstacle avoidance. Cloud-based operations facilitate the aggregation of data from multiple UAVs or sources. This enables collaborative obstacle avoidance and object identification, improving overall system efficiency and accuracy.
[037] Cloud-based systems allow for continuous learning and updating of methods based on new data and insights gathered from various UAV operations. This leads to improved performance and adaptability over time.
[038] The infrastructure can easily scale up or down based on the demand, allowing for flexibility in managing multiple drones or handling peak loads.
[039] All data and control mechanisms can be centralized, making it easier to manage and coordinate fleets of drones.
[040] The multi-sensor data collected and fused on the edge platform followed by processing the data with the help of AI-method to detect the object or incident detection. Collecting and processing the multi sensor data on edge server takes the lesser time and will be useful to UAV for a quicker response. The UAV integrated with 4G/5G dongle through which it can connected to the internet. The edge server connected to the UAV through internet to transfer the data. Data fusion and processing taken place at edge platform and based on the AI-based algorithm instructions sent to UAV through the internet (fig 3). The communication happens through the internet.
[041] Numerous modifications and adaptations of the system of the present invention will be apparent to those skilled in the art, and thus it is intended by the appended claims to cover all such modifications and adaptations which fall within the true spirit and scope of this invention.
, Claims:WE CLAIM:
1. A cloud-based system and method for remote piloting of drones involves using a cloud infrastructure to control drones from a remote location comprises-
a) An unmanned aerial vehicle (UAV) characterized in that flight controller (PX4) (1), raspberry Pi (2), camera module (3) and wireless communication module (4) so that users can interact with the UAV via a cloud platform wherein the PX4 flight controller (1) and Raspberry Pi (2) are integrated, with the companion computer handling the onboard camera modules (3) and an autonomous navigation method for waypoint-based flight is pre-installed in the companion computer, ensuring safe UAV navigation. Using a wireless communication module (4),
b) a cloud-based control system for interfacing between a piloting device and the UAV using wireless communication module (4), the cloud-based control system receives commands from the piloting device and transmits commands and applications to the UAV processor.
2. The cloud-based system and method for remote piloting of drones, as claimed in claim 1, wherein the data processing in the cloud enables person detection from the onboard camera feed (3).
3. The cloud-based system and method for remote piloting of drones, as claimed in claim 1, wherein during flight, cloud-based navigation systems are continuously update the UAV's route in response to real-time data. Cloud computing allows for the efficient management of waypoints, UAVs receives updated waypoint instructions in real-time, enables dynamic adjustments to the flight path based on mission objectives or changing circumstances.
4. The cloud-based system and method for remote piloting of drones, as claimed in claim 1, wherein the system enables the storage and analysis of flight data, including telemetry, sensor data, and mission logs which is used for performance evaluation, improving future missions, and refining navigation method.
5. The cloud-based system and method for remote piloting of drones, as claimed in claim 1, wherein the UAV is integrated with 4G/5G dongle through which it is connected to the internet and the edge server connected to the UAV through internet to transfer the data.
6. The cloud-based system and method for remote piloting of drones, as claimed in claim 1, wherein the data fusion and processing taken place at edge platform and based on the artificial intelligence (AI) based communication with UAV through the internet.