Description:FIELD OF INVENTION AND USE OF INVENTION

[0001] The present invention is made in the field of Computer Science and Electronics & Communication Engineering more particularly a system to control pests by delivering magnetic pulses via multiple aerial drones.
[0002] Here a system is designed to sense various types of real-time pests by Single Shot Multibox Detector (SSD) algorithm. This system consists of multiple aerial drones where each drone includes a camera, a pest sensor, a gyroscope sensor, a gps sensor, a drone on-board computer, and a magnetron. The pest sensor is an algorithm designed into the drone on-board computer which senses the type of the pest based on the images of the pest collected from the drone camera via detection process, establishes a threat level posed by the presence of the pest based on the pest type via counting process and control other leading drones by measuring their movement and position based on the Particle Swarm Optimization (PSO) algorithm. The pest control mechanism performs a pest control by delivering magnetic pulses towards the pest based on the pest type and the threat level posed by the presence of the pest which mainly includes locust swarm, terrestial pest and also other insects that affects vegetation and crops.

BACKGROUND OF INVENTION

[0003] Pest Control Mechanism have been known in the past basically for controlling various types of pests by using chemical pesticides. But it has been seen that some of these substances may cause cancer and are very harmful to the environment. Another most natural pest control method known as biological pest control method which reduces the number of pest populations by natural enemies without leaving any consequences for the rest of the environment. When it comes to cost effective method, organic pest control method comes to seen where Sodium Fluorocetate (FCH2CO2Na) is used which is a biodegradable poison to eradicate a range of pests. The drawback of these pest control methods are it typically needs an active human role to control pests.
[0004] Along with the various types of these pest control methods, various systems or methods have also been proposed which have been used for pest control mechanism. System proposed in 2017, US patent 2017/0231213 A1 shows a drone based pest abatement mechanism where the system consists of an aerial drone . The aerial drone have a pest sensor to sense the pests based on their emissions, an environmental sensor which detects the environment of the pest and a drone on-board computer identifies the pest type based on the emissions of the pest and establishes a risk level based on the pest type. After establishing risk level, the sprinkling system gets activated which contains the chemical pesticides to abate the pests. System also have been proposed in 1988 where the flying swarms is only three miles away and the magnetrons on the helicopters emit specially-tuned microwave pulses several times per seconds which sweep out and destroy a square mile of locusts in approximately two to three minutes.
[0005] The major drawback in the art is that the system using helicopter is much more costly as the fuel cost is much more and it is also inefficient. So instead of helicopter if the system uses an aerial drone then it will be much more effective. Another drawback in the art is that the system which uses an aerial drone cannot sweep out and destroy a square mile of locusts so the system needs multiple aerial drones which will sweep out and destroy the pests in an optimized way.
[0006] Yet another drawback in the art is the sensors that have been used to sense the pests are quite costly in comparison to software and to abate the pests various chemical pesticides are used which are very harmful to the environment as the chemicals can be passed through the food chain and may also cause chemical pollution.

OBJECT OF INVENTION

[0007] The principal objective of the invention is a system capable of controlling pests which mainly includes locust swarm, terrestial pest and also other insects that affects vegetation and crops by emitting specially-tuned magnetic pulses using multiple aerial drones in a very efficient and optimize way by designing various algorithms into the aerial drones. The system can be useful in any location where the pests are present.
[0008] The further objective of the invention is to sense various types of real-time pests a Single Shot Multibox Detector (SSD) algorithm is used as pest sensor which mainly includes detection of pests and counting of pests. This algorithm makes the invention more cost effective as most of the proposed system are using various chemical sensors to sense the pests which is much more costlier than using an computer vision algorithm.
[0009] The further objective of the invention is the system consists of multiple aerial drones where the number of drones depends on the swarm of pests and each drone is capable to measure the movement of other leading drones and also their own position based on Swarm Intelligence. The multiple aerial drones can easily sweep out and destroy a square mile of pests in an optimized way instead of using an aerial drone for pest control. Although some of the proposed system are using helicopters to control pests which is much more costlier than drones as it includes the fuel cost.
[0010] The further objective of the invention is the magnetrons in the aerial drones that emits magnetic pulses several times per seconds which can easily sweep out and destroy a swarm of pests in an efficient way without causing any harmful to the environment. Although most of the proposed system are using chemical pesticides to control pests which are very harmful to the environment. Moreover the magnetic pulses can destroy pests even when they are hiding where these chemical pesticides can’t reach which makes the system more effective to control pests.

STATEMENT OF INVENTION

[0011] The statement of the invention herein is a drone augmented pest control system comprising multiple aerial drones where each drone includes a camera, a pest sensor, a gyroscope sensor, a gps sensor, a drone on-board computer, a magnetron and position and control unit. The main feature of this invention is the system capable of controlling pests which mainly includes locust swarms, terrestial pests and various other insects that affects vegetation and crops. If large number of pests comes to seen, then other leading drones also comes to seen for controlling those pests in an optimized and efficient way. The pests are controlled by emitting specially-tuned electromagnetic pulses several times per seconds which sweep out and destroy a square mile of locusts in approximately two to three minutes.

SUMMARY OF INVENTION

[0012] In an embodiment of the present invention, multiple aerial drone-based method or computer program controls the pests problem. A pest sensor on each aerial drone is a computer vision algorithm naming Single Shot Multibox Detector (SSD) which senses various types of pests by matching phase during the training of model. One or more processors determine , based on the frames captured on the aerial drone , identify the type of the pest based on the training set of SSD algorithm. The processor(s) establish a threat level posed by the presence of the pest based on the pest type and image of the pest. The aerial drone then initiates a pest control mechanism of the pests based on the pest type and the threat level posed by the presence of the pest which mainly includes locust swarm, terrestial pest and also other insects that affects vegetation and crops.
[0013] In an embodiment of the present invention , multiple aerial drones are present where each drone includes a camera, a gyroscope sensor, a gps sensor, a drone on-board computer, and a magnetron. The pest sensor is an algorithm designed into the drone on-board computer which determines the location of the pest, senses the type of the pest based on the images of the pest collected from the drone camera via detection process, establishes a threat level posed by the presence of the pest based on the pest type via counting process and control other leading drones by measuring their movement based on the Particle Swarm Optimization (PSO) algorithm. The pest control mechanism performs a pest control using multiple aerial drones by delivering specially-tuned magnetic pulses towards the pest which can sweep out and destroy a square miles of pest in an optimized way based on the pest type and the threat level posed by the presence of the pest.

BRIEF DESCRIPTION OF DRAWINGS
[0014] FIG. 1 depicts the entire internal system and network of each aerial drone in which the present disclosure may be implemented;
[0015] FIG. 2 depicts additional detail of each exemplary aerial drones in accordance with one or more embodiments of the present invention ;
[0016] FIG. 3 illustrates control hardware and other hardware features of each exemplary aerial drones in accordance with one or more embodiments of the present invention ;
[0017] FIG. 4 depicts an overall pest control system which shows how multiple aerial drones are being utilized to control swarm of pests according to an embodiment of the present invention;
[0018] FIG. 5 illustrates main components of each aerial drone of FIG. 4 which are being utilized to control various types of pests in accordance with one or more embodiments of the present invention ;
[0019] FIG. 6 illustrates a high-level flow chart of one or more steps performed by one or more computing and/or other hardware devices to ameliorate a pest control problem in accordance with one or more embodiments of the present invention ;
[0020] FIG. 7 illustrates a Single Shot Multibox Detector (SSD) network architecture which is mainly used to sense various types of pests according to an embodiment of the present invention;
DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention is integrated systems or a computer program product . The computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention. This invention minimize the need of manpower, overcome bottlenecks in the flow of information and as a whole solve or at least ameliorate the problems outlined above.
[0022] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. And the computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing processing device. Also the computer readable program instructions are for carrying out operations of the present invention may be assembler instructions , machine instructions, machine dependent instructions ,microcode , firmware instructions, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java , Python or similar programming languages .
[0023] The present invention aims to develop a system which is capable of controlling various types of pests by sweeping out or destroying them which mainly includes locust swarm, terrestial pest and also other insects. The invention can be also useful to control other pests that affects vegetation, trees and crops.
[0024] Aspects of the present invention are described herein with reference to flowchart illustrations or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer readable program instructions .
[0025] The flowchart and block diagrams in the Figures illustrates the architecture, functionality, and operation of possible implementations of systems , methods , and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module , segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions. In some alternative implementations , the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
[0026] With reference now to the figures , and in particular to FIG. 1, there is depicted a block diagram of an entire internal system and network of each aerial drone that may be utilized by and in the implementation of the present invention. Some or all of the exemplary architecture, including both depicted hardware and software, shown for and within computer 101 may be utilized by drone on-board computer and/or SH69P25K microcontroller 151 shown in FIG. 1 , and/or drone on-board computer 223 shown in FIG. 2, and/or drone on-board computer 323 shown in FIG. 3 , and/or drone on-board computer 523 shown in FIG. 5 .
[0027] Exemplary computer 101 includes a processor 103 that is coupled to a system bus 105. Processor 103 may utilize one or more processors , each of which has one or more processor cores . A video adapter 107 , which supports a display 109 , is also coupled to system bus 105 . System bus 105 is coupled via a bus bridge 111 to an input/output (1/0) bus 113. An I/O interface 115 is coupled to I/O bus 113. I/O interface 115 affords communication with various I/O devices, including a camera 119 ( i.e. , a digital camera capable of capturing still and moving images), and external USB port(s) 125 . While the format of the ports connected to I/O interface 115 may be any known to those skilled in the art of computer architecture, in one embodiment some or all of these ports are universal serial bus (USB) ports .
[0028] Also coupled to I/O interface 115 is a positioning unit 151, which determines a position of computer 101 and other devices using positioning sensors 153. Positioning sensors 153 , may be any type of sensors that are able to determine a position of aerial drones, including computer 101, an aerial drone 200 shown in FIG. 2 , etc . Positioning sensors 153 may utilize, without limitation, satellite based positioning devices (e.g., global positioning system GPS based devices), gyroscope sensor (to measure change in movement of other leading drones and itself) and barometers (to measure change in altitude).
[0029] As depicted, computer 101 is able to communicate with a software deploying server 149 or other devices /systems ( e.g., drone on - board computer 123 ) using a network interface 129. Network interface 129 is a hardware network interface, such as a network interface card (NIC) ,etc . Network 127 may be an external network such as the Internet , or an internal network such as an Ethernet or a virtual private network (VPN) . In one or more embodiments , network 127 is a wireless network , such as a Wi-Fi network , a cellular network , etc .
[0030] A hard drive interface 131 is also coupled to system bus 105 . Hard drive interface 131 interfaces with a hard drive 133. In one embodiment , hard drive 133 populates a system memory 135 , which is also coupled to system bus 105 . System memory is defined as a lowest level of volatile memory in computer 101. This volatile memory includes additional higher levels of volatile memory (not shown), including, but not limited to, cache memory, registers and buffers. Data that populates system memory 135 includes computer 101's operating system (OS) 137 and application programs 143.
[0031] Operating System (OS) 137 includes a shell 139 , for providing transparent user access to resources such as application programs 143. Generally, shell 139 is a program that provides an interpreter and an interface between the user and the operating system . More specifically, shell 139 executes commands that are entered into a command line user interface or from a file. Thus, shell 139, also called a command processor, is generally the highest level of the operating system software hierarchy and serves as a command interpreter. The shell provides a system prompt, interprets commands entered by keyboard, mouse, or other user input media, and sends the interpreted commands to the appropriate lower levels of the operating system (e.g., a kernel 141) for processing. While shell 139 is a text-based, line-oriented user interface, the present invention will equally well support other user interface modes, such as graphical, voice, etc.
[0032] As depicted, OS 137 also includes kernel 141, which includes lower levels of functionality for OS 137, including providing essential services required by other parts of OS 137 and application programs 143, including memory management, process and task management, disk management, and mouse and keyboard management.
[0033] Application programs 143 in computer 101's system memory include Single Shot Multibox Detector (SSD) Algorithm 145. SSD Algorithm 145 includes code for implementing the processes described in FIGS. 6-7. In one embodiment, computer 101 is able to download SSD Algorithm 145 from software deploying server 150 for detection of pests and also establishes the threat level posed by the presence of the pest, including in an on-demand basis. In one embodiment of the present invention, software deploying server 150 performs all of the functions associated with the present invention ( including execution of SSD 145), thus freeing computer 101 from having to use its own internal computing resources to execute SSD 145 .
[0034] Application programs 143 in computer 101's system memory also include Logic for Positioning-Particle Swarm Optimization(PSO) Algorithm 147. PSO Algorithm 147 includes code for implementing the processes described below, including those in FIGS.2-6. In one embodiment, computer 101 is able to download PSO Algorithm 147 from software deploying server 150 for positioning of drone, including in an on-demand basis, wherein the code of PSO 147 is not downloaded until needed for execution. In one embodiment of the present invention, software deploying server 150 performs all of the functions associated with the present invention (including execution of PSO 147) , thus freeing computer 101 from having to use its own internal computing resources to execute PSO 147.
[0035] Application programs 143 in computer 101's system memory also include Logic for Turning on the Magnetron 149. It includes code for implementing the processes described below, including those in FIGS. 2-6. In one embodiment, computer 101 is able to download Logic for Turing on the Magnetron 149 from software deploying server 150 for sweeping out and destroying the pests, including in an on-demand basis, wherein the code of Turning on the Magnetron 149 is not downloaded until needed for execution. In one embodiment of the present invention, software deploying server 150 performs all of the functions associated with the present invention (including Turning on Magnetron 149), thus freeing computer 101 from having to use its own internal computing resources for turning on magnetron 149.
[0036] The hardware elements depicted in computer 101 are not intended to be exhaustive, but rather are representative to highlight essential components required by the present invention. For instance, computer 101 may include alternate memory storage devices such as magnetic cassettes, digital versatile disks (DVDs) , Bernoulli cartridges, and the like. These and other variations are intended to be within the spirit and scope of the present invention.
[0037] FIG. 2 illustrates an exemplary aerial drone 200 in accordance with one or more embodiments of the present invention. The terms “aerial drone”, “drone”, and " unmanned aerial vehicle”( “UAV” ) are used interchangeably herein to identify and describe an airborne vehicle that is capable of pilot-less flight and controlling the pests as described herein.
[0038] As shown in FIG. 2, aerial drone 200 includes a body 202, which is attached to supports such as support 204. Supports such as support 204 support stanchions such as stanchion 206 . Such stanchions provide a housing for a driveshaft within each of the stanchions ,such as the depicted driveshaft 208 within stanchion 206. These driveshafts are connected to propellers. For example , driveshaft 208 within stanchion 206 is connected to propeller 210.
[0039] A power transfer mechanism 212 (e.g., a chain, a primary driveshaft, etc .) transfers power from a geared transmission 214 to the driveshafts within the stanchions (e.g., from geared transmission 214 to the driveshaft 208 inside stanchion 206) , such that propeller 210 is turned , thus providing lift and steering to the aerial drone 200 . Geared transmission 214 preferably contains a plurality of gears, such that a gear ratio inside geared transmission 214 can be selectively changed . Power to the geared transmission 214 is preferably provided by an electric motor 216, which is quieter than an internal combustion engine , and thus is better suited to the present invention.
[0040] Affixed to the bottom of body 202 is a camera controller 224, which is logic that controls movement of a camera 226 via a camera support 228 (which includes actuators for movement of camera 226 ). The camera controller 224 is able to focus, as well as aim camera 226, while under the control of a drone on-board computer 223 ( analogous to drone on - board computer 123 shown in FIG. 1 ). In a preferred embodiment, drone on-board computer 223 controls all components of aerial drone 200 depicted in FIG . 2 , and performs all or some of the analytics described herein (i.e., identifying a pest and pest type, controlling a pest, etc .). Also affixed to the body 202 of aerial drone 200 is a nozzle 230, which receives specially-tuned magnetic pulses (by a magnetron 232) and is controlled by SH69P25K Microcontroller 234 which is a One Time Programmable device.
[0041] While aerial drone 200 is depicted in FIG. 2 as having multiple propellers, in another embodiment a single propeller elevates and adjusts pitch and roll of the aerial drone while a rotor adjusts yaw of the aerial drone using a set of positioning sensors (e.g., gyroscope sensors) that cause the propeller and rotor to change attitude. Similar positioning sensors can likewise adjust the attitude of multiple propellers .
[0042] In an embodiment of the present invention, aerial drone 200 is "miniaturized”, thus allowing it to be flown in confined spaces, such as between walls, in attics, within silos, pressure vessels, and other storage structures, etc. That is, in this embodiment a maximum dimension of aerial drone 200 may be less than 6 inches , or even less than one inch, based on the level of available miniaturized components .
[0043] With reference now to FIG. 3, exemplary control hardware and other hardware components within the aerial drone 200 presented in FIG. 2 are depicted. A drone on- board computer 323 ( analogous to drone on-board computer 223 shown in FIG. 2) controls a drone mechanism controller 301, which is a computing device that controls a set of drone physical control mechanisms 303. The set of drone physical control mechanisms 303 include, but are not limited to, throttles for electric motor 216, selectors for selecting gear ratios within the geared transmission 214, controls for adjusting the pitch, roll, and angle of attack of propellers such as propeller 210 and other controls used to control the operation and movement of the aerial drone 200 depicted in FIG. 2.
[0044] Whether in autonomous mode or remotely-piloted mode (based on control signals sent via computer 101 to the drone on-board computer 123 shown in FIG. 1), the drone on-board computer 323 controls the operation of aerial drone 200. This control includes the use of outputs from navigation and control unit 305 to control the aerial drone 200. Navigation and control unit 305 include hardware sensors that (1) determine the location of the aerial drone 200; (2) sense pests and other aerial drones around aerial drone 200; (3) measure the speed and direction of the aerial drone 200 ;and (4) provide any other inputs needed to safely control the movement of the aerial drone 200.
[0045] With respect to the feature of (1) determining the location of the aerial drone 200 , this is achieved in one or more embodiments of the present invention through the use of a positioning system such as positioning system 151( shown in FIG. 1) , which may be part of the drone on-board computer 323, combined with positioning sensor 353. Positioning system 151 may use a global positioning system(GPS Sensor 355), which uses space-based satellites that provide positioning signals that are triangulated by a GPS receiver to determine a 3-D geophysical position of the aerial drone 200. Positioning system 151 may also use, either alone or in conjunction with a GPS system, physical movement sensors such as gyroscope sensor 354, barometers (which measure altitude changes by the aerial drone), etc.
[0046] With respect to the feature of (2) sensing pests and other aerial drones around aerial drone 200, the drone on-board computer 323 utilize Single Shot Multibox Detector (SSD) Algorithm 145 and PSO (Particle Swarm Optimization) Algorithm 147 by downloading it from the software deploying server 150. The SSD Algorithm 145 senses various types of pest via detection process and also establishes threat label posed by the presence of the pest whereas the PSO Algorithm 147 controls the other leading drones through monitoring and search stage by measuring the movement and position of other drones.
[0047] With respect to the feature of (3) measuring the speed and direction of the aerial drone 200, this is accomplished in one or more embodiments of the present invention by taking readings from an on-board airspeed indicator (not depicted) on the aerial drone 200 and or detecting movements to the control mechanisms (depicted in FIG. ) on the aerial drone 200 and the positioning system 151 discussed above .
[0048] With respect to the feature of (4) providing any other inputs needed to safely control the movement of the aerial drone 200, such inputs include, but are not limited to, control signals to fly the aerial drone 200 to land aerial drone 200 (e.g. to make an emergency landing), etc.
[0049] Also on aerial drone 200 in one or more embodiments of the present invention is a camera 326, which is capable of sending still or moving visible light digital photographic images (or video) to the drone on-board computer 323 and when it captures video then on that case the extraction of frames from video is done by SSD algorithm 145. Besides capturing images of pests as described herein, camera 326 is able to capture images of physical objects. These images can be used to determine the location of the aerial drone 200 (e.g. , by matching to known landmarks), to sense other drones and to determine speed (by tracking changes to images passing by), as well as to receive visual images of pests as described herein.
[0050] Also on aerial drone 200 in one or more embodiments of the present invention is magnetron 332 . The magnetron 332 is a high-powered vaccum tube that works as a self-excited microwave oscillator which emits specially-tuned electromagnetic waves instead of sound waves so the human can’t hear the resonant energy. The magnetron emits those electromagnetic waves to destroy the pests from the nozzle 230 of aerial drone 200.
[0051] Also on aerial drone 200 in one or more embodiments of the present invention are lights 311. Lights 311 are activated by drone on-board computer 323 to provide visual warnings ,alerts ,etc. That is, once the aerial drone 200 detects a certain type of pest (s), an alert light may be displayed by lights 311.
[0052] With reference now to FIG. 4, shows a pest control system 402 which consists of N number of aerial drones 400 where each aerial drone contains all the functionalities described in FIGS. 1-7 and each drone emits specially-tuned electromagnetic pulses several times per seconds from nozzle 230 as mentioned in FIG. 2 to sweep out and destroy a swarm of pests 414 in approximately two to three minutes with very efficient and optimized way.
[0053] With reference now to FIG. 5, an exemplary aerial drone 400 (analogous to aerial drone 200 and aerial drone 400 shown in FIG. 2-4) is depicted in use according to one or more embodiments of the present invention. Aerial drone 400 has a camera, a pest sensor, positioning unit, a magnetron, a drone on-board computer , and a pest control mechanism. As described herein, the pest sensor is a SSD Algorithm 145 is used which is described in FIG. 7. The detection process of pests may be a camera which captures still and moving images or captures video which is then transformed to multiple frames.
[0054] Similarly, positioning unit 551 may be a gyroscope sensor 553 which detects the angular velocity in three axis. So it can detect rate of change of angle in pitch, roll and yaw. The change in angle information is used to provide stability to drone and to prevent it from wobbling. The information from gyroscope is fed to electric motor 216 in FIG. 2 to control the speed of motors dynamically to provide the stability to motor. Gyroscope also ensure that drones rotate at exact angle and also measures the movement and position of other leading drones with respect to itself in accordance to one or more embodiments of the present invention.
[0055] The drone on-board computer 523 identifies a pest type of the pest based on the images of the pest via detection process of SSD algorithm, and also establishes a risk level posed by the presence of the pest based on the pest type and the environment of the pest. Also examples of pest control mechanism 502 includes mechanical, chemical, aural, light, and other control mechanisms. For example, pest control mechanism 502 may be the combination of SH69P25K Controller 234 , magnetron 232 , and nozzle 230 shown in FIG. 2 for emitting electromagnetic pulses in order to disturb or kill the pest (e.g., swarm of airborne pests 508 or terrestrial pest 512). In another embodiment, pest control mechanism 502 is a mechanical probe ( not shown in the figures ) or merely the body 202 or propeller 210 or camera support 228 shown in FIG. 2 that can mechanically damage, knock off of an eave ( in the case of a wasps' nest ) , etc . the pest nest 510. In another embodiment, the pest control mechanism 502 may be the propeller 210 shown in FIG. 2, which generates sufficient wind to blow a flying pest (e.g., swarm of airborne pests 508) out of an area. In another embodiment, the pest control mechanism 502 may be the lights 311 shown in FIG. 3, which can strobe brightly enough to cause pests ( for example, bats ) to vacate the area.
[0056] For example, assume that the drone on-board computer 523, based on pest sensors which is SSD algorithm 145, determines that the pest being monitored is a venomous insect having no useful agricultural purpose (e.g., a wasp but not a bee). Assume further that the wasp is identified as being next to a children's playground (and not in the middle of uninhabited woods). As such, the drone on-board computer 523 will determine that this wasp (or swarm of wasps ) poses a threat to the children on the playground, and will take steps to control ( i.e., remove ) the threat posed by the wasps.
[0057] Thus, the drone on-board computer 423 will initiate a pest control of the pest based on the pest type and the risk level posed by the presence of the pest . Examples of such pest control include killing the terrestrial pest 512 and locust swarm 514 (e.g., by emitting electromagnetic pulses from the magnetron 332 shown in FIG. 5); physically forcing the swarm of pests to leave the area e. g., by maneuvering the aerial drone 400 so close to the wasps that they are encouraged to leave the area).
[0058] With reference now to FIG. 6, a high level flowchart of one or more steps performed by one or more processors and/or other hardware devices to control the presence of a pest is presented . A pest is defined as any non - human biological creature predetermined to be unduly harmful (beyond a standard set by the entity making the predetermination) to persons, to domesticated animals or to the environment. Such pests include, but are not limited to, venomous insects, terrestrial pests, locust swarm and various other insects.
[0059] After initiator block 601, camera 326 captures video of the pests and cut it into several frames (i.e. extract frames 603) .This cut be done in such a way that a sequence of images can be obtained and that allows the motion detection of the objects therin. Initially, the region of interest is selected and information such as trees and sidewalks is ignored. Then removal of background 605 takes place in three steps: (1) the difference between two frames, f and f+1 is calculated. (2) the comparison of difference is done and (3) then the pixels having the same value in the difference calculation are eliminated. Then the image resulting from the background removal is converted to grayscale 607. The grayscale images goes to morphological operations 609 in the object segmentation process. After the segmentation process of the objects, the contours are identified in the image, and for each object contained in the scene, a minimum rectangle 611 is inserted, in this way, each rectangle obtained is the representation of a pest. A minimum rectangle size is defined so that it is possible to identify valid objects in the image (i.e. pests). This entire process is known as detection process.
[0060] After detection process, the counting process comes to seen where the objects of detection process has been kept. Valid rectangles are taken as pests on the scene and the centroids of each object are calculated 613. In the first case, each of the centroids found is assumed as a pest and are stored in a list. In the second case, it is necessary to compare the pests stored in the list, and with those found in the current frame, such comparison is necessary so that there is a link between the same pest in different frames. So it is checked that pest (or centroid) crossed counting process or not 615. If it is yes then the pest takes into account 617 and the new positions are ignored to the same centroid 619. Then the block 621 comes to action where the fitness value is calculated for each aerial drone. And if the centroid crossed counting point is no, then it is checked whether distance between centroid is less than threshold or not 649. If it is yes then connect the current centroid (or pest) to the previous 651 and then the process loops 653 back to the condition 615. And if the condition 649 is no then new pest can be found 655 and if the new pest is found then it loops 657 back to the condition 615. In this way the entire process continues until the new pest comes into account. In the present invention the SSD Algorithm 145 for detection of pests goes through this detection and counting process.
[0061] After the detection of pests is done, then the positioning of drones needs to be done by Particle Swarm Optimization (PSO) algorithm. For these, the fitness evaluation 623 needs to be done for each aerial drone. The aerial drone which have the highest fitness value is fittest as described in 625 which means that the ones with highest fitness value is having the shortest distance then the drone can easily travel to the target path in shortest time. Then the condition is checked whether fitness value is better than the best fitness value (pBest) as in 627. If it is true then set the current fitness value as the new pBest (or lBest or local best) and then go through 631 to another condition 633. If it is false then also go to condition 633 which determines whether fitness value is better than the global fitness value (gBest) where gBest is tracked by the particle swarm optimizer. If the condition 633 is true then set the current fitness value as the new gBest (or global best) and then go through 637 to block 639. If the condition 633 is false then calculate drone velocity and position as in 639 via gyroscope sensor 354 in FIG. 3 which measures the movement of other drones and also position of other drones and itself. After that the condition 641 is checked whether the maximum iteration criteria is attained or gBest value is better than the desired goal. If the condition 641 is true then the signal is sent to the SH69P25K microcontroller 234 in FIG. 2 and if the condition 641 is false then it loops 647 back to the fitness evaluation 623 process again. In this way the multiple aerial drones are set for the pest control mechanism according to an embodiment of the present invention.
[0062] Once the aerial drones get arranged according to the number of pests (or locust swarm 514) described above, the signal is send to the SH69P25K Microcontroller from the aerial drone 200 in FIG. 2 as said in the block 643. The SH69P25K is a 4 Bit Microcontroller which is known as One Time Programmable device so there is no chance of reprogramming the chip. The main motive of SH69P25K Microcontroller is to control the magnetron by activating and deactivating when needed as said in the block 645. Once the magnetron 232 in FIG. 2 gets activated, it emits specially-tuned magnetic pulses towards the pests 414 in FIG. 4 several times per seconds to sweep out and destroy them in approximately two to three minutes.
The flow chart ends at terminator block 647.
[0063] With reference now to FIG. 7 illustrates a Single Shot Multibox Detector (SSD) network architecture where the algorithm or logic is described above in respect to FIG. 6. SSD is capable of detecting multiple objects from a single image/frame. SSD object detection mainly comprises of two parts: (1) extract feature maps and (2) apply convolution filter to detect objects (i.e. various types of pests). SSD’s architecture builds on the venerable VGG-16 architecture, but discards the fully connected layers. The reason VGG-16 was used as the base network is because of its (1) strong performance in high quality image classification tasks and (2) its popularity for problems where transfer learning helps in improving results. Instead of the original VGG fully connected layers, a set of auxiliary convolutional layers (from conv6 onwards) were added, thus enabling to extract features at multiple scales and progressively decrease the size of the input to each subsequent layer. Then it detects objects using the Conv5_3 layer. The SSD Algorithm 145 in FIG. 1 is based on a feed-forward convolutional network that produces a fixed-size collection of bounding boxes and scores for the presence of object class instances in those boxes, followed by a non-maximum suppression step to produce the final detections. Thus, in an embodiment of the present invention, the SSD algorithm detects the various types of pests and their threat level based on training of this model.
[0064] Thus, in an embodiment of the present invention , the detection of pest process is a camera (e.g., camera 326 ) , and the Single Shot Multibox Detector (SSD) algorithm described above, such that the visual image of the pest is captured by the camera. This captured image is transmitted from the aerial drone to the display 109. The drone on-board computer 123 shown in FIG. 1 will transmit (e.g., stream in real time) images of the pest to the display, so that any user can check the pest images later.
[0065] In an embodiment of the present invention, a camera (e.g., camera 326 shown in FIG . 3 ) on the aerial drone detects a pest nest ( e . g . , pest nest 510 shown in FIG. 5) of the pest. In response to detecting the pest nest, the pest control mechanism 502 destroys the pest nest by emitting magnetic pulses from magnetron.
[0066] In an embodiment of the present invention, electromagnetic wave pulses(e.g. , a combination of SH69P25K Microcontroller 234, magnetron 232, and nozzle 230 shown in FIG. 2) performs the pest control by applying magnetic pulses to the pest where the pest is one or more insects from a group of insects consisting of wasps, hornets, scorpions, locusts swarm, carpenter ants, termites, cockroaches, ticks, and yellow jackets.
[0067] In an embodiment of the present invention, the pest is an insect that is part of an insect swarm (e.g., the depicted swarm of airborne pests 508 shown in FIG. 5). In this embodiment, the aerial drone-based method may further detect, by the aerial drone, a behavioural pattern of insect swarm, where the behavioural pattern includes insect swarming movement indicative of aggressive behaviour, insect swarm density changes, and flight patterns of the insect swarm towards a pest nest. The aerial drone will then adjust the pest control according to the behavioural pattern of the insect swarm. Such swarms have characteristic shapes, movement , etc. that are compared to known swarm traits via training of Single Shot Multibox Detector (SSD) Model, in order to determine if the swarm if in an aggressive attack mode, is merely flying back to their hive , is merely foraging for food, etc. Based on this determination, the type of pest control is adjusted accordingly.
[0068] One benefit of the invention presented herein is that a human is less likely to be stung or otherwise be adversely affected, as may be the case if a human must climb a ladder to reach a pest nest(e.g., a wasp nest). Also, the present invention allows insect extermination in hard-to-reach places, such as crawl spaces, between walls, in tight attics, in confined spaces such as storage silos and storage tanks, etc . The smart drone presented herein also ensures that magnetic pulses does not affects the environment except the pests.
[0069] In an embodiment of the present invention, before a drone takes an extermination action, it must wait for signal and enlist other drones to increase confidence levels and when such other drones have appropriate extermination features. For example , assume that a particular aerial drone has a 70% confidence level that a pest swarm being observed and magnetic pulses being emitted from that aerial drone , and that there is a 50% chance that the particular aerial drone releasing at the swarm will eradicate it . In order to increase these confidence levels , this first aerial drone will call for(i.e. , send a signal to other aerial drones in the area ) other aerial drones to come to where the first aerial drone is in order to (1) confirm the type and state of insects in the swarm, and (2) assist the first aerial drone in releasing magnetic pulses, in order to wipe out the swarm.
[0070] In some cases, the drone may seek to identify helpful insects (e.g., honeybees needed for pollination). In such cases, the drone may take a helpful action such as “doing no harm", by not emitting magnetic pulses on that area. As the motive of the present invention is to sweep out and destroy those insects which are harmful for the environment.
[0071] A high-definition camera, an image processor, and a main controller may be employed for insect analysis, etc. The image processor may be connected with the high-definition camera and used for performing picture processing on the potential insect swarming pictures to obtain pest types. If desired, according to this pest detecting system, the pest types can be automatically obtained to provide convenience for relevant agricultural management departments(or exterminators ) to invoke during returning of the unmanned aerial vehicle, so that pointed pest prevention and control measurements can be made. Alternatively, the drone can be smart and take action.
[0072] Insect identification can be performed by one or more means. In an embodiment of the present invention, Single Shot Multibox Detector (SSD) algorithm is used for insect identification which is a well known computer vision algorithm. Also insects may be identified by digital image progressing, pattern recognition and the theory of taxonomy . Artificial neural networks (ANNs) and a support vector machine (SVM) are used as pattern recognition methods for the identifications. Similarly, other input parameters may be considered such as body shape and pattern characteristics , body eccentricity , color complexity , center of gravity of insect silhouette , etc . For example , as ample image ( taken in real time of a pest ) and a recognition image ( of a known pest ) may be input into a preprocess image , which contains common features of both images( e.g. ,size, color, shape, etc. of the pest) . The common features are extracted in order to (1) recognize the type of pest , and (2) train the system to look for certain features in future pests , in order to recognize future presences of the pest.
[0073] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and “comprising ,”when used in this specification, specify the presence of stated features , integers ,steps , operations, elements, and components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components , and groups thereof .
[0074] Having thus described embodiments of the present invention of the present application in detail and by reference to illustrative embodiments thereof , it will be apparent that modifications and variations are possible without departing from the scope of the present invention defined in the appended claims. , C , Claims: 1. A drone-augmented pest control system for controlling various types of pests there among locust swarm, terrestrial pest and other insects, comprising:
sensing, by a pest sensor on each aerial drone, a pest, wherein the pest sensor detects pest type of the pest;
determining, by one or more processors and based on the readings from pest sensor on each aerial drone, quantity of the pest;
identifying, by one or more processor, other leading aerial drones based on the movement of other drones and position of itself;
establishing, by one or more processors, a threat level posed by the presence of the pest based on the pest type; and
initiating, via each aerial drone, a pest control of the pest based on the pest type and the threat level posed by the presence of the pest.
2. The drone-augmented pest control system of claim 1, wherein the pest sensor is a camera, wherein the pest type of the pest is detected from the video of the pest, and the video of the pest is captured by camera.
3. The drone-augmented pest control system of claim 2, further comprising: transmitting, from the aerial drone to a display for visualization of image, the visual image or video of the pest.
4. The drone-augmented pest control system of claim 1, wherein the pest sensor is an algorithm designed into the system to detect pests, and the frames are extracted from the video captured by camera.
5. The drone-augmented pest control system of claim 1, further comprising: detecting, by a camera on the aerial drone, swarm of pest; and in response to detecting the swarm of pest, destroying, by the pest control mechanism, the swarm of pest.
6. The drone-augmented pest control system of claim 5, further comprising: destroying, by a magnetron on the aerial drone, swarm of pests; by emitting specially-tuned magnetic pulses several times per seconds towards the swarm of pest.
7. The drone-augmented pest control system of claim 6, further comprising: SH69P25K microcontroller, wherein the magnetron of each aerial drone is controlled by the microcontroller.
8. The drone-augmented pest control system of claim 1, wherein the aerial drones are controlled by Swarm Intelligence, wherein the movement of the other aerial drones are measured by gyroscope sensor.