CN111402541A - Forest fire extinguishing method and system based on unmanned aerial vehicle cluster - Google Patents
Forest fire extinguishing method and system based on unmanned aerial vehicle cluster Download PDFInfo
- Publication number
- CN111402541A CN111402541A CN202010164595.XA CN202010164595A CN111402541A CN 111402541 A CN111402541 A CN 111402541A CN 202010164595 A CN202010164595 A CN 202010164595A CN 111402541 A CN111402541 A CN 111402541A
- Authority
- CN
- China
- Prior art keywords
- fire
- unmanned aerial
- aerial vehicle
- image
- rotor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000006399 behavior Effects 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000013135 deep learning Methods 0.000 claims abstract description 16
- 239000000779 smoke Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000006855 networking Effects 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 2
- 239000002699 waste material Substances 0.000 abstract description 3
- 238000005070 sampling Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
- G08B17/125—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
-
- A—HUMAN NECESSITIES
- A62—LIFE-SAVING; FIRE-FIGHTING
- A62C—FIRE-FIGHTING
- A62C3/00—Fire prevention, containment or extinguishing specially adapted for particular objects or places
- A62C3/02—Fire prevention, containment or extinguishing specially adapted for particular objects or places for area conflagrations, e.g. forest fires, subterranean fires
- A62C3/0228—Fire prevention, containment or extinguishing specially adapted for particular objects or places for area conflagrations, e.g. forest fires, subterranean fires with delivery of fire extinguishing material by air or aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Forests & Forestry (AREA)
- Public Health (AREA)
- Ecology (AREA)
- Biodiversity & Conservation Biology (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a forest fire extinguishing method and system based on unmanned aerial vehicle clusters, wherein the method comprises the following steps: receiving images and coordinates returned by the fixed-wing unmanned aerial vehicle; processing the image by using a deep learning algorithm, judging the fire behavior and responding; dispatching a plurality of multi-rotor unmanned aerial vehicles in batches according to an autonomous decision algorithm to extinguish the fire; receiving an image returned after the multi-rotor unmanned aerial vehicle throws the fire extinguishing bomb; and judging the fire behavior according to the real-time image and responding. The forest fire situation is continuously monitored through the multiple fixed-wing unmanned aerial vehicles, and the fire is timely found; the multi-rotor unmanned aerial vehicle is dispatched in batches to execute tasks, the fire behavior is judged immediately according to the fire scene pictures returned in real time, the arrangement is reasonable, and the waste of resources is reduced.
Description
Technical Field
The invention relates to the field of forest fire prevention, in particular to a forest fire extinguishing method and system based on unmanned aerial vehicle clusters.
Background
Forests are important components of natural resources and habitats of many animals and plants. But forests are also areas with high fire. The most effective means for reducing the loss of forest fires is to discover and extinguish the forest fires in time. The satellite remote sensing is seriously influenced by weather, and the real-time performance and the resolution ratio are poor; manual patrol is high in labor cost. In addition, the fire fighting truck is difficult to drive in for forest fire fighting, and the resource consumption is serious for small-sized fire disasters when an airplane is used for fire fighting. With the rise of unmanned aerial vehicle technology, another new way is provided for forest inspection and fire extinguishing.
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art and provides a forest fire extinguishing method and system based on an unmanned aerial vehicle cluster.
The technical scheme adopted by the invention for solving the problems is as follows:
in a first aspect of the invention, a forest fire-extinguishing method based on a cluster of unmanned aerial vehicles, the cluster of unmanned aerial vehicles comprising a plurality of fixed-wing unmanned aerial vehicles for inspecting fire and a plurality of multi-rotor unmanned aerial vehicles for throwing fire extinguishing bombs to extinguish fire, comprises the following steps:
receiving a first fire scene image and a fire scene coordinate returned by the fixed-wing unmanned aerial vehicle;
processing the first fire scene image by using a deep learning algorithm and judging the fire, if the fire is judged to be in an initial stage, establishing an unmanned aerial vehicle fire extinguishing task, otherwise, notifying artificial fire fighting;
dispatching a plurality of multi-rotor unmanned aerial vehicles in batches according to an autonomous decision algorithm to extinguish the fire;
receiving a second fire scene image returned by a plurality of multi-rotor unmanned aerial vehicles after the fire extinguishing bomb is thrown;
processing the second fire scene image by using a deep learning algorithm and judging the fire behavior, if the fire behavior is judged to be in a non-initial stage, notifying manual fire fighting, and if not, continuously dispatching the multi-rotor unmanned aerial vehicle to extinguish the fire until the fire behavior is judged to be extinguished;
when the fire is judged to be extinguished, the fire extinguishing task of the unmanned aerial vehicle is cancelled.
According to the first aspect of the invention, the processing of the fire image by using the deep learning algorithm and the judgment of the fire behavior comprise the following steps:
preprocessing the fire image;
inputting the preprocessed fire image into a Mask R-CNN network model trained by a large number of fire sample images comprising a plurality of stages of fire behaviors to identify a flame class and a smoke class;
if the continuous x frames of fire images obtained by identification are only smoke, judging the fire images to be in an initial stage; if the fire images are identified to have flames, judging the fire condition according to the average area of the flames in the continuous x-frame fire images;
wherein the fire image comprises the first fire scene image or the second fire scene image, and the fire condition comprises the initial stage and the non-initial stage.
According to a first aspect of the invention, the pre-processing comprises image enhancement, smoothing filtering, edge detection and image segmentation.
According to the first aspect of the present invention, the calculation formula of the area of the flame class in one frame of the fire image is as follows: s (flame class) ═ S (boundary box) × (Num (mask of flame class)/Num (boundary box)); in the formula, S (flame) is the area of flame, S (boundary box) is the area of boundary box, Num (flame mask) is the number of pixel points of flame mask, Num (boundary box) is the number of pixel points of boundary box.
According to a first aspect of the invention, the fire images are taken with the fixed-wing drone or the multi-rotor drone at the same height relative to the ground plane.
According to a first aspect of the invention, said dispatching in batches a plurality of multi-rotor drones for fire extinguishing according to an autonomous decision algorithm comprises in particular the following steps:
respectively calculating the distance and the flight time of each multi-rotor unmanned aerial vehicle to reach the task place of the new task;
sequentially sequencing the multi-rotor unmanned aerial vehicles meeting the requirements from near to far according to the distance from the fire place, and then adding the multi-rotor unmanned aerial vehicles into a task queue, wherein the multi-rotor unmanned aerial vehicles meeting the requirements are multi-rotor unmanned aerial vehicles which are provided with resources capable of completing the fire extinguishing task of the unmanned aerial vehicles and are in a task-free state;
and sending an execution instruction to the multi-rotor unmanned aerial vehicles in the task queue according to a set time interval until the fire extinguishing task of the unmanned aerial vehicles is cancelled.
In a second aspect of the invention, a forest fire extinguishing system based on an unmanned aerial vehicle cluster comprises a control background and the unmanned aerial vehicle cluster, wherein the unmanned aerial vehicle cluster comprises a plurality of fixed-wing unmanned aerial vehicles and a plurality of multi-rotor unmanned aerial vehicles;
the control background comprises:
the communication module is used for receiving a first fire scene image returned by the fixed-wing unmanned aerial vehicle, coordinates of a fire scene and a second fire scene image returned by the multi-rotor unmanned aerial vehicle after the multi-rotor unmanned aerial vehicle throws a fire extinguishing bomb;
the image processing module is used for processing the first fire scene image and the second fire scene image by utilizing a deep learning algorithm;
the fire behavior judging module is used for judging the fire behavior according to the image output by the image processing module;
the task processing module is used for establishing and canceling an unmanned aerial vehicle fire extinguishing task according to the judgment result of the fire judgment module;
and the autonomous decision module is used for dispatching a plurality of multi-rotor unmanned aerial vehicles to extinguish fire according to an autonomous decision algorithm.
According to a second aspect of the invention, the forest fire extinguishing system based on the unmanned aerial vehicle cluster further comprises a ground control station, wherein the ground control station is used for relaying information interacted between the control background and the unmanned aerial vehicle cluster and supplementing resources and fire extinguishing bombs for the unmanned aerial vehicle cluster.
According to a second aspect of the invention, the fixed-wing drone and the multi-rotor drone are provided with a camera, a GPS positioning module, a networking module, an automatic obstacle avoidance module and a height measurement device; many rotor unmanned aerial vehicle still is equipped with fire extinguishing bomb throwing device.
According to a second aspect of the invention, the height measuring device comprises a sonar device for measuring the distance to the ground level, an accelerometer for measuring vertical acceleration, a barometer for measuring atmospheric pressure, and a height calculation module for calculating the current height in combination with data collected by the sonar device, the accelerometer and the barometer.
The forest fire extinguishing system based on the unmanned aerial vehicle cluster at least has the following beneficial effects: the fixed-wing unmanned aerial vehicle inspects the fire disaster, and returns the fire disaster scene image and the coordinates of the fire disaster scene once the fire disaster is found; utilize the deep learning algorithm to judge the intensity of a fire according to the conflagration scene image and make the decision-making, dispatch many rotor unmanned aerial vehicle in batches and put out a fire, the unmanned aerial vehicle of dispatching can passback the conflagration scene image at once after throwing the fire extinguishing bomb, the control backstage judges the intensity of a fire once more, whether need continue to dispatch many rotor unmanned aerial vehicle according to the intensity of a fire decision and put out a fire or inform artifical fire control. The multiple fixed-wing unmanned aerial vehicle can continuously inspect forests in a large range and find fires in time; the multi-rotor unmanned aerial vehicle can quickly cross the forest to reach a fire scene for fire extinguishing, is suitable for small-sized fire, and is safe and quick; and dispatching the unmanned aerial vehicle to execute tasks in batches, and judging the fire behavior in real time according to the real-time returned fire scene pictures, so that the arrangement is reasonable, and the waste of resources is reduced.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The invention is further illustrated with reference to the following figures and examples.
FIG. 1 is a flow chart of a forest fire extinguishing method based on unmanned aerial vehicle clustering according to an embodiment of the invention;
FIG. 2 is an exemplary diagram of image processing and fire determination;
fig. 3 is a structural diagram of a forest fire extinguishing system based on unmanned aerial vehicle clustering according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to the present preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
Referring to fig. 1, an embodiment of the present invention provides a forest fire extinguishing method based on a cluster of unmanned aerial vehicles, the cluster of unmanned aerial vehicles including a plurality of fixed-wing unmanned aerial vehicles 31 for inspecting fire and a plurality of multi-rotor unmanned aerial vehicles 32 for throwing fire extinguishing bomb to extinguish fire, the method including the steps of:
step S100, receiving a first fire scene image and a fire scene coordinate returned by the fixed-wing unmanned aerial vehicle 31;
s200, processing the first fire scene image by using a deep learning algorithm and judging the fire, if the fire is judged to be in an initial stage, establishing an unmanned aerial vehicle fire extinguishing task, otherwise, notifying artificial fire fighting;
step S300, dispatching a plurality of multi-rotor unmanned aerial vehicles 32 in batches according to an autonomous decision algorithm to extinguish a fire;
step S400, receiving a second fire scene image returned after the fire extinguishing bomb is thrown by the multiple multi-rotor unmanned aerial vehicles 32;
step S500, processing the second fire scene image by using a deep learning algorithm and judging the fire, if the fire is judged to be in a non-initial stage, notifying artificial fire fighting, and if not, continuously dispatching the multi-rotor unmanned aerial vehicle 32 to extinguish the fire until the fire is judged to be extinguished;
and S600, when the fire is judged to be extinguished, canceling the fire extinguishing task of the unmanned aerial vehicle.
Firstly, in step S100, each fixed-wing drone 31 inspects in a defined area according to a predetermined route, measures the temperature through a temperature sensor, and once the temperature of a certain position of the forest is higher than that of other positions, shoots a picture at a fixed height above the position where the drone flies vertically through a camera; meanwhile, the coordinates of the fire place are determined through a GPS positioning module. The fixed-wing drone 31 measures its own height relative to the ground level by means of a height measuring device to determine the position at which it reaches a fixed height. The fixed-wing drone 31 transmits the photographed first fire scene image and the coordinates of the fire scene to the control background 10 through the plurality of ground control stations 20 by means of the networking module.
It should be noted that the fixed-wing drone 31 has the advantages of fast flying speed, long range and no possibility of falling down immediately after stalling, and is suitable for executing a long-time task of patrolling the forest from high altitude. The multi-rotor unmanned aerial vehicle 32 has the advantages of flexibility, high site adaptability, low-altitude flight, stable flight, capability of realizing fixed-point hovering, multiple flight modes, capability of realizing automatic flight of path planning and the like, has high carrying capacity, and is suitable for executing the task of throwing fire extinguishing bombs to extinguish fire.
Referring to fig. 2, further, the processing the fire image and judging the fire behavior using the deep learning algorithm in steps S200 and S500 includes the steps of:
preprocessing a fire image; the preprocessing comprises image enhancement, smooth filtering, edge detection and image segmentation, so that background noise can be reduced, and the image identification precision is improved; of course, in other embodiments, image recognition can be performed without image preprocessing;
inputting the preprocessed fire image into a Mask R-CNN network model trained by a large number of fire sample images comprising a plurality of stages of fire behaviors to identify a flame class and a smoke class;
if the continuous x frames of fire images obtained by identification are only smoke, judging the fire images to be in an initial stage; if the fire images are identified to have flames, judging the fire condition according to the average area of the flames in the continuous x-frame fire images;
the fire image comprises a first fire scene image or a second fire scene image, and the fire condition comprises an initial stage and a non-initial stage.
It should be noted that the Mask R-CNN network model is composed of three core portions, namely a backbone network, a full convolution network and region of interest alignment. The main network is used for realizing target detection and classification of the fire image to obtain flame and smoke; the full convolution network has the function of adding masks to the detected flames and smoke; the region of interest alignment uses a bilinear interpolation method to replace the traditional quantization operation so as to reduce the error.
Further, in a frame of fire image, the calculation formula of the area of the flame class is as follows: s (flame class) ═ S (boundary box) × (Num (mask of flame class)/Num (boundary box)); in the formula, S (flame) is the area of flame, S (boundary box) is the area of boundary box, Num (flame mask) is the number of pixel points of flame mask, Num (boundary box) is the number of pixel points of boundary box.
In this embodiment, S (bounding box) is calculated by taking the pixel value coordinates of two diagonally opposite points of the bounding box. And sampling horizontal and vertical pixel points at intervals in the boundary frame by taking the step length as 2, wherein the number of horizontal sampling points in the boundary frame region is a, the number of vertical sampling points in the boundary frame region is b, and then Num (boundary frame) is a. And matching the coordinates of all sampling points according to the pixel point coordinate values of the flame type mask region, and counting the total number of the overlapped sampling points as Num (flame type mask).
In step S200, if it is determined that the fire is in the initial stage, an unmanned aerial vehicle fire extinguishing task is established, otherwise, an artificial fire fighting is notified.
It should be noted that the fire image is taken by the fixed-wing drone 31 or the multi-rotor drone 32 at the same height relative to the ground level. This makes it possible to make the areas of the flames calculated uniform, and to reduce errors without requiring normalization before averaging.
Further, in step S300, the step of dispatching the multiple multi-rotor drones 32 in batches according to the autonomous decision algorithm for fire extinguishing specifically includes the steps of:
the control background 10 calculates the distance and flight time of each multi-rotor unmanned aerial vehicle 32 to the task site of the new task; certainly, the multi-rotor unmanned aerial vehicle 32 in the task state does not fall into the calculation range, so that the calculation efficiency is improved;
sequentially sequencing the multi-rotor unmanned aerial vehicles 32 meeting the requirements from near to far according to the distance from the fire place, and then adding the sequenced multi-rotor unmanned aerial vehicles into a task queue, wherein the multi-rotor unmanned aerial vehicles 32 meeting the requirements are the multi-rotor unmanned aerial vehicles 32 which are provided with resources capable of completing the fire extinguishing task of the unmanned aerial vehicles and are in a task-free state; the multi-rotor drone 32, equipped with resources able to accomplish the drone fire fighting task, satisfies the following conditions: the fire extinguishing bomb is provided, the maximum flight distance provided by the stored electric quantity is greater than the sum of the distance from the fire place to the fire place and the distance from the fire place to the nearest ground control station 20, and the maximum flight time provided by the stored electric quantity is greater than the sum of the flight time from the fire place to the nearest ground control station 20;
and sending execution instructions to the multi-rotor unmanned aerial vehicles 32 in the task queue according to the set time interval until the fire extinguishing task of the unmanned aerial vehicles is cancelled. Specifically, in this embodiment, a group of 4 multi-rotor drones 32 are dispatched in a mission queue to perform a mission in batches of a group of multi-rotor drones 32.
For step S400, during the task executed by the multi-rotor drone 32, the control back station 10 receives the second fire scene image returned after the multi-rotor drone 32 throws the fire extinguishing bomb;
for step S500, processing the second fire scene image by using a deep learning algorithm and judging the fire, if the fire is judged to be in a non-initial stage, notifying artificial fire fighting, and if not, continuously dispatching the multi-rotor unmanned aerial vehicle 32 to extinguish the fire until the fire is judged to be extinguished;
and step S600, when the fire is judged to be extinguished, the fire extinguishing task of the unmanned aerial vehicle is cancelled.
In addition, the multi-rotor drone 32 that has thrown the fire-extinguishing bomb will return to the nearest ground control station 20. Similarly, a multi-rotor drone 32 traveling to a fire location may return to the nearest ground control station 20 upon receiving a signal from the drone that a fire suppression mission is being signaled.
In this embodiment, the fixed-wing drone 31 performs fire inspection, and returns a fire scene image and coordinates of a fire scene once a fire is found; utilize the deep learning algorithm to judge the intensity of a fire according to the scene of a fire image and make the decision, dispatch many rotor unmanned aerial vehicle 32 in batches and put out a fire, the unmanned aerial vehicle of dispatching can passback the scene of a fire image immediately after throwing the fire extinguishing bomb, control backstage 10 judges the intensity of a fire once more, whether need continue to dispatch many rotor unmanned aerial vehicle 32 according to the intensity of a fire decision and put out a fire or inform artifical fire control. The multiple fixed-wing unmanned aerial vehicles 31 can continuously inspect forests in a large range and find fires in time; the multi-rotor unmanned aerial vehicle 32 can quickly cross the forest to reach the fire scene for fire extinguishing, is suitable for small-sized fire, and is safe and quick; and dispatching the unmanned aerial vehicle to execute tasks in batches, and judging the fire behavior in real time according to the real-time returned fire scene pictures, so that the arrangement is reasonable, and the waste of resources is reduced.
Referring to fig. 3, another embodiment of the present invention provides a forest fire extinguishing system based on a cluster of unmanned aerial vehicles, for performing the forest fire extinguishing method based on the cluster of unmanned aerial vehicles, including a control background 10 and a cluster of unmanned aerial vehicles, where the cluster of unmanned aerial vehicles includes a plurality of fixed-wing unmanned aerial vehicles 31 and a plurality of multi-rotor unmanned aerial vehicles 32;
the control background 10 includes:
the communication module 11 is configured to receive a first fire scene image and coordinates of a fire scene returned by the fixed-wing drone 31, and a second fire scene image returned by the multi-rotor drone 32 after throwing a fire extinguishing bomb;
the image processing module 12 is configured to process the first fire scene image and the second fire scene image by using a deep learning algorithm;
the fire behavior judging module 13 is used for judging the fire behavior according to the image output by the image processing module 12;
the task processing module 14 is used for establishing and canceling the fire extinguishing task of the unmanned aerial vehicle according to the judgment result of the fire judgment module 13;
and an autonomous decision module 15 for dispatching a plurality of multi-rotor drones 32 for fire suppression according to an autonomous decision algorithm.
Further, forest fire extinguishing system based on unmanned aerial vehicle cluster still includes ground control station 20, and ground control station 20 is used for relaying control backstage 10 and the interactive information between the unmanned aerial vehicle cluster and is used for supplementing resource and fire extinguishing bomb for the unmanned aerial vehicle cluster.
Further, the fixed-wing drone 31 and the multi-rotor drone 32 are provided with cameras, GPS positioning modules, networking modules, automatic obstacle avoidance modules, and height measurement devices; the multi-rotor drone 32 is also provided with a fire extinguishing bomb throwing device.
The fixed-wing unmanned aerial vehicle 31 and the multi-rotor unmanned aerial vehicle 32 realize automatic obstacle avoidance in the flight process through the automatic obstacle avoidance module; specifically, the automatic obstacle avoidance module comprises a binocular stereoscopic vision camera and a radar.
Further, the height measuring device comprises a sonar device for measuring the distance from the ground level, an accelerometer for measuring vertical acceleration, a barometer for measuring atmospheric pressure and a height calculating module for calculating the current height by combining data collected by the sonar device, the accelerometer and the barometer.
In another embodiment of the invention, a storage medium is provided, the storage medium storing executable instructions for causing a cluster of computer-controlled drones to perform the forest fire fighting method as described above.
Examples of storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and the present invention shall fall within the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means.
Claims (10)
1. A forest fire extinguishing method based on an unmanned aerial vehicle cluster is characterized in that the unmanned aerial vehicle cluster comprises a plurality of fixed wing unmanned aerial vehicles for inspecting fire and a plurality of multi-rotor unmanned aerial vehicles for throwing fire extinguishing bombs to extinguish fire, and the method comprises the following steps:
receiving a first fire scene image and a fire scene coordinate returned by the fixed-wing unmanned aerial vehicle;
processing the first fire scene image by using a deep learning algorithm and judging the fire, if the fire is judged to be in an initial stage, establishing an unmanned aerial vehicle fire extinguishing task, otherwise, notifying artificial fire fighting;
dispatching a plurality of multi-rotor unmanned aerial vehicles in batches according to an autonomous decision algorithm to extinguish the fire;
receiving a second fire scene image returned by a plurality of multi-rotor unmanned aerial vehicles after the fire extinguishing bomb is thrown;
processing the second fire scene image by using a deep learning algorithm and judging the fire behavior, if the fire behavior is judged to be in a non-initial stage, notifying manual fire fighting, and if not, continuously dispatching the multi-rotor unmanned aerial vehicle to extinguish the fire until the fire behavior is judged to be extinguished;
and when the fire is judged to be extinguished, the fire extinguishing task of the unmanned aerial vehicle is cancelled.
2. A forest fire fighting method based on unmanned aerial vehicle cluster as claimed in claim 1, wherein the processing of fire images and the judgment of fire behavior using a deep learning algorithm comprises the steps of:
preprocessing the fire image;
inputting the preprocessed fire image into a Mask R-CNN network model trained by a large number of fire sample images comprising a plurality of stages of fire behaviors to identify a flame class and a smoke class;
if the continuous x frames of fire images obtained by identification are only smoke, judging the fire images to be in an initial stage;
if the fire images are identified to have flames, judging the fire condition according to the average area of the flames in the continuous x-frame fire images;
wherein the fire image comprises the first fire scene image or the second fire scene image, and the fire condition comprises the initial stage and the non-initial stage.
3. A forest fire fighting method based on unmanned aerial vehicle clusters as claimed in claim 2 wherein the pre-processing comprises image enhancement, smoothing filtering, edge detection and image segmentation.
4. A forest fire extinguishing method based on unmanned aerial vehicle cluster as claimed in claim 2, wherein the calculation formula of the area of flame class in one frame of fire image is as follows:
s (flame class) ═ S (boundary box) × (Num (mask of flame class)/Num (boundary box));
in the formula, S (flame) is the area of flame, S (boundary box) is the area of boundary box, Num (flame mask) is the number of pixel points of flame mask, Num (boundary box) is the number of pixel points of boundary box.
5. A forest fire fighting method based on a cluster of drones as defined in claim 4, characterized in that said fire images are taken with the fixed wing drone or the multi-rotor drone at the same height with respect to the ground level.
6. A forest fire fighting method based on a cluster of drones as defined in claim 1, wherein said batch dispatching of multiple multi-rotor drones for fire fighting according to an autonomous decision algorithm comprises in particular the following steps:
respectively calculating the distance and the flight time of each multi-rotor unmanned aerial vehicle to reach the task place of the new task;
sequentially sequencing the multi-rotor unmanned aerial vehicles meeting the requirements from near to far according to the distance from the fire place, and then adding the multi-rotor unmanned aerial vehicles into a task queue, wherein the multi-rotor unmanned aerial vehicles meeting the requirements are multi-rotor unmanned aerial vehicles which are provided with resources capable of completing the fire extinguishing task of the unmanned aerial vehicles and are in a task-free state;
and sending an execution instruction to the multi-rotor unmanned aerial vehicles in the task queue according to a set time interval until the fire extinguishing task of the unmanned aerial vehicles is cancelled.
7. The forest fire extinguishing system based on the unmanned aerial vehicle cluster is characterized by comprising a control background and the unmanned aerial vehicle cluster, wherein the unmanned aerial vehicle cluster comprises a plurality of fixed-wing unmanned aerial vehicles and a plurality of multi-rotor unmanned aerial vehicles;
the control background comprises:
the communication module is used for receiving a first fire scene image returned by the fixed-wing unmanned aerial vehicle, coordinates of a fire scene and a second fire scene image returned by the multi-rotor unmanned aerial vehicle after the multi-rotor unmanned aerial vehicle throws a fire extinguishing bomb;
the image processing module is used for processing the first fire scene image and the second fire scene image by utilizing a deep learning algorithm;
the fire behavior judging module is used for judging the fire behavior according to the image output by the image processing module;
the task processing module is used for establishing and canceling an unmanned aerial vehicle fire extinguishing task according to the judgment result of the fire judgment module;
and the autonomous decision module is used for dispatching a plurality of multi-rotor unmanned aerial vehicles to extinguish fire according to an autonomous decision algorithm.
8. The forest fire suppression system based on unmanned aerial vehicle cluster of claim 7, further comprising a ground control station for relaying information interacted between the control back office and the unmanned aerial vehicle cluster and for supplementing resources and fire extinguishing bombs for the unmanned aerial vehicle cluster.
9. A forest fire extinguishing system based on unmanned aerial vehicle cluster according to claim 7 or 8, characterized in that the fixed wing unmanned aerial vehicles and the multi-rotor unmanned aerial vehicles are provided with cameras, GPS positioning modules, networking modules, automatic obstacle avoidance modules and height measuring devices; many rotor unmanned aerial vehicle still is equipped with fire extinguishing bomb throwing device.
10. A forest fire suppression system based on unmanned aerial vehicle cluster according to claim 9, characterized in that the height measurement device includes a sonar device for measuring distance from ground level, an accelerometer for measuring vertical acceleration, a barometer for measuring atmospheric pressure, and a height calculation module for calculating the current height in combination with data collected by the sonar device, the accelerometer and the barometer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010164595.XA CN111402541A (en) | 2020-03-11 | 2020-03-11 | Forest fire extinguishing method and system based on unmanned aerial vehicle cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010164595.XA CN111402541A (en) | 2020-03-11 | 2020-03-11 | Forest fire extinguishing method and system based on unmanned aerial vehicle cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111402541A true CN111402541A (en) | 2020-07-10 |
Family
ID=71428722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010164595.XA Pending CN111402541A (en) | 2020-03-11 | 2020-03-11 | Forest fire extinguishing method and system based on unmanned aerial vehicle cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111402541A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112327914A (en) * | 2020-10-30 | 2021-02-05 | 广东工业大学 | Indoor collaborative path planning method and system for high-rise fire-fighting multi-unmanned aerial vehicle |
| CN112365673A (en) * | 2020-11-12 | 2021-02-12 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112462778A (en) * | 2020-11-30 | 2021-03-09 | 国网四川省电力公司达州供电公司 | Operation method for man-machine cooperative inspection of power distribution network |
| CN112915421A (en) * | 2021-01-21 | 2021-06-08 | 湖南林科达农林技术服务有限公司 | Unmanned aerial vehicle-based forest fire retardant spraying method and unmanned aerial vehicle |
| CN112933481A (en) * | 2021-03-16 | 2021-06-11 | 中国地质大学（武汉） | Fire-fighting early warning robot, system and method based on image analysis |
| CN113409642A (en) * | 2021-06-28 | 2021-09-17 | 山东科技大学 | Small-size fire smoke flow simulation experiment and numerical simulation combined system |
| CN113426044A (en) * | 2021-07-07 | 2021-09-24 | 广东诺海科技集团有限公司 | Forest fire extinguishing method for cluster unmanned aerial vehicle |
| CN113947863A (en) * | 2021-10-13 | 2022-01-18 | 上海翼枭航空科技有限公司 | Remote control method and system for unmanned aerial vehicle |
| CN114063638A (en) * | 2021-10-15 | 2022-02-18 | 中电科翌智航(宁夏)科技有限公司 | Unmanned aerial vehicle patrolling system and smart energized city emergency equipment |
| CN114815870A (en) * | 2022-05-06 | 2022-07-29 | 成都软智科技有限公司 | Extensible multi-unmanned aerial vehicle cooperation fire processing method and system |
| CN114949663A (en) * | 2022-05-13 | 2022-08-30 | 成都软智科技有限公司 | Many unmanned aerial vehicle fire extinguishing system |
| CN115240347A (en) * | 2022-06-02 | 2022-10-25 | 中国人民解放军军事科学院国防科技创新研究院 | Self-maintaining robot cluster system for fire monitoring and control |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065413A (en) * | 2012-12-13 | 2013-04-24 | 中国电子科技集团公司第十五研究所 | Method and device of acquiring fire class information |
| KR101259478B1 (en) * | 2012-08-13 | 2013-05-06 | 이한주 | Fire suppression system |
| CN105719421A (en) * | 2016-04-27 | 2016-06-29 | 丛静华 | Big data mining based integrated forest fire prevention informatization system |
| CN106390331A (en) * | 2016-10-11 | 2017-02-15 | 北京电子工程总体研究所 | High-rise building fire extinguishing system based on multi-rotor-wing unmanned aerial vehicle (UAV) |
| CN106422115A (en) * | 2016-10-11 | 2017-02-22 | 北京电子工程总体研究所 | Remote control fire extinguishing method for high-rise buildings |
| WO2017137393A1 (en) * | 2016-02-10 | 2017-08-17 | Tyco Fire & Security Gmbh | A fire detection system using a drone |
| CN207085114U (en) * | 2017-04-28 | 2018-03-13 | 成都谍翼科技有限公司 | Clustered control fire-fighting system |
| CN108241349A (en) * | 2018-01-15 | 2018-07-03 | 梁晓龙 | Fire-fighting unmanned plane group system and fire-fighting method |
| CN108806162A (en) * | 2018-07-24 | 2018-11-13 | 烟台伺维特智能科技有限公司 | A kind of forest fire protection and extinguishing system and method for unmanned aerial vehicle platform |
| CN208444408U (en) * | 2018-07-24 | 2019-01-29 | 烟台伺维特智能科技有限公司 | A kind of forest fire protection and extinguishing system of unmanned aerial vehicle platform |
| CN110232380A (en) * | 2019-06-13 | 2019-09-13 | 应急管理部天津消防研究所 | Fire night scenes restored method based on Mask R-CNN neural network |
| KR20200002271A (en) * | 2018-06-29 | 2020-01-08 | (주)샤픈고트 | Method and apparatus for fire suppression using UAV(unmanned aerial vehicle) |
| CN110841220A (en) * | 2019-12-09 | 2020-02-28 | 国网智能科技股份有限公司 | Intelligent fire-fighting system and method for transformer substation |
| CN110841217A (en) * | 2019-11-20 | 2020-02-28 | 国网安徽省电力有限公司蚌埠供电公司 | Substation waterless fire-fighting system based on ubiquitous Internet of things technology |
-
2020
- 2020-03-11 CN CN202010164595.XA patent/CN111402541A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101259478B1 (en) * | 2012-08-13 | 2013-05-06 | 이한주 | Fire suppression system |
| CN103065413A (en) * | 2012-12-13 | 2013-04-24 | 中国电子科技集团公司第十五研究所 | Method and device of acquiring fire class information |
| WO2017137393A1 (en) * | 2016-02-10 | 2017-08-17 | Tyco Fire & Security Gmbh | A fire detection system using a drone |
| CN105719421A (en) * | 2016-04-27 | 2016-06-29 | 丛静华 | Big data mining based integrated forest fire prevention informatization system |
| CN106390331A (en) * | 2016-10-11 | 2017-02-15 | 北京电子工程总体研究所 | High-rise building fire extinguishing system based on multi-rotor-wing unmanned aerial vehicle (UAV) |
| CN106422115A (en) * | 2016-10-11 | 2017-02-22 | 北京电子工程总体研究所 | Remote control fire extinguishing method for high-rise buildings |
| CN207085114U (en) * | 2017-04-28 | 2018-03-13 | 成都谍翼科技有限公司 | Clustered control fire-fighting system |
| CN108241349A (en) * | 2018-01-15 | 2018-07-03 | 梁晓龙 | Fire-fighting unmanned plane group system and fire-fighting method |
| KR20200002271A (en) * | 2018-06-29 | 2020-01-08 | (주)샤픈고트 | Method and apparatus for fire suppression using UAV(unmanned aerial vehicle) |
| CN108806162A (en) * | 2018-07-24 | 2018-11-13 | 烟台伺维特智能科技有限公司 | A kind of forest fire protection and extinguishing system and method for unmanned aerial vehicle platform |
| CN208444408U (en) * | 2018-07-24 | 2019-01-29 | 烟台伺维特智能科技有限公司 | A kind of forest fire protection and extinguishing system of unmanned aerial vehicle platform |
| CN110232380A (en) * | 2019-06-13 | 2019-09-13 | 应急管理部天津消防研究所 | Fire night scenes restored method based on Mask R-CNN neural network |
| CN110841217A (en) * | 2019-11-20 | 2020-02-28 | 国网安徽省电力有限公司蚌埠供电公司 | Substation waterless fire-fighting system based on ubiquitous Internet of things technology |
| CN110841220A (en) * | 2019-12-09 | 2020-02-28 | 国网智能科技股份有限公司 | Intelligent fire-fighting system and method for transformer substation |
Non-Patent Citations (1)
| Title |
|---|
| 李梁等: "改进Mask R-CNN在航拍灾害检测的应用研究", 《计算机工程与应用》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112327914A (en) * | 2020-10-30 | 2021-02-05 | 广东工业大学 | Indoor collaborative path planning method and system for high-rise fire-fighting multi-unmanned aerial vehicle |
| CN112327914B (en) * | 2020-10-30 | 2023-08-22 | 广东工业大学 | Indoor collaborative path planning method and system for high-rise fire-fighting multiple unmanned aerial vehicles |
| CN112365673A (en) * | 2020-11-12 | 2021-02-12 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112365673B (en) * | 2020-11-12 | 2022-08-02 | 光谷技术有限公司 | Forest fire monitoring system and method |
| CN112462778A (en) * | 2020-11-30 | 2021-03-09 | 国网四川省电力公司达州供电公司 | Operation method for man-machine cooperative inspection of power distribution network |
| CN112915421B (en) * | 2021-01-21 | 2022-05-03 | 湖南林科达农林技术服务有限公司 | Unmanned aerial vehicle-based forest fire retardant spraying method and unmanned aerial vehicle |
| CN112915421A (en) * | 2021-01-21 | 2021-06-08 | 湖南林科达农林技术服务有限公司 | Unmanned aerial vehicle-based forest fire retardant spraying method and unmanned aerial vehicle |
| CN112933481A (en) * | 2021-03-16 | 2021-06-11 | 中国地质大学（武汉） | Fire-fighting early warning robot, system and method based on image analysis |
| CN113409642A (en) * | 2021-06-28 | 2021-09-17 | 山东科技大学 | Small-size fire smoke flow simulation experiment and numerical simulation combined system |
| CN113426044B (en) * | 2021-07-07 | 2022-04-26 | 广东诺海科技集团有限公司 | Forest fire extinguishing method for cluster unmanned aerial vehicle |
| CN113426044A (en) * | 2021-07-07 | 2021-09-24 | 广东诺海科技集团有限公司 | Forest fire extinguishing method for cluster unmanned aerial vehicle |
| CN113947863A (en) * | 2021-10-13 | 2022-01-18 | 上海翼枭航空科技有限公司 | Remote control method and system for unmanned aerial vehicle |
| CN114063638A (en) * | 2021-10-15 | 2022-02-18 | 中电科翌智航(宁夏)科技有限公司 | Unmanned aerial vehicle patrolling system and smart energized city emergency equipment |
| CN114815870A (en) * | 2022-05-06 | 2022-07-29 | 成都软智科技有限公司 | Extensible multi-unmanned aerial vehicle cooperation fire processing method and system |
| CN114949663A (en) * | 2022-05-13 | 2022-08-30 | 成都软智科技有限公司 | Many unmanned aerial vehicle fire extinguishing system |
| CN115240347A (en) * | 2022-06-02 | 2022-10-25 | 中国人民解放军军事科学院国防科技创新研究院 | Self-maintaining robot cluster system for fire monitoring and control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111402541A (en) | Forest fire extinguishing method and system based on unmanned aerial vehicle cluster | |
| CN105913604B (en) | Assay method and its device occur for the fire based on unmanned plane | |
| CN110069071B (en) | Unmanned aerial vehicle navigation method and device, storage medium and electronic equipment | |
| CN106054928B (en) | A kind of full region fire generation measuring method based on unmanned plane network | |
| CN111679695B (en) | Unmanned aerial vehicle cruising and tracking system and method based on deep learning technology | |
| KR102263307B1 (en) | System and method for controlling cluster flight of unmanned aerial vehicle | |
| CN111508181A (en) | Forest fire prevention system based on multiple unmanned aerial vehicles and method thereof | |
| CN111796603A (en) | Smoke inspection unmanned aerial vehicle system, inspection detection method and storage medium | |
| KR20190004176A (en) | Apparatus and method for the obstacle collision avoidance of unmanned aerial vehicle | |
| CN114200471B (en) | Forest fire source detection system and method based on unmanned aerial vehicle, storage medium and equipment | |
| KR102131723B1 (en) | System for monitoring and suppressing fires using unmanned aircraft and Driving method thereof | |
| Kasyap et al. | Early detection of forest fire using mixed learning techniques and UAV | |
| CN106168807A (en) | The flight control method of a kind of aircraft and flight control assemblies | |
| CN113268075A (en) | Unmanned aerial vehicle control method and system | |
| CN114283548A (en) | Fire continuous monitoring method and system for unmanned aerial vehicle | |
| CN116820128B (en) | Automatic patrol system for realizing large forest | |
| CN112818964A (en) | Unmanned aerial vehicle detection method based on FoveaBox anchor-free neural network | |
| Minwalla et al. | Experimental evaluation of PICAS: An electro-optical array for non-cooperative collision sensing on unmanned aircraft systems | |
| CN112365763A (en) | Unmanned aerial vehicle inspection training method and system for power equipment | |
| CN111243270A (en) | Accident processing method and device, electronic equipment and computer readable storage medium | |
| CN114534134B (en) | Online unmanned full-automatic fire prevention rescue unmanned aerial vehicle device and system that puts out a fire | |
| CN115588267A (en) | Natural fire early warning and response system based on remote sensing | |
| CN113778134A (en) | Ground station for coal mine environment | |
| JP7130409B2 (en) | Control device | |
| Cheng | Research on the application of computer vision technology in power system UAV line inspection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200710 |
|
| RJ01 | Rejection of invention patent application after publication |