CN111307291A - Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle - Google Patents
Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle Download PDFInfo
- Publication number
- CN111307291A CN111307291A CN202010134618.2A CN202010134618A CN111307291A CN 111307291 A CN111307291 A CN 111307291A CN 202010134618 A CN202010134618 A CN 202010134618A CN 111307291 A CN111307291 A CN 111307291A
- Authority
- CN
- China
- Prior art keywords
- temperature
- camera
- coordinate system
- unmanned aerial
- aerial vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000002159 abnormal effect Effects 0.000 claims abstract description 36
- 238000006243 chemical reaction Methods 0.000 claims abstract description 35
- 230000005856 abnormality Effects 0.000 claims abstract description 30
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 claims description 42
- 230000003287 optical effect Effects 0.000 claims description 42
- 230000005540 biological transmission Effects 0.000 claims description 31
- 230000000007 visual effect Effects 0.000 claims description 17
- 238000009529 body temperature measurement Methods 0.000 claims description 6
- 230000001788 irregular Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims 2
- 230000010354 integration Effects 0.000 abstract description 2
- 238000011897 real-time detection Methods 0.000 abstract 1
- 230000018109 developmental process Effects 0.000 description 17
- 230000006870 function Effects 0.000 description 16
- 238000011161 development Methods 0.000 description 12
- 230000003993 interaction Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000630 rising effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000010259 detection of temperature stimulus Effects 0.000 description 1
- 230000010006 flight Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- WABPQHHGFIMREM-UHFFFAOYSA-N lead(0) Chemical compound [Pb] WABPQHHGFIMREM-UHFFFAOYSA-N 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/007—Radiation pyrometry, e.g. infrared or optical thermometry for earth observation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/04—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
- G01J5/485—Temperature profile
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Automation & Control Theory (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental & Geological Engineering (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geology (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a surface temperature anomaly detection and positioning method, device and system based on an unmanned aerial vehicle, belonging to the field of integration and application of intelligent microsystems, and the method is realized by the following steps: controlling the unmanned aerial vehicle to fly to a target area according to the visible light data, and acquiring infrared data of the target area; judging whether temperature abnormality exists in the infrared data, and if the temperature abnormality exists, acquiring pixel coordinates of the temperature abnormality; obtaining coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly; and converting the coordinates of the temperature anomaly in the world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly. The invention can realize real-time detection and positioning of abnormal targets and output accurate detection results.
Description
Technical Field
The invention belongs to the field of integration and application of intelligent microsystems, and particularly relates to a surface temperature anomaly detection and positioning method, device and system based on an unmanned aerial vehicle.
Background
Since the advent of unmanned aerial vehicles, unmanned aerial vehicles have increasingly played an important role in military and civilian fields. At present, unmanned aerial vehicles are receiving wide attention in the fields of disaster prevention and reduction, smart cities, anomaly detection, agriculture and the like. Most of these drones have a single optical camera mounted thereon, enabling the operator to observe an abnormal situation in the screen at the ground station. However, a single visible light camera cannot be adapted for surface temperature anomaly detection. The surface temperature is an important parameter reflecting the surface environment, and has wide application in the aspects of farmland, urban heating pipelines, fire detection and the like. The infrared camera can receive infrared heat radiation emitted by a ground target, and the image color can reflect the earth surface temperature of an area, but the defects of missing texture details and the like exist. Most unmanned aerial vehicles on the existing market generally only have a single camera, and the requirement of urban earth surface temperature anomaly detection cannot be met. Therefore, the anomaly detection of the surface temperature urgently needs to combine aerial video images of the visible light camera and the infrared camera to make up for the defect of a single imaging sensor.
It is far from not enough to observe the abnormality in the picture, and in order to achieve the purpose of detecting and positioning the surface temperature abnormality, the target needs to be detected and positioned in real time so as to be beneficial to timely processing the abnormal target. The abnormality detection can be carried out by means of images observed by an infrared camera, but most of the existing temperature abnormality detection methods only rely on the function of detecting overhigh temperature by the infrared camera, and the application range is limited. For target positioning, the method can be divided into active and passive modes according to the working mechanism. Active positioning means that the unmanned aerial vehicle carries other auxiliary devices such as laser range finder, acquires the depth information or other auxiliary information of target. Passive positioning refers to positioning a target using only flight parameters obtained by sensors such as a Global Positioning System (GPS) and an Inertial Measurement Unit (IMU). In outdoor operation, the active target positioning algorithm has the problems of too high hardware cost, overweight and the like, and is not suitable for a small unmanned aerial vehicle, so that a geographical positioning technology needs to be carried out on the target with the abnormal surface temperature under a passive condition.
In the unusual detection of actual earth's surface temperature, in order to realize real-time unusual target detection and location, need carry out real-time processing to airborne sensor observation data, the accurate testing result of output, therefore, how to reform transform current unmanned aerial vehicle observation platform, design unmanned aerial vehicle earth's surface temperature unusual detection and positioning system, how on unmanned aerial vehicle, the integrated all kinds of sensor time synchronization are observed, earth's surface temperature unusual detection and location, main functions such as multisource data download, the realization is to earth's surface temperature unusual rapid detection and location, and the sharing of detection information is the technological problem that needs to solve at present urgently.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides a ground surface temperature anomaly detection and positioning method, device and system based on an unmanned aerial vehicle, so that the technical problems of high cost, limitation on detection precision and the like of the existing unmanned aerial vehicle in ground surface temperature anomaly monitoring and positioning are solved.
In order to achieve the above object, according to one aspect of the present invention, there is provided a surface temperature anomaly detection and location method based on an unmanned aerial vehicle, including:
controlling an unmanned aerial vehicle to fly to a target area according to visible light data, and acquiring infrared data of the target area;
judging whether temperature abnormality exists in the infrared data, and if the temperature abnormality exists, acquiring pixel coordinates of the temperature abnormality;
obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly;
and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly.
Preferably, the determining whether the infrared data has a temperature abnormality includes:
and judging whether an area with the temperature higher than a preset highest temperature threshold value, an area with the temperature lower than a preset lowest temperature threshold value or an area with the temperature difference larger than a highest temperature difference threshold value from the surrounding preset area exists in the infrared data.
Preferably, the determining whether there is an area with a temperature higher than a preset maximum temperature threshold and an area with a temperature lower than a preset minimum temperature threshold in the infrared data includes:
carrying out graying processing on the infrared data to obtain grayscale data;
acquiring a maximum gray value corresponding to a pixel point with the maximum gray value and a minimum gray value corresponding to a pixel point with the minimum gray value in the gray data;
establishing a temperature-gray value conversion relation according to the pixel value of the reference pixel point provided by the current infrared data and the corresponding temperature thereof, the average temperature of the temperature measurement range and the pixel value corresponding to the average temperature;
obtaining the highest temperature corresponding to the maximum gray value and the lowest temperature corresponding to the minimum gray value based on the temperature-gray value conversion relation;
if the highest temperature is greater than the preset highest temperature threshold, the temperature of the area corresponding to the maximum gray value is abnormal; if the lowest temperature is smaller than the preset lowest temperature threshold, the temperature of the area corresponding to the minimum gray value is abnormal.
Preferably, the determining whether there is an area having a temperature difference greater than a maximum temperature difference threshold from a preset area around includes:
graying the infrared data to obtain grayscale data, establishing a temperature-grayscale value conversion relation according to a pixel value of a reference pixel point provided by the current infrared data and a corresponding temperature thereof, an average temperature of a temperature measurement range and a pixel value corresponding to the average temperature, and obtaining a maximum grayscale value difference corresponding to the maximum temperature difference threshold value based on the temperature-grayscale value conversion relation;
dividing the gray data into uniform irregular pixel blocks, comparing the average gray value of each pixel block with the average gray value of the pixel blocks in the preset area around the pixel block, and if the gray value difference of the two exceeds the maximum gray value difference, determining that the temperature of the pixel block is abnormal.
Preferably, the dividing the gray data into uniform irregular pixel blocks includes:
determining initial clustering centers according to the number of pre-divided super pixels, acquiring gradient values of all pixel points in a first preset field of each initial clustering center, and moving each initial clustering center to a position with the minimum corresponding gradient value to obtain a new clustering center;
searching pixel points in a second preset neighborhood around each new clustering center, acquiring the distance between each searched target pixel point and the corresponding new clustering center, taking the new clustering center corresponding to the minimum distance as the clustering center of the target pixel point, repeating the searching operation until the clustering center is not changed any more, and determining pixel blocks belonging to each clustering center.
Preferably, the obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relationship between the camera coordinate system and the world coordinate system, the conversion relationship between the camera coordinate system and the pixel coordinate system, and the pixel coordinates of the temperature anomaly comprises:
establishing a reference coordinate system with an origin at an infrared camera optical center and three axes parallel to a world coordinate system, and obtaining an actual distance from the infrared camera optical center to the temperature anomaly position according to the reference coordinate system and the flight height of the unmanned aerial vehicle;
and obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinate system and the actual distance from the optical center of the infrared camera to the temperature anomaly.
According to another aspect of the invention, an airborne observation device based on surface temperature anomaly detection and positioning of an unmanned aerial vehicle is provided, comprising: the system comprises an unmanned aerial vehicle flight control module, a visual observation module and an abnormity detection and positioning module;
the unmanned aerial vehicle flight control module is used for controlling the unmanned aerial vehicle to fly to a target area according to a received flight control instruction, then receiving a camera control instruction and sending the camera control instruction to the visual observation module, wherein the flight control instruction is obtained by analyzing visible light data acquired by an optical camera in the visual observation module and then triggering;
the visual observation module is used for analyzing the camera control instruction, controlling an optical camera and an infrared camera in the visual observation module to execute corresponding operation and acquiring infrared data of the target area;
the abnormal detection and positioning module is used for judging whether temperature abnormality exists in the infrared data or not, and if the temperature abnormality exists, acquiring pixel coordinates of the abnormal temperature position; obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly; and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly.
Preferably, the unmanned aerial vehicle flight control module includes: the system comprises a wireless image transmission module, an airborne wireless data transceiving module, a flight control submodule and an unmanned aerial vehicle flight platform;
the wireless image transmission module is used for transmitting the optical data and the infrared data shot by the unmanned aerial vehicle to the ground control device for displaying;
the airborne wireless data transceiver module is used for receiving a remote control instruction sent by the ground control device and sending telemetering data on the unmanned aerial vehicle to the ground control device;
and the flight control submodule is used for controlling the unmanned aerial vehicle flight platform to act according to the remote control command so as to control the flight of the unmanned aerial vehicle.
Preferably, the visual observation module comprises: the system comprises a camera control module, an airborne wireless receiving module, a holder optical camera and a holder infrared camera;
the camera control module is used for receiving a camera control command sent by the flight control sub-module and controlling the holder optical camera and the holder infrared camera to execute an operation corresponding to the camera control command;
and the airborne wireless receiving module is used for receiving a camera attitude angle adjusting instruction sent by the ground control device so as to control the attitude angles of the holder optical camera and the holder infrared camera.
According to another aspect of the invention, there is provided a drone-based surface temperature anomaly detection and location system comprising: an airborne observation device and a ground control device;
the ground control device is used for receiving and displaying the optical data and the infrared data sent by the airborne observation device, receiving a flight control instruction and a camera control instruction, and sending the flight control instruction and the camera control instruction to the airborne observation device;
the airborne observation device is used for receiving the flight control instruction and controlling the unmanned aerial vehicle to fly to a target area according to the flight control instruction; receiving the camera control instruction, controlling an optical camera and an infrared camera to execute corresponding operation according to the camera control instruction, and acquiring infrared data of the target area;
the airborne observation device is also used for judging whether temperature abnormality exists in the infrared data, and if the temperature abnormality exists, the pixel coordinates of the temperature abnormality position are obtained; obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly; and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly, sending the geographical position information of the temperature anomaly to the ground control device and displaying the geographical position information on the temperature anomaly in the ground control device.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
1. the invention designs a temperature abnormal target detection and passive target geographical positioning method based on multi-sensor observation information of an unmanned aerial vehicle platform. The method can detect and position the abnormal target at a fixed position, and can also detect and position the abnormal target which moves. Firstly, establishing a temperature-gray value relation by utilizing an infrared image, and estimating a target temperature in a picture by using a pixel value; detecting three types of temperature abnormal targets by using a super-pixel segmentation method, and acquiring pixel positions of the abnormal targets on the image; and further establishing an image-camera-unmanned aerial vehicle-ground coordinate system projection model, and calculating the geographic position of the abnormal target. All detection and positioning calculation are carried out in real time on a temperature abnormity detection and positioning module of the onboard observation device, an accurate detection result is output, and information of a monitored target and the geographical position of the monitored target is downloaded to a ground control device for a user to refer.
2. The invention designs a method for controlling an infrared camera and an optical camera to jointly observe the earth surface temperature. And simultaneously transmitting the optical image and the infrared image acquired by the airborne observation device to the ground control device, and respectively displaying the optical image and the infrared image on the mobile terminal equipment of the ground control device and the HDMI display of the remote controller. The user can observe and control the visible light camera and the infrared camera on the displays of the mobile terminal device and the remote controller in the ground control apparatus, respectively. The visible light camera provides a normal flight visual field, and assists in controlling the flight of the unmanned aerial vehicle by matching with real-time flight data, so that a user can control the unmanned aerial vehicle to fly to an area where surface temperature abnormality may exist; the infrared camera is used for observing a thermal imaging picture of a target earth surface area and monitoring abnormal conditions. The process comprises a set of camera control protocol, and after the onboard camera control module receives a command from the ground remote controller, the onboard camera control module analyzes the command according to the content of the command frame and executes the corresponding command. The protocol provided by the invention supports extension, and a new data transmission protocol can be defined for the newly added equipment at any time under the condition that hardware conditions are met.
3. The invention designs an earth surface temperature anomaly detection system based on an unmanned aerial vehicle platform. The system comprises an unmanned aerial vehicle airborne observation device and an unmanned aerial vehicle ground control device. Unmanned aerial vehicle machine carries observation device and is entire system's sky end, include: unmanned aerial vehicle main part framework, observation sensor, have two kinds of cloud platform cameras of infrared and optics of cloud platform and temperature anomaly detection and orientation module, unmanned aerial vehicle detects and realizes through temperature anomaly detection and orientation module to the geographical location of target in unusual region. The ground control device includes: take remote controller and mobile terminal of display, the user can pass through remote controller control unmanned aerial vehicle's flight and cruise at ground controlling means, simultaneously, observes two kinds of videos respectively on the display of mobile terminal and remote controller. When the user shoots on the ground control optical or infrared camera, the ground remote controller transmits the time stamp and the position of the unmanned aerial vehicle longitude and latitude, attitude angle, relative height information and abnormal target which are transmitted to the ground control device by the airborne observation device through the wireless link, and the position can be recorded in the mobile terminal device in the form of a relational database. The time stamp is used for matching and corresponding the optical and infrared camera images, thus ensuring the integrity of the information and the one-to-one correspondence relationship.
Drawings
FIG. 1 is a system architecture diagram provided by an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a method for detecting and locating temperature anomaly according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a data transmission process according to an embodiment of the present invention;
fig. 4 is a schematic view of a video transmission flow according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating a communication flow of camera control commands according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of an implementation method of a camera control module according to an embodiment of the present invention;
fig. 7 is a model diagram of an imaging projection of a pan-tilt camera of an unmanned aerial vehicle according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In the present examples, "first", "second", "third", etc. are used for distinguishing different objects, and are not necessarily used for describing a particular order or sequence.
The Mobile terminal device related to the embodiment of the present application may be a Mobile terminal having a display screen and a networking function, and the Mobile terminal may include various smart phones, tablet computers, or computing devices, and the like, as well as various forms of User Equipment (UE), Mobile Station (MS), terminal device (terminal device), and the like.
Fig. 1 is a schematic structural diagram of a surface temperature anomaly detection and positioning system based on an unmanned aerial vehicle according to an embodiment of the present invention. The method comprises the following steps: unmanned aerial vehicle machine carries observation device and unmanned aerial vehicle ground control device. The unmanned aerial vehicle airborne observation device mainly comprises three modules of flight control, visual observation and abnormity detection and positioning of the unmanned aerial vehicle;
1) unmanned aerial vehicle flight control module includes: the system comprises a wireless image transmission module, an airborne wireless data transceiving module, a flight control module and an unmanned aerial vehicle flight platform;
the wireless transmission link of the unmanned aerial vehicle is divided into a remote control link and an image transmission link, wherein the downlink image transmission link adopts a wireless image transmission module and is responsible for transmitting videos and images shot by a camera on the unmanned aerial vehicle to a ground control device for displaying. And the remote control link is used for data communication between the ground remote controller and the airborne wireless data transceiver module and is responsible for transmitting uplink remote control commands and downlink remote measurement data between the ground control device and the unmanned aerial vehicle. At present, a remote controller of an unmanned aerial vehicle is provided with a remote control link function module, a special image transmission device needs to be additionally arranged on an image transmission link, and when the remote control link and the image transmission link are used simultaneously, the problem of interference possibly generated between an uplink link and a downlink link needs to be considered. Therefore, a combined uplink and downlink mapping device, such as a DJI Lightbridge2 device, may be employed. The device adopts a time division multiplexing mechanism, and the uplink and downlink data links adopt different frequency hopping technologies and signal modulation modes, so that the device is completely independent, and can upload remote sensing signals and download map transmission signals in the same frequency band (such as 2.4 GHz).
The wireless image transmission module and the airborne wireless data receiving and transmitting module are used for video transmission from the airborne observation device to the ground control device and data transmission between the ground control device and the airborne observation device. In an embodiment of the present invention, the DJI Lightbridge2 sky module may be used for transmission. The unmanned aerial vehicle flight control submodule is used for controlling the flight of the whole unmanned aerial vehicle, is connected with the airborne wireless data transceiver module and carries out two-way data communication with the ground remote controller. The unmanned aerial vehicle flight platform comprises a hardware system necessary for the unmanned aerial vehicle to fly and also comprises various sensors such as a GPS (global positioning system), an IMU (inertial measurement unit) and the like. The unmanned aerial vehicle flight platform is controlled by the flight control submodule, and can transmit the position information and the attitude information of the unmanned aerial vehicle to the flight control submodule.
2) The unmanned aerial vehicle visual observation module utilizes the transparent transmission function of data between airborne observation device and the ground control device, makes infrared camera and optical camera can be by ground control device remote control, carries out the earth's surface temperature and observes. The unmanned aerial vehicle visual observation module consists of an unmanned aerial vehicle holder infrared camera, an unmanned aerial vehicle holder optical camera, a camera control module and an airborne wireless receiving module.
The software development kit can be specifically implemented by an SDK, where the SDK is a software development kit used for developing software suitable for a certain product or a certain platform, and is generally provided by a manufacturer of the product or the platform for a developer to use. The Onboard SDK is a software development kit for controlling the drone, provided by DJI, running on an embedded development board that carries the observation devices. The camera control module runs on the development board through the Onboard SDK and interacts data with the flight control sub-module through the serial communication interface. The flight control sub-module transmits a camera control command, and the camera control module analyzes the command according to a formulated command protocol, controls the infrared camera and the optical camera to execute corresponding functions and completes related operations. The airborne wireless receiving module is communicated with a ground wireless transmitting module of the ground control device of the unmanned aerial vehicle, and can be used for controlling the attitude angle of the optical camera and the infrared camera.
3) The temperature anomaly detection and anomaly positioning module is a core part of the whole airborne observation device. For detecting a target with an abnormal temperature and calculating its geographical location. The temperature anomaly detection and anomaly positioning module can be realized by an embedded development board of an onboard observation device in hardware. Temperature anomaly detection and orientation module can acquire unmanned aerial vehicle's flight data to and the video and the image picture of unmanned aerial vehicle cloud platform camera shooting, supply anomaly detection and location to use, then, the geographical position of the target that appears anomaly will be sent to ground controlling means through machine carries wireless data transceiver module.
4) The ground control device comprises an unmanned aerial vehicle image transmission control module and a holder attitude control module. The unmanned aerial vehicle image transmission control module comprises a human-computer interaction interface and a ground wireless data transceiver module. The holder attitude control module comprises a mobile display screen and a ground wireless transmitting module.
In the embodiment of the present invention, the system may specifically include a ground side of the DJI Lightbridge2, a mobile terminal device, a mobile HDMI display screen, and a universal model airplane remote controller. The Mobile SDK is a software development kit DJI for controlling the unmanned aerial vehicle, which is developed autonomously, and runs on the Mobile terminal device of the ground control device. The mobile terminal equipment can be a tablet personal computer, and a human-computer interaction interface is designed. Through this interface user can control optics and infrared camera and carry out corresponding function, can observe the flight data that optics video and unmanned aerial vehicle machine carried observation device and descend simultaneously in real time. The remote controller of Lightbridge2 ground end for controlling unmanned aerial vehicle flight passes through USB interface connection with the panel computer, links to each other with removing the HDMI display screen simultaneously. The user can observe the infrared video through the HDMI display screen. The universal model airplane remote controller can be communicated with an airborne wireless receiving module of the airborne observation device and is used for controlling the attitude angles of the two holder cameras.
Fig. 2 is a schematic flow chart of a method for detecting and locating surface temperature anomaly based on an unmanned aerial vehicle according to an embodiment of the present invention.
After the unmanned aerial vehicle takes off, the visible light camera is started, and an operator controls the unmanned aerial vehicle to fly to an area where surface temperature abnormality may exist on a remote controller of the ground control device by observing a picture shot by the visible light camera. And starting the infrared camera through a remote controller of the ground control device to scan the ground surface area. The operator can simultaneously observe the optical image and the infrared image on the ground. And transmitting a control command through the mobile terminal equipment to detect the airborne temperature anomaly. If no abnormity exists, the unmanned aerial vehicle is continuously controlled to cruise; if the ground surface temperature is abnormal, the infrared camera sends an alarm signal, an operator controls the unmanned aerial vehicle to record or shoot videos of the visible light and the infrared camera, abnormal temperature detection and positioning are carried out, infrared video data are extracted, abnormal ground surface temperature is monitored, the geographical position of an abnormal ground surface area is calculated, then the unmanned aerial vehicle downloads the self pose and the calculated target geographical position, optical images and infrared images to the ground simultaneously, and one-time operation is completed.
In the embodiment of the invention, the combined observation control method of the infrared camera and the visible light camera is mainly realized by means of a flight control module and a visual observation module. The concrete implementation is as follows:
1) data transmission channel
Fig. 3 is a schematic diagram of a data transmission flow of Lightbridge2 of an unmanned aerial vehicle according to an embodiment of the present invention. The transmission link of Lightbridge2 is divided into upstream and downstream data. The uplink data comprises an unmanned aerial vehicle flight control command sent by a remote controller and control commands generated by various flight function keys and camera control keys on a human-computer interaction interface; the downlink data comprises real-time flight data acquired by the sensor, result feedback data of ground control command executed by the camera and infrared camera temperature detection and positioning data. These data are transmitted wirelessly through Lightbridge 2. The uplink data are directly transmitted to the flight control sub-module through a D-Bus interface of the Lightbridge2 sky-side module, and if the uplink data are flight control commands, the flight control sub-module directly executes the flight control commands; and if the command is a camera control command, transmitting the command to the camera control module through the serial port for execution. And the data received by the ground terminal of the Lightbridge2 is transmitted to the mobile terminal device through the USB interface and displayed on the man-machine interaction interface.
Fig. 4 is a schematic view of a video transmission flow of an unmanned aerial vehicle according to an embodiment of the present invention. The map pass function of Lightbridge2 supports the simultaneous transmission of two videos from the sky end to the ground end. The video streams of the visible light camera and the infrared camera are respectively input from an AV port and an HDMI port of the Lightbridge2 sky-side module, and are transmitted to the ground of the Lightbridge2 after being integrated by the inner code of the Lightbridge2 sky-side module. After receiving the image data stream, the ground terminal decodes and separates two paths of videos in the remote controller, transmits the visible light camera video to the mobile terminal equipment through the USB interface to be displayed on the human-computer interaction interface, and transmits the infrared video to the HDMI display screen through the HDMI interface to be displayed.
2) Airborne camera control command communication flow
Fig. 5 is a schematic view of a communication flow of a camera control command according to an embodiment of the present invention. The camera control command is sent by a user key of a human-computer interaction interface and comprises functions of photographing, video recording, focusing and the like, the command is sent to the Lightbridge2 sky end through the Lightbridge2 ground end, and after the Lightbridge2 sky end receives the command, the command is transmitted to the camera control module through the flight control sub-module and the serial port in sequence. The camera control module analyzes the command, PWM waves with corresponding frequencies are generated at GPIO ports appointed by the airborne development board according to the analyzed result, and the camera distinguishes different types of commands according to the GPIO port pins for receiving PWM and the received PWM waves, so that corresponding functions are executed. After the execution is finished, the camera control module sends out a feedback signal, and the signal is transmitted to the ground end of the Lightbridge2 through the flight control sub-module and the Lightbridge2 sky end in sequence and displayed on the man-machine interaction interface.
3) Camera control protocol
The minimum unit for data transmission between the Onboard SDK and the flight control sub-module is a protocol frame. Each frame protocol contains a frame header segment, a frame data segment, and a frame trailer segment. The frame header section defines the basic information of the frame, and comprises the following steps: length, sequence number, type, and reserved bits. The protocol frame is divided into a command frame and a response frame, the frame data segment of the command frame is divided into a command set, a command ID and a command value, and the frame data segment of the response frame only has a response value. The end of frame segment contains the checksum of the frame. The Mobile SDK is a software development kit for controlling the drone provided by DJI, running on the Mobile terminal device of the ground control device. In the embodiment of the invention, the data transmitted between the Onboard SDK and the Mobile SDK are command frames, and when the Onboard SDK and the Mobile SDK communicate, the command set and the command ID of the frame data section of the transmitted data are specified by a manufacturer, and the actually transmitted data are the command value part of the frame data section. Thus, the part is redefined to formulate a set of camera control protocols. The embodiment of the invention only sets a set of control protocol for the infrared camera, and the principle and the method of the optical camera are the same as those of the infrared camera.
The command value part of the frame data segment is redefined into a data header segment and a data segment, the header segment is divided into SOF, DES and TYPE, and the data segment is divided into CMD SET and CMD VAL, and the specific definition is shown in table 1 below.
TABLE 1 frame data segment Command value part definition
The command value part of the frame data segment redefines 4 bytes, and the first byte SOF is a starting identification bit; the second bytes DES, TYPE represent the transmission direction and TYPE of the command, which is 0x11 for the control command of the infrared camera as an example; the third byte CMD SET represents a role object of the data, the embodiment of the present invention uses 0x11 as a control command SET of the infrared camera, and the fourth byte CMD VAL represents functions of the infrared camera, such as photographing and recording, and the specific definitions are shown in table 2 below.
TABLE 2 CMD VAL and Infrared Camera function index Table
The infrared camera pin and PWM wave frequency data in table 2 are for example XM6A unmanned aerial vehicle thermal imager. The signal pins of the infrared camera can receive PWM wave input, and the command value is the specific numerical value of CMD VAL. The PWM wave frequency means the duration of the high level, that is, the PWM wave periods corresponding to 1.0ms, 1.5ms, and 2.0ms in table 1 are respectively 2.0ms, 3.0ms, and 4.0ms, the frequencies (reciprocal of the period) are respectively 500Hz, 333Hz, and 250Hz, and the output time of all PWM waves is 10 ms. And the development board sends PWM waves with corresponding frequencies to the infrared camera pins from the corresponding GPIO ports according to different CMD VAL values, so that the infrared camera is controlled to execute corresponding functions. After the execution is successful, the embedded development board sends data with CMD VAL value of 0x3F to the Mobile SDK through the Onboard SDK, and the Mobile terminal equipment running the Mobile SDK displays that the command is successfully executed after receiving the data.
Besides the four PWM input signal lines and the ground wire, the infrared camera is also provided with an output lead, when the temperature anomaly detection module detects temperature anomaly, 3.3V voltage can be continuously output at the lead, and the voltage disappears when the target temperature returns to normal. The lead wire also needs to be connected with a GPIO port of the embedded development board, the development board constantly monitors the GPIO port, if a rising edge or a falling edge occurs, the target temperature is abnormal or the target temperature returns to normal, the development board can send corresponding data to the Mobile SDK through the OnboardSDK, after the Mobile terminal device receives the data, a prompt and screenshot can be generated on a human-computer interaction interface, time information and geographical position information are recorded simultaneously, and matching of images and information is facilitated.
Fig. 6 is a schematic diagram of an execution flow of a camera control module according to an embodiment of the present invention, which runs on an embedded development board in real time through an onboarddsdk. After configuration operation, aircraft activation and flight control right acquisition are carried out on the main program, the main program is divided into two threads. The sub-thread continuously reads the voltage state of the GPIO5 port, if the voltage state is high level, the global variable flag is set to 1, otherwise the flag is set to 0, the main thread judges whether a rising edge or a falling edge is generated according to whether the value of the flag is changed, so as to determine whether to send a temperature alarm signal or an alarm release signal to the Mobile SDK, then continuously read the data sent by the Mobile SDK to the Onboard SDK, the command value part of the frame data section is analyzed according to a preset protocol, if the data is an infrared camera control command, PWM waves with corresponding frequency are sent from the corresponding GPIO port to the infrared camera according to the command value, after the infrared camera successfully executes the function, a successful response signal is sent to the Mobile SDK, and finally the steps are repeated after 20ms of delay.
In the embodiment of the invention, the temperature anomaly detection and positioning module of the airborne observation device runs on the embedded development board in real time through the Onboard SDK. The specific implementation process is as follows:
1) abnormality detection method
Surface temperature anomalies are typically manifested in infrared images of three types: the temperature of a certain position in the picture is too high, the temperature of a certain position in the picture is too low, and the temperature of a certain position in the picture has sudden change compared with the ambient temperature. Therefore, the detection of the surface abnormal temperature in the embodiment of the invention mainly aims at the three situations.
The detection method 1: detection of temperature at a point in an image above or below a threshold temperature
If the maximum temperature threshold is set as TmaxThe lowest temperature threshold is TminMaximum temperature difference of Δ Tmax. These three thresholds can be set by the operator himself in the actual operation. For a certain infrared image, because the infrared image is in the same temperature measurement range, the gray values of all the pixel points and the actual temperature present a linear positive correlation relationship, and the linear coefficient can be obtained through calculation. For a certain infrared image, the steps of calculating the highest temperature position, the lowest temperature position and the corresponding temperature are as follows:
1. the infrared image is a color image, so the three-channel color image is firstly converted into a single-channel gray image. In the embodiment of the present invention, the color image is converted into the gray image by a weighted average method, that is:
I(x,y)＝0.3×I_R(x,y)+0.59×I_G(x,y)+0.11×I_B(x,y)
wherein, (x, y) is the pixel coordinates, I (x, y) is the pixel of the gray image, I _ R (x, y), I _ G (x, y), I _ B (x, y) are the three-channel pixel values of the color image pixel.
2. Traversing all pixel points of the gray level image, obtaining pixel coordinate positions maxP and minP of the points with the maximum gray level value and the minimum gray level value, and recording the corresponding gray level maxValue and minValue.
3. And establishing a temperature-gray value conversion relation according to the reference points of the current image provided by the infrared camera, namely the temperature values corresponding to some pixel points in the current image, the average temperature of the temperature measurement range and the pixel values corresponding to the average temperature. Theoretically, the conversion relationship between the gray-scale value and the temperature is:
I(x,y)＝kTxy+b
where k represents the slope in the linear relationship, TxyThe temperature values corresponding to the (x, y) points are shown, and b represents the offset in the linear relationship.
4. And substituting the obtained gray values maxValue and minValue of the highest temperature point and the lowest temperature point in the current image into a temperature-gray value conversion relational expression to obtain the temperature values of the highest temperature point and the lowest temperature point. If the maximum temperature is greater than the threshold value TmaxOr the lowest temperature is less than the threshold value TminIt is considered that there is a temperature abnormality. And recording the pixel coordinate position of the temperature abnormal point for the subsequent abnormal target positioning method.
The detection method 2 comprises the following steps: detecting temperature abnormality that the temperature of a certain position in an image has a sudden change compared with the ambient temperature, comprising the following steps of:
1. converting the color image into a gray image, and establishing a temperature-gray value conversion relation of the infrared image according to the parameters of the infrared camera, wherein the method is the same as the temperature conversion linear relation in the detection method 1; will have the highest temperature difference Delta TmaxBringing the conversion relation into, obtaining the maximum pixel value difference delta Imax；
2. The image is divided into uniform irregular blocks of pixels. The specific method comprises the following steps:
i. initializing a clustering center: if the gray level image has N pixel points, the number of the pre-divided super pixels is set to be K, and the size of each super pixel is set to be K
Distance between adjacent cluster centers is
Reselecting cluster centers within an n × n domain of cluster centers: calculating gradient values of all pixel points in the neighborhood, and then moving a clustering center to a position with the minimum gradient;
assigning a class label to each pixel point in the neighborhood around each cluster center: limiting the search range to be 2S multiplied by 2S, and accelerating convergence;
distance metric: for each searched pixel, calculating its distance from the cluster center, i.e.
Wherein d iscIs the distance of the grey values, dsAs a function of the spatial distance,
is the maximum spatial distance within the class. N is a radical ofcThe distance of the maximum gray value can be changed with different pictures or clusters and is set as a constant. And each pixel point has a distance with the surrounding clustering centers, and the clustering center corresponding to the minimum value of the distance measurement D' is taken as the clustering center of the pixel point.
v. iterative optimization: the above steps are iterated until the cluster center is no longer changed.
In the embodiment of the invention, efficiency and time are measured, and the number of iterations is selected to be 10.
Therefore, the pixels in the divided pixel blocks have the characteristics of similar gray value and the like. Recording the central point positions of all the pixel blocks after final convergence and the average gray value of each pixel block;
3. comparing the average gray value of each pixel block with the average gray values of the adjacent pixel blocks around the pixel block, and if the gray value difference exceeds the maximum gray value difference delta ImaxIf so, judging that the temperature abnormal condition occurs, and recording the pixel coordinate position of one pixel block for the subsequent abnormal target positioning method.
2) Abnormal target positioning method
To calculate the geographical position of the target, the pixel position of the abnormality detection output is converted into the geographical position. The following steps are specifically executed:
1. and establishing an unmanned aerial vehicle projection model as shown in fig. 7. Assuming that all coordinate systems meet the right-hand rule, an ENU (East-North-Up) coordinate system is a world coordinate system and is represented by a corner mark (E); the origin of the coordinate system of the unmanned aerial vehicle infrared camera is coincided with the optical center of the infrared camera, the x axis is coincided with the optical axis, and the x axis is represented by an angle mark (C); the pixel coordinate system is denoted by (uv).
2. According to the rigid body motion, the ENU coordinate system can be coincided with the camera coordinate system after rotation and translation operations. Specifically, it can be described by the formula:
wherein,
respectively represent the position of the target under an ENU coordinate system and a camera coordinate system,
t is the rotation matrix and translation vector from the camera coordinate system to the ENU coordinate system. The rotation matrix can be decomposed into a rotation matrix around the x, y, z axes, gamma, theta,
The pitch angle, the roll angle and the course angle of the unmanned aerial vehicle camera are respectively,
representing the position of the drone in the ENU world coordinate system. According to the above formula, the position of the target in the world coordinate system can be solved according to the position of the target in the camera coordinate system.
3. Also according to the imaging model, the pixel coordinates of the object can be converted into the position of the object in the camera coordinate system. Firstly, the position of a point of the target on the imaging plane under a camera coordinate system is calculated:
where K is the camera reference matrix, fx、fy、cx、cyIs a parameter of K. f is the focal length of the camera and fx＝αf，fy＝βf。
Is the pixel coordinates of the object and,
and representing the position of the pixel point of the target on the imaging surface under the world coordinate system. In combination with the above formula, the specific process of converting the pixel coordinates of the target into a world coordinate system is as follows:
wherein d is the actual distance from the optical center of the infrared camera to the target, i.e. the depth.
4. Establishing a reference coordinate system (R) with an origin at the camera optical center and three axes parallel to the ENU coordinate system, the depth can be described as:
in the formula, H is the flying height,
is a unit vector in a reference coordinate system, and omega represents a vector sum of a camera optical center to a target
The included angle between the two parts is included,
is a vector
Unit vector of (1), point PimtRepresenting the position of the object on the imaging plane. In combination with the above formula, the conversion of the target from the pixel coordinate system to the ENU coordinate system can be accomplished.
5. The target finally needs to be converted from the ENU coordinate system to GPS coordinates, i.e. to the WGS84(World geodetic system-1984) coordinate system (labeled I above). The concrete description is as follows:
in the formula, rEThe radius of the earth, where the target is located, B, L longitude and latitude,
a rotation matrix representing the ENU coordinate system to the ECI coordinate system,
the position of the origin of the ENU coordinate system in the ECI coordinate system is represented, and therefore the geographic position of the abnormal target can be calculated. The geographic location is downloaded to a surface control device for reference by an operator.
The invention provides a method for combining infrared image and optical image combined observation and earth surface temperature anomaly detection based on an unmanned aerial vehicle platform. The unmanned aerial vehicle carries two kinds of cameras of visible light and infrared at airborne observation device. The visible camera may be used to provide a normal view for unmanned aerial vehicle flight, while the infrared camera may be used to detect surface temperature anomalies as well as night flights. The two paths of videos are transmitted to the ground control device by the image transmission equipment, and the two paths of videos can be displayed separately after being received by the ground control device.
The invention provides a surface temperature anomaly detection and positioning method based on an infrared camera video, which automatically detects three types of surface temperature anomalies by establishing a gray value-temperature relation and combining super-pixel segmentation, wherein the three types of anomalies are respectively the conditions of overhigh temperature, overlow temperature and sudden change of the temperature of a certain position compared with the ambient temperature; after the pixel value of the abnormal target is obtained, the geographical position of the target is calculated by establishing a projection model of the unmanned aerial vehicle, and the geographical position is transmitted to the ground control device for display through an airborne wireless data transceiver module of the unmanned aerial vehicle.
In order to realize the joint control of the optical camera and the infrared camera and realize the surface temperature anomaly detection and positioning, the unmanned aerial vehicle airborne observation device is provided with an embedded development board, and the method can be realized through an Onboard SDK.
The invention integrates a landmark temperature anomaly detection and positioning system based on an unmanned aerial vehicle platform.
It should be understood that parts of the specification not set forth in detail are well within the prior art.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. An earth surface temperature anomaly detection and positioning method based on an unmanned aerial vehicle is characterized by comprising the following steps:
controlling an unmanned aerial vehicle to fly to a target area according to visible light data, and acquiring infrared data of the target area;
judging whether temperature abnormality exists in the infrared data, and if the temperature abnormality exists, acquiring pixel coordinates of the temperature abnormality;
obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly;
and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly.
2. The method of claim 1, wherein the determining whether a temperature anomaly exists in the infrared data comprises:
and judging whether an area with the temperature higher than a preset highest temperature threshold value, an area with the temperature lower than a preset lowest temperature threshold value or an area with the temperature difference larger than a highest temperature difference threshold value from the surrounding preset area exists in the infrared data.
3. The method of claim 2, wherein the determining whether there is an area with a temperature higher than a preset maximum temperature threshold and an area with a temperature lower than a preset minimum temperature threshold in the infrared data comprises:
carrying out graying processing on the infrared data to obtain grayscale data;
acquiring a maximum gray value corresponding to a pixel point with the maximum gray value and a minimum gray value corresponding to a pixel point with the minimum gray value in the gray data;
establishing a temperature-gray value conversion relation according to the pixel value of the reference pixel point provided by the current infrared data and the corresponding temperature thereof, the average temperature of the temperature measurement range and the pixel value corresponding to the average temperature;
obtaining the highest temperature corresponding to the maximum gray value and the lowest temperature corresponding to the minimum gray value based on the temperature-gray value conversion relation;
if the highest temperature is greater than the preset highest temperature threshold, the temperature of the area corresponding to the maximum gray value is abnormal; if the lowest temperature is smaller than the preset lowest temperature threshold, the temperature of the area corresponding to the minimum gray value is abnormal.
4. The method of claim 2, wherein the determining whether there is an area having a temperature difference from a surrounding preset area greater than a maximum temperature difference threshold comprises:
graying the infrared data to obtain grayscale data, establishing a temperature-grayscale value conversion relation according to a pixel value of a reference pixel point provided by the current infrared data and a corresponding temperature thereof, an average temperature of a temperature measurement range and a pixel value corresponding to the average temperature, and obtaining a maximum grayscale value difference corresponding to the maximum temperature difference threshold value based on the temperature-grayscale value conversion relation;
dividing the gray data into uniform irregular pixel blocks, comparing the average gray value of each pixel block with the average gray value of the pixel blocks in the preset area around the pixel block, and if the gray value difference of the two exceeds the maximum gray value difference, determining that the temperature of the pixel block is abnormal.
5. The method of claim 4, wherein the dividing the gray data into uniform irregular blocks of pixels comprises:
determining initial clustering centers according to the number of pre-divided super pixels, acquiring gradient values of all pixel points in a first preset field of each initial clustering center, and moving each initial clustering center to a position with the minimum corresponding gradient value to obtain a new clustering center;
searching pixel points in a second preset neighborhood around each new clustering center, acquiring the distance between each searched target pixel point and the corresponding new clustering center, taking the new clustering center corresponding to the minimum distance as the clustering center of the target pixel point, repeating the searching operation until the clustering center is not changed any more, and determining pixel blocks belonging to each clustering center.
6. The method according to claim 3 or 5, wherein the obtaining the coordinates of the temperature anomaly in the world coordinate system according to the transformation relationship between the camera coordinate system and the world coordinate system, the transformation relationship between the camera coordinate system and the pixel coordinate system, and the pixel coordinates of the temperature anomaly comprises:
establishing a reference coordinate system with an origin at an infrared camera optical center and three axes parallel to a world coordinate system, and obtaining an actual distance from the infrared camera optical center to the temperature anomaly position according to the reference coordinate system and the flight height of the unmanned aerial vehicle;
and obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinate system and the actual distance from the optical center of the infrared camera to the temperature anomaly.
7. The utility model provides an earth's surface temperature anomaly detection and airborne observation device of location based on unmanned aerial vehicle which characterized in that includes: the system comprises an unmanned aerial vehicle flight control module, a visual observation module and an abnormity detection and positioning module;
the unmanned aerial vehicle flight control module is used for controlling the unmanned aerial vehicle to fly to a target area according to a received flight control instruction, then receiving a camera control instruction and sending the camera control instruction to the visual observation module, wherein the flight control instruction is obtained by analyzing visible light data acquired by an optical camera in the visual observation module and then triggering;
the visual observation module is used for analyzing the camera control instruction, controlling an optical camera and an infrared camera in the visual observation module to execute corresponding operation and acquiring infrared data of the target area;
the abnormal detection and positioning module is used for judging whether temperature abnormality exists in the infrared data or not, and if the temperature abnormality exists, acquiring pixel coordinates of the abnormal temperature position; obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly; and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly.
8. The apparatus of claim 7, wherein the drone flight control module comprises: the system comprises a wireless image transmission module, an airborne wireless data transceiving module, a flight control submodule and an unmanned aerial vehicle flight platform;
the wireless image transmission module is used for transmitting the optical data and the infrared data shot by the unmanned aerial vehicle to the ground control device for displaying;
the airborne wireless data transceiver module is used for receiving a remote control instruction sent by the ground control device and sending telemetering data on the unmanned aerial vehicle to the ground control device;
and the flight control submodule is used for controlling the unmanned aerial vehicle flight platform to act according to the remote control command so as to control the flight of the unmanned aerial vehicle.
9. The apparatus of claim 8, wherein the visual observation module comprises: the system comprises a camera control module, an airborne wireless receiving module, a holder optical camera and a holder infrared camera;
the camera control module is used for receiving a camera control command sent by the flight control sub-module and controlling the holder optical camera and the holder infrared camera to execute an operation corresponding to the camera control command;
and the airborne wireless receiving module is used for receiving a camera attitude angle adjusting instruction sent by the ground control device so as to control the attitude angles of the holder optical camera and the holder infrared camera.
10. The utility model provides an earth's surface temperature anomaly detection and positioning system based on unmanned aerial vehicle which characterized in that includes: an airborne observation device and a ground control device;
the ground control device is used for receiving and displaying the optical data and the infrared data sent by the airborne observation device, receiving a flight control instruction and a camera control instruction, and sending the flight control instruction and the camera control instruction to the airborne observation device;
the airborne observation device is used for receiving the flight control instruction and controlling the unmanned aerial vehicle to fly to a target area according to the flight control instruction; receiving the camera control instruction, controlling an optical camera and an infrared camera to execute corresponding operation according to the camera control instruction, and acquiring infrared data of the target area;
the airborne observation device is also used for judging whether temperature abnormality exists in the infrared data, and if the temperature abnormality exists, the pixel coordinates of the temperature abnormality position are obtained; obtaining the coordinates of the temperature anomaly in the world coordinate system according to the conversion relation between the camera coordinate system and the world coordinate system, the conversion relation between the camera coordinate system and the pixel coordinates of the temperature anomaly; and converting the coordinates of the temperature anomaly in a world coordinate system into GPS coordinates to acquire the geographical position information of the temperature anomaly, sending the geographical position information of the temperature anomaly to the ground control device and displaying the geographical position information on the temperature anomaly in the ground control device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010134618.2A CN111307291B (en) | 2020-03-02 | 2020-03-02 | Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010134618.2A CN111307291B (en) | 2020-03-02 | 2020-03-02 | Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111307291A true CN111307291A (en) | 2020-06-19 |
| CN111307291B CN111307291B (en) | 2021-04-20 |
Family
ID=71152117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010134618.2A Active CN111307291B (en) | 2020-03-02 | 2020-03-02 | Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111307291B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111982298A (en) * | 2020-08-14 | 2020-11-24 | 扬州大学 | Unmanned aerial vehicle-based rice canopy temperature detection method |
| CN112860946A (en) * | 2021-01-18 | 2021-05-28 | 四川弘和通讯有限公司 | Method and system for converting video image information into geographic information |
| CN112950565A (en) * | 2021-02-25 | 2021-06-11 | 山东英信计算机技术有限公司 | Method and device for detecting and positioning water leakage of data center and data center |
| WO2022056654A1 (en) * | 2020-09-15 | 2022-03-24 | 华为技术有限公司 | Temperature measurement method and temperature measurement apparatus |
| CN116527119A (en) * | 2023-06-29 | 2023-08-01 | 众芯汉创（北京）科技有限公司 | Intelligent analysis and understanding componentization system and method based on onboard image video |
| CN117232658A (en) * | 2023-11-03 | 2023-12-15 | 苏州简测科技有限公司 | Echelle grating spectrometer |
| CN118730600A (en) * | 2024-09-04 | 2024-10-01 | 西安英孚瑞科技有限公司 | Uncooled infrared detector thermal imaging anomaly detection method and uncooled infrared detector thermal imaging anomaly detection device |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101650866A (en) * | 2009-09-22 | 2010-02-17 | 华南理工大学 | Fire detecting system applied to unmanned helicopter and fire detecting method thereof |
| CN102819926A (en) * | 2012-08-24 | 2012-12-12 | 华南农业大学 | Fire monitoring and warning method on basis of unmanned aerial vehicle |
| CN102881106A (en) * | 2012-09-10 | 2013-01-16 | 南京恩博科技有限公司 | Dual-detection forest fire identification system through thermal imaging video and identification method thereof |
| CN104143248A (en) * | 2014-08-01 | 2014-11-12 | 江苏恒创软件有限公司 | Forest fire detection, prevention and control method based on unmanned aerial vehicle |
| CN108033015A (en) * | 2017-12-20 | 2018-05-15 | 西安科技大学 | A kind of unmanned plane device and method for the monitoring of coal gangue hill ignition point |
| CN108535321A (en) * | 2018-03-30 | 2018-09-14 | 吉林建筑大学 | A kind of building thermal technique method for testing performance based on three-dimensional infrared thermal imaging technique |
| CN108629297A (en) * | 2018-04-19 | 2018-10-09 | 北京理工大学 | A kind of remote sensing images cloud detection method of optic based on spatial domain natural scene statistics |
| JP2019027840A (en) * | 2017-07-27 | 2019-02-21 | 東京電力ホールディングス株式会社 | Observation device |
| CN110155327A (en) * | 2019-05-18 | 2019-08-23 | 太原裕昌泰科贸有限公司 | A kind of low-to-medium altitude flight monitoring unmanned system |
| US20190311203A1 (en) * | 2018-04-09 | 2019-10-10 | Accenture Global Solutions Limited | Aerial monitoring system and method for identifying and locating object features |
| CN110446159A (en) * | 2019-08-12 | 2019-11-12 | 上海工程技术大学 | A kind of system and method for interior unmanned plane accurate positioning and independent navigation |
| CN110525650A (en) * | 2016-12-22 | 2019-12-03 | 深圳市大疆创新科技有限公司 | Unmanned plane and its control method |
-
2020
- 2020-03-02 CN CN202010134618.2A patent/CN111307291B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101650866A (en) * | 2009-09-22 | 2010-02-17 | 华南理工大学 | Fire detecting system applied to unmanned helicopter and fire detecting method thereof |
| CN102819926A (en) * | 2012-08-24 | 2012-12-12 | 华南农业大学 | Fire monitoring and warning method on basis of unmanned aerial vehicle |
| CN102881106A (en) * | 2012-09-10 | 2013-01-16 | 南京恩博科技有限公司 | Dual-detection forest fire identification system through thermal imaging video and identification method thereof |
| CN104143248A (en) * | 2014-08-01 | 2014-11-12 | 江苏恒创软件有限公司 | Forest fire detection, prevention and control method based on unmanned aerial vehicle |
| CN110525650A (en) * | 2016-12-22 | 2019-12-03 | 深圳市大疆创新科技有限公司 | Unmanned plane and its control method |
| JP2019027840A (en) * | 2017-07-27 | 2019-02-21 | 東京電力ホールディングス株式会社 | Observation device |
| CN108033015A (en) * | 2017-12-20 | 2018-05-15 | 西安科技大学 | A kind of unmanned plane device and method for the monitoring of coal gangue hill ignition point |
| CN108535321A (en) * | 2018-03-30 | 2018-09-14 | 吉林建筑大学 | A kind of building thermal technique method for testing performance based on three-dimensional infrared thermal imaging technique |
| US20190311203A1 (en) * | 2018-04-09 | 2019-10-10 | Accenture Global Solutions Limited | Aerial monitoring system and method for identifying and locating object features |
| CN108629297A (en) * | 2018-04-19 | 2018-10-09 | 北京理工大学 | A kind of remote sensing images cloud detection method of optic based on spatial domain natural scene statistics |
| CN110155327A (en) * | 2019-05-18 | 2019-08-23 | 太原裕昌泰科贸有限公司 | A kind of low-to-medium altitude flight monitoring unmanned system |
| CN110446159A (en) * | 2019-08-12 | 2019-11-12 | 上海工程技术大学 | A kind of system and method for interior unmanned plane accurate positioning and independent navigation |
Non-Patent Citations (1)
| Title |
|---|
| 何慧敏: "一种基于红外热图的温度提取方法", 《计算技术与自动化》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111982298A (en) * | 2020-08-14 | 2020-11-24 | 扬州大学 | Unmanned aerial vehicle-based rice canopy temperature detection method |
| WO2022056654A1 (en) * | 2020-09-15 | 2022-03-24 | 华为技术有限公司 | Temperature measurement method and temperature measurement apparatus |
| CN112860946A (en) * | 2021-01-18 | 2021-05-28 | 四川弘和通讯有限公司 | Method and system for converting video image information into geographic information |
| CN112860946B (en) * | 2021-01-18 | 2023-04-07 | 四川弘和通讯集团有限公司 | Method and system for converting video image information into geographic information |
| CN112950565A (en) * | 2021-02-25 | 2021-06-11 | 山东英信计算机技术有限公司 | Method and device for detecting and positioning water leakage of data center and data center |
| CN116527119A (en) * | 2023-06-29 | 2023-08-01 | 众芯汉创（北京）科技有限公司 | Intelligent analysis and understanding componentization system and method based on onboard image video |
| CN116527119B (en) * | 2023-06-29 | 2023-09-08 | 众芯汉创（北京）科技有限公司 | Intelligent analysis and understanding componentization system and method based on onboard image video |
| CN117232658A (en) * | 2023-11-03 | 2023-12-15 | 苏州简测科技有限公司 | Echelle grating spectrometer |
| CN117232658B (en) * | 2023-11-03 | 2024-01-30 | 苏州简测科技有限公司 | Echelle grating spectrometer |
| CN118730600A (en) * | 2024-09-04 | 2024-10-01 | 西安英孚瑞科技有限公司 | Uncooled infrared detector thermal imaging anomaly detection method and uncooled infrared detector thermal imaging anomaly detection device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111307291B (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111307291B (en) | Surface temperature anomaly detection and positioning method, device and system based on unmanned aerial vehicle | |
| US11797009B2 (en) | Unmanned aerial image capture platform | |
| CN109596118B (en) | Method and equipment for acquiring spatial position information of target object | |
| WO2021189507A1 (en) | Rotor unmanned aerial vehicle system for vehicle detection and tracking, and detection and tracking method | |
| US20220077820A1 (en) | Method and system for soar photovoltaic power station monitoring | |
| US20170371353A1 (en) | Automatic Tracking Mode For Controlling An Unmanned Aerial Vehicle | |
| US20200012756A1 (en) | Vision simulation system for simulating operations of a movable platform | |
| EP3629309A2 (en) | Drone real-time interactive communications system | |
| JP6583840B1 (en) | Inspection system | |
| CN112115607B (en) | Mobile intelligent body digital twin system based on multidimensional microblog space | |
| CN109656319B (en) | Method and equipment for presenting ground action auxiliary information | |
| WO2018059398A1 (en) | Method, apparatus, and system for controlling multi-rotor aircraft | |
| US20210221502A1 (en) | Method and a system for real-time data processing, tracking, and monitoring of an asset using uav | |
| CN113890977B (en) | Airborne video processing device and unmanned aerial vehicle with same | |
| Li et al. | Verification of monocular and binocular pose estimation algorithms in vision-based UAVs autonomous aerial refueling system | |
| US10557718B2 (en) | Auxiliary control method and system for unmanned aerial vehicle | |
| JP2023100642A (en) | inspection system | |
| WO2021251441A1 (en) | Method, system, and program | |
| Gur et al. | Image processing based approach for crime scene investigation using drone | |
| CN111708916A (en) | Unmanned aerial vehicle cluster video intelligent processing system and method | |
| CN218041561U (en) | Individual soldier infrared thermal imaging reconnaissance system | |
| CN117930981A (en) | Unmanned aerial vehicle eye movement control method, device and system | |
| WO2022113482A1 (en) | Information processing device, method, and program | |
| KR101858989B1 (en) | System for analyzing character of target | |
| WO2024084925A1 (en) | Information processing apparatus, program, and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |