CN111444801A - Real-time detection method for infrared target of unmanned aerial vehicle - Google Patents
Real-time detection method for infrared target of unmanned aerial vehicle Download PDFInfo
- Publication number
- CN111444801A CN111444801A CN202010191052.7A CN202010191052A CN111444801A CN 111444801 A CN111444801 A CN 111444801A CN 202010191052 A CN202010191052 A CN 202010191052A CN 111444801 A CN111444801 A CN 111444801A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- infrared
- detection
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention discloses a real-time detection method for an infrared target of an unmanned aerial vehicle, which belongs to the field of real-time monitoring of the unmanned aerial vehicle and comprises the following steps of (1) constructing an IT-YO L O unmanned aerial vehicle infrared target detection network, (2) acquiring and manufacturing an unmanned aerial vehicle infrared target detection data set, (3) training to generate an unmanned aerial vehicle real-time infrared target detection model, (4) detecting the infrared target by the unmanned aerial vehicle, improving a trunk characteristic extraction network and replacing a network detection layer on the basis of a Tiny-YO L OV3 lightweight target detection network, constructing the IT-YO L O unmanned aerial vehicle infrared target detection network, acquiring and manufacturing an unmanned aerial vehicle infrared target detection data set under the vision of the unmanned aerial vehicle by using infrared imaging, training to generate the unmanned aerial vehicle real-time infrared target detection model, applying to the detection of the unmanned aerial vehicle under the detection environment which can not be matched by a visible light camera such as the weather influence environment such as no light at night, rain fog and the like and a common digital night vision device, acquiring clear infrared images of pedestrians and.
Description
Technical Field
The invention belongs to the technical field of real-time monitoring of unmanned aerial vehicles, and particularly relates to a real-time detection method for an infrared target of an unmanned aerial vehicle.
Background
Nowadays, the military application of unmanned aerial vehicles is more mature and deeper. With the rapid development of technologies such as flight control, communication, positioning and navigation, more application fields and values of the unmanned aerial vehicle are excavated by people, such as movie and television aerial photography, entertainment and self-shooting, agricultural plant protection, traffic management and the like. Under the dark environment at night, under the bad weather condition, the unmanned aerial vehicle needs to detect the target at a long distance and high precision, and the device relying on visible light illumination or low-light night vision has great limitation.
The infrared thermal imaging system has the advantages that the imaging principle is infrared spectrum radiation imaging, the infrared thermal imaging system is independent of a light source, is little influenced by weather, has long detection distance, and has strong application value in the fields of target identification and detection, search and rescue, military and driving assistance in a completely dark environment at night. The application of the thermal imaging technology enables the unmanned aerial vehicle to have stronger capability in the aspects of fast searching for people and fast determining sources at night, and the unmanned aerial vehicle can become an important tool for searching people for rescue after a disaster, confirming fire sources and searching daily police.
In the future, three words of 'intelligent' are added between 'thermal imaging + unmanned aerial vehicle'. At present, thermal imaging unmanned aerial vehicle is still only a data collection instrument, and the thermal imaging picture either relies on the manual work, or relies on other machines, still is in the post processing stage of manpower ization and automation, and real-time intellectuality in the ideal has not been realized yet. In view of this, the development trend in the future should be to directly load the intelligent analysis function on the thermal imaging unmanned aerial vehicle body, let it not only fly, can see but also can think. Through intelligent adding holding, also be an analytical tool when letting unmanned aerial vehicle become to collect the instrument, the just better supplementary scientific decision of two unifications.
A large number of existing target technologies are based on a visible light environment, and a target detection method under the environment of full black, no light, rain and fog weather and the like is lack of research and practice. Secondly, the intelligent real-time target detection research work under the vision of the unmanned aerial vehicle is also in a starting stage, and related research results are few.
Disclosure of Invention
The invention aims to: the method for detecting the infrared target of the unmanned aerial vehicle in real time is provided, and the defects that the existing mass target technologies are based on a visible light environment and the target detection method under the environments of full darkness, no light, rain and fog weather and the like is lack of research and practice are overcome.
The technical scheme adopted by the invention is as follows: an unmanned aerial vehicle infrared target real-time detection method comprises the following steps:
(1) an IT-YO L O unmanned aerial vehicle infrared target detection network is constructed;
(2) collecting and manufacturing an unmanned aerial vehicle infrared target detection data set;
(3) training to generate an unmanned aerial vehicle real-time infrared target detection model;
(4) and detecting the infrared target by the unmanned plane.
According to the technical scheme, improvement of a trunk characteristic extraction network and replacement of a network detection layer are carried out on the basis of a Tiny-YO L OV3 lightweight target detection network, an IT-YO L O unmanned aerial vehicle infrared target detection network is constructed, an unmanned aerial vehicle infrared target detection data set is collected and manufactured under the vision of an unmanned aerial vehicle by using infrared imaging, an unmanned aerial vehicle real-time infrared target detection model is generated by training, the unmanned aerial vehicle infrared target detection model is applied to detecting targets under the detection environments which cannot be met by a visible light camera and a common digital night vision device in environments with no light at night, weather influence such as rain and fog and the like, clear pedestrians can be collected around hectometer in infrared spectrum, vehicle imaging is carried out in real time and high-precision target detection, and the defects that the existing large number of target technologies are based on the visible light environment and a target detection method under the environments with no light, rain and fog weather lacks research and practice.
Preferably, the IT-YO L O unmanned aerial vehicle infrared target detection network in the step (1) is to improve a main network and a detection network on the basis of a Tiny-YO L OV3 lightweight target detection network, the IT-YO L O unmanned aerial vehicle infrared target detection network adopts a basic structure of Tiny-YO L OV3, according to the characteristics of infrared images, the characteristics of a shallow convolutional layer are extracted, the infrared small target detection capability is improved, a single-channel convolutional core is used, the operand is reduced, and a detection part uses a detection mode based on a CenterNet structure to reduce the false detection rate and improve the detection speed.
Preferably, for feature extraction of a long-distance low-resolution infrared small target, a Tiny layer of a Tiny convolution Conv4 layer of a Tiny-YO L OV3 target detection network can effectively represent semantic information of the infrared small target, a Conv3 layer is too small in receptive field, a Conv5 layer is too large in receptive field and contains certain background noise interference, and therefore in order to improve detection capability of the infrared small target, a Maxpool3 layer in a Tiny-YO L OV3 is replaced by a Conv4 layer, a Conv5 layer is added, meanwhile, an upper sampling layer sample2 is added on the basis of a Tiny-YO L OV3 model, a Conv5 layer and an Upestle 2 are connected in channel dimension, Conv7 is subjected to sampling raising operation through an Upestle 2 to form a feature added by a feature pyramid, a first convolution kernel in the upper sampling layer is replaced by a single-channel convolution kernel, the operation amount is reduced, the real-time performance is improved, a detection network structure is further optimized by replacing a Center 68692 real-time detection method, and the target detection method is used for detecting the infrared small target detection network.
Preferably, the feature pyramid network is a 3 detection scale structure of 13 × 13, 26 × 26 and 52 × 52 pixels the feature pyramid network is promoted by 13 × 13 and 26 × 26 pixels of Tiny-YO L OV3 to 3 detection scale structures of 13 × 13, 26 × 26 and 52 × 52 pixels, using the centret detection structure on 3 detection scales.
Preferably, in order to further improve the detection precision of the infrared target in the infrared image, improve the detection real-time performance and reduce the computation amount, a detection network part uses a centret detection structure on 3 detection scales, and replaces the original YO L O detection layer based on an anchor (anchor box) with an anchor free mode (anchor free), so that the improved target detection network is more suitable for detecting the infrared small target.
The basic idea of CenterNet is that a large number of incorrect bounding boxes often appear in an anchor box (anchor box) -based method, and due to the lack of additional supervision on relevant clipping regions, the original Tiny-YO L OV3 needs a large number of anchor boxes in a detection layer for target detection, while CenterNet is a single-stage key point detection model, each target object to be detected is regarded as three key points (a central point, an upper corner and a lower corner), so that the generation of a large number of anchor boxes is avoided, the operation amount is reduced, the real-time performance is improved, and the detection accuracy and the recall rate are improved.
The Center pool model in the centret network consists of 2 convolution normalized residual fusion layers (Conv-BN-Re L U), 1 left Pooling layer (L eft pool), 1 Right Pooling layer (Right pool), 1 Top Pooling layer (toppool), 1 Bottom Pooling layer (Bottom pool), which functions to predict branches of central key points, facilitating the central acquisition of more targets, and thus making it more perceptible to the central area of the propofol.
Preferably, when the centret structure is used for detection, the scale sensitive region is adopted to adapt to the targets with different sizes, a central region with a relatively small target and a relatively large target is generated, and whether a bounding box I is reserved is acquired in the following manner: tlx，tlyPoints, br, representing the upper left corner of the framex，bryRepresenting the point at the bottom right corner of the box, a center region j is defined, and the coordinates of the point at the top left corner are defined as (ctl)x,ctly) Lower right corner point (cbr)x,cbry)，ctlx，ctly，cbrx，cbrySatisfies the following formula (a) in the following formula,
(1) where n is an odd number and represents the size of the central region j, n is 3 when the bounding box is smaller than 150, and n is 5 when the bounding box is larger than 150.
Preferably, in step (2), adopt unmanned aerial vehicle to carry on infrared thermal imaging platform respectively in the low latitude under the night environment, the high altitude distance is shot, and clear infrared image that can supply discernment to detect is gathered to the visual angle from top to bottom, collects 8000 infrared thermal imaging images of unmanned aerial vehicle visual angle lower night scene and detects the data set as unmanned aerial vehicle infrared target.
More preferably, an unmanned aerial vehicle infrared target detection data set is divided into a training set and a test set according to a ratio of 5:1, a YO L O-MARK tool is used for labeling 2 types of targets to be detected, pedestrians and vehicles, in a model training link, all image samples in the data set are converted into images of 416 × 416 pixels, during training, 100 images are used as a batch for small-batch training, a batch of images are trained, a weight is updated once, the attenuation rate of the weight is set to be 0.0005, momentum is set to be 0.9, the initial learning rate is set to be 0.001, an IT-YO L O network is iterated 20000 times, the model is stored once after every 2000 iterations, the average loss of the final model is reduced to be less than 0.2, and the model with the highest precision is selected as an unmanned aerial vehicle real-time infrared target detection model.
The hardware platform used for model training and testing is Core i7-8750H 2.2Ghz processor +16GB memory + Geforce GTX 10808 GB video card, and the software platform uses Win10+ tensoflow1.9.0 + CUDA9.2+ VS2017+ opencv 4.0.
Preferably, the average accuracy Mp, the average false detection rate Mf, the average missed detection rate Mm, the average calculation speed Mo, and the detection model size Mw index are selected for evaluation, and the calculation formula is as follows:
wherein TP represents the number of infrared targets correctly detected in the visible infrared image, FN represents the number of infrared targets which are not detected in the infrared image, FP represents the number of infrared targets which are detected by mistake in the infrared image, TN represents the number of infrared targets which are not detected by mistake in the infrared image, Mo and Mw are obtained by actual test and training, and the indexes of Mp, Mf, Mm, Mo and Mw are tested and the results are analyzed.
The unmanned aerial vehicle vision infrared video with the length of 600 frames is used for carrying out actual test, and the pedestrian and vehicle actual test identification is carried out.
According to the actual test result, comparing the representative target detection network at present, testing the indexes by adopting a hardware platform and a software platform which are the same as those used for model training and testing, a data set and training parameters, and analyzing the test result.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. the invention is applied to the detection of targets in the detection environment which cannot be handled by a visible light camera and a common digital night vision device, such as an unmanned aerial vehicle in the absence of light at night and weather influence environments such as rain fog and the like, can acquire clear infrared spectrum imaging of pedestrians and vehicles about one hundred meters, and can perform real-time and high-precision target detection;
2. the IT-YO L O unmanned aerial vehicle infrared target detection network adopts a basic structure of Tiny-YO L OV3, according to the characteristics of infrared images, the characteristics of a shallow convolution layer are extracted, the infrared small target detection capability is improved, a single-channel convolution kernel is used, the operation amount is reduced, and a detection part uses a detection mode based on a CenterNet structure to reduce the false detection rate and improve the detection speed;
3. forming a feature image layer added to a feature pyramid, wherein the feature pyramid network is improved to a 3-detection scale structure of 13 × 13 pixels, 26 × 26 pixels and 52 × 52 pixels from 13 × 13 pixels and 26 × 26 pixels of a Tiny-YO L OV3, and then a convolution kernel in a first convolution layer is replaced by a single-channel convolution kernel, so that the operand is reduced, the real-time performance is improved, in the detection network part, a CenterNet structure is used for replacing an original YO L O detection layer, the detection method is optimized, the false detection rate is reduced, the real-time performance is further improved, and the detection network is the IT-YO L O unmanned aerial vehicle infrared target detection network;
4. the IT-YO L O adds a certain number of convolution layers on the basis of the original Tiny-YO L OV3 target detection network to extract low-layer convolution characteristics and increase the detection scale, and a CenterNet structure is used in a detection part to improve the detection real-time property and reduce the false detection rate, so that the network structure is deepened on the basis of the original target detection network, the weight of a training model is increased from 34MB to 96MB, the IT-YO L O belongs to a light-weight target detection network, and the model weight can meet the deployment on an airborne embedded system and an edge calculation unit of an unmanned aerial vehicle;
5. adopt unmanned aerial vehicle to carry on infrared thermal imaging platform vision collection preparation data set under the night environment, be in the low latitude respectively, the high altitude angle is shot with the unmanned aerial vehicle vision to carry out the mark of pedestrian and vehicle target with the supervision learning mode, divide training and test set, gather the preparation and be applicable to the infrared target detection data set of unmanned aerial vehicle night to ground pedestrian and vehicle, and generated the real-time infrared target detection model of unmanned aerial vehicle through the training.
Drawings
FIG. 1 is a flow chart of a real-time detection method for an infrared target of an unmanned aerial vehicle according to the present invention;
FIG. 2 is a diagram of a network structure of a Tiny-YO L OV3 according to the present invention;
FIG. 3 is a network architecture diagram of IT-YO L O according to the present invention;
figure 4 is a diagram of the architecture of the centret network according to the present invention;
FIG. 5 is a graph of model training mean loss;
FIG. 6 is a diagram of real-time infrared target detection test results for an unmanned aerial vehicle;
fig. 7 is a flowchart of an airborne thermal imaging target acquisition and detection platform.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
Referring to fig. 1, a real-time detection method for an infrared target of an unmanned aerial vehicle includes the following steps:
(1) an IT-YO L O unmanned aerial vehicle infrared target detection network is constructed;
(2) collecting and manufacturing an unmanned aerial vehicle infrared target detection data set;
(3) training to generate an unmanned aerial vehicle real-time infrared target detection model;
(4) and detecting the infrared target by the unmanned plane.
According to the technical scheme, improvement of a trunk characteristic extraction network and replacement of a network detection layer are carried out on the basis of a Tiny-YO L OV3 lightweight target detection network, an IT-YO L O unmanned aerial vehicle infrared target detection network is constructed, an unmanned aerial vehicle infrared target detection data set is collected and manufactured under the vision of an unmanned aerial vehicle by using infrared imaging, an unmanned aerial vehicle real-time infrared target detection model is generated by training, the unmanned aerial vehicle infrared target detection model is applied to detecting targets under the detection environments which cannot be met by a visible light camera and a common digital night vision device in environments with no light at night, weather influence such as rain and fog and the like, clear pedestrians can be collected around hectometer in infrared spectrum, vehicle imaging is carried out in real time and high-precision target detection, and the defects that the existing large number of target technologies are based on the visible light environment and a target detection method under the environments with no light, rain and fog weather lacks research and practice.
Example 2
Referring to the attached figures 1-4, on the basis of embodiment 1, in the step (1), the IT-YO L O unmanned aerial vehicle infrared target detection network is to improve a backbone network and a detection network on the basis of a Tiny-YO L OV3 lightweight target detection network, the IT-YO L O unmanned aerial vehicle infrared target detection network adopts a Tiny-YO L OV3 basic structure, extracts the characteristics of a shallow convolutional layer according to the characteristics of an infrared image, improves the infrared small target detection capability, reduces the operand by using a single-channel convolutional kernel, and reduces the false detection rate and improves the detection speed by using a detection mode based on a CenterNet structure in a detection part.
For feature extraction of a long-distance low-resolution infrared small target, a Tiny-YO L OV3 target detection network shallow layer convolution Conv4 layer can represent semantic information of the infrared small target more effectively, a Conv3 layer is too small in receptive field, a Conv5 layer is too large in receptive field and contains certain background noise interference, therefore, in order to improve detection capability of the infrared small target, a Maxpool3 layer in a Tiny-YO L OV3 is replaced by a Conv4 layer, a Conv5 layer is added, an upper sampling layer Upessample 2 is added on the basis of a Tiny-YO L OV3 model, a Conv5 layer and an Upessample 2 are connected on a channel dimension, Conv7 performs up-sampling operation through the Upessample 2 to form a feature pyramid added feature layer, a first feature layer is replaced by a single-channel convolution kernel, the operation amount is reduced, the real-time performance is improved, a detection network part is used for replacing a Cente structure, and the target detection method is further used for improving the detection error rate of an original unmanned aerial vehicle detection method for detecting the infrared small target.
The feature pyramid network was lifted by the 13 × and 26 × pixels of the Tiny-YO L OV3 to 3 detection scale structures of 13 ×, 26 × and 52 × pixels, using the CenterNet detection structure on 3 detection scales.
In order to further improve the detection precision of the infrared target in the infrared image, improve the detection real-time performance and reduce the computation amount, a detection network part uses a CenterNet detection structure on 3 detection scales, and replaces the original YO L O detection layer based on an anchor (anchor box) by an anchor free mode (anchor free), so that the improved target detection network is more suitable for detecting the infrared small target.
When the CenterNet structure is used for detection, the scale sensitive area is adopted to adapt to the target objects with different sizes, a central area with a relatively small target and a relatively large target is generated, and whether a bounding box I is reserved or not is acquired in the following manner: tlx，tlyPoints, br, representing the upper left corner of the framex，bryRepresenting the point at the bottom right corner of the box, a center region j is defined, and the coordinates of the point at the top left corner are defined as (ctl)x,ctly) Lower right corner point (cbr)x,cbry)，ctlx，ctly，cbrx，cbrySatisfies the following formula (a) in the following formula,
(1) where n is an odd number and represents the size of the central region j, n is 3 when the bounding box is smaller than 150, and n is 5 when the bounding box is larger than 150.
Example 3
As shown in fig. 1-5, on the basis of embodiment 1, an unmanned aerial vehicle is adopted to carry an infrared thermal imaging platform at low altitude and high altitude distance respectively in night environment for shooting, clear infrared images for identification and detection are collected from top to bottom, and infrared thermal imaging images of night scenes under 8000 unmanned aerial vehicle visual angles are collected as an unmanned aerial vehicle infrared target detection data set.
In a model training link, image samples in the data set are all converted into images with 416 × pixels, small-batch training is carried out by taking 100 images as one batch during training, a batch of images are trained, the weight is updated once, the attenuation rate of the weight is set to be 0.0005, the momentum is set to be 0.9, the initial learning rate is set to be 0.001, 20000 iterations are carried out on an IT-YO L O network, the model is stored once after every 2000 iterations, the average loss of the final model is reduced to be below 0.2, and the model with the highest precision is selected as the real-time infrared target detection model of the unmanned aerial vehicle.
The hardware platform used for model training and testing is Core i7-8750H 2.2Ghz processor +16GB memory + Geforce GTX 10808 GB video card, and the software platform uses Win10+ tensoflow1.9.0 + CUDA9.2+ VS2017+ opencv 4.0.
And selecting the indexes of average accuracy Mp, average false detection rate Mf, average missed detection rate Mm, average operation speed Mo and detection model size Mw for evaluation, wherein the calculation formula is as follows:
wherein TP represents the number of infrared targets correctly detected in the visible infrared image, FN represents the number of infrared targets which are not detected in the infrared image, FP represents the number of infrared targets which are detected by mistake in the infrared image, TN represents the number of infrared targets which are not detected by mistake in the infrared image, Mo and Mw are obtained by actual test and training, and the indexes of Mp, Mf, Mm, Mo and Mw are tested and the results are analyzed.
Example 4
As shown in fig. 1-6, on the basis of embodiment 3, an actual test is performed by using a visual infrared video of the unmanned aerial vehicle with a length of 600 frames, and actual test identification is performed on detected pedestrians and vehicles, and the result is shown in fig. 6.
According to actual test results, comparing the current representative target detection network, testing the indexes by adopting a hardware platform and a software platform which are the same as those used for model training and testing, a data set and training parameters, and analyzing the test results, wherein the test results are shown in table 1.
TABLE 1 comparative analysis of the comprehensive Properties test
From table 1, IT is known that the average detection rate of an IT-YO L O infrared target detection network reaches 95% of a YO L OV3 target detection network, exceeds 24% of an SSD 300L 0300 target detection network, 1% of a RetinaNet-50-500 target detection network, 25% of a Tiny-YO L OV 695 2 target detection network, reduces the average false detection rate by 5% relative to an SSD 300L 3300 target detection network, reduces by 9% relative to a RetinaNet-50-500 target detection network, reduces by 10% relative to a YO L2 OV3 target detection network, reduces by 6% relative to a Tiny-YO L4 OV3 target detection network, exceeds 6% of a YO L OV3 target detection network, reduces by 19% relative to a SSD300 target detection network, reduces by 1% relative to a retinet-50-500 target detection network, reduces by 20% relative to a Tiny-YO 3 target detection network, reduces by 19% relative to an average operational speed higher than that of a tie net-300 target detection network, and does not greatly reduce the weight of an embedded unmanned aerial vehicle infrared target detection network by 20% when the unmanned aerial vehicle detects by a Tiny unmanned aerial vehicle model, the unmanned aerial vehicle, the unmanned.
Example 5
Like figure 7, adopt unmanned aerial vehicle to carry on infrared thermal imaging detection module and carry out front end infrared image collection, be applied to under the totally dark environment of no light source night, and do not influence target detection and recognition effect under the condition that has certain rain and fog weather, because thermal imager output is AV format single channel signal, consequently carry out format conversion through the data acquisition integrated circuit board, convert single channel digital image format into, adopt embedded system's Edge calculation module on the unmanned aerial vehicle platform to carry on the light-weight target detection network based on deep learning like Google Edge TPU.
In fig. 5 of the present application, Avg L oss Curve is the model training average loss Curve, Avg L oss is the model training average loss, and batchs is the model training batch.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. An unmanned aerial vehicle infrared target real-time detection method is characterized by comprising the following steps:
(1) an IT-YO L O unmanned aerial vehicle infrared target detection network is constructed;
(2) collecting and manufacturing an unmanned aerial vehicle infrared target detection data set;
(3) training to generate an unmanned aerial vehicle real-time infrared target detection model;
(4) and detecting the infrared target by the unmanned plane.
2. The real-time detection method for infrared targets of unmanned aerial vehicles according to claim 1, wherein in step (1), the IT-YO L O unmanned aerial vehicle infrared target detection network is improved by a backbone network and a detection network based on a Tiny-YO L OV3 lightweight target detection network.
3. The real-time detection method for the infrared target of the unmanned aerial vehicle as claimed in claim 2, wherein a Maxpool3 layer in a Tiny-YO L OV3 is replaced by a Conv4 layer, the Conv5 layer is added, an upsampling layer Upesple 2 is added on the basis of a Tiny-YO L OV3 model, the Conv5 layer and the Upesple 2 are connected in a channel dimension, the Conv7 is subjected to upsampling operation through the Upesple 2 to form a feature image layer with an added feature pyramid, the convolution kernel in the first convolution layer is replaced by a single-channel convolution kernel, and a CenterNet structure is used for replacing an original YO L O detection layer, namely, the IT-YO L O unmanned aerial vehicle infrared target detection network.
4. The real-time detection method of the infrared target of the unmanned aerial vehicle as claimed in claim 3, wherein the feature pyramid network is 3 detection scale structures of 13 × 13, 26 × 26 and 52 × 52 pixels.
5. The real-time detection method of the infrared target of the unmanned aerial vehicle as claimed in claim 4, wherein a CenterNet detection structure is used in 3 detection scales, and an original YO L O detection layer based on the anchor point is replaced by an anchor point-free mode.
6. The real-time detection method for the infrared target of the unmanned aerial vehicle as claimed in claim 5, wherein when the centret structure is used for detection, a size sensitive area is adopted to adapt to targets with different sizes, a central area with a relatively small target and a relatively large target is generated, and whether a bounding box I is reserved or not is obtained by the following method: tlx，tlyPoints, br, representing the upper left corner of the framex，bryRepresenting the point at the bottom right corner of the box, a center region j is defined, and the coordinates of the point at the top left corner are defined as (ctl)x,ctly) Lower right corner point (cbr)x,cbry)，ctlx，ctly，cbrx，cbrySatisfies the following formula (a) in the following formula,
where n is an odd number and represents the size of the central region j, n is 3 when the bounding box is smaller than 150, and n is 5 when the bounding box is larger than 150.
7. The real-time detection method for the infrared target of the unmanned aerial vehicle as claimed in claim 1, wherein in step (2), the unmanned aerial vehicle is adopted to carry the infrared thermal imaging platform to shoot at low altitude and high altitude distance respectively in the night environment, clear infrared images for identification and detection are collected from the top to bottom view angles, and infrared thermal imaging images of night scenes at 8000 visual angles of the unmanned aerial vehicle are collected as the infrared target detection data set of the unmanned aerial vehicle.
8. The real-time detection method of the infrared target of the unmanned aerial vehicle as claimed in claim 7, characterized in that an unmanned aerial vehicle infrared target detection data set is divided into a training set and a test set according to a ratio of 5:1, class 2 targets to be detected, pedestrians and vehicles are labeled by using a YO L O-MARK tool, all image samples in the data set are converted into 416 × 416 pixel images in a model training link, during training, a small batch of training is performed by taking 100 images as a batch, a batch of images are trained, a weight is updated once, the attenuation rate of the weight is set to 0.0005, momentum is set to 0.9, an initial learning rate is set to 0.001, 20000 iterations are performed on an IT-YO L O network, a model is stored once after every 2000 iterations, and finally the average loss of the model is reduced to below 0.2, and the model with the highest precision is selected as the unmanned aerial vehicle real-time infrared target detection model.
9. The real-time detection method for the infrared target of the unmanned aerial vehicle as claimed in claim 8, wherein the average accuracy Mp, the average false detection rate Mf, the average missed detection rate Mm, the average operation speed Mo, and the detection model size Mw index are selected for evaluation, and the calculation formula is as follows:
wherein TP represents the number of infrared targets correctly detected in the visible infrared image, FN represents the number of infrared targets which are not detected in the infrared image, FP represents the number of infrared targets which are detected by mistake in the infrared image, TN represents the number of infrared targets which are not detected by mistake in the infrared image, Mo and Mw are obtained by actual test and training, and the indexes of Mp, Mf, Mm, Mo and Mw are tested and the results are analyzed.
10. The real-time detection method of the infrared target of the unmanned aerial vehicle as claimed in claim 7, wherein the low altitude is within 30 meters from the ground, the high altitude is within 30-150 meters from the ground, and the visual angle in the infrared image which is clear and can be identified and detected is collected from top to bottom is 30-60 degrees.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010191052.7A CN111444801A (en) | 2020-03-18 | 2020-03-18 | Real-time detection method for infrared target of unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010191052.7A CN111444801A (en) | 2020-03-18 | 2020-03-18 | Real-time detection method for infrared target of unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111444801A true CN111444801A (en) | 2020-07-24 |
Family
ID=71653366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010191052.7A Pending CN111444801A (en) | 2020-03-18 | 2020-03-18 | Real-time detection method for infrared target of unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111444801A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016449A (en) * | 2020-08-27 | 2020-12-01 | 西华大学 | Vehicle identification and detection method based on deep learning |
| CN112052802A (en) * | 2020-09-09 | 2020-12-08 | 上海工程技术大学 | Front vehicle behavior identification method based on machine vision |
| CN112071058A (en) * | 2020-08-14 | 2020-12-11 | 深延科技(北京)有限公司 | Road traffic monitoring and vehicle abnormity, contraband and fire detection method and system based on deep learning |
| CN112686340A (en) * | 2021-03-12 | 2021-04-20 | 成都点泽智能科技有限公司 | Dense small target detection method based on deep neural network |
| CN112767297A (en) * | 2021-02-05 | 2021-05-07 | 中国人民解放军国防科技大学 | Infrared unmanned aerial vehicle group target simulation method based on image derivation under complex background |
| CN113688748A (en) * | 2021-08-27 | 2021-11-23 | 武汉大千信息技术有限公司 | Fire detection model and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180284743A1 (en) * | 2016-05-09 | 2018-10-04 | StrongForce IoT Portfolio 2016, LLC | Methods and systems for industrial internet of things data collection for vibration sensitive equipment |
| CN109508710A (en) * | 2018-10-23 | 2019-03-22 | 东华大学 | Based on the unmanned vehicle night-environment cognitive method for improving YOLOv3 network |
| CN110334661A (en) * | 2019-07-09 | 2019-10-15 | 国网江苏省电力有限公司扬州供电分公司 | Infrared power transmission and transformation abnormal heating point target detecting method based on deep learning |
| US20190339688A1 (en) * | 2016-05-09 | 2019-11-07 | Strong Force Iot Portfolio 2016, Llc | Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial internet of things |
-
2020
- 2020-03-18 CN CN202010191052.7A patent/CN111444801A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180284743A1 (en) * | 2016-05-09 | 2018-10-04 | StrongForce IoT Portfolio 2016, LLC | Methods and systems for industrial internet of things data collection for vibration sensitive equipment |
| US20190339688A1 (en) * | 2016-05-09 | 2019-11-07 | Strong Force Iot Portfolio 2016, Llc | Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial internet of things |
| CN109508710A (en) * | 2018-10-23 | 2019-03-22 | 东华大学 | Based on the unmanned vehicle night-environment cognitive method for improving YOLOv3 network |
| CN110334661A (en) * | 2019-07-09 | 2019-10-15 | 国网江苏省电力有限公司扬州供电分公司 | Infrared power transmission and transformation abnormal heating point target detecting method based on deep learning |
Non-Patent Citations (4)
| Title |
|---|
| ZHENG YANG等: "Detecting Small Objects in Urban Settings Using SlimNet Model", 《IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING》 * |
| 刘军等: "基于增强Tiny YOLOV3算法的车辆实时检测与跟踪", 《农业工程学报》 * |
| 易诗等: "基于红外热成像与改进YOLOV3的夜间野兔监测方法", 《农业工程学报》 * |
| 王殿伟等: "改进的YOLOv3红外视频图像行人检测算法", 《西安邮电大学学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112071058A (en) * | 2020-08-14 | 2020-12-11 | 深延科技(北京)有限公司 | Road traffic monitoring and vehicle abnormity, contraband and fire detection method and system based on deep learning |
| CN112016449A (en) * | 2020-08-27 | 2020-12-01 | 西华大学 | Vehicle identification and detection method based on deep learning |
| CN112052802A (en) * | 2020-09-09 | 2020-12-08 | 上海工程技术大学 | Front vehicle behavior identification method based on machine vision |
| CN112052802B (en) * | 2020-09-09 | 2024-02-20 | 上海工程技术大学 | Machine vision-based front vehicle behavior recognition method |
| CN112767297A (en) * | 2021-02-05 | 2021-05-07 | 中国人民解放军国防科技大学 | Infrared unmanned aerial vehicle group target simulation method based on image derivation under complex background |
| CN112767297B (en) * | 2021-02-05 | 2022-09-23 | 中国人民解放军国防科技大学 | Infrared unmanned aerial vehicle group target simulation method based on image derivation under complex background |
| CN112686340A (en) * | 2021-03-12 | 2021-04-20 | 成都点泽智能科技有限公司 | Dense small target detection method based on deep neural network |
| CN113688748A (en) * | 2021-08-27 | 2021-11-23 | 武汉大千信息技术有限公司 | Fire detection model and method |
| CN113688748B (en) * | 2021-08-27 | 2023-08-18 | 武汉大千信息技术有限公司 | Fire detection model and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111444801A (en) | Real-time detection method for infrared target of unmanned aerial vehicle | |
| CN109146929B (en) | Object identification and registration method based on event-triggered camera and three-dimensional laser radar fusion system | |
| CN108109385B (en) | System and method for identifying and judging dangerous behaviors of power transmission line anti-external damage vehicle | |
| CN111460968B (en) | Unmanned aerial vehicle identification and tracking method and device based on video | |
| CN106462737A (en) | Systems and methods for haziness detection | |
| CN111582234B (en) | Large-scale oil tea tree forest fruit intelligent detection and counting method based on UAV and deep learning | |
| CN108537122A (en) | Include the image co-registration acquisition system and image storage method of meteorologic parameter | |
| CN115439424A (en) | Intelligent detection method for aerial video image of unmanned aerial vehicle | |
| CN113192646B (en) | Target detection model construction method and device for monitoring distance between different targets | |
| CN114255407B (en) | High-resolution-based anti-unmanned aerial vehicle multi-target identification and tracking video detection method | |
| CN110660222A (en) | Intelligent environment-friendly electronic snapshot system for black smoke vehicle on road | |
| CN113743358A (en) | Landscape visual feature recognition method based on all-dimensional acquisition and intelligent calculation | |
| CN112349150A (en) | Video acquisition method and system for airport flight guarantee time node | |
| CN115962757A (en) | Unmanned aerial vehicle surveying and mapping method, system and readable storage medium | |
| CN111476314B (en) | Fuzzy video detection method integrating optical flow algorithm and deep learning | |
| CN110837771A (en) | Bird identification method and device based on convolutional neural network | |
| CN113723833B (en) | Method, system, terminal equipment and storage medium for evaluating quality of forestation actual results | |
| CN112734794B (en) | Moving target tracking and positioning method based on deep learning | |
| CN114973028A (en) | Aerial video image real-time change detection method and system | |
| CN114943984A (en) | Intelligent bird detection and bird repelling method and device based on radar point cloud and visual image | |
| CN114119713A (en) | Forest land baldness detection method based on artificial intelligence and unmanned aerial vehicle remote sensing | |
| CN113903002A (en) | Tower crane below abnormal intrusion detection method based on tower crane below personnel detection model | |
| CN116843691B (en) | Photovoltaic panel hot spot detection method, storage medium and electronic equipment | |
| CN116229286B (en) | Knowledge-driven space target situation awareness method and system | |
| Zhang et al. | Vehicle identification and speed calculation method for general road camera perspective |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200724 |
|
| RJ01 | Rejection of invention patent application after publication |