CN116109950A - Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method - Google Patents
Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method Download PDFInfo
- Publication number
- CN116109950A CN116109950A CN202310002983.1A CN202310002983A CN116109950A CN 116109950 A CN116109950 A CN 116109950A CN 202310002983 A CN202310002983 A CN 202310002983A CN 116109950 A CN116109950 A CN 116109950A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- target
- model
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 115
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000000007 visual effect Effects 0.000 title claims abstract description 26
- 230000007246 mechanism Effects 0.000 claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000010586 diagram Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 5
- 238000011179 visual inspection Methods 0.000 claims 2
- 238000004422 calculation algorithm Methods 0.000 description 13
- 238000013135 deep learning Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/52—Scale-space analysis, e.g. wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
Abstract
The invention provides a low airspace anti-unmanned aerial vehicle visual detection, identification and tracking method, which belongs to the field of unmanned aerial vehicle countering, and comprises the following steps: firstly, acquiring unmanned aerial vehicle samples, constructing a target detection training sample set, a target fine-granularity recognition training sample set and a target tracking data set, then optimizing a frame of a target detection model to obtain an improved target detection model, detecting and positioning the unmanned aerial vehicle in an actual visual image to obtain an unmanned aerial vehicle global image, then improving the target fine-granularity recognition model, introducing a scale self-adaptive attention mechanism and joint probability prediction, extracting and recognizing unmanned aerial vehicle parts on the unmanned aerial vehicle global image, and finally, tracking the unmanned aerial vehicle, and recording the track and the tracking video image of each model unmanned aerial vehicle. The method has higher detection and identification precision, and can solve the problems that the prior art has higher missed detection, can not identify the model of the unmanned aerial vehicle and is easy to fail in tracking the unmanned aerial vehicle.
Description
Technical Field
The invention belongs to the field of anti-unmanned aerial vehicles, and particularly relates to a low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method.
Background
In recent years, unmanned aerial vehicle industry rapidly develops, unmanned aerial vehicles are widely applied in military and civil fields, and screening and countering of unmanned aerial vehicles are necessary means for guaranteeing orderly operation of unmanned aerial vehicles. The vision detection, identification and tracking technology is an important means for screening and countering the unmanned aerial vehicle, and due to the fact that the low-airspace unmanned aerial vehicle has complex aircraft background (trees, buildings and the like), large scale (occupation proportion) change and more foreign matter interference (flying birds and flying insects), the problems of detection omission, identification errors, tracking failure and the like often occur by adopting a traditional image processing method.
The invention discloses a method for identifying and positioning a low-altitude target, which is applied to an unmanned aerial vehicle countercar, wherein the unmanned aerial vehicle countercar comprises a task load and a car carrying platform, the task load comprises a low-altitude detection radar, a radio detection device and an image sensor module, the target information in the image sensor is detected and identified by constructing an unmanned aerial vehicle identification model, the attribute information of the target is obtained, the azimuth and pitch information of the target are obtained by combining the azimuth and pitch information of the turning of a turntable of the image sensor module, the azimuth and distance information of the target is obtained by utilizing the low-altitude detection radar, the position information and the attribute information of the target are obtained by utilizing the radio detection device, and then the position information of the target is detected more accurately by fusing the information.
The invention relates to an intelligent autonomous unmanned aerial vehicle countering system, which is claimed to be protected by a patent application of the name of the intelligent autonomous unmanned aerial vehicle countering system of the Chengdu longitudinal aviation intelligent science and technology limited company, and belongs to the technical field of unmanned aerial vehicle intrusion countering, and the intelligent autonomous unmanned aerial vehicle countering system comprises an AI energy-imparting network attack module, an intelligent unmanned aerial vehicle network capturing module, a GPS navigation decoy module and a multi-frequency band radio interference suppression module; and under the optimal configuration of multiparty resources, realizing real-time networking deployment of various defense countermeasures and automatically suppressing intrusion targets. After the unmanned aerial vehicle is identified and tracked, the AI (automatic identification) energy-imparting network attack module can invade the unmanned aerial vehicle control link and take over the black flying unmanned aerial vehicle, the intelligent unmanned aerial vehicle network capturing module can safely drop the invasive unmanned aerial vehicle to the ground, the GPS navigation decoy module induces the unmanned aerial vehicle to fly to a preset area, and the multi-band radio interference suppression module releases interference signals to technically block the unmanned aerial vehicle from flying. The invention can be provided with the reaction equipment suitable for application according to different application scenes.
The thirty-eighth research of China electronic technology group company discloses a patent application named as a high-resolution-based anti-unmanned aerial vehicle multi-target recognition and tracking video detection method, and specifically discloses a high-resolution-based anti-unmanned aerial vehicle multi-target recognition and tracking video detection method, which comprises the following steps: adopting a 4K high-definition monitoring camera to collect photoelectric data; acquiring a training sample and labeling the training sample; dividing a sample into a plurality of images, and labeling the divided images; constructing a multi-scale depth target detection network; inputting the images of the sample library into a multi-scale depth target detection network for model training to obtain a trained parameter model; the parameter model respectively identifies the images after segmentation and the images before segmentation, and the identification results before and after segmentation of the images are synthesized to obtain the position information of the unmanned aerial vehicle; after the unmanned aerial vehicle target frame is identified, the unmanned aerial vehicle target is locked. The advantages are that: the visual field range is effectively improved, the tracking target is easier, the unmanned aerial vehicle is captured faster and more conveniently, the identification accuracy is effectively improved, the possibility of false alarm is greatly reduced, and the unmanned aerial vehicle searching time is shortened.
The institute of electronic technology university (Huzhou) discloses a patent application named as a deep learning-based anti-unmanned aerial vehicle tracking method, and in particular relates to a deep learning-based anti-unmanned aerial vehicle tracking method. Based on the current detection technology state and considering factors such as detection precision, distance, cost and the like, the visual correlation technology is adopted to realize the detection of the unmanned aerial vehicle, the visual detection is divided into two stages of detection and tracking, and a method which is more suitable for tracking the unmanned aerial vehicle is obtained by using a deep learning method. In order to ensure the final tracking speed, a lightweight detection network is designed, the whole detection network has high detection precision while ensuring the speed, and the designed tracking network is a further improvement on the original network. Overall, the last tracking network can effectively and quickly track the unmanned aerial vehicle.
In addition, journal articles research on target recognition and tracking algorithm in anti-unmanned aerial vehicle system, and research on target detection and tracking algorithm in anti-unmanned aerial vehicle scene is performed respectively. Aiming at the problem of low detection rate of weak and small targets, a moving target detection algorithm based on space-time continuity is provided, and a target recognition algorithm is operated to recognize the targets. Then, an anti-unmanned aerial vehicle tracking algorithm test platform is also constructed, comparison analysis is carried out on several real-time target tracking algorithms, and target tracking in an anti-unmanned aerial vehicle scene is carried out by using an STC algorithm. Aiming at the problems of poor scale transformation effect and model updating mechanism of STC algorithm, an improved STC algorithm is provided. Finally, the anti-unmanned aerial vehicle system photoelectric module is subjected to software and hardware design.
The journal paper 'research on low airspace anti-unmanned aerial vehicle visual detection and tracking method' starts from visual means, combines with the current deep learning method, explores a method for unmanned aerial vehicle detection and tracking, and mainly works as follows: 1) An anchor-frame-free detection network MS-Center Net is designed based on a space pyramid module and a lightweight characteristic extraction module. The network starts from practical application, optimizes the scene of unmanned aerial vehicle target detection, and the finally obtained network model is small in parameter and calculation amount and improves the precision compared with the network model before optimization. 2) A twin neural network G-Siam CPP for tracking is designed based on GFL loss. The network solves the problem of asynchronous classification evaluation and quality evaluation in the original network structure, and simultaneously uses more general distribution to replace special Dirac distribution on the prediction of the frame position. Experiments show that the network has higher positioning accuracy. 3) Based on the idea of re-detection, an LG-Siam CPP network is designed by combining MS-Center Net and G-Siam CPP. The network relieves the offset problem caused by error accumulation in the tracking process or the tracking failure problem caused by the fact that the tracking template is not updated. The new network can update the changed template in time in the tracking process and correct the position error of the template in time, and better robustness is shown.
According to the conditions of domestic and foreign patents and related journal articles, it is known that more patent documents and journal articles relate to the design and use of yolo algorithm to detect and identify the target, and a filtering algorithm is often used to track the target. However, the target detection algorithm adopted by the existing anti-unmanned aerial vehicle method detects that the unmanned aerial vehicle has higher missed detection, the model of the unmanned aerial vehicle cannot be identified, and tracking failure can be caused by fast illumination, deformation and movement speed when the unmanned aerial vehicle is tracked.
Therefore, a novel low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method needs to be developed to overcome the defects of the prior art.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method, which is used for improving a target detection model and a target fine-granularity identification model to obtain higher detection and identification precision and solving the problems that the prior art has higher omission, the model of the unmanned aerial vehicle cannot be identified and the unmanned aerial vehicle tracking is easy to fail.
In order to achieve the above purpose, the invention provides a low airspace anti-unmanned aerial vehicle visual detection, identification and tracking method, which comprises the following steps:
s1: collecting unmanned aerial vehicle samples, constructing a target detection training sample set, a target fine granularity recognition training sample set and a target tracking data set,
s2: optimizing the frame of the target detection model, adding a small target layer to be suitable for the actual condition that the unmanned aerial vehicle presents small characteristics in the low airspace visual image, obtaining an improved target detection model,
training an improved target detection model by adopting the target detection training sample set in the step S1, detecting and positioning the unmanned aerial vehicle in the actual visual image by adopting the trained improved target detection model to acquire an unmanned aerial vehicle global image,
s3: improving the target fine granularity recognition model, introducing a scale self-adaptive attention mechanism and joint probability prediction to improve recognition accuracy, obtaining the scale self-adaptive attention mechanism target fine granularity recognition model,
training the scale self-adaptive attention mechanism target fine granularity recognition model by adopting the target fine granularity recognition training sample set in the step S1, extracting and recognizing unmanned aerial vehicle components by adopting the trained scale self-adaptive attention mechanism target fine granularity recognition model to the unmanned aerial vehicle global image obtained in the step S2, further completing recognition of unmanned aerial vehicle models,
s4: tracking the unmanned aerial vehicle obtained in the step S3, and recording the track and tracking video image of each model unmanned aerial vehicle.
Further, in step S1, at least five types of unmanned aerial vehicle samples are collected, at least 100 unmanned aerial vehicle sample images of each type are collected, and the shooting angle, gesture and size of each unmanned aerial vehicle sample image are different, so that the unmanned aerial vehicle sample images can be used for constructing a target detection training sample set, a target fine-grained recognition training sample set and a target tracking data set.
Further, in step S2, the target detection model is a yolov5S target detection model, and the frame of the yolov5S target detection model is optimized, that is, an anchor initialization parameter is modified in a yolov5S target detection model configuration file, a group of initialization values capable of being used for detecting targets with a size of more than 16×16 pixels are added, a small target detection layer is added to a head layer of yolov5S, and a feature map of the head middle detection layer is fused with a feature map of a corresponding layer of a backhaul network layer, so that a larger feature map is obtained for small target detection
Further, in step S3, the extraction and recognition of the unmanned aerial vehicle component are specifically performed, namely, the feature images of different components of the unmanned aerial vehicle are extracted first, global pooling operation is performed on the feature images of different components of the unmanned aerial vehicle to obtain one-dimensional feature descriptors, the one-dimensional feature descriptors are processed, each channel of the feature images of different components of the unmanned aerial vehicle is endowed with different weights, attention feature vector distribution is performed to obtain a new feature image, the new feature image is polymerized in the channel dimension to generate a two-dimensional feature image, further, a mask image is obtained, an unmanned aerial vehicle component image is obtained according to the mask image and the foreground scale image, and the feature of the unmanned aerial vehicle component image is compared with the feature images of different components of the unmanned aerial vehicle, so that the extraction and recognition of the unmanned aerial vehicle component are completed.
Further, in step S3, the unmanned aerial vehicle model identification is specifically:
first, the output of each unmanned aerial vehicle branch in the scale-adaptive attention mechanism target fine-granularity recognition model is expressed as logits y E R N N represents the total number of classes of drones,
each branch in the scale-adaptive attention mechanism target fine-granularity recognition model is then given a dynamic probability parameter, λ, to weight the uncertainty, λ representing the confidence score used to evaluate the branch output logits y,
then, inputting the output of each branch in the scale self-adaptive attention mechanism target fine granularity recognition model to a softMax function, calculating to obtain the confidence score of each branch, further obtaining the dynamic probability parameter of logits y,
finally, adding the output of each branch after uncertain weighting in the scale self-adaptive attention mechanism target fine granularity identification model to obtain y concat Obtaining y according to dynamic probability parameters of logits y concat Category value, select y concat The class corresponding to the maximum value of the class is used as a model prediction result, and is an unmanned model number.
Further, in step S4, the unmanned aerial vehicle obtained in step S3 is tracked by using a TCTrack target tracking model trained by the target tracking dataset, where the TCTrack target tracking model refers to a model for tracking an air target by using a temporal context. The temporal context refers to a method of tracking the execution of processes in a model.
In general, the above technical solutions conceived by the present invention have the following compared with the prior art
The beneficial effects are that:
with the rapid development of the deep learning technology, the precision of the visual detection, recognition and tracking model based on the deep learning is greatly improved. The invention provides a low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method, which realizes the detection of unmanned aerial vehicles by improving a yolov5s target detection model, identifies the model of the unmanned aerial vehicle by utilizing an improved target fine-grained identification model, and realizes the tracking of the unmanned aerial vehicle by utilizing a TCTrack tracking model. Specifically, based on deep learning, the detection of the unmanned aerial vehicle by applying the yolov5s target detection model, the identification of target fine granularity are applied to the identification of the model of the unmanned aerial vehicle, the TCTrack tracking model is applied to the tracking of the unmanned aerial vehicle detected and identified, and the yolov5s target detection model and the target fine granularity identification model are improved, so that the detection, identification and tracking of the unmanned aerial vehicle in a low airspace are realized, and the reaction of the unmanned aerial vehicle is achieved.
Through the improvement, the precision of visual detection, identification and tracking models is greatly improved, the detection, identification and tracking of the low-airspace unmanned aerial vehicle are realized, and the countering of the unmanned aerial vehicle is realized.
Drawings
FIG. 1 is a flow chart of a low airspace anti-unmanned aerial vehicle visual detection, identification and tracking method provided by an embodiment of the invention;
FIG. 2 is a flowchart of a specific implementation of a low airspace anti-unmanned aerial vehicle vision detection, identification and tracking method provided by an embodiment of the invention;
fig. 3 is a diagram of a frame extracted based on unmanned aerial vehicle target attention mechanism components provided by an embodiment of the present invention;
fig. 4 is a frame diagram of target fine-grained target recognition of a unmanned aerial vehicle based on joint prediction, which is provided by an embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of a low airspace anti-unmanned aerial vehicle visual detection, identification and tracking method provided by an embodiment of the present invention, which is mainly divided into the following steps:
s1: collecting unmanned aerial vehicle samples, constructing a target detection training sample set, a target fine granularity recognition training sample set and a target tracking data set,
s2: optimizing the frame of the target detection model, adding a small target layer to obtain an improved target detection model, training the improved target detection model by adopting the target detection training sample set in the step S1, detecting and positioning the unmanned aerial vehicle in the actual visual image by adopting the trained improved target detection model to obtain a global image of the unmanned aerial vehicle,
s3: improving the target fine granularity recognition model, introducing a scale self-adaptive attention mechanism and joint probability prediction, adopting the target fine granularity recognition training sample set in the step S1 to train the target fine granularity recognition model of the scale self-adaptive attention mechanism, adopting the trained target fine granularity recognition model of the scale self-adaptive attention mechanism to extract and recognize unmanned aerial vehicle components of the unmanned aerial vehicle global image obtained in the step S2, further completing recognition of unmanned aerial vehicle models,
s4: tracking the unmanned aerial vehicle obtained in the step S3, and recording the track and tracking video image of each model unmanned aerial vehicle.
Fig. 2 is a flowchart of a specific implementation of the low airspace anti-unmanned aerial vehicle vision detection, identification and tracking method provided by the embodiment of the present invention, as can be seen in conjunction with fig. 1 and 2,
in step S1, unmanned aerial vehicle samples of at least multiple model are collected, unmanned aerial vehicle sample images of each model are more, and the shooting angle, gesture and size of each unmanned aerial vehicle sample image are different, the sample is annotated to the manual work, unmanned aerial vehicles of different models are put into the file of corresponding model manually, in order to be used for constructing target detection training sample set, target fine granularity discernment training sample set and target tracking data set.
In step S2, the target detection model is a yolov5S target detection model, and optimizing a frame of the yolov5S target detection model is specifically to modify an anchor initialization value in a yolov5S model configuration file, and increase a group of initialization values which can be used for detecting targets with a size of more than 16×16 pixels; adding a small target detection layer on a head layer of yolov5s, and fusing a feature map of the head middle detection layer with a corresponding hierarchical feature map of a backhaul network layer to obtain a larger feature map for small target detection
Specifically, extracting and identifying unmanned aerial vehicle parts, namely extracting to obtain feature images of different unmanned aerial vehicle parts, carrying out global pooling operation on the feature images of the different unmanned aerial vehicle parts to obtain one-dimensional feature descriptors, processing the one-dimensional feature descriptors, endowing each channel of the feature images of the different unmanned aerial vehicle parts with different weights, carrying out attention feature vector distribution to obtain a new feature image, carrying out aggregation on the new feature images in channel dimension to generate two-dimensional feature images, further obtaining a mask image, obtaining an unmanned aerial vehicle part image according to the mask image and a foreground scale image, and comparing features in the feature images of the unmanned aerial vehicle part image and the feature images of the different unmanned aerial vehicle parts to finish extraction and identification of unmanned aerial vehicle parts.
Specifically, the unmanned aerial vehicle model identification is specifically:
first, the output of each unmanned aerial vehicle branch in the scale-adaptive attention mechanism target fine-granularity recognition model is expressed as logits y E R N N represents unmanned aerial vehicleThe total number of categories,
each branch in the scale-adaptive attention mechanism target fine-granularity recognition model is then given a dynamic probability parameter, λ, to weight the uncertainty, λ representing the confidence score used to evaluate the branch output logits y,
then, inputting the output of each branch in the scale self-adaptive attention mechanism target fine granularity recognition model to a softMax function, calculating to obtain the confidence score of each branch, further obtaining the dynamic probability parameter of logits y,
finally, adding the output of each branch after uncertain weighting in the scale self-adaptive attention mechanism target fine granularity identification model to obtain y concat Obtaining y according to dynamic probability parameters of logits y concat Category value, select y concat The class corresponding to the maximum value of the class is used as a model prediction result, and is an unmanned model number.
Select y concat And taking the class corresponding to the maximum value of the middle class as a model prediction result, namely the unmanned aerial vehicle type.
Specifically, in step S4, the unmanned aerial vehicle obtained in step S3 is tracked by using a TCtrack target tracking model, which refers to an algorithm model for tracking an air target by using a time context.
Fig. 3 is an extraction frame diagram based on an unmanned aerial vehicle target attention mechanism component provided by an embodiment of the present invention, fig. 4 is a fine-grained target recognition frame diagram based on a joint prediction unmanned aerial vehicle target provided by an embodiment of the present invention, and the following further details the method of the present invention with reference to fig. 2, 3 and 4:
1. sample collection, specifically, collect 5 unmanned aerial vehicle samples of type, unmanned aerial vehicle sample 100 of each type requires unmanned aerial vehicle scale different in the sample, and shooting angle is different, and the gesture is different.
2. And constructing a target detection data set, specifically, uniformly fusing 5 model unmanned aerial vehicle sample sets into one type of unmanned aerial vehicle sample set, marking the samples by using a labelmg marking tool, and constructing a VOC target detection training data set.
3. And constructing a fine-grained identification data set, specifically, respectively classifying 5 types of unmanned aerial vehicle samples, respectively placing 5 types of unmanned aerial vehicles into corresponding folders of different types, and constructing a weak supervision training sample data set.
4. And constructing a target tracking data set, specifically, labeling the collected unmanned aerial vehicle video series of each model, and constructing an OTB target tracking data set.
5. Improving a target detection model, specifically improving a target detection model yolov5s, aiming at the problem that the unmanned aerial vehicle presents small characteristics in a low airspace visual image, modifying a yolov5s model configuration file, and adding a group of initialization values which can be used for detecting targets with the size of more than 16 multiplied by 16 pixels into an anchor initialization value; and adding a small target detection layer on a head layer of yolov5s, and fusing the feature map of the head middle detection layer with the corresponding hierarchical feature map of the backhaul network layer to adapt to better detection of the unmanned aerial vehicle in the image.
The yolov5s target detection model configuration file comprises a category number, a model depth width parameter, an anchor parameter, a background network layer and a head layer, wherein the anchor is a mode initial detection parameter, the background network layer has the function of extracting image features, the head layer comprises two parts, namely a Neck Neck and a Prediction, the Neck Neck completes multi-scale fusion of the features extracted by the background network, and the Prediction completes detection of targets in images.
6. The unmanned aerial vehicle component characteristic representation, in particular, is favorable for improving the bilinear target fine granularity recognition model, introduces a scale self-adaptive attention mechanism, and obtains the scale self-adaptive attention mechanism target fine granularity recognition model. The method specifically comprises the following steps:
(1) Feature representation, and a foreground scale feature extractor branches to obtain a feature map F of the unmanned aerial vehicle O ∈R h×w×c The different parts of the unmanned aerial vehicle are characterized by F O ＝[x 1 x x …x c ]∈R h×w×c Wherein c represents the feature map channel, h represents the feature map height, w represents the feature map width, x 1 、x 2 、···、x c Representing characteristicsGraph, x 1 Is the 1 st channel of the characteristic diagram, x c Representing the c-th channel of the feature map.
(2) Global pooling operation, performing global pooling operation on the feature map, and using formula z l ＝
Changing the feature map into a one-dimensional feature descriptor z= [ z ] 1 z 2 …z c ]∈R c Wherein h and w represent the width and height of the feature map, x l ∈R h×w Is the first channel of the feature map, z l Representing the first pass of the pooled signature. R is R x Represents the c-th channel characteristic diagram, R h×w A feature map expressed as width w and height h, z 1 、z 2 、···、z c Respectively showing the pooling characteristic diagram 1 2./v c channels.
(3) Attention feature vector operation, substituting the one-dimensional feature descriptor z into the formula m=σ (W 2 ∝(W 1 z)) to obtain a one-dimensional vector m with re-measured weights, and the one-dimensional vector m= [ m ] of re-measurement 1 m 2 m 3 …m c ]Formula m=σ (W 2 ∝(W 1 z)) of the variables σ and oc represent the ReLU activation function and Sigmoid activation function, respectively, and the variable W 1 And W is 2 Different weights are generated for each channel of the feature map, m being the attention feature vector;
(4) The feature map represents: re-measure the eigenvector to u= [ m ] 1 x 1 m 2 x 2 …m c x c ]∈R h×w×c Will characteristic diagram U 1 ∈R h×w×c Aggregation in channel dimension to generate a two-dimensional feature map
M∈R h×w Two-dimensional feature map mean->
Mask pattern->
Mask map intersection area is +.>
Wherein m is 1 、m 2 、···、m c Respectively expressed as one-dimensional vector weights, U l Representing a profile of 1 attention mechanism activation, c representing a profile with c channels, M (x, y) representing the value of the two-dimensional profile M at position (x, y), +.>
Representation represents mask 1->
A mask map of the type 2 is shown,
(5) Unmanned aerial vehicle part drawing identification: unmanned aerial vehicle part image
I o Representing a view of the foreground scale in the foreground,
mask patterns representing characteristic regions, wherein the mask patterns are also expressed as exclusive OR and are used for judging whether the mask patterns are overlapped with the foreground patterns, if so, the mask patterns are unmanned aerial vehicle parts, and the unmanned aerial vehicle parts are imaged I p And F is equal to O ＝[x 1 x 2 …x c ]∈R h×w×c And comparing the characteristics to finish the identification of the unmanned aerial vehicle parts. />
7. The target fine granularity recognition model of the scale self-adaptive attention mechanism is further optimized, and the aim of the operation is to realize unmanned aerial vehicle model number recognition by utilizing the bilinear fine granularity target recognition model, improve the target fine granularity recognition model, introduce joint probability prediction and improve recognition accuracy.
First, the output of each unmanned aerial vehicle branch in the scale-adaptive attention mechanism target fine-granularity recognition model is expressed as logits y E R N N represents the total number of classes of drones,
next, each branch design dynamic probability parameter λ in the scale-adaptive attention mechanism target fine-granularity recognition model is weighted for uncertainty, λ is used to evaluate the confidence score of the branch output logits y,
then, inputting the output of each branch in the scale self-adaptive attention mechanism target fine granularity recognition model to a softMax function, calculating to obtain the confidence score of each branch, further obtaining the dynamic probability parameter of logits y,
finally, adding the weighted outputs of each branch in the scale self-adaptive attention mechanism target fine granularity recognition model to obtain y concat Select y concat And the category corresponding to the maximum value of the category is used as a model prediction result. Specific:
(1) The characteristic is represented as follows: the output of each branch of the scale-adaptive attention mechanism target fine granularity recognition model represents logits y epsilon R N N represents the total number of unmanned categories.
(2) Classification estimation: will y concat Representing the sum, y, of the outputs of each branch of the unmanned aerial vehicle concat ＝y global +y part ,y global Representing the global output of the unmanned aerial vehicle, y part And representing the local branch output of the unmanned aerial vehicle. Each branch designs a dynamic probability parameter λ to weight the uncertainty, at which time y concat Denoted as y concat ＝λ global ×y global +λ part ×y part Wherein lambda is global 、λ part Respectively represent the output logits y to be estimated global And logits y part Confidence score of (c).
(3) Confidence represents: outputting each branch to the logic y E R N Input to SoftMax function, and calculate confidence score
y j Represents the j-th element of the branch output, N represents the total number of the identified categories, p i Representing a predictive confidence score vector.
(4) Probability parameters: obtaining logits y according to confidence score global And logits y part Dynamic probability parameters, logits y global And logits y part The dynamic probability parameter is lambda global And lambda (lambda) part ，λ global ＝max(p global ),λ part ＝max(p part )。
(5) Operation of unmanned aerial vehicle model identification: lambda is set to global 、λ part Substitution into y concat ＝λ global ×y global +λ part ×y part ，y concat And the corresponding category is used as a model prediction result, and the model or category of the unmanned aerial vehicle is obtained according to the model prediction result.
8. And training an improved yolov5s target detection model, a scale self-adaptive attention mechanism target fine granularity recognition model and a TCTrack target tracking model.
9. And detecting the unmanned aerial vehicle in the video image by using an improved target detection model yolov5 s.
10. And identifying the model number of the unmanned aerial vehicle detected by the target detection model yolov5s by using a scale self-adaptive attention mechanism target fine granularity identification model.
11. And tracking the detected and identified unmanned aerial vehicle by using a TCTrack tracking model, and recording a tracking track and a tracking video image.
According to the method, the unmanned aerial vehicle is detected in real time by utilizing the improved yolov5s target detection model, the model identification is further carried out on the detected unmanned aerial vehicle by adopting the scale self-adaptive attention mechanism target fine granularity identification model, the detected and identified unmanned aerial vehicle is tracked by adopting the TCTrack tracking model, the detection, identification and tracking precision of the unmanned aerial vehicle are effectively improved, and technical support is provided for the unmanned aerial vehicle reverse production industrialization.
In the invention, a TCTrack tracking model can be specifically referred to an article TCTrack: temporal Contexts for Aerial Tracking published in CVPR 2022 (IEEE Conference on Computer Vision and Pattern Recognition 2022) by the combination of university of Shangji, shang Tang technology-Nanyang university combined AI research center S-Lab and the like, and the article provides a twin network frame based on time sequence information, and the frame introduces the time sequence information through two dimensions so as to better realize the balance of speed and performance in the tracking process, thereby meeting challenges brought by factors such as illumination change, target scale change, target shape change, background interference and the like in an air scene. The TCTrack continuously integrates time sequence information through the feature dimension and the similarity map dimension, in the feature extraction process, the time sequence information is efficiently introduced into the feature dimension by adopting the Online TAdaConv, a more efficient time sequence information strategy is used, the feature map is corrected through the continuously accumulated time sequence information, the requirement of instantaneity is met under the condition that acceleration is not used, and the precision similar to that of other SOTA trackers is also obtained. The yolov5s target detection model can be specifically referred to an Ultralytics LLC company which issues a yolov5s open source model in the github, and the yolov5s model is used as one of a yolov5 model series, and has the advantage of higher speed compared with other models of a yolov4 model and a yolov5 system.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (6)
1. The low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method is characterized by comprising the following steps of:
s1: collecting unmanned aerial vehicle samples, constructing a target detection training sample set, a target fine granularity recognition training sample set and a target tracking data set,
s2: optimizing the frame of the target detection model, adding a small target layer to be suitable for the actual condition that the unmanned aerial vehicle presents small characteristics in the low airspace visual image, obtaining an improved target detection model,
training an improved target detection model by adopting the target detection training sample set in the step S1, detecting and positioning the unmanned aerial vehicle in the actual visual image by adopting the trained improved target detection model to acquire an unmanned aerial vehicle global image,
s3: improving the target fine granularity recognition model, introducing a scale self-adaptive attention mechanism and joint probability prediction to improve recognition accuracy, obtaining the scale self-adaptive attention mechanism target fine granularity recognition model,
training a scale self-adaptive attention mechanism target fine granularity recognition model by adopting the target fine granularity recognition training sample set in the step S1, extracting and recognizing unmanned aerial vehicle components by adopting the trained scale self-adaptive attention mechanism target fine granularity recognition model to the unmanned aerial vehicle global image obtained in the step S2, further completing recognition of unmanned aerial vehicle models,
s4: tracking the unmanned aerial vehicle obtained in the step S3, and recording the track and tracking video image of each model unmanned aerial vehicle.
2. The low airspace anti-unmanned aerial vehicle vision detecting, identifying and tracking method according to claim 1, wherein in the step S1, at least five types of unmanned aerial vehicle samples are collected, each type of unmanned aerial vehicle sample image is at least 100, and the shooting angle, gesture and size of each unmanned aerial vehicle sample image are different, so that the unmanned aerial vehicle sample image can be used for constructing a target detection training sample set, a target fine-grained identification training sample set and a target tracking data set.
3. The method for visual detection, identification and tracking of the low airspace anti-unmanned aerial vehicle according to claim 2, wherein in the step S2, the target detection model is a yolov5S target detection model, the frame of the yolov5S target detection model is optimized, specifically, an anchor initialization parameter is modified in a yolov5S target detection model configuration file, a group of initialization values capable of being used for detecting targets with the size of more than 16×16 pixels is added, a small target detection layer is added in a head layer of the yolov5S, and a feature map of the head middle detection layer and a corresponding hierarchical feature map of a backhaul network layer are fused, so that a larger feature map is obtained, and small target detection can be performed.
4. The method for visually detecting, identifying and tracking the low-airspace anti-unmanned aerial vehicle is characterized in that in the step S3, the unmanned aerial vehicle component is extracted and identified, specifically, the characteristic diagrams of different components of the unmanned aerial vehicle are extracted and obtained, global pooling operation is carried out on the characteristic diagrams of the different components of the unmanned aerial vehicle to obtain one-dimensional characteristic descriptors, the one-dimensional characteristic descriptors are processed, different weights are given to each channel of the characteristic diagrams of the different components of the unmanned aerial vehicle, attention characteristic vector distribution is carried out, a new characteristic diagram is obtained, the new characteristic diagram is aggregated in channel dimensions to generate a two-dimensional characteristic diagram, a mask diagram is further obtained, the unmanned aerial vehicle component image is obtained according to the mask diagram and the foreground scale diagram, and the characteristic of the unmanned aerial vehicle component image and the characteristic diagrams of the different components of the unmanned aerial vehicle are compared, so that the unmanned aerial vehicle component extraction and identification are completed.
5. The method for visual inspection, identification and tracking of low airspace anti-unmanned aerial vehicle according to claim 4, wherein in step S3, the identification of the unmanned aerial vehicle model number is specifically:
first, the output of each unmanned aerial vehicle branch in the scale-adaptive attention mechanism target fine-granularity recognition model is expressed as logits y E R N N represents the total number of classes of drones,
each branch in the scale-adaptive attention mechanism target fine-granularity recognition model is then given a dynamic probability parameter, λ, to weight the uncertainty, λ representing the confidence score used to evaluate the branch output logits y,
then, inputting the output of each branch in the scale self-adaptive attention mechanism target fine granularity recognition model to a softMax function, calculating to obtain the confidence score of each branch, further obtaining the dynamic probability parameter of logits y,
finally, adding the output of each branch after uncertain weighting in the scale self-adaptive attention mechanism target fine granularity identification model to obtain y concat Obtaining y according to dynamic probability parameters of logits y concat Category value, select y concat The class corresponding to the maximum value of the class is used as a model prediction result, and is an unmanned model number.
6. The method for visual inspection, recognition and tracking of a low airspace anti-unmanned aerial vehicle according to claim 5, wherein in step S4, the unmanned aerial vehicle obtained in step S3 is tracked by using a TCTrack target tracking model trained by a target tracking data set, and the TCTrack target tracking model tracks an air target by using a time context.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310002983.1A CN116109950A (en) | 2023-01-03 | 2023-01-03 | Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310002983.1A CN116109950A (en) | 2023-01-03 | 2023-01-03 | Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116109950A true CN116109950A (en) | 2023-05-12 |
Family
ID=86266801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310002983.1A Pending CN116109950A (en) | 2023-01-03 | 2023-01-03 | Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116109950A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079196A (en) * | 2023-10-16 | 2023-11-17 | 长沙北斗产业安全技术研究院股份有限公司 | Unmanned aerial vehicle identification method based on deep learning and target motion trail |
| CN117111624A (en) * | 2023-10-23 | 2023-11-24 | 江苏苏启智能科技有限公司 | Anti-unmanned aerial vehicle method and system based on electromagnetic anti-control technology |
| CN117611668A (en) * | 2023-11-27 | 2024-02-27 | 河海大学 | Unmanned aerial vehicle target visual positioning method based on multi-source priori |
| CN118549971A (en) * | 2024-07-29 | 2024-08-27 | 北京领云时代科技有限公司 | Unmanned aerial vehicle bee colony attack intention recognition system and method based on HMM model |
-
2023
- 2023-01-03 CN CN202310002983.1A patent/CN116109950A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079196A (en) * | 2023-10-16 | 2023-11-17 | 长沙北斗产业安全技术研究院股份有限公司 | Unmanned aerial vehicle identification method based on deep learning and target motion trail |
| CN117079196B (en) * | 2023-10-16 | 2023-12-29 | 长沙北斗产业安全技术研究院股份有限公司 | Unmanned aerial vehicle identification method based on deep learning and target motion trail |
| CN117111624A (en) * | 2023-10-23 | 2023-11-24 | 江苏苏启智能科技有限公司 | Anti-unmanned aerial vehicle method and system based on electromagnetic anti-control technology |
| CN117111624B (en) * | 2023-10-23 | 2024-02-02 | 江苏苏启智能科技有限公司 | Anti-unmanned aerial vehicle method and system based on electromagnetic anti-control technology |
| CN117611668A (en) * | 2023-11-27 | 2024-02-27 | 河海大学 | Unmanned aerial vehicle target visual positioning method based on multi-source priori |
| CN118549971A (en) * | 2024-07-29 | 2024-08-27 | 北京领云时代科技有限公司 | Unmanned aerial vehicle bee colony attack intention recognition system and method based on HMM model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zheng et al. | Air-to-air visual detection of micro-uavs: An experimental evaluation of deep learning | |
| CN107862705B (en) | Unmanned aerial vehicle small target detection method based on motion characteristics and deep learning characteristics | |
| CN108510467B (en) | SAR image target identification method based on depth deformable convolution neural network | |
| CN109829398B (en) | Target detection method in video based on three-dimensional convolution network | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN111914664A (en) | Vehicle multi-target detection and track tracking method based on re-identification | |
| CN104200495B (en) | A kind of multi-object tracking method in video monitoring | |
| CN116109950A (en) | Low-airspace anti-unmanned aerial vehicle visual detection, identification and tracking method | |
| CN113807187A (en) | Unmanned aerial vehicle video multi-target tracking method based on attention feature fusion | |
| CN106570490B (en) | A kind of pedestrian's method for real time tracking based on quick clustering | |
| Wang et al. | Bottle detection in the wild using low-altitude unmanned aerial vehicles | |
| CN107273832B (en) | License plate recognition method and system based on integral channel characteristics and convolutional neural network | |
| CN108062574A (en) | A kind of Weakly supervised object detection method based on particular category space constraint | |
| CN112818905B (en) | Finite pixel vehicle target detection method based on attention and spatio-temporal information | |
| CN114627447A (en) | Road vehicle tracking method and system based on attention mechanism and multi-target tracking | |
| CN109859246B (en) | Low-altitude slow unmanned aerial vehicle tracking method combining correlation filtering and visual saliency | |
| CN110334703B (en) | Ship detection and identification method in day and night image | |
| CN112232240A (en) | Road sprinkled object detection and identification method based on optimized intersection-to-parallel ratio function | |
| CN113297982A (en) | Target detection method for improving combination of KCF and DSST in aerial photography | |
| CN115272882A (en) | Discrete building detection method and system based on remote sensing image | |
| CN117036404A (en) | Monocular thermal imaging simultaneous positioning and mapping method and system | |
| Fang et al. | A real-time anti-distractor infrared UAV tracker with channel feature refinement module | |
| Delleji et al. | An upgraded-yolo with object augmentation: Mini-uav detection under low-visibility conditions by improving deep neural networks | |
| CN112347967B (en) | Pedestrian detection method fusing motion information in complex scene | |
| Sazdić-Jotić et al. | Feature extraction for drone classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |