CN116363412A - Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics - Google Patents
Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics Download PDFInfo
- Publication number
- CN116363412A CN116363412A CN202310121385.6A CN202310121385A CN116363412A CN 116363412 A CN116363412 A CN 116363412A CN 202310121385 A CN202310121385 A CN 202310121385A CN 116363412 A CN116363412 A CN 116363412A
- Authority
- CN
- China
- Prior art keywords
- track
- unmanned aerial
- classification model
- aerial vehicle
- classifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013145 classification model Methods 0.000 title claims abstract description 58
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000001133 acceleration Effects 0.000 claims abstract description 48
- 239000012634 fragment Substances 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000010801 machine learning Methods 0.000 claims abstract description 9
- 238000002790 cross-validation Methods 0.000 claims abstract description 6
- 241000271566 Aves Species 0.000 claims description 28
- 238000011156 evaluation Methods 0.000 claims description 17
- 238000012360 testing method Methods 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 11
- 238000002372 labelling Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 5
- 241000272201 Columbiformes Species 0.000 claims description 4
- 210000004556 brain Anatomy 0.000 claims description 4
- WOAFDHWYKSOANX-UHFFFAOYSA-N diisopropyl methylphosphonate Chemical compound CC(C)OP(C)(=O)OC(C)C WOAFDHWYKSOANX-UHFFFAOYSA-N 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 4
- 239000010977 jade Substances 0.000 claims description 4
- 238000012706 support-vector machine Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 238000010183 spectrum analysis Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000003936 working memory Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
- G06V20/47—Detecting features for summarising video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for constructing a classification model of a flying bird and an unmanned aerial vehicle based on centroid movement characteristics, and video data of flying of the flying bird and the unmanned aerial vehicle are shot; preprocessing video data, and then cutting the preprocessed video data into a plurality of tracks with fixed duration to form a video data set; marking a centroid movement track by adopting a computer vision algorithm, and performing wavelet denoising pretreatment; calculating the speed and acceleration value motion characteristics of the track fragments in the centroid motion track, and extracting the maximum fluctuation amplitude of the track fragments in different frequency intervals; respectively establishing a speed classifier, an acceleration classifier and a track fluctuation classifier aiming at the speed, acceleration and track fluctuation characteristics of the flying bird and the unmanned aerial vehicle by adopting machine learning; weighting and integrating the prediction results of the three classifiers, obtaining the optimal weight in a five-fold cross validation mode, and establishing a classification model of the flying bird and the unmanned aerial vehicle; the classification model of the flying bird and the unmanned aerial vehicle provided by the invention has the advantages of high speed and high accuracy, does not need manual judgment, and liberates labor cost.
Description
Technical Field
The invention relates to the technical field of target classification, in particular to a method and a system for constructing a classification model of a bird and an unmanned aerial vehicle based on centroid movement characteristics, and a real-time target classifier is established.
Background
Birds and unmanned aerial vehicles are typically "low-altitude" aircraft, i.e., aircraft having a flight altitude of less than 1000 meters, a flight speed of less than 200 km/h, and a radar reflection area of less than 2 square meters. They are a major factor in threatening the security of low-altitude airspace. The flying bird collides with the aircraft to damage the structure of the aircraft, and the aircraft is seriously destroyed. Bird strikes are a major symptom factor in aircraft operation. In addition, with the development of unmanned aerial vehicle technology, unmanned aerial vehicles gradually become a novel threat. Because of the characteristics of small volume, portability, easy operation and the like, the device is easy to be used for illegal activities such as illegal ground mapping and interference with normal flight of civil aviation. For different threats of birds and unmanned aerial vehicles, different measures need to be taken, and the premise of taking proper measures is to correctly identify and classify the flying object. At a greater distance, the bird and the unmanned aerial vehicle are small and have similar appearance characteristics, so that it is difficult to accurately distinguish the two.
Starting in 2017, a safe coastline (SafeShore) project sponsored by the european union "horizon 2020" program started a bird, unmanned aerial vehicle, to detect a challenge race, which was held once every two years, three times so far. In this game, most algorithms are mainly based on static image information, i.e. based on the difference in the appearance characteristics of birds and unmanned aerial vehicles, to build a convolutional neural network model. However, these algorithms are less effective at identifying small objects at long distances, and when the object image size is less than 32 pixels, the false positive rate and the false negative rate are both high.
Some researchers have proposed categorization based on movement information of flying objects. Compared with image information, the motion information has the advantages that: the robustness to the distance is high, and the influence of object deformation, environment brightness and background interference objects is avoided. Srsigarom and the like calculate 5 motion characteristics of each track, including a speed average value, an acceleration average value, a turning angle, periodicity and a curvature radius, and dimension reduction is performed by using a principal component analysis method, a support vector machine is established according to the first two main characteristics, and the accuracy is more than 80%, but the algorithm has the defects that: the method is established based on the statistical index of the motion information, and the statistics can only reflect partial information, is easily influenced by the track segment length and cannot be used for real-time classification tasks.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method and a system for constructing a classification model of a flying bird and an unmanned aerial vehicle based on centroid movement characteristics, which are based on the centroid movement characteristic difference of the flying bird and the unmanned aerial vehicle and refer to strategies adopted when a human visual system classifies moving targets to establish a real-time flying target classifier.
The invention adopts the following technical scheme:
on one hand, the invention provides a method for constructing a classification model of a bird and an unmanned aerial vehicle based on centroid motion characteristics, which is characterized by comprising the following steps:
step 2, preprocessing video data, and cutting the video data into a plurality of tracks with fixed duration to form a video data set;
marking the centroid position of each frame of flying target in the video data set by adopting a computer vision algorithm to form a centroid movement track, and carrying out wavelet denoising pretreatment;
step 4, dividing each track into a plurality of track segments with fixed duration without overlapping, calculating the speed, acceleration and track fluctuation motion characteristics of the track segments to obtain all speed values and acceleration values, and extracting the maximum fluctuation amplitude of the track segments in different frequency intervals;
step 5, respectively establishing a speed classifier, an acceleration classifier and a track fluctuation classifier aiming at the speed, the acceleration and the track fluctuation characteristics of the flying bird and the unmanned aerial vehicle by adopting machine learning;
and 6, weighting and integrating the prediction results of the three classifiers obtained in the step 5 by using a weighted integration method, obtaining optimal weights by using a cross-validation method, establishing a motion trail classification model, and identifying the video data sets of the birds and/or the unmanned aerial vehicles in the test set.
Further, the method further comprises a step 7 of calculating the track fragments in the track by the motion track classification model to obtain the class probability of the track fragments; and (3) performing stability evaluation on the category probability of the track fragments by adopting a stability evaluation module based on a working memory mechanism and a decision mechanism of the human brain, and outputting a classification result.
In step 7, the motion characteristics in the track segments are input into the motion track classification model constructed in step 6 for calculation, so as to obtain the class probability of the track segments; the motion trail classification model continuously inputs category probability information to the stability evaluation module; and outputting a classification result when the input same class probability has at least 7 class probabilities greater than 0.8 in 10 consecutive classes and the class judgment of the speed classifier, the acceleration classifier and the track fluctuation classifier is consistent.
Preferably, the constructed motion trail classification model performs class probability calculation on trail fragments with the duration of 200ms and containing 24 frames.
Preferably, in the step 2, when the original video data is preprocessed, motion characteristics of the bird and the unmanned aerial vehicle in the original video data in taking off, landing and hovering are removed, and a track with each section being at least 2 seconds is formed.
Preferably, in the step 3, marking the centroid position of each frame of flying object in the acquired pigeon video data set by adopting an ECO object tracking algorithm, and marking the centroid position of each frame of flying object in the acquired yellow jade bird, pearl bird and unmanned aerial vehicle video data set by adopting a DIMP object tracking algorithm.
Preferably, in the step 4, each track is divided into a plurality of track segments with a duration of 200ms without overlapping, and the method for calculating the track fluctuation motion characteristics in the track segments is as follows: performing moving average smoothing on the centroid movement track in the step 3 to obtain a smooth track; subtracting the horizontal coordinate and the vertical coordinate of the smooth track from the horizontal coordinate and the vertical coordinate of the centroid movement track respectively to obtain track fluctuation in the horizontal direction and the vertical direction; and carrying out frequency spectrum analysis on the track fluctuation by adopting fast Fourier transform to respectively obtain the maximum fluctuation amplitude in the frequency range of 0-10Hz,10-20Hz and 20-30Hz.
Preferably, in the step 5, a naive bayes algorithm is adopted to respectively establish a velocity classifier and an acceleration classifier, and a support vector machine algorithm is adopted to establish a track fluctuation classifier.
In another aspect, the present invention also provides a classification model system for a bird and an unmanned aerial vehicle based on centroid motion characteristics, the system comprising: the camera is used for shooting video data of flying birds and unmanned aerial vehicles;
the preprocessing module is used for preprocessing video data and cutting the video data into a plurality of tracks with fixed duration to form a video data set;
the visual algorithm labeling module is used for labeling the centroid position of each frame of flight target in the preprocessed bird and unmanned plane tracks to form a centroid movement track of the flight target;
the wavelet denoising module is used for carrying out wavelet denoising pretreatment on the centroid movement track and removing high-frequency noise and annotation errors in the centroid movement track;
the data processing module divides each track into a plurality of track segments with fixed duration without overlapping, calculates the speed, acceleration and track fluctuation motion characteristics of the track segments in each track segment, obtains all speed values and acceleration values, and extracts the maximum fluctuation amplitude of the track segments in different frequency intervals;
the classifier construction module is used for extracting the speed value, the acceleration value and the track fluctuation amplitude in the data processing module and respectively constructing a speed classifier, an acceleration classifier and a track fluctuation classifier by adopting machine learning;
and the classification model construction module is used for weighting and integrating the prediction results of the obtained speed classifier, the acceleration classifier and the track fluctuation classifier by using a weighted integration method, and obtaining the optimal weight by using a cross verification method to obtain the motion track classification model.
Further, the system also comprises a stability evaluation module, wherein the motion trail classification model carries out category probability calculation on trail fragments in each section of trail, and a classification result is output after probability evaluation is carried out by the stability evaluation module.
The technical scheme of the invention has the following advantages:
A. the classification model of the flying bird and the unmanned aerial vehicle provided by the invention has the advantages of high speed and high accuracy. According to the visual system, the motion trail classification model is established based on the short-time trail segments, so that the automatic and real-time judgment of the category of the moving target can be realized, the manual judgment is not needed, and the labor cost is liberated; the established motion trail classification model has high accuracy rate of classifying the motion trail of the flying bird and the unmanned aerial vehicle, which reaches more than 90 percent, and the frame is simple and can be migrated to other tasks based on the motion characteristic classification targets.
B. The invention requires a smaller sample size for training. The image characteristics of the existing flying targets are influenced by factors such as target types, observation visual angles, deformation, shielding and the like, the variety is various, and a large number of samples are required to train for building the classifier according to the image characteristics. The movement characteristics are limited by flight dynamics, and the movement characteristics of the flying birds and the unmanned aerial vehicle have intrinsic differences, and the differences are not influenced by the types of the flying birds and the types of the unmanned aerial vehicle, so that the invention can obtain stable results by adopting small sample training and is applicable to the identification of different types of flying birds and unmanned aerial vehicles.
Drawings
In order to more clearly illustrate the embodiments of the present invention, the drawings that are required for the embodiments will be briefly described, and it will be apparent that the drawings in the following description are some embodiments of the present invention and that other drawings may be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart I of a motion trail classification model construction method provided by the invention;
FIG. 2 is a flowchart II of a motion trail classification model construction method provided by the invention;
FIG. 3 is a flow chart of the motion trail classification model application provided by the invention;
FIG. 4 is an exemplary diagram of a flight path provided by the present invention;
FIG. 5 is a flow chart of a bird and unmanned classifier provided by the present invention;
FIG. 6 is a diagram of a motion trail classification model construction system provided by the invention;
FIG. 7 is a sample bird test track classification provided by the present invention-pearl bird;
fig. 8 is a schematic diagram of a test track classification example-teleo of an unmanned aerial vehicle provided by the invention.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, 3 and 5, the invention provides a method for constructing a classification model of a bird and an unmanned aerial vehicle based on centroid motion characteristics, which comprises the following steps:
and S01, shooting the flying bird by using a camera, and controlling the unmanned aerial vehicle to simulate the flying bird to fly along the movement track, so as to obtain video data of the flying bird and the flying unmanned aerial vehicle respectively.
In order to improve the application range of the model, 3 types of flying birds and unmanned aerial vehicles are respectively selected, wherein the flying birds are pearl birds, yellow jade birds and pigeons in sequence from small to large; the unmanned aerial vehicle is, in order from small to large, terlo (telelo), danshengzhu Pro (Mavic Pro) and danshengzhu fairy 4 (Phantom 4). And 3 types of flying birds are collected indoors, the video duration is 2 seconds at the shortest, then a windless and clear day is selected, and in an outdoor open parking lot, the flight hands with rich experience are required to control the 3 types of unmanned aerial vehicles to simulate the movement track flight of the flying birds, so that the flying video data of the unmanned aerial vehicles are obtained.
S02, preprocessing the photographed original video data, cutting the original video data into a plurality of tracks with fixed duration, and forming a video data set by all the tracks.
The preprocessing includes editing all shot video data into 2 seconds by video software, and removing typical motion characteristics of the flying bird and the unmanned aerial vehicle, such as take-off, landing, hovering and the like, for example, each type of flying object comprises 30 segments of flying video data of 2 seconds per segment to form a video data set, and the duration and the number of the segments of the flying video data set are not particularly limited.
And S03, marking the centroid position of each frame of flying target in the video data set by adopting a computer vision algorithm, forming a centroid movement track, and performing wavelet denoising pretreatment.
Marking the centroid position of each frame of flying target in the video data set by adopting a computer vision algorithm (comprising ECO and DIMP), wherein the specific method comprises the following steps: a bounding box is drawn around the flying object with the center of the bounding box defined as the centroid, as shown in fig. 4. The position of the centroid target in each frame in the video forms the motion trail of the centroid. And after all the labels are finished, manually detecting the accuracy of the video labels. The ECO target tracking algorithm is high in accuracy when the pigeons are marked, and the DIMP target tracking algorithm is high in accuracy when other 5 types of flying targets (yellow jade birds, pearl birds, tello, mavic Pro and Phantom 4) are marked.
And then carrying out wavelet denoising pretreatment on the centroid movement track formed after labeling, and removing high-frequency noise and possible labeling errors in the track.
The unmanned aerial vehicle used in the invention is a four-rotor unmanned aerial vehicle, high-frequency fluctuation in the track can be caused by high-frequency rotation of the spiral wings, and abnormal fluctuation in the track can be caused by jitter (random noise) and labeling errors. In order to remove the noise and obtain a relatively accurate centroid position, the original track data needs to be preprocessed, and the original signal is subjected to noise reduction processing by adopting a wavelet denoising method. The wavelet denoising method can remove high-frequency noise and retain signal characteristics.
The invention divides the video dataset into a training set and a test set in a ratio of 4:1. The 2 second trajectory in the training set was split into short-time-interval (200 ms) trajectory segments without overlapping. And calculating 3 motion characteristics of the track segments, including speed, acceleration and track fluctuation motion characteristics, obtaining all speed values and acceleration values, and extracting the maximum fluctuation amplitude of each track segment in different frequency intervals.
Track fluctuation here refers to up-and-down fluctuation of the original centroid motion track compared with the smooth track. The smooth track is a track obtained by performing moving average smoothing on the original centroid movement track, and the smooth windows are 15. The calculation method of the track fluctuation respectively subtracts the horizontal and vertical coordinates of the smooth track from the horizontal and vertical coordinates of the original centroid movement track to obtain the fluctuation in the horizontal and vertical directions; then, the trace fluctuation is subjected to spectrum analysis by fast Fourier transformation, and the maximum fluctuation amplitude in 3 frequency intervals including 0-10Hz,10-20Hz and 20-30Hz is taken.
And S05, respectively establishing a speed classifier, an acceleration classifier and a track fluctuation classifier aiming at the speed, the acceleration and the track fluctuation characteristics of the flying bird and the unmanned aerial vehicle by adopting machine learning. The method is characterized in that a naive Bayesian algorithm in machine learning is preferably adopted, a speed classifier and an acceleration classifier are respectively established according to all the speeds and accelerations obtained in the step S04 and the corresponding categories of the speeds and accelerations, a support vector machine algorithm in machine learning is adopted, and a track fluctuation classifier is established according to the track fluctuation amplitude extracted in the step S04.
S06, calculating the class probability of each speed value in the track segment according to the three classifiers obtained in the step S05, wherein the average value of the class probabilities is the speed class probability of the track segment for any 200ms track segment; the acceleration classifier calculates the class probability of each acceleration value in the track segment, and the average value of the class probabilities is the acceleration class probability; the track fluctuation classifier calculates fluctuation class probability according to the amplitude characteristics. And (3) weighting and integrating the prediction results of the three classifiers by using a weighting and integrating method, obtaining optimal weight by using a cross-validation method, establishing a motion trail classification model, namely establishing a final target classifier, and identifying the video data set of the flyer and/or the unmanned aerial vehicle in the test set.
The invention preferably adopts a five-fold cross validation method to calculate the optimal weight, and the specific method is as follows: the training set was equally divided into 5 subsets, 4 of which were used for training in turn, and 1 subset was used for verification. This process is repeated 5 times, each time a correct rate is calculated, the average value of the 5 correct rates is used as an estimate of the correct rate corresponding to the weight, and the optimal weight is the weight corresponding to the highest correct rate.
In order to apply the established motion trail classification model to real-time classification tasks, as shown in fig. 2, the invention also simulates the working mechanism of a human in step S07, and constructs a machine observer which is used for classifying the trail in the test set. The machine observer as in fig. 5 comprises two modules, the first being a constructed motion trajectory classification model (i.e. a bird-drone classifier) that simulates a feature detection network in the brain, constantly extracting the motion features of the trajectory segments and assigning them different weights. The motion trail classification model calculates the class probability of trail fragments (200 ms, 24 frames in total) with fixed duration, and tests in a real-time trail data input mode when the motion trail classification model is applied in a test set. When the input track length is less than 200 milliseconds, the input track is not classified; when the input track length reaches 200 milliseconds, judging the track fragments of the 200 milliseconds, and outputting class probability; every time one frame of data is newly input, the latest 200 millisecond track segment is acquired, and a new class probability is output. In an actual classification task, it is not enough to calculate the class probabilities alone, but work memory and decision-making mechanism assistance is also required. Therefore, the invention simulates the working principle of the human brain, and also constructs a second module, namely a stability evaluation module. The motion trail classification model continuously inputs updated probability information to the stability evaluation module, when the probability of the same category is input, at least 7 of the probabilities are larger than 0.8 in 10 continuous categories, and the category judgment of the three classifiers is consistent at the moment, the three classifiers are regarded as stable and reliable judgment, and a final classification result is output; if the classification judgment of the three classifiers is inconsistent, the classification result is not output. If the motion trail classification model does not make a judgment at the trail end point, namely the stability is not achieved, the accuracy is marked as 0.5, and the response time is marked as the average response time of other judged trails.
Inputting track data in the test set into the motion track classification model established in the step S06, and obtaining the prediction category of each track; and comparing the consistency of the predicted category and the real category to obtain the classification accuracy of the motion trail classification model.
On the other hand, as shown in fig. 6, the invention also provides a classification model system of the flying bird and the unmanned aerial vehicle based on the centroid motion characteristics, which comprises a camera, a preprocessing module, a visual algorithm labeling module, a wavelet denoising module, a data processing module, a classifier construction module and a classification model construction module which are arranged in a computer. The video camera is used for shooting video data of flying birds and unmanned aerial vehicles; the preprocessing module is used for preprocessing video data and cutting the video data into a plurality of tracks with fixed duration to form a video data set containing a plurality of continuous track fragments; the visual algorithm labeling module is used for labeling the centroid position of each frame of flying target in the preprocessed flying bird and unmanned plane track fragments to form a centroid movement track of the flying target; the wavelet denoising module is used for performing wavelet denoising pretreatment on the centroid movement track and removing high-frequency noise and annotation errors in the centroid movement track; the data processing module divides each track into a plurality of track segments with fixed duration without overlapping, calculates the speed, acceleration and track fluctuation motion characteristics of the track segments in each track segment, obtains all speed values and acceleration values, and extracts the maximum fluctuation amplitude of the track segments in different frequency intervals; the classifier construction module extracts the speed value, the acceleration value and the track fluctuation amplitude in the data processing module, and adopts machine learning to respectively construct a speed classifier, an acceleration classifier and a track fluctuation classifier; the classification model construction module utilizes a weighted integration method to integrate the prediction results of the obtained speed classifier, the acceleration classifier and the track fluctuation classifier in a weighted way, and adopts a cross validation method to obtain the optimal weight so as to obtain the motion track classification model.
In order to improve classification accuracy, a stability evaluation module is further arranged in the system, the motion trail classification model carries out class probability calculation on trail fragments in each section of trail, and after probability evaluation is carried out by the stability evaluation module, classification results are output.
As shown in fig. 7 and 8, the classification model constructed by the present invention is schematically shown. The speed and acceleration probability distribution differences and the track fluctuation differences of the flying birds and the unmanned aerial vehicle are distinguished. As can be seen in the figure: compared with unmanned aerial vehicle, speed and acceleration distribution range of flying birds are bigger, and track fluctuation is also bigger.
The left side of the first row in fig. 7 and 8 is the original centroid movement track, and the right side of the first row is the classifier probability judgment real-time output result, wherein the white vertical line represents the output moment of the result (380.79 ms in the case of average reaction), and the classification result is output in a reaction frame. The second row is the extracted trajectory features, speed, acceleration and trajectory fluctuations in the horizontal and vertical directions, respectively. The speed and acceleration are based on the track data in the training set, and the obtained probability distribution templates of the speed and the acceleration of the flying bird and the unmanned aerial vehicle are obtained. The speed and the acceleration of the flying bird are distributed in a probability template of the flying bird, and the track of the flying bird fluctuates greatly, as shown in fig. 7; the speed and acceleration of the unmanned aerial vehicle are distributed in a probability template of the unmanned aerial vehicle, and the track fluctuation of the unmanned aerial vehicle is small, as shown in fig. 8.
After testing 30 sections of tracks in the test set, the accuracy rate reaches 100%.
The invention has the advantages that the sample quantity required by training is less, only part of the flight tracks are required to be acquired, calculation is not required to be carried out on all the flight tracks, and the invention can obtain stable output results by adopting small sample training and can be suitable for accurately identifying different types of flying birds and unmanned aerial vehicles.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While obvious variations or modifications are contemplated as falling within the scope of the present invention.
Claims (10)
1. A method for constructing a classification model of a bird and an unmanned aerial vehicle based on centroid movement features is characterized by comprising the following steps:
step 1, shooting flying birds, and simultaneously controlling an unmanned aerial vehicle to simulate the flying birds in a motion track to fly so as to obtain video data of flying birds and flying unmanned aerial vehicles respectively;
step 2, preprocessing video data, and cutting the video data into a plurality of tracks with fixed duration to form a video data set;
marking the centroid position of each frame of flying target in the video data set by adopting a computer vision algorithm to form a centroid movement track, and carrying out wavelet denoising pretreatment;
step 4, dividing each track into a plurality of track segments with fixed duration without overlapping, calculating the speed, acceleration and track fluctuation motion characteristics of the track segments to obtain all speed values and acceleration values, and extracting the maximum fluctuation amplitude of the track segments in different frequency intervals;
step 5, respectively establishing a speed classifier, an acceleration classifier and a track fluctuation classifier aiming at the speed, the acceleration and the track fluctuation characteristics of the flying bird and the unmanned aerial vehicle by adopting machine learning;
and 6, weighting and integrating the prediction results of the three classifiers obtained in the step 5 by using a weighted integration method, obtaining optimal weights by using a cross-validation method, establishing a motion trail classification model, and identifying the video data sets of the birds and/or the unmanned aerial vehicles in the test set.
2. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid motion characteristics according to claim 1, wherein the method further comprises the step 7 of calculating the track fragments in the track by the motion track classification model to obtain the class probabilities of the track fragments; and (3) performing stability evaluation on the category probability of the track fragments by adopting a stability evaluation module based on a working memory mechanism and a decision mechanism of the human brain, and outputting a classification result.
3. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid motion characteristics according to claim 2, wherein in the step 7, the motion characteristics in the track segments are input into the motion track classification model constructed in the step 6 for calculation, so as to obtain the class probability of the track segments; the motion trail classification model continuously inputs category probability information to the stability evaluation module; and outputting a classification result when the input same class probability has at least 7 class probabilities greater than 0.8 in 10 consecutive classes and the class judgment of the speed classifier, the acceleration classifier and the track fluctuation classifier is consistent.
4. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid motion features according to claim 3, wherein the constructed classification model of the motion trail performs class probability calculation on the trail segment with the duration of 200ms and the 24 frames.
5. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid movement characteristics according to claim 1, wherein in the step 2, when the original video data is preprocessed, the movement characteristics of the flying bird and the unmanned aerial vehicle in the original video data during take-off, landing and hovering are removed, and a track of at least 2 seconds is formed.
6. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid movement characteristics according to claim 1, wherein in the step 3, the centroid position of each frame of flying object in the acquired pigeon video data set is marked by adopting an ECO object tracking algorithm, and the centroid position of each frame of flying object in the acquired yellow jade bird, pearl bird and unmanned aerial vehicle video data set is marked by adopting a DIMP object tracking algorithm.
7. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid motion feature according to claim 1, wherein in the step 4, each track is divided into a plurality of track segments with the duration of 200ms without overlapping, and the method for calculating the track fluctuation motion feature in the track segments is as follows: performing moving average smoothing on the centroid movement track in the step 3 to obtain a smooth track; subtracting the horizontal coordinate and the vertical coordinate of the smooth track from the horizontal coordinate and the vertical coordinate of the centroid movement track respectively to obtain track fluctuation in the horizontal direction and the vertical direction; and carrying out frequency spectrum analysis on the track fluctuation by adopting fast Fourier transform to respectively obtain the maximum fluctuation amplitude in the frequency range of 0-10Hz,10-20Hz and 20-30Hz.
8. The method for constructing the classification model of the flying bird and the unmanned aerial vehicle based on the centroid motion features according to claim 1, wherein in the step 5, a naive Bayesian algorithm is adopted to respectively establish a speed classifier and an acceleration classifier, and a support vector machine algorithm is adopted to establish a track fluctuation classifier.
9. A classification model system for a bird and unmanned aerial vehicle based on centroid motion features, the system comprising: the camera is used for shooting video data of flying birds and unmanned aerial vehicles;
the preprocessing module is used for preprocessing video data and cutting the video data into a plurality of tracks with fixed duration to form a video data set;
the visual algorithm labeling module is used for labeling the centroid position of each frame of flight target in the preprocessed bird and unmanned plane tracks to form a centroid movement track of the flight target;
the wavelet denoising module is used for carrying out wavelet denoising pretreatment on the centroid movement track and removing high-frequency noise and annotation errors in the centroid movement track;
the data processing module divides each track into a plurality of track segments with fixed duration without overlapping, calculates the speed, acceleration and track fluctuation motion characteristics of the track segments in each track segment, obtains all speed values and acceleration values, and extracts the maximum fluctuation amplitude of the track segments in different frequency intervals;
the classifier construction module is used for extracting the speed value, the acceleration value and the track fluctuation amplitude in the data processing module and respectively constructing a speed classifier, an acceleration classifier and a track fluctuation classifier by adopting machine learning;
and the classification model construction module is used for weighting and integrating the prediction results of the obtained speed classifier, the acceleration classifier and the track fluctuation classifier by using a weighted integration method, and obtaining the optimal weight by using a cross verification method to obtain the motion track classification model.
10. The classification model system of the flying bird and the unmanned aerial vehicle based on the centroid motion features according to claim 9, wherein the system further comprises a stability evaluation module, the motion trail classification model performs class probability calculation on trail fragments in each trail, and the stability evaluation module outputs classification results after probability evaluation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310121385.6A CN116363412A (en) | 2023-02-03 | 2023-02-03 | Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310121385.6A CN116363412A (en) | 2023-02-03 | 2023-02-03 | Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116363412A true CN116363412A (en) | 2023-06-30 |
Family
ID=86912413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310121385.6A Pending CN116363412A (en) | 2023-02-03 | 2023-02-03 | Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116363412A (en) |
-
2023
- 2023-02-03 CN CN202310121385.6A patent/CN116363412A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108037770B (en) | Unmanned aerial vehicle power transmission line inspection system and method based on artificial intelligence | |
| CN113269073B (en) | Ship multi-target tracking method based on YOLO V5 algorithm | |
| EP3696566A1 (en) | System and method for identification of an airborne object | |
| CN110018453B (en) | Intelligent model identification method based on aircraft track characteristics | |
| CN105892471A (en) | Automatic automobile driving method and device | |
| CN104463249B (en) | A kind of remote sensing images airfield detection method based on Weakly supervised learning framework | |
| CN112434643A (en) | Classification and identification method for low-slow small targets | |
| CN107273816B (en) | Traffic speed limit label detection recognition methods based on vehicle-mounted forward sight monocular camera | |
| CN113128342B (en) | Flight path data preprocessing method and aerial target identification method | |
| CN106570490B (en) | A kind of pedestrian's method for real time tracking based on quick clustering | |
| CN103559508B (en) | A kind of based on continuous Adaboost video vehicle detection method | |
| CN108830246B (en) | Multi-dimensional motion feature visual extraction method for pedestrians in traffic environment | |
| CN115064009B (en) | Terminal area unmanned aerial vehicle and man-machine conflict risk level classification method | |
| CN110674674A (en) | Rotary target detection method based on YOLO V3 | |
| CN113362374A (en) | High-altitude parabolic detection method and system based on target tracking network | |
| CN114139373B (en) | Multi-sensor automatic collaborative management method for unmanned aerial vehicle reverse vehicle | |
| CN112950634A (en) | Method, equipment and system for identifying damage of wind turbine blade based on unmanned aerial vehicle routing inspection | |
| CN113807314A (en) | Millimeter wave radar video fusion method based on micro-Doppler effect | |
| Sim et al. | Effective ground-truthing of supervised machine learning for drone classification | |
| CN115657002A (en) | Vehicle motion state estimation method based on traffic millimeter wave radar | |
| US10935938B1 (en) | Learning from operator data for practical autonomy | |
| CN116994077A (en) | Regression prediction method for flight attitude under action of complex wind field | |
| CN116363412A (en) | Method and system for constructing classification model of flying bird and unmanned aerial vehicle based on centroid movement characteristics | |
| CN115311580A (en) | Unmanned aerial vehicle threat determination method based on image recognition and related equipment | |
| CN115984319A (en) | Unmanned aerial vehicle target tracking method based on improved YoLov7 and Deepsort |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |