CN114035604A - Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method - Google Patents
Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method Download PDFInfo
- Publication number
- CN114035604A CN114035604A CN202111135520.XA CN202111135520A CN114035604A CN 114035604 A CN114035604 A CN 114035604A CN 202111135520 A CN202111135520 A CN 202111135520A CN 114035604 A CN114035604 A CN 114035604A
- Authority
- CN
- China
- Prior art keywords
- abnormal target
- abnormal
- ant
- unmanned aerial
- aerial vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 261
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 238000012544 monitoring process Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000012795 verification Methods 0.000 claims abstract description 15
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 10
- 239000003016 pheromone Substances 0.000 claims description 40
- 238000013527 convolutional neural network Methods 0.000 claims description 21
- 241000257303 Hymenoptera Species 0.000 claims description 16
- 238000000034 method Methods 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 9
- 239000000126 substance Substances 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000005295 random walk Methods 0.000 claims description 5
- 238000007689 inspection Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000037361 pathway Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 241000411851 herbal medicine Species 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 230000006378 damage Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an abnormal target detection method for video monitoring and unmanned aerial vehicle air-ground linkage, which comprises the following steps: step S1: extracting characteristics of historical abnormal target video data to be used as input parameters, and importing the input parameters into a constructed model for training; after training is finished, new abnormal target data are imported into the model to obtain a judgment result; meanwhile, calculating a weight value of the abnormal target according to the type and the confidence F of the abnormal target; step S2: planning an optimal path for the unmanned aerial vehicle to complete the cruise abnormal target point task according to the ant colony algorithm; step S3: combining the abnormal target detection result of video monitoring and the abnormal target detection result of the unmanned aerial vehicle, checking whether the abnormal type and the detection result of the abnormal target at the same position are consistent, and if the mutual check results are consistent, returning the abnormal target information and storing the abnormal target information to the server background; and if the verification results are inconsistent, the relevant responsible person performs manual verification, and the verification results are returned and stored in the server background.
Description
Technical Field
The invention relates to the field of abnormal target detection, in particular to an abnormal target detection method based on video monitoring and unmanned aerial vehicle air-ground linkage.
Background
In recent years, with the rapid development of science and technology and the requirement and concern of people on safety, monitoring devices are more and more widely applied. Detection of abnormal behavior in video is an important problem in the field of computer vision, and aims to give a video or process the video in real time in an online system and find abnormal targets in the video in time.
In the land resource supervision or ocean resource monitoring, a plurality of monitoring cameras are usually set up for monitoring or an unmanned aerial vehicle is set for regular cruise monitoring due to a wide monitoring field. Although the monitoring camera can detect monitoring points all day long, a visual blind area exists; although the problem of monitoring video monitoring blind areas can be solved in unmanned aerial vehicle cruise monitoring, the unmanned aerial vehicle has limited cruising ability and cannot monitor cruising of monitoring points all day long, abnormal targets of the monitoring points are easy to miss detection, and therefore irreparable loss is caused. In order to overcome the defects of a single detection method, an abnormal target detection method for linking video monitoring and an unmanned aerial vehicle in the air-ground is needed so as to improve the accuracy and stability of abnormal target detection.
In recent years, some foreign object identification detection methods combining surveillance video and unmanned aerial vehicles have been proposed, for example:
the invention patent with publication number CN111814721A discloses an airport runway foreign matter detection and classification method based on unmanned aerial vehicle high-low altitude combined scanning, which trains a Yolo neural network model, obtains a fine detection area by adopting a method of combining two-wheel high-low altitude thickness detection, photographs the fine detection area, inputs pictures into the trained Yolo neural network model to detect and classify foreign matters, and accurately detects and classifies the foreign matters. However, in the method, a Yolo neural network model is trained, a method combining two rounds of high-low altitude thickness detection is adopted to obtain a fine detection area, the fine detection area is photographed, pictures are input into the Yolo neural network model which is trained to detect and classify foreign matters, accurate detection and classification of the foreign matters are achieved, the flying height of the unmanned aerial vehicle during fine detection is set in a self-adaptive mode according to the density of the distribution of points to be detected, optimal planning of the path of the aircraft based on the positions of the foreign matters cannot be achieved, mutual verification between high-altitude rapid coarse detection and low-altitude fine detection cannot be achieved, and due to the fact that high-low altitude switching flight is needed, long-time flight of the unmanned aerial vehicle cannot be met, and the requirement for reducing loss of the unmanned aerial vehicle is met.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide an abnormal target detection method for video monitoring and unmanned aerial vehicle air-ground linkage, which trains an abnormal target detection model through a convolutional neural network, realizes automatic identification and detection of an abnormal target, and reduces the conditions of misjudgment, missing report and the like of abnormal target detection; and then, the unmanned aerial vehicle cruising dynamic route planning is carried out on the abnormal target points through the route planning based on the ant colony algorithm, so that the loss of the unmanned aerial vehicle is reduced. And finally, mutually verifying the fused unmanned aerial vehicle abnormal target detection result and the monitored abnormal target detection result so as to improve the accuracy and stability of detection.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
an abnormal target detection method for video monitoring and unmanned aerial vehicle air-ground linkage comprises the following steps:
step S1: acquiring monitoring video data through video monitoring, wherein the monitoring video data comprises abnormal target video data, the video monitoring can detect abnormal targets in real time, feature extraction is carried out on historical abnormal target video data in the video data to be used as input parameters, and the input parameters are imported into a constructed model for training; after training is finished, new abnormal target data are imported into the model to obtain a judgment result, so that prediction inspection is carried out; meanwhile, calculating a weight value of the abnormal target according to the obtained type and confidence F of the abnormal target, and sorting and dividing the abnormal target from high to low according to the weight value;
step S2: taking the weight value of each abnormal target obtained in the step S1 as an initial pheromone concentration, and planning an optimal path for the unmanned aerial vehicle to complete the task of cruising at the abnormal target point according to the ant colony algorithm based on the initial pheromone concentration, so that the unmanned aerial vehicle can complete the detection of the abnormal target on the optimal path;
step S3: combining the video monitoring abnormal target detection result obtained in the step S1 and the unmanned aerial vehicle abnormal target detection result obtained in the step S2, checking whether the abnormal type and the detection result of the abnormal target at the same position are consistent, and if the mutual checking results are consistent, returning abnormal target information and storing the abnormal target information to a server background; and if the verification results are inconsistent, the relevant responsible person performs manual verification, and the verification results are returned and stored in the server background.
Preferably, the model in step S1 is a convolutional neural network model.
Preferably, the step S1 specifically includes the following steps:
step S1.1: image preprocessing, namely acquiring a monitoring video data set, acquiring an image sequence from the monitoring video data set, and performing normalization processing on the image sequence to be detected;
step S1.2: extracting the characteristics of the historical abnormal target video data to obtain the characteristic attributes of the historical abnormal target, and dividing the processed historical abnormal target video data into a training set and a test set;
step S1.3: building a convolutional neural network model, training the convolutional neural network model based on the data training set related to the abnormal target features processed in the step S1.2, verifying the convolutional neural network model by adopting a test set to obtain the trained convolutional neural network model, taking the trained convolutional neural network model as an abnormal target detection model, and outputting an abnormal target type and an abnormal target confidence F through the abnormal target detection model;
step S1.4: defining n abnormal measurement indexes for measuring importance degree of abnormal target, and establishing abnormal measurement matrix A for m abnormal targets detected in video monitoringm×nWherein A isijThe representation is the value of the ith abnormal target on the jth abnormal measurement index;
step S1.5: the formula for calculating the weight value of each abnormal target is as follows:
wherein, FiIs the confidence corresponding to the ith abnormal target in the step S1.3, AijThe value of the ith abnormal target on the jth abnormal measurement index in the step S1.4 is obtained;
step S1.6: and (4) carrying out normalization processing on the weight values of the abnormal targets calculated in the step (S1.5).
Preferably, the feature extraction process performed on the historical abnormal target video data in step S1.2 is as follows:
acquiring an RGBD image sequence in a historical abnormal target video data set, and performing background removal processing on color information according to depth information to acquire a target characteristic zone bit corresponding to a target; and performing feature extraction on pixel points corresponding to the target feature location in the color information to obtain historical abnormal target feature attributes.
Preferably, in the step S1.2, the processed historical abnormal target video data is divided into a training set and a testing set according to a ratio of 7: 3.
Preferably, the abnormal metric index in step S1.4 at least includes a degree of damage index, an emergency index, and an influence range index.
Preferably, the step S2 specifically includes the following steps:
step S2.1: initializing pheromones and setting parameters, and obtaining weight values Y of various abnormal targets subjected to normalization processing according to step S1.6iAs the initial pheromone concentration, the abnormal target weight value YiNamely the weighted score value of the abnormal target, the weighted value Y of the abnormal targetiIf the concentration is high, the initial pheromone concentration is high; setting a cruise initial position S, an abnormal target task point and a maximum iteration number NmaxAnd the number g of ants, at the initial moment, placing g ants at the starting point S, and enabling the ants to randomly walk to the abnormal target point;
step S2.2: the ant selects a circuit according to the probability, and the calculation formula of the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t is as follows:
wherein the content of the first and second substances,
the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t, wherein b is the abnormal target task point which is not migrated; tau isabPheromone concentration for pathway (a, b); etaabHeuristic information for the distance between the outlier target task points,
dabthe distance between the abnormal target task points a and b is obtained; alpha and beta are given parameters; gkThe abnormal target task points which are not walked by the ants are collected; setting the number of abnormal target task points as n, initial stage, GkN-1 elements, i.e. including other abnormal target task points except the abnormal target task from ant k, over time, GkThe elements in the Chinese herbal medicine are less and less until the Chinese herbal medicine is empty; alpha is an pheromone importance factor, which is called an information factor for short, and the larger the value of the alpha is, the larger the information influence strength is; beta is a factor of the importance degree of the heuristic function, which is called the heuristic function factor for short, and the larger the value of the factor is, the larger the influence of the heuristic function is.
Step S2.3: when an ant transfers to an abnormal target task point, whether the ant can return to the initial point is judged: if the ant can return to the starting point, the next abnormal target task point to which the ant wants to walk is continuously calculated according to the step S2.2, and then the ant is transferred to the next abnormal target task point until the ant returns to the starting point; if the ant can not return to the starting point, returning to the previous abnormal target task point to reselect the next wandering abnormal target point, and when all the next abnormal target task points of the ant can not return to the starting point, directly returning the ant to the starting point, and ending the cycle; so that the updated pheromone table has ants to finish accessing all the abnormal target task points, thereby finishing the random walk of the ants;
step S2.4: when the ant k finishes random walk, recording the optimal solution in the current iteration times, simultaneously updating the pheromone concentration on each abnormal target task point connection path of the current walk, and locally updating the path to correct the pheromone concentration in the following calculation mode:
wherein the content of the first and second substances,
wherein the content of the first and second substances,
the pheromone concentration on the connecting path (a, b) of the abnormal target task point a and the abnormal target task point b at the time t,
the pheromone concentration increased by releasing pheromones on the connecting paths (a, b) of the abnormal target task points a and b of the ant k in the current cycle is represented, namely the pheromone increment on the sides (a, b); lkL is the total length of path of kth ant, mu is given parameter, QkIs the weighted sum of all abnormal targets passed by ant k;
step S2.5: judging whether the iteration reaches the maximum iteration number NmaxIf yes, outputting the optimal path, and if not, returning to the step S2.2 to continue circulation;
step S2.6: and (5) obtaining an optimal path for the unmanned aerial vehicle to finish the cruise abnormal target task according to the step S2.5, finishing the planning of the abnormal detection route of the unmanned aerial vehicle based on the ant colony algorithm, importing the optimal path into an unmanned aerial vehicle control system, so that the unmanned aerial vehicle finishes the detection and classification of the abnormal target through the optimal path, and finally returning the detection result of the abnormal target.
Compared with the prior art, the invention has the beneficial technical effects that:
according to the method, the convolutional neural network is used for training the abnormal target detection model, so that the automatic identification and detection of the abnormal target are realized, and the conditions of misjudgment, missing report and the like of the abnormal target detection are reduced; and then, the unmanned aerial vehicle cruising dynamic route planning is carried out on the abnormal target points through the route planning based on the ant colony algorithm, so that the loss of the unmanned aerial vehicle is reduced. And finally, mutually verifying the abnormal target detection result of the fused unmanned aerial vehicle and the monitoring abnormal target detection result, so that the abnormal target detection is mutually verified by the monitoring video and the unmanned aerial vehicle cruise video, the conditions of missing judgment and erroneous judgment of the abnormal target are reduced, and the accuracy and stability of the abnormal detection are improved.
Drawings
Fig. 1 is a flowchart of an abnormal target detection method for video surveillance and unmanned aerial vehicle air-ground linkage in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments, but the scope of the present invention is not limited to the following embodiments.
Examples
Referring to fig. 1, the embodiment discloses an abnormal target detection method for video monitoring and unmanned aerial vehicle air-ground linkage, which includes the following steps:
step S1: acquiring monitoring video data through video monitoring, wherein the monitoring video data comprises abnormal target video data, the video monitoring can detect abnormal targets in real time, feature extraction is carried out on historical abnormal target video data in the video data to be used as input parameters, and the input parameters are imported into a constructed model for training; after training is finished, new abnormal target data are imported into the model to obtain a judgment result, so that prediction inspection is carried out; meanwhile, calculating a weight value of the abnormal target according to the obtained type and confidence F of the abnormal target, and sorting and dividing the abnormal target from high to low according to the weight value;
step S2: taking the weight value of each abnormal target obtained in the step S1 as an initial pheromone concentration, and planning an optimal path for the unmanned aerial vehicle to finish the task of the cruise abnormal target point according to an ant colony algorithm based on the initial pheromone concentration so that the unmanned aerial vehicle finishes the detection of the abnormal target on the optimal path and finishes the detection of the unmanned aerial vehicle cruise video on the abnormal target;
step S3: combining the video monitoring abnormal target detection result obtained in the step S1 and the unmanned aerial vehicle abnormal target detection result obtained in the step S2, checking whether the abnormal type and the detection result of the abnormal target at the same position are consistent, and if the mutual checking results are consistent, returning abnormal target information and storing the abnormal target information to a server background; and if the verification results are inconsistent, the relevant responsible person performs manual verification, and the verification results are returned and stored in the server background.
The model in step S1 is a convolutional neural network model.
The step S1 specifically includes the following steps:
step S1.1: image preprocessing, namely acquiring a monitoring video data set, acquiring an image sequence from the monitoring video data set, and performing normalization processing on the image sequence to be detected;
step S1.2: extracting the characteristics of the historical abnormal target video data to obtain the characteristic attributes of the historical abnormal target, and dividing the processed historical abnormal target video data into a training set and a test set;
step S1.3: building a convolutional neural network model, training the convolutional neural network model based on the data training set related to the abnormal target features processed in the step S1.2, verifying the convolutional neural network model by adopting a test set to obtain the trained convolutional neural network model, taking the trained convolutional neural network model as an abnormal target detection model, and outputting an abnormal target type and an abnormal target confidence F through the abnormal target detection model;
step S1.4: defining n abnormal measurement indexes for measuring importance degree of abnormal target, and establishing abnormal measurement matrix A for m abnormal targets detected in video monitoringm×nWherein A isijRepresented as the ith anomaly target on the jth anomaly measureTaking values;
step S1.5: the formula for calculating the weight value of each abnormal target is as follows:
wherein, FiIs the confidence corresponding to the ith abnormal target in the step S1.3, AijThe value of the ith abnormal target on the jth abnormal measurement index in the step S1.4 is obtained;
step S1.6: and (4) carrying out normalization processing on the weight values of the abnormal targets calculated in the step (S1.5).
Preferably, the feature extraction process performed on the historical abnormal target video data in step S1.2 is as follows:
acquiring an RGBD image sequence in a historical abnormal target video data set, and performing background removal processing on color information according to depth information to acquire a target characteristic zone bit corresponding to a target; and performing feature extraction on pixel points corresponding to the target feature location in the color information to obtain historical abnormal target feature attributes.
In the step S1.2, the processed historical abnormal target video data is divided into a training set and a test set according to a ratio of 7: 3. The anomaly measure index in step S1.4 at least includes a degree of harm index, an urgency index, and an influence range index.
The step S2 specifically includes the following steps:
step S2.1: initializing pheromones and setting parameters, and obtaining weight values Y of various abnormal targets subjected to normalization processing according to step S1.6iAs the initial pheromone concentration, the abnormal target weight value YiNamely the weighted score value of the abnormal target, the weighted value Y of the abnormal targetiIf the concentration is high, the initial pheromone concentration is high; setting a cruise initial position S, an abnormal target task point and a maximum iteration number NmaxAnd the number g of ants, at the initial moment, placing g ants at the starting point S, and enabling the ants to randomly walk to the abnormal target point;
step S2.2: the ant selects a circuit according to the probability, and the calculation formula of the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t is as follows:
wherein the content of the first and second substances,
the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t, wherein b is the abnormal target task point which is not migrated; tau isabPheromone concentration for pathway (a, b); etaabHeuristic information for the distance between the outlier target task points,
dabthe distance between the abnormal target task points a and b is obtained; alpha and beta are given parameters; gkThe abnormal target task points which are not walked by the ants are collected; setting the number of abnormal target task points as n, initial stage, GkN-1 elements, i.e. including other abnormal target task points except the abnormal target task from ant k, over time, GkThe elements in the Chinese herbal medicine are less and less until the Chinese herbal medicine is empty; alpha is an pheromone importance factor, which is called an information factor for short, and the larger the value of the alpha is, the larger the information influence strength is; beta is a factor of the importance degree of the heuristic function, which is called the heuristic function factor for short, and the larger the value of the factor is, the larger the influence of the heuristic function is.
Step S2.3: when an ant transfers to an abnormal target task point, whether the ant can return to the initial point is judged: if the ant can return to the starting point, the next abnormal target task point to which the ant wants to walk is continuously calculated according to the step S2.2, and then the ant is transferred to the next abnormal target task point until the ant returns to the starting point; if the ant can not return to the starting point, returning to the previous abnormal target task point to reselect the next wandering abnormal target point, and when all the next abnormal target task points of the ant can not return to the starting point, directly returning the ant to the starting point, and ending the cycle; so that the updated pheromone table has ants to finish accessing all the abnormal target task points, thereby finishing the random walk of the ants;
step S2.4: when the ant k finishes random walk, recording the optimal solution in the current iteration times, simultaneously updating the pheromone concentration on each abnormal target task point connection path of the current walk, and locally updating the path to correct the pheromone concentration in the following calculation mode:
wherein the content of the first and second substances,
wherein the content of the first and second substances,
the pheromone concentration on the connecting path (a, b) of the abnormal target task point a and the abnormal target task point b at the time t,
the pheromone concentration increased by releasing pheromones on the connecting paths (a, b) of the abnormal target task points a and b of the ant k in the current cycle is represented, namely the pheromone increment on the sides (a, b); lkL is the total length of path of kth ant, mu is given parameter, QkIs the weighted sum of all abnormal targets passed by ant k;
step S2.5: judging whether the iteration reaches the maximum iteration number NmaxIf yes, outputting the optimal path, and if not, returning to the step S2.2 to continue circulation;
step S2.6: and (5) obtaining an optimal path for the unmanned aerial vehicle to finish the cruise abnormal target task according to the step S2.5, finishing the planning of the abnormal detection route of the unmanned aerial vehicle based on the ant colony algorithm, importing the optimal path into an unmanned aerial vehicle control system, so that the unmanned aerial vehicle finishes the detection and classification of the abnormal target through the optimal path, and finally returning the detection result of the abnormal target.
According to the method, the convolutional neural network is used for training the abnormal target detection model, so that the automatic identification and detection of the abnormal target are realized, and the conditions of misjudgment, missing report and the like of the abnormal target detection are reduced; and then, performing unmanned aerial vehicle cruising dynamic route planning on the abnormal target point through the route planning based on the ant colony algorithm, reducing the loss of the unmanned aerial vehicle, finally performing mutual verification on the detection result of the abnormal target of the unmanned aerial vehicle in the step S2 and the detection result of the monitoring abnormal target in the step S1, realizing the mutual verification of the monitoring video and the unmanned aerial vehicle cruising video on the detection of the abnormal target, reducing the conditions of missing judgment and erroneous judgment of the abnormal target, and improving the accuracy and stability of the abnormal detection.
Variations and modifications to the above-described embodiments may occur to those skilled in the art, which fall within the scope and spirit of the above description. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and variations of the present invention should fall within the scope of the claims of the present invention. Furthermore, although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
Claims (7)
1. An abnormal target detection method for video monitoring and unmanned aerial vehicle air-ground linkage is characterized by comprising the following steps:
step S1: acquiring monitoring video data through video monitoring, wherein the monitoring video data comprises abnormal target video data, the video monitoring can detect abnormal targets in real time, feature extraction is carried out on historical abnormal target video data in the video data to be used as input parameters, and the input parameters are imported into a constructed model for training; after training is finished, new abnormal target data are imported into the model to obtain a judgment result, so that prediction inspection is carried out; meanwhile, calculating a weight value of the abnormal target according to the obtained type and confidence F of the abnormal target, and sorting and dividing the abnormal target from high to low according to the weight value;
step S2: taking the weight value of each abnormal target obtained in the step S1 as an initial pheromone concentration, and planning an optimal path for the unmanned aerial vehicle to complete the task of cruising at the abnormal target point according to the ant colony algorithm based on the initial pheromone concentration, so that the unmanned aerial vehicle can complete the detection of the abnormal target on the optimal path;
step S3: combining the video monitoring abnormal target detection result obtained in the step S1 and the unmanned aerial vehicle abnormal target detection result obtained in the step S2, checking whether the abnormal type and the detection result of the abnormal target at the same position are consistent, and if the mutual checking results are consistent, returning abnormal target information and storing the abnormal target information to a server background; and if the verification results are inconsistent, the relevant responsible person performs manual verification, and the verification results are returned and stored in the server background.
2. The method for detecting the abnormal target of the video surveillance and unmanned aerial vehicle air-ground linkage according to claim 1, wherein the model in the step S1 is a convolutional neural network model.
3. The method for detecting the abnormal target of the video monitoring and unmanned aerial vehicle air-ground linkage according to claim 1 or 2, wherein the step S1 specifically comprises the following steps:
step S1.1: image preprocessing, namely acquiring a monitoring video data set, acquiring an image sequence from the monitoring video data set, and performing normalization processing on the image sequence to be detected;
step S1.2: extracting the characteristics of the historical abnormal target video data to obtain the characteristic attributes of the historical abnormal target, and dividing the processed historical abnormal target video data into a training set and a test set;
step S1.3: building a convolutional neural network model, training the convolutional neural network model based on the data training set related to the abnormal target features processed in the step S1.2, verifying the convolutional neural network model by adopting a test set to obtain the trained convolutional neural network model, taking the trained convolutional neural network model as an abnormal target detection model, and outputting an abnormal target type and an abnormal target confidence F through the abnormal target detection model;
step S1.4: defining n abnormal measurement indexes for measuring importance degree of abnormal target, and establishing abnormal measurement matrix A for m abnormal targets detected in video monitoringm×nWherein A isijThe representation is the value of the ith abnormal target on the jth abnormal measurement index;
step S1.5: the formula for calculating the weight value of each abnormal target is as follows:
wherein, FiIs the confidence corresponding to the ith abnormal target in the step S1.3, AijThe value of the ith abnormal target on the jth abnormal measurement index in the step S1.4 is obtained;
step S1.6: and (4) carrying out normalization processing on the weight values of the abnormal targets calculated in the step (S1.5).
4. The method for detecting the abnormal target of the video monitoring and unmanned aerial vehicle air-ground linkage according to claim 3, wherein the step S1.2 of extracting the characteristics of the historical abnormal target video data comprises the following steps:
acquiring an RGBD image sequence in a historical abnormal target video data set, and performing background removal processing on color information according to depth information to acquire a target characteristic zone bit corresponding to a target; and performing feature extraction on pixel points corresponding to the target feature location in the color information to obtain historical abnormal target feature attributes.
5. The method for detecting the abnormal target of the video monitoring and unmanned aerial vehicle air-ground linkage according to claim 4, wherein the processed historical abnormal target video data is divided into a training set and a testing set according to a ratio of 7:3 in the step S1.2.
6. The method according to claim 3, wherein the abnormal measurement indexes in step S1.4 at least include a risk degree index, an emergency degree index and an influence range index.
7. The method for detecting the abnormal target of the video monitoring and unmanned aerial vehicle air-ground linkage according to claim 3, wherein the step S2 specifically comprises the following steps:
step S2.1: initializing pheromones and setting parameters, and obtaining weight values Y of various abnormal targets subjected to normalization processing according to step S1.6iA weight value Y of the abnormal target as an initial pheromone concentrationiIf the concentration is high, the initial pheromone concentration is high; setting a cruise initial position S, an abnormal target task point and a maximum iteration number NmaxAnd the number g of ants, placing g ants at the starting point S, and randomly walking the abnormal target points by the ants;
step S2.2: the ant selects a circuit according to the probability, and the calculation formula of the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t is as follows:
wherein the content of the first and second substances,
the probability that the ant k is transferred from the abnormal target task point a to the abnormal target task point b at the moment t, wherein b is the abnormal target task point which is not migrated; tau isabPheromone concentration for pathway (a, b); etaabHeuristic information for the distance between the outlier target task points,
dabthe distance between the abnormal target task points a and b is obtained; alpha and beta are given parameters; gkIs the difference of ants not migratingA common target task point set;
step S2.3: when an ant transfers to an abnormal target task point, whether the ant can return to the initial point is judged: if the ant can return to the starting point, the next abnormal target task point to which the ant wants to walk is continuously calculated according to the step S2.2, and then the ant is transferred to the next abnormal target task point until the ant returns to the starting point; if the ant can not return to the starting point, returning to the previous abnormal target task point to reselect the next wandering abnormal target point, and when all the next abnormal target task points of the ant can not return to the starting point, directly returning the ant to the starting point, and ending the cycle;
step S2.4: when the ant k finishes random walk, recording the optimal solution in the current iteration times, simultaneously updating the pheromone concentration on each abnormal target task point connection path of the current walk, and locally updating the path to correct the pheromone concentration in the following calculation mode:
wherein the content of the first and second substances,
wherein the content of the first and second substances,
the pheromone concentration on the connecting path (a, b) of the abnormal target task point a and the abnormal target task point b at the time t,
the pheromone concentration increased by releasing pheromones on the connecting paths (a, b) of the abnormal target task points a and b of the ant k in the current cycle is represented, namely the pheromone increment on the sides (a, b); lkL is the total length of path of kth ant, mu is given parameter, QkIs the weighted sum of all abnormal targets passed by ant k;
step S2.5: judging whether the iteration reaches the maximum iteration number NmaxIf yes, outputting the optimal path, and if not, returning to the step S2.2 to continue circulation;
step S2.6: and (5) obtaining an optimal path for the unmanned aerial vehicle to finish the cruise abnormal target task according to the step S2.5, finishing the planning of the abnormal detection route of the unmanned aerial vehicle based on the ant colony algorithm, importing the optimal path into an unmanned aerial vehicle control system, so that the unmanned aerial vehicle finishes the detection and classification of the abnormal target through the optimal path, and finally returning the detection result of the abnormal target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111135520.XA CN114035604B (en) | 2021-09-27 | 2021-09-27 | Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111135520.XA CN114035604B (en) | 2021-09-27 | 2021-09-27 | Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114035604A true CN114035604A (en) | 2022-02-11 |
| CN114035604B CN114035604B (en) | 2022-06-17 |
Family
ID=80140259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111135520.XA Active CN114035604B (en) | 2021-09-27 | 2021-09-27 | Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114035604B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935286A (en) * | 2023-08-03 | 2023-10-24 | 广州城市职业学院 | Short video identification system |
| CN116973522A (en) * | 2023-09-22 | 2023-10-31 | 深圳市天地互通科技有限公司 | Integrated automatic atmosphere monitoring control system |
| CN117576100A (en) * | 2024-01-16 | 2024-02-20 | 浙江合丰科技有限公司 | Surface defect grading detection and evaluation method for FPC connector |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582117A (en) * | 2020-04-29 | 2020-08-25 | 长江大学 | Unmanned aerial vehicle illegal building inspection method, equipment and storage medium |
| CN111814720A (en) * | 2020-07-17 | 2020-10-23 | 电子科技大学 | Airport runway foreign matter detection and classification method based on unmanned aerial vehicle vision |
| CN111814721A (en) * | 2020-07-17 | 2020-10-23 | 电子科技大学 | Airfield runway foreign matter detection and classification method based on unmanned aerial vehicle high-low altitude combined scanning |
| WO2020236788A1 (en) * | 2019-05-20 | 2020-11-26 | Tg-17, Llc | Systems and methods for real-time adjustment of neural networks for autonomous tracking and localization of moving subject |
| CN112729248A (en) * | 2020-12-18 | 2021-04-30 | 湖南文理学院 | Remote sensing detection system and method based on unmanned aerial vehicle platform |
| CN112910410A (en) * | 2021-01-21 | 2021-06-04 | 合肥工业大学 | Hot spot effect monitoring system of photovoltaic power station |
-
2021
- 2021-09-27 CN CN202111135520.XA patent/CN114035604B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020236788A1 (en) * | 2019-05-20 | 2020-11-26 | Tg-17, Llc | Systems and methods for real-time adjustment of neural networks for autonomous tracking and localization of moving subject |
| CN111582117A (en) * | 2020-04-29 | 2020-08-25 | 长江大学 | Unmanned aerial vehicle illegal building inspection method, equipment and storage medium |
| CN111814720A (en) * | 2020-07-17 | 2020-10-23 | 电子科技大学 | Airport runway foreign matter detection and classification method based on unmanned aerial vehicle vision |
| CN111814721A (en) * | 2020-07-17 | 2020-10-23 | 电子科技大学 | Airfield runway foreign matter detection and classification method based on unmanned aerial vehicle high-low altitude combined scanning |
| CN112729248A (en) * | 2020-12-18 | 2021-04-30 | 湖南文理学院 | Remote sensing detection system and method based on unmanned aerial vehicle platform |
| CN112910410A (en) * | 2021-01-21 | 2021-06-04 | 合肥工业大学 | Hot spot effect monitoring system of photovoltaic power station |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935286A (en) * | 2023-08-03 | 2023-10-24 | 广州城市职业学院 | Short video identification system |
| CN116935286B (en) * | 2023-08-03 | 2024-01-09 | 广州城市职业学院 | Short video identification system |
| CN116973522A (en) * | 2023-09-22 | 2023-10-31 | 深圳市天地互通科技有限公司 | Integrated automatic atmosphere monitoring control system |
| CN117576100A (en) * | 2024-01-16 | 2024-02-20 | 浙江合丰科技有限公司 | Surface defect grading detection and evaluation method for FPC connector |
| CN117576100B (en) * | 2024-01-16 | 2024-03-22 | 浙江合丰科技有限公司 | Surface defect grading detection and evaluation method for FPC connector |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114035604B (en) | 2022-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114035604B (en) | Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method | |
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| KR102328734B1 (en) | Method for automatically evaluating labeling reliability of training images for use in deep learning network to analyze images, and reliability-evaluating device using the same | |
| CN110866887A (en) | Target situation fusion sensing method and system based on multiple sensors | |
| CN113361354B (en) | Track component inspection method and device, computer equipment and storage medium | |
| CN114299417A (en) | Multi-target tracking method based on radar-vision fusion | |
| CN109255286A (en) | A kind of quick detection recognition method of unmanned plane optics based on YOLO deep learning network frame | |
| CN105930768A (en) | Spatial-temporal constraint-based target re-identification method | |
| CN110018453A (en) | Intelligent type recognition methods based on aircraft track feature | |
| CN111062303A (en) | Image processing method, system and computer storage medium | |
| CN101794437A (en) | Method for detecting abnormal target in hyperspectral remotely sensed image | |
| CN112465854A (en) | Unmanned aerial vehicle tracking method based on anchor-free detection algorithm | |
| CN111259736B (en) | Real-time pedestrian detection method based on deep learning in complex environment | |
| CN114841920A (en) | Flame identification method and device based on image processing and electronic equipment | |
| CN116258980A (en) | Unmanned aerial vehicle distributed photovoltaic power station inspection method based on vision | |
| CN115690564A (en) | Outdoor fire smoke image detection method based on Recursive BIFPN network | |
| CN115690545B (en) | Method and device for training target tracking model and target tracking | |
| CN116580322A (en) | Unmanned aerial vehicle infrared small target detection method under ground background | |
| CN112633262B (en) | Channel monitoring method and device, electronic equipment and medium | |
| CN114494845A (en) | Artificial intelligence hidden danger troubleshooting system and method for construction project site | |
| CN112017213A (en) | Target object position updating method and system | |
| CN114550074B (en) | Image recognition method and system based on computer vision | |
| Khudov et al. | The Choice of Quality Indicator for the Image Segmentation Evaluation | |
| CN115410102A (en) | SAR image airplane target detection method based on combined attention mechanism | |
| CN113780462A (en) | Vehicle detection network establishment method based on unmanned aerial vehicle aerial image and application thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 510000 room 1301 (Location: room 1301-1), No. 68, yueken Road, Wushan street, Tianhe District, Guangzhou City, Guangdong Province (office only) Applicant after: Guangzhou Fu'an Digital Technology Co.,Ltd. Address before: 510000 No. 1501, 68 yueken Road, Tianhe District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU FUAN ELECTRONIC TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |