CN115147385A - Intelligent detection and judgment method for repeated damage in aviation hole exploration video - Google Patents
Intelligent detection and judgment method for repeated damage in aviation hole exploration video Download PDFInfo
- Publication number
- CN115147385A CN115147385A CN202210820374.2A CN202210820374A CN115147385A CN 115147385 A CN115147385 A CN 115147385A CN 202210820374 A CN202210820374 A CN 202210820374A CN 115147385 A CN115147385 A CN 115147385A
- Authority
- CN
- China
- Prior art keywords
- detection
- frame
- track
- damage
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 115
- 230000006378 damage Effects 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000001914 filtration Methods 0.000 claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims abstract description 22
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 8
- 230000003252 repetitive effect Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 230000003902 lesion Effects 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 4
- 238000012790 confirmation Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 208000027418 Wounds and injury Diseases 0.000 claims description 2
- 230000006735 deficit Effects 0.000 claims description 2
- 208000014674 injury Diseases 0.000 claims description 2
- 238000012360 testing method Methods 0.000 claims description 2
- 239000000523 sample Substances 0.000 claims 2
- 238000013528 artificial neural network Methods 0.000 claims 1
- 230000004927 fusion Effects 0.000 abstract description 5
- 230000008034 disappearance Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 2
- UFHFLCQGNIYNRP-UHFFFAOYSA-N Hydrogen Chemical compound [H][H] UFHFLCQGNIYNRP-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 229910052739 hydrogen Inorganic materials 0.000 description 1
- 239000001257 hydrogen Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent detection and judgment method for repeated damage in an aviation hole exploration video, which comprises the steps of firstly carrying out target detection on internal damage, reading the position of a target detection frame of a current frame and the depth characteristics of image blocks of each detection frame; then filtering the detection frame according to the confidence coefficient; the maximum value of the detection frames is suppressed so as to eliminate the phenomenon that a plurality of detection frames exist in the same damage; predicting the position of a target current frame by using a Kalman filtering tracker; calculating a cost matrix of the track information and the detection information according to the Mahalanobis distance based on the appearance information, and sequentially performing cascade matching and IOU matching; and finally, updating parameters and feature sets of the Kalman filtering tracker, and judging the disappearance of the target and the appearance of a new target. In the process, in order to solve the problem of repeated damage misjudgment in the hole exploration video, the invention provides an intelligent detection and judgment method for repeated damage in the aviation hole exploration video based on the fusion of a twin convolutional neural network and an optimal stack block.
Description
Technical Field
The invention relates to an intelligent detection and judgment method for repetitive damage in an aviation hole exploration video based on a twin network and an optimal stacking block, and belongs to the field of computer vision.
Background
The traditional target tracking method has made great progress over years, and can obtain good tracking effect to a certain extent. For example, target tracking is performed through feature space and compressive sampling, a filter is used for relieving the influence caused by shielding in the tracking process, and multi-target tracking is performed through a swarm optimization algorithm. Although the method can achieve the target tracking effect to a certain extent, the designed tracker cannot sufficiently learn the characteristics of the target content in the image, so that the tracking effect is unstable, and the work of marking and judging whether the damage is repeated in real time is difficult to complete.
In recent years, the development of target detection and tracking technology is rapid, and the tracking-detection technology has become a mainstream technology in the field of multi-target tracking. But the method mainly focuses on detection and tracking application of pedestrians and vehicles, has less damage research on the aero-engine, and has very important significance for effectively tracking the damage of the aero-engine as the core power of the aero field. The YOLO algorithm is an efficient identification algorithm, has excellent performance for the detection and identification of the target, can be used as a target detector in a detection stage in target tracking, and is used for providing an initial position of a tracked damaged target of the aircraft engine.
In the application of the actual aeroengine hole detection technology, the internal structure of the engine is complex, the change of the hole detection shooting angle is large, the problems that a detection target is shielded, the shooting angle is changed, illumination is changed and the like can be solved, the existing technology is not suitable for the actual hole detection technology, and the requirements of real-time detection and judgment on whether repeated damage exists can not be met.
Disclosure of Invention
In order to solve the problems, the invention provides an intelligent detection and judgment method for repeated damage in an aviation hole exploration video based on a twin network and an optimal stacking block, the method is high in precision and high in processing speed, the repetition rate of damage detection is reduced, and the method is more suitable for objects with large angle change.
Firstly, carrying out target detection on internal (such as internal engine) damage through an aviation hole detection video (such as an aviation engine internal hole detection video), and reading the position of a current frame target detection frame and the depth characteristics of each detection frame image block;
then filtering the detection frame according to the confidence coefficient; carrying out maximum suppression on the detection frames so as to eliminate the phenomenon that a plurality of detection frames exist in the same damage; predicting the position of a target current frame by using a Kalman filtering tracker;
calculating a cost matrix of the track information and the detection information according to the Mahalanobis distance based on the appearance information, and sequentially performing cascade matching and IOU matching;
and finally, updating parameters and feature sets of the Kalman filtering tracker, and judging the disappearance of the target and the appearance of a new target.
In the process, in order to solve the problem of repeated damage interference caused by online statistics of damage, an intelligent detection and judgment method for repeated damage in an aviation hole exploration video is provided based on a fusion technology of a twin convolutional neural network and an optimal stack block, and the specific steps comprise:
firstly, establishing a corresponding track frame for a result detection frame detected by a first frame in an original frame of a given video in an acquired hole detection video in the aircraft engine;
step two, performing IOU matching on the frame of target detection and the frame of the previous frame passing through the track prediction one by one, and calculating a cost matrix of the frame;
step three, taking all the cost matrixes obtained in the step two as input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm;
step four, the step two and the step three are circulated until the matched result shows a track of a confirmation state or the video is finished;
predicting a prediction frame corresponding to the track of the confirmed state and the track of the unconfirmed state of the matching result through Kalman filtering; then, cascade matching is carried out on the prediction frame of the track in the confirmed state and the detection frame of the detection result of the original video frame, and a corresponding cost matrix is calculated according to the matching result;
taking all the cost matrixes obtained in the step five as input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm;
step seven, sending the matching result obtained in the step six into a model fusing a twin convolutional neural network and the optimal stacking block for similarity judgment, and detecting and judging the obtained damage;
and step eight, circulating the step five to the step seven until the video is finished.
The detection frame is the target detection frame.
The invention has the beneficial effects that:
according to the technical scheme, the semantic features of the objects which are large in shooting angle change, shielded and the like can be accurately propagated, a tracking algorithm is added on the basis of target detection, repeated damage is removed, and accuracy of damage detection is improved.
The technical reasons for achieving the above results in the technical scheme are as follows:
1) The existing detector such as YOLOV5 is used as a target detection detector, and the weight value and the network of the detector are modified so as to solve the problem that the same damage identification is greatly influenced by the change of the shooting angle.
2) The method for solving the association between the detection result and the tracking prediction result is to use the Hungarian algorithm, and the association of the motion information and the association of the target appearance information are considered here.
3) By introducing cascade matching, when a target is shielded for a long time, the uncertainty of Kalman filtering prediction is greatly increased, and the observability in a state space is greatly reduced.
If two trackers compete for the matching right of the same detection result, the uncertainty of tracking the predicted position is larger because the position information is not updated for a long time, that is, the covariance is larger, the inverse of the covariance is used in the mahalanobis distance calculation, and therefore the mahalanobis distance is smaller, so that the detection result is more likely to be associated with the track with longer occlusion time, and this undesirable effect often destroys the continuity of tracking. The final stage of matching also performs IOU-based matching on uncertain states and outdated unmatched successful tracks.
4) The fusion of a twin network and the optimal stacking speed is added, so that the problem of repeated judgment of the same damage caused by shielding, illumination and the like of an engine damage target is effectively solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of the invention as a whole.
FIG. 2 is a fusion twin network and optimal stacked block flow diagram.
Detailed Description
The invention discloses an intelligent detection and judgment method for repeated damage in an aviation hole exploration video, which comprises the steps of firstly carrying out target detection on internal damage, reading the position of a target detection frame of a current frame and the depth characteristics of image blocks of each detection frame; then filtering the detection frame according to the confidence coefficient; the maximum value of the detection frames is suppressed so as to eliminate the phenomenon that a plurality of detection frames exist in the same damage; predicting the position of a target current frame by using a Kalman filtering tracker; calculating a cost matrix of the track information and the detection information according to the Mahalanobis distance based on the appearance information, and sequentially performing cascade matching and IOU matching; and finally, updating parameters and feature sets of the Kalman filtering tracker, and judging the disappearance of the target and the appearance of a new target. In the process, in order to solve the problem of repeated damage misjudgment in the hole exploration video, the invention provides an intelligent detection and judgment method for repeated damage in the aviation hole exploration video based on the fusion of a twin convolutional neural network and an optimal stack block.
The invention is described in further detail below with reference to the following figures and detailed description:
the specific embodiment of the invention is as follows:
firstly, in the acquired hole detection video in the aircraft engine, for an original frame of a given video, establishing a corresponding track of a detection frame of a first frame. Specifically, the first step includes the following substeps:
step 1.1, detecting a video frame by adopting a target detector to obtain a detection frame;
step 1.2, marking out the corresponding damage in all the target frames, and extracting the characteristics; the characteristics comprise apparent characteristics and motion characteristics;
and 1.3, initializing the motion variables of Kalman filtering, and predicting the corresponding prediction frame through Kalman filtering.
And step two, performing IOU matching on the detection frame and the track frame of the previous frame one by one, and calculating a cost matrix of the detection frame and the track frame.
And step three, taking all the cost matrixes obtained in the step two as the input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm. Specifically, step three includes the following substeps:
3.1, subtracting the minimum element of each row of the cost matrix;
3.2, generating each row of elements of the matrix and subtracting the minimum element of the row;
step 3.3, connecting all zeros in the new matrix by using the least row lines and column lines, and checking whether the current distribution is optimal; if the row line and the column line do not connect all the elements of the matrix, step 3.4 is entered, otherwise step 3.5 is entered;
3.4, finding the minimum element from the elements which are not connected with the row lines and the column lines, subtracting the minimum element from the rest elements, and adding the minimum element to the element corresponding to the intersection point of the row lines and the column lines;
step 3.5, finding out 0 elements corresponding to each row and 0 elements corresponding to the columns, and finding out optimal distribution according to the 0 elements;
step 3.6, to incorporate the motion information, the square of the Mahalanobis distance between the predicted Kalman Filter State and the newly arrived measurement value is used, i.e. the first metric d (1) Represents:
wherein d is (1) (i, j) represents the motion matching degree between the jth detection frame and the ith track, x i Is the predicted observed quantity of the trajectory at the current moment, S i Is a covariance matrix of the observation space at the current moment predicted by a Kalman filter, using d j Indicating the detected current track state of the target;
step 3.7, using the index of step 3.6, eliminates low probability correlations by setting the threshold for mahalanobis distance to 95% of the confidence interval calculated by the inverse chi-square distribution, with an indicator representing this decision:
wherein the corresponding March threshold is t (1) When d is greater than d (1) (i, j) is greater than t (1) Then, then
Is set to 0 and otherwise is 1.
When the number of the detection frames is 0, the Markov distance between the ith track frame and the jth detection frame is not matched, and when the number of the detection frames is 1, the Markov distance is matched;
step 3.8, measuring the minimum cosine distance between the ith track frame and the jth detection frame by using a second metric, and using d (2) Represents:
wherein R is i Is a container for storing the feature vector of the track frame of the matching result. r is a radical of hydrogen j Is the feature vector of the jth detection box,
k feature vectors representing the ith trajectory box;
step 3.9, introduce a binary variable to indicate whether the metric can accept a correlation:
wherein, t (2) Is a threshold value, if d (2) (i, j) is greater than t (2) Then, then
Is set to 0, otherwise is 1;
when the cosine distance is 0, the cosine distance between the ith track frame and the jth detection frame is not matched, and when the cosine distance is 1, the cosine distance is matched;
step 3.10, to build the correlation problem, two metrics are combined using a weighted sum:
h (i，j) ＝λd (1) (i，j)+(1-λ)d (2) (i，j)
wherein d is (1) (i, j) is defined in step 3.6, d (2) (i, j) is defined in step 3.8, lambda represents weight, and the value range is 0-1;
step 3.11, if an association is within the gated region for both metrics, it is said to be an acceptable association:
the results of linear matching are three:
the first is track mismatch, and the mismatched track is directly deleted;
the second is a mismatch between the predicted result and the detected result, when such a detected result is initialized as a new trajectory;
and the third is that the detection frame of the next frame and the prediction frame of the previous frame are successfully matched, namely the two adjacent frames are successfully tracked, and at the moment, the corresponding track variable of the detection result of the next frame is updated through Kalman filtering.
And step four, circulating the step two and the step three until the matching result shows a track of a confirmation state or all frames are processed.
And step five, predicting frames corresponding to the track of the confirmed state and the track of the unconfirmed state through Kalman filtering, and performing cascade matching on the track frame of the confirmed state and the detection frame. Specifically, the step five includes the following substeps:
step 5.1, kalman filtering prediction, namely predicting a value at the next moment according to an estimated value at the previous moment, which is called prior estimation, and predicting an error at the next moment at the same time, which is called prior error;
there are three kinds of cascade matching results:
firstly, track matching, namely updating the corresponding track variable of the track through Kalman filtering;
secondly, track mismatching, namely, carrying out IOU matching on the track in the previous uncertain state and the mismatching track together with a mismatching detection frame one by one, and calculating a cost matrix of the mismatching track through the result of the IOU matching;
and thirdly, mismatching is detected, the track in the previous uncertain state and the mismatching track are matched with the mismatching detection frame one by one, and then the cost matrix of the mismatching detection frame is calculated according to the result of the IOU matching.
And step six, taking all the cost matrixes obtained in the step five as input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm. Specifically, the sixth step includes the following substeps:
step 6.1, if the tracks are mismatched, directly deleting the mismatched tracks, otherwise, entering step 6.2;
step 6.2, if the detection results are mismatched, initializing the detection results into a new track, otherwise, entering step 6.3;
and 6.3, if the detection frame and the predicted frame are successfully matched, namely the previous frame and the next frame are successfully tracked, updating the corresponding track variables of the detection results through Kalman filtering. The method comprises the following specific steps:
step 6.3.1, updating Kalman filtering, namely firstly calculating Kalman gain, then calculating posterior estimation by using the prior estimation of the previous step, and updating the prior error to obtain a posterior error;
6.3.2, performing feature extraction on the damage at the moment, and identifying again; and carrying out duplicate removal operation on the condition that the same damage occurs in the previous frame and the next frame and different ID values are distributed.
And step seven, sending the matching result obtained in the step six into a model fusing the twin convolutional neural network and the optimal stacking block for similarity judgment, and detecting and judging the obtained damage. Specifically, step seven includes the following substeps:
7.1, for two adjacent frames of images, respectively segmenting the two adjacent frames of images to obtain a plurality of stacked blocks, wherein the stacked blocks are used as sub-images;
step 7.2, each sub-image is independently processed into a preliminary damage similarity verification decision;
7.3, judging the similarity of the two groups of sub-images through damage, and outputting a similarity score;
if the score is not greater than the threshold, not the same lesion; if the score is smaller than the threshold value, discarding the irrelevant sub-image with low judgment, and selecting the optimal verification precision of the sub-image with high judgment;
7.4, fusing the twin convolutional neural network and the optimal stack block, training to obtain a pre-training model, extracting the characteristics of the optimal selection sub-image, and fusing the characteristics into a single characteristic vector;
7.5, transmitting the feature set formed by the feature vector to a classifier to obtain a final damage similarity verification decision;
and 7.6, obtaining a damage similarity verification result, comparing, and reducing the misjudgment rate of the same damage in the aviation hole detection video.
Further, step 7.4 comprises the following sub-steps:
step 7.4.1, adopting a supervised end-to-end deep connected convolutional neural network based on feature similarity; especially the one-piece network architecture based on the feature similarity, the method is different from other methods, a twin network is used for verifying the damage similarity from the whole image through one-time feature extraction, then the single similarity measurement is applied to merge and calculate the similarity between feature vectors, and further whether two kinds of damages are the same damage is judged;
and 7.4.2, training by using the twin convolutional neural network and the optimal stacking block to obtain the best training model so as to enhance the judgment of injury similarity.
Further, step 7.5 comprises the following sub-steps:
and (3) further processing through a result judged by the classifier to improve the performance of judging the damage similarity:
deleting irrelevant parts from the sub-image blocks, and only keeping the public sub-images with strong correlation as relation evidence of damage similarity so as to minimize the distance between the damaged sub-image pairs (same damage) or maximize the distance between the damaged sub-image pairs (non-same damage);
if the original image is segmented into m blocks, i.e. m sub-images, they form m image pairs with the image to be detected;
calling similarity scores of the previous m image pairs; suppose there are N containers C = (C) 1 ，c 2 ，c 3 ，...，c n ) Each container having a parent level
And subsets
Given P a ∈F p ＝(P a，1 ，P a，2 ，P a，3 ，...，P a，n ) And P b ∈S c ＝(P b，1 ，P b，2 ，P b，3 ，...，P b，n ) In which P is a ∈F p And P b ∈S c Respectively indicate that it belongs to F p And S c M pairs of sub-images in a pair, P a And P b Representing each sub-image;
for each container C, a subpicture P a And P b Is C = (C) 1 (P a -P b )，c 2 (P a -P b )，c 3 (P a -P b )，...，c n (P a -P b ))；
Then, based on the test set T = { e = { e } 1 (c 1 (P a -P b ))，e 2 (c 2 (P a -P b ))，e 3 (c 3 (P a -P b ))，...，e n (c n (P a -P b ) B) in which e n Representing similarity accuracy assessment, c n Denotes a container, P a And P b Representing a sub-image;
using the accuracy e obtained in the previous step n To determine the most similar appearance component between the two lesions;
after obtaining the accuracy score, the impairment component is located and selected according to a threshold that specifies similarity, with the sub-image representation that shares the most similar cues ignoring the dissimilar portions.
And step eight, repeatedly circulating the step five, the step six and the step seven until the video frame is finished.
The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and embellishments can be made without departing from the principle of the present invention, and these should also be construed as the scope of the present invention.
Claims (10)
1. An intelligent detection and judgment method for repeated damage in an aviation hole detection video is characterized by comprising the following steps:
firstly, establishing a corresponding track frame for a detection frame of a first frame in an acquired aviation hole detection video for an original frame of a given video;
step two, carrying out IOU matching on the detection frame and the track frame of the previous frame one by one, and calculating a cost matrix of the detection frame and the track frame;
step three, taking all the cost matrixes obtained in the step two as input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm;
step four, the step two and the step three are circulated until a matched result shows a track of a confirmation state or all frames are processed;
predicting a confirmed state track prediction frame and an unconfirmed state track prediction frame of a matching result through Kalman filtering; then, cascade matching is carried out on the track prediction frame in the confirmed state and the detection frame of the original video frame detection result, and a corresponding cost matrix is calculated according to the matching result;
step six, taking all the cost matrixes obtained in the step five as input of the Hungarian algorithm, and obtaining a linear matching result through the Hungarian algorithm;
step seven, sending the matching result obtained in the step six into a model fusing the twin convolutional neural network and the optimal stacking block for similarity judgment, and detecting and judging the obtained damage;
and step eight, circulating the step five to the step seven until all the frames are processed.
2. The method for intelligently detecting and determining repetitive damage in an aviation hole detection video according to claim 1, wherein in the first step:
step 1.1, detecting a video frame by adopting a target detector to obtain a detection frame;
step 1.2, marking out corresponding damages in all target frames, and performing feature extraction; the characteristics comprise apparent characteristics and motion characteristics;
and 1.3, initializing the motion variables of Kalman filtering, and predicting the corresponding prediction frame through Kalman filtering.
3. The method for intelligently detecting and determining the repetitive damage in the aviation hole detection video according to the claim 1, characterized in that, in the third step:
step 3.1, subtracting the minimum element of each row of the cost matrix;
3.2, generating each column of elements of the matrix and subtracting the minimum element of the column;
3.3, connecting all zeros in the new matrix by using the least row lines and column lines, and checking whether the current distribution is optimal; if the row and column lines do not connect all elements of the matrix, go to step 3.4, otherwise go to step 3.5;
3.4, finding the minimum element in the elements which are not connected with the row lines and the column lines, subtracting the minimum element from the residual elements, and adding the minimum element to the element corresponding to the intersection point of the row lines and the column lines;
step 3.5, finding out 0 elements corresponding to each row and 0 elements corresponding to the columns, and finding out optimal distribution according to the 0 elements;
step 3.6, to incorporate the motion information, the square of the mahalanobis distance between the predicted kalman filter state and the newly arrived measurement is used, i.e. the first metric is denoted d (1):
wherein d is (1) (i, j) represents the motion matching degree between the jth detection frame and the ith track, x i Is the predicted observed quantity of the trajectory at the current time, S i Is a covariance matrix of the observation space at the current time predicted by a Kalman filter, using d j Representing the detected current trajectory state of the target;
step 3.7, using the index of step 3.6, eliminates low probability associations by setting the threshold for mahalanobis distance to 95% of the confidence interval calculated by the inverse chi-square distribution, with an indicator to indicate this decision:
wherein the corresponding March threshold is t (1) When d is greater than d (1) (i, j) is greater than t (1) Then, then
Is set to 0 and otherwise is 1.
When the number of the detection frames is 0, the Markov distance between the ith track frame and the jth detection frame is not matched, and when the number of the detection frames is 1, the Markov distance is matched;
step 3.8, measuring the minimum cosine distance between the ith track frame and the jth detection frame by using a second metric, and using d (2) Represents:
d (2) (i，j)＝min{1-r j T r i (k) |r i (k) ∈R i }
wherein R is i Is a container for storing the feature vector of the track frame of the matching result; r is j Is the feature vector of the jth detection box, r i (k) K feature vectors representing the ith trajectory box;
step 3.9, introduce a binary variable to indicate whether the metric can accept a correlation:
wherein, t (2) Is a threshold value, if d (2) (i, j) is greater than t (2) Then, then
Is set to 0, otherwise is 1;
when the number is 0, the cosine distance between the ith track frame and the jth detection frame is not matched, and when the number is 1, the cosine distance is matched;
step 3.10, to build the correlation problem, two metrics are combined using a weighted sum:
h (i，j) ＝λd (1) (i，j)+(1-λ)d (2) (i，j)
wherein d is (1) (i, j) is defined in step 3.6, d (2) (i, j) is defined in step 3.8, λ represents weight, and the value range is 0-1;
step 3.11, if an association is within the gated region of both metrics, it is said to be an acceptable association:
the results of linear matching are three:
the first is track mismatch, and the mismatched track is directly deleted;
the second is that the prediction result and the detection result are mismatched, and the detection result is initialized to a new track;
and the third is that the detection frame of the next frame is successfully matched with the prediction frame of the previous frame, namely the two adjacent frames are successfully tracked, and at the moment, the corresponding track variable of the detection result of the next frame is updated through Kalman filtering.
4. The intelligent detection and determination method for repetitive damage in an aviation hole detection video according to claim 1, characterized in that in the fifth step:
step 5.1, kalman filtering prediction, namely predicting a value at the next moment according to an estimated value at the previous moment, namely priori estimation, and predicting an error at the next moment, namely a priori error, at the same time;
there are three kinds of cascade matching results:
firstly, track matching, namely updating the corresponding track variable of the track by Kalman filtering;
secondly, track mismatching, namely, carrying out IOU matching on the track in the previous uncertain state and the mismatching track together with a mismatching detection frame one by one, and calculating a cost matrix of the mismatching track through the result of the IOU matching;
and thirdly, mismatching is detected, the track in the previous uncertain state, the mismatching track and the mismatching detection frame are subjected to IOU matching one by one, and then a cost matrix of the mismatching is calculated through the result of the IOU matching.
5. The intelligent detection and determination method for repetitive damage in an aviation hole detection video according to claim 1, characterized in that in the sixth step:
6.1, if the tracks are mismatched, directly deleting the mismatched tracks, otherwise, entering the step 6.2;
step 6.2, if the detection results are mismatched, initializing the detection results into a new track, otherwise, entering step 6.3;
and 6.3, if the detection frame and the predicted frame are successfully matched, namely the previous frame and the next frame are successfully tracked, updating the corresponding track variable of the detection result through Kalman filtering.
6. The intelligent detection and determination method for repetitive damage in an aviation hole detection video according to claim 5, characterized in that in step 6.3:
step 6.3.1, kalman filtering updating, namely firstly calculating Kalman gain, then calculating posterior estimation by using the prior estimation of the previous step, and updating the prior error to obtain a posterior error;
step 6.3.2, performing feature extraction on the damage at the moment, and identifying again; and performing duplicate removal operation on the condition that the same damage occurs in the previous frame and the next frame and different ID values are allocated.
7. The intelligent detection and judgment method for repetitive damage in an aviation hole detection video according to claim 1, characterized in that in step seven:
step 7.1, for two adjacent frames of images, respectively dividing the two adjacent frames of images into a plurality of stacked blocks, wherein the stacked blocks are used as sub-images;
step 7.2, each sub-image is independently processed into a preliminary damage similarity verification decision;
7.3, judging the damage similarity of the two groups of subimages, and outputting a similarity score;
if the score is not greater than the threshold, not the same lesion; if the score is smaller than the threshold value, discarding the irrelevant sub-image with low judgment, and selecting the optimal verification precision of the sub-image with high judgment;
7.4, fusing a twin convolution neural network and the optimal stack block, training to obtain a pre-training model, extracting the characteristics of the optimal selection sub-image, and fusing the characteristics into a single characteristic vector;
7.5, transmitting the feature set formed by the feature vector to a classifier to obtain a final damage similarity verification decision;
and 7.6, obtaining a damage similarity verification result, comparing, and reducing the misjudgment rate of the same damage in the aviation hole detection video.
8. The method as claimed in claim 7, wherein said step 7.4 comprises:
step 7.4.1, adopting a supervised end-to-end deep connected convolutional neural network based on characteristic similarity;
the convolutional neural network is a connected network architecture based on feature similarity, a twin network is used for verifying damage similarity from the whole image through one-time feature extraction, and then single similarity measurement is applied to merge and calculate the similarity between feature vectors so as to judge whether two kinds of damages are the same damage;
and 7.4.2, training by using a twin convolutional neural network and the optimal stacking block to obtain the best training model so as to enhance the judgment of injury similarity.
9. The method as claimed in claim 7, wherein said step 7.5 comprises:
and (3) further processing through a result judged by the classifier to improve the performance of judging the damage similarity:
deleting irrelevant parts from the sub-image blocks, and only keeping the public sub-images with strong correlation as relation evidence of damage similarity so as to minimize the distance between the damaged sub-image pairs, namely the same damage, or maximize the distance between the damaged sub-image pairs, namely the non-same damage;
if the original image is segmented into m blocks, i.e. m sub-images, they form m image pairs with the image to be detected;
calling similarity scores of the previous m image pairs; suppose there are N containers C = (C) 1 ，c 2 ，c 3 ，...，c n ) Each container having a parent level
And subsets
Given P a ∈F p ＝(P a，1 ，P a，2 ，P a，3 ，...，P a，n ) And P b ∈S c ＝(P b，1 ，P b，2 ，P b，3 ，...，P b，n ) In which P is a ∈F p And P b ∈S c Respectively indicate that it belongs to F p And S c M pairs of sub-images in a pair, P a And P b Representing each sub-image;
for each container C, a subpicture P a And P b Is C = (C) 1 (P a -P b )，c 2 (P a -P b )，c 3 (P a -P b )，...，c n (P a -P b ))；
Then, based on the test set T = { e = { e } 1 (c 1 (P a -P b ))，e 2 (c 2 (P a -P b ))，e 3 (c 3 (P a -P b ))，...，e n (c n (P a -P b ) B) in which e n Representing similarity accuracy assessment, c n Denotes a container, P a And P b Representing a sub-image;
using the accuracy e obtained in the previous step n To determine the most similar appearance component between the two lesions;
after obtaining the accuracy score, the impairment component is located and selected according to a threshold specifying similarity, with the sub-image representations that share the most similar cues ignoring the dissimilar parts.
10. The method as claimed in claim 1, wherein the aviation hole probe video is an aviation engine internal hole probe video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210820374.2A CN115147385A (en) | 2022-07-13 | 2022-07-13 | Intelligent detection and judgment method for repeated damage in aviation hole exploration video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210820374.2A CN115147385A (en) | 2022-07-13 | 2022-07-13 | Intelligent detection and judgment method for repeated damage in aviation hole exploration video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115147385A true CN115147385A (en) | 2022-10-04 |
Family
ID=83412847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210820374.2A Pending CN115147385A (en) | 2022-07-13 | 2022-07-13 | Intelligent detection and judgment method for repeated damage in aviation hole exploration video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147385A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117854011A (en) * | 2024-03-07 | 2024-04-09 | 福建南亿智能科技有限公司 | Intelligent AI camera recognition comparison method and system |
-
2022
- 2022-07-13 CN CN202210820374.2A patent/CN115147385A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117854011A (en) * | 2024-03-07 | 2024-04-09 | 福建南亿智能科技有限公司 | Intelligent AI camera recognition comparison method and system |
| CN117854011B (en) * | 2024-03-07 | 2024-05-03 | 福建南亿智能科技有限公司 | Intelligent AI camera recognition comparison method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113012203B (en) | High-precision multi-target tracking method under complex background | |
| CN108492272B (en) | Cardiovascular vulnerable plaque identification method and system based on attention model and multitask neural network | |
| CN108447080B (en) | Target tracking method, system and storage medium based on hierarchical data association and convolutional neural network | |
| CN110516556A (en) | Multi-target tracking detection method, device and storage medium based on Darkflow-DeepSort | |
| CN114972418B (en) | Maneuvering multi-target tracking method based on combination of kernel adaptive filtering and YOLOX detection | |
| CN112184752A (en) | Video target tracking method based on pyramid convolution | |
| CN107424171B (en) | Block-based anti-occlusion target tracking method | |
| CN111161315B (en) | Multi-target tracking method and system based on graph neural network | |
| CN112288773A (en) | Multi-scale human body tracking method and device based on Soft-NMS | |
| CN112884059B (en) | Small sample radar working mode classification method fusing priori knowledge | |
| CN110349188B (en) | Multi-target tracking method, device and storage medium based on TSK fuzzy model | |
| Zhao et al. | Uncertainty-optimized deep learning model for small-scale person re-identification | |
| CN113033555B (en) | Visual SLAM closed loop detection method based on metric learning | |
| CN111429481B (en) | Target tracking method, device and terminal based on adaptive expression | |
| Wu et al. | Deep reinforcement learning with parameterized action space for object detection | |
| CN115147385A (en) | Intelligent detection and judgment method for repeated damage in aviation hole exploration video | |
| CN117576079A (en) | Industrial product surface abnormality detection method, device and system | |
| CN116343080A (en) | Dynamic sparse key frame video target detection method, device and storage medium | |
| CN110472607A (en) | A kind of ship tracking method and system | |
| Zhang et al. | Residual memory inference network for regression tracking with weighted gradient harmonized loss | |
| CN116883457B (en) | Light multi-target tracking method based on detection tracking joint network and mixed density network | |
| CN117291250A (en) | Neural network pruning method for image segmentation | |
| Altundogan et al. | Multiple object tracking with dynamic fuzzy cognitive maps using deep learning | |
| CN116245913A (en) | Multi-target tracking method based on hierarchical context guidance | |
| CN116109649A (en) | 3D point cloud instance segmentation method based on semantic error correction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |