CN114626449A - Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method - Google Patents
Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method Download PDFInfo
- Publication number
- CN114626449A CN114626449A CN202210207915.4A CN202210207915A CN114626449A CN 114626449 A CN114626449 A CN 114626449A CN 202210207915 A CN202210207915 A CN 202210207915A CN 114626449 A CN114626449 A CN 114626449A
- Authority
- CN
- China
- Prior art keywords
- network
- target
- degradation
- weight
- targets
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000006731 degradation reaction Methods 0.000 title claims abstract description 64
- 230000015556 catabolic process Effects 0.000 title claims abstract description 52
- 238000004458 analytical method Methods 0.000 title claims abstract description 11
- 238000012360 testing method Methods 0.000 claims abstract description 68
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000011158 quantitative evaluation Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 4
- 239000011159 matrix material Substances 0.000 claims description 27
- 238000007619 statistical method Methods 0.000 claims description 15
- 238000013211 curve analysis Methods 0.000 claims description 14
- 238000012512 characterization method Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 7
- 230000014509 gene expression Effects 0.000 claims description 6
- 238000000513 principal component analysis Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 2
- 238000011156 evaluation Methods 0.000 claims 2
- 230000003042 antagnostic effect Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000019771 cognition Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
- G06F18/2193—Validation; Performance evaluation; Active pattern learning techniques based on specific statistical tests
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses a target recognition network degradation analysis method under a countermeasure environment based on a least square weight determination method, which comprises the following steps: acquiring an undisturbed target image, and dividing the undisturbed target image into an undisturbed data set; training a target recognition network by using a data set to obtain a pre-training model; respectively testing an undisturbed target image and an interfered target image by utilizing a pre-training model to obtain test results of an interference-free test set and an interference-containing test set model; and processing the test result through a constructed quantitative evaluation system for the target identification network performance degradation, representing the target identification network degradation result, and analyzing the network performance degradation process. The invention can quantitatively and qualitatively analyze the network degradation process under the antagonistic environment from three aspects by constructing a target recognition network performance degradation quantitative evaluation system, and can provide important data support for solving the degradation problem of the target recognition network.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a target identification network degradation analysis method under an antagonistic environment based on a least square weight determination method.
Background
The image target identification network plays an important role in intelligent combat, can identify various targets timely and accurately, and assists in completing accurate attack tasks. However, the various interferences present in a realistic confrontation environment can degrade the performance of the target recognition network, exposing the vulnerability of the recognition network. How to characterize the degradation degree of the target recognition network in the countermeasure environment is a precondition for solving the problem of network degradation. At present, for the use of a target recognition network, generally, engineering experience of a user is relied on, understanding and explanation of an internal working principle are relatively slow, and the method is generally developed from five aspects of visualization of network hidden layer results, feature analysis, defect and optimization, explanation by using a traditional machine learning model and construction of an interpretable module, for example, a feature graph of a network internal hidden layer is visualized by deconvolution, or a class activation method is utilized to focus attention on features which have the greatest influence on a network output result on an input picture, or a decision tree is utilized to simulate a network decision process, and the like. These methods increase the interpretability of the network through post-explanation, but the measure of the degree of network degradation in the countermeasure environment is still an index characterization that needs to be determined. At present, indexes for representing network performance comprise average precision, accuracy, recall rate and the like, but no index can be synthesized.
Disclosure of Invention
The invention aims to: in order to overcome the defects in the prior art, the method for analyzing the degradation of the target identification network in the countermeasure environment based on the least square weight determination method is provided, the weight of each output parameter of the target identification network in the countermeasure environment influenced by the countermeasure environment is determined by mainly utilizing the least square weight determination method combining the subjective and objective weight determination methods, the degradation degree of the target identification network is represented, a quantitative evaluation system of the performance degradation of the target identification network in the countermeasure environment is constructed by combining an ROC curve analysis method and a probability distribution statistical method, and the degradation process of the network performance is analyzed.
The technical scheme is as follows: in order to achieve the above object, the present invention provides a target identification network degradation analysis method under an antagonistic environment based on a least square weight determination method, comprising the following steps:
s1: acquiring an undisturbed target image, and dividing the undisturbed target image into an undisturbed data set, wherein the undisturbed data set comprises a training set, a verification set and an undisturbed test set;
obtaining a target image added with interference, and making an interference test set;
s2: training a target recognition network by utilizing a training set and a verification set to obtain a pre-training model;
s3: respectively testing an undisturbed target image and an interfered target image by using the pre-training model through an interference-free test set and an interference test set to obtain test results of the interference-free test set and the interference test set model;
s4: and (4) processing the test result of the step S3 through a constructed quantitative evaluation system for the target identification network performance degradation, representing the target identification network degradation result, and analyzing the network performance degradation process.
Further, the target recognition network in the step S2 includes fast R-CNN and YOLOv5 recognition networks, and the pre-training models are a fast R-CNN training model and a YOLOv5 training model, respectively.
Further, the test results of the non-interference test set and the interference test set in the step S3 include average precision of the mean, accuracy, recall rate, F1 score, and coincidence degree of the prediction box and the real box.
Further, the expression of the model test result is as follows:
mean average precision (mAP):
accuracy (precision):
recall (recall):
f1 score:
degree of coincidence (IOU) of prediction box and real box:
where i represents the ith image, j represents a certain object, PjIs the average precision of a certain class of objects, which is the average of the precisions of N samples containing such objects,
the number of the real targets is the number of the targets,
meaning the number of targets that can be detected, M represents the number of types of targets, TP, FP, FN represent the number of targets that are detected as positive and correctly, the number of targets that can be detected as positive and incorrectly, the number of targets that are detected as negative and actually correctly, Rdet represents the bounding box detected by the network, Rgt represents the area where the actual target is located, the condition that Rdet is correctly detected is IOU > 0.5, and the target labels match.
Further, the quantitative evaluation system for the performance degradation of the target identification network in the step S4 includes a least square weight determination method portion, an ROC curve analysis method portion, and a probability distribution statistical method portion.
And determining the weight of each parameter of the test results of the interference-free test set and the interference test set model by a least square weight determination method, and representing the different degradation degrees of different parameters output by the target recognition network in the confrontation environment. The specific characterization method comprises the following steps:
improved hierarchy method for determining weight mu1n: firstly, a hierarchical structure model is established according to decision-making purposes and criteriaThe scheme is divided into three layers of high, middle and low, and then a judgment matrix is constructed, wherein the element a of the matrixijGiven by a 1-9 scale method of Saaty, after each factor is compared pairwise, the maximum characteristic root lambda of a judgment matrix is determinedmaxNormalizing and recording as weight W, performing hierarchical single ordering (the elements in W are ordered according to the relative importance of the same layer element to the previous layer element), performing consistency check, performing hierarchical total ordering and consistency check (i.e. calculating the weight of all the factors of a certain layer relative to the highest layer), and calculating the weight of the ith factor of the lowest layer to the highest layer
Wherein, ajIs the weight ordering of the middle layer to the highest layer, bijIs the ith factor of the lowest layer to the middle layer ajThe weight value hierarchy of (1) is ordered;
entropy weight method for determining weight v1n: first, the proportion of the j index of the i object is calculated
Then, the information entropy of the j index is calculated
Wherein, K is a constant,
finally, the weight of the jth index is calculated,
determination of weight v by principal component analysis2n: obtaining and transferring a normalized network performance index matrix A, solving a correlation coefficient matrix for the matrix A by using two columns, then solving an eigenvalue lambda and a corresponding eigenvector of the correlation coefficient matrix according to the eigenvalue lambda and the corresponding eigenvector
P is the total number of eigenvalues, m is the number of eigenvalues conforming to the inequalityDetermining the number of principal components according to m, standardizing the eigenvectors corresponding to the eigenvalues, converting the eigenvectors into principal component expressions, and finally performing weighted summation on the m principal components to obtain a weight vector v2n；
Constructing an objective function:
the result of the combined weight-determining is:
when the number mu is 2/3, the number mu is,
in time, there are:
identifying a comprehensive performance characterization formula after the network multi-index weight determination:
performance＝ω1·precision+ω2·recall+ω3·F1+ω4·mAP。
further, the ROC curve analysis method comprises the following steps: respectively drawing a True Positive Rate (TPR) graph and a False Positive Rate (FPR) graph of the non-interference test set and the interference test set under different probability thresholds,
TP indicates the number of samples with positive true value and positive predicted value, FN indicates the number of samples with positive true value and negative predicted value, FP indicates negative true value and negative predicted valueThe number of samples with positive values, TN, the number of samples with negative values for true values and negative values for predicted values.
Further, the probability distribution statistical method comprises the following steps: the output probabilities of the undisturbed and disturbed images containing a certain class target classified into various class targets after passing through the network are respectively recorded as P ═ { P ═ P1,p2,…,pnP'1,p'2,…,p'nAnd measuring the degradation condition delta p of the network performance as p by adopting the change of the correct detection probability of the identification network for the target imagej-p'j。
The least square weight determination method can represent the degradation phenomenon of an identification network from the angle of network identification precision, the ROC curve analysis rule can measure the stability and the openness degree of the identification network, the probability distribution statistical method can represent the performance of the identification network facing different types of targets, and a target identification network performance degradation quantitative evaluation system combines the three methods and analyzes the degradation process of the target identification network under the antagonistic environment in a more comprehensive mode from three angles.
Has the advantages that: compared with the prior art, the method integrates the subjective weight determination method and the objective weight determination method through the least square weight determination method, introduces human cognition and engineering experience, considers objective facts, and achieves dynamic weight assignment, so that different degradation degrees of target identification network output parameters are determined; meanwhile, the stability and openness degree of the identification network can be measured by combining an ROC curve analysis method; the probability distribution statistical method is combined, the performance condition of the recognition network facing different kinds of targets can be represented, the whole target recognition network degradation analysis framework integrates three methods, the degradation process of the network under the antagonistic environment can be quantitatively and qualitatively analyzed from three aspects, and important data support can be provided for solving the degradation problem of the target recognition network.
Drawings
FIG. 1 is an overall flow diagram of the process of the present invention;
FIG. 2 is a schematic diagram of a portion of data sets without interference and corresponding data sets with interference in the present embodiment;
FIG. 3 is a ROC curve variation diagram of an undisturbed SAR remote sensing ground vehicle image;
FIG. 4 is a ROC curve variation diagram of an SAR remote sensing ground vehicle image with interference added.
Detailed Description
The present invention is further illustrated by the following figures and specific examples, which are to be understood as illustrative only and not as limiting the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalent modifications thereof which may occur to those skilled in the art upon reading the present specification.
The invention provides a target identification network degradation analysis method under a countermeasure environment based on a least square weight determination method, as shown in figure 1, the method comprises the following steps:
s1: obtaining a target image without interference, and dividing the target image into an interference-free data set, wherein the interference-free data set comprises a training set, a verification set and an interference-free test set, and the ratio of the training set to the verification set to the interference-free test set is 8:1: 1;
obtaining a target image added with interference, and making an interference test set;
s2: respectively training two target recognition networks of Faster R-CNN and Yolov5 by using a training set and a verification set to obtain pre-training models, namely a Faster R-CNN training model and a Yolov5 training model;
s3: respectively testing an undisturbed target image and an interfered target image by using a Faster R-CNN training model and a YOLOv5 training model through an interference-free testing set and an interference testing set to obtain testing results of the interference-free testing set and the interference testing set;
the test results of the interference-free test set and the interference test set model are as follows:
mean average precision (mAP):
accuracy (precision):
recall (recall):
f1 score:
degree of coincidence (IOU) of prediction box and real box:
where i represents the ith image, j represents a certain object, PjIs the average precision of a certain class of objects, which is the average of the precisions of N samples containing such objects,
the number of the real targets is the number of the targets,
meaning the number of targets that can be detected, M represents the number of types of targets, TP, FP, FN represent the number of targets that are detected as positive and correctly, the number of targets that can be detected as positive and incorrectly, the number of targets that are detected as negative and actually correctly, Rdet represents the bounding box detected by the network, Rgt represents the area where the actual target is located, the condition that Rdet is correctly detected is IOU > 0.5, and the target labels match.
S4: and (4) processing the test result of the step S3 through a constructed quantitative evaluation system for the target identification network performance degradation, representing the target identification network degradation result, and analyzing the network performance degradation process.
The target identification network performance degradation quantitative evaluation system comprises a least square weight determination method part, an ROC curve analysis method part and a probability distribution statistical method part.
And determining the weight of each parameter of the test results of the interference-free test set and the interference test set model by a least square weight determination method, and representing the different degradation degrees of different parameters output by the target recognition network in the confrontation environment. The specific characterization method comprises the following steps:
improved hierarchy method for determining weight mu1n: firstly, establishing a hierarchical structure model, dividing the model into three layers of high, medium and low according to decision-making purposes, criteria and schemes, and then constructing a judgment matrix, wherein elements a of the matrixijGiven by a 1-9 scale method of Saaty, comparing every two factors, and judging the maximum characteristic root lambda of the matrixmaxNormalizing and recording as weight W, performing hierarchical single ordering (the elements in W are ordered according to the relative importance of the same layer element to the previous layer element), performing consistency check, performing hierarchical total ordering and consistency check (i.e. calculating the weight of all the factors of a certain layer relative to the highest layer), and calculating the weight of the ith factor of the lowest layer to the highest layer
Wherein, ajIs the weight ordering of the middle layer to the highest layer, bijIs the ith factor of the lowest layer to the middle layer ajThe weight value hierarchy of (2) is sorted;
entropy weight method for determining weight v1n: first, the proportion of the jth index of the ith object is calculated
Then, the information entropy of the j index is calculated
Wherein, K is a constant,
finally, the weight of the jth index is calculated,
determination of weight v by principal component analysis2n: obtaining and transferring a normalized network performance index matrix A, solving a correlation coefficient matrix for the matrix A by using two columns, and then solving a characteristic value lambda and a corresponding characteristic value lambda of the correlation coefficient matrixIs according to
P is the total number of the characteristic values, m is the number of the characteristic values conforming to the inequality, the number of the principal components is determined according to m, the characteristic vectors corresponding to the characteristic values are standardized and converted into principal component expressions, and finally the m principal components are subjected to weighted summation to obtain a weight vector v2n；
Constructing an objective function:
the result of the combined weight determination is:
when the value of mu is 2/3,
in time, there are:
identifying a comprehensive performance characterization formula after the network multi-index weight determination:
performance＝ω1·precision+ω2·recall+ω3·F1+ω4·mAP。
the ROC curve analysis method comprises the following steps: respectively drawing a True Positive Rate (TPR) graph and a False Positive Rate (FPR) graph of the non-interference test set and the interference test set under different probability thresholds,
TP means the number of samples whose true value is positive and whose predicted value is positive, FN means the number of samples whose true value is positive and whose predicted value is negative, FP means the number of samples whose true value is negative and whose predicted value is positive, TN means the number of samples whose true value is negative and whose predicted value is negative.
The probability distribution statistical method comprises the following steps: the output probabilities of the undisturbed and disturbed images containing a certain class target classified into various class targets after passing through the network are respectively recorded as P ═ { P ═ P1,p2,…,pnP'1,p'2,…,p'nAnd measuring the degradation condition delta p of the network performance as p by adopting the change of the correct detection probability of the identification network for the target imagej-p'j。
The degradation phenomena of two target identification networks, namely FasterR-CNN and Yolov5, can be characterized from the angle of network identification precision through a least square weight determination method, the stability and the openness degree of the two target identification networks, namely FasterR-CNN and Yolov5, can be measured through an ROC curve analysis method, the performance conditions of the two target identification networks, namely FasterR-CNN and Yolov5, facing different kinds of targets can be characterized through a probability distribution statistical method, a quantitative evaluation system for the performance degradation of the target identification network combines the three methods, and the degradation processes of the two target identification networks, namely FasterR-CNN and Yolov5, in an antagonistic environment are comprehensively analyzed from three angles.
Based on the above scheme, in order to verify the effect of the scheme of the present invention, the embodiment applies the scheme of the present invention as an example, specifically as follows:
in this embodiment, the target identification network degradation analysis method provided by the present invention is applied to radar sea surface ship images, SAR remote sensing ground vehicle images, and infrared remote sensing aircraft target images at the same time, referring to fig. 2, the specific process is as follows:
step 1: obtaining an undisturbed radar sea surface ship image, an SAR remote sensing ground vehicle image and an infrared remote sensing airplane target image, and making 3 data sets, wherein each data set is internally provided with a training set, a verification set and a test set, and the ratio of the training set to the verification set to the test set is 8:1: 1;
step 2: acquiring radar sea surface ship images added with the suppression interference, SAR remote sensing ground vehicle images added with the deception interference and infrared remote sensing airplane target images added with the laser interference, and respectively making 3 test sets;
and step 3: respectively training two target recognition networks of Faster R-CNN and YOLOv5 by using the 3 data sets obtained in the step 1 to respectively obtain 6 training models;
and 4, step 4: respectively testing radar sea surface ship images, SAR remote sensing ground vehicle images and infrared remote sensing airplane target images without interference by using the 6 training models obtained in the step 3 to obtain test results;
mean average precision (mAP):
accuracy (precision):
recall (recall):
f1 score:
intersection ratio (IOU) index between prediction box and real box:
where i represents the ith image, j represents a certain object, PjIs the average precision of a certain class of objects, which is the average of the precisions of N samples containing such objects,
the number of the real targets is the number of the targets,
meaning the number of targets that can be detected, M represents the number of types of targets, TP, FP, FN represent the number of targets that are detected as positive and correctly, the number of targets that can be detected as positive and incorrectly, the number of targets that are detected as negative and actually correctly, Rdet represents a bounding box detected by the network, Rgt represents the area where the actual targets are located, the condition that Rdet is correctly detected is IOU > 0.5, and the target labels match;
and 5: respectively testing the three test sets obtained in the step 2 by using the 6 training models obtained in the step 3 to obtain test results, wherein the parameter types contained in the results are the same as those in the step 4;
step 6: constructing a performance characterization method comprising a least square weight determination method, an ROC curve analysis method and a probability distribution statistical method 3, orienting to a performance degradation quantitative evaluation system of FasterR-CNN and Yolov5 two target identification network models, characterizing a target image intelligent identification network degradation result, and analyzing a network performance degradation process:
improved hierarchy method for determining weight mu1n: firstly, establishing a hierarchical structure model, dividing the model into three layers of high, medium and low according to decision-making purposes, criteria and schemes, and then constructing a judgment matrix, wherein elements a of the matrixijGiven by a 1-9 scale method of Saaty, comparing every two factors, and judging the maximum characteristic root lambda of the matrixmaxNormalizing and recording as weight W, performing hierarchical single ordering (the elements in W are ordered according to the relative importance of the same layer element to the previous layer element), performing consistency check, performing hierarchical total ordering and consistency check (i.e. calculating the weight of all the factors of a certain layer relative to the highest layer), and calculating the weight of the ith factor of the lowest layer to the highest layer
Wherein, ajIs the weight ordering of the middle layer to the highest layer, bijIs the ith factor of the lowest layer to the middle layer ajThe weight value hierarchy of (1) is ordered;
entropy weight method for determining weight v1n: first, the proportion of the j index of the i object is calculated
Then, the information entropy of the j index is calculated
Wherein, K is a constant,
finally, the weight of the jth index is calculated,
determination of weight v by principal component analysis2n: obtaining and transferring a normalized network performance index matrix A, solving a correlation coefficient matrix for the matrix A by using two columns, then solving an eigenvalue lambda and a corresponding eigenvector of the correlation coefficient matrix according to the eigenvalue lambda and the corresponding eigenvector
P is the total number of the characteristic values, m is the number of the characteristic values conforming to the inequality, the number of the principal components is determined according to m, the characteristic vectors corresponding to the characteristic values are standardized and converted into principal component expressions, and finally the m principal components are subjected to weighted summation to obtain a weight vector v2n；
Constructing an objective function:
the result of the combined weight-determining is:
when the value of mu is 2/3,
in time, there are:
identifying a comprehensive performance characterization formula after the network multi-index weight determination:
performance＝ω1·precision+ω2·recall+ω3·F1+ω4·mAP
and then combining an ROC curve analysis method and a probability distribution statistical method to analyze the degradation process of the target recognition network under the countermeasure environment in multiple angles: respectively drawing a True Positive Rate (TPR) graph and a False Positive Rate (FPR) graph of the non-interference test set and the interference test set under different probability thresholds,
TP means the number of samples with positive true value and positive predicted value, FN means the number of samples with positive true value and negative predicted value, FP means the number of samples with negative true value and positive predicted value, TN means the number of samples with negative true value and negative predicted value. The output probabilities of the undisturbed and disturbed images containing a certain class target classified into various class targets after passing through the network are respectively recorded as P ═ { P ═ P1,p2,…,pnP } and P ═ P'1,p'2,…,p'nAnd measuring the degradation condition delta p of the network performance as p by adopting the change of the correct detection probability of the identification network for the target imagej-p'j。
The experimental data of the least square weight determination method, the ROC curve analysis method, and the probability distribution statistical method finally obtained in this embodiment are as follows:
the weight determination index obtained by the least square weight determination method has the calculation formula as follows:
performance＝0.2589·percision+0.2868·recall+0.2350·F1+0.2193·map
the change of the ROC curve under the influence of different degrees of interference is shown in FIGS. 3 and 4:
by comparing the two graphs and combining the definition of the ROC curve, the stability of the network is reduced for the image after the interference is added.
The probability distribution statistical method analyzes one frame of image sample as follows:
the probability that the targets in the undisturbed image are identified as various types of targets 2S1, BRDM _2, BTR _60, D7, SN _132, SN _9563, SN _ C71, T62, ZIL131, ZSU _23_4 in the data set is P ═ 0.9932, 0.3762, 0.0258, 0.0000, 0.0000, 0.0148, 0.0000, 0.0401, 0.0483, 0.0100}, respectively; the probabilities that the objects in the added interference image are identified as various objects 2S1, BRDM _2, BTR _60, D7, SN _132, SN _9563, SN _ C71, T62, ZIL131, and ZSU _23_4 in the data set are P' ═ 0.0463, 0.08357, 0.4578, 0.0000, 0.0000, 0.0000, 0.0000, 0.0467, and 0.0000, respectively, and the probability change condition that the identified objects are correct before and after the added interference is calculated: 0.9932-0.0463 ═ 0.9469. By counting the change conditions of the recognition rate of other images in the test set, the degradation of the network recognition classification performance in the antagonistic environment can be reflected.
Therefore, the target identification network degradation analysis method under the countermeasure environment based on the least square weight determination method provided by the invention integrates the main weight determination method and the objective weight determination method, for example, an improved hierarchy method is adopted in the main weight determination method, an entropy weight method and a principal component analysis weight determination method are adopted in the objective weight determination method, the human cognition and the engineering experience are introduced, the objective fact is considered, the dynamic weight assignment is realized, and the degradation phenomenon of an identification network is represented from the angle of network identification precision. Meanwhile, the method is combined with an ROC curve analysis method for measuring the stability and openness of an identification network and a probability distribution statistical method for representing the performance condition of the identification network facing different kinds of targets. The whole framework integrates the three target recognition network performance characterization methods, and analyzes the network degradation process in the countermeasure environment in a relatively comprehensive mode.
Claims (9)
1. A target identification network degradation analysis method under a confrontation environment based on a least square weight determination method is characterized by comprising the following steps:
s1: acquiring 2000 frames of undisturbed target images, and dividing the undisturbed target images into an undisturbed data set, wherein the undisturbed data set comprises a training set, a verification set and an undisturbed test set;
obtaining a target image added with interference, and making an interference test set;
s2: training a target recognition network by utilizing a training set and a verification set to obtain a pre-training model;
s3: respectively testing an undisturbed target image and an interfered target image by utilizing a pre-training model through an interference-free test set and an interference test set to obtain test results of the interference-free test set and the interference test set model;
s4: and (4) processing the test result of the step S3 through a constructed quantitative evaluation system for the target identification network performance degradation, representing the target identification network degradation result, and analyzing the network performance degradation process.
2. The method as claimed in claim 1, wherein the target recognition network in the step S2 includes fast R-CNN and YOLOv5 recognition networks, and the pre-training models are Faster R-CNN training model and YOLOv5 training model, respectively.
3. The method for analyzing the degradation of the target recognition network under the countermeasure environment based on the least square weight-determining method of claim 1, wherein the test results of the non-interference test set and the interference test set in the step S3 each include average accuracy, recall, F1 score, and coincidence degree of the prediction box and the real box.
4. The method for analyzing the degradation of the target recognition network in the confrontation environment based on the least square weight determination method as claimed in claim 3, wherein the expression of the model test result is as follows:
mean average precision (mAP):
accuracy (precision):
recall (recall):
f1 fraction:
degree of coincidence (IOU) of prediction box and real box:
where i represents the ith image, j represents a certain object, PjIs the average precision of a certain class of objects, which is the average of the precisions of N samples containing such objects,
the number of the targets is the real number of the targets,
meaning the number of targets that can be detected, M represents the number of types of targets, TP, FP, FN represent the number of targets that are detected as positive and correctly, the number of targets that can be detected as positive and incorrectly, and the number of targets that are detected as negative and actually correctly, Rdet represents the bounding box detected by the network, and Rgt represents the area where the actual target is located.
5. The method for analyzing the degradation of the target identification network under the countermeasure environment based on the least square weight determination method of claim 1, wherein the quantitative evaluation system for the performance degradation of the target identification network in the step S4 comprises a least square weight determination method part, a ROC curve analysis method part and a probability distribution statistical method part.
6. The method as claimed in claim 5, wherein in step S4, the least square weight-determining method is used to partially determine the weights of the parameters of the test results obtained from the non-interference test set and the interference test set, so as to characterize the degradation degree of the different parameters output by the target recognition network in the countermeasure environment.
7. The method for analyzing the degradation of the target recognition network in the countermeasure environment based on the least square weight determination method as claimed in claim 6, wherein the specific characterization method of the least square weight determination method in the step S4 is:
improved hierarchy method for determining weight mu1n: firstly, establishing a hierarchical structure model, dividing the model into three layers of high, medium and low according to decision-making purposes, criteria and schemes, and then constructing a judgment matrix, wherein elements a of the matrixijGiven by a 1-9 scale method of Saaty, comparing every two factors, and judging the maximum characteristic root lambda of the matrixmaxNormalizing and recording as weight W, performing hierarchical single ordering, wherein the elements in W are ordered according to the relative importance of the same layer element to the previous layer element, performing consistency check, and finally performing hierarchical total ordering and consistency check, namely calculating the weight of all the factors of a certain layer relative importance to the highest layer, and calculating the weight of the ith factor of the lowest layer to the highest layer
Wherein, ajIs the weight ordering of the middle layer to the highest layer, bijIs the ith factor of the lowest layer to the middle layer ajThe weight value hierarchy of (1) is ordered;
entropy weight method for determining weight v1n: first, the proportion of the j index of the i object is calculated
Then, the information entropy of the j index is calculated
Wherein, K is a constant,
finally, the weight of the jth index is calculated,
determination of weight v by principal component analysis2n: obtaining and transferring a normalized network performance index matrix A, solving a correlation coefficient matrix for the matrix A by using two columns, then solving an eigenvalue lambda and a corresponding eigenvector of the correlation coefficient matrix according to the eigenvalue lambda and the corresponding eigenvector
P is the total number of the characteristic values, m is the number of the characteristic values conforming to the inequality, the number of the principal components is determined according to m, the characteristic vectors corresponding to the characteristic values are standardized and converted into principal component expressions, and finally the m principal components are subjected to weighted summation to obtain a weight vector v2n；
Constructing an objective function:
the result of the combined weight determination is:
when the value of mu is 2/3,
in time, there are:
identifying a comprehensive performance characterization formula after the network multi-index weight determination:
performance＝ω1·precision+ω2·recall+ω3·F1+ω4·mAP。
8. the method for analyzing the degradation of the target identification network in the confrontation environment based on the least square weight-determining method as claimed in claim 5, wherein the ROC curve analysis section in the step S4 is used to realize the evaluation of the stability and openness of the target identification network, and specifically comprises: plotting a graph of true positive rate versus false positive rate at different probability thresholds,
TP means the number of samples whose true value is positive and whose predicted value is positive, FN means the number of samples whose true value is positive and whose predicted value is negative, FP means the number of samples whose true value is negative and whose predicted value is positive, TN means the number of samples whose true value is negative and whose predicted value is negative.
9. The method for analyzing degradation of a target recognition network under a confrontation environment based on a least square weight-determining method as claimed in claim 5, wherein the probability distribution statistical method in the step S4 is used to realize the evaluation of the classification effect of the target recognition network, and specifically comprises: if the targets in each image i are respectively marked as a class label CjWhen the network is pairedWhen image class is predicted, j label classes c allowing model to returni1,…cijIf there is cij＝CjIf the model prediction is correct, otherwise, if the model prediction is wrong, the output probabilities of the undisturbed and disturbed images containing a certain class of targets classified into various class targets after passing through the network are respectively recorded as P ═ { P ═ P1,p2,…,pnP'1,p'2,…,p'nAnd measuring the degradation condition delta p of the network performance as p by adopting the change of the correct detection probability of the identification network for the target imagej-p'j。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210207915.4A CN114626449A (en) | 2022-03-03 | 2022-03-03 | Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210207915.4A CN114626449A (en) | 2022-03-03 | 2022-03-03 | Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114626449A true CN114626449A (en) | 2022-06-14 |
Family
ID=81899400
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210207915.4A Pending CN114626449A (en) | 2022-03-03 | 2022-03-03 | Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114626449A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130138428A1 (en) * | 2010-01-07 | 2013-05-30 | The Trustees Of The Stevens Institute Of Technology | Systems and methods for automatically detecting deception in human communications expressed in digital form |
| CN106355030A (en) * | 2016-09-20 | 2017-01-25 | 浙江大学 | Fault detection method based on analytic hierarchy process and weighted vote decision fusion |
| CN108430069A (en) * | 2018-02-11 | 2018-08-21 | 重庆邮电大学 | A kind of V2X applied in network performance test and comprehensive evaluation analysis method |
| CN110288005A (en) * | 2019-05-30 | 2019-09-27 | 南京理工大学 | Millimeter wave detector New Jamming Effects Evaluation Method based on SVM |
| CN111241933A (en) * | 2019-12-30 | 2020-06-05 | 南京航空航天大学 | Pig farm target identification method based on universal countermeasure disturbance |
| CN111582571A (en) * | 2020-04-30 | 2020-08-25 | 中国电力科学研究院有限公司 | Power grid operation situation sensing method and system with model driving and data driving integrated |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
| CN112487694A (en) * | 2020-11-29 | 2021-03-12 | 西北工业大学 | Complex equipment residual life prediction method based on multiple degradation indexes |
| CN112634237A (en) * | 2020-12-25 | 2021-04-09 | 福州大学 | Long bamboo strip surface defect detection method and system based on YOLOv3 improved network |
-
2022
- 2022-03-03 CN CN202210207915.4A patent/CN114626449A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130138428A1 (en) * | 2010-01-07 | 2013-05-30 | The Trustees Of The Stevens Institute Of Technology | Systems and methods for automatically detecting deception in human communications expressed in digital form |
| CN106355030A (en) * | 2016-09-20 | 2017-01-25 | 浙江大学 | Fault detection method based on analytic hierarchy process and weighted vote decision fusion |
| CN108430069A (en) * | 2018-02-11 | 2018-08-21 | 重庆邮电大学 | A kind of V2X applied in network performance test and comprehensive evaluation analysis method |
| CN110288005A (en) * | 2019-05-30 | 2019-09-27 | 南京理工大学 | Millimeter wave detector New Jamming Effects Evaluation Method based on SVM |
| CN111241933A (en) * | 2019-12-30 | 2020-06-05 | 南京航空航天大学 | Pig farm target identification method based on universal countermeasure disturbance |
| CN111582571A (en) * | 2020-04-30 | 2020-08-25 | 中国电力科学研究院有限公司 | Power grid operation situation sensing method and system with model driving and data driving integrated |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
| CN112487694A (en) * | 2020-11-29 | 2021-03-12 | 西北工业大学 | Complex equipment residual life prediction method based on multiple degradation indexes |
| CN112634237A (en) * | 2020-12-25 | 2021-04-09 | 福州大学 | Long bamboo strip surface defect detection method and system based on YOLOv3 improved network |
Non-Patent Citations (2)
| Title |
|---|
| 周鹏: "高分辨率光学遥感卫星效能评估与参数 敏感性分析研究", 《中国优秀硕士学位论文全文数据库》 * |
| 顾兴建: "多尺度Ｕ网络实现番茄叶部病斑分割与识别", 《计算机科学》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113705478B (en) | Mangrove single wood target detection method based on improved YOLOv5 | |
| CN109284786B (en) | SAR image terrain classification method for generating countermeasure network based on distribution and structure matching | |
| CN112101426A (en) | Unsupervised learning image anomaly detection method based on self-encoder | |
| CN110018453B (en) | Intelligent model identification method based on aircraft track characteristics | |
| CN111191629B (en) | Image visibility detection method based on multiple targets | |
| CN111507370A (en) | Method and device for obtaining sample image of inspection label in automatic labeling image | |
| CN108492298A (en) | Based on the multispectral image change detecting method for generating confrontation network | |
| CN109766926B (en) | Radar radiation source signal intra-pulse characteristic comprehensive evaluation method and system | |
| CN114509266A (en) | Bearing health monitoring method based on fault feature fusion | |
| CN111582387A (en) | Rock spectral feature fusion classification method and system | |
| CN112668613A (en) | Satellite infrared imaging effect prediction method based on weather forecast and machine learning | |
| US20040096107A1 (en) | Method and computer program product for determining an efficient feature set and an optimal threshold confidence value for a pattern recogniton classifier | |
| Chou et al. | SHM data anomaly classification using machine learning strategies: A comparative study | |
| CN113343225B (en) | Poisoning defense method and device based on deep learning of neural pathway | |
| CN115935187B (en) | Nuclear sensitivity alignment network-based mechanical fault diagnosis method under variable working conditions | |
| CN114626449A (en) | Target recognition network degradation analysis method under countermeasure environment based on least square weight determination method | |
| Lin et al. | Optimization of a multi-stage ATR system for small target identification | |
| CN113553708B (en) | Method and device for tracing key influence factors of simulation model | |
| CN114821322A (en) | Small sample remote sensing image classification method and system based on attention mechanism | |
| CN113673105A (en) | Design method of true value comparison strategy | |
| CN111985826A (en) | Visual quality grading method and system for multi-index industrial product | |
| CN113534129B (en) | Method and system for evaluating high-speed target detection performance of foundation broadband radar | |
| Zhang et al. | Improved Sparse Representation of Open Set HRRP Recognition Method | |
| JP2021108031A (en) | Model generation device, vehicle simulation system, model generation method, vehicle simulation method and computer program | |
| CN111914898B (en) | Adaptive threshold-based airborne SAR task adaptability evaluation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220614 |