CN111950635B - Robust feature learning method based on layered feature alignment - Google Patents
Robust feature learning method based on layered feature alignment Download PDFInfo
- Publication number
- CN111950635B CN111950635B CN202010809932.6A CN202010809932A CN111950635B CN 111950635 B CN111950635 B CN 111950635B CN 202010809932 A CN202010809932 A CN 202010809932A CN 111950635 B CN111950635 B CN 111950635B
- Authority
- CN
- China
- Prior art keywords
- features
- feature
- sample
- neural network
- different
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a robust feature learning method based on layered feature alignment, which comprises the following steps: using a deep convolutional neural network to conduct hierarchical extraction of depth features from input samples in different fields; for the extracted layered features, the channel and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns more abundant feature representations; using Wasserstein distance based on optimal transmission theory to accurately measure the difference between sample characteristic representations in different fields; differences between layered features extracted from different domain samples are used as part of a model loss function to help the model learn more robust features, thereby improving the robustness against deep neural network models. According to the technical scheme, the depth network model can learn robust features, damage to an attack resisting method is avoided, and therefore a safe and reliable depth system is obtained.
Description
Technical Field
The invention relates to the technical field of robust machine learning, in particular to a robust feature learning method based on layered feature alignment.
Background
In recent years, deep convolutional neural networks have made breakthroughs in many computer vision tasks such as image classification, object detection, and the like. However, researchers have found that these deep convolutional neural networks are susceptible to spoofing by specially designed anti-disturbance samples that are not readily perceived by the human eye. These challenge samples generated by challenge attack methods pose serious challenges for systems with high requirements for safety and stability, including autopilot systems, medical diagnostic systems, security systems, and the like. In addition, if a deep network model changes its prediction with high confidence given a sample with a small amount of disturbance as input, it can be determined that these models do not learn well from the input of the sample from scratch to the task related inherent properties, nor from the sample to the robust visual concept. Thus, designing a deep network model that is sufficiently robust to combat disturbances is critical for safe and reliable computer vision applications.
In recent years of research work, researchers have proposed a variety of challenge defense mechanisms to overcome different challenge approaches. These defense mechanisms can be roughly divided into two categories. The first category of defense approaches mainly uses various pre-treatments on the input image to overcome the challenge. Dziugaite et al and Das et al compress JPEG images as an countermeasure against defenses. These methods use discrete fourier transforms in the field of input images to handle noise immunity. However, these methods based on JPEG image compression have not achieved the objective of successfully removing the noise countermeasure. By fully utilizing the strong representation capability of the generated countermeasure network, the method of the Defence-GAN is proposed by Samantouue et al to defend various countermeasure attacks; the method achieves the objective of removing noise by regenerating an image sufficiently similar to the input image. Mustafa et al propose to use image super-resolution as a means of countermeasures, and by using a deep super-resolution network as a mapping function, the method maps samples from the countermeasures to the normal fields, thereby achieving the purpose of removing countermeasures noise, and finally inputting the mapped images into an image recognition system for normal recognition. Another approach to combat defense is to improve the robustness of the model by mainly modifying the training process or the network structure to handle the combat disturbances. Challenge training is an effective means of improving the robustness of a model by adding specifically designed challenge samples to the training data. Goodfellow et al train a network model by adding to the clean samples challenge samples generated using the FGSM (Fast Gradient Sign Method ) challenge method. Madry et al used Min-Max optimization to conduct challenge training, which used PGD (Project Gradient Descent, map gradient descent) attack methods to generate challenge samples. Integrated challenge training is also a novel challenge defense method that uses challenge samples generated from a variety of different depth networks as training data to optimize model parameters. In addition, song et al used a domain-adaptive approach to training a network model in order to improve the generalization ability of the depth model to the challenge samples.
Although the above-described methods have made good progress in improving the robustness against deep convolutional neural networks, they often fail to achieve satisfactory results for different kinds of white-box attack methods, which are limited by the poor generalization performance of the model.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention aims to provide a robust feature learning method based on layered feature alignment, which enables a model based on a deep convolutional neural network to learn more robust image features through layered feature alignment operation, thereby solving the problem of limited generalization capability of an anti-sample model in different fields in the prior art and providing effective reliability and safety guarantee for deployment and application of a deep model system.
In order to achieve the above purpose, the present invention provides the following technical solutions: a robust feature learning method based on hierarchical feature alignment comprises the following steps:
(1) Lifting depth features of different layers from samples in different fields by using a depth convolution neural network;
(2) For the extracted layered features, the channel and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns more abundant feature representations;
(3) Using Wasserstein distance based on optimal transmission theory to accurately measure the difference between sample characteristic representations in different fields;
(4) Differences between layered features extracted from different domain samples are used as part of a model loss function to help the model learn more robust features, thereby improving the robustness against deep neural network models.
Preferably, the image samples of different fields include a normal field image sample and an countermeasure field image sample.
Preferably, in the step (1), the ResNet-110 network structure is used for extracting the characteristics of the image, the ResNet-110 network structure is divided into 4 different structural layers, and after a normal sample or an countermeasure sample is input, the convolution structure is used for extracting the image characteristics with different scales and different abstract degrees at the 4 different structural layers when the forward reasoning is carried out on the network.
Preferably, in step (2), a graph convolution operation is performed using two one-dimensional convolutions, the graph convolution operation being formulated as follows:
formula (1):
GCN(f)＝Conv1D[Conv1D(f)]
in the formula (1), GCN (not shown) represents a graph convolution neural network, f represents a feature vector subjected to dimension reduction processing, and f represents input of graph convolution operation; in addition, conv1D (the) represents a one-dimensional convolution operation, feature extraction is performed using two different-direction one-dimensional graph convolution operations, which enhance the representation of relationships between different regions in the feature after sufficient end-to-end training.
Preferably, in the step (3), the characteristic at a certain layer extracted from the sample in the normal field using the deep neural network is represented by X, and the characteristic at the same layer extracted from the sample in the countermeasure field using the same deep neural network is represented by Y, and the optimal transmission distance between the two characteristic distributions X and Y is formulated as follows:
formula (2):
equation (2) is a definition of Wasserstein distance (bulldozer distance). Wherein: representing this is a definition, defining the calculation result on the right as the representation form on the left, P X And P Y Respectively represent the edge distribution forms of the features X and Y, and P (X-P X ，Y～P Y ) Representing the joint distribution of features X and Y, c (X, Y) is any measurable error function that measures the distance between X and Y; in addition, E (X，Y)～Γ Representing the mathematical expectation under joint probability, inf represents the infinitesimal bound where the calculation is the mathematical expectation, and therefore, W c (P X ，P Y ) Is defined as the edge distribution P of features X and Y under the premise of a measurable error function c X And P Y For input, among all the metric distance methods, the method in which the X to Y distance is the smallest is called an optimal transmission method, and the calculated distance value is the required optimal transmission distance.
Preferably, step (4), specifically including hierarchically extracted feature representations from the normal-domain image sample and the challenge-domain image sample, calculating the differences between them using Wasserstein distance after processing using graph convolution, adding Wasserstein distance of the challenge-sample feature representation and the normal-sample feature representation in different hierarchies to a final loss function used to optimize network parameters, and allowing the network model to learn a more robust feature representation by using feature alignment gradually through sufficient end-to-end training;
the final loss function is shown below:
equation (3):
wherein in formula (3), F represents a method for classifying imagesθ is a parameter of the deep neural network that is learned during end-to-end training of the network, L CE Representing a cross entropy loss function, and simultaneously calculating cross entropy loss of the normal sample and the corresponding countermeasure sample, so that the network can successfully classify the normal sample and the countermeasure sample; x is x clean Represents a normal sample, x adv Representing challenge samples, y true A correct tag representing the data is displayed,and->Representing image feature representations extracted from normal and challenge samples at the first layer of the deep neural network F, respectively, l=1, 2 or l=1, 2,3,4; LC means linear combination of features; λ represents the relative weights between multiple loss functions, and when training a model using a training set, the final loss function shown in formula (3) is used to calculate classification errors and differences between sample features in different fields, and then a random gradient descent algorithm is used to optimize model parameters of the network according to the errors, so as to finally find the optimal model parameters.
The invention has the advantages that: compared with the prior art, the invention provides a novel hierarchical feature alignment method from the field self-adaption point of view, so that the deep convolutional neural network can learn robust feature representation from the countermeasure sample; in order to better let the model learn the robust feature representation when the similarity of the antagonistic sample feature and the normal sample feature is improved by progressively improving the model network structure, the invention provides a Wasserstein distance method for measuring the difference between the antagonistic sample feature and the normal sample feature based on the optimal transmission theory.
The method provided by the invention can effectively improve the generalization capability of the model based on the deep convolutional neural network to samples in different countermeasure fields, and provides effective defense even if the model is attacked by a white box which is difficult to process by the prior method;
the model based on the depth convolution neural network can learn more robust image features through layered feature alignment operation, so that the problem that the generalization capability of an anti-sample model is limited in different fields in the prior art is solved, and effective reliability and safety guarantee are provided for deployment and application of a depth model system.
The invention is further described below with reference to the drawings and specific examples.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention;
FIG. 2 is a schematic diagram showing a model structure for counterdefense according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a robust feature learning process over challenge samples according to an embodiment of the present invention;
FIG. 4 is a schematic view of a visualization of the decision space of a normal sample and an challenge sample, respectively, on top of three exemplary classification datasets in accordance with an embodiment of the present invention.
Detailed Description
Referring to fig. 1,2 and 3, the robust feature learning method based on hierarchical feature alignment disclosed by the invention comprises the following steps:
(1) Lifting depth features of different layers from samples in different fields by using a depth convolution neural network;
(2) For the extracted layered features, the channel and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns more abundant feature representations;
(3) Using Wasserstein distance based on optimal transmission theory to accurately measure the difference between sample characteristic representations in different fields;
(4) Differences between layered features extracted from different domain samples are used as part of a model loss function to help the model learn more robust features, thereby improving the robustness against deep neural network models.
Step (1), the specific process is to use a PGD (Project Gradient Descent, mapping gradient descent) attack method to generate corresponding countermeasure field image samples with different degrees of disturbance from the image samples in the normal field. For normal field image samples and contrast field image samples, the method uses a deep convolutional neural network to extract multi-level features of the images. To make the extracted features more representative, we divide the depth network into multiple layers according to its structure. Since the present invention proposes a framework, we use different "layers" to represent the structural hierarchy of these partitions for feature extraction, rather than explicitly specifying which layer is the network, the structural hierarchy partitioning criteria proposed by the present invention are shown in fig. 2.
Taking the structure shown in fig. 2 as an example, we use a res net-110 ("res net-110" refers here to a residual network with 110 network layers) network structure for feature extraction of images, where we divide it into 4 different structural hierarchies. When forward reasoning is carried out on the network after normal samples or countermeasure samples are input, image features with different scales and different abstract degrees are extracted at 4 different levels by using a convolution structure.
In order to better perform feature alignment operation during model training, the invention proposes to perform model training on a clean sample, and then perform model training based on a robust feature learning process of hierarchical feature alignment by using the clean sample and a corresponding countermeasure sample together.
And (2) extracting image features at different levels by using a deep convolutional neural network, and processing the extracted features at different levels by using the convolutional neural network for representing the representative image features so as to make the network learn more rich image feature representations. Graph convolution can better capture the relationship between different areas in the depth feature from a global perspective; and simultaneously, stronger limitation can be added to the characteristics when the characteristics are aligned subsequently. Since we calculate the distance between feature vectors when calculating the optimal transmission Wasserstein distance, we turn the tensor-form image features into the form of feature vectors; and, to speed up the calculation of the distance metric, we will take a series of feature selection and dimension reduction operations.
To reduce the complexity of dimension reduction, we use a representative linear combination of features to process the extracted features from both the channel and feature nodes,
after dimension reduction, we perform a graph convolution operation using two one-dimensional convolutions, which can be formulated as follows:
formula (1):
GCN(f)＝Conv1D[Conv1D(f)]
in the formula (1), GCN (, a,) represents a graph convolution neural network, f represents a feature vector subjected to a dimension reduction process, where f represents an input of a graph convolution operation; in addition, conv1D (, the figure) represents a one-dimensional convolution operation where we use two different-direction one-dimensional graph convolution operations for feature extraction. After sufficient end-to-end training, the graph convolution operation may enhance the representational capacity for relationships between different regions in the feature.
The specific details are shown in fig. 2. After sufficient end-to-end training, the graph convolution operation may enhance the representational capacity for relationships between different regions in the feature. In addition, for features extracted from normal samples and features extracted from challenge samples at different network structure levels, the present invention uses graph convolution to process the features before calculating Wasserstein distance between them. As shown in fig. 2, we calculated Wasserstein distance at 4 different locations to measure the differences in characteristics of the different domain samples when using the res net-110 structure.
And (3) performing hierarchical image feature extraction by using the step (1), performing feature selection, dimension reduction and graph convolution operation by using the step (2), and calculating differences among samples in different fields by using optimal transmission Wasserstein distance with regularization terms, wherein the step is to perform feature alignment operation among the features of the samples in different fields, and align the hierarchical features of the countermeasure samples to the hierarchical features of the normal samples, so that the neural model has enough robustness.
In this embodiment, we use X and Y to represent a set of feature vectors of two different distributions, more specifically, X represents a feature at a certain layer extracted from a sample in the normal domain using a deep neural network, and Y represents a feature at the same layer extracted from a sample in the challenge domain using the same deep neural network, and the optimal transmission distance between the two feature distributions X and Y can be formulated as follows:
formula (2):
equation (2) is a definition of Wasserstein distance (bulldozer distance). Wherein, the symbol: representing this is a definition, we define the right calculation result to the left representation. In the formula, P X And P Y Respectively represent the edge distribution forms of the features X and Y, and P (X-P X ，Y～P Y ) Representing the joint distribution of features X and Y. c (X, Y) is any measurable error function that measures the distance between X and Y. Also, in the formula, E (X，Y)～Γ Representing the mathematical expectation under joint probability, inf represents the inf where the calculation is the infinitesimal of the mathematical expectation. Thus W is c (P X ，P Y ) Is defined as the edge distribution P of features X and Y under the premise of a measurable error function c X And P Y For input, among all the measurement distance modes, the mode with the smallest X-to-Y distance is called an optimal transmission mode, and the calculated distance value is the optimal transmission distance required here.
In the present embodiment, use is made ofTo calculate the distance between the feature vectors. Thus, the formula can be expressed as follows:
in practical applications, it may be discretized into the following formula form:
where in the formula <, represents the Hadamard product between the matrices P and C, where P and C are both in the form of a two-dimensional matrix, and thus represent the sum of the products of the elements at each corresponding position of the matrices P and C, and min represents an optimization problem where the minimum is calculated. Because the calculation cost of the method can rise rapidly along with the increase of the data quantity, the method uses an entropy regularization mode to improve the algorithm and uses a sink horn iterative algorithm to optimize. The entropy regularization term for matrix P is shown as follows:
thus, an optimal transmission Wasserstein distance calculation method with regularization can be obtained:
where e is used to balance the approximation of the regularization problem to the original problem in the formula, when e goes towards 0, the regularization problem is converted into the original problem, and in the present invention, e=0.1. Furthermore, since this problem is a convex optimization problem, it has a unique solution. In addition, wasserstein distance is used in the present invention to measure the difference between intermediate feature representations extracted from normal and challenge samples using deep convolutional neural networks.
In addition, we choose to use the Sinkhorn iterative algorithm when optimizing the optimal transmission distance.
Step (4), the specific process is that Wasserstein distance of the anti-sample characteristic representation and the normal sample characteristic representation in different layers are added into a final loss function used for optimizing network parameters, and the network model gradually learns more robust characteristic representation by utilizing characteristic alignment through sufficient end-to-end training.
The final loss function is shown in the following formula:
formula (3)
Wherein in formula (3), F represents a deep neural network for image classification, θ is a parameter of the deep neural network that is learned during end-to-end training of the network, L CE Representing a cross entropy loss function, and simultaneously calculating cross entropy loss of the normal sample and the corresponding countermeasure sample, so that the network can successfully classify the normal sample and the countermeasure sample; x is x clean Represents a normal sample, x adv Representing challenge samples, y true A correct tag representing the data is displayed,and->Representing image feature representations extracted from normal and challenge samples at the first layer of the deep neural network F, respectively, l=1, 2 or l=1, 2,3,4; LC means linear combination of features; λ represents the relative weights between multiple loss functions, and when training a model using a training set, the final loss function shown in formula (3) is used to calculate classification errors and differences between sample features in different fields, and then a random gradient descent algorithm is used to optimize model parameters of the network according to the errors, so as to finally find the optimal model parameters.
The embodiment of the invention has the following beneficial effects:
compared with the prior art, the invention provides a novel hierarchical feature alignment method from the field self-adaption point of view, so that the deep convolutional neural network can learn robust feature representation from the countermeasure sample; in order to better let the model learn the robust feature representation when the similarity of the antagonistic sample feature and the normal sample feature is improved by progressively improving the model network structure, the invention provides a Wasserstein distance method for measuring the difference between the antagonistic sample feature and the normal sample feature based on the optimal transmission theory.
The method provided by the invention can effectively improve the generalization capability of the model based on the deep convolutional neural network to samples in different countermeasure fields, and provides effective defense even if the model is attacked by a white box which is difficult to process by the prior method;
the model based on the depth convolution neural network can learn more robust image features through layered feature alignment operation, so that the problem that the generalization capability of an anti-sample model is limited in different fields in the prior art is solved, and effective reliability and safety guarantee are provided for deployment and application of a depth model system.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in implementing the methods of the above embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc.
The foregoing embodiments are provided for further explanation of the present invention and are not to be construed as limiting the scope of the present invention, and some insubstantial modifications and variations of the present invention, which are within the scope of the invention, will be suggested to those skilled in the art in light of the foregoing teachings.
Claims (4)
1. A robust feature learning method based on hierarchical feature alignment is characterized in that: the method comprises the following steps:
(1) Lifting depth features of different layers from image samples in different fields by using a depth convolution neural network;
(2) For the extracted layered features, the channel and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns more abundant feature representations;
(3) Using Wasserstein distance based on optimal transmission theory to accurately measure the difference between sample characteristic representations in different fields;
(4) Taking the difference between layered features extracted from different field samples as a part of a model loss function to help the model learn more robust features, thereby improving the antagonism robustness of the deep neural network model;
step (2) performing a graph convolution operation using two one-dimensional convolutions, the graph convolution operation formulated as:
GCN(f)＝Conv1D[Conv1D(f)]
wherein, in the formula, GCN (or) represents a graph convolution neural network, f represents a feature vector subjected to dimension reduction treatment, and f represents the input of graph convolution operation; in addition, conv1D (the) represents a one-dimensional convolution operation, the feature extraction is carried out by using two one-dimensional graph convolution operations with different directions, and after sufficient end-to-end training, the graph convolution operation enhances the representation capability of the relation between different areas in the feature;
step (3), using X to represent the features at a certain layer extracted from the samples in the normal field using the deep neural network, and Y to represent the features at the same layer extracted from the samples in the countermeasure field using the same deep neural network, the optimal transmission distance between the two feature distributions X and Y being formulated as follows:
wherein, in the formula: representing this is a definition, defining the calculation result on the right as the representation form on the left, P X And P Y Respectively represent the edge distribution forms of the features X and Y, and P (X-P X ,Y～P Y ) Representing the joint distribution of features X and Y, c (X, Y) is any measurable error function that measures the distance between X and Y; this isIn addition, E (X,Y)～Γ Representing the mathematical expectation under joint probability, inf represents the infinitesimal bound where the calculation is the mathematical expectation, and therefore, W c (P X ,P Y ) Is defined as the edge distribution P of features X and Y under the premise of a measurable error function c X And P Y For input, among all the metric distance methods, the method in which the X to Y distance is the smallest is called an optimal transmission method, and the calculated distance value is the required optimal transmission distance.
2. A robust feature learning method based on hierarchical feature alignment as claimed in claim 1, wherein: the different domain image samples include a normal domain image sample and a countermeasure domain image sample.
3. A robust feature learning method based on hierarchical feature alignment as claimed in claim 2, wherein: and (1) extracting the characteristics of the image by using a ResNet-110 network structure, dividing the image into 4 different structural layers, and extracting the image characteristics with different scales and different abstract degrees by using a convolution structure at the 4 different structural layers when forward reasoning is performed on the network after a normal sample or a countermeasure sample is input.
4. A robust feature learning method based on hierarchical feature alignment according to claim 3, characterized in that: step (4), specifically including the characteristic representation hierarchically extracted from the normal field image sample and the contrast field image sample, after processing by using graph convolution, calculating the difference between them by using Wasserstein distance, adding Wasserstein distance the contrast sample characteristic representation and the normal sample characteristic representation in different levels to the final loss function used for optimizing the network parameters, and gradually utilizing characteristic alignment to learn the more robust characteristic representation by full end-to-end training;
the final loss function is shown in the following formula:
wherein, in the formula, F represents a deep neural network for image classification, θ is a parameter of the deep neural network, the parameter is learned during end-to-end training of the network, L CE Representing a cross entropy loss function, and simultaneously calculating cross entropy loss of the normal sample and the corresponding countermeasure sample, so that the network can successfully classify the normal sample and the countermeasure sample; x is x clean Represents a normal sample, x adv Representing challenge samples, y true A correct tag representing the data is displayed,and->Representing image feature representations extracted from normal and challenge samples at the first layer of the deep neural network F, respectively, l=1, 2 or l=1, 2,3,4; LC means linear combination of features; λ represents the relative weights between multiple loss functions, and when training a model using a training set, the final loss function calculates the classification errors and the differences between the sample features of different fields, and then optimizes the model parameters of the network using a random gradient descent algorithm according to these errors, finally finding the optimal model parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809932.6A CN111950635B (en) | 2020-08-12 | 2020-08-12 | Robust feature learning method based on layered feature alignment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809932.6A CN111950635B (en) | 2020-08-12 | 2020-08-12 | Robust feature learning method based on layered feature alignment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111950635A CN111950635A (en) | 2020-11-17 |
| CN111950635B true CN111950635B (en) | 2023-08-25 |
Family
ID=73331806
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010809932.6A Active CN111950635B (en) | 2020-08-12 | 2020-08-12 | Robust feature learning method based on layered feature alignment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111950635B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465019B (en) * | 2020-11-26 | 2022-12-27 | 重庆邮电大学 | Countermeasure sample generation and countermeasure defense method based on disturbance |
| CN113436073B (en) * | 2021-06-29 | 2023-04-07 | 中山大学 | Real image super-resolution robust method and device based on frequency domain |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028255A1 (en) * | 2016-08-11 | 2018-02-15 | 深圳市未来媒体技术研究院 | Image saliency detection method based on adversarial network |
| CN110674866A (en) * | 2019-09-23 | 2020-01-10 | 兰州理工大学 | Method for detecting X-ray breast lesion images by using transfer learning characteristic pyramid network |
| CN110728219A (en) * | 2019-09-29 | 2020-01-24 | 天津大学 | 3D face generation method based on multi-column multi-scale graph convolution neural network |
| CN110738622A (en) * | 2019-10-17 | 2020-01-31 | 温州大学 | Lightweight neural network single image defogging method based on multi-scale convolution |
| CN111126258A (en) * | 2019-12-23 | 2020-05-08 | 深圳市华尊科技股份有限公司 | Image recognition method and related device |
| CN111178504A (en) * | 2019-12-17 | 2020-05-19 | 西安电子科技大学 | Information processing method and system of robust compression model based on deep neural network |
| CN111242227A (en) * | 2020-01-16 | 2020-06-05 | 天津师范大学 | Multi-modal foundation cloud identification method based on heterogeneous depth features |
| CN111476200A (en) * | 2020-04-27 | 2020-07-31 | 华东师范大学 | Face de-identification generation method based on generation of confrontation network |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10698063B2 (en) * | 2017-11-01 | 2020-06-30 | Siemens Healthcare Gmbh | Motion artifact reduction of magnetic resonance images with an adversarial trained network |
| US11030485B2 (en) * | 2018-03-30 | 2021-06-08 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems and methods for feature transformation, correction and regeneration for robust sensing, transmission, computer vision, recognition and classification |
| US11481617B2 (en) * | 2019-01-22 | 2022-10-25 | Adobe Inc. | Generating trained neural networks with increased robustness against adversarial attacks |
-
2020
- 2020-08-12 CN CN202010809932.6A patent/CN111950635B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028255A1 (en) * | 2016-08-11 | 2018-02-15 | 深圳市未来媒体技术研究院 | Image saliency detection method based on adversarial network |
| CN110674866A (en) * | 2019-09-23 | 2020-01-10 | 兰州理工大学 | Method for detecting X-ray breast lesion images by using transfer learning characteristic pyramid network |
| CN110728219A (en) * | 2019-09-29 | 2020-01-24 | 天津大学 | 3D face generation method based on multi-column multi-scale graph convolution neural network |
| CN110738622A (en) * | 2019-10-17 | 2020-01-31 | 温州大学 | Lightweight neural network single image defogging method based on multi-scale convolution |
| CN111178504A (en) * | 2019-12-17 | 2020-05-19 | 西安电子科技大学 | Information processing method and system of robust compression model based on deep neural network |
| CN111126258A (en) * | 2019-12-23 | 2020-05-08 | 深圳市华尊科技股份有限公司 | Image recognition method and related device |
| CN111242227A (en) * | 2020-01-16 | 2020-06-05 | 天津师范大学 | Multi-modal foundation cloud identification method based on heterogeneous depth features |
| CN111476200A (en) * | 2020-04-27 | 2020-07-31 | 华东师范大学 | Face de-identification generation method based on generation of confrontation network |
Non-Patent Citations (1)
| Title |
|---|
| 基于卷积对抗网络的多通道图像修复方法;范宝杰，等;《计算机应用与软件》;第37卷(第7期);第176-179、265页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111950635A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110941794B (en) | Challenge attack defense method based on general inverse disturbance defense matrix | |
| Esmaeili et al. | Fast-at: Fast automatic thumbnail generation using deep neural networks | |
| CN107844743A (en) | A kind of image multi-subtitle automatic generation method based on multiple dimensioned layering residual error network | |
| CN111462210B (en) | Monocular line feature map construction method based on epipolar constraint | |
| CN111950635B (en) | Robust feature learning method based on layered feature alignment | |
| CN107871105A (en) | Face authentication method and device | |
| CN111310582A (en) | Turbulence degradation image semantic segmentation method based on boundary perception and counterstudy | |
| Elaskily et al. | Deep learning based algorithm (ConvLSTM) for copy move forgery detection | |
| CN110309835B (en) | Image local feature extraction method and device | |
| CN105184260A (en) | Image characteristic extraction method, pedestrian detection method and device | |
| CN114255403A (en) | Optical remote sensing image data processing method and system based on deep learning | |
| CN111400572A (en) | Content safety monitoring system and method for realizing image feature recognition based on convolutional neural network | |
| CN112381733B (en) | Image recovery-oriented multi-scale neural network structure searching method and network application | |
| Liu et al. | Target recognition of sport athletes based on deep learning and convolutional neural network | |
| CN111091129A (en) | Image salient region extraction method based on multi-color characteristic manifold sorting | |
| CN114399630A (en) | Countercheck sample generation method based on belief attack and significant area disturbance limitation | |
| CN112329764A (en) | Infrared dim target detection method based on TV-L1 model | |
| CN116912804A (en) | Efficient anchor-frame-free 3-D target detection and tracking method and model | |
| CN113935396A (en) | Manifold theory-based method and related device for resisting sample attack | |
| CN117152486A (en) | Image countermeasure sample detection method based on interpretability | |
| Liu et al. | Iris double recognition based on modified evolutionary neural network | |
| CN116109649A (en) | 3D point cloud instance segmentation method based on semantic error correction | |
| Jafrasteh et al. | Generative adversarial networks as a novel approach for tectonic fault and fracture extraction in high resolution satellite and airborne optical images | |
| CN113920159B (en) | Infrared air small and medium target tracking method based on full convolution twin network | |
| CN113487506A (en) | Countermeasure sample defense method, device and system based on attention denoising |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |