CN111950635A - Robust feature learning method based on hierarchical feature alignment - Google Patents

Robust feature learning method based on hierarchical feature alignment Download PDF

Info

Publication number
CN111950635A
CN111950635A CN202010809932.6A CN202010809932A CN111950635A CN 111950635 A CN111950635 A CN 111950635A CN 202010809932 A CN202010809932 A CN 202010809932A CN 111950635 A CN111950635 A CN 111950635A
Authority
CN
China
Prior art keywords
feature
sample
features
model
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010809932.6A
Other languages
Chinese (zh)
Other versions
CN111950635B (en
Inventor
张笑钦
王金鑫
赵丽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wenzhou University
Original Assignee
Wenzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wenzhou University filed Critical Wenzhou University
Priority to CN202010809932.6A priority Critical patent/CN111950635B/en
Publication of CN111950635A publication Critical patent/CN111950635A/en
Application granted granted Critical
Publication of CN111950635B publication Critical patent/CN111950635B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention discloses a robust feature learning method based on hierarchical feature alignment, which comprises the following steps: carrying out hierarchical extraction of depth features from input samples in different fields by using a depth convolution neural network; for the extracted hierarchical features, the channels and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns richer feature representation; accurately measuring the difference between the sample feature representations in different fields by using Wasserstein distance based on the optimal transmission theory; differences between layered features extracted from samples in different fields are used as a part of a model loss function to help a model to learn more robust features, so that the robustness of the deep neural network model is improved. By the technical scheme, the deep network model can learn the robust characteristics, and the damage of the anti-attack method is avoided, so that a safe and reliable deep system is obtained.

Description

Robust feature learning method based on hierarchical feature alignment
Technical Field
The invention relates to the technical field of robust machine learning, in particular to a robust feature learning method based on hierarchical feature alignment.
Background
In recent years, deep convolutional neural networks have broken through many computer vision tasks such as image classification, target detection, and the like. However, researchers have found that these deep convolutional neural networks are vulnerable to spoofing by specially designed anti-perturbation samples that are not easily detectable by the human eye. These challenge samples generated by the challenge-challenge method pose serious challenges to systems with high requirements for safety and stability, including automatic driving systems, medical diagnostic systems, and security systems. In addition, if a deep network model changes its prediction result with high confidence when a sample with a small amount of disturbance is given as an input, it can be judged that the models do not learn task-related inherent attributes from the input sample from the beginning, and a robust visual concept cannot be learned from the sample. Therefore, designing a deep network model that is robust enough to combat disturbances is crucial for safe and reliable computer vision applications.
In recent research work, researchers have proposed a variety of defense mechanisms to overcome different approaches to combat attacks. These defense mechanisms can be roughly divided into two categories. The first category of defense methods mainly employs various pre-processing on the input image to overcome the counterattack. Dziugaite et al and Das et al use JPEG image compression as a countermeasure defense. These methods use discrete fourier transforms in the field of input images to process the anti-noise. However, these JPEG image compression-based methods have far failed to successfully remove the counternoise. By fully utilizing the strong representation capability of generating an anti-network, the Defense-GAN method is proposed by Samangouei and the like to defend various anti-attacks; the method achieves the aim of removing the counternoise by regenerating an image sufficiently similar to the input image. Mustafa et al propose to use image super-resolution as a countermeasure defense means, and by using a depth super-resolution network as a mapping function, the method maps samples from the countermeasure domain to the normal domain, thereby achieving the purpose of removing the countermeasure noise, and finally inputting the mapped image into an image recognition system for normal recognition. Another countermeasure is to improve the robustness of the model by modifying the training process or the network structure to deal with the counterdisturbance. The confrontation training is an effective means for improving the confrontation robustness of the model, and achieves the aim by adding specially designed confrontation samples to training data. Goodfellow et al trained the network model by adding to the clean samples countersamples generated using the FGSM (Fast Gradient Sign Method) counterattack Method. Madry et al performed challenge training using a Min-Max optimization method that generated challenge samples using a PGD (Project Gradient determination) attack method. Integrated countermeasure training is also a novel countermeasure defense method that uses countermeasure samples generated from a variety of different deep networks as training data to optimize model parameters. In addition, in order to improve the generalization ability of the depth model to the countermeasure sample, Song et al trains the network model using a domain adaptive method.
Although the above methods have made good progress in improving robustness of the deep convolutional neural network, they are often unable to achieve satisfactory results for different kinds of white-box attack methods, which are limited by poor generalization performance of the model.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a robust feature learning method based on hierarchical feature alignment, which enables a model based on a deep convolutional neural network to obtain more robust image features through the operation of hierarchical feature alignment, thereby solving the problem of limited ability of resisting sample model generalization in different fields in the prior art and providing effective reliability and safety guarantee for the deployment and application of a deep model system.
In order to achieve the purpose, the invention provides the following technical scheme: a robust feature learning method based on hierarchical feature alignment comprises the following steps:
(1) extracting depth features of different levels from samples of different fields by using a depth convolution neural network;
(2) for the extracted hierarchical features, the channels and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns richer feature representation;
(3) accurately measuring the difference between the sample feature representations in different fields by using Wasserstein distance based on the optimal transmission theory;
(4) differences between layered features extracted from samples in different fields are used as a part of a model loss function to help a model to learn more robust features, so that the robustness of the deep neural network model is improved.
Preferably, the image samples of different domains include normal domain image samples and confrontation domain image samples.
Preferably, in step (1), the feature extraction of the image is performed by using a ResNet-110 network structure, the image is divided into 4 different structural levels, and after a normal sample or a countermeasure sample is input, when the network performs forward reasoning, the image features with different scales and different abstraction degrees are extracted by using a convolution structure at the 4 different structural levels.
Preferably, in step (2), the graph convolution operation is performed using two one-dimensional convolutions, and is formulated as follows:
formula (1):
GCN(f)＝Conv1D[Conv1D(f)]
in formula (1), GCN (∙) represents a graph convolution neural network, f represents a feature vector subjected to dimension reduction processing, and f represents an input of a graph convolution operation; in addition, Conv1D (∙) represents a one-dimensional convolution operation that uses two differently oriented one-dimensional graph convolution operations for feature extraction that, after sufficient end-to-end training, enhances the representation of the relationships between different regions in a feature.
Preferably, in step (3), X represents a feature at a certain layer extracted from a sample in the normal domain by using a deep neural network, Y represents a feature at the same layer extracted from a sample in the countermeasure domain by using the same deep neural network, and the optimal transmission distance between the two feature distributions X and Y is formulated as follows:
formula (2):
Figure BDA0002628485790000041
equation (2) is the definition of Wasserstein distance. Wherein: meaning that this is a definition, the right calculation result is defined as the representation to the left, PXAnd PYRespectively represent edge distribution forms of the features X and Y, and P (X to P)X，Y～PY) Representing the joint distribution of features X and Y, c (X, Y) being an arbitrary measurable error function that measures the distance between X and Y; furthermore, E(X，Y)～Representing the mathematical expectation under joint probability, inf represents the infimum bound where the computation is the mathematical expectation, and thus, Wc(PX，PY) Is defined as the distribution P of the edges of the features X and Y with the premise of a measurable error function cXAnd PYFor input, among all the distance measuring methods, the method in which the distance from X to Y is the smallest is called the optimal transmission method, and the calculated distance value is the required optimal transmission distance.
Preferably, the step (4) specifically includes extracting feature representations hierarchically from the normal domain image sample and the confrontation domain image sample, calculating a difference between the normal domain image sample and the confrontation domain image sample by using Wasserstein distance after processing by using graph volume, adding the Wasserstein distance of the confrontation sample feature representation and the normal sample feature representation in different layers into a final loss function for optimizing network parameter use, and gradually learning a more robust feature representation by using feature alignment for the network model through sufficient end-to-end training;
the final loss function is shown below:
formula (3):
Figure BDA0002628485790000051
wherein, in formula (3), F represents a deep neural network for image classification, θ is a parameter of the deep neural network, the parameter is learned when the network is trained end-to-end, and LCERepresenting a cross entropy loss function, and simultaneously calculating the cross entropy loss of the normal sample and the corresponding confrontation sample, so that the network can successfully classify the normal sample and the confrontation sample; x is the number ofcleanDenotes a normal sample, xadvRepresenting a challenge sample, ytrueThe correct tag that represents the data is,
Figure BDA0002628485790000052
and
Figure BDA0002628485790000053
image feature representations extracted from the normal sample and the confrontation sample at the l-th layer of the deep neural network F are respectively represented, l is 1, 2 or 1, 2, 3, 4; LC represents linear combination of features; and lambda represents the relative weight among a plurality of loss functions, when the model is trained by using a training set, the final loss function shown in the formula (3) is used for calculating classification errors and differences among sample characteristics in different fields, and then a random gradient descent algorithm is used for optimizing model parameters of the network according to the errors, so that the optimal model parameters are finally found.
The invention has the advantages that: compared with the prior art, the invention provides a novel hierarchical feature alignment method from the field self-adaption perspective, so that the deep convolutional neural network can learn robust feature representation from a confrontation sample; when the similarity of the feature of the countermeasure sample and the feature of the normal sample is improved progressively along the network structure of the model, in order to better enable the model to learn robust feature representation, the invention provides a Wasserstein distance based on the optimal transmission theory to measure the difference between the feature of the countermeasure sample and the feature of the normal sample.
The method provided by the invention can effectively improve the generalization capability of the model based on the deep convolutional neural network to samples in different confrontation fields, and can provide effective defense even if the model is attacked by a white box which is difficult to process by the conventional method;
the model based on the deep convolutional neural network can obtain more robust image characteristics through the operation of hierarchical characteristic alignment, so that the problem that the generalization capability of the anti-sample model in different fields is limited in the prior art is solved, and effective reliability and safety guarantee are provided for the deployment and application of a deep model system.
The invention is further described with reference to the drawings and the specific embodiments in the following description.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention;
FIG. 2 is a schematic diagram of a model structure proposed for defense fighting according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a robust feature learning process on a challenge sample according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating the visualization of the decision space for normal samples and confrontation samples on three typical classification data sets according to the embodiment of the present invention.
Detailed Description
Referring to fig. 1, fig. 2 and fig. 3, the robust feature learning method based on hierarchical feature alignment disclosed by the present invention includes the following steps:
(1) extracting depth features of different levels from samples of different fields by using a depth convolution neural network;
(2) for the extracted hierarchical features, the channels and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns richer feature representation;
(3) accurately measuring the difference between the sample feature representations in different fields by using Wasserstein distance based on the optimal transmission theory;
(4) differences between layered features extracted from samples in different fields are used as a part of a model loss function to help a model to learn more robust features, so that the robustness of the deep neural network model is improved.
Step (1), the specific process is that a PGD (Project Gradient) attack method is used to generate corresponding confrontation domain image samples with different degrees of disturbance from the image samples in the normal domain. For normal domain image samples and confrontation domain image samples, the method uses a deep convolutional neural network to extract multilevel characteristics of the image. To make the extracted features more representative, we divide the deep network into multiple levels according to its structure. Since the present invention proposes a framework, we use different "layers" to represent the divided structural levels for feature extraction, rather than to explicitly specify which layer of the network is, the proposed structural level division standard of the present invention is shown in fig. 2.
Taking the structure shown in fig. 2 as an example, we use the network structure of ResNet-110 ("ResNet-110" here refers to a residual network with 110 network layers) to perform feature extraction of an image, where we divide it into 4 different structural levels. After a normal sample or a confrontation sample is input, when the network carries out forward reasoning, image features with different scales and different abstraction degrees are extracted by using a convolution structure at 4 different levels.
When model training is carried out, in order to better carry out the operation of feature alignment, the invention proposes that the model training is carried out on a clean sample, and then the clean sample and a corresponding countermeasure sample are jointly used for carrying out the model training of a robust feature learning process based on hierarchical feature alignment.
And (2) after extracting image features at different levels by using a deep convolutional neural network, for the representative image features, in order to enable the network to learn richer image feature representation, processing the extracted features at different levels by using the graph convolutional neural network. Graph convolution can better capture the relation between different areas in the depth feature from the global perspective; and may also impose stronger constraints on the features for subsequent alignment of the features. When the optimal transmission Wasserstein distance is calculated, the distance between the characteristic vectors is calculated, so that the image characteristics in the tensor form are converted into the characteristic vector form; and, in order to speed up the calculation of the distance metric, we will take a series of operations of feature selection and dimension reduction.
In order to reduce the complexity of dimension reduction, a representative characteristic linear combination mode of the characteristics is used for processing the extracted characteristics from two aspects of channels and characteristic nodes,
after dimensionality reduction, we perform the graph convolution operation using two one-dimensional convolutions, which can be formulated as:
formula (1):
GCN(f)＝Conv1D[Conv1D(f)]
in formula (1), GCN (∙) represents a graph convolution neural network, f represents a feature vector subjected to dimensionality reduction, where f represents the input of a graph convolution operation; in addition, Conv1D (∙) represents a one-dimensional convolution operation where we perform feature extraction using two one-dimensional graph convolution operations in different directions. After sufficient end-to-end training, the graph convolution operation can enhance the representation capability of the relationship between different regions in the feature.
The specific detail information is shown in fig. 2. After sufficient end-to-end training, the graph convolution operation can enhance the representation capability of the relationship between different regions in the feature. In addition, for features extracted from normal samples and features extracted from challenge samples at different network structure levels, the present invention processes the features using graph convolution before computing the Wasserstein distance between them. As shown in FIG. 2, when using the ResNet-110 structure, we calculated Wasserstein distances at 4 different locations to make a measure of the difference in sample characteristics in different domains.
And (3) performing a specific process that after the step (1) is used for extracting the hierarchical image features and the step (2) is used for performing feature selection, dimension reduction and graph volume operations, the difference between samples in different fields is calculated by using the optimal transmission Wasserstein distance with the regularization term, and the step aligns the hierarchical features of the countermeasure samples to the hierarchical features of the normal samples in order to perform feature alignment operations between the characteristics of the samples in the different fields, so that the neural model has sufficient robustness.
In this embodiment, we use X and Y to represent a set of feature vectors of two different distributions, more specifically, X represents a feature at a certain layer extracted from a sample in the normal domain using a deep neural network, and Y represents a feature at the same layer extracted from a sample in the countermeasure domain using the same deep neural network, and the optimal transmission distance between the two feature distributions X and Y can be formulated as follows:
formula (2):
Figure BDA0002628485790000091
equation (2) is the definition of Wasserstein distance. Wherein, the symbol: to indicate that this is a definition, we define the results of the calculations on the right as the representation on the left. In the formula, PXAnd PYRespectively represent edge distribution forms of the features X and Y, and P (X to P)X，Y～PY) Representing the joint distribution of features X and Y. c (X, Y) is an arbitrary measurable error function that measures the distance between X and Y. Further, in the formula, E(X，Y)～It represents the mathematical expectation under joint probability, inf represents the infimum bound where the computation is the mathematical expectation. Thus, Wc(PX，PY) Is defined as the distribution P of the edges of the features X and Y with the premise of a measurable error function cXAnd PYFor input, among all the distance measuring methods, the method with the smallest distance from X to Y is called the optimal transmission method, and the calculated distance value is the optimal transmission distance required here.
In this embodiment, use is made of
Figure BDA0002628485790000092
The distance between the feature vectors is calculated. Thus, the formula can be expressed as follows:
Figure BDA0002628485790000093
in practical application, it can be discretized into the form of the following formula:
Figure BDA0002628485790000094
wherein <, > represents the Hadamard (Hadamard) product between the matrices P and C, since both P and C are two-dimensional matrices, the sum of the products of the elements at each corresponding position of the matrix P and the matrix C is represented here, and min represents the optimization problem of calculating the minimum value here. As the calculation cost of the method can rise rapidly along with the increase of the data volume, the method improves the algorithm by using an entropy regularization mode and optimizes by using a Sinkhorn iterative algorithm. The entropy regularization term for the matrix P is shown by the following equation:
Figure BDA0002628485790000101
therefore, an optimal transmission Wasserstein distance calculation method with regularization can be obtained:
Figure BDA0002628485790000102
wherein in the formula, e is used to balance the approximation degree of the regularization problem and the original problem, when e is close to 0, the regularization problem is converted into the original problem, and in the invention, e is 0.1. Furthermore, since the problem is a convex optimization problem, it has a unique solution. In addition, in the present invention, Wasserstein distance is used to measure the difference between intermediate feature representations extracted using deep convolutional neural networks from normal and challenge samples.
In addition, we choose to use Sinkhorn iterative algorithm when optimizing the optimal transmission distance.
And (4) adding the Wasserstein distance represented by the confrontation sample characteristic representation and the normal sample characteristic representation in different layers into a final loss function used for optimizing network parameters, and gradually learning a more robust characteristic representation by utilizing characteristic alignment of a network model through sufficient end-to-end training.
The final loss function is shown in the following equation:
formula (3)
Figure BDA0002628485790000103
Wherein, in formula (3), F represents a deep neural network for image classification, θ is a parameter of the deep neural network, the parameter is learned when the network is trained end-to-end, and LCERepresenting a cross entropy loss function, and simultaneously calculating the cross entropy loss of the normal sample and the corresponding confrontation sample, so that the network can successfully classify the normal sample and the confrontation sample; x is the number ofcleanDenotes a normal sample, xadvRepresenting a challenge sample, ytrueThe correct tag that represents the data is,
Figure BDA0002628485790000111
and
Figure BDA0002628485790000112
image feature representations extracted from the normal sample and the confrontation sample at the l-th layer of the deep neural network F are respectively represented, l is 1, 2 or 1, 2, 3, 4; LC represents linear combination of features; lambda represents relative weight among a plurality of loss functions, when a model is trained by using a training set, a classification error and differences among sample characteristics in different fields are calculated by using a final loss function shown in formula (3), and then a random gradient is used according to the classification error and the differencesAnd optimizing the model parameters of the network by using a descent algorithm, and finally finding the optimal model parameters.
The embodiment of the invention has the following beneficial effects:
compared with the prior art, the invention provides a novel hierarchical feature alignment method from the field self-adaption perspective, so that the deep convolutional neural network can learn robust feature representation from a confrontation sample; when the similarity of the feature of the countermeasure sample and the feature of the normal sample is improved progressively along the network structure of the model, in order to better enable the model to learn robust feature representation, the invention provides a Wasserstein distance based on the optimal transmission theory to measure the difference between the feature of the countermeasure sample and the feature of the normal sample.
The method provided by the invention can effectively improve the generalization capability of the model based on the deep convolutional neural network to samples in different confrontation fields, and can provide effective defense even if the model is attacked by a white box which is difficult to process by the conventional method;
the model based on the deep convolutional neural network can obtain more robust image characteristics through the operation of hierarchical characteristic alignment, so that the problem that the generalization capability of the anti-sample model in different fields is limited in the prior art is solved, and effective reliability and safety guarantee are provided for the deployment and application of a deep model system.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by relevant hardware instructed by a program, and the program may be stored in a computer-readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc.
The above embodiments are described in detail for the purpose of further illustrating the present invention and should not be construed as limiting the scope of the present invention, and the skilled engineer can make insubstantial modifications and variations of the present invention based on the above disclosure.

Claims (6)

1. A robust feature learning method based on hierarchical feature alignment is characterized in that: the method comprises the following steps:
(1) extracting depth features of different levels from samples of different fields by using a depth convolution neural network;
(2) for the extracted hierarchical features, the channels and the spatial relation of the features are limited through a graph convolution neural network, so that the model learns richer feature representation;
(3) accurately measuring the difference between the sample feature representations in different fields by using Wasserstein distance based on the optimal transmission theory;
(4) differences between layered features extracted from samples in different fields are used as a part of a model loss function to help a model to learn more robust features, so that the robustness of the deep neural network model is improved.
2. The robust feature learning method based on hierarchical feature alignment as claimed in claim 1, wherein: the image samples of different domains include normal domain image samples and confrontation domain image samples.
3. The robust feature learning method based on hierarchical feature alignment as claimed in claim 2, wherein: and (1) extracting the features of the image by using a ResNet-110 network structure, wherein the image features are divided into 4 different structural levels, and after a normal sample or a confrontation sample is input, when the network carries out forward reasoning, the image features with different scales and different abstraction degrees are extracted by using a convolution structure at the 4 different structural levels.
4. The robust feature learning method based on hierarchical feature alignment as claimed in claim 3, wherein: step (2), using two one-dimensional convolutions to perform graph convolution operation, wherein the graph convolution operation is formulated as the following form:
GCN(f)＝Conv1D[Conv1D(f)]
in the formula, GCN (∙) represents a graph convolution neural network, f represents a feature vector subjected to dimensionality reduction, and f represents the input of a graph convolution operation; in addition, Conv1D (∙) represents a one-dimensional convolution operation that uses two differently oriented one-dimensional graph convolution operations for feature extraction that, after sufficient end-to-end training, enhances the representation of the relationships between different regions in a feature.
5. The robust feature learning method based on hierarchical feature alignment as claimed in claim 4, wherein: step (3), using X to represent the feature at a certain layer extracted from the sample in the normal field by using the deep neural network, and using Y to represent the feature at the same layer extracted from the sample in the confrontation field by using the same deep neural network, wherein the optimal transmission distance between the two feature distributions X and Y is formulated as follows:
Figure FDA0002628485780000021
wherein, in the formula,
Figure FDA0002628485780000023
indicating that this is a definition, the right calculation result is defined as the left representation, PXAnd PYRespectively represent edge distribution forms of the features X and Y, and P (X to P)X，Y～PY) Representing the joint distribution of features X and Y, c (X, Y) being an arbitrary measurable error function that measures the distance between X and Y; furthermore, E(X，Y)～Representing the mathematical expectation under joint probability, inf represents the infimum bound where the computation is the mathematical expectation, and thus, Wc(PX，PY) Is defined as the distribution P of the edges of the features X and Y with the premise of a measurable error function cXAnd PYFor input, among all the distance measuring methods, the method in which the distance from X to Y is the smallest is called the optimal transmission method, and the calculated distance value is the required optimal transmission distance.
6. The robust feature learning method based on hierarchical feature alignment as claimed in claim 5, wherein: step (4), specifically, the feature representation extracted in a layering mode from the normal domain image sample and the confrontation domain image sample is processed by using graph convolution, the difference between the normal domain image sample and the confrontation domain image sample is calculated by using Wasserstein distance, the confrontation sample feature representation and the normal sample feature representation in different layers are added into a final loss function used for optimizing network parameters, and the network model is gradually learned to the more robust feature representation by using feature alignment through sufficient end-to-end training;
the final loss function is shown in the following equation:
Figure FDA0002628485780000022
wherein in the formula, F represents a deep neural network for image classification, theta is a parameter of the deep neural network, the parameter is learned when the network is trained end to end, and LCERepresenting a cross entropy loss function, and simultaneously calculating the cross entropy loss of the normal sample and the corresponding confrontation sample, so that the network can successfully classify the normal sample and the confrontation sample; x is the number ofcleanDenotes a normal sample, xadvRepresenting a challenge sample, ytrueThe correct tag that represents the data is,
Figure FDA0002628485780000031
and
Figure FDA0002628485780000032
image feature representations extracted from the normal sample and the confrontation sample at the l-th layer of the deep neural network F are respectively represented, l is 1, 2 or 1, 2, 3, 4; LC represents linear combination of features; lambda represents the relative weight between multiple loss functions, and when the model is trained by using the training set, the final loss function calculates the classification error and the difference between the sample characteristics in different fields, and then the final loss function is used according to the errorsAnd optimizing the model parameters of the network by using a stochastic gradient descent algorithm, and finally finding the optimal model parameters.
CN202010809932.6A 2020-08-12 2020-08-12 Robust feature learning method based on layered feature alignment Active CN111950635B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010809932.6A CN111950635B (en) 2020-08-12 2020-08-12 Robust feature learning method based on layered feature alignment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010809932.6A CN111950635B (en) 2020-08-12 2020-08-12 Robust feature learning method based on layered feature alignment

Publications (2)

Publication Number Publication Date
CN111950635A true CN111950635A (en) 2020-11-17
CN111950635B CN111950635B (en) 2023-08-25

Family

ID=73331806

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010809932.6A Active CN111950635B (en) 2020-08-12 2020-08-12 Robust feature learning method based on layered feature alignment

Country Status (1)

Country Link
CN (1) CN111950635B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112465019A (en) * 2020-11-26 2021-03-09 重庆邮电大学 Countermeasure sample generation and countermeasure defense method based on disturbance
CN113436073A (en) * 2021-06-29 2021-09-24 中山大学 Real image super-resolution robust method and device based on frequency domain

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018028255A1 (en) * 2016-08-11 2018-02-15 深圳市未来媒体技术研究院 Image saliency detection method based on adversarial network
US20190128989A1 (en) * 2017-11-01 2019-05-02 Siemens Healthcare Gmbh Motion artifact reduction of magnetic resonance images with an adversarial trained network
US20190303720A1 (en) * 2018-03-30 2019-10-03 Arizona Board Of Regents On Behalf Of Arizona State University Systems and methods for feature transformation, correction and regeneration for robust sensing, transmission, computer vision, recognition and classification
CN110674866A (en) * 2019-09-23 2020-01-10 兰州理工大学 Method for detecting X-ray breast lesion images by using transfer learning characteristic pyramid network
CN110728219A (en) * 2019-09-29 2020-01-24 天津大学 3D face generation method based on multi-column multi-scale graph convolution neural network
CN110738622A (en) * 2019-10-17 2020-01-31 温州大学 Lightweight neural network single image defogging method based on multi-scale convolution
CN111126258A (en) * 2019-12-23 2020-05-08 深圳市华尊科技股份有限公司 Image recognition method and related device
CN111178504A (en) * 2019-12-17 2020-05-19 西安电子科技大学 Information processing method and system of robust compression model based on deep neural network
CN111242227A (en) * 2020-01-16 2020-06-05 天津师范大学 Multi-modal foundation cloud identification method based on heterogeneous depth features
US20200234110A1 (en) * 2019-01-22 2020-07-23 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN111476200A (en) * 2020-04-27 2020-07-31 华东师范大学 Face de-identification generation method based on generation of confrontation network

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018028255A1 (en) * 2016-08-11 2018-02-15 深圳市未来媒体技术研究院 Image saliency detection method based on adversarial network
US20190128989A1 (en) * 2017-11-01 2019-05-02 Siemens Healthcare Gmbh Motion artifact reduction of magnetic resonance images with an adversarial trained network
US20190303720A1 (en) * 2018-03-30 2019-10-03 Arizona Board Of Regents On Behalf Of Arizona State University Systems and methods for feature transformation, correction and regeneration for robust sensing, transmission, computer vision, recognition and classification
US20200234110A1 (en) * 2019-01-22 2020-07-23 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN110674866A (en) * 2019-09-23 2020-01-10 兰州理工大学 Method for detecting X-ray breast lesion images by using transfer learning characteristic pyramid network
CN110728219A (en) * 2019-09-29 2020-01-24 天津大学 3D face generation method based on multi-column multi-scale graph convolution neural network
CN110738622A (en) * 2019-10-17 2020-01-31 温州大学 Lightweight neural network single image defogging method based on multi-scale convolution
CN111178504A (en) * 2019-12-17 2020-05-19 西安电子科技大学 Information processing method and system of robust compression model based on deep neural network
CN111126258A (en) * 2019-12-23 2020-05-08 深圳市华尊科技股份有限公司 Image recognition method and related device
CN111242227A (en) * 2020-01-16 2020-06-05 天津师范大学 Multi-modal foundation cloud identification method based on heterogeneous depth features
CN111476200A (en) * 2020-04-27 2020-07-31 华东师范大学 Face de-identification generation method based on generation of confrontation network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
范宝杰，等: "基于卷积对抗网络的多通道图像修复方法", 《计算机应用与软件》, vol. 37, no. 7, pages 176 - 179 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112465019A (en) * 2020-11-26 2021-03-09 重庆邮电大学 Countermeasure sample generation and countermeasure defense method based on disturbance
CN113436073A (en) * 2021-06-29 2021-09-24 中山大学 Real image super-resolution robust method and device based on frequency domain

Also Published As

Publication number Publication date
CN111950635B (en) 2023-08-25

Similar Documents

Publication Publication Date Title
CN111275115B (en) Method for generating counterattack sample based on generation counternetwork
CN110941794B (en) Challenge attack defense method based on general inverse disturbance defense matrix
CN113554089B (en) Image classification countermeasure sample defense method and system and data processing terminal
CN111461307A (en) General disturbance generation method based on generation countermeasure network
CN112396129B (en) Challenge sample detection method and universal challenge attack defense system
CN107844743A (en) A kind of image multi-subtitle automatic generation method based on multiple dimensioned layering residual error network
CN113283599B (en) Attack resistance defense method based on neuron activation rate
CN111967006A (en) Adaptive black box anti-attack method based on neural network model
CN113627543B (en) Anti-attack detection method
CN115860112B (en) Model inversion method-based countermeasure sample defense method and equipment
CN111950635A (en) Robust feature learning method based on hierarchical feature alignment
Suzuki et al. Adversarial example generation using evolutionary multi-objective optimization
CN116250020A (en) Detecting an antagonism example using a potential neighborhood graph
CN114419413A (en) Method for constructing sensing field self-adaptive transformer substation insulator defect detection neural network
CN111178504B (en) Information processing method and system of robust compression model based on deep neural network
CN113033822A (en) Antagonistic attack and defense method and system based on prediction correction and random step length optimization
CN115239760B (en) Target tracking method, system, equipment and storage medium
CN114387449A (en) Image processing method and system for coping with adversarial attack of neural network
CN116912568A (en) Noise-containing label image recognition method based on self-adaptive class equalization
CN112487933B (en) Radar waveform identification method and system based on automatic deep learning
CN116109649A (en) 3D point cloud instance segmentation method based on semantic error correction
CN113177599B (en) Reinforced sample generation method based on GAN
CN115062306A (en) Black box anti-attack method for malicious code detection system
CN113487506A (en) Countermeasure sample defense method, device and system based on attention denoising
CN115510986A (en) Countermeasure sample generation method based on AdvGAN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant