CN112115963A - Method for generating unbiased deep learning model based on transfer learning - Google Patents
Method for generating unbiased deep learning model based on transfer learning Download PDFInfo
- Publication number
- CN112115963A CN112115963A CN202010750897.5A CN202010750897A CN112115963A CN 112115963 A CN112115963 A CN 112115963A CN 202010750897 A CN202010750897 A CN 202010750897A CN 112115963 A CN112115963 A CN 112115963A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- learning model
- unbiased
- sample image
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013136 deep learning model Methods 0.000 title claims abstract description 121
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000013526 transfer learning Methods 0.000 title claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 43
- 238000013508 migration Methods 0.000 claims abstract description 8
- 230000005012 migration Effects 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 43
- 238000004364 calculation method Methods 0.000 claims description 10
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 5
- 238000012360 testing method Methods 0.000 description 13
- 238000013135 deep learning Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000001914 filtration Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000003094 perturbing effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for generating an unbiased deep learning model based on transfer learning, which comprises the following steps: (1) constructing an original data set with task labels and bias labels of sample images; (2) training a biased deep learning model by using an original data set; (3) constructing and training an anti-attack network, and attacking an original data set by using the trained anti-attack network without a bias data set; (4) training an initial unbiased deep learning model with the same structure as the biased deep learning model by using an unbiased data set; (5) and preparing a third feature extractor, and forming an unbiased deep learning model by the third feature extractor with the parameters and the parameters determined based on the migration learning strategy and a second classifier contained in the trained initial unbiased deep learning model so as to ensure the fairness of the deep learning model in automatic decision making according to the input image and improve the accuracy of image recognition.
Description
Technical Field
The invention belongs to the field of deep learning, and particularly relates to a method for generating an unbiased deep learning model based on transfer learning.
Background
Deep learning helps people make decisions automatically and solves many complex pattern recognition problems by virtue of the powerful inherent rule of the learning sample data set and the capability of highly abstracting features, so that the deep learning is applied to the fields of medical diagnosis, voice recognition, image recognition, natural language understanding, advertisement, credit, employment, education, criminal judicial law and the like, and plays a good role. With continuous exploration and innovation of researchers, the performance of deep learning is continuously improved, the application is more and more extensive, and the deep learning has profound influence on the daily life of people.
Although deep learning can help people obtain more accurate predictions, recent studies show that deep learning models can also have bias in automatic decision making, which may be manifested in: the probability of predicting that the black person is reported to crime again is far higher than that of the white person, the accuracy rate of the white person is far higher than that of the black person when the sex of one person in the picture is predicted, and the male proportion is far higher than that of the female when a software engineer is searched. In some important occasions, such as enterprises, if deep learning models are used for decision making, the bias may cause the enterprises to be in a high-risk business environment, and if the enterprises abandon the deep learning models, advantages may be lost and eliminated in business competition, because automatic decision support of deep learning is a trend of the era. Therefore, the bias existing in the deep learning model can cause many negative effects on the society, and the bias is deep into various fields, so that the study on the deep learning algorithm measurement and the fairness thereof is particularly important.
The main reasons for the bias of the deep learning model are that the sample data set has bias, the bias can be amplified by the deep learning model, and the bias is given to the evaluation of the deep learning model. Therefore, at present, the work of researchers for eliminating the bias of the deep learning model mainly includes preprocessing a sample data set to eliminate the bias, performing small-scale modification on parameters of the deep learning model to eliminate the bias existing in the model, and performing fairness evaluation on the deep learning model. However, in the existing method for eliminating the deep learning model bias, only one factor of the model bias is considered. For example, the bias is eliminated by directly preprocessing the sample data set, which has the problem that the trained model does not learn the bias-containing data set, so that the original bias-containing data can be very sensitive to some biased or irrelevant features when being recognized, and meanwhile, the influence of the bias is amplified by the deep learning model is not considered, so that the trained model can only eliminate a part of the bias.
In view of the bias of the deep learning model and the limitation of the existing bias elimination method, a method for generating the unbiased deep learning model based on the transfer learning is researched, and the unbiased deep learning model generated to help people to make automatic decisions has extremely important theoretical and practical significance.
Disclosure of Invention
The invention aims to provide a method for generating an unbiased deep learning model based on transfer learning. And automatically filtering the biased features when the deep learning model learns sample data through knowledge migration, thereby ensuring the fairness of the deep learning model when automatically making a decision according to an input image and improving the accuracy of image identification.
In order to achieve the purpose, the invention provides the following technical scheme:
a method for generating an unbiased deep learning model based on transfer learning comprises the following steps:
(1) acquiring a sample image, marking a task label and a bias label of the sample image, and constructing an original data set;
(2) training a biased deep learning model consisting of a first feature extractor and a first classifier by using image data and task labels in an original data set to obtain a trained biased deep learning model;
(3) constructing and training an anti-attack network, attacking the original data set by using the trained anti-attack network to obtain an unbiased data set corresponding to the original data set, so that bias labels in the unbiased data set cannot be predicted;
(4) training an initial unbiased deep learning model with the same structure as the biased deep learning model by using an unbiased data set;
(5) preparing a third feature extractor, constructing a loss function by using feature distribution extracted from the original sample image by the third feature extractor and feature distribution extracted from an unbiased image corresponding to the original sample image by using a second feature extractor of the unbiased deep learning model, optimizing parameters of the third feature extractor by using the loss function, and forming the unbiased deep learning model by the third feature extractor determined by the parameters and a second classifier contained in the trained initial unbiased deep learning model.
Preferably, the constructing and training of the anti-attack network comprises:
constructing an anti-attack network, wherein the anti-attack network comprises a convolution layer and a full connection layer, a ReLU function is adopted as an activation function, the input of the anti-attack network is the feature distribution of an original sample image extracted by a trained first feature extractor, the output of the anti-attack network is a logits layer, and the prediction probability distribution of a bias label of the original sample image is obtained through a softmax function;
constructing a Loss function Loss _ NAdv of the anti-attack network, wherein the Loss function aims to enable the anti-attack network to predict the probability distribution of the bias label according to the characteristic distribution corresponding to the original sample image, and the calculation formula is as follows:
wherein z isiIs the original sample image xiThe output of the first feature extractor of the trained biased deep learning model; b isiIs the original sample image xiTrue bias tags of (1); nadv (. cndot.) denotesAn output of the anti-attack network; l (-) represents a cross entropy function, i is the index of the original sample image, and N is the total number of the original sample images;
and training the anti-attack network by using the Loss function Loss _ NAdv so as to optimize the model parameters of the anti-attack network.
Preferably, attacking the original data set by using the trained countermeasure network to obtain an unbiased data set corresponding to the original data set includes:
(a) designing a disturbance variable r;
(b) the disturbance variable r is added to the original sample image xiObtaining a disturbance sample image, and extracting the disturbance feature distribution of the disturbance sample image by using a first feature extractor of a trained biased deep learning model;
(c) calculating disturbance characteristic distribution by using a trained anti-attack network to obtain predicted probability distribution, calculating Loss Loss _ Adv according to the predicted probability distribution, updating a disturbance variable r according to the Loss Loss _ Adv when the iteration times do not reach the maximum iteration times, skipping to execute the step (b), and outputting a disturbance sample image obtained by using the latest disturbance variable r as an unbiased image until the maximum iteration times are reached to form an unbiased data set;
the Loss _ Adv is calculated by the formula:
Loss_Adv＝-αLoss_NAdv+Loss_Y
wherein alpha is a hyper-parameter, the value range is 0-1, Loss _ Y is a Loss function value of the task label except the bias label, and the calculation formula is as follows:
wherein, c1(. represents the predicted output of the first classifier of the biased deep learning model, yiRepresenting the original sample image xiThe task tag of (1).
The Loss function Loss _ tl for optimizing the parameters of the third feature extractor is as follows:
Loss_tl＝∑L(h,h')
where h denotes the original sample image xiH' represents the feature distribution output by the third feature extractor, and the second feature extractor of the unbiased deep learning model performs on the original sample image xiAnd extracting the characteristic distribution of the corresponding unbiased image.
After the sample image is obtained, the sample image is rotated, turned over, color enhanced, added with Gaussian noise and randomly scaled to expand the sample image, and the bias labels comprise race labels, region labels and gender labels.
Preferably, the first and second feature extractors employ a ResNet-50 model;
the first classifier and the second classifier of the initial unbiased deep learning model employ a network composed of fully connected layers.
The first classifier and a second classifier of the initial unbiased deep learning model adopt a network consisting of 4 full connection layers;
the anti-attack network adopts a network consisting of 3 convolutional layers and 4 fully-connected layers, and the activation function adopts a ReLU function.
Preferably, the training parameters of the anti-attack network, the initial unbiased deep learning model and the third feature extractor are set as: the Batch size is set to 32, the maximum number of iterations trained is set to 60, the optimizer uses Adam, the learning rate is set to 0.001, and the first and second estimated exponential decay rates are set to 0.9 and 0.999, respectively.
Compared with the prior art, the invention has the beneficial effects that at least:
according to the method for generating the unbiased deep learning model based on the transfer learning, provided by the embodiment of the invention, the deep learning model can obtain the bias characteristic capability of automatically filtering sample data based on the strategy of the transfer learning, so that the fairness of model decision is ensured, and the accuracy of image identification is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for generating an unbiased deep learning model based on transfer learning according to an embodiment of the present invention;
FIG. 2 is a system framework diagram of an unbiased deep learning model based on transfer learning according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for generating an unbiased data set according to an embodiment of the present invention;
fig. 4 is a training flowchart of an unbiased deep learning model based on transfer learning according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The method aims to solve the problem that image identification is inaccurate due to the fact that a bias problem exists in a deep learning model. The embodiment provides a method for generating an unbiased deep learning model based on transfer learning, and as shown in fig. 1, the method for generating an unbiased deep learning model based on transfer learning includes the following steps:
(1) and (3) defining the deep learning model bias.
The invention defines the wrong decision caused by the dependence of the deep learning model on false relevant features during automatic decision making as the decision bias of the deep learning model. For example, when the bias features are gender, and the resume of the software engineer is screened by using a deep learning model, the rejection rate of women is often high, so that the discrimination of the profession to women and the unfairness of model decision are caused. Therefore, the invention eliminates the bias of the model by filtering the irrelevant features and then classifying.
(2) Data set preparation and preprocessing.
The embodiment selects an image dataset with multi-label classification, such as a COCO dataset, and uses one bias label B as a bias feature, such as a gender feature. One or more of other tags are selected as task tags, the task tags can be professional tags and the like, the data set is preprocessed, and in order to improve the recognition accuracy of the deep learning model, data enhancement operations can be added to expand the data set, wherein the operations comprise rotation, overturning, color enhancement, Gaussian noise addition and random scaling. And (3) forming an original data set by the images subjected to the data enhancement operation, dividing the original data set into a training set and a test set according to the proportion of 7:3, and respectively using the training set and the test set for the migration learning and the test of the unbiased model.
(3) And establishing and training a bias deep learning model.
In this embodiment, the constructed biased deep learning model includes two parts, namely a first feature extractor and a first classifier, where the first feature extractor adopts a ResNet-50 model structure, and the first classifier adopts a network formed by 4 full-connection layers. Training the biased deep learning model by using a training set of an original data set, and testing and optimizing the biased deep learning model by using a testing set to enable the biased deep learning model to reach a preset identification accuracy rate. Since the original data set used for training contains bias features, the model obtained by training is called a bias deep learning model.
(4) And constructing and training the anti-attack network.
The anti-attack network aims to predict the bias label B through the output characteristics of the first characteristic extractor of the biased deep learning model, and is mainly used for generating unbiased data sets. The specific process is as follows:
and (4.1) constructing a structure of the anti-attack network, wherein the anti-attack network adopts a network consisting of 3 convolutional layers and 4 full-connection layers, and the activation function adopts a ReLU function. The input of the anti-attack network is the output z of the original data set through the first feature extractor of the biased deep learning model in the step (3), wherein z represents the feature distribution of the original data set. The output of the anti-attack network is a logits layer, and the predicted probability distribution of the bias labels of the original data set is obtained through a softmax function.
(4.2) designing a loss function of the anti-attack network, wherein the loss function aims to enable the anti-attack network to predict the probability distribution of the bias label according to the corresponding characteristic distribution of the original sample image, and the calculation formula is as follows:
wherein z isiIs the original sample image xiOutputting by a first feature extractor of the biased deep learning model in the step (3); b isiIs the original sample image xiTrue bias tags of (1); nadv (·) represents the output of the anti-attack network; l (-) represents a cross entropy function.
(4.3) training the anti-attack network, setting the Batch size to be 32, training the maximum iteration number to be 60, adopting Adam by the optimizer, setting the learning rate to be 0.001, and setting the exponential decay rates of the first estimation and the second estimation to be 0.9 and 0.999 respectively. And training the anti-attack network by using the training set of the original data set and the bias label B thereof, and testing and optimizing the anti-attack network by using the test set so as to ensure that the anti-attack network reaches the preset identification accuracy.
(5) And generating an unbiased data set, and attacking the original data set by using an anti-attack network, wherein the attack is that the biased label B of the generated unbiased data set is unpredictable by adding disturbance in the original data set, but other labels are not influenced. Wherein the original data sets correspond one-to-one to the unbiased data sets.
Designing a loss function for resisting attacks, wherein the calculation formula is as follows:
Loss_Adv＝-αLoss_NAdv+Loss_Y (2)
wherein, α is a hyper-parameter, and Loss _ Y is a Loss function value of other classification labels except the bias label B, and the calculation formula is as follows:
wherein, c1(. h) represents the output of the first classifier of the biased deep learning model in step (3); y isiRepresenting the original sample image xiThe real tag of (1).
And designing a disturbance variable r, wherein r is a matrix of weight multiplied by high multiplied by 3, wherein the weight and the high are respectively the width and the height of the sample data image, 3 represents three channels of RGB of the sample data image, and the disturbance variable r is initialized to a zero matrix. And (4) optimizing the disturbance variable r by adopting an Adam optimizer, wherein the parameters of the optimizer are the same as those in the step (4.3).
Setting the maximum number of iterations of a single sample to be 1000, as shown in fig. 3, the specific process of generating unbiased data for each original data set sample is as follows:
(5.1) input original sample image xiTurning to step (5.2);
(5.2) original sample image xiObtaining a disturbance sample image x by superposing with a disturbance variable ri', go to step (5.3);
(5.3) perturbing the sample image xi' input to the first feature extractor of the biased deep learning model in step (3) and output ziTurning to step (5.4);
(5.4) adding ziInputting the input into the anti-attack network of step (4), and outputting nadv (z)i) Turning to step (5.5);
(5.5) calculating a loss function value according to the formulas (1) to (3), and turning to the step (5.6);
(5.6) judging whether the maximum iteration number is reached, if so, outputting a disturbance sample image xi' as an unbiased image, forming an unbiased data set, ending the iteration, if not, turning to step (5.7);
(5.7) updating the disturbance variable r with Adam according to the loss function value, and turning to step (5.2).
(6) And (5) building and training an initial unbiased deep learning model.
And (4) the structure of the initial unbiased deep learning model is the same as that of the biased deep learning model in the step (3). That is, the initial unbiased deep learning model includes a second feature extractor having the same structure as the first feature extractor, and also includes a second classifier having the same result as the first classifier.
Setting the Batch size to 32, training the maximum number of iterations to 60, using Adam for the optimizer, setting the learning rate to 0.001, and setting the first and second estimated exponential decay rates to 0.9 and 0.999, respectively. And (5) training an initial unbiased deep learning model by using the unbiased data set generated in the step (5), and testing and optimizing the model by using the test set to enable the initial unbiased deep learning model to reach a preset identification accuracy. The output of the second feature extractor for recording the initial unbiased deep learning model is h', and the output of the second classifier for recording the initial unbiased deep learning model is c2(·)。
(7) And designing an unbiased deep learning model training framework based on the transfer learning.
As shown in fig. 2, the specific process of step (7) is:
(7.1) design unbiased deep learning model structure based on transfer learning, can adopt the third feature extractor, its input is the original sample image xiThe output is the intermediate characteristic h; the third classifier adopts a second classifier c which is trained by an initial unbiased deep learning model2(·)；
(7.2) designing a loss function of the unbiased deep learning model based on the migration learning, wherein the loss function aims to enable the unbiased deep learning model to learn the knowledge of the initial unbiased deep learning model, so that the unbiased deep learning model can automatically filter the biased features when facing the original data set, and the calculation formula of the loss function is as follows:
Loss_tl＝∑L(h,h') (4)
where h represents the output of the third feature extractor of the unbiased deep learning model and h' represents the output of the second feature extractor of the initial unbiased deep learning model.
(7.3) designing initialized training parameters, setting the Batch size to be 32, setting the maximum iteration number of training to be 60, adopting Adam by an optimizer, setting the learning rate to be 0.001, and setting the exponential decay rates of the first estimation and the second estimation to be 0.9 and 0.999 respectively.
(7.4) designing a training process of the unbiased deep learning model, and combining with the graph 4, wherein the specific process is as follows:
(7.4.1) original sample image xiInputting the h into a third feature extractor and outputting the h, and turning to the step (7.4.2);
(7.4.2) original sample image xiCorresponding unbiased image xiInputting the initial unbiased deep learning model into a second feature extractor and outputting h', and turning to the step (7.4.3);
(7.4.3) calculating a loss function value according to the formula (4), and turning to the step (7.4.4);
(7.4.4) updating unbiased deep learning model parameters according to the loss function values, and turning to the step (7.4.5);
(7.4.5) judging whether the maximum iteration times is reached, if so, saving the model and ending the training; if not, go to step (7.4.1).
(8) And training and testing the unbiased deep learning model.
And (4) according to the training process in the step (7.4), training a third feature extractor of the unbiased deep learning model by utilizing the original data set and the unbiased data set, and connecting the third feature extractor with the second classifier trained by the initial unbiased deep learning model in the step (6) after the training is finished to be used as the unbiased deep learning model, namely the unbiased deep learning model generated on the basis of transfer learning. Testing the bias degree lambda of an unbiased deep learning model generated based on transfer learning by using a test set of an original data set, wherein the smaller the bias degree lambda value of the model is, the more fair the model is in decision making, and the calculation formula is as follows:
wherein n represents the total number of sample data in the test set in the original data set; nadv (h)i) Representing sample data xiBy means of mobilityLearning a feature extractor of the unbiased model and outputting of an anti-attack network; function l [. C]Indicating a function, the value is 1 when the equation holds in parentheses, and is 0 otherwise.
The method for generating the unbiased deep learning model based on the migration learning provides a new unbiased model training framework, the unbiased deep learning model is generated through the knowledge migration of the unbiased model and the attack resisting network, and the unbiased model training method solves the problem that training data and model structure parameters are biased at the same time, so that the fairness of model decision is further ensured. The proposed migration learning unbiased model can select a simple structure, and greatly reduces the calculation time of the deep learning model in practical application. The provided strategy based on the transfer learning can ensure that the deep learning model can automatically filter bias characteristics under the condition of ensuring the precision of the original target classification task, ensure the fairness of the deep learning model in decision making and provide guidance for researching and eliminating the bias of the deep learning model.
The above-mentioned embodiments are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions, equivalents, etc. made within the scope of the principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for generating an unbiased deep learning model based on transfer learning is characterized by comprising the following steps:
(1) acquiring a sample image, marking a task label and a bias label of the sample image, and constructing an original data set;
(2) training a biased deep learning model consisting of a first feature extractor and a first classifier by using image data and task labels in an original data set to obtain a trained biased deep learning model;
(3) constructing and training an anti-attack network, attacking the original data set by using the trained anti-attack network to obtain an unbiased data set corresponding to the original data set, so that bias labels in the unbiased data set cannot be predicted;
(4) training an initial unbiased deep learning model with the same structure as the biased deep learning model by using an unbiased data set;
(5) preparing a third feature extractor, constructing a loss function by using feature distribution extracted from the original sample image by the third feature extractor and feature distribution extracted from an unbiased image corresponding to the original sample image by using a second feature extractor of the unbiased deep learning model, optimizing parameters of the third feature extractor by using the loss function, and forming the unbiased deep learning model by the third feature extractor determined by the parameters and a second classifier contained in the trained initial unbiased deep learning model.
2. The method for generating an unbiased deep learning model based on transfer learning of claim 1, wherein the constructing and training of the anti-attack network includes:
constructing an anti-attack network, wherein the anti-attack network comprises a convolution layer and a full connection layer, a ReLU function is adopted as an activation function, the input of the anti-attack network is the feature distribution of an original sample image extracted by a trained first feature extractor, the output of the anti-attack network is a logits layer, and the prediction probability distribution of a bias label of the original sample image is obtained through a softmax function;
constructing a Loss function Loss _ NAdv of the anti-attack network, wherein the Loss function aims to enable the anti-attack network to predict the probability distribution of the bias label according to the characteristic distribution corresponding to the original sample image, and the calculation formula is as follows:
wherein z isiIs the original sample image xiThe output of the first feature extractor of the trained biased deep learning model; b isiIs the original sample image xiTrue bias tags of (1); nadv (·) represents the output of the anti-attack network; l (-) represents a cross-entropy function,i is the index of the original sample image, and N is the total number of the original sample images;
and training the anti-attack network by using the Loss function Loss _ NAdv so as to optimize the model parameters of the anti-attack network.
3. The method for generating an unbiased deep learning model based on transfer learning of claim 2, wherein attacking the original data set with a trained confrontation network to obtain an unbiased data set corresponding to the original data set includes:
(a) designing a disturbance variable r;
(b) the disturbance variable r is added to the original sample image xiObtaining a disturbance sample image, and extracting the disturbance feature distribution of the disturbance sample image by using a first feature extractor of a trained biased deep learning model;
(c) calculating disturbance characteristic distribution by using a trained anti-attack network to obtain predicted probability distribution, calculating Loss Loss _ Adv according to the predicted probability distribution, updating a disturbance variable r according to the Loss Loss _ Adv when the iteration times do not reach the maximum iteration times, skipping to execute the step (b), and outputting a disturbance sample image obtained by using the latest disturbance variable r as an unbiased image until the maximum iteration times are reached to form an unbiased data set;
the Loss _ Adv is calculated by the formula:
Loss_Adv＝-αLoss_NAdv+Loss_Y
wherein alpha is a hyper-parameter, the value range is 0-1, Loss _ Y is a Loss function value of the task label except the bias label, and the calculation formula is as follows:
wherein, c1(. represents the predicted output of the first classifier of the biased deep learning model, yiRepresenting the original sample image xiThe task tag of (1).
4. The method for generating an unbiased deep learning model based on transfer learning of claim 1, wherein the penalty function Loss _ tl for optimizing the third feature extractor parameters is:
Loss_tl＝∑L(h,h')
where h denotes the original sample image xiH' represents the feature distribution output by the third feature extractor, and the second feature extractor of the unbiased deep learning model performs on the original sample image xiAnd extracting the characteristic distribution of the corresponding unbiased image.
5. The method for generating an unbiased deep learning model based on transfer learning of claim 1, wherein after the sample image is obtained, the sample image is rotated, flipped, color enhanced, gaussian noise added, and randomly scaled to expand the sample image, and the bias labels include race labels, region labels, and gender labels.
6. The method for generating an unbiased deep learning model based on transfer learning of claim 1, wherein the first feature extractor and the second feature extractor employ a ResNet-50 model;
the first classifier and the second classifier of the initial unbiased deep learning model employ a network composed of fully connected layers.
7. The method for generating an unbiased deep learning model based on migration learning of claim 1, wherein the first classifier and the second classifier of the initial unbiased deep learning model employ a network consisting of 4 fully connected layers;
the anti-attack network adopts a network consisting of 3 convolutional layers and 4 fully-connected layers, and the activation function adopts a ReLU function.
8. The method for generating an unbiased deep learning model based on transfer learning as claimed in claim 1, wherein the training parameters of the anti-attack network, the initial unbiased deep learning model and the third feature extractor are set as: the Batch size is set to 32, the maximum number of iterations trained is set to 60, the optimizer uses Adam, the learning rate is set to 0.001, and the first and second estimated exponential decay rates are set to 0.9 and 0.999, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010750897.5A CN112115963B (en) | 2020-07-30 | 2020-07-30 | Method for generating unbiased deep learning model based on transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010750897.5A CN112115963B (en) | 2020-07-30 | 2020-07-30 | Method for generating unbiased deep learning model based on transfer learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115963A true CN112115963A (en) | 2020-12-22 |
| CN112115963B CN112115963B (en) | 2024-02-20 |
Family
ID=73799596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010750897.5A Active CN112115963B (en) | 2020-07-30 | 2020-07-30 | Method for generating unbiased deep learning model based on transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115963B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112651459A (en) * | 2020-12-31 | 2021-04-13 | 厦门易仕特仪器有限公司 | Defense method, device, equipment and storage medium for confrontation sample of deep learning image |
| CN113077057A (en) * | 2021-04-20 | 2021-07-06 | 中国科学技术大学 | Unbiased machine learning method |
| CN113256621A (en) * | 2021-06-25 | 2021-08-13 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113297572A (en) * | 2021-06-03 | 2021-08-24 | 浙江工业大学 | Deep learning sample-level anti-attack defense method and device based on neuron activation mode |
| CN113298254A (en) * | 2021-06-10 | 2021-08-24 | 浙江工业大学 | Deskewing method and device for deep migration learning |
| CN113705787A (en) * | 2021-08-27 | 2021-11-26 | 电子科技大学 | Digital modulation signal identification method based on deep collaborative training |
| CN113988291A (en) * | 2021-10-26 | 2022-01-28 | 支付宝(杭州)信息技术有限公司 | Training method and device for user representation network |
| CN114419379A (en) * | 2022-03-30 | 2022-04-29 | 浙江大学 | System and method for improving fairness of deep learning model based on antagonistic disturbance |
| CN114848019A (en) * | 2022-04-19 | 2022-08-05 | 山东大学 | Automatic heart ultrasonic detection system and method based on flexible cooperative robot |
| CN115017290A (en) * | 2022-07-15 | 2022-09-06 | 浙江星汉信息技术股份有限公司 | File question-answering system optimization method and device based on cooperative confrontation training |
| WO2022247448A1 (en) * | 2021-05-25 | 2022-12-01 | 华为云计算技术有限公司 | Data processing method and apparatus, computing device, and computer readable storage medium |
| CN115937994A (en) * | 2023-01-06 | 2023-04-07 | 南昌大学 | Data detection method based on deep learning detection model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520202A (en) * | 2018-03-15 | 2018-09-11 | 华南理工大学 | Confrontation robustness image characteristic extracting method based on variation spherical projection |
| CN109816032A (en) * | 2019-01-30 | 2019-05-28 | 中科人工智能创新技术研究院(青岛)有限公司 | Zero sample classification method and apparatus of unbiased mapping based on production confrontation network |
| CN110837570A (en) * | 2019-11-12 | 2020-02-25 | 北京交通大学 | Method for unbiased classification of image data |
-
2020
- 2020-07-30 CN CN202010750897.5A patent/CN112115963B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520202A (en) * | 2018-03-15 | 2018-09-11 | 华南理工大学 | Confrontation robustness image characteristic extracting method based on variation spherical projection |
| CN109816032A (en) * | 2019-01-30 | 2019-05-28 | 中科人工智能创新技术研究院(青岛)有限公司 | Zero sample classification method and apparatus of unbiased mapping based on production confrontation network |
| CN110837570A (en) * | 2019-11-12 | 2020-02-25 | 北京交通大学 | Method for unbiased classification of image data |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112651459A (en) * | 2020-12-31 | 2021-04-13 | 厦门易仕特仪器有限公司 | Defense method, device, equipment and storage medium for confrontation sample of deep learning image |
| CN113077057A (en) * | 2021-04-20 | 2021-07-06 | 中国科学技术大学 | Unbiased machine learning method |
| CN113077057B (en) * | 2021-04-20 | 2022-09-06 | 中国科学技术大学 | Unbiased machine learning method |
| WO2022247448A1 (en) * | 2021-05-25 | 2022-12-01 | 华为云计算技术有限公司 | Data processing method and apparatus, computing device, and computer readable storage medium |
| CN113297572A (en) * | 2021-06-03 | 2021-08-24 | 浙江工业大学 | Deep learning sample-level anti-attack defense method and device based on neuron activation mode |
| CN113297572B (en) * | 2021-06-03 | 2022-05-17 | 浙江工业大学 | Deep learning sample-level anti-attack defense method and device based on neuron activation mode |
| CN113298254A (en) * | 2021-06-10 | 2021-08-24 | 浙江工业大学 | Deskewing method and device for deep migration learning |
| CN113256621A (en) * | 2021-06-25 | 2021-08-13 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113256621B (en) * | 2021-06-25 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113705787A (en) * | 2021-08-27 | 2021-11-26 | 电子科技大学 | Digital modulation signal identification method based on deep collaborative training |
| CN113988291A (en) * | 2021-10-26 | 2022-01-28 | 支付宝(杭州)信息技术有限公司 | Training method and device for user representation network |
| CN113988291B (en) * | 2021-10-26 | 2024-06-04 | 支付宝(杭州)信息技术有限公司 | Training method and device for user characterization network |
| CN114419379A (en) * | 2022-03-30 | 2022-04-29 | 浙江大学 | System and method for improving fairness of deep learning model based on antagonistic disturbance |
| CN114848019A (en) * | 2022-04-19 | 2022-08-05 | 山东大学 | Automatic heart ultrasonic detection system and method based on flexible cooperative robot |
| CN115017290A (en) * | 2022-07-15 | 2022-09-06 | 浙江星汉信息技术股份有限公司 | File question-answering system optimization method and device based on cooperative confrontation training |
| CN115937994A (en) * | 2023-01-06 | 2023-04-07 | 南昌大学 | Data detection method based on deep learning detection model |
| CN115937994B (en) * | 2023-01-06 | 2023-05-30 | 南昌大学 | Data detection method based on deep learning detection model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115963B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112115963A (en) | Method for generating unbiased deep learning model based on transfer learning | |
| CN110598029B (en) | Fine-grained image classification method based on attention transfer mechanism | |
| CN110048827B (en) | Class template attack method based on deep learning convolutional neural network | |
| CN112395442B (en) | Automatic identification and content filtering method for popular pictures on mobile internet | |
| CN111753918B (en) | Gender bias-removed image recognition model based on countermeasure learning and application | |
| CN115131618B (en) | Semi-supervised image classification method based on causal reasoning | |
| CN111984817B (en) | Fine-grained image retrieval method based on self-attention mechanism weighting | |
| Li et al. | Robust deep neural networks for road extraction from remote sensing images | |
| CN114998603B (en) | Underwater target detection method based on depth multi-scale feature factor fusion | |
| CN109903339A (en) | A kind of video group personage's position finding and detection method based on multidimensional fusion feature | |
| CN114998615B (en) | Collaborative saliency detection method based on deep learning | |
| CN112883931A (en) | Real-time true and false motion judgment method based on long and short term memory network | |
| CN114708637A (en) | Face action unit detection method based on meta-learning | |
| CN112733602A (en) | Relation-guided pedestrian attribute identification method | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN116433909A (en) | Similarity weighted multi-teacher network model-based semi-supervised image semantic segmentation method | |
| CN116503618B (en) | Method and device for detecting remarkable target based on multi-mode and multi-stage feature aggregation | |
| CN116664623A (en) | Video target long-term tracking method based on twin network joint tracking and detection | |
| Anggoro et al. | Classification of Solo Batik patterns using deep learning convolutional neural networks algorithm | |
| CN115223033A (en) | Synthetic aperture sonar image target classification method and system | |
| CN111860441B (en) | Video target identification method based on unbiased depth migration learning | |
| CN114462466A (en) | Deep learning-oriented data depolarization method | |
| CN111754459A (en) | Dyeing counterfeit image detection method based on statistical depth characteristics and electronic device | |
| Khan et al. | The Art of Seeing: A Computer Vision Journey into Object Detection | |
| CN114973357B (en) | Facial expression recognition method based on double-label learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |