WO2021052159A1 - Adversarial transfer learning-based face beauty prediction method and device - Google Patents

Abstract

An adversarial transfer learning-based face beauty prediction method and device, the method comprising the following steps: screening the most relevant auxiliary tasks from among a plurality of face factor recognition auxiliary tasks by means of similarity measurement, and on this basis, constructing a first face beauty prediction model; transferring general feature parameters formed after by pre-training an adversarial network to a second face beauty prediction model; and inputting a face image to be measured to perform recognition. The present method reduces the training cost of pre-training, reduces the negative transfer caused by auxiliary tasks with unrelated factors, and also reduces the calculation complexity for retraining the second face beauty prediction model, so that a more accurate model is obtained using fewer training images.

Description

Face beauty prediction method and device based on anti-migration learning Technical field

The invention relates to the field of image processing, in particular to a method and device for predicting the beauty of a face based on anti-migration learning.

Background technique

Face beauty prediction technology is widely used in the field of photography. At the same time, with the development of deep learning technology, the application of deep learning technology to facial beauty prediction technology makes facial beauty prediction results more accurate and more in line with people's cognition. However, single-task learning ignores the association between tasks, and multi-task learning adds unnecessary combinations to the deep learning network, which increases the redundancy of deep learning tasks, and also increases the burden of network training, which seriously affects The efficiency of classification and recognition.

Summary of the invention

The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, to provide a method and device for predicting the beauty of a face based on multi-task migration, and to reduce the amount of calculation through the similarity measurement.

The technical solutions adopted by the present invention to solve its problems are:

In the first aspect of the present invention, the method for predicting the beauty of a face based on anti-migration learning includes the following steps:

Measure the similarity between N auxiliary tasks and the main task, and obtain A auxiliary tasks with the highest similarity. The main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the beauty factors of the face, N>A;

Establish A first face beauty prediction model corresponding to A secondary tasks with the highest similarity, and establish a second face beauty prediction model for face beauty prediction;

Taking A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain, through adversarial network pre-training to find the common feature parameters of the source domain relative to the target domain, the common feature The parameters are transferred to the second face beauty prediction model;

Input the face image to be tested to the retrained second face beauty prediction model to output the face beauty prediction result.

According to the first aspect of the present invention, measuring the similarity between the N auxiliary tasks and the main task, and obtaining A auxiliary tasks with the highest similarity includes the following steps:

Construct a fully-supervised specific network for the N auxiliary tasks and the main task respectively and perform training to obtain the characteristic expression E _s (I) of each task;

Construct a migration network between the N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task, and the task tightness is calculated as follows:

Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

Use the analytic hierarchy process to normalize the loss of the migration network to obtain an incidence matrix;

Optimize the incidence matrix to get A auxiliary tasks with the highest similarity.

According to the first aspect of the present invention, the first face beauty prediction model includes a first preprocessing layer for preprocessing face images connected in sequence, a first feature sharing layer for extracting shared image features, and A first independent feature extraction layer that extracts independent features from shared image features, and a first classification layer; the second face beauty prediction model includes a second pre-processing layer connected in sequence, a second feature sharing layer, and a second independent The feature extraction layer is used to fuse independent features and the geometric features and texture features of the corresponding facial beauty prediction task, and the second classification layer.

According to the first aspect of the present invention, the A first face beauty prediction model is used as the source domain and the second face beauty prediction model is used as the target domain, and the counter network is pre-trained to find the source domain relative to the target domain Migrating the universal feature parameters to the second face beauty prediction model specifically includes the following steps:

Extraction step: extract the source feature f ^s =G(x ^s ) corresponding to the face image input to the first face beauty prediction model and the target feature f ^t =G( x ^t );

Mapping step: map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

Distinguish step: distinguish the source of the target feature and the pseudo-target feature and calculate the error through the loss function;

Optimization step: use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo-target feature, and then combine the error to optimize the mapping of the source feature to the target feature space;

Repeat the mapping step, the distinguishing step and the optimization step until the source domain and the target domain are adapted to obtain the common feature parameters;

Migration step: Migrate the general feature parameters to the second face beauty prediction model.

According to the first aspect of the present invention, the fusion method of the feature fusion layer to fuse geometric features, texture features, and independent features is summation, and the calculation method is F _fusion =[F _CNN ,G,H], where F _fusion is Fusion features, F _CNN is an independent feature, G is a geometric feature, H is a texture feature.

The above-mentioned face beauty prediction method based on anti-migration learning has at least the following beneficial effects: find the highest correlation among the auxiliary tasks of identifying face factors through the similarity measure, and construct the first face beauty prediction based on this. The model is pre-trained; the training cost of pre-training is reduced, the deviation caused by the auxiliary tasks with unrelated factors to the first face beauty prediction model is reduced, and the negative migration is avoided. The general feature parameters formed after the pre-training of the confrontation network are migrated to the second face beauty prediction model to achieve the final face beauty prediction. Through the confrontation migration learning, the calculation amount of the second face beauty prediction model training is reduced, and the training time is compressed. Use fewer training images to obtain a more accurate model effect.

In the second aspect of the present invention, a face beauty prediction device based on anti-transfer learning includes:

The similarity measurement module is used to measure the similarity between N auxiliary tasks and the main task to obtain A auxiliary tasks with the highest similarity. The main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the beauty factors of the face. N>A;

The first model establishment module is used to establish A first face beauty prediction model corresponding to A auxiliary tasks with the highest similarity;

The second model building module is used to build a second face beauty prediction model for face beauty prediction;

The parameter migration module is configured to use A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain to find the common feature parameters of the source domain relative to the target domain through pre-training of the confrontation network , Migrating the general feature parameters to the second face beauty prediction model;

The calculation module is used to input the face image to be tested to the second face beauty prediction model that is retrained to output the face beauty prediction result.

According to the second aspect of the present invention, the similarity measurement module includes:

The feature expression acquisition module is used to construct a fully-supervised specific network for the N auxiliary tasks and the main task respectively and perform training to obtain the feature expression E _s (I) of each task;

The tightness measurement module is used to construct a migration network between N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task, and the task tightness is calculated as follows:

Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

The normalization processing module is used to normalize the loss of the migration network through the analytic hierarchy process to obtain the correlation matrix;

The optimization processing module is used to optimize the incidence matrix to obtain A auxiliary tasks with the highest similarity.

According to the second aspect of the present invention, the first face beauty prediction model includes a first preprocessing layer for preprocessing face images that are sequentially connected, a first feature sharing layer for extracting shared image features, and A first independent feature extraction layer that extracts independent features from shared image features, and a first classification layer; the second face beauty prediction model includes a second pre-processing layer connected in sequence, a second feature sharing layer, and a second independent The feature extraction layer is used to fuse independent features and the geometric features and texture features of the corresponding facial beauty prediction task, and the second classification layer.

According to the second aspect of the present invention, the parameter migration module includes:

The first extraction module is used to extract the source feature f ^s =G(x ^s ) of the face image corresponding to the input first face beauty prediction model;

^{The second extraction module is used to extract the target feature f t} =G(x ^t ) of the face image corresponding to the input to the second face beauty prediction model;

The mapping module is used to map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

The distinguishing module is used to distinguish the source of the target feature and the pseudo-target feature and calculate the error through the loss function;

The optimization module is used to use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo-target feature, and then combine the error to optimize the source feature to the target feature space Mapping

The parameter acquisition module is used to acquire general characteristic parameters when both the source domain and the target domain are adapted;

The migration sub-module is used to migrate the general feature parameters to the second face beauty prediction model.

_{According to the second aspect of the present invention, the calculation method of the fusion} of geometric features, texture features, and independent features by the feature fusion layer is F fusion =[F _CNN ,G,H], where F _fusion is the fusion feature, and F _CNN is an independent feature, G is a geometric feature, and H is a texture feature.

The above-mentioned facial beauty prediction device based on anti-migration learning has at least the following beneficial effects: the similarity measurement module finds the highest correlation among the auxiliary tasks of identifying facial factors, and the pre-training module builds the first The face beauty prediction model is pre-trained; the training cost of pre-training is reduced, and the deviation caused by the auxiliary tasks with irrelevant factors to the first face beauty prediction model is avoided, and negative migration is avoided. The parameter migration module migrates the general migration parameters formed after the pre-training of the confrontation network to the second face beauty prediction model, and realizes the final face beauty prediction. Through the confrontation migration learning, the calculation amount is reduced, the training time is compressed, and the calculation module is used more. Fewer training images can get a more accurate model effect.

Description of the drawings

The present invention will be further explained below with reference to the drawings and examples.

FIG. 1 is a step diagram of a face beauty prediction method based on anti-migration learning according to an embodiment of the present invention;

FIG. 2 is a specific step diagram of step S10;

FIG. 3 is a schematic diagram of a face beauty prediction method based on anti-migration learning according to an embodiment of the present invention;

FIG. 4 is another schematic diagram of a face beauty prediction method based on anti-migration learning according to an embodiment of the present invention;

FIG. 5 is a specific step diagram of step S30;

FIG. 6 is a structural diagram of an apparatus for predicting face beauty based on anti-migration learning according to an embodiment of the present invention;

Figure 7 is a structural diagram of the parameter migration module.

detailed description

This section will describe the specific embodiments of the present invention in detail. The preferred embodiments of the present invention are shown in the accompanying drawings. The function of the accompanying drawings is to supplement the description of the text part of the manual with graphics, so that people can understand the present invention intuitively and vividly. Each technical feature and overall technical solution of the invention cannot be understood as a limitation of the protection scope of the present invention.

In the description of the present invention, if the first and second are described for the purpose of distinguishing technical features, they cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating The precedence of the indicated technical characteristics.

In the description of the present invention, unless otherwise clearly defined, terms such as setting, installation, and connection should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above terms in the present invention in combination with the specific content of the technical solution.

Referring to Fig. 1, an embodiment of the present invention provides a face beauty prediction method based on anti-migration learning, which includes the following steps:

Step S10, measure the similarity between the N auxiliary tasks and the main task, and obtain A auxiliary tasks with the highest similarity, where the main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the beauty factors of the face, N>A;

Step S20: Establish A first face beauty prediction model corresponding to A secondary tasks with the highest similarity, and establish a second face beauty prediction model for face beauty prediction;

Step S30: Use A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain. Through pre-training of the confrontation network to find the common feature parameters of the source domain relative to the target domain, Transferring the general feature parameters to the second face beauty prediction model;

Step S40: Input the face image to be tested to the second trained face beauty prediction model to output the face beauty prediction result.

In this embodiment, the similarity measure is used to find the most relevant among the auxiliary tasks of recognizing face factors, and based on this, the first face beauty prediction model is constructed for pre-training; the training cost of pre-training is reduced, Reduce the bias caused by the auxiliary tasks with irrelevant factors to the first face beauty prediction model, and avoid negative transfer. The auxiliary tasks of multiple recognition of facial factors mainly include recognition of facial expression, age, gender, skin color, eye size, eye distance, ear shape, nose size, nose bridge height, lip shape, etc.

Transfer learning is to improve the learning of new tasks by transferring knowledge from related tasks that have been learned. Migrating the general feature parameters formed after the adversarial network pre-training to the second face beauty prediction model can effectively reduce the calculation amount of the second face beauty prediction model training, compress the training time, and achieve more accurate results with fewer training images The effect of the model.

Referring to FIG. 2, further, step S10 specifically includes the following steps:

Step S11. Construct a fully-supervised specific network for each of the N auxiliary tasks and the main task and perform training to obtain the feature expression E _s (I) of each task. Each specific network has an encoder and a decoder, and all codes are The decoders all have the same ResNet50 structure, and the decoders correspond to different tasks;

Step S12. Construct a migration network between N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task. The task tightness calculation method is:

Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

Step S13: Normalize the loss of the migration network through the analytic hierarchy process to obtain the correlation matrix; specifically, for each task pair (i, j) of the source task pointing to the target task, the test set is taken out by the leave-out method after the migration ; For each task, construct a matrix W _t , and then use the Laplace smoothing method to _{control the output result of the matrix W t} within the range [0.001, 0.999], and then transform to obtain an incidence matrix, which reflects the similarity between tasks Probability. The calculation method of each element w′ _{i, j} in W′ _{t is:}

Step S14: Perform optimization processing on the incidence matrix to obtain A auxiliary tasks with the highest similarity, that is, obtain a subgraph selection problem based on the incidence matrix.

Referring to FIG. 3, further, the first face beauty prediction model corresponding to each auxiliary task includes a first preprocessing layer 11 for preprocessing face images connected in sequence, and a first feature sharing layer for extracting shared image features. 12. A first independent feature extraction layer 13 for extracting independent features from shared image features, and a first classification layer 14. 4, in other embodiments, the first face beauty prediction model of all auxiliary tasks can be combined into one, multiple first preprocessing layers 11 are combined into one, and multiple first feature sharing layers 12 are combined into one. One, the shared image features of all tasks are placed in the first feature sharing layer 12; after the first feature sharing layer 12, multiple first independent feature extraction layers 13 and multiple first classification layers 14 are connected corresponding to different tasks.

The second face beauty prediction model includes a second preprocessing layer 21, a second feature sharing layer 22, and a second independent feature extraction layer 23, which are connected in sequence, and are used to fuse independent features with geometric features and textures corresponding to face beauty prediction tasks. The feature fusion layer 24 of features, and the second classification layer 25. Wherein, the first independent feature extraction layer 13 and the second independent feature extraction layer 23 both include 1 convolutional layer, 1 BN layer, 1 activation function layer, and 1 pooling layer that are connected in sequence, and the first classification layer 14 Both the second classification layer 25 and the second classification layer 25 include two fully connected layers.

5, further, step S30 specifically includes the following steps:

Step S31. Extraction step: The generator extracts the source feature f ^s =G(x ^s ) corresponding to the face image input to the first face beauty prediction model and the target feature corresponding to the face image input to the second face beauty prediction model f ^t =G(x ^t );

Step S32. Mapping step: map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

Step S33, distinguishing step: the discriminator distinguishes the source of the target feature and the pseudo-target feature and calculates the error through the loss function; wherein the loss function can be a commonly used loss function, such as a cross entropy loss function or a mean square error loss function;

Step S34, optimization step: use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo-target feature, and then combine the error to optimize the source feature to the target feature space The mapping;

Step S35, repeat the mapping step, the distinguishing step and the optimization step until the source domain and the target domain are adapted to obtain the common feature parameters; among them, the source domain and the target domain are adapted to each other, that is, the error of the loss function calculation is less than the set Threshold; the smaller the error, the closer the source domain to the target domain, the better the migration effect;

Step S36. Migration step: Migrate the general feature parameters to the second face beauty prediction model.

The confrontation network is used to reduce the distribution difference between the source domain and the target domain, that is, the distribution difference between the first face beauty prediction model and the second face beauty prediction model; to achieve the purpose of knowledge transfer and reuse from the source domain to the target domain.

In addition, the overall migration of the general feature parameters of the first face beauty prediction model to the second face beauty prediction model is specifically as follows: each of the first feature sharing layer 12, the first independent feature extraction layer 13, and the first classification layer 14 The parameters are correspondingly migrated to the second feature sharing layer 22, the second independent feature extraction layer 23, and the second classification layer 25; the entire second face beauty prediction model accepts the parameters of the migration learning, and the overall model is optimized.

Further, the fusion method of the feature fusion layer 24 to fuse geometric features, texture features and independent features is summation, and the calculation method is F _fusion = [F _CNN ,G,H], where F _fusion is the fusion feature and F _CNN is the independent feature Features, G is geometric feature, H is texture feature.

The above face beauty prediction method measures the similarity of multiple auxiliary tasks, selects the auxiliary tasks with high similarity to the main task, builds a pre-training network model based on this, and transfers the parameters of the pre-trained network model to the main task face In the beautiful recognition network model, the network model of the main task is optimized to avoid the negative transfer caused by useless parameters caused by the training of irrelevant auxiliary tasks, which can greatly reduce the amount of training, improve the efficiency of classification and recognition, and improve the accuracy of classification and recognition.

Referring to FIG. 6, another embodiment of the present invention, an apparatus for predicting face beauty based on anti-transfer learning, applying the above-mentioned face beauty prediction method includes:

The similarity measurement module 100 is used to measure the similarity between N auxiliary tasks and the main task to obtain A auxiliary tasks with the highest similarity, where the main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the factors of the beauty of the face , N>A;

The first model building module 210 is used to build A first face beauty prediction models corresponding to A auxiliary tasks with the highest similarity;

The second model building module 220 is used to build a second face beauty prediction model used for face beauty prediction;

The parameter migration module 300 is configured to use A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain to find the common features of the source domain relative to the target domain through adversarial network pre-training Parameters, transferring the general feature parameters to the second face beauty prediction model;

The calculation module 400 is used to input the face image to be tested to the second face beauty prediction model that is retrained to output the face beauty prediction result.

In this embodiment, the similarity measurement module is used to find the most relevant among the auxiliary tasks of identifying face factors, and to construct the first face beauty prediction model for pre-training; reducing the training cost of pre-training , To reduce the bias caused by the auxiliary tasks with irrelevant factors to the first face beauty prediction model, and avoid negative migration. The parameter migration module 300 migrates the general feature parameters formed after pre-training to the second face beauty prediction model, and realizes the final face beauty prediction, reduces the calculation amount through migration learning, compresses the training time, and reaches the calculation module 400 to use less The training images to obtain a more accurate model effect.

Referring to FIG. 6, further, the similarity measurement module 100 includes:

The feature expression acquisition module 110 is used to construct a fully-supervised specific network for the N auxiliary tasks and the main task respectively and perform training to obtain the feature expression E _s (I) of each task;

The tightness measurement module 120 is used to construct a migration network between N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task. The calculation method of the task tightness is:

Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

The normalization processing module 130 is used to normalize the loss of the migration network through the analytic hierarchy process to obtain an incidence matrix;

The optimization processing module 140 is used for optimizing the incidence matrix to obtain A auxiliary tasks with the highest similarity.

Further, the first face beauty prediction model includes a first preprocessing layer 11 for preprocessing face images, a first feature sharing layer 12 for extracting shared image features, and a first feature sharing layer 12 for extracting features from the shared image. The first independent feature extraction layer 13 of independent features, and the first classification layer 14; the second face beauty prediction model includes a second preprocessing layer 21, a second feature sharing layer 22, and a second independent feature extraction layer 23 connected in sequence , A feature fusion layer 24 and a second classification layer 25 used to fuse independent features and geometric features and texture features of the corresponding facial beauty prediction task.

Referring to FIG. 7, further, the parameter migration module 300 includes:

The first extraction module 310 is configured to extract the source feature f ^s =G(x ^s ) of the face image corresponding to the input first face beauty prediction model;

The second extraction module 320 is configured to extract the target feature f ^t =G(x ^t ) of the face image corresponding to the input second face beauty prediction model;

The mapping module 330 is used to map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

The distinguishing module 340 is used to distinguish the source of the target feature and the pseudo-target feature and calculate the error through the loss function;

The optimization module 350 is configured to use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo target feature, and then combine the error to optimize the source feature to the target feature space The mapping;

The parameter acquisition module 360 is used to acquire general characteristic parameters when both the source domain and the target domain are adapted;

The migration sub-module 370 is configured to migrate the general feature parameters to the second face beauty prediction model.

Further, the parameter migration module 300 correspondingly migrates the respective parameters of the first feature sharing layer 12, the first independent feature extraction layer 13, and the first classification layer 14 to the second feature sharing layer 22, the second independent feature extraction layer 23, and the second feature sharing layer. Classification layer 25.

Further, the calculation method of the feature fusion layer 24 for the fusion of geometric features, texture features and independent features is F _fusion =[F _CNN ,G,H], where F _fusion is the fusion feature, F _CNN is the independent feature, and G is Geometric feature, H is texture feature.

The above-mentioned facial beauty prediction device measures the similarity of multiple auxiliary tasks, selects the auxiliary tasks with high similarity to the main task, and builds a parameter migration network model based on the confrontation network based on this, and transfers the parameters to the general features generated by the network model The parameters are migrated to the main task face beauty recognition network model, so that the main task network model is optimized, and the negative transfer caused by useless parameters caused by the training of irrelevant auxiliary tasks can be avoided, which can greatly reduce the amount of training and improve the efficiency of classification and recognition. Improve classification and recognition accuracy.

Another embodiment of the present invention provides a storage medium that stores executable instructions, and the executable instructions enable a processor connected to the storage medium to perform facial beauty prediction methods based on the above-mentioned anti-migration learning-based face beauty prediction method. The image is processed to obtain the beautiful face recognition result.

The above are only preferred embodiments of the present invention. The present invention is not limited to the above-mentioned embodiments. As long as they achieve the technical effects of the present invention by the same means, they should fall within the protection scope of the present invention.

Claims (10)

1. The face beauty prediction method based on anti-migration learning is characterized in that it includes the following steps:

   Measure the similarity between N auxiliary tasks and the main task, and obtain A auxiliary tasks with the highest similarity. The main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the beauty factors of the face, N>A;

   Establish A first face beauty prediction model corresponding to A secondary tasks with the highest similarity, and establish a second face beauty prediction model for face beauty prediction;

   Taking A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain, through adversarial network pre-training to find the common feature parameters of the source domain relative to the target domain, the common feature The parameters are transferred to the second face beauty prediction model;

   Input the face image to be tested to the trained second face beauty prediction model to output the face beauty prediction result.
2. The face beauty prediction method based on adversarial migration learning according to claim 1, wherein the measuring the similarity between the N auxiliary tasks and the main task, and obtaining A auxiliary tasks with the highest similarity comprises the following steps:

   Construct a fully-supervised specific network for the N auxiliary tasks and the main task respectively and perform training to obtain the characteristic expression E _s (I) of each task;

   Construct a migration network between N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task, and the task tightness is calculated as follows:

   

   Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

   Use the analytic hierarchy process to normalize the loss of the migration network to obtain an incidence matrix;

   Optimize the incidence matrix to get A auxiliary tasks with the highest similarity.
3. The face beauty prediction method based on adversarial migration learning according to claim 1 or 2, wherein the first face beauty prediction model includes a first preprocessing layer for preprocessing face images connected in sequence , A first feature sharing layer for extracting shared image features, a first independent feature extraction layer for extracting independent features from shared image features, and a first classification layer; the second face beauty prediction model includes sequential connections The second preprocessing layer, the second feature sharing layer, and the second independent feature extraction layer are used to fuse independent features with geometric features and texture features corresponding to the facial beauty prediction task, and the second classification layer.
4. The method for predicting face beauty based on anti-migration learning according to claim 3, wherein the A first face beauty prediction models are used as the source domain and the second face beauty prediction model is used as the source domain. The target domain searches for the common feature parameters of the source domain relative to the target domain through adversarial network pre-training, and migrating the common feature parameters to the second face beauty prediction model specifically includes the following steps:

   Extraction step: extract the source feature f ^s =G(x ^s ) corresponding to the face image input to the first face beauty prediction model and the target feature f ^t =G( x ^t );

   Mapping step: map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

   Distinguish step: distinguish the source of the target feature and the pseudo-target feature and calculate the error through the loss function;

   Optimization step: use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo-target feature, and then combine the error to optimize the mapping of the source feature to the target feature space;

   Repeat the mapping step, the distinguishing step and the optimization step until the source domain and the target domain are adapted to obtain the common feature parameters;

   Migration step: Migrate the general feature parameters to the second face beauty prediction model.
5. The face beauty prediction method based on anti-migration learning according to claim 4, wherein the fusion method of the feature fusion layer fusing geometric features, texture features, and independent features is summation, and the calculation method is F _fusion =[ F _CNN , G, H], where F _fusion is a fusion feature, F _CNN is an independent feature, G is a geometric feature, and H is a texture feature.
6. The facial beauty prediction device based on anti-migration learning is characterized in that it includes:

   The similarity measurement module is used to measure the similarity between N auxiliary tasks and the main task to obtain A auxiliary tasks with the highest similarity. The main task is the task of predicting the beauty of the face, and the auxiliary task is the task of identifying the beauty factors of the face. N>A;

   The first model establishment module is used to establish A first face beauty prediction model corresponding to A auxiliary tasks with the highest similarity;

   The second model building module is used to build a second face beauty prediction model for face beauty prediction;

   The parameter migration module is configured to use A of the first face beauty prediction model as the source domain and the second face beauty prediction model as the target domain to find the common feature parameters of the source domain relative to the target domain through pre-training of the confrontation network , Migrating the general feature parameters to the second face beauty prediction model;

   The calculation module is used to input the face image to be tested to the second face beauty prediction model that is retrained to output the face beauty prediction result.
7. The device for predicting the beauty of faces based on adversarial transfer learning according to claim 6, wherein the similarity measurement module comprises:

   The feature expression acquisition module is used to construct a fully-supervised specific network for the N auxiliary tasks and the main task respectively and perform training to obtain the feature expression E _s (I) of each task;

   The tightness measurement module is used to construct a migration network between N auxiliary tasks and the main task, and measure the task tightness between the N auxiliary tasks and the main task, and the task tightness is calculated as follows:

   

   Where I is the input, D is the data set, f _t (I) is the true value of the t-th input I, L _t is the loss between the true value and the predicted value, and E _I∈D represents the expected value;

   The normalization processing module is used to normalize the loss of the migration network through the analytic hierarchy process to obtain the correlation matrix;

   The optimization processing module is used to optimize the incidence matrix to obtain A auxiliary tasks with the highest similarity.
8. The device for predicting face beauty based on adversarial transfer learning according to claim 6 or 7, wherein the first face beauty prediction model comprises a first preprocessing layer for preprocessing face images connected in sequence , A first feature sharing layer for extracting shared image features, a first independent feature extraction layer for extracting independent features from shared image features, and a first classification layer; the second face beauty prediction model includes sequential connections The second preprocessing layer, the second feature sharing layer, and the second independent feature extraction layer are used to fuse independent features with geometric features and texture features corresponding to the facial beauty prediction task, and the second classification layer.
9. The device for predicting the beauty of a face based on adversarial transfer learning according to claim 8, wherein the parameter transfer module comprises:

   The first extraction module is used to extract the source feature f ^s =G(x ^s ) of the face image corresponding to the input first face beauty prediction model;

   ^{The second extraction module is used to extract the target feature f t} =G(x ^t ) of the face image corresponding to the input to the second face beauty prediction model;

   The mapping module is used to map the source feature to the target feature space to obtain the pseudo target feature T(G(x ^s ));

   The distinguishing module is used to distinguish the source of the target feature and the pseudo-target feature and calculate the error through the loss function;

   The optimization module is used to use the regularization term r(G(x ^s ), T(G(x ^s ))) to measure the distance between the source feature and the pseudo-target feature, and then combine the error to optimize the source feature to the target feature space Mapping

   The parameter acquisition module is used to acquire general characteristic parameters when both the source domain and the target domain are adapted;

   The migration module is used to migrate the general feature parameters to the second face beauty prediction model.
10. The face beauty prediction device based on anti-migration learning according to claim 9, wherein the calculation method of the _fusion of geometric features, texture features, and independent features of the feature fusion layer is F fusion =[F _CNN ,G ,H], where F _fusion is a fusion feature, F _CNN is an independent feature, G is a geometric feature, and H is a texture feature.

Images