CN111275651B - Face bright removal method based on antagonistic neural network - Google Patents

Abstract

The invention discloses a method for removing light from a human face based on an adversarial neural network. The method includes: synthesizing a human face through a method of computer graphics, and establishing a virtual human face data set corresponding to a virtual human face with light and a human face with no light , obtain the real face data set corresponding to the real face with light and face without light through ordinary shooting and shooting with polarized lenses; build an adversarial neural network, use the data set to train network parameters, and obtain a network model with light removal, there will be The light face image is input to the trained light removal network model, and the light-free image of the face is obtained. The present invention can realize more natural and real human face light removal by fighting against the neural network.

Description

A method for removing light from human face based on adversarial neural network

Technical Field

The invention relates to the field of pattern recognition, and in particular to a method for removing light from a human face based on an adversarial neural network.

Background Art

The skin surface will reflect light due to the irradiation of the ambient light source, which is manifested as shiny skin in the image. The shiny skin surface covers up the original color and texture of the part, and the local shiny skin on the face has an impact on computer vision, computer graphics and medical examination. In computer vision, the shiny skin on the face will bring difficulties to face detection and face recognition, resulting in problems such as failure to detect and recognition errors; in computer graphics, the shiny skin on the real face needs to be consistent with the environment. Under the light source that does not produce shiny skin, the skin should not be highlighted; in medical examination, it will interfere with the detection of facial pigmented skin diseases. It is difficult to distinguish whether the skin covered by shiny skin has such skin diseases, which hinders the diagnosis of the disease and the design of treatment plans. Therefore. It is necessary to remove the shiny skin on the face.

Since the skin material is relatively complex, general methods for removing light are not applicable. Existing research on light removal technology has not yet achieved the goal of naturally and realistically removing light from the face skin, especially when there are dense spots such as freckles on the face. While naturally removing the light, it is necessary to restore the original skin and the spots, moles and other features on the skin as much as possible. Therefore, removing light from the face is a complex and challenging problem. The light removal method that establishes a light model is limited by the physical light model and cannot fully reflect the real light environment, especially for complex materials such as face and skin. Therefore, the effect of removing light is not realistic enough.

With the rise of artificial intelligence technology, deep learning algorithms are constantly developing. Compared with the de-illumination method by establishing a lighting model, the end-to-end neural network can directly predict the corresponding lightless image based on the original illuminated image. It is not restricted by a fixed lighting model and can better and more realistically restore the skin of the illuminated part of the face.

Summary of the invention

Purpose of the invention: The technical problem to be solved by the present invention is to propose a method for removing light from the face based on an adversarial neural network. This method can use an end-to-end neural network to repair the light areas on the face, so as to naturally and realistically remove the light areas on the face and restore the original skin color and texture of the corresponding part. The method is suitable for all kinds of light on the face.

Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: the present invention designs a method for removing light from human faces based on an adversarial neural network. The method comprises the following steps: A method for removing light from human faces based on an adversarial neural network, the method comprising the following steps:

(1) Synthesize human faces through computer graphics methods, generate virtual human face image pairs with and without light, and establish a virtual face dataset;

(2) By taking ordinary photos and using polarized lenses to take photos of human faces, we obtain image pairs of human faces with light and without light in real environments, and establish a real face dataset;

(3) Build a generative adversarial neural network model;

(4) Pre-training the neural network using a virtual face dataset;

(5) Use the real face dataset to adjust the pre-trained neural network parameters to obtain the final light removal network model;

(6) The face image to be illuminated is input into the illuminated removal network model to obtain the corresponding dark face image.

Furthermore, the method of establishing the virtual face dataset in step (1) is as follows:

(1.1) Collecting non-glossy images of human faces on the Internet as materials for generating three-dimensional faces;

(1.2) Generate the 3D face corresponding to the image through HeadShot, add light source, set different reflection levels for the face material, and obtain different illuminated images of the face under different expressions through the expressions of the preset 3D face model. By intercepting, obtain the face dark image in the dark environment and the face illuminated image in the light environment, and establish a virtual face dataset.

Furthermore, the method for establishing a real face dataset in step (2) is as follows:

(2.1) Collect data with a camera, set the camera position, the subject's position, the polarizing filter position, and the light source position and angle so that the polarizing filter can eliminate light during shooting;

(2.2) When collecting images, the subject remains motionless in a fixed position, and different illuminated real face images are captured under different light intensities. A polarizing filter is then used to capture the dark version of the corresponding image to establish a real face dataset.

Furthermore, in step (3), a generative adversarial neural network model is constructed as follows:

Construct an adversarial neural network, including a generator and a discriminator. The generator adopts the U-net structure with a total of 15 layers. The features of the i-th layer are connected to the features of the 15-i layer, where i∈[1,7], and the corresponding layers share the underlying information, thereby optimizing the output results. The size of the generator input image is 256*256, and it is downsampled after convolution with a convolution kernel size of 4*4, batch normalization and ReLu activation. After 8 times of the above downsampling, the image obtained after each sampling is The dimensions of the features are: 128*128, 64*64, 32*32, 16*16, 8*8*, 4*4, 2*2 and 1*1; then activation, deconvolution with a convolution kernel size of 4*4, and batch normalization are performed to achieve upsampling. After 8 times of the above upsampling, the dimensions of the features obtained after each sampling are: 2*2, 4*4, 8*8, 16*16, 32*32, 64*64, 128*128 and the output size is 256*256;

The discriminator is a Markov discriminator (PatchGAN), which works by dividing the input image into N*N small blocks for judgment, and taking the average of the judgment results of all small blocks as the output result. By changing the size of the input image without changing the parameter N of the discriminator, multi-scale discrimination is achieved. The discriminator result is the average of the results of two discriminators of different scales. The training process is optimized using the Adam optimizer.

由于最开始生成器生成的结果不真实，判别器认为该图像为真实图像概率为0，即判别器的结果

为0，而将训练集中有光的图像a与训练集中对应的无光图像a‘输入到判别器D1、D2中，得到判别器结果

判别器认为输入图像为真实图像，故判别器的结果

为1，更新生成器的参数，使生成图为真实图像的概率

最大化，该最大值大于0.5，此时判别器认为生成器生成的图像为真实图像；再保持生成器参数不变，训练判别器，使生成图为真实图像的概率

最小化，该最小值小于0.5，此时判别器认为生成器生成的图像不真实；如此交替训练生成器和判别器，直到判别器认为真实图像与生成图像一致，即生成器和判别器之间达到纳什均衡，则训练完成；Furthermore, in step (3), the virtual face dataset is used to pre-train the neural network. The method is as follows: for the neural network constructed in step (3), the virtual face dataset is first used for pre-training to obtain the parameters of the generator and discriminator of the pre-trained model. The specific adversarial training principle is as follows: the lighted image a in the training set is input into the generator G to obtain the generated image G(a), and the generated image G(a) and the corresponding lighted image a in the training set are input into the discriminators D1 and D2 of different scales to obtain the discriminator results

Since the result generated by the generator at the beginning is not real, the discriminator believes that the probability of the image being a real image is 0, that is, the result of the discriminator

is 0, and the light image a in the training set and the corresponding dark image a' in the training set are input into the discriminators D1 and D2 to obtain the discriminator results

The discriminator believes that the input image is a real image, so the result of the discriminator is

To 1, update the parameters of the generator to make the generated image a real image.

Maximize, the maximum value is greater than 0.5, at this time the discriminator believes that the image generated by the generator is a real image; then keep the generator parameters unchanged and train the discriminator to make the probability of the generated image being a real image

Minimize, the minimum value is less than 0.5, at which point the discriminator believes that the image generated by the generator is not real; the generator and the discriminator are trained alternately in this way until the discriminator believes that the real image is consistent with the generated image, that is, the Nash equilibrium is reached between the generator and the discriminator, and the training is completed;

其中生成器和判别器的损失函数

训练判别器D时，目标是使L_GAN最小，训练生成器G时，目标是使L_GAN最大，L1范数损失函数L1＝E_a,a‘[||a‘-G(a)||₁]，E为期望，λ为系数。In order to ensure the similarity between the generated image G(a) and the matte image a' in the training set, the L1 distance between G(a) and a' is added as part of the loss function. The overall loss function is:

The loss functions of the generator and the discriminator are

When training the discriminator D, the goal is to minimize L _GAN , and when training the generator G, the goal is to maximize L _GAN . The L1 norm loss function is L1 = E _a,a' [||a'-G(a)|| ₁ ], where E is the expectation and λ is the coefficient.

由于最开始生成器生成的结果不真实，判别器认为该图像为真实图像概率为0，即判别器的结果

为0，而将训练集中有光的图像a与训练集中对应的无光图像a‘输入到判别器D1、D2中，得到判别器结果

判别器认为输入图像为真实图像，故判别器的结果

为1，更新生成器的参数，使生成图为真实图像的概率

最大化，该最大值大于0.5，此时判别器认为生成器生成的图像为真实图像；再保持生成器参数不变，训练判别器，使生成图为真实图像的概率

最小化，该最小值小于0.5，此时判别器认为生成器生成的图像不真实；如此交替训练生成器和判别器，直到判别器判别认为真实图像与生成图像一致，即生成器和判别器之间达到纳什均衡，则训练完成；Furthermore, in step (5), the parameters of the pre-trained neural network are adjusted using a real face dataset to obtain a final light removal network model. The method is as follows: Based on step (4), the parameters of the generator and the discriminator are adjusted using a real face dataset for training, so that the final light removal network model can generate a matte face image that is the same as a real face. The specific adversarial training principle is as follows: the light image a in the training set is input into the generator G to obtain a generated image G(a), and the generated image G(a) and the corresponding light image a in the training set are input into the discriminators D1 and D2 of different scales to obtain the discriminator result

Since the result generated by the generator at the beginning is not real, the discriminator believes that the probability of the image being a real image is 0, that is, the result of the discriminator

is 0, and the light image a in the training set and the corresponding dark image a' in the training set are input into the discriminators D1 and D2 to obtain the discriminator results

The discriminator believes that the input image is a real image, so the result of the discriminator is

To 1, update the parameters of the generator to make the generated image a real image.

Maximize, the maximum value is greater than 0.5, at this time the discriminator believes that the image generated by the generator is a real image; then keep the generator parameters unchanged and train the discriminator to make the probability of the generated image being a real image

Minimize, the minimum value is less than 0.5, at which point the discriminator believes that the image generated by the generator is not real; the generator and the discriminator are trained alternately in this way until the discriminator determines that the real image is consistent with the generated image, that is, the Nash equilibrium is reached between the generator and the discriminator, and the training is completed;

其中生成器和判别器的损失函数L_GAN＝E_a,a‘[logD(a,a‘)]+E_a[log(1-D(a,G(a)))]，训练判别器D时，目标是使L_gAN最小，训练生成器G时，目标是使L_GAN最大，L1范数损失函数L1＝E_a,a‘[||a‘-G(a)||₁]，E为期望，λ为系数。In order to ensure the similarity between the generated image G(a) and the matte image a' in the training set, the L1 distance between G(a) and a' is added as part of the loss function. The overall loss function is:

The loss function of the generator and discriminator is L _GAN = E _a,a' [logD(a,a')]+E _a [log(1-D(a,G(a)))], when training the discriminator D, the goal is to minimize L _gAN , when training the generator G, the goal is to maximize L _GAN , L1 norm loss function L1 = E _a,a' [||a'-G(a)|| ₁ ], E is the expectation, λ is the coefficient.

Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

(1) The present invention achieves the effect of removing the light from the face by using an end-to-end neural network, is not limited to a fixed physical illumination model, and can generate a more realistic light removal effect;

(2) In addition, compared with the light removal of other materials, the present invention has good adaptability to complex materials such as skin;

(3) In addition, the present invention can not only restore the original skin color of the light area, but also has a certain degree of restoration ability for spots, color blocks, etc. in the area;

(4) Other features and advantages of the present invention will be described in the following description and, in part, will become apparent from the description or will be understood through the implementation of the present invention. The purposes and other advantages of the present invention can be realized and obtained through the structures particularly pointed out in the description, claims and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for removing light from a human face based on an adversarial neural network implemented in the present invention;

FIG2 is a flow chart of training an adversarial neural network according to the present invention;

FIG3 is a schematic diagram of the structure of the adversarial neural network generator constructed by the present invention;

FIG. 4 is a schematic diagram of the light removal effect of the present invention.

DETAILED DESCRIPTION

The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.

In addition, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be performed in an order different from that shown here.

FIG1 is a flow chart of a method for removing light from a human face based on an adversarial neural network implemented in the present invention. Each step will be described in detail below with reference to FIG1 .

Step S110, since there is no public dataset of illuminated face images, and collecting a large number of paired real face illuminated data is time-consuming and labor-intensive and difficult to achieve, computer graphics methods are used to add light to the face images to obtain paired virtual face illuminated images and face dark images, and to establish a virtual face dataset.

Face images are collected from the Internet, and HeadShot generates the corresponding 3D face. Light sources are added, different reflectivity levels are set for the face material, and different light images of the face with different expressions are obtained through the expressions of the preset 3D face model. The virtual face data set includes black, white, and yellow men and women, ensuring the comprehensiveness of the data set and providing a basis for the results of subsequent neural network training to be applicable to all kinds of people.

Step S120: Since the human face generated by computer graphics has a certain sense of falseness, in order to compensate for this, corresponding images of real illuminated human faces and dark human faces are required. To this end, when collecting images, the subject remains motionless at a fixed position, and different illuminated real human face images are taken under different light intensities (with a fixed light source angle), and then a polarized lens placed at a fixed angle is used to remove the light to obtain a dark version of the same face. In this way, a small number of real human face data sets are established.

Step S130, constructing an adversarial neural network, including a generator and a discriminator, and obtaining parameters of the generator and the discriminator through subsequent training, so that the generator can generate a natural matte image, and the discriminator believes that the generated image is a real and natural matte image.

Step S140: Pre-train the constructed neural network using a virtual face dataset to obtain the parameters of the generator and discriminator of the pre-trained model.

Step S150, based on the pre-trained model, the real face data set is used for training, and the parameters of the generator and the discriminator are adjusted so that the final light removal network model can generate a natural and non-false light-free face image like a real face. Since the number of real face data sets is small, data enhancement is performed by image flipping to increase the data volume and make the performance of the trained network better. The parameters of the generator and the discriminator obtained by the final training are saved.

Step S160, input the illuminated face image into the illuminated removal network model obtained in the previous step to obtain the corresponding dark face image.

Fig. 2 is a flowchart of the training of the adversarial neural network of the present invention, which is divided into two steps: pre-training and parameter adjustment. The adversarial neural network is divided into two parts, a generator and a discriminator. The generator adopts the structure of U-net, with a total number of layers of 15. The features of the i-th layer are connected to the features of the 15-i-th layer, and the corresponding layers share the underlying information, thereby optimizing the output results. The commonly used Markov discriminator (PatchGAN) is used as the discriminator. In view of the limited perception range of a single discriminator, the present invention adopts a multi-scale approach on the discriminator to grasp the detail information and global information respectively. By changing the size of the input image and inputting it into the discriminator of the same structure, it is possible to extract detail information of different scales of the image. The discriminator loss is the sum of the losses of each scale discrimination.

Both virtual face datasets and real face datasets are paired. In order to focus the network's attention on the differences in the lighted areas, the paired images are connected as input. The network can automatically learn the residual of the image, thereby effectively learning the features of the lighted areas. The overall loss function is the adversarial loss of the generator and the discriminator plus the L1 norm loss between the generated image output by the generator and the corresponding dark image in the dataset. The training process is optimized using the Adam optimizer.

由于最开始生成器生成的结果真实性很差，判别器认为该图像为真实图像概率为0，即判别器的结果

为0，而将训练集中有光的图像a与训练集中对应的无光图像a‘输入到判别器D1、D2中，得到判别器结果

判别器认为输入图像为真实图像，故判别器的结果

为1，更新生成器的参数，使生成图为真实图像的概率

最大化，该最大值大于0.5，此时判别器认为生成器生成的图像为真实图像；再保持生成器参数不变，训练判别器，使生成图为真实图像的概率

最小化，该最小值小于0.5，此时判别器认为生成器生成的图像不真实；如此交替训练生成器和判别器，直到判别器无法判别生成器生成的图像是否真实(认为真实图像与生成图像无法区分)，即生成器和判别器之间达到纳什均衡，则训练完成。为了保证生成图G(a)与训练集中无光图像a‘之间的相似性，加入了G(a)与a‘的L1距离作为损失函数的一部分。整体损失函数为

其中生成器和判别器的损失函数L_GAN＝E_a,a‘[logD(a,a‘)]+E_a[log(1-D(a,G(a)))]，训练判别器D时，目标是使L_GAN最小，训练生成器G时，目标是使L_GAN最大，L1范数损失函数L1＝E_a,a‘[||a‘-G(a)||₁]，E为期望，λ为系数，训练时设为100。The specific training steps are: first pre-train with a virtual face dataset to get the approximate parameters of the generator and discriminator, then train with a real face dataset, and fine-tune the parameters of the generator and discriminator based on real face images, so that the generator's generation results are more realistic and natural, and the discriminator's judgment on image authenticity and light removal effect is more accurate. The specific adversarial training principle is as follows: input the lighted image a in the training set into the generator G to obtain the generated image G(a), and input the generated image G(a) and the corresponding lighted image a in the training set into the discriminators D1 and D2 of different scales to obtain the discriminator results.

Since the results generated by the generator at the beginning are not very realistic, the discriminator believes that the probability of the image being a real image is 0, that is, the result of the discriminator

is 0, and the light image a in the training set and the corresponding dark image a' in the training set are input into the discriminators D1 and D2 to obtain the discriminator results

The discriminator believes that the input image is a real image, so the result of the discriminator is

To 1, update the parameters of the generator to make the generated image a real image.

Maximize, the maximum value is greater than 0.5, at this time the discriminator believes that the image generated by the generator is a real image; then keep the generator parameters unchanged and train the discriminator to make the probability of the generated image being a real image

Minimize, the minimum value is less than 0.5, at which point the discriminator believes that the image generated by the generator is not real; the generator and the discriminator are trained alternately until the discriminator cannot determine whether the image generated by the generator is real (it believes that the real image and the generated image are indistinguishable), that is, the generator and the discriminator reach a Nash equilibrium, and the training is completed. In order to ensure the similarity between the generated image G(a) and the light-free image a' in the training set, the L1 distance between G(a) and a' is added as part of the loss function. The overall loss function is

The loss function of the generator and discriminator is L _GAN = E _a,a' [logD(a,a')]+E _a [log(1-D(a,G(a)))], when training the discriminator D, the goal is to minimize L _GAN , when training the generator G, the goal is to maximize L _GAN , L1 norm loss function L1 = E _a,a' [||a'-G(a)|| ₁ ], E is the expectation, λ is the coefficient, which is set to 100 during training.

Figure 3 is a schematic diagram of the structure of the adversarial neural network generator built by the present invention, which adds dimension splicing on the basis of a simple Encoder-Decoder structure. The generator adopts the U-net structure with a total of 15 layers, connecting the features of the i-th layer with the features of the 15-i-th layer (where i∈[1,7]), and the corresponding layers share the underlying information, thereby optimizing the output results. The size of the generator input image is 256*256. After convolution (the convolution kernel size is 4*4), batch normalization and ReLu activation, downsampling is achieved. After 8 times of the above downsampling, the dimensions of the features obtained after each sampling are: 128*128, 64*64, 32*32, 16*16, 8*8*, 4*4, 2*2 and 1*1; then activation, deconvolution (the convolution kernel size is 4*4), batch normalization are performed to achieve upsampling. After 8 times of the above upsampling, the dimensions of the features obtained after each sampling are: 2*2, 4*4, 8*8, 16*16, 32*32, 64*64, 128*128 and an output size of 256*256.

FIG. 4 is a diagram showing the light removal effect of the present invention.

Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims (3)

1. A method for removing light from a human face based on an adversarial neural network, characterized in that the method comprises the following steps: (1) Synthesize human faces through computer graphics methods, generate virtual human face image pairs with and without light, and establish a virtual face dataset; (2) By taking ordinary photos and using polarized lenses to take photos of human faces, we obtain image pairs of human faces with light and without light in real environments, and establish a real face dataset; (3) Build a generative adversarial neural network model; (4) Pre-training the neural network using a virtual face dataset; (5) Use the real face dataset to adjust the pre-trained neural network parameters to obtain the final light removal network model; (6) Inputting the face image to be light-removed into the light-removal network model to obtain the corresponding light-free face image; In step (3), a generative adversarial neural network model is built as follows: Construct an adversarial neural network, including a generator and a discriminator. The generator adopts the U-net structure with a total of 15 layers. The features of the i-th layer are connected to the features of the 15-i layer, where i∈[1,7], and the corresponding layers share the underlying information, thereby optimizing the output results. The size of the generator input image is 256*256, and after convolution with a convolution kernel size of 4*4, batch normalization and ReLu activation, downsampling is achieved. After 8 times of the above downsampling, the image obtained after each sampling is The dimensions of the features are: 128*128, 64*64, 32*32, 16*16, 8*8*, 4*4, 2*2 and 1*1; then activation, deconvolution with a convolution kernel size of 4*4, and batch normalization are performed to achieve upsampling. After 8 times of the above upsampling, the dimensions of the features obtained after each sampling are: 2*2, 4*4, 8*8, 16*16, 32*32, 64*64, 128*128 and the output size is 256*256; The discriminator is a Markov discriminator (PatchGAN), which works by dividing the input image into N*N small blocks for judgment, and taking the average of the judgment results of all small blocks as the output result. By changing the input image size without changing the discriminator parameter N, multi-scale discrimination is achieved. The discriminator result is the average of the results of two discriminators of different scales. The training process is optimized by the Adam optimizer. In step (3), the virtual face dataset is used to pre-train the neural network. The method is as follows: for the neural network constructed in step (3), the virtual face dataset is first used for pre-training to obtain the parameters of the generator and discriminator of the pre-trained model. The specific adversarial training principle is as follows: the lighted image a in the training set is input into the generator G to obtain the generated image G(a), and the generated image G(a) and the corresponding lighted image a in the training set are input into the discriminators D1 and D2 of different scales to obtain the discriminator result

Since the result generated by the generator at the beginning is not real, the discriminator believes that the probability of the image being a real image is 0, that is, the result of the discriminator

is 0, and the light image a in the training set and the corresponding dark image a' in the training set are input into the discriminators D1 and D2 to obtain the discriminator results

The discriminator believes that the input image is a real image, so the result of the discriminator is

To 1, update the parameters of the generator to make the generated image a real image.

Maximize, the maximum value is greater than 0.5, at this time the discriminator believes that the image generated by the generator is a real image; then keep the generator parameters unchanged and train the discriminator to make the probability of the generated image being a real image

Minimize, the minimum value is less than 0.5, at this time the discriminator thinks that the image generated by the generator is not real; the generator and the discriminator are trained alternately in this way until the discriminator thinks that the real image is consistent with the generated image, that is, the Nash equilibrium is reached between the generator and the discriminator, and the training is completed; In order to ensure the similarity between the generated image G(a) and the matte image a' in the training set, the L1 distance between G(a) and a' is added as part of the loss function. The overall loss function is:

The loss functions of the generator and the discriminator are

When training the discriminator D, the goal is to minimize L _GAN , and when training the generator G, the goal is to maximize L _GAN . The L1 norm loss function is L1 = E _a,a' [||a'-G(a)|| ₁ ], where E is the expectation and λ is the coefficient; In step (5), the parameters of the pre-trained neural network are adjusted using a real face dataset to obtain the final light removal network model. The method is as follows: Based on step (4), the parameters of the generator and the discriminator are adjusted using a real face dataset for training so that the final light removal network model can generate a matte face image that is the same as a real face. The specific adversarial training principle is as follows: the light image a in the training set is input into the generator G to obtain the generated image G(a), and the generated image G(a) and the corresponding light image a in the training set are input into the discriminators D1 and D2 of different scales to obtain the discriminator result

Since the result generated by the generator at the beginning is not real, the discriminator believes that the probability of the image being a real image is 0, that is, the result of the discriminator

is 0, and the light image a in the training set and the corresponding dark image a' in the training set are input into the discriminators D1 and D2 to obtain the discriminator results

The discriminator believes that the input image is a real image, so the result of the discriminator is

To 1, update the parameters of the generator to make the generated image a real image.

Maximize, the maximum value is greater than 0.5, at this time the discriminator believes that the image generated by the generator is a real image; then keep the generator parameters unchanged and train the discriminator to make the probability of the generated image being a real image

Minimize, the minimum value is less than 0.5, at this time the discriminator thinks that the image generated by the generator is not real; the generator and the discriminator are trained alternately in this way until the discriminator determines that the real image is consistent with the generated image, that is, the Nash equilibrium is reached between the generator and the discriminator, and the training is completed; In order to ensure the similarity between the generated image G(a) and the matte image a' in the training set, the L1 distance between G(a) and a' is added as part of the loss function. The overall loss function is:

The loss function of the generator and discriminator is L _GAN = E _{a, a′} [logD(a, a')] + E _a [log(1-D(a, G(a)))], when training the discriminator D, the goal is to minimize L _GAN , when training the generator G, the goal is to maximize L _GAN , L1 norm loss function L1 = E _{a, a'} [||a'-G(a)|| ₁ ], E is the expectation, λ is the coefficient. 2. According to the method for removing light from human face based on adversarial neural network in claim 1, it is characterized in that the method of establishing the virtual human face data set in step (1) is as follows: (1.1) Collecting non-glossy images of human faces on the Internet as materials for generating three-dimensional faces; (1.2) Generate the 3D face corresponding to the image through HeadShot, add light source, set different reflection levels for the face material, and obtain different illuminated images of the face under different expressions through the expressions of the preset 3D face model. By intercepting, obtain the face dark image in the dark environment and the face illuminated image in the light environment, and establish a virtual face dataset. 3. According to the method for removing light from human faces based on adversarial neural networks in claim 1, it is characterized in that the method for establishing a real human face dataset in step (2) is as follows: (2.1) Collect data with a camera, set the camera position, the subject's position, the polarizing filter position, and the light source position and angle so that the polarizing filter can eliminate light during shooting; (2.2) When collecting images, the subject remains motionless in a fixed position, and different illuminated real face images are captured under different light intensities. A polarizing filter is then used to capture the dark version of the corresponding image to establish a real face dataset.

Images