CN108229381B - Face image generation method and device, storage medium and computer equipment - Google Patents
Face image generation method and device, storage medium and computer equipment Download PDFInfo
- Publication number
- CN108229381B CN108229381B CN201711480435.0A CN201711480435A CN108229381B CN 108229381 B CN108229381 B CN 108229381B CN 201711480435 A CN201711480435 A CN 201711480435A CN 108229381 B CN108229381 B CN 108229381B
- Authority
- CN
- China
- Prior art keywords
- network
- target detection
- face image
- layer
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 86
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 24
- 230000006870 function Effects 0.000 claims description 24
- 238000000605 extraction Methods 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 16
- 238000010276 construction Methods 0.000 claims description 12
- 230000001815 facial effect Effects 0.000 claims description 10
- 230000017105 transposition Effects 0.000 claims description 10
- 210000000697 sensory organ Anatomy 0.000 claims description 9
- 238000006073 displacement reaction Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000011478 gradient descent method Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 3
- 210000000214 mouth Anatomy 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 210000001331 nose Anatomy 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 210000004709 eyebrow Anatomy 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G06T5/73—
Abstract
The invention relates to a face image generation method, a device, a storage medium and computer equipment, comprising a target detection network constructed based on a residual error network and a fast area network with convolutional neural network characteristics; constructing an confrontation generating network, wherein the confrontation generating network is used for reconstructing the face characteristics; cascading a target detection network and a countermeasure generation network to obtain a super-resolution network; inputting the face image to be detected into a super-resolution network to obtain a super-resolution face image; the target detection network in the super-resolution network extracts and identifies the face features from the face image to be detected, and the anti-generation network reconstructs the face features to obtain the super-resolution face image.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for generating a face image, a storage medium, and a computer device.
Background
In the fields of computer vision, image processing and pattern recognition, a face image is one of research hotspots, and a face recognition detection technology is an important component of a biological recognition technology. In video monitoring, due to the defects of hardware camera equipment and the influence of a shooting environment, the acquired face image may be unclear to different degrees, so that the image quality is reduced.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a face image generation method, apparatus, storage medium, and computer device that can improve image quality.
A face image generation method comprises the following steps:
constructing a target detection network based on a residual error network and a fast regional network with convolutional neural network characteristics, wherein the target detection network is used for extracting and identifying face characteristics from a face image to be detected;
constructing an confrontation generating network, wherein the confrontation generating network is used for reconstructing the face characteristics;
cascading a target detection network and a countermeasure generation network to obtain a super-resolution network;
and inputting the face image to be detected into a super-resolution network to obtain a super-resolution face image.
A face image generation apparatus comprising:
the target detection network construction module is used for constructing a target detection network based on a residual error network and a fast regional network with convolutional neural network characteristics, and the target detection network is used for extracting and identifying face characteristics from a face image to be detected;
the confrontation generation network construction module is used for constructing a confrontation generation network which is used for reconstructing the face characteristics;
the super-resolution network module is used for cascading the target detection network and the countermeasure generation network to obtain a super-resolution network;
and the face image output module is used for inputting the face image to be detected into the super-resolution network to obtain the super-resolution face image.
A storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the above-mentioned method.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the steps of the method being performed when the program is executed by the processor.
The face image generation method, the face image generation device, the storage medium and the computer equipment comprise a target detection network constructed on the basis of a residual error network and a fast area network with convolutional neural network characteristics; constructing an confrontation generating network, wherein the confrontation generating network is used for reconstructing the face characteristics; cascading a target detection network and a countermeasure generation network to obtain a super-resolution network; inputting the face image to be detected into a super-resolution network to obtain a super-resolution face image; the target detection network in the super-resolution network extracts and identifies the face features from the face image to be detected, and the anti-generation network reconstructs the face features to obtain the super-resolution face image.
Drawings
FIG. 1 is a schematic flow chart diagram of a face image generation method in one embodiment;
FIG. 2 is a diagram illustrating a residual error network in the face image generation method according to an embodiment;
FIG. 3 is a flow diagram illustrating one step of a method for generating a face image according to one embodiment;
FIG. 4 is a schematic structural diagram of a face image generation apparatus according to an embodiment;
FIG. 5 is a diagram illustrating a target detection network in the face image generation method according to an embodiment;
fig. 6 is a schematic diagram of a countermeasure generation network in the face image generation method in one embodiment.
Detailed Description
As shown in fig. 1, a face image generation method includes:
s100, constructing a target detection network based on the residual error network and the fast regional network with the convolutional neural network characteristics, wherein the target detection network is used for extracting and identifying the human face characteristics from the human face image to be detected.
The Residual Network (ResNet) provides a reference for the input of each layer, and a Residual function is formed by learning, so that the Residual function is easier to optimize, and the number of Network layers can be greatly deepened. In computer vision, the level of image features becomes higher as the depth of a network becomes deeper, which is an important factor in achieving a good image processing effect. However, gradient dispersion or explosion becomes an obstacle to training the deep network, resulting in failure to converge. The depth of the converged network can be improved to ten times by means of normalization initialization, input normalization of each layer and the like, but the network starts to degrade, namely, increasing the number of network layers can cause larger errors. By stacking layers of y ═ x on a shallow network basis, the network can be made to grow with depth without degradation.
ResNet learns the residual function f (x) ═ h (x) -x, without introducing additional parameters and computational complexity, the residual function will typically have less response fluctuation. In practice, considering the cost of computation, the residual block is computationally optimized by replacing two 3 × 3 convolutional layers with 1 × 1+3 × 3+1 × 1, as shown in fig. 2, the middle 3 × 3 convolutional layer is first reduced in computation under one dimension-reduced 1 × 1 convolutional layer, and then reduced under another 1 × 1 convolutional layer, which both maintains the accuracy and reduces the computation amount.
Fast Regions with convolutional Neural network features (fast Regions with convolutional Neural Networks, fast Rcnn), including inputting a test image, inputting a whole picture into a CNN (convolutional Neural Networks, convolutional Neural network) for feature extraction; generating suggestion windows by using an RPN (resilient packet network), and generating 300 suggestion windows for each picture; mapping the suggestion window to the last layer of convolution characteristic graph of the CNN; generating a feature map of fixed size for each RoI (region of interest) by posing layer; the classification probability and Bounding box regression (Bounding box regression) are jointly trained using Softmax Loss and Smooth L1 Loss.
The convolutional neural network is composed of neurons according to a hierarchical structure like a common neural network, and weights and offsets between the neurons can be obtained through training. Similarly, the input data and the weight are operated, the output result is input into the excitation neuron, and then the result is output. In general, the entire neural network calculates scores of the final categories for the image data inputted at the pixel level using a score function, and then obtains an optimal weight by minimizing a loss function.
S200, constructing a confrontation generating network, wherein the confrontation generating network is used for reconstructing the face features.
The confrontation generation network GAN is inspired by two-player zero-sum games (two-player games) in the game theory, and two game parties in the GAN model are respectively served by a generating model (generating model) and a discriminant model (discriminant model). The generative model G captures the distribution of sample data, and noise z obeying certain distribution (such as uniform distribution, Gaussian distribution and the like) is used for generating a sample similar to real training data, wherein the pursuit effect is that the more the real sample is, the better the pursuit effect is; the discriminant model D is a two-classifier that estimates the probability that a sample is from training data (rather than from generative data), and if the sample is from real training data, D outputs a large probability; otherwise, D outputs a small probability. In the training process, one party is fixed, the network weight of the other party is updated, and the two parties are iterated alternately, in the process, the two parties optimize own networks to the utmost extent, so that competitive confrontation is formed until the two parties reach a dynamic balance (Nash equilibrium), at the moment, the generated model G recovers the distribution of training data (a sample which is the same as real data is created), the result cannot be judged by the discriminant model, the accuracy is 50%, and the guess is approximately equal to the misstatement. When G is fixed, for the optimization of D, it can be understood that: inputting real data, and optimizing a network structure to output 1 by the D; the input comes from the generated data and the D-optimized network structure outputs 0 itself. When D is fixed, G optimizes the network of itself to make itself output the sample as much as the real data, and makes D output high probability after the generated sample is judged by D.
And S300, cascading the target detection network and the countermeasure generation network to obtain the super-resolution network.
And connecting the trained target detection network with the countermeasure generation network to obtain a super-resolution network, and outputting a face image with low resolution to obtain a high-definition face image and facial five-sense feature characteristics after the face image with low resolution is input into the super-resolution network.
And S400, inputting the face image to be detected into a super-resolution network to obtain a super-resolution face image.
The super-resolution is to improve the resolution of the original image, and the process of obtaining a high-resolution image through a low-resolution image is super-resolution reconstruction. In a number of electronic image applications, high resolution images are often desired. High resolution means that the density of pixels in the image is high, providing more detail that is essential in many practical applications.
The face image generation method comprises the steps of constructing a target detection network based on a residual error network and a fast area network with convolutional neural network characteristics; constructing an confrontation generating network, wherein the confrontation generating network is used for reconstructing the face characteristics; cascading a target detection network and a countermeasure generation network to obtain a super-resolution network; inputting the face image to be detected into a super-resolution network to obtain a super-resolution face image; the target detection network in the super-resolution network extracts and identifies the face features from the face image to be detected, and the anti-generation network reconstructs the face features to obtain the super-resolution face image.
In one embodiment, as shown in fig. 3, the step S100 of constructing the target detection network in the face image generation method based on the residual error network and the fast area network having the convolutional neural network feature includes:
s120, constructing a feature extraction layer for extracting face features in the face image based on a residual error network;
s140, connecting the feature extraction layer with a target detection layer in a fast regional network with convolutional neural network features to form a sample target detection network, wherein the sample target detection network is used for identifying the human face features in the human face image;
and S160, training the sample target detection network to obtain the target detection network.
Specifically, a residual network (RestNet-101 layer) can be built, the residual network has 5 blocks in total, except for the first input block, the number of cycles of the convolutional network in the other four blocks is 3, 4, 23 and 3 respectively, the network is trained by adopting data in an ImageNet database to enable parameters to be fitted, and the accuracy of the previous classification can be 72.6% and the accuracy of the previous five is 93.7% after training is completed. The last convolutional layer of each cycle in the middle block is extracted by adopting picture input with the size of 321 × 321 pixels, the sizes of the four convolutional layers are respectively 160 × 160 × 256, 80 × 80 × 512, 40 × 40 × 1024 and 20 × 20 × 2048, and the parameters of the convolutional layers can be represented by (s, n), wherein n is the number of convolution kernels, and s is the size of the convolution kernels. After the characteristic quantity is reduced by one convolution layer from the convolution layer at the bottommost layer, the characteristic quantity is amplified by 2 times by using the transposition convolution layer and then is connected with the convolution layer at the upper layer, finally, the output layer of 160 multiplied by 256 is obtained, and the residual error network is used as a characteristic extraction network. Adjusting 1000 types of ResNet which are trained and based on ImageNet, changing the classification number of the ResNet into 6 types, inputting human faces and other 4 types of images as data sets, retraining parameters, achieving 100% accuracy, not fitting, and obtaining the required feature extraction network after the first-stage training is finished. And after the trained feature extraction network is obtained, connecting the obtained feature extraction network with a subsequent target detection layer according to the connection mode of the Faster Rcnn, and detecting five sense organs and a human face by using the fast Rcnn, wherein the categories to be classified are eyes, eyebrows, a nose, a mouth, the human face and other six categories.
In one embodiment, the step of training the sample target detection network to obtain the target detection network includes: inputting the face image carrying the feature marks into a sample target detection network, and outputting the positions and classification types of features in the face image, wherein the features comprise face five sense organs; and correcting the sample target detection network through a loss function to obtain the target detection network, wherein the loss function is the Euclidean distance between the position marked by the five sense organs and the position of the output five sense organs. When the loss function is minimized, iterative solution can be performed step by step through a gradient descent method, and the minimized loss function and the model parameter value are obtained. Euclidean distance is a commonly used definition of distance, referring to the true distance between two points in m-dimensional space, or the natural length of a vector (i.e., the distance of the point from the origin).
In one embodiment, the step of constructing a countermeasure generation network in the face image generation method includes: building a convolution network and a pixel displacement network; and connecting the convolution network with the pixel displacement network to form a countermeasure generation network. In another embodiment, the step of constructing a countermeasure generation network in the face image generation method includes: building a convolution layer and a transposition convolution layer, and connecting the convolution layer with the transposition convolution layer to form a convolution network; and connecting the transposed convolution layer in the convolution network with the pixel displacement network to form a countermeasure generation network. Wherein, the step of building the convolution layer and the transposition convolution layer, connecting the convolution layer with the transposition convolution layer and forming the convolution network comprises the following steps: building a first convolution layer, a second convolution layer and a third convolution layer which are connected in sequence; building a first transposition convolution layer and a second transposition convolution layer which are connected in sequence; and connecting the third convolution layer with the first convolution layer to form a convolution network.
Specifically, the countermeasure generation network may adopt 5 layers of convolutional networks plus 1 layer of pixel shift networks, the first three layers are convolutional layers with a step size of 2, the picture may be reduced by 8 times, the second two layers are transposed convolutional layers with a step size of 2, and the picture is enlarged by 4 times. The number of the characteristic layers is 132, the last layer of pixel displacement layer extracts the 132 characteristic layers into 3-bit channels, the length and the width of the picture are amplified by 8 times, a shallow confrontation generation network with the length and the width respectively amplified by 4 times is obtained, and the network is trained to be fitted.
In one embodiment, the step of cascading the target detection network and the countermeasure generation network in the face image generation method to obtain the super-resolution network further includes: reducing a network loss function by a gradient descent method, and correcting the super-resolution network, wherein the network loss function is the Euclidean distance between the facial image features output by the super-resolution network and the facial image features of a preset sample; the method for obtaining the super-resolution face image comprises the following steps of inputting a face image to be detected into a super-resolution network, and obtaining the super-resolution face image: and inputting the face image to be detected into the corrected super-resolution network to obtain the super-resolution face image.
In the machine learning algorithm, when the loss function is minimized, iterative solution can be performed step by a gradient descent method to obtain the minimized loss function and the model parameter value.
In one embodiment, a storage medium is further provided, on which a computer program is stored, wherein the program is executed by a processor to implement any one of the face image generation methods in the above embodiments. The storage medium may be an optical disc, a read-only memory, a random access memory, or the like.
The storage medium and the stored computer program can obtain super-resolution face images by implementing the processes of the embodiments of the face image generation methods, and the processing can effectively improve the definition of input images, thereby improving the image quality of the input images.
In one embodiment, a computer device is further provided, which includes a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the face image generation method as in any one of the above embodiments. When a processor of the computer device executes a program, the super-resolution face image can be obtained by implementing any one of the face image generation methods in the embodiments, and the processing can effectively improve the definition of the input image, thereby improving the image quality of the input image.
In one embodiment, a face image generation apparatus, as shown in fig. 4, includes:
the target detection network construction module 100 is configured to construct a target detection network based on a residual error network and a fast area network with convolutional neural network characteristics, where the target detection network is used to extract and identify facial characteristics from a facial image to be detected;
the confrontation generation network construction module 200 is used for constructing a confrontation generation network, and the confrontation generation network is used for reconstructing the face characteristics;
the super-resolution network module 300 is used for cascading the target detection network and the countermeasure generation network to obtain a super-resolution network;
the face image output module 400 is configured to input the face image to be detected to a super-resolution network to obtain a super-resolution face image.
The face image generation device comprises a target detection network construction module 100, an antagonism generation network construction module 200, a super-resolution network module 300 and a face image output module 400, wherein the target detection network construction module 100 constructs a target detection network based on a residual error network and a fast area network with convolutional neural network characteristics; the confrontation generation network construction module 200 constructs a confrontation generation network, and the confrontation generation network is used for reconstructing the face features; the super-resolution network module 300 cascades the target detection network and the countermeasure generation network to obtain a super-resolution network; the face image output module 400 inputs the face image to be detected into a super-resolution network to obtain a super-resolution face image; the target detection network in the super-resolution network extracts and identifies the face features from the face image to be detected, and the anti-generation network reconstructs the face features to obtain the super-resolution face image.
In one embodiment, a method for generating a face image specifically includes: 1. and (5) building a feature extraction network. And (2) a residual error network (ResNet-101) is built, 5 blocks are arranged in the residual error network in total, except a first input block, the cycle number of the convolution network in each block is respectively 3, 4, 23 and 3, the data of an ImageNet database is adopted to train the network so as to enable parameters to be fitted, and the accuracy of the previous classification can reach 72.6 percent and the accuracy of the previous five classification can reach 93.7 percent after the training is completed. The last convolutional layer of each cycle in the middle block is extracted by adopting picture input with the size of 321 × 321 pixels, the sizes of the four convolutional layers are respectively 160 × 160 × 256, 80 × 80 × 512, 40 × 40 × 1024 and 20 × 20 × 2048, and the parameters of the convolutional layers can be represented by (s, n), wherein n is the number of convolution kernels, and s is the size of the convolution kernels. After the characteristic quantity is reduced by one convolution layer from the convolution layer at the lowest layer, the characteristic quantity is amplified by 2 times by using the transposition convolution layer and is connected with the convolution layer at the previous layer, and finally, the output layer of 160 multiplied by 256 is obtained.
After the trained feature extraction layer is obtained, a fast regional network with convolutional neural network features is adopted to detect five sense organs and a human face, the categories to be classified are eyes, eyebrows, noses, mouths, the human face and other six categories, so that 1000 categories ResNet based on ImageNet which are trained need to be adjusted, the classification number of the 1000 categories ResNet is changed into 6 categories, the human face and other 4 categories of images are made into a data set to be input, parameters are retrained, the accuracy rate can reach 100%, fitting cannot be achieved, the first-stage training is finished, and the required feature extraction network is obtained.
2. And constructing a target detection network. The obtained feature extraction network is connected with a subsequent target detection layer according to the connection mode of the fast area network with the convolutional neural network feature to form a target detection network, as shown in fig. 5. And inputting the face picture with the five sense organs mark into the target detection network, outputting the face picture as the position and classification category of each five sense organs, and outputting a loss function as the Euclidean distance between the mark position and the predicted position. Based on the human face characteristics, the aspect ratio of the detection frame of the eyes and the mouth is 2: 1, the length-width ratio of the detection frame of the nose is 1: 2. the positions and the types of the facial features can be accurately detected through the trained target detection network.
3. And constructing a countermeasure generation network. There are two kinds of confrontation generating networks, one adopts deep network to extract better characteristics; and the other method adopts a shallow network and keeps more characteristics of the original image. As shown in fig. 6, the picture can be reduced by 8 times by using 5 layers of convolutional network plus 1 layer of pixel shift network, and the first three layers are convolutional layers with step length of 2; the latter two layers are transposed convolution layers with step size 2, magnifying the picture by 4 times. The number of the characteristic layers is 132, the last layer of pixel displacement layer extracts the 132 characteristic layers into 3-bit channels, the length and the width of the picture are amplified by 8 times, then a shallow confrontation generation network with the length and the width respectively amplified by 4 times is obtained, and the network is trained to be fitted.
4. The countermeasure generation network is cascaded with the target detection network. And connecting the trained target detection network with the countermeasure generation network, and obtaining a high-definition picture and the characteristics of the five sense organs of the high-definition picture after a low-resolution picture is input into the connected network. And simultaneously, inputting the average value of the high-definition face pictures in the database to obtain the comprehensive facial feature value. And for corresponding features, using an L2 universal number as a loss function, deriving the high-definition picture by using the loss function, and reducing the loss function by using a gradient descent method to improve the accuracy of the five-sense-organ features in the high-definition picture.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A face image generation method is characterized by comprising the following steps:
constructing a target detection network based on a residual error network and a fast regional network with convolutional neural network characteristics, wherein the target detection network is used for extracting and identifying human face characteristics from a human face image to be detected, and the target detection network is obtained by connecting a characteristic extraction layer constructed by the residual error network with a target detection layer in the fast regional network;
constructing a confrontation generating network, wherein the confrontation generating network is used for reconstructing the human face features;
cascading the target detection network and the countermeasure generation network to obtain a super-resolution network;
inputting the face image to be detected into the super-resolution network to obtain a super-resolution face image;
the step of constructing the target detection network based on the residual error network and the fast area network with the convolutional neural network characteristics comprises the following steps:
constructing a feature extraction layer for extracting face features in a face image based on a residual error network, wherein the feature extraction layer is an output layer obtained by starting from a convolution layer at the bottommost layer in the residual error network, reducing the number of features through one convolution layer, amplifying by using a transposed convolution layer and then connecting with a convolution layer at the previous layer;
connecting the feature extraction layer with a target detection layer in a fast area network with convolutional neural network features to form a sample target detection network, wherein the sample target detection network is used for identifying the human face features in the human face image;
and training the sample target detection network to obtain a target detection network.
2. The method of claim 1, wherein the step of training the sample target detection network to obtain a target detection network comprises:
inputting a face image carrying feature marks into the sample target detection network, and outputting positions and classification categories of features in the face image, wherein the features comprise face five sense organs;
and correcting the sample target detection network through a loss function to obtain a target detection network, wherein the loss function is the Euclidean distance between the position of the feature mark and the output feature position.
3. The method of generating a face image according to claim 1, wherein the step of constructing a countermeasure generating network comprises:
building a convolution network and a pixel displacement network;
and connecting the convolution network with the pixel displacement network to form a countermeasure generation network.
4. The face image generation method according to claim 3, wherein the step of constructing a countermeasure generation network includes:
building a convolutional layer and a transposed convolutional layer, and connecting the convolutional layer with the transposed convolutional layer to form a convolutional network;
and connecting the transposed convolution layer in the convolution network with the pixel displacement network to form a countermeasure generation network.
5. The method according to claim 4, wherein the step of building a convolutional layer and a transpose convolutional layer, connecting the convolutional layer and the transpose convolutional layer to form a convolutional network comprises:
building a first convolution layer, a second convolution layer and a third convolution layer which are connected in sequence;
building a first transposition convolution layer and a second transposition convolution layer which are connected in sequence;
and connecting the third convolution layer with the first convolution layer to form a convolution network.
6. The face image generation method according to claim 1,
the step of cascading the target detection network and the countermeasure generation network to obtain a super-resolution network further comprises:
reducing a network loss function by a gradient descent method, and correcting the super-resolution network, wherein the network loss function is the Euclidean distance between the facial image features output by the super-resolution network and the facial image features of a preset sample;
the step of inputting the face image to be detected into the super-resolution network to obtain the super-resolution face image comprises the following steps:
and inputting the face image to be detected into the corrected super-resolution network to obtain a super-resolution face image.
7. A face image generation apparatus, comprising:
the target detection network construction module is used for constructing a target detection network based on a residual error network and a fast regional network with convolutional neural network characteristics, the target detection network is used for extracting and identifying human face characteristics from a human face image to be detected, and the target detection network is obtained by connecting a characteristic extraction layer constructed by the residual error network with a target detection layer in the fast regional network;
the confrontation generation network construction module is used for constructing a confrontation generation network, and the confrontation generation network is used for reconstructing the face features;
the super-resolution network module is used for cascading the target detection network and the countermeasure generation network to obtain a super-resolution network;
the face image output module is used for inputting the face image to be detected into the super-resolution network to obtain a super-resolution face image;
the target detection network construction module is also used for constructing a feature extraction layer for extracting face features in the face image based on a residual error network, wherein the feature extraction layer is an output layer obtained by starting from a convolution layer at the bottommost layer in the residual error network, reducing the feature quantity through one convolution layer, amplifying by using a transposed convolution layer and then connecting with a convolution layer at the previous layer;
connecting the feature extraction layer with a target detection layer in a fast area network with convolutional neural network features to form a sample target detection network, wherein the sample target detection network is used for identifying the human face features in the human face image;
and training the sample target detection network to obtain a target detection network.
8. The apparatus of claim 7, wherein the target detection network construction module is further configured to input a face image carrying feature labels to the sample target detection network, and output positions and classification categories of features in the face image, where the features include facial features;
and correcting the sample target detection network through a loss function to obtain a target detection network, wherein the loss function is the Euclidean distance between the position of the feature mark and the output feature position.
9. A storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the steps of the method of any one of claims 1 to 6.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1-6 are implemented when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711480435.0A CN108229381B (en) | 2017-12-29 | 2017-12-29 | Face image generation method and device, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711480435.0A CN108229381B (en) | 2017-12-29 | 2017-12-29 | Face image generation method and device, storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108229381A CN108229381A (en) | 2018-06-29 |
| CN108229381B true CN108229381B (en) | 2021-01-08 |
Family
ID=62646179
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711480435.0A Active CN108229381B (en) | 2017-12-29 | 2017-12-29 | Face image generation method and device, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108229381B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110675324B (en) * | 2018-07-02 | 2023-10-10 | 上海寰声智能科技有限公司 | 4K ultra-high definition image sharpening processing method |
| CN110728626A (en) * | 2018-07-16 | 2020-01-24 | 宁波舜宇光电信息有限公司 | Image deblurring method and apparatus and training thereof |
| CN109300107B (en) | 2018-07-24 | 2021-01-22 | 深圳先进技术研究院 | Plaque processing method, device and computing equipment for magnetic resonance blood vessel wall imaging |
| CN109460708A (en) * | 2018-10-09 | 2019-03-12 | 东南大学 | A kind of Forest fire image sample generating method based on generation confrontation network |
| CN109447962A (en) * | 2018-10-22 | 2019-03-08 | 天津工业大学 | A kind of eye fundus image hard exudate lesion detection method based on convolutional neural networks |
| CN109447018B (en) * | 2018-11-08 | 2021-08-03 | 天津理工大学 | Road environment visual perception method based on improved Faster R-CNN |
| CN111598891B (en) * | 2019-02-20 | 2023-08-08 | 深圳先进技术研究院 | Plaque stability identification method, plaque stability identification device, plaque stability identification equipment and storage medium |
| CN109961507B (en) * | 2019-03-22 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Face image generation method, device, equipment and storage medium |
| CN110210314B (en) * | 2019-05-06 | 2023-06-13 | 深圳华付技术股份有限公司 | Face detection method, device, computer equipment and storage medium |
| CN110119746B (en) * | 2019-05-08 | 2021-11-30 | 北京市商汤科技开发有限公司 | Feature recognition method and device and computer readable storage medium |
| CN110189255B (en) * | 2019-05-29 | 2023-01-17 | 电子科技大学 | Face detection method based on two-stage detection |
| CN110298384B (en) * | 2019-06-03 | 2021-03-12 | 西华大学 | Countermeasure sample image generation method and apparatus |
| CN110533004A (en) * | 2019-09-07 | 2019-12-03 | 哈尔滨理工大学 | A kind of complex scene face identification system based on deep learning |
| CN110570355B (en) * | 2019-09-12 | 2020-09-01 | 杭州海睿博研科技有限公司 | Multi-scale automatic focusing super-resolution processing system and method |
| CN112836554A (en) * | 2019-11-25 | 2021-05-25 | 广东博智林机器人有限公司 | Image verification model construction method, image verification method and device |
| CN111047546A (en) * | 2019-11-28 | 2020-04-21 | 中国船舶重工集团公司第七一七研究所 | Infrared image super-resolution reconstruction method and system and electronic equipment |
| CN111221118B (en) * | 2020-02-26 | 2022-06-28 | 南京理工大学 | Microscopic imaging method based on phase coding single lens |
| CN111428591A (en) * | 2020-03-11 | 2020-07-17 | 天津华来科技有限公司 | AI face image processing method, device, equipment and storage medium |
| CN111310732A (en) * | 2020-03-19 | 2020-06-19 | 广东宜教通教育有限公司 | High-precision face authentication method, system, computer equipment and storage medium |
| CN111862040B (en) * | 2020-07-20 | 2023-10-31 | 中移（杭州）信息技术有限公司 | Portrait picture quality evaluation method, device, equipment and storage medium |
| CN112528851B (en) * | 2020-12-09 | 2024-03-29 | 南京航空航天大学 | Face detection method and system |
| CN114240844B (en) * | 2021-11-23 | 2023-03-14 | 电子科技大学 | Unsupervised key point positioning and target detection method in medical image |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106683067A (en) * | 2017-01-20 | 2017-05-17 | 福建帝视信息科技有限公司 | Deep learning super-resolution reconstruction method based on residual sub-images |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
-
2017
- 2017-12-29 CN CN201711480435.0A patent/CN108229381B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106683067A (en) * | 2017-01-20 | 2017-05-17 | 福建帝视信息科技有限公司 | Deep learning super-resolution reconstruction method based on residual sub-images |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
Non-Patent Citations (1)
| Title |
|---|
| 欧阳宁.基于并列卷积神经网络的超分辨率重建.《计算机应用》.2017, * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108229381A (en) | 2018-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229381B (en) | Face image generation method and device, storage medium and computer equipment | |
| JP6807471B2 (en) | Semantic segmentation model training methods and equipment, electronics, and storage media | |
| CN107679477B (en) | Face depth and surface normal vector prediction method based on cavity convolution neural network | |
| CN107977932B (en) | Face image super-resolution reconstruction method based on discriminable attribute constraint generation countermeasure network | |
| CN108230278B (en) | Image raindrop removing method based on generation countermeasure network | |
| CN105069400B (en) | Facial image gender identifying system based on the sparse own coding of stack | |
| CN110991299A (en) | Confrontation sample generation method aiming at face recognition system in physical domain | |
| CN111783748B (en) | Face recognition method and device, electronic equipment and storage medium | |
| WO2019227479A1 (en) | Method and apparatus for generating face rotation image | |
| CN107103285B (en) | Face depth prediction method based on convolutional neural network | |
| CN112801015B (en) | Multi-mode face recognition method based on attention mechanism | |
| CN110059728B (en) | RGB-D image visual saliency detection method based on attention model | |
| CN111126412B (en) | Image key point detection method based on characteristic pyramid network | |
| CN112950561B (en) | Optical fiber end face defect detection method, device and storage medium | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN113011253B (en) | Facial expression recognition method, device, equipment and storage medium based on ResNeXt network | |
| CN113610046B (en) | Behavior recognition method based on depth video linkage characteristics | |
| CN112581540A (en) | Camera calibration method based on human body posture estimation in large scene | |
| CN114842343A (en) | ViT-based aerial image identification method | |
| CN111898571A (en) | Action recognition system and method | |
| CN110991563B (en) | Capsule network random routing method based on feature fusion | |
| CN110503002B (en) | Face detection method and storage medium | |
| CN111553250B (en) | Accurate facial paralysis degree evaluation method and device based on face characteristic points | |
| CN109101984B (en) | Image identification method and device based on convolutional neural network | |
| CN111368831B (en) | Positioning system and method for vertical text |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| DD01 | Delivery of document by public notice | ||
| DD01 | Delivery of document by public notice |
Addressee: Hunan happy Technology Co., Ltd. Document name: Notification of Approving Refund |
|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |