CN107895358A - The Enhancement Method and system of facial image - Google Patents
The Enhancement Method and system of facial image Download PDFInfo
- Publication number
- CN107895358A CN107895358A CN201711422950.3A CN201711422950A CN107895358A CN 107895358 A CN107895358 A CN 107895358A CN 201711422950 A CN201711422950 A CN 201711422950A CN 107895358 A CN107895358 A CN 107895358A
- Authority
- CN
- China
- Prior art keywords
- model
- facial image
- face
- image
- generation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001815 facial effect Effects 0.000 title claims abstract description 237
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000002708 enhancing effect Effects 0.000 claims abstract description 99
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims description 51
- 238000005728 strengthening Methods 0.000 claims description 24
- 238000004519 manufacturing process Methods 0.000 claims description 16
- 238000003062 neural network model Methods 0.000 claims description 13
- 230000007935 neutral effect Effects 0.000 claims description 10
- 210000005036 nerve Anatomy 0.000 claims 1
- 230000006870 function Effects 0.000 description 32
- 230000000694 effects Effects 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 238000003672 processing method Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 210000001747 pupil Anatomy 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 208000027534 Emotional disease Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention provides a kind of Enhancement Method of facial image and system, this method includes：Obtain current facial image；Face burst processing is carried out to the current facial image, to obtain face slice images；The first image enhaucament model that the face slice images are separately input to each build in advance, the face slice images strengthened；The face slice images of the enhancing are replaced to the corresponding part of the current facial image respectively, once to be strengthened facial image.The face obtained using the present invention is true careful in face details, can obtain the facial image of high-resolution, high quality.
Description
Technical field
The present invention relates to image processing field, and in particular to the Enhancement Method and system of a kind of facial image.
Background technology
In the last few years, deep learning neutral net achieved many progress in image domains, such as target detection, face are known
Not, image segmentation etc..For the task of Face image synthesis, traditional method based on neutral net is training one for spy
Determine the neutral net of reconstruction task, then to the width of network inputs one facial image to be reconstructed (such as low-quality facial image),
The complete high quality facial image of a width is directly exported by neutral net.
But the face quality of above-mentioned existing technical scheme generation is uneven, can be obtained for some facial images
Preferable quality reconstruction is obtained, but may then cause Face image synthesis second-rate for other facial images.Effect at present
Preferable face generation scheme is the method based on production confrontation neutral net.
Production confrontation neutral net (Generative Adversarial Network, abbreviation GAN) is a kind of special
The convolutional neural networks (CNN) of framework, it is mainly used in image generation field.Production resists neutral net generally by two parts group
Into i.e. production network model (Generator, G model) and discriminate network model (Discriminator, D model).G moulds
Type is used to generate target image (Generated Example), and D models then replace the loss function in traditional convolutional neural networks
Layer, by the contrast with true picture (Real Example), judge generation target image it is true and false, then instruct G models
Preferably generation target image.
In facial image reconstruction task, conventional method is to use whole facial image of G model reconstructions, then by D models
Judge the quality of whole facial image.What is done due to D models is two classification tasks, that is, the quality of human face image generated it is good
Bad therefore bigger image includes more Pixel Informations, is more difficult to accurate judgement for D models, this directly results in
Conventional face reconstruction model is difficult to the facial image for reconstructing high-resolution high quality.
The content of the invention
It is an object of the invention to provide a kind of Enhancement Method of facial image and system, the face figure of high quality is desirably to obtain
Picture.
The invention provides a kind of Enhancement Method of facial image, wherein, including：
Obtain current facial image；
Face burst processing is carried out to the current facial image, to obtain face slice images；
The first image enhaucament model that the face slice images are separately input to each build in advance, is strengthened
Face slice images；
The face slice images of the enhancing are replaced to the corresponding part of the current facial image respectively, to obtain one
Secondary enhancing facial image.
Preferably, methods described also includes：
The once enhancing facial image is input to the second image enhaucament model built in advance, obtains secondary enhancing people
Face image.
Preferably, described first image enhancing model is that production resists neural network model, including the first generation model
With the first discrimination model；
The input of first generation model is the face slice images；The output of first generation model is described
The face slice images of enhancing；
The face slice images inputted as the enhancing of first discrimination model and the face slice images of high quality,
The output of first discrimination model is two classification judgment values.
Preferably, the second image enhaucament model includes production confrontation neural network model, including the second generation mould
Type and the second discrimination model；
The input of second generation model once strengthens facial image to be described；The output of second generation model is
The secondary enhancing facial image；
The input of second discrimination model is the secondary enhancing facial image and the facial image of high quality, described the
The output of two discrimination models is two classification judgment values.
Preferably, the second image enhaucament model is trained by following training method：
The training set for once strengthening facial image is input to second generation model；
The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is defeated
Enter into second discrimination model；
The two classification judgment values exported according to second discrimination model, obtain first object function, by first mesh
The gradient anti-pass of scalar functions is to second generation model；
When two classification judgment values of second discrimination model output reach the first setting value, deconditioning.
Preferably, the second image enhaucament model also includes the human face recognition model based on convolutional neural networks structure；
The input of the human face recognition model is the secondary enhancing facial image, and the output of the human face recognition model is
The characteristic vector of the secondary enhancing facial image.
Preferably, the second image enhaucament model is trained by following training method：：
The training set for once strengthening facial image is input to second generation model；
The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is defeated
Enter into second discrimination model；Meanwhile the secondary enhancing facial image for generating second generation model is input to institute
State in human face recognition model；
The two classification judgment values exported according to second discrimination model, obtain first object function, by first mesh
The gradient anti-pass of scalar functions is to second generation model；
The secondary enhancing people exported according to the characteristic vector of the facial image of high quality and the human face recognition model
The distance between characteristic vector of face image, the second object function is obtained, by the gradient anti-pass of second object function to institute
State the second generation model；
When two classification judgment values of second discrimination model output reach the first setting value, and the face of the high quality
The distance between characteristic vector of secondary enhancing facial image that the characteristic vector of image exports with the human face recognition model arrives
During up to the second setting value, deconditioning.
Preferably, it is described that face burst processing is carried out to the current facial image, to obtain face slice images bag
Include：
Face characteristic point mark is carried out to the current facial image；
The current facial image is rotated and/or scaled according to the characteristic point of mark；
Facial image after the rotation and/or scaling is cut out, to obtain face slice images.
Present invention also offers a kind of strengthening system of facial image, wherein, including：
Acquisition module, for obtaining current facial image；
Face burst module, for carrying out face burst processing to the current facial image, to obtain face burst
Image；
First image enhancement module, for the first figure for being separately input to each build in advance by the face slice images
Image intensifying model, the face slice images strengthened；
Image replacement module, for the face slice images of the enhancing to be replaced into the current facial image respectively
Corresponding part, once to be strengthened facial image.
Preferably, the system also includes：
Second image enhancement module, increase for the once enhancing facial image to be input into the second image built in advance
Strong model, obtain secondary enhancing facial image.
Preferably, the system also includes：
Second structure module, for building the second image enhaucament model；The second image enhaucament model includes life
An accepted way of doing sth resists neural network model, including the second generation model and the second discrimination model；
The input of second generation model once strengthens facial image to be described；The output of second generation model is
The secondary enhancing facial image；
The input of second discrimination model is the secondary enhancing facial image and the facial image of high quality, described the
The output of two discrimination models is two classification judgment values.
Preferably, the second structure module includes training unit, is used for：
The training set for once strengthening facial image is input to second generation model；
The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is defeated
Enter into second discrimination model；
The two classification judgment values exported according to second discrimination model, obtain first object function, by first mesh
The gradient anti-pass of scalar functions is to second generation model；
When two classification judgment values of second discrimination model output reach the first setting value, deconditioning.
Preferably, the second image enhaucament model also includes the human face recognition model based on convolutional neural networks structure；
The input of the human face recognition model is the secondary enhancing facial image, and the output of the human face recognition model is
The characteristic vector of the secondary enhancing facial image.
Preferably, the second structure module includes training unit, is used for：
The training set for once strengthening facial image is input to second generation model；
The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is defeated
Enter into second discrimination model；Meanwhile the secondary enhancing facial image for generating second generation model is input to institute
State in human face recognition model；
The two classification judgment values exported according to second discrimination model, obtain first object function, by first mesh
The gradient anti-pass of scalar functions is to second generation model；
The secondary enhancing people exported according to the characteristic vector of the facial image of high quality and the human face recognition model
The distance between characteristic vector of face image, the second object function is obtained, by the gradient anti-pass of second object function to institute
State the second generation model；
When two classification judgment values of second discrimination model output reach the first setting value, and the face of the high quality
The distance between characteristic vector of secondary enhancing facial image that the characteristic vector of image exports with the human face recognition model arrives
During up to the second setting value, deconditioning.
Preferably, the face burst module includes：
Unit is marked, for carrying out face characteristic point mark to the current facial image；
Unit for scaling is rotated, the current facial image is rotated and/or contracted for the characteristic point according to mark
Put；
Unit is cut out, for being cut out to the facial image after the rotation and/or scaling, to obtain face burst figure
Picture.
The Enhancement Method and system of facial image provided by the invention, current facial image is carried out at face burst
Reason, and the first image enhaucament model by building in advance is once strengthened, the face slice images strengthened；Will enhancing
Face slice images the corresponding part of the current facial image is replaced after, once strengthened facial image.
Compared with prior art, method and system provided in an embodiment of the present invention improve the disposal ability to high-definition picture, energy
Access the facial image of high-resolution, high quality.
Further, secondary enhancing, final output are carried out to once strengthening facial image by the second image enhaucament model
Secondary enhancing facial image be further enhance high-resolution, the facial image of high quality.
Brief description of the drawings
Fig. 1 is the flow chart of the Enhancement Method of facial image provided in an embodiment of the present invention；
Fig. 2 is another flow chart of the Enhancement Method of facial image provided in an embodiment of the present invention；
Fig. 3 is the structural representation of the first image enhaucament model；
Fig. 4 is the structural representation of the second image enhaucament model；
Fig. 5 is the flow chart of the training method of the second image enhaucament model in the embodiment of the present invention；
Fig. 6 is another flow chart of the training method of the second image enhaucament model in the embodiment of the present invention；
Fig. 7 is the flow chart of face slice images processing method in the embodiment of the present invention；
Fig. 8 is the effect diagram after the processing of face slice images in the embodiment of the present invention；
Fig. 9 is the structural representation of the strengthening system of facial image provided in an embodiment of the present invention；
Figure 10 is the structural representation of the strengthening system of second of facial image provided in an embodiment of the present invention；
Figure 11 is the structural representation of the strengthening system of the third facial image provided in an embodiment of the present invention；
Figure 12 is the structural representation of the strengthening system of the 4th kind of facial image provided in an embodiment of the present invention；
Figure 13 is the structural representation of face burst module in the embodiment of the present invention.
Embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, is only used for explaining the present invention, and is not construed as limiting the claims.
The facial image photo being related in the embodiment of the present invention is only used for carrying out effect explanation, and it is big that it comes from Hong Kong Chinese
A data set CUFS disclosed in.
As shown in figure 1, the flow chart of the Enhancement Method for facial image provided in an embodiment of the present invention, this method include with
Lower step (it will be appreciated by persons skilled in the art that the step of being related in the present embodiment is not limited sequentially with this, is only entered with this
Row explanation)：
Step S100, current facial image is obtained.
Current facial image can be low-quality fuzzy facial image or the facial image of Freehandhand-drawing.
Step S200, face burst processing is carried out to the current facial image, to obtain face slice images.
Human face five-sense-organ is positioned first, takes face slice images after positioning respectively." five be related in the present embodiment
Official " refers to left eye, right eye, nose and mouth respectively, because the slice images of ear area shared in whole facial image is smaller, and
Visual effect and recognition effect are not influenceed, " face " being related in the present embodiment do not consider ear.Therefore, five in the present embodiment
Official's slice images refer to left eye slice images, right eye slice images, nose slice images and face slice images.
Face burst processing method will be specifically addressed below, so as to which the face slice images for making to obtain have setting
Size.
Step S300, the face slice images are separately input to the first image enhaucament model each built in advance,
The face slice images strengthened.
The first image enhaucament model can be that production of the prior art resists neural network model, its training method
It will be described in more detail later.
Each face slice images are strengthened by the first image enhaucament model, obtain the face burst figure of enhancing
Picture.
Step S400, the face slice images of the enhancing are replaced to the corresponding portion of the current facial image respectively
Point, once to be strengthened facial image.
The face slice images of the enhancing obtained in step S300 are replaced to the corresponding part of current facial image, wherein
The lap pixel value of face slice images takes average, so as to once be strengthened facial image.
The Enhancement Method of facial image provided in an embodiment of the present invention, current facial image is carried out at face burst
Reason, and the first image enhaucament model by building in advance is strengthened, the face slice images strengthened；By the five of enhancing
Official's slice images are replaced to the corresponding part of current facial image, so as to once be strengthened facial image.With it is existing
Technology is compared, and method provided in an embodiment of the present invention improves the disposal ability to high-definition picture, can obtain high-resolution
The facial image of rate, high quality.
As shown in Fig. 2 another flow chart of the Enhancement Method for facial image provided in an embodiment of the present invention, preferably
Ground, after step S400, methods described also includes：
Step S500, the once enhancing facial image is input to the second image enhaucament model built in advance, obtained
Secondary enhancing facial image.
Due to the once enhancing facial image that is obtained in step S400 it is possible that face collocation is uncoordinated, face point
The problems such as sheet border is excessively obvious, the second image enhaucament model, the secondary enhancing people of final output are introduced in step S500
Face image is high-resolution, the facial image of high quality further enhanced.
Mentioned above, the first image enhaucament model can be production confrontation neural network model of the prior art, raw
Accepted way of doing sth confrontation neural network model is generally made up of two parts, i.e. generation model (Generator, G model) and discrimination model
(Discriminator, D model).G models are used to generate target image (Generator Example), and D models then replace damage
Lose function layer, by the contrast with true picture (Real Example), judge generation target image it is true and false, then instruct
G models preferably generate target image.
Specific to the present embodiment, the first image enhaucament model includes the first generation model and the first discrimination model.First life
Input into model is face slice images, exports the face slice images for enhancing.The input of first discrimination model is enhancing
Face slice images and high quality face slice images, export for two classification judgment values.
Lower mask body introduces the training process of the first image enhaucament model：In this network model, the first generation model
The input of (G models) is face slice images, and output is the face slice images of enhancing；First discrimination model (D models) only exists
Training stage uses, and input is the output of face slice images, i.e. G models that we strengthen, and the face burst figure of high quality
Picture.D models receive one true (the face slice images of high quality) one vacation (the face slice images of the enhancing of G models generation), enter
The classification of row two judges, exports judgment value, so as to instruct the generation of G models closer to real image.
Using D models output as image classification foundation, its output between 0 to 1,0 represent D models judge image as
Vacation, 1, which represents D models, judges image to be true.
Make pdataAnd p (x)g(x) distribution of the enhancing image of distribution with the generation of true high quality graphic is represented respectively, is sentenced
The object function of other model is as follows：
The function target is to enable D models to be determined as very, being false by enhanced image discriminating, that is, doing by high quality graphic
Go out correct judgement.And the target of G models is then that can generate the image of fascination D models, mixes the spurious with the genuine, allow D models cannot be distinguished by
True picture and generation image, then the object function of whole model is as follows：
The fixed party during training, updates the network weight of the opposing party, alternating iteration, when G models are fixed, most
Excellent D models are：
In an iterative process, both sides strongly optimize the network of oneself, so as to form competition confrontation, when both sides reach one
During dynamic balance, pg(x)=pdata(x), optimal D models are：
This explanation, D models cannot be distinguished by true and false (true and false probability respectively accounts for 50%) of input picture, and training terminates.This
When it is considered that G models can generate face slice images real enough.Using the enhancing model of right eye slice images as
Example, optimum state are as shown in Figure 3.
It is previously mentioned because true and false judgement of the D models to big image is easily made a fault, therefore traditional directly using is given birth to
The method of accepted way of doing sth confrontation neutral net reconstructed image is hardly produced the big image of high quality.But herein, we input D models
Facial image face burst carried out enhancing, the now effect of D models, which can be considered as to eliminate face bursts and replace, to be caused
Edge blocky effect, and the enhanced quality in other regions of image, so judge that difficulty substantially reduces, can accomplish pair
The judgement of higher resolution image (being 500*500 in the present embodiment).
Similarly, the second image enhaucament model can also be production confrontation neural network model of the prior art, wrap
Include the second generation model and the second discrimination model.The input of second generation model is exported to be secondary once to strengthen facial image
Strengthen facial image.The input of second discrimination model is secondary enhancing facial image and the facial image of high quality, is exported as two
Classification judgment value.
Preferably, the second image enhaucament model also includes the human face recognition model based on convolutional neural networks structure.Such as Fig. 4
It is shown, it is the structural representation of the second image enhaucament model, the preferable second image enhaucament model introduces an optimization mould
Type, the Optimized model is the good human face recognition model (face recognize model, F model) of training in advance, for keeping
Face characteristic consistency.F models include but is not limited to disclosed human face recognition model (such as VGG_face models) or voluntarily trained
The human face recognition model with certain recognition of face effect.
The input of the human face recognition model (F models) is secondary enhancing facial image, is exported as secondary enhancing facial image
Characteristic vector.
As shown in figure 5, be the flow chart of the training method of the second image enhaucament model in the embodiment of the present invention, the training side
Method is trained for being not introduced into the second image enhaucament model of F models, including：
Step S510, the training set for once strengthening facial image is input to second generation model.
Step S520, by the secondary enhancing facial image of second generation model generation and the facial image of high quality
Training set is input in second discrimination model.
The training set source of the facial image of high quality includes but is not limited to open face data set (such as human face data collection
CUHK, include hundreds of high quality facial images and corresponding face hand-drawing image), the human face data collection that voluntarily handles (such as
The high quality facial image voluntarily collected, Fuzzy Processing is then carried out to facial image and obtains corresponding fuzzy facial image).
Step S530, the two classification judgment values exported according to second discrimination model, obtain first object function, by institute
The gradient anti-pass of first object function is stated to second generation model.
First object function is：
Step S540, when two classification judgment values of second discrimination model output reach the first setting value, instruction is stopped
Practice.
The training process of above-mentioned second image enhaucament model and the training process of the first image enhaucament model are essentially identical, tool
Body repeats no more.
As shown in fig. 6, be another flow chart of the training method of the second image enhaucament model in the embodiment of the present invention, should
Training method is trained for introducing the second image enhaucament model of F models, including：
Step S501, the training set for once strengthening facial image is input to second generation model.
Step S502, by the secondary enhancing facial image of second generation model generation and the facial image of high quality
Training set is input in second discrimination model；Meanwhile the secondary enhancing facial image for generating second generation model
It is input in the human face recognition model.
Step S503, the two classification judgment values exported according to second discrimination model, obtain first object function, by institute
The gradient anti-pass of first object function is stated to second generation model.
First object function is：
Step S504, the institute exported according to the characteristic vector of the facial image of the high quality and the human face recognition model
The distance between characteristic vector of secondary enhancing facial image is stated, the second object function is obtained, by second object function
Gradient anti-pass is to second generation model.
Step S505, when two classification judgment values of second discrimination model output reach the first setting value, and the height
The characteristic vector of the secondary enhancing facial image of the characteristic vector of the facial image of quality and the human face recognition model output it
Between distance reach the second setting value when, deconditioning.
F models can export the characteristic vector of a facial image, pass through the face figure of the face feature vector and high quality
Euclidean distance between the characteristic vector of picture, judge the similitude of face.
The object function of F models, i.e. the second object function are：
F models and the second discrimination model (D models) are connected in parallel to behind the second generation model (G models), are referred to jointly
Lead the training of the second generation model.F models train in itself, it is not necessary to its parameters weighting of retraining.
The gradient of F models the second object function of anti-pass gives G models, for updating the parameters weighting of G models.Wherein D models
G models are instructed to generate face real enough, F models instruct the reconstruct facial image of G models generation on feature Euclidean distance
It is close enough with true high quality facial image, so as to ensure that the face of generation will not become strange facial image.
Overall goals function of the second image enhaucament model after F models are introduced is as follows：
Wherein wdWeight factor, w are lost for D modelsfWeight factor, p are lost for F modelsdataAnd p (x)g(x) represent respectively
The distribution of the enhancing image of distribution with the generation of true high quality graphic.The present embodiment provides a kind of preferable weight value：
As shown in fig. 7, it is the flow chart of face slice images processing method in the embodiment of the present invention, at face slice images
Reason method comprises the following steps：
Step S210, face characteristic point mark is carried out to the current facial image.
Features described above point includes left eye, right eye, nose and lip center, mask method include but is not limited to artificial mark,
Facial feature points detection model mark (the facial feature points detection interface in such as OpenCV+Dlib storehouses, and some god for increasing income
, can be with the features described above point of quick detection facial image through network model).Demarcate pupil of left eye coordinate (xleye,yleye), right eye
Pupil coordinate (xreye,yreye), nose coordinate (xnose,ynose), lip centre coordinate (xmouth,ymouth)。
Step S220, the current facial image is rotated and/or scaled according to the characteristic point of mark.
By the means for rotating and/or scaling, facial image is set to snap to unified size and posture facial image (in this case
In for face vertical pendulum just, size it is basically identical).It shown below is the rotation used in the present embodiment, scaling formula：
Rotation formula is as follows：
Calculate the anglec of rotation of image：
The x coordinate conversion formula of arbitrfary point：X=(y0-ynose)sinθ+(x0-xnose)cosθ+xnose
The y-coordinate rotation formula of arbitrfary point：Y=(y0-ynose)cosθ-(x0-xnose)sinθ+ynose
Above-mentioned rotation transformation is centered on nose point coordinates, with De Salle's line (i.e. vertical line in T) vertically for standard, by original coordinates
Point (x0,y0) coordinate points (x, y) are moved to, keep face to be vertically aligned after rotation；Then postrotational facial image is contracted
Put, facial image is zoomed to unified size.The standard used in the present embodiment is：The ordinate at right and left eyes line midpoint
With L=150 pixel of ordinate distance of lip central point.This numerical value for reference, but can be not limited to this referential data.Scaling
Yardstick s calculation formula are：
Step S230, the facial image after the rotation and/or scaling is cut out, to obtain face slice images.
Centered on right and left eyes line midpoint, the high quality facial image of the alignment demarcation of fixed size is cut out, it is corresponding
Facial image to be reconstructed make same treatment.Face burst is taken centered on characteristic point, the image slices of long a width of S*S sizes,
Training data as face burst enhancing model.
In the present embodiment, the fixed size for cutting facial image is 500*500, and face burst size is 120*120, but not
It is limited to this size, it is as shown in Figure 8 cuts out effect.
Correspondingly, the embodiment of the present invention additionally provides a kind of facial image strengthening system, as shown in figure 9, being the system
Structural representation.
In this embodiment, the system includes：
Acquisition module 111, for obtaining current facial image；
Face burst module 112, for carrying out face burst processing to the current facial image, to obtain face point
Picture；
First image enhancement module 113, for the face slice images are separately input to each to build in advance the
One image enhaucament model, the face slice images strengthened；
Image replacement module 114, for the face slice images of the enhancing to be replaced into the current face figure respectively
The corresponding part of picture, once to be strengthened facial image.
Acquisition module 111 obtains current facial image, and current facial image is carried out by face burst module 112
The processing of face burst, and face slice images are separately input to what is each built in advance by the first image enhancement module 113
First image enhaucament model is strengthened, the face slice images strengthened；By image replacement module 114 by the five of enhancing
After official's slice images are replaced to the corresponding part of current facial image, once strengthened facial image.With existing skill
Art is compared, and system provided in an embodiment of the present invention improves the disposal ability to high-definition picture, can obtain high-resolution,
The facial image of high quality.
As shown in Figure 10, for second of facial image provided in an embodiment of the present invention strengthening system structural representation,
Unlike Fig. 9, in the embodiment, the system further comprises：
Second image enhancement module 115, for the once enhancing facial image to be input into the second figure built in advance
Image intensifying model, obtain secondary enhancing facial image.
Because obtained once enhancing facial image is it is possible that face collocation is uncoordinated, face slice boundaries are excessively bright
The problems such as aobvious, introduce the second image enhaucament model in Figure 10 embodiment, the secondary enhancing facial image of final output be into
The high-resolution of one step enhancing, the facial image of high quality.
As shown in figure 11, for the third facial image provided in an embodiment of the present invention strengthening system structural representation,
Unlike Fig. 9 or Figure 10, in the embodiment, the system further comprises：
First structure module 116, for building the first image enhaucament model.First image enhaucament model make a living an accepted way of doing sth confrontation
Neural network model, including the first generation model and the first discrimination model.
The structure of the first image enhaucament model is having been described above, and will not be repeated here.
As shown in figure 12, for the 4th kind of facial image provided in an embodiment of the present invention strengthening system structural representation,
Unlike Figure 11, the system still further comprises：
Second structure module 117, for building the second image enhaucament model；The second image enhaucament model includes
Production resists neural network model, including the second generation model and the second discrimination model.
The input of second generation model is exported as secondary enhancing facial image once to strengthen facial image.Second differentiates
The input of model is secondary enhancing facial image and the facial image of high quality, is exported as two classification judgment values.
Preferably, the second image enhaucament model also includes the human face recognition model based on convolutional neural networks structure.Face
The input of identification model is secondary enhancing facial image, is exported as the characteristic vector of secondary enhancing facial image.
The structure of the preferable second image enhaucament model is as shown in figure 4, will not be repeated here.
Preferably, the second structure module 117 also includes training unit, when the second image that the second structure module 117 is built
Strengthen model using production confrontation neural network model, without introduce F models when, the training unit is used for：
The training set for once strengthening facial image is input to second generation model；
The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is defeated
Enter into second discrimination model；
When two classification judgment values of second discrimination model output reach the first setting value, deconditioning.
Preferably, when the second image enhaucament model that the second structure module 117 is built resists neutral net using production
Model, and during introducing F models, the training unit is used for：
The secondary enhancing facial image that second generation model generates is input in the human face recognition model；
The secondary increasing exported according to the characteristic vector of the facial image of the high quality and the human face recognition model
The distance between characteristic vector of strong man's face image, the second object function is obtained, by the gradient anti-pass of second object function
To second generation model；
While two classification judgment values of second discrimination model output reach the first setting value, and the high quality
Facial image characteristic vector and the human face recognition model output secondary enhancing facial image characteristic vector between
When distance reaches the second setting value, deconditioning.
Neural network model is resisted by the production of structure, and introduces the human face recognition model of an optimization, to generation
Formula confrontation neural network model is trained, and is obtained the second optimal image enhaucament model, is realized to once strengthening facial image
Secondary enhancing is carried out, obtains the facial image of high quality.
As shown in figure 13, it is the structural representation of face burst module in the embodiment of the present invention, in this embodiment, face
Burst module 112 specifically includes：
Unit 1121 is marked, for carrying out face characteristic point mark to the current facial image；
Rotate unit for scaling 1122, for the characteristic point according to mark to the current facial image carry out rotation and/
Or scaling；
Unit 1123 is cut out, for being cut out to the facial image after the rotation and/or scaling, to obtain face point
Picture.
Can be by marking unit 1121, using the method for artificial mark or facial feature points detection model mark to current
Facial image carry out face characteristic point mark, then the facial image after mark is rotated by rotating unit for scaling 1122
And/or scaling, facial image is cut out finally by unit 1123 is cut out, so as to obtain the face burst figure of suitable dimension
Picture.
Using facial image strengthening system provided in an embodiment of the present invention, obtained face is true thin in face details
Cause, the facial image of high-resolution, high quality can be obtained.
Although the present invention is described with reference to above example, the present invention is not limited to above-described embodiment, and
Only limited by claim, those of ordinary skill in the art easily can modify and change to it, but and without departing from
The essential idea and scope of the present invention.
Claims (15)
- A kind of 1. Enhancement Method of facial image, it is characterised in that including：Obtain current facial image；Face burst processing is carried out to the current facial image, to obtain face slice images；The first image enhaucament model that the face slice images are separately input to each build in advance, the face strengthened Slice images；The face slice images of the enhancing are replaced to the corresponding part of the current facial image respectively, once to be increased Strong man's face image.
- 2. according to the method for claim 1, it is characterised in that methods described also includes：The once enhancing facial image is input to the second image enhaucament model built in advance, obtains secondary enhancing face figure Picture.
- 3. according to the method for claim 1, it is characterised in that described first image enhancing model is production confrontation nerve Network model, including the first generation model and the first discrimination model；The input of first generation model is the face slice images；The output of first generation model is the enhancing Face slice images；The face slice images inputted as the enhancing of first discrimination model and the face slice images of high quality, it is described The output of first discrimination model is two classification judgment values.
- 4. according to the method for claim 2, it is characterised in that the second image enhaucament model includes production confrontation god Through network model, including the second generation model and the second discrimination model；The input of second generation model once strengthens facial image to be described；The output of second generation model is described Secondary enhancing facial image；The input of second discrimination model is the secondary enhancing facial image and the facial image of high quality, and described second sentences The output of other model is two classification judgment values.
- 5. according to the method for claim 4, it is characterised in that the second image enhaucament model passes through following training method It is trained：The training set for once strengthening facial image is input to second generation model；The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is input to In second discrimination model；The two classification judgment values exported according to second discrimination model, obtain first object function, by the first object letter Several gradient anti-pass is to second generation model；When two classification judgment values of second discrimination model output reach the first setting value, deconditioning.
- 6. according to the method for claim 4, it is characterised in that the second image enhaucament model is also included based on convolution god Human face recognition model through network struction；The input of the human face recognition model is the secondary enhancing facial image, and the output of the human face recognition model is described The characteristic vector of secondary enhancing facial image.
- 7. according to the method for claim 6, it is characterised in that the second image enhaucament model passes through following training method It is trained：：The training set for once strengthening facial image is input to second generation model；The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is input to In second discrimination model；Meanwhile the secondary enhancing facial image for generating second generation model is input to the people In face identification model；The two classification judgment values exported according to second discrimination model, obtain first object function, by the first object letter Several gradient anti-pass is to second generation model；The secondary enhancing face figure exported according to the characteristic vector of the facial image of high quality and the human face recognition model The distance between characteristic vector of picture, the second object function is obtained, by the gradient anti-pass of second object function to described Two generation models；When two classification judgment values of second discrimination model output reach the first setting value, and the facial image of the high quality The distance between the characteristic vector of secondary enhancing facial image of characteristic vector and the human face recognition model output reach the During two setting values, deconditioning.
- 8. according to the method described in claim any one of 1-7, it is characterised in that described that the current facial image is carried out The processing of face burst, is included with obtaining face slice images：Face characteristic point mark is carried out to the current facial image；The current facial image is rotated and/or scaled according to the characteristic point of mark；Facial image after the rotation and/or scaling is cut out, to obtain face slice images.
- A kind of 9. strengthening system of facial image, it is characterised in that including：Acquisition module, for obtaining current facial image；Face burst module, for carrying out face burst processing to the current facial image, to obtain face slice images；First image enhancement module, the first image for being separately input to each build in advance by the face slice images increase Strong model, the face slice images strengthened；Image replacement module, for the face slice images of the enhancing to be replaced to the correspondence of the current facial image respectively Part, once to be strengthened facial image.
- 10. system according to claim 9, it is characterised in that the system also includes：Second image enhancement module, for the once enhancing facial image to be input into the second Image Enhancement Based built in advance Type, obtain secondary enhancing facial image.
- 11. system according to claim 10, it is characterised in that the system also includes：Second structure module, for building the second image enhaucament model；The second image enhaucament model includes production Resist neural network model, including the second generation model and the second discrimination model；The input of second generation model once strengthens facial image to be described；The output of second generation model is described Secondary enhancing facial image；The input of second discrimination model is the secondary enhancing facial image and the facial image of high quality, and described second sentences The output of other model is two classification judgment values.
- 12. system according to claim 11, it is characterised in that the second structure module includes training unit, is used for：The training set for once strengthening facial image is input to second generation model；The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is input to In second discrimination model；The two classification judgment values exported according to second discrimination model, obtain first object function, by the first object letter Several gradient anti-pass is to second generation model；When two classification judgment values of second discrimination model output reach the first setting value, deconditioning.
- 13. system according to claim 11, it is characterised in that the second image enhaucament model also includes being based on convolution The human face recognition model of neutral net structure；The input of the human face recognition model is the secondary enhancing facial image, and the output of the human face recognition model is described The characteristic vector of secondary enhancing facial image.
- 14. system according to claim 13, it is characterised in that the second structure module includes training unit, is used for：The training set for once strengthening facial image is input to second generation model；The training set of the secondary enhancing facial image of second generation model generation and the facial image of high quality is input to In second discrimination model；Meanwhile the secondary enhancing facial image for generating second generation model is input to the people In face identification model；The two classification judgment values exported according to second discrimination model, obtain first object function, by the first object letter Several gradient anti-pass is to second generation model；The secondary enhancing face figure exported according to the characteristic vector of the facial image of high quality and the human face recognition model The distance between characteristic vector of picture, the second object function is obtained, by the gradient anti-pass of second object function to described Two generation models；When two classification judgment values of second discrimination model output reach the first setting value, and the facial image of the high quality The distance between the characteristic vector of secondary enhancing facial image of characteristic vector and the human face recognition model output reach the During two setting values, deconditioning.
- 15. according to the system described in claim any one of 9-14, it is characterised in that the face burst module includes：Unit is marked, for carrying out face characteristic point mark to the current facial image；Unit for scaling is rotated, the current facial image is rotated and/or scaled for the characteristic point according to mark；Unit is cut out, for being cut out to the facial image after the rotation and/or scaling, to obtain face slice images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711422950.3A CN107895358A (en) | 2017-12-25 | 2017-12-25 | The Enhancement Method and system of facial image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711422950.3A CN107895358A (en) | 2017-12-25 | 2017-12-25 | The Enhancement Method and system of facial image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107895358A true CN107895358A (en) | 2018-04-10 |
Family
ID=61808271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711422950.3A Pending CN107895358A (en) | 2017-12-25 | 2017-12-25 | The Enhancement Method and system of facial image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107895358A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596062A (en) * | 2018-04-12 | 2018-09-28 | 清华大学 | The real-time high-intensity region method and device of face picture based on deep learning |
| CN108648163A (en) * | 2018-05-17 | 2018-10-12 | 厦门美图之家科技有限公司 | A kind of Enhancement Method and computing device of facial image |
| CN108830209A (en) * | 2018-06-08 | 2018-11-16 | 西安电子科技大学 | Based on the remote sensing images method for extracting roads for generating confrontation network |
| CN109448083A (en) * | 2018-09-29 | 2019-03-08 | 浙江大学 | A method of human face animation is generated from single image |
| CN109459043A (en) * | 2018-12-12 | 2019-03-12 | 上海航天控制技术研究所 | A kind of spacecraft Relative Navigation based on production reconstructed image |
| CN109493297A (en) * | 2018-11-01 | 2019-03-19 | 重庆中科云丛科技有限公司 | Low quality facial image Enhancement Method, system, equipment and storage medium |
| CN109859113A (en) * | 2018-12-25 | 2019-06-07 | 北京奇艺世纪科技有限公司 | Model generating method, image enchancing method, device and computer readable storage medium |
| CN109858364A (en) * | 2018-12-28 | 2019-06-07 | 维沃移动通信有限公司 | A kind of processing method and mobile terminal of facial image |
| CN110570358A (en) * | 2018-09-04 | 2019-12-13 | 阿里巴巴集团控股有限公司 | vehicle loss image enhancement method and device based on GAN network |
| WO2020056769A1 (en) * | 2018-09-21 | 2020-03-26 | Intel Corporation | Method and system of facial resolution upsampling for image processing |
| WO2020056901A1 (en) * | 2018-09-20 | 2020-03-26 | 北京字节跳动网络技术有限公司 | Method and device for processing image |
| WO2020062120A1 (en) * | 2018-09-29 | 2020-04-02 | 浙江大学 | Method for generating facial animation from single image |
| CN112308043A (en) * | 2020-11-26 | 2021-02-02 | 腾讯科技（深圳）有限公司 | Image processing method, image processing apparatus, and computer-readable storage medium |
| CN112699887A (en) * | 2020-12-30 | 2021-04-23 | 科大讯飞股份有限公司 | Method and device for obtaining mathematical object labeling model and mathematical object labeling |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1525401A (en) * | 2003-02-28 | 2004-09-01 | ��˹���´﹫˾ | Method and system for enhancing portrait images that are processed in a batch mode |
| US20140254932A1 (en) * | 2012-11-27 | 2014-09-11 | Texas Instruments Incorporated | Content adaptive edge and detail enhancement for image and video processing |
| CN106023104A (en) * | 2016-05-16 | 2016-10-12 | 厦门美图之家科技有限公司 | Human face eye area image enhancement method and system and shooting terminal |
| CN106056562A (en) * | 2016-05-19 | 2016-10-26 | 京东方科技集团股份有限公司 | Face image processing method and device and electronic device |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
-
2017
- 2017-12-25 CN CN201711422950.3A patent/CN107895358A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1525401A (en) * | 2003-02-28 | 2004-09-01 | ��˹���´﹫˾ | Method and system for enhancing portrait images that are processed in a batch mode |
| US20140254932A1 (en) * | 2012-11-27 | 2014-09-11 | Texas Instruments Incorporated | Content adaptive edge and detail enhancement for image and video processing |
| CN106023104A (en) * | 2016-05-16 | 2016-10-12 | 厦门美图之家科技有限公司 | Human face eye area image enhancement method and system and shooting terminal |
| CN106056562A (en) * | 2016-05-19 | 2016-10-26 | 京东方科技集团股份有限公司 | Face image processing method and device and electronic device |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596062A (en) * | 2018-04-12 | 2018-09-28 | 清华大学 | The real-time high-intensity region method and device of face picture based on deep learning |
| CN108648163A (en) * | 2018-05-17 | 2018-10-12 | 厦门美图之家科技有限公司 | A kind of Enhancement Method and computing device of facial image |
| CN108830209A (en) * | 2018-06-08 | 2018-11-16 | 西安电子科技大学 | Based on the remote sensing images method for extracting roads for generating confrontation network |
| CN108830209B (en) * | 2018-06-08 | 2021-12-17 | 西安电子科技大学 | Remote sensing image road extraction method based on generation countermeasure network |
| CN110570358A (en) * | 2018-09-04 | 2019-12-13 | 阿里巴巴集团控股有限公司 | vehicle loss image enhancement method and device based on GAN network |
| WO2020048271A1 (en) * | 2018-09-04 | 2020-03-12 | 阿里巴巴集团控股有限公司 | Gan network-based vehicle damage image enhancement method and apparatus |
| WO2020056901A1 (en) * | 2018-09-20 | 2020-03-26 | 北京字节跳动网络技术有限公司 | Method and device for processing image |
| WO2020056769A1 (en) * | 2018-09-21 | 2020-03-26 | Intel Corporation | Method and system of facial resolution upsampling for image processing |
| WO2020062120A1 (en) * | 2018-09-29 | 2020-04-02 | 浙江大学 | Method for generating facial animation from single image |
| CN109448083A (en) * | 2018-09-29 | 2019-03-08 | 浙江大学 | A method of human face animation is generated from single image |
| US11544887B2 (en) | 2018-09-29 | 2023-01-03 | Zhejiang University | Method for generating facial animation from single image |
| CN109493297A (en) * | 2018-11-01 | 2019-03-19 | 重庆中科云丛科技有限公司 | Low quality facial image Enhancement Method, system, equipment and storage medium |
| CN109493297B (en) * | 2018-11-01 | 2020-08-18 | 重庆中科云从科技有限公司 | Low-quality face image enhancement method, system, equipment and storage medium |
| CN109459043A (en) * | 2018-12-12 | 2019-03-12 | 上海航天控制技术研究所 | A kind of spacecraft Relative Navigation based on production reconstructed image |
| CN109859113A (en) * | 2018-12-25 | 2019-06-07 | 北京奇艺世纪科技有限公司 | Model generating method, image enchancing method, device and computer readable storage medium |
| CN109858364A (en) * | 2018-12-28 | 2019-06-07 | 维沃移动通信有限公司 | A kind of processing method and mobile terminal of facial image |
| CN112308043A (en) * | 2020-11-26 | 2021-02-02 | 腾讯科技（深圳）有限公司 | Image processing method, image processing apparatus, and computer-readable storage medium |
| CN112699887A (en) * | 2020-12-30 | 2021-04-23 | 科大讯飞股份有限公司 | Method and device for obtaining mathematical object labeling model and mathematical object labeling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107895358A (en) | The Enhancement Method and system of facial image | |
| CN108765319A (en) | A kind of image de-noising method based on generation confrontation network | |
| CN110222628A (en) | A kind of face restorative procedure based on production confrontation network | |
| CN109344693A (en) | A kind of face multizone fusion expression recognition method based on deep learning | |
| CN108416752A (en) | A method of image is carried out based on production confrontation network and removes motion blur | |
| JP6788264B2 (en) | Facial expression recognition method, facial expression recognition device, computer program and advertisement management system | |
| CN108520504A (en) | A kind of blurred picture blind restoration method based on generation confrontation network end-to-end | |
| CN109376582A (en) | A kind of interactive human face cartoon method based on generation confrontation network | |
| CN106780466A (en) | A kind of cervical cell image-recognizing method based on convolutional neural networks | |
| CN107481188A (en) | A kind of image super-resolution reconstructing method | |
| CN107358626A (en) | A kind of method that confrontation network calculations parallax is generated using condition | |
| CN109584251A (en) | A kind of tongue body image partition method based on single goal region segmentation | |
| CN108470320A (en) | A kind of image stylizing method and system based on CNN | |
| CN104850825A (en) | Facial image face score calculating method based on convolutional neural network | |
| CN108629753A (en) | A kind of face image restoration method and device based on Recognition with Recurrent Neural Network | |
| CN108665463A (en) | A kind of cervical cell image partition method generating network based on confrontation type | |
| CN108932536A (en) | Human face posture method for reconstructing based on deep neural network | |
| CN107145893A (en) | A kind of image recognition algorithm and system based on convolution depth network | |
| CN105426695A (en) | Health status detecting system and method based on irises | |
| CN108388874A (en) | Prawn morphological parameters method for automatic measurement based on image recognition and cascade classifier | |
| CN112541422B (en) | Expression recognition method, device and storage medium with robust illumination and head posture | |
| CN111242865A (en) | Fundus image enhancement method based on generation type countermeasure network | |
| CN107832740A (en) | The Method of Teaching Quality Evaluation and system of a kind of remote teaching | |
| CN109359527A (en) | Hair zones extracting method and system neural network based | |
| CN107316032A (en) | One kind sets up facial image identifier method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180410 |