CN110197462A - A kind of facial image beautifies in real time and texture synthesis method - Google Patents
A kind of facial image beautifies in real time and texture synthesis method Download PDFInfo
- Publication number
- CN110197462A CN110197462A CN201910305804.5A CN201910305804A CN110197462A CN 110197462 A CN110197462 A CN 110197462A CN 201910305804 A CN201910305804 A CN 201910305804A CN 110197462 A CN110197462 A CN 110197462A
- Authority
- CN
- China
- Prior art keywords
- image
- face
- dimensional
- eyebrow
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001815 facial effect Effects 0.000 title claims abstract description 43
- 238000001308 synthesis method Methods 0.000 title claims abstract description 13
- 210000004709 eyebrow Anatomy 0.000 claims abstract description 50
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000006243 chemical reaction Methods 0.000 claims abstract description 20
- 230000004927 fusion Effects 0.000 claims abstract description 12
- 230000008569 process Effects 0.000 claims abstract description 12
- 230000008439 repair process Effects 0.000 claims abstract description 5
- 238000013507 mapping Methods 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 5
- 210000001508 eye Anatomy 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 23
- 238000001514 detection method Methods 0.000 abstract description 6
- 238000010276 construction Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 11
- 238000005286 illumination Methods 0.000 description 6
- 238000012937 correction Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 210000004209 hair Anatomy 0.000 description 3
- 210000000214 mouth Anatomy 0.000 description 3
- 210000001331 nose Anatomy 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000003796 beauty Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000002087 whitening effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G06T5/77—
-
- G06T5/94—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of model construction method, in particular to a kind of facial image towards three-dimensional facial reconstruction beautifies in real time and texture synthesis method.The present invention can realize automatic detection and beautification, to obtain preferable human face rebuilding result by extracting human face characteristic point, color conversion, face is seamless fusion and processes such as eyebrow repair process, three-dimensional facial reconstruction using ASM algorithm to face.The experimental results showed that this paper algorithm can obtain satisfactorily rebuilding effect, there is superior operation efficiency including the brow region being blocked can be rebuild, and compared with other methods, real-time can be reached.The application is based on the standard colour of skin to facial image and carries out landscaping treatment, generates face texture maps, is applied to three-dimensional facial reconstruction, obtains personalized three-dimensional faceform.
Description
Technical field
The present invention relates to a kind of model construction method, in particular to a kind of facial image towards three-dimensional facial reconstruction are real-time
Beautification and texture synthesis method.
Background technique
In recent years, facial image textures synthesis and three-dimensional facial reconstruction have become graph and image processing, computer vision, people
The research hotspot content in the fields such as work intelligence and pattern-recognition.Body based on the three-dimensional facial reconstruction of image since user can be promoted
Sense is tested, is had a wide range of applications value in fields such as three-dimensional animation, computer game, virtual costume fittings.
Currently, the three-dimensional facial reconstruction based on image can be divided into reconstruction based on depth image, based on more image sequences
It rebuilds and the modes such as reconstruction based on single image.Based on the three-dimensional facial reconstruction of single image because using human face photo less, grasp
It is convenient to make to realize, it has also become the hot research direction in human face rebuilding.Reconstruction process based on single image includes facial image
The operations such as feature point extraction, flesh correction, facial image fusion, the synthesis of face texture maps and throe-dimensional temperature.It is existing
Algorithm for reconstructing does not account for the influence for the factors such as illumination is bad, and eyebrow blocks in face textures synthesis, so that reconstructed results are not
Enough beauties.Therefore, herein with emphasis on flesh correction, face is seamless fusion and eyebrow rebuild etc. optimize, sufficiently benefit
Realistic and aesthetic feeling face texture maps are generated with the information on single image.
In human face characteristic point extraction, active shape model (Active Shape Model) and active apparent model
(Active Appearance Model) is two methods the most classical, has started the two class classics framves for solving problems
Structure；Depth learning technology is a kind of information analysis modeling technique outstanding for appearing in machine learning field in recent years, the technology
It to the simulation of complex model and is approached based on the realization of multilayer neural network framework, there is announcement data inner link outstanding and knot
The ability of structure has effectively pushed the development of human face characteristic point extractive technique.On flesh correction, Qiu Jialiang et al. proposes colour of skin beauty
White algorithm can preferably highlight the insufficient face of illumination, but the algorithm can not realize colour of skin deviation and correct；
Reinhard et al. proposes classical color transfer algorithm, which has the tone reversal of image overall certain effect
Fruit, but there are problems in local color conversion；Chen et al. is propagated using the editor based on Local Liner Prediction, the party
Method requires user to choose the region for needing region to be converted and being not converted manually, is not carried out full automation.Scheming
As in seamless fusion, common method has weighted mean method, gaussian filtering method, multi-resolution method, the fusion method based on gradient field
Deng.Wherein, weighted mean method is simple, calculating speed is fast, but syncretizing effect is poor, is difficult elimination moving target and is formed by fuzzy yin
Shadow；Gaussian filtering method is smoothed image overlapping region using gaussian kernel function, easily causes On Local Fuzzy；Multiresolution
Method in different frequency bands, to realize the fusion transition of whole image, but need to repeatedly filter picture breakdown, and computationally intensive, Yi Zao
Weaken at signal, causes image fuzzy；The synthesis for realizing image in gradient field based on the method for gradient field, utilizes known image
Gradient information guides interpolation fusion, is not in blooming, syncretizing effect is good.On contours extract, Li et al. people proposes to be not required to
The Level Set Models to be initialized solve the problems, such as that traditional Level Set Models reinitialize, and improve contour fitting speed
Degree.
The work of the application, which will use a facial image, is propagated based on the editor of Local Liner Prediction to realize
The correction of the colour of skin realizes automatic choose using the characteristic point that ASM algorithm extracts for the area of skin color that needs are converted；Using pool
Loose edit methods merge in human face five-sense-organ region with standard broca scale；Meanwhile eyebrow is blocked, set forth herein a kind of combination Li
The method of model and Corner Detection rebuilds eyebrow outline；It finally will be on the face texture map to three-dimensional face model of generation.
Summary of the invention
In view of the deficiencies of the prior art, the present invention proposes a kind of algorithm frames, combine mark to the facial image of user's input
The quasi- colour of skin carries out landscaping treatment, generates texture maps, realizes three-dimensional facial reconstruction.Firstly, extracting face characteristic using ASM algorithm
Point is used for range constraint and texture mapping；Secondly, being realized using being propagated based on the editor of Local Liner Prediction to face
The color of the colour of skin is converted；Then, seamless the merging of human face region and standard colour of skin image is realized by graph cut, meanwhile, point
Situation carries out reduction reconstruction to eyebrow is blocked.By this series of processes, face texture maps after being beautified are mapped that
On three-dimensional face model, obtain rebuilding face.
The present invention is achieved by following technical proposals:
A kind of facial image beautifies in real time and texture synthesis method, it is characterised in that includes the following steps:
(1) human face characteristic point is extracted using ASM algorithm
Establish active shape model and local gray level model；Original shape matches target image, according to part
Gray level model is adjusted, until convergence；It detects obtained characteristic point and selects the region that eyebrow, eyes, nose, mouth are constituted
As region to be fused；
(2) color is converted
The tone of original image and the tone of target image are consistent by following color conversion formula:
In formula, c ∈ (r, g, b) c ∈ (r, g, b), IrPixel value after indicating conversion, IsIndicate original image pixel value,
means, meantRespectively indicate the pixel mean value of original image and target image, devs, devtRespectively indicate original image and target figure
The pixel criterion of picture is poor；
(3) the seamless fusion of face and eyebrow repair process
First setting on image S has a closure subregion Ω, and boundary isV is the gradient in region to be fused in original image
Vector field, f are the scalar function being defined on S, it is known that ff existsOn value be f*, then under ff takes V to guide in Ω
Interpolating function, i.e. solution extreme-value problem
In formula,For the gradient of f；
(4) three-dimensional facial reconstruction
By on the face texture map to threedimensional model obtained by above-mentioned algorithm, face texture mapping can be by two dimensional image
The upper characteristic point coordinate extracted by ASM algorithm is mapped with characteristic point coordinate corresponding on threedimensional model, utilizes OpenGL
The function library of offer realizes the texture mapping of three-dimensional face model.
Preferably, a kind of above-mentioned facial image beautifies in real time and local face of the texture synthesis method in step (2)
Method used in color conversion process is as follows:
Firstly, by data from High Dimensional Mapping to low-dimensional while, keep data manifold structure it is constant, use vector xiTo indicate
A pixel i in some feature space；Give a data set x1..., xN, for each pixel xi, choose it
K nearest-neighbor, be expressed as xi1..., xik, one group of weights omega will be calculatedijThe reconstruction for enabling the k neighborhood best
Pixel xi, calculated by minimizing following energy function
Constraint conditionIt acquiresAfterwards, with xiXiNeighborhood linear combination can rebuild
xixi；
Secondly, the method for using editor to propagate to be realized to the conversion of color；Given original image and target image, by most
Smallization following energy function transmits color:
Wherein, SiIt is edited at pixel i as a result, ti(i ∈ R) is the specified standard colour of skin, and R is to need to convert
Region.
Preferably, a kind of above-mentioned facial image beautifies in real time and texture synthesis method is in formula (3) further earth's surface
It is shown as matrix form:
E=[(I-W) S]T(I-W)S+(S-T)TΛ(S-T) (4)
S is that i-th of element is siVector, I is unit matrix, and Λ is the diagonal matrix that diagonal entry is λ, and T is i-th
A element is tiVector, WW be element be ωijMatrix.
The utility model has the advantages that the present invention can realize automatic detection and beautification, to obtain by an algorithm frame to face
Preferable human face rebuilding result.The experimental results showed that this paper algorithm can obtain satisfactorily rebuilding effect, including can weigh
The brow region being blocked is built, and there is superior operation efficiency compared with other methods, real-time can be reached.
Detailed description of the invention
The algorithm flow chart of Fig. 1 the application
Fig. 2-1 human face characteristic point extracts sample one
Fig. 2-2 human face characteristic point extracts sample two
Fig. 2-3 human face characteristic point extracts sample three
Fig. 3-1 color conversion effect compares (original image)
Fig. 3-2 color conversion effect compares (target figure)
Fig. 3-3 color conversion effect compares (reinhard method)
Fig. 3-4 color conversion effect compares (the application method)
Fig. 4-1 graph cut schematic diagram process (original image)
Fig. 4-2 graph cut schematic diagram process (image S)
Fig. 4-1 graph cut schematic diagram process (blending image)
Fig. 5 eyebrow mirror image merges (when side eyebrow is complete)
Fig. 6 eyebrow template merges (when two sides eyebrow is imperfect)
Fig. 7 rebuilds eyebrow outline process (a-b-c-d-e-f)
Fig. 8 rebuilds eyebrow effect picture
Fig. 9 three-dimensional face texture mapping
Figure 10-1 is in improper colour of skin situation human face rebuilding effect picture (original image)
Figure 10-2 is in improper colour of skin situation human face rebuilding effect picture (beautification texture maps)
Figure 10-3 is in improper colour of skin situation human face rebuilding effect picture (reconstructed results)
Figure 11-1 eyebrow is blocked the human face rebuilding effect picture (original image) of situation
Figure 11-2 eyebrow is blocked the human face rebuilding effect picture (beautification texture maps) of situation
Figure 11-3 eyebrow is blocked the human face rebuilding effect picture (reconstructed results) of situation
Specific embodiment
With reference to the accompanying drawing, implementation of the invention is illustrated:
Embodiment 1
Flow chart shown in 1 with reference to the accompanying drawings, a kind of facial image beautifies in real time and texture synthesis method, it is characterised in that including
Following step:
(1) human face characteristic point is extracted using ASM algorithm
Establish active shape model and local gray level model；Original shape matches target image, according to part
Gray level model is adjusted, until convergence；It detects obtained characteristic point and selects the region that eyebrow, eyes, nose, mouth are constituted
As region to be fused；
Human face characteristic point is extracted using ASM algorithm in the application.ASM algorithm passes through model training and pattern search two steps
It is rapid to realize feature point extraction.In model training stage, training sample set establishes active shape model and local gray level model；In mould
Type search phase, the original shape that the first step is obtained match target image, are then adjusted according to local gray level model
Parameter, until convergence.Final facial modeling is obtained in this way, as a result such as Fig. 2 (including Fig. 2-1, Fig. 2-2, Fig. 2-3)
It is shown.
(2) color is converted
The tone of original image and the tone of target image are consistent by following color conversion formula:
In formula, c ∈ (r, g, b) c ∈ (r, g, b), IrPixel value after indicating conversion, IsIndicate original image pixel value,
means, meantRespectively indicate the pixel mean value of original image and target image, devs, devtRespectively indicate original image and target figure
The pixel criterion of picture is poor；
For whitening or the amendment colour of skin deviation as caused by the extraneous factors such as illumination for realizing face complexion, we use color
The former colour of skin is converted to the specified colour of skin by conversion.The algorithm counts original image and the target image pixel in L α β color space
Mean value and standard deviation.
(3) the seamless fusion of face and eyebrow repair process
First setting on image S has a closure subregion Ω, and boundary isV is the gradient in region to be fused in original image
Vector field, f are the scalar function being defined on S, it is known that ff existsOn value be f*, then under ff takes V to guide in Ω
Interpolating function, i.e. solution extreme-value problem
In formula,For the gradient of f；
Because the problem is deconstructed into Poisson's equation, it is referred to as graph cut in this way.Fusion process such as Fig. 4 institute
Show.
The characteristic point that the application is detected by ASM algorithm selects eyebrow, eyes, nose, mouth automatically and constitutes
Region tentatively eliminates the problem of hair blocks as region to be fused in this way.Target image is a standard broca scale, setting
Fusion center is located at broca scale center, is in the human face region of fusion among broca scale, obtains a seamless face in this way
Texture maps are used for three-dimensional facial reconstruction.Since the color of standard broca scale is chosen according to the color of threedimensional model, texture maps
The solid colour of cheek edge color and threedimensional model, so Fusion Edges problem can be efficiently solved.
In addition it is contemplated that the case where eyebrow is blocked is detected by judging the gray value in eyebrow lateral angle vertex neighborhood
Obtain the different situations that eyebrow is blocked.When eyebrow occur and blocking, graph cut is divided to two regions to carry out, and a part is eye
The region of eyeball, nose, mouth composition, another part brow region are then individually handled, and can divide three kinds of situations:
(1) if side eyebrow is complete, other side eyebrow is blocked completely, using facial symmetry, by complete eyebrow water
Flat mirror picture is mapped to the other side, as shown in Figure 5.
(2) when eyebrow is blocked when two sides, a pair of of eyebrow template is chosen, two sides brow region is incorporated, such as Fig. 6 institute
Show.
(3) when side, eyebrow is complete, and when other side eyebrow is only partially blocked, we utilize complete eyebrow outline, simultaneously
In conjunction with the part eyebrow outline that the other side is not blocked, to rebuild the side eyebrow, to achieve the effect that closer to true eyebrow.I
Repair the eyebrow of this kind of situation using Li models coupling Corner Detection and block.Firstly, we are detected using Canny operator
To the eyebrow outline of complete side, then by the contour line flip horizontal, it is blocked the initial profile line of eyebrow as the other side.
Intermediate angle point is found in the brow region that is blocked simultaneously, is in this way the part not being blocked that demarcates with angle point, in this, as
Constraint to Li model evolution, evolution obtain final eyebrow outline.As shown in fig. 7, figure a is complete brow region, it is adopted
It is detected to obtain profile diagram b with Canny operator, is figure c by its flip horizontal.Then angle is found in the brow region for having partial occlusion
Point such as figure d, it is finally boundary with angle point that the profile for scheming c is then fitted in the region on the basis of eyebrow endpoint, which such as schemes e,
The brow region not being blocked obtains final reconstruction eyebrow outline by Li model evolution.It obtains after rebuilding eyebrow outline, it will
Original image two sides eyebrow is that boundary is merged with angle point.As shown in figure 8, a chooses the region not being blocked, b chooses complete eyebrow
The eyebrow c after being rebuild is merged in the two regions by the lateral area after overturning, which, which is utilized, partly blocks eyebrow
Partial information, closer in true eyebrow.
(4) three-dimensional facial reconstruction
By on the face texture map to threedimensional model obtained by above-mentioned algorithm, face texture mapping can be by two dimensional image
The upper characteristic point coordinate extracted by ASM algorithm is mapped with characteristic point coordinate corresponding on threedimensional model, utilizes OpenGL
The function library of offer realizes the texture mapping of three-dimensional face model.Process is as shown in Figure 9.
The present embodiment is based on the standard colour of skin to facial image and carries out landscaping treatment, generates face texture maps, is applied to
Three-dimensional facial reconstruction obtains personalized three-dimensional faceform.Since there may be hairs for the facial image that shoots in actual life
It blocks, situations such as illumination condition is bad, beautiful effect is often unable to get to the three-dimensional reconstruction of facial image, is mentioned herein thus
An algorithm frame out can realize automatic detection and beautification, to obtain preferable human face rebuilding result to face.
Embodiment 2
During as described in Example 1, beautifies in real time in a kind of above-mentioned facial image and texture synthesis method is in step
(2) method used in the local color conversion process in is as follows:
Firstly, by data from High Dimensional Mapping to low-dimensional while, keep data manifold structure it is constant, use vector xiTo indicate
A pixel i in some feature space；Give a data set x1..., xN, for each pixel xi, choose it
K nearest-neighbor, be expressed asOne group of weights omega will be calculatedijSo that
The k neighborhood can be best reconstruction pixel xi, calculated by minimizing following energy function
Constraint conditionIt acquiresAfterwards, with xiXiNeighborhood linear combination can rebuild
xiXi；
Secondly, the method for using editor to propagate to be realized to the conversion of color；Given original image and target image, by most
Smallization following energy function transmits color:
Wherein, SiIt is edited at pixel i as a result, ti(i ∈ R) is the specified standard colour of skin, and R is to need to convert
Region.
A kind of above-mentioned facial image beautifies in real time and texture synthesis method in formula (3) is further expressed as rectangular
Formula:
E=[(I-W) S]T(I-W)S+(S-T)TΛ(S-T) (4)
S is that i-th of element is siVector, I is unit matrix, and Λ is the diagonal matrix that diagonal entry is λ, and T is i-th
A element is tiVector, WW be element be wijMatrix.
The embodiment chooses 100 width particular photos as test set, and server end image processing algorithm computer CPU is
4 core 4.20GHz of Intel Duo, inside saves as 16GB, in MyEclipse running environment, by java applet to face figure to be measured
As carrying out the sequence of operations such as feature point extraction, color conversion, graph cut and eyebrow reparation.
Have preferable resistance to verify the application algorithm to environment, we to the facial image under different illumination backgrounds into
Row experiment, it can be seen that for the environment that light is excessively dark, algorithm can the preferably whitening colour of skin, the case where for colour of skin deviation,
Also it is able to achieve good flesh correction effect, experimental result is as shown in Figure 10.
Meanwhile the application method and other methods time compare, it may have relative advantage, as shown in the table:
The application is based on the standard colour of skin to facial image and carries out landscaping treatment, generates face texture maps, is applied to three
Human face rebuilding is tieed up, personalized three-dimensional faceform is obtained.Since there may be hair screenings for the facial image that shoots in actual life
Gear, situations such as illumination condition is bad, are often unable to get beautiful effect to the three-dimensional reconstruction of facial image, thus set forth herein
One algorithm frame can realize automatic detection and beautification, to obtain preferable human face rebuilding result to face.Experimental result
Show that this paper algorithm can obtain satisfactorily rebuilding effect, including the brow region being blocked can be rebuild, and compared with it
He has superior operation efficiency at method, can reach real-time.
The application is when face texture maps generate, it is desirable that input picture is front face image, how to pass through video acquisition
The human face image information of more perspective carries out the generation of face texture, and considers that correcting posture etc. is realized in face deflection, reaches essence
Spending higher reconstruction effect will be our work from now on.Meanwhile the ornament (such as glasses) on face will affect texture maps
Effect is generated, the important process that ornament influence is also for we how is removed.In addition, work on hand is to pass through standard texture
Facial image is mapped on three-dimensional face model by mapping algorithm, and it is high-quality how to construct more accurate texture-mapping algorithm generation
Measure three-dimensional reconstruction face, and groundwork from now on.
Bibliography
1、Stylianou G,Lanitis A.Image Based 3D Face Reconstruction:a Survey
[J].International Journal of Image and Graphics,2009,9(2):217-250
2、Newcombe RA,Izadi S,Hilliges O.KinectFusion:Real-time Dense Surface
Mapping and Tracking[J].IEEE International Symposium on Mixed and Augmented
Reality,2012,4(6):127-136
3, Face Image with Pose Variations recognition methods [J] the pattern-recognition and people that Wang Yanan, Su Jianbo are synthesized based on image
Work intelligence, 2015,28 (9): 848-856
4、Richardson E,Sela M,Orel R,Kimmel R.Learning Detailed Face
Reconstruction from a Single Image[C].IEEE Conference on Computer Vision and
Pattern Recognition,2017:5553-5562
5、Ginneken BV,Frangi AF,Staal JJ.Active Shape Model Segmentation with
Optimal Features[J].IEEE Transactions on Medical Imaging,2002,21(8):924-933
6, Wu demonstrate,proves, Zhou Yue colour image human face positioning feature point algorithm research [J] electronic letters, vol, 2008,36 (2):
309-313
7、Tzimiropoulos G,Alabortimedina J,Zafeiriou S.Generic Active
Appearance Models Revisited[J].Springer,2012,7726:650-663
8, Qiu Jialiang, Dai Shengkui skin tone texture retain real-time face beautification algorithm [J] CAD and figure
Learn journal, 2018,30 (2): 336-345
9、Reinhard E,Ashikhmin M,Gooch B,Shirley P.Color Transfer between
Images[J].IEEE Computer Graphics and Applications,2002,21(5):34-41
10、Chen X,Zou D,Zhao Q,Tan P.Manifold Preserving Edit Propagation[J]
.ACM Transactions on Graphics,2012；31(6):1-7
11、Li CM,Huang R,Ding ZH.A Level Set Method for Image Segmentation in
the Presence of Intensity Inhomogeneities with Application to MRI[J].IEEE
Transactions and Image Processing,2011,20(7):2007-2016
12、Rez P,Gangnet M,Blake A.Poisson Image Editing[J].ACM Transactions
on Graphics,2003,22(3):313-318
13、Dominitz A,Tannenbaum A.Texture Mapping via Optimal Mass Transport
[J].IEEE Transactions on Visualization and Computer Graphics,2010,16(3):419-
433
14、Jackson AS,Bulat A.Large Pose 3D Face Reconstruction from a Single
Image via Direct Volumetric CNN Regression[C].IEEE International Conference
on Computer Vision,2017:1031-1039。
Claims (3)
1. a kind of facial image beautifies in real time and texture synthesis method, it is characterised in that include the following steps:
(1) human face characteristic point is extracted using ASM algorithm
Establish active shape model and local gray level model；Original shape matches target image, according to local gray level
Model is adjusted, until convergence；It detects obtained characteristic point and selects the region conduct that eyebrow, eyes, nose, mouth are constituted
Region to be fused；
(2) color is converted
The tone of original image and the tone of target image are consistent by following color conversion formula:
In formula, c ∈ (r, g, b) c ∈ (r, g, b), IrPixel value after indicating conversion, IsIndicate original image pixel value, means,
meantRespectively indicate the pixel mean value of original image and target image, devs, devtRespectively indicate the picture of original image and target image
Plain standard deviation；
(3) the seamless fusion of face and eyebrow repair process
First setting on image S has a closure subregion Ω, and boundary isV is the gradient vector in region to be fused in original image
, f is the scalar function being defined on S, it is known that ff existsOn value be ff, then ff takes the interpolation under V guidance in Ω
Function, i.e. solution extreme-value problem
In formula,For the gradient of f；
(4) three-dimensional facial reconstruction
By on the face texture map to threedimensional model obtained by above-mentioned algorithm, face texture mapping can will lead on two dimensional image
The characteristic point coordinate for crossing the extraction of ASM algorithm is mapped with characteristic point coordinate corresponding on threedimensional model, is provided using OpenGL
Function library realize three-dimensional face model texture mapping.
2. a kind of facial image according to claim 1 beautifies in real time and texture synthesis method, it is characterised in that in step
(2) method used in the local color conversion process in is as follows:
Firstly, by data from High Dimensional Mapping to low-dimensional while, keep data manifold structure it is constant, use vector xiTo indicate at certain
A pixel i in a feature space；Give a data set x1..., xN, for each pixel xi, choose its k
Nearest-neighbor is expressed as xi1..., xix, one group of weights omega will be calculatedijEnable the reconstruction pixel that the k neighborhood is best
Point xi, calculated by minimizing following energy function
Constraint conditionIt acquiresAfterwards, with xixiNeighborhood linear combination can rebuild xixi；
Secondly, the method for using editor to propagate to be realized to the conversion of color；Given original image and target image, pass through minimum
Following energy function transmits color:
Wherein, siIt is edited at pixel i as a result, ti(j ∈ R) is the specified standard colour of skin, and R is the area for needing to convert
Domain.
3. a kind of facial image according to claim 2 beautifies in real time and texture synthesis method, it is characterised in that formula (3)
Further it is expressed as matrix form:
E=[(I-W) S]T(I-W)S+(S-T)TΛ(S-T) (4)
S is that i-th of element is siVector, I is unit matrix, and Λ is the diagonal matrix that diagonal entry is λ, and T is i-th yuan
Element is tiVector, WW be element be ωijMatrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910305804.5A CN110197462A (en) | 2019-04-16 | 2019-04-16 | A kind of facial image beautifies in real time and texture synthesis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910305804.5A CN110197462A (en) | 2019-04-16 | 2019-04-16 | A kind of facial image beautifies in real time and texture synthesis method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110197462A true CN110197462A (en) | 2019-09-03 |
Family
ID=67751974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910305804.5A Pending CN110197462A (en) | 2019-04-16 | 2019-04-16 | A kind of facial image beautifies in real time and texture synthesis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110197462A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929651A (en) * | 2019-11-25 | 2020-03-27 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111080511A (en) * | 2019-11-18 | 2020-04-28 | 杭州时光坐标影视传媒股份有限公司 | End-to-end face exchange method for high-resolution multi-feature extraction |
| CN111104878A (en) * | 2019-12-06 | 2020-05-05 | 江西开心玉米网络科技有限公司 | Image processing method, device, server and storage medium |
| CN111402394A (en) * | 2020-02-13 | 2020-07-10 | 清华大学 | Three-dimensional exaggerated cartoon face generation method and device |
| CN111563855A (en) * | 2020-04-29 | 2020-08-21 | 百度在线网络技术（北京）有限公司 | Image processing method and device |
| CN111738930A (en) * | 2020-05-12 | 2020-10-02 | 北京三快在线科技有限公司 | Face image synthesis method and device, electronic equipment and storage medium |
| WO2021170127A1 (en) * | 2020-02-29 | 2021-09-02 | 华为技术有限公司 | Method and apparatus for three-dimensional reconstruction of half-length portrait |
| CN113628327A (en) * | 2021-08-12 | 2021-11-09 | 聚好看科技股份有限公司 | Head three-dimensional reconstruction method and equipment |
| US11404086B2 (en) | 2020-02-14 | 2022-08-02 | Perfect Mobile Corp. | Systems and methods for segment-based virtual application of makeup effects to facial regions displayed in video frames |
| US11922540B2 (en) | 2020-02-14 | 2024-03-05 | Perfect Mobile Corp. | Systems and methods for segment-based virtual application of facial effects to facial regions displayed in video frames |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103136513A (en) * | 2013-02-05 | 2013-06-05 | 山东神思电子技术股份有限公司 | Improved automatic storage management (ASM) facial feature point locating method |
| CN103839223A (en) * | 2012-11-21 | 2014-06-04 | 华为技术有限公司 | Image processing method and image processing device |
| CN105184273A (en) * | 2015-09-18 | 2015-12-23 | 桂林远望智能通信科技有限公司 | ASM-based dynamic image frontal face reconstruction system and method |
| CN105787974A (en) * | 2014-12-24 | 2016-07-20 | 中国科学院苏州纳米技术与纳米仿生研究所 | Establishment method for establishing bionic human facial aging model |
| CN107316340A (en) * | 2017-06-28 | 2017-11-03 | 河海大学常州校区 | A kind of fast human face model building based on single photo |
-
2019
- 2019-04-16 CN CN201910305804.5A patent/CN110197462A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103839223A (en) * | 2012-11-21 | 2014-06-04 | 华为技术有限公司 | Image processing method and image processing device |
| CN103136513A (en) * | 2013-02-05 | 2013-06-05 | 山东神思电子技术股份有限公司 | Improved automatic storage management (ASM) facial feature point locating method |
| CN105787974A (en) * | 2014-12-24 | 2016-07-20 | 中国科学院苏州纳米技术与纳米仿生研究所 | Establishment method for establishing bionic human facial aging model |
| CN105184273A (en) * | 2015-09-18 | 2015-12-23 | 桂林远望智能通信科技有限公司 | ASM-based dynamic image frontal face reconstruction system and method |
| CN107316340A (en) * | 2017-06-28 | 2017-11-03 | 河海大学常州校区 | A kind of fast human face model building based on single photo |
Non-Patent Citations (1)
| Title |
|---|
| 阳策 等: "基于标准肤色的人脸图像纹理合成与三维重建应用", 《WWW.C-S-A.ORG.CN/HTML/2019/5/6878.HTML》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080511A (en) * | 2019-11-18 | 2020-04-28 | 杭州时光坐标影视传媒股份有限公司 | End-to-end face exchange method for high-resolution multi-feature extraction |
| CN110929651A (en) * | 2019-11-25 | 2020-03-27 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| US11678734B2 (en) | 2019-11-25 | 2023-06-20 | Beijing Dajia Internet Information Technology Co., Ltd. | Method for processing images and electronic device |
| CN110929651B (en) * | 2019-11-25 | 2022-12-06 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111104878A (en) * | 2019-12-06 | 2020-05-05 | 江西开心玉米网络科技有限公司 | Image processing method, device, server and storage medium |
| CN111402394B (en) * | 2020-02-13 | 2022-09-20 | 清华大学 | Three-dimensional exaggerated cartoon face generation method and device |
| CN111402394A (en) * | 2020-02-13 | 2020-07-10 | 清华大学 | Three-dimensional exaggerated cartoon face generation method and device |
| US11404086B2 (en) | 2020-02-14 | 2022-08-02 | Perfect Mobile Corp. | Systems and methods for segment-based virtual application of makeup effects to facial regions displayed in video frames |
| US11922540B2 (en) | 2020-02-14 | 2024-03-05 | Perfect Mobile Corp. | Systems and methods for segment-based virtual application of facial effects to facial regions displayed in video frames |
| WO2021170127A1 (en) * | 2020-02-29 | 2021-09-02 | 华为技术有限公司 | Method and apparatus for three-dimensional reconstruction of half-length portrait |
| CN111563855A (en) * | 2020-04-29 | 2020-08-21 | 百度在线网络技术（北京）有限公司 | Image processing method and device |
| CN111563855B (en) * | 2020-04-29 | 2023-08-01 | 百度在线网络技术（北京）有限公司 | Image processing method and device |
| CN111738930A (en) * | 2020-05-12 | 2020-10-02 | 北京三快在线科技有限公司 | Face image synthesis method and device, electronic equipment and storage medium |
| CN113628327A (en) * | 2021-08-12 | 2021-11-09 | 聚好看科技股份有限公司 | Head three-dimensional reconstruction method and equipment |
| CN113628327B (en) * | 2021-08-12 | 2023-07-25 | 聚好看科技股份有限公司 | Head three-dimensional reconstruction method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110197462A (en) | A kind of facial image beautifies in real time and texture synthesis method | |
| Song et al. | Geometry-aware face completion and editing | |
| Zhao et al. | M3d-vton: A monocular-to-3d virtual try-on network | |
| US9367940B2 (en) | Method for single-view hair modeling and portrait editing | |
| CN105354876B (en) | A kind of real-time volume fitting method based on mobile terminal | |
| Chai et al. | Single-view hair modeling for portrait manipulation | |
| CN105427385B (en) | A kind of high-fidelity face three-dimensional rebuilding method based on multilayer deformation model | |
| Lee et al. | Fast head modeling for animation | |
| CN109427007B (en) | Virtual fitting method based on multiple visual angles | |
| CN109801380A (en) | A kind of method, apparatus of virtual fitting, storage medium and computer equipment | |
| CN107274493A (en) | A kind of three-dimensional examination hair style facial reconstruction method based on mobile platform | |
| Fyffe et al. | Multi‐view stereo on consistent face topology | |
| Zheng et al. | Avatarrex: Real-time expressive full-body avatars | |
| CN105913416A (en) | Method for automatically segmenting three-dimensional human face model area | |
| Wang et al. | Thanka mural inpainting based on multi-scale adaptive partial convolution and stroke-like mask | |
| CN104850847B (en) | Image optimization system and method with automatic thin face function | |
| CN109886144B (en) | Virtual trial sending method and device, computer equipment and storage medium | |
| US11587288B2 (en) | Methods and systems for constructing facial position map | |
| Zheng et al. | Image-based clothes changing system | |
| CN115689869A (en) | Video makeup migration method and system | |
| CN112221145A (en) | Game face model generation method and device, storage medium and electronic equipment | |
| KR20230085931A (en) | Method and system for extracting color from face images | |
| CN111028354A (en) | Image sequence-based model deformation human face three-dimensional reconstruction scheme | |
| CN111127642A (en) | Human face three-dimensional reconstruction method | |
| KR20230110787A (en) | Methods and systems for forming personalized 3D head and face models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190903 |