CN105528763A - Self-adaptive region sensing mask generation method based on multi-grid approximate algorithm - Google Patents

Abstract

The invention provides a self-adaptive region sensing mask generation method based on a multi-grid approximate algorithm, wherein the generation method utilizes the multi-grid approximate algorithm to solve a high-dimension linear system and performs algorithm optimization and improvement for face beautification. An optimization method select switch relative to resolution is set, a conjugate gradient descent method or a sparse matrix are selected for problem solving; On the basis of an actual condition that one image is used as an input image and an orientated feature graph, a down-sampling matrix is directly applied to the input image and the orientated feature graph, then approximate rough matrix parameters of the linear system are obtained, and in this way, a roughing process is remarkably simplified. The processing efficiency of the provided region sensing mask generation technology is increased by four times of that of the prior art; the system real-time request is satisfied in face beautification.

Description

Plate generation method is covered in a kind of adaptive region perception based on multi grid approximate data

Technical field

The invention belongs to pattern-recognition and field of artificial intelligence, plate generation method is covered in the adaptive region perception particularly related to based on multi grid approximate data.

Background technology

The social high speed development of mobile Internet, to the rush of demand of beautifying faces product.Beautifying faces related algorithm has sizable researching value in sphere of learning, also has broad application prospects in actual life meanwhile.Unprofessional user carries out portrait landscaping treatment by beautifying faces system, and planar design personnel carry out virtual image Design assistant etc. by beautifying faces system.

If the portrait photo of shooting comprises the illumination condition of more complicated, background environment and different personage's attitudes, beautifying faces can meet with difficulty., the face on facial image and background need be distinguished for this reason, and according to existing face priori, more meticulously face is divided into zones of different, corresponding localized processing parameter is set, carry out process in various degree.The object of the invention is to design the method automatically can carrying out corresponding localized processing parameter setting according to face characteristic to zones of different.

Prosperous 2012 of Chen Yili and Liao Wen respectively in Master's thesis " beautifying faces based on data-driven " and " the beautifying faces application based on data-driven is developed " first by positioning face key feature points, obtain the unique point that can identify face significant points profile, and distinguished point based divides human face region, finally realize the object that zones of different is processed in various degree.But because landscaping treatment process depends on unique point, when the feature point detection at face edge is not accurate enough, the result of beautifying of generation there will be obvious process vestige, causes beautifying unsuccessfully.For this reason, utilize the people such as P é rez within 2003, to be published in the Poisson image editing algorithms of the middle proposition of paper " Poissonimageediting " on ACMTransactionsonGraphics, post-processed is carried out to beautifying picture, to remove factitious edges of regions.However, this effect need to improve in the sense of reality and natural degree.Liang etc. are published in the paper " Facialskinbeautificationusingadaptiveregion-awaremasks " of IEEEtransactionsoncybernetics the beautification method proposing to cover based on self-adaptation plate for 2014, face key position is not obtained roughly (as eyes by means of only the face prior imformation comprised in unique point, nose, face etc.) distribution, and by edge feature obvious in Canny operator extraction to face.Then by editor's propagation model, it is spread again, obtain the illiteracy plate that can control beautifying faces editing operation.Got by broadcast algorithm owing to covering plate, thus can not produce obvious edge, the image after processing is not had and significantly processes vestige, but therefore bring calculated amount increase, the problem that arithmetic speed is slow.

Summary of the invention

The object of this invention is to provide a kind of adaptive region perception based on multi grid approximate data and cover plate generation method, meet the requirement of beautifying faces technology in real-time and practicality, plate generative process is covered to its core algorithm and carries out adaptive region sensing and optimizing and time complexity optimization, utilize multi grid approximate data to solve and cover plate generation High-dimensional Linear system, while keeping better beautifying faces effect, significantly reduce Algorithms T-cbmplexity.

The technical solution used in the present invention is as follows.

Plate generation method is covered in a kind of adaptive region perception based on multi grid approximate data, multi grid approximate data is utilized to solve the High-dimensional Linear system of covering plate generation, make optimized algorithm improved efficiency 4 times, be applied to beautifying faces process and can meet system real time and practicality requirement.

Described adaptive region perception is covered plate generation method and is comprised:

(1), adaptive edge keeps energy minimization models definition;

(2), multi-grid method model parameter solves;

(3), multi grid approximate data is optimized;

Described step (1) is that edge keeps energy minimization models definition, its objective is and determines that self-adaptation covers the mathematical optimization principle in plate generative process.The present invention selects edge to keep energy minimization models as the mathematical model of region perception self-adaption template, and its expression formula is:

$f=\underset{f}{\mathrm{argmin}}\left\{\underset{x}{\Σ}w\left(x\right){\left(f\left(x\right)-g\left(x\right)\right)}^2+\mathrm{\λ}\underset{x}{\Σ}h\left(\▿f,\▿L\right)\right\}$

In formula:

$h(\▿f,\▿L)=\frac{\left|\right|f_x|{|}_p^2}{\left|\right|L_x|{|}_p^{\mathrm{\α}\left(x\right)}+\mathrm{\ϵ}}+\frac{\left|\right|f_y|{|}_p^2}{\left|\right|L_y|{|}_p^{\mathrm{\α}\left(x\right)}+\mathrm{\ϵ}}$

F is the scalar function of a supposition, how to adjust in order to the pixel value on Description Image, and the pixel value namely after the x place adjustment of f (x) expression point, can be understood as Output rusults. be data item, representative of consumer input information is to the restriction of Output rusults, and wherein data item weight w (x) specifies the affined degree of pixel, its value is [0,1], and g (x) be mode input restriction, be namely the rough range of face. for smooth item, its objective is and ensure that the gradient of f is little as far as possible, wherein L is guidance feature figure, uses the brightness of image passage processed through taking the logarithm as L.Respectively show partial derivative with lower target symbol table, namely parameter ε is a very little but non-vanishing constant (such as, can choose ε=1e-10), is used for preventing when denominator is zero occurring zero situation about removing.α (x) is the parametric function with image space characteristic variations, and this makes it possible to utilize existing face priori to arrange different α (x) values to zones of different, thus controls the smoothness of face zones of different.Parameter lambda controls the proportion of level and smooth item entirety, and when λ is larger, the proportion of smooth item is larger, and now image is smooth-out, otherwise then image is tending towards the original-shape inputting restriction.It is more careful that the selection of λ needs, if λ is too little, diffuse images is not enough, and edge transition is nature, if λ is too large, image is too level and smooth, easily loses important marginal information.Can find out, " edge maintenance model " utilizes guidance feature figure to carry out the diffusion of control inputs information, in particular, is control according to the graded of the brightness layer of image.When the graded of L is less, diffusion is comparatively large, otherwise when the graded of L is larger, diffusion is less, and this diffusion process can be regulated by λ and ε two parameters.|| || _pfor p normal form, p is model parameter.Above-mentioned edge keeps energy minimization models to be equivalent to solving of following linear system: Ax=B, in formula

$A_{ij}=\left\{\begin{array}{c}-\mathrm{\λ}{\left({\left|L_i-L_j\right|}^{\mathrm{\α}}+\mathrm{\ϵ}\right)}^{-1},j\∈N_4\left(i\right)\\ w_i-{\mathrm{\Σ}}_{k\∈N_4\left(i\right)}{A}_{ik},i=j\\ 0,otherwise\end{array}\right.$

B _i＝w _ig _i

Wherein, the pixel in i, j representative image, N ₄i () represents the point in four neighborhoods of i.

Described step (2) is multi-grid method solving model parameter, its objective is that solving edge by multiple grid method keeps energy minimization models parameter, basic operational steps is: first problem roughening, former Grid Projection is calculated to a fairly simple new grid, and Fast Convergent returns original system via interpolation later more by the time.

Self-adaptation perception is covered plate and is equivalent to the linear equation of shape as Ax=B, and suppose that A is n × n matrix, x and B is n × 1 matrix.So, final purpose obtains x=A ^-1b.Definition error function e (t)=x-x (t), when e (t) is less than certain value, thinks that x has converged to suitable value.In order to use alternative manner, that compare in iterative solution procedure is consecutive value e ^*(t)=x (t+1)-x (t).Algorithm steps comprises: 1) A is carried out unusual decomposition A=D-C; 2) solve: x=D ^-1cx+D ^-1b; 3) iterative: x (t+1)=D ^-1cx (t)+D ^-1b.Make M=D ^-1c, N=D ^-1, then x (t+1)=Mx (t)+NB is had.N numerical value is larger, restrains slower.Suppose p (i), (i=1,2 ..., n) be original system finite element basis function, q (i) (i=1,2..., m, m < n) is roughening grid.A kind of roughening method is structural matrix H, and make p=Hq, H is m * n matrix.Make A'=HAH ^t, x'=Hx, B'=HB, then A'x'=B' is the grid system that a m ties up roughization.More specifically, for the problem of a k dimension, if k is less than a dimension of specifying, directly solve with Jacobian technique.Otherwise, turn to roughly lower dimension, such as become original 1/2nd, be finally transformed to the dimension of original system again by interpolation method.

Described step (3) is the optimization of multi grid approximate data, and under its objective is the prerequisite ensureing modelling effect, the treatment effeciency of boosting algorithm, accelerates self-adaptation perception and cover plate generation.

First, the present invention designs a switch, (is more than or equal to 1000x1000 pixel) when photo resolution is enough high, just adopts conjugate gradient decent to solve linear system, and arranges higher solving precision and iterations for it.Internal memory required for the method is 40% of direct solution mode, makes in an ordinary PC, and this algorithm can process the large resolution picture more than 2000x2000.

The initiation parameter of conjugate gradient decent affects algorithm the convergence speed to a great extent.Be 0.0001, when maximum iteration time does not limit in precision, the picture of a process 1675*2200 approximately needs 14 minutes.By reducing the precision (0.01) that solves and maximum iteration time (100), the processing time can drop to the rank of tens seconds.But meanwhile, because solving result inaccurate, for little resolution picture, it is 0 that the illiteracy plate obtained has some regional values, causes final process result occurring obvious blackspot.When iterations is insufficient, similar defect can be more obvious.For this defect that iterations causes not, can be solved by increase iterations and solving precision.

The present invention adopts down-sampled matrix as changing matrix H roughly.Keep the equivalent linearity system definition Ax=B of energy minimization models according to adaptive edge, systematic parameter is:

$A_{ij}=\left\{\begin{array}{c}-\mathrm{\&lambda;}{\left({\left|L_i-L_j\right|}^{\mathrm{\&alpha;}}+\mathrm{\&epsiv;}\right)}^{-1},j\&Element;N_4\left(i\right)\\ w_i-{\mathrm{\&Sigma;}}_{k\&Element;N_4\left(i\right)}{A}_{ik},i=j\\ 0,otherwise\end{array}\right.$

B _i＝w _ig _i

Therefore, it is the matrix that one (m × n) × (m × n) ties up that the present invention covers plate generation model parameter A, and parameter B is the column vector that (m × n) × 1 is tieed up, and wherein m, n are respectively width and the height of original image.Linear system parameter after roughization is:

B'＝H _downSampleB

$A^{\&prime;}=H_{downSample}{\mathrm{AH}}_{downSample}^{-1}$

To the solution that this system is tried to achieve be:

X'＝H _downSampleX

Wherein H _downSamplerepresent down-sampled matrix, represent down-sampled inverse of a matrix matrix, in the present invention, complete this operation by bilinear interpolation.Net result is this formula represents carries out bilinear interpolation to the solving result of roughization system.

In order to simplify the operation further, reduce calculated amount, the present invention simplifies roughization process.Because parameter matrix A and B is got by same input picture and guidance feature figure, in the present invention, input picture and guidance feature figure use same image, therefore down-sampled matrix is directly applied on input picture and guidance feature figure, try to achieve the approximate of linear system again and change matrix parameter roughly, and then use sparse matrix derivation algorithm to obtain approximately to dissolve X' roughly.

Ultimate principle of the present invention is: utilize multi grid approximate data to solve and cover the High-dimensional Linear system of plate generation and be optimized, down-sampled matrix is directly applied on input picture and guidance feature figure, try to achieve the approximate of linear system again and change matrix parameter roughly, and then use sparse matrix derivation algorithm to obtain approximately to dissolve roughly, make optimized algorithm improved efficiency 4 times, be applied to beautifying faces process and can meet system real time and practicality requirement.

The present invention, compared with existing illiteracy plate generation method, has following advantage and beneficial effect:

(1), cover plate generate there is region perception, Automatic adjusument parameter is to adapt to the process in different characteristic region.

(2) treatment effeciency, covering plate generation technique improves 4 times.

(3) landscaping treatment requirement of real time, is made.

Accompanying drawing explanation

Fig. 1 is that the process flow diagram of plate generation method for beautifying faces is covered in the adaptive region perception based on multi grid approximate data of the present invention;

Fig. 2 is conjugate gradient decent design sketch of the present invention;

Fig. 3 be illiteracy plate effect of the present invention and other cover plate generation method comparison diagram.

Embodiment

Below in conjunction with accompanying drawing, the present invention will be further described, and carry out each core algorithm of beautifying faces with computing machine and realize, those skilled in the art can adopt C++ programming language to work out all kinds of handling procedure to implement the present invention.For improving linear system solution performance, this example introduces scripting programming language python expansion, makes this algorithms library can by the high-performance mathematical algorithm storehouse Nympy of python, and the sparse matrix in Scipy solves interface.

Adaptive region perception based on multi grid approximate data of the present invention is covered plate generation method flow and is seen accompanying drawing 1.Block layer decomposition is carried out to the facial image of input, obtains smoothness, brightness and color three layer.To three layer input adaptive region perception template generation algorithm respectively, calculate the template of three layer respectively.Adaptive region perception template generation needs to build adaptive edge and keeps energy minimization models and adopt multi-grid iteration Optimization Method.Actual for beautifying faces, replace changing matrix roughly with down-sampled matrix, and roughization process is simplified, optimize the time complexity of the template generation algorithm of beautifying faces core.

Conjugate gradient decent effect of the present invention as shown in Figure 2.The initiation parameter of conjugate gradient decent affects convergence of algorithm speed to a great extent.By reducing solving precision and maximum iteration time, the processing time can significantly decline, but brings new problem thus.Because solving result inaccurate, for little resolution picture, it is 0 that the illiteracy plate obtained has some regional values, and this causes final process result occurring obvious blackspot.Figure 2 shows this defect, when iterations is insufficient, similar defect can be more obvious.Although this phenomenon effect for large resolution picture is not very obvious, for little resolution picture, then what belong to failure beautifies process.

The method that adaptive region perception based on multi grid optimized algorithm illiteracy plate generation effect of the present invention and Liang Lingyu propose in the PhD dissertation " self-adaptation of facial image is beautified and played up research " of 2014 contrasts sees accompanying drawing 3.The former figure of first behavior, the effect of second behavior this paper algorithm, the third line is the effect of beam Ling Yu PhD dissertation method.Contrast the illiteracy plate effect of illiteracy plate effect of the present invention and beam Ling Yu PhD dissertation method, can see, plate is covered in the brightness of two kinds of algorithms and color illiteracy plate is more similar, and smoothness illiteracy plate differs greatly, this is mainly because two algorithms fine degree when extracting the texture characteristics of face is different.Can see from the 4th row, algorithm of the present invention remains more face texture information, although differ greatly both seeming from the illiteracy plate generated, consider that its physical significance controls the editor's degree to human face region, as long as its transition is level and smooth, so landscaping effect can not produce very big-difference.

Claims (7)

1. plate generation method is covered in the adaptive region perception based on multi grid approximate data, utilizes multi grid approximate data to solve high-dimensional linear system, and carries out algorithm optimization improvement for beautifying faces; Conjugate gradient decent is used to replace sparse matrix solving method, with down-sampled matrix as changing matrix roughly; The treatment effeciency covering plate generation technique is made to promote 4 times; It is characterized in that:

The described adaptive region perception based on multi grid approximate data is covered plate generation method and is specifically comprised:

1) build adaptive edge and keep energy minimization models: adaptive region perception is covered plate generation technique mathematics and turn to adaptive edge maintenance energy minimization models, make to cover plate technique can obtain in image areal distribution according to the similarity of image in pixel scale, by rough face component information and lines information are spread, obtain interregional level and smooth transition effect, concrete model is defined as follows:

$f=\underset{f}{\mathrm{argmin}}\{\underset{x}{\&Sigma;}w\left(x\right){\left(f\right(x)-g(x\left)\right)}^2+\mathrm{\&lambda;}\underset{x}{\&Sigma;}h(\&dtri;f,\&dtri;L)\},$

In formula

$h(\&dtri;f,\&dtri;L)=\frac{\left|\right|f_x|{|}_p^2}{\left|\right|L_x|{|}_p^{\mathrm{\&alpha;}\left(x\right)}+\mathrm{\&epsiv;}}+\frac{\left|\right|f_y|{|}_p^2}{\left|\right|L_y|{|}_p^{\mathrm{\&alpha;}\left(x\right)}+\mathrm{\&epsiv;}},$

F is a scalar function, how to adjust in order to the pixel value on Description Image, and the pixel value namely after the x place adjustment of f (x) expression point, can be regarded as Output rusults; expression makes get the f of minimum value; be data item, representative of consumer input information is to the restriction of Output rusults, and data item weight w (x) indicates the affined degree of pixel, and its value is [0,1], and g (x) is mode input restriction, i.e. the rough range of face; be smooth item, its objective is and ensure that the gradient of f is little as far as possible, wherein L is guidance feature figure; with represent the gradient information of f and L; Parameter lambda controls the proportion of level and smooth item entirety; Respectively show local derviation with lower target symbol table, that is: parameter ε is non-vanishing constant, is used for preventing when denominator is zero occurring zero situation about removing; α (x) is the parametric function with image space characteristic variations, in order to regulate the impact of graded on output image of guidance feature figure L; Be p normal form, p is model parameter;

2) multi-grid method solving model parameter: keep energy minimization models parameter with Multigrid Computations adaptive edge, first ask || || _psolve a coarse result, then interpolation between coarse result and meticulous result; Kept by adaptive edge energy minimization models equivalence transformation to be High-dimensional Linear system Ax=B, matrix A, B is defined as follows:

B _i＝w _ig _i，

Wherein, the pixel in i, j representative image, N ₄i () represents the point in four neighborhoods of i; Utilize the above-mentioned High-dimensional Linear system of the solution by iterative method of multi grid, definition error function e (t)=x-x (t), when e (t) is less than the threshold value of setting, thinks that x has converged to suitable value; In fact, in order to use alternative manner, that compare in the iterative process solved is adjacent value e ^*(t)=x (t+1)-x (t), its algorithm is summed up as 3 steps: 1) A is carried out unusual decomposition A=D-C; 2) solve: x=D ^-1cx+D ^-1b; 3) iterative: x (t+1)=D ^-1cx (t)+D ^-1b; Make M=D ^-1c, N=D ^-1, then x (t+1)=Mx (t)+NB is had; Suppose that p (i) is original system finite element basis function, wherein i=1,2 ..., n; Q (i) is the grid of roughening, wherein i=1,2..., m, m < n; Roughening method is structural matrix H, makes p=Hq, and H is m * n matrix; Make A'=HAH ^t, x'=Hx, B'=HB, then A'x'=B' is the grid system that a m ties up roughization; For the problem of a k dimension, if k is less than a dimension of specifying, Jacobi (Jacobi) process of iteration method is so directly used to solve, otherwise, turn to roughly lower dimension, be finally transformed to the dimension of original system again by methods such as interpolation;

3) multi grid approximate data is optimized: apply for beautifying faces, under the prerequisite keeping modelling effect, reduce step 2) in multi grid optimized algorithm consuming time, remove practical time performance bottleneck, specifically comprise two aspect optimisation strategy: use conjugate gradient decent to replace sparse matrix solving method, with down-sampled matrix as changing matrix roughly.

2. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that adaptive edge keeps the object of energy minimization models definition to be determine that self-adaptation covers mathematical optimization principle in plate generative process, in described step (1), α (x) is the parametric function with image space characteristic variations, make it possible to utilize existing face priori to arrange different α (x) values to zones of different, thus control the smoothness of face zones of different.

3. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that, in described step (1) selection of λ need more careful, to ensure that diffuse images is abundant, edge transition nature, utilizing guidance feature figure to carry out the diffusion of control inputs information, in particular, is control according to the graded of the brightness layer L of image; The graded of L and diffusion are inverse relation, and namely the graded of L is less, and diffusion is larger; Otherwise when the graded of L is larger, diffusion is less; This diffusion process is regulated by λ and ε two parameters.

4. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that, the basic operational steps of described step (2) is: first problem roughening, former Grid Projection is calculated to a fairly simple new grid, by the time Fast Convergent returns original computation process via interpolation (Interpolation) later again, its objective is equivalent linearity system solution self-adaptation perception being covered to plate generation.

5. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that, described step (3) conjugate gradient decent replaces sparse matrix solving method to solve linear system, design a switch for this reason, only have when photo resolution is enough high, namely be greater than setting value, adopt conjugate gradient decent to solve linear system, and higher solving precision and iterations are set for this reason; Internal memory required for conjugate gradient decent is 40% of direct solution mode.

6. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that, described step changes matrix roughly mentioned by (3), adopting down-sampled matrix as changing matrix roughly, being defined by linear system parameter:

B _i＝w _ig _i，

Known, parameter A is that one (m × n) × (m × n) ties up matrix, and parameter B is (m × n) × 1 dimensional vector, and wherein m, n are respectively width and the height of original image; Linear system parameter after roughization is: B'=H _downSampleb and the solution of trying to achieve is: X'=H _downSamplex;

Wherein H _downSamplerepresent down-sampled matrix, represent down-sampled inverse of a matrix matrix, operate by bilinear interpolation, net result is: this formula represents carries out bilinear interpolation to the solving result of roughization system.

7. plate generation method is covered in the adaptive region perception based on multi grid approximate data according to claim 1, it is characterized in that, the described roughization process of step (3) is simplified the operation, input picture and guidance feature figure use same image, down-sampled matrix is directly applied on input picture and guidance feature figure, try to achieve the approximate roughization matrix parameter A and B of linear system again, and then use sparse matrix derivation algorithm is obtained to be similar to and is dissolved X' roughly.