CN102044085B - Full-automatic three-dimensional human face detection and posture correction method - Google Patents

Abstract

The invention belongs to the fields of computer vision and computer graphics, relating to a full-automatic robust three-dimensional human face detection and posture correction method. The method comprises the following steps of: by using three-dimensional curved surfaces of human faces with complex interference, various expressions and different postures as input and carrying out multi-dimensional moment analysis on three-dimensional curved surfaces of human faces, roughly detecting the curved surfaces of the human faces by using face regional characteristics and accurately positioning the positions of the nose tips by using nose tip regional characteristics; further accurately segmenting to form completed curved surfaces of the human faces; detecting the positions of the nose roots by using nose root regional characteristics according to distance information of the curved surfaces of the human faces; establishing a human face coordinate system; automatically correcting the postures of the human faces according to the human face coordinate system; and outputting the trimmed, complete and posture-corrected three-dimensional human faces. The method can be used for a large-scale three-dimensional human face base. The result shows that the method has the advantages of high speed, high accuracy and high reliability.

Description

The method that a kind of full-automatic three-dimensional people face detects and posture is corrected

Technical field

The invention belongs to computer vision and field of Computer Graphics, be specifically related to a kind of three-dimensional face full-automatic, robust and detect and the attitude correcting method.

Background technology

In recent years, along with the progress that 3 D stereo is rebuild software and hardware technology, three-dimensional face is widely used in a plurality of fields, for example authentication, the behavioural analysis of people's face and computer animation design etc.The correction that three-dimensional face is detected with attitude is that first also is a very crucial step in the above-mentioned application.And these operations of manual carrying out will very be wasted time and energy, and in application such as similar authentication, the correction of detection and posture also is necessary automatically.But similar complex background (like clothes, hair etc.), expression etc. all make this work very challenging.

The background technology that document [1], [2] are described all is to utilize the curvature information of curved surface to carry out three-dimensional face detection or feature location, and characteristics such as curvature can show very unstablely under interference of noise.The background technology that document [3] is described then utilizes the method for seeking people's face plane of symmetry to solve the problem that three-dimensional face detects and attitude is corrected, but its people's face plane of symmetry is easy to make mistakes under the situation that expression exists.The background technology that document [4], [5] are described has been utilized the algorithm of multiple filter combining classification device, and the shortcoming of these algorithms is: its result depends on the performance of sorter to a great extent.And the point of proximity iteration ICP method for registering such as (iterativecloset point) of similar utilization carries out the background technology that attitude is corrected, and then needs good starting condition.The above-mentioned background technology all exists inevitable problem, and it is restricting inevitably, and three-dimensional face detects and the development of attitude correction technology.

List of references:

[1]A.Colombo，C.Cusano，and?R.Schettini，“3D?face?detection?using?curvature?analysis，”Pattern?Recognition，39(3)，2006，pp.444-455；

[2]F.B.ter?Haar?and?R.C.Veltkamp，“A?3d?face?matching?framework?for?facial?curves，”Graph.Models，71(2)，2009，pp.77-91；

[3]G.Pan?et?al.“Finding?Symmetry?Plane?of?3D?Face?Shape”Proc.Int.Conf.on?PatternRecogtion，Conference?on，2006，pp.1143-1146；

[4]C.Xu?et.Al.“Combining?local?features?for?robust?nose?location?in?3D?facial?data，”PatternRecognition?Letters，27(13)，2006，pp.1487-1494；

[5]A.Rajwade?and?M.Levine，“Facial?pose?from?3D?data，”Image?and?Vision?Computing，24(8)，2006，pp.849-856

Summary of the invention

The objective of the invention is to propose a kind of method that three-dimensional face detects and posture is corrected of full automatic, robust, and do not receive complex background, expression and interference of noise.

The present invention is achieved in that a kind of full-automatic three-dimensional people face detects and the posture correcting method, and its step is following:

Step 1: the pending three-dimensional face curved surface input detection systems that will contain complicated disturbance, various expression and different gestures;

Step 2: the three-dimensional face curved surface is carried out multiple dimensioned square analysis, suppose that S represents the three-dimensional face curved surface, for any 1 p on the curved surface, its neighborhood can use a radius to be positioned at the p spheroid as r, the centre of sphere and define, and its neighborhood is designated as B _r(p) IS, then the square of this neighborhood is orientated as:

$M_r^0\left(p\right)={\∫}_{B_r\left(p\right)\mathrm{IS}}\mathrm{xdx}$

$M_r^1\left(p\right)={\∫}_{B_r\left(p\right)\mathrm{IS}}[x-M_r^0\left(p\right)]\⊗[x-M_r^0\left(p\right)]\mathrm{dx}$

Wherein $y\⊗z:={\left(y_i{z}_j\right)}_{i,j=\mathrm{1,2,3}}.$ M _r ¹(p) three eigenwerts have been described the statistics geological information of its neighborhood; The metric space of three-dimensional face curved surface S can be realized through following smoothing process:

$\frac{\∂S}{\∂t}=\mathrm{\δ}\mathrm{\ΔS}$

Wherein δ is the parameter of the level and smooth speed of control; Three-dimensional face curved surface S can approach with discrete tri patch, and said smoothing process can realize through discrete Laplace operator;

Step 3: propose the face area characteristic $F_i=1-\frac{{\mathrm{\λ}}_2^i}{{\mathrm{\λ}}_3^i}(1-\frac{{\mathrm{\λ}}_1^i}{{\mathrm{\λ}}_3^i})$ Come to detect cursorily three-dimensional face curved surface S, wherein λ ' ₃>=λ ' ₂>=λ ' ₁>=0 is said M _r ¹Three eigenwerts;

Step 4: on the three-dimensional face curved surface S of asperity detection, propose the nose provincial characteristics $T_i=\frac{{\mathrm{\λ}}_2^i}{{\mathrm{\λ}}_3^i+{\mathrm{\λ}}_2^i+{\mathrm{\λ}}_1^i}$ Come to navigate to exactly the position of nose, come further accurately to be partitioned into complete three-dimensional face curved surface according to this position then;

Step 5: the range information according to the three-dimensional face curved surface S that is partitioned into proposes nasion provincial characteristics $\frac{d_G(\mathrm{Ref},i)}{d_E(\mathrm{Ref},i)}\≤R_{\mathrm{Thr}}$ Detect the position of the nasion, wherein d _G(ref, i) and d _G(on behalf of nose, ref i) arrive geodesic line distance and the Euclidean distance of an i respectively; Compare with other zones, very near straight line, so its geodesic line distance and Euclidean distance are more approaching to line between the nasion for nose, can selected threshold R _Thr≈ 1 confirms the position of the nasion;

Step 6:, set up three-dimensional face surface coordinates system according to the position that step 3,4 and 5 is confirmed: at first with the position of nose as initial point; In people's face curved surface space of high yardstick, carry out principal component analysis (PCA) then, minimal eigenvalue characteristic of correspondence vector is the z axle; At last detected nasal area is carried out principal component analysis (PCA), can confirm the direction of x axle and y axle in conjunction with the position of the nasion

Step 7: according to the three-dimensional face surface coordinates is to come automatically to carry out pruning of people's face curved surface and posture correction;

Step 8: output is through the three-dimensional face curved surface that pruned, complete, that posture was corrected.

Three-dimensional face of the present invention detects with the posture correcting method and is used for large-scale three-dimensional face storehouse (comprising from the people's face under the different expressions of 100 experimental subjectss); Experimental result shows that this method has that speed is fast, accuracy is high, the fiduciary level advantages of higher; Can be used for handling the human face expression that following occasion has complicated disturbance: (1) is in the medical application relevant with face; Like shape of face recovery after face's cosmetic surgery, the unexpected injury etc., the present invention can assist the doctor automatically to three-dimensional face detect, pruning and posture correct; (2) utilizing three-dimensional face to carry out under the occasion of identification or authentication, the present invention can be automatically, promptly with people's face location and utilize the people's face coordinate system FCS that is built that it is mated; (3) analyze in the application with facial expression animation in three-dimensional face expression, the present invention can with people's face that complex expression is arranged and amimia under people's face compare fast, thereby reliable data are provided for the face's curved surface variation under the espressiove.

Description of drawings

Fig. 1 is the process flow diagram of three-dimensional face detection of the present invention and posture correcting method;

Fig. 2 is the inventive method under the situation that level and smooth yardstick increases, the variation comparison diagram of people's face curved surface; Light color represents face's character numerical value bigger, and dark color represents face's character numerical value smaller;

Fig. 3 is the inventive method under the situation that level and smooth yardstick increases, the variation comparison diagram of people's face curved surface and nose character numerical value; Light color represents the nose character numerical value bigger, and dark color represents the nose character numerical value smaller;

Fig. 4 a is that the inventive method nose is to face other regional geodesic line distance (light color) and Euclidean distance (dark color) synoptic diagram;

Fig. 4 b is that the inventive method detects nose position (dark color, its candidate point are light color) and nasion position (dark color, its candidate point are light color) synoptic diagram at last;

Fig. 5 is in the BU-3DFE three-dimensional face database, utilizes the inventive method to carry out people's face and the error map behind amimia people's face registration under 24 kinds of complex expressions after attitude is corrected;

Fig. 6 is to use the inventive method partly to prune and the posture example of correcting, above the original surface sweeping curved surface data of a behavior, below pruning and people's face that posture was corrected of behavior correspondence.

Embodiment

Three-dimensional face of the present invention detect and posture correcting method (as shown in Figure 1) with the three-dimensional face curved surface that contains complicated disturbance, various expression and different gestures as input; Through people's face three-dimension curved surface is carried out multiple dimensioned square analysis; At first propose the face area characteristic and come to detect cursorily people's face curved surface; Secondly propose the position that the nose provincial characteristics locatees exactly nose again, come further accurately to be partitioned into complete people's face curved surface according to this position then, then proposed the position that nasion provincial characteristics detects the nasion according to the range information of people's face curved surface; Last position according to these unique points; Set up people's face coordinate system, and automatically carried out the correction of face posture in view of the above, be output as three-dimensional face that pruned, complete, that posture was corrected.

Do detailed explanation in the face of the detailed process of the inventive method down.

Suppose the three-dimensional face curved surface to be detected of S representative input, for any 1 p on the curved surface, its neighborhood can use a radius to be positioned at the p spheroid as r, the centre of sphere and define, and its neighborhood is designated as B _r(p) IS, then the square of this neighborhood is orientated as:

$M_r^0\left(p\right)={\∫}_{B_r\left(p\right)\mathrm{IS}}\mathrm{xdx}$

$M_r^1\left(p\right)={\∫}_{B_r\left(p\right)\mathrm{IS}}[x-M_r^0\left(p\right)]\⊗[x-M_r^0\left(p\right)]\mathrm{dx}$

Wherein $y\⊗z:={\left(y_i{z}_j\right)}_{i,j=\mathrm{1,2,3}}.$ M _r ¹(p) three eigenwerts have been described the statistics geological information of its neighborhood.The metric space of a three-dimension curved surface can be realized through following smoothing process:

$\frac{\∂S}{\∂t}=\mathrm{\δ}\mathrm{\ΔS}$

Wherein δ is the parameter of the level and smooth speed of control.In practical operation, S can approach with discrete tri patch, and therefore above-mentioned smoothing process can realize through discrete Laplace operator.We use N ₁(i) the ring next-door neighbour of expression summit i, X representes the set of all apex coordinates, then the Laplace operator of summit i is defined as:

$\mathrm{\Δ}{x}_i={\mathrm{\Σ}}_{j\∈N_1\left(i\right)}{w}_{i,j}(x_i-x_j)$

X wherein _iThe coordinate of expression summit i, w _{I, j}Be that summation is 1 weights.According to following formula, the Laplace operator of S is defined as:

L＝I-W

W wherein _{I, j}=W _{I, j}, I is a unit matrix.Therefore, the metric space of S can be realized by following formula:

X _t+1＝X _t-δLX _t

The first step of algorithm promptly detects and is partitioned into people's face from the curved surface of original scanning.Can find out that by shown in Figure 2 along with the increase of level and smooth yardstick, compare with other zones, it is more level and smooth that face area becomes, and therefore, is characterized as face feature below we define:

$F_i=1-\frac{{\mathrm{\λ}}_2^i}{{\mathrm{\λ}}_3^i}(1-\frac{{\mathrm{\λ}}_1^i}{{\mathrm{\λ}}_3^i})$

λ ' wherein ₃>=λ ' ₂>=λ ' ₁>=0 is M _r ¹Three eigenwerts.When summit i and its neighbour are similar in one plane gradually, λ ' ₃≈ λ ' ₂＞0, λ ' ₁≈ 0, then F _i≈ 0, so its value can be used for weighing the level and smooth degree of regional area.

After people's face, second of algorithm goes on foot and is used for accurately navigating to nose in asperity detection, because it is the most also to be the position that receives expression influence least easily on the human face region.The nose characterizing definition is:

$T_i=\frac{{\mathrm{\λ}}_2^i}{{\mathrm{\λ}}_3^i+{\mathrm{\λ}}_2^i+{\mathrm{\λ}}_1^i}$

When summit i constitutes a spheroid with its neighborhood, λ ' is arranged ₃≈ λ ' ₂≈ λ ' ₁, T _iReach maximal value 1/3.As illustrate shown in 3, along with the growth of space scale, the T in nose zone _iThan the similar partial sphere in other zones, T _iAlso bigger, so it can be used as the characteristic of choosing the nose zone.

After the position that accurately navigates to nose, can with the nose centre of sphere, the people who crosses in asperity detection is further refinement human face region on the face.All nose eigenwerts are greater than certain threshold value T _ThrThe summit as whole nasal area.

The 3rd step of algorithm is the position that navigates to the nasion, and its characterizing definition is following:

$\frac{d_G(\mathrm{ref},i)}{d_E(\mathrm{ref},i)}\≤R_{\mathrm{thr}}$

D wherein _G(ref, i) and d _G(ref i) represents geodesic line distance and the Euclidean distance of RP (being nose) to an i respectively.As illustrate shown in 4, to compare with other zones, very near straight line, so its geodesic line distance is more approaching with Euclidean distance to line between the nasion for nose, so can selected threshold R in can testing _Thr≈ 1 confirms the position of the nasion.

The 4th step of algorithm is according to detected people's face and the characteristic point position (nose and the nasion) that navigates to, and sets up stable people's face coordinate system.Detailed process is following: at first with the position of nose as initial point, in people's face curved surface space of high yardstick, carry out principal component analysis (PCA) then, minimal eigenvalue characteristic of correspondence vector is the z axle; Then detected nasal area is carried out principal component analysis (PCA), can confirm the direction of x axle and y axle in conjunction with the position of the nasion.

Further describe the present invention through Embodiment B U-3DFE three-dimensional face database below.

BU-3DFE three-dimensional face database (L.Yin et al; " A 3D facial expression database forfacial behavior research; " Proc.Int.Conf.on Automatic Face and GestureRecognition; 2006, pp.211-216.) contain 2500 original scanning people faces of 100 people.Each original scanning people face also comprises 7 kinds of expressions in various degree except comprising complicated disturbance such as garment-like, hair.Main variable element is:

1. level and smooth yardstick elects 40 as, because the various characteristics that propose keep stable gradually when yardstick continues to increase.

2. face area characteristic F _iNormalizing between 0 to 1, F _ThrElect 0.8 as, think too little F _ThrWill make that detecting people's face comprises a lot of wrong interference.

3. face area characteristic T _iNormalizing between 0 to 1, T _ThrElect 0.1 as, think too little T _ThrWill make detected nasal area comprise the expression interference sections.

4. when detecting nasion zone, R _ThrBe defined as 1.07

The present invention successfully detects 2435 people's faces in 2500 three-dimensional faces; Be that success ratio is 97.4%; Wherein have 2419 people's faces successfully to navigate to face feature (nose and the nasion), promptly the face feature position success rate is 99.2% in the detected people's face of success.In order to quantize to weigh the result that human face posture of the present invention is corrected, we utilize parameter to weigh: suppose S _RefBe people's face curved surface that posture was corrected, we are with its curved surface as a reference.Carry out people's face that attitude was corrected with the present invention for any one, we quantize transformation matrix R between the two with ICP.R can resolve into two parts, i.e. translating sections R _t=(x _t, y _t, z _t) and rotating part R _r=R _x(γ) R _y(β) R _z(α).

In order to check the robustness of the present invention under complex expression; For each experimental subjects; We with people's face of neutrality expression as a reference; The ICP that carries out with it that the attitude of other 24 complex expressions was corrected matees, and the people's face and the error profile result behind amimia people's face registration that utilize the inventive method to carry out under 24 kinds of complex expressions after attitude is corrected are as shown in Figure 5.Equally in order to check the stability of invention in all subjects, we at first choose one and carry out people's face that attitude corrects as a reference with manual work, and 2419 are detected and people's face and its coupling that attitude was corrected, and error is as shown in table 1.Partial results is as shown in Figure 6.

Table 1: rotation error and translation error

α	β	γ	x t	y t	z t
						2.08°	1.50°	1.78°	1.52mm	2.47mm	2.29mm

Claims (1)

1. a full-automatic three-dimensional people face detects and the posture correcting method, it is characterized in that it comprises the steps:

Step 1: the pending three-dimensional face curved surface input detection systems that will contain complicated disturbance, various expression and different gestures;

Step 2: the three-dimensional face curved surface is carried out multiple dimensioned square analysis, suppose that S represents the three-dimensional face curved surface, for any 1 p on the curved surface, its neighborhood can use a radius to be positioned at the p spheroid as r, the centre of sphere and define, and its neighborhood is designated as B _r(p) I S, then the square of this neighborhood is orientated as:

Wherein

, three eigenwerts of

have been described the statistics geological information of its neighborhood; The metric space of three-dimensional face curved surface S can be realized through following smoothing process:

Wherein δ is the parameter of the level and smooth speed of control; Three-dimensional face curved surface S can approach with discrete tri patch, and said smoothing process can realize through discrete Laplace operator;

Step 3: propose face area characteristic

and come to detect cursorily three-dimensional face curved surface S, wherein

is three eigenwerts of said

;

Step 4: on the three-dimensional face curved surface S of asperity detection; Propose nose provincial characteristics

and come to locate exactly the position of nose; Come further accurately to be partitioned into complete three-dimensional face curved surface according to this position then

Wherein, the position of said nose is T _iThe position of corresponding nose when reaching maximal value;

Step 5: the range information according to the three-dimensional face curved surface S that is partitioned into proposes nasion provincial characteristics

Detect the position of the nasion, wherein d _G(ref, i) and d _E(on behalf of nose, ref i) arrive geodesic line distance and the Euclidean distance of an i respectively; Compare with other zones, very near straight line, so its geodesic line distance and Euclidean distance are more approaching, selected threshold R to line between the nasion for nose _Thr≈ 1 confirms the position of the nasion;

Step 6:, set up three-dimensional face surface coordinates system according to the position that step 3,4 and 5 is confirmed: at first with the position of nose as initial point; In people's face curved surface space of high yardstick, carry out principal component analysis (PCA) then, minimal eigenvalue characteristic of correspondence vector is the z axle; At last detected nasal area is carried out principal component analysis (PCA), can confirm the direction of x axle and y axle in conjunction with the position of the nasion

Step 7: according to the three-dimensional face surface coordinates is to come automatically to carry out pruning of people's face curved surface and posture correction;

Step 8: output is through the three-dimensional face curved surface that pruned, complete, that posture was corrected.

Images