CN100504911C - Method for automatically extracting face area from three-dimensional scanning original data of head and shoulder - Google Patents

Abstract

The invention discloses a kind of method to abstract automatically the area of the face from the initial 3D-scan data of head and shoulder, and its steps is: (1) curved face reconfiguration of 3D face: with the method of sub mesh creating piecewise smooth triangular mesh curved face with the initial 3D data of head and shoulder; (2) calculating the Gaussian curvature: calculating the discrete Gaussian curvature of every apex basing on the triangular mesh data which is gotten during curved face reconfiguration; (3) abstracting the area of the face. The beneficial effect of the invention is abstracting the core of the face from the 3D data of the head and shoulder automatically and accurately.

Description

Automatically extract the method for face area in a kind of three-dimensional scanning original data of from the beginning shoulder

Technical field

The present invention relates to the extraction method of face area in a kind of the shoulder three-dimensional scanning original data, relate in particular to a kind of with Gaussian curvature be communicated with the method that the sheet search groups is combined into core technology, automatically extracts face's nucleus from three-dimensional scanning original data.

Background technology

Development along with the three-dimensional acquisition technology, three-dimensional data becomes the 4th kind of multimedia data type after sound, image and video just gradually, three-dimensional face model particularly is in the internet, aspect such as home entertaining, computer animation, video display amusement and security monitoring is with a wide range of applications.The three-dimensional face data that major applications needs are nucleuses of front face.

But people's face data that existing three-dimensional face data acquisition equipment obtains not only comprise the positive zone of people's face, also comprise the neighboring area except that people's face, as zones such as neck, shoulder, hairs.These region shape instabilities belong to interfere information not in the know, effect and accuracy that the influence great majority are used.Adopting manual method to remove that information not in the know can cause using can't robotization, lacks practicality (as the recognition of face in the security monitoring) in some cases.Existing automatic cutting technique requires the assistance of two dimensional image, is difficult to be directly used in three-dimensional face model.

Summary of the invention

The invention provides a kind ofly based on curvature be communicated with that sheet is search, full automatic face area extracting method, what mainly solve is the automatic problem of pretreatment of head shoulder three-dimensional scanning original data during large-scale three dimensional people face is used.

Automatically extract the method for face area in a kind of three-dimensional scanning original data of from the beginning shoulder, its step is as follows:

(1), the surface reconstruction of three-dimensional face: adopt the thought of subdivision mesh, original three-dimensional head shoulder data (coordinate point set) are made up piecewise smooth triangular mesh curved surface;

(2), the calculating of Gaussian curvature: the discrete Gaussian curvature of on the triangular mesh data that surface reconstruction obtains, calculating each summit;

(3), human face region extracts: (this step divides three little steps to finish)

At first be the calculating that is communicated with sheet: the Gaussian curvature based on the 2nd step calculated is divided into a plurality of connection sheets by certain threshold value with original three-dimensional face data;

Next is that rough human face region is selected: in the connection sheet of cutting apart, select to comprise the rough calculation result of the connection sheet of people's face nucleus maximum as human face region by certain rule;

Be the accurate extraction of human face region at last:, cut out accurate face nucleus by the connection sheet of rough selection and original three-dimensional face data.

The method of the subdivision mesh described in the step (1) comprises that network topology estimation, grid optimization and piecewise smooth surface optimize three steps.

The computing method of the Gaussian curvature that adopts in the step (2) are to one of the calculating of Gaussian curvature on the continuous curve surface formula approximate, that employing is following:

$\underset{A}{\∫\∫}\mathrm{KdA}=2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i---\left(1\right)$

$K=\frac{2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i}{\frac{1}{3}A}---\left(2\right)$

When in the step (3) original three-dimensional face data being divided into a plurality of connection sheet, carry out the expansion of connected relation according to 1 ring neighborhood of grid.

In the step (3), what described threshold values was got is the mean-Gaussian curvature of all grid vertexes.

The rough selective rule of face area need calculate three amount: TR in the step (3) _m(upper right point), BL _m(lower-left point) and Span _m, it is as follows that it calculates used formula:

${\mathrm{TR}}_m(x,y,z)={(\max \left(p_i\left(x\right)\right),\max \left(p_i\left(y\right)\right),\max p_i\left(z\right)))}_{p_i\∈P_m}---\left(3\right)$

${\mathrm{BL}}_m(x,y,z)={(\min \left(p_i\left(x\right)\right),\min \left(p_i\left(y\right)\right),\min \left(p_i\left(z\right)\right))}_{p_i\∈P_m}---\left(4\right)$

Span _m＝‖TR _m-BL _m‖ (5)

If rough selective rule is Span _M1〉=th _b, Span _M2〉=th _b, then select Span less as face area, otherwise select bigger as face area.

Th in the rough selection of face area in the step (3) _bValue be taken as 180.

Used cutting radius was the center of gravity of the rough connection sheet the selected distance apart from the center of gravity solstics to the set when face area accurately extracted in the step (3).

The problem of face area cutting can be defined as follows in the three-dimensional data of head shoulder: the data P={p of a given initial three-dimensional head shoulder _i, p wherein _i={ x _i, y _i, z _iBe the three-dimensional coordinate that i is ordered, suppose how automatically to extract the subclass P ' { p of P without any about the attitude of raw data P and the information of direction _i', this subclass only comprises the zone of front face.

To correctly distinguish human face region, need analyze and add up the character of people's face curved surface.Curvature has been expressed the intrinsic characteristic of curved surface, and its tolerance and the orientation independent of curved surface, is fit to very much to be used for the feature of analytic surface.Because curvature is only just meaningful for curved surface, meaningless to the coordinate points set, we intend reconstruct three-dimension curved surface from P, calculate on the curved surface every Gaussian curvature then, seek to extract the method for human face region by the distribution situation of analyzing this curved surface Gaussian curvature.

Say on directly perceived, although human face region causes the curve form more complicated because of eye, nose, mouth and eye socket, move towards, move towards to neck to the both sides trend with from chin from cheek to hair but examine from forehead, the common feature of these three directions is that curve form has a change procedure from smooth to bending.Gaussian curvature is to weigh this variation tendency well to measure.Therefore, at first reconstruct curved surface information from P is calculated the curvature on each summit on the surface of reconstruct; Big young pathbreaker's data according to curvature are divided into different connection sheets then, search for the zone that comprises maximum front face again from be communicated with sheet, and are last

The front face of maximum is communicated with sheet to be adjusted and obtains face area result accurately.

The effect that the present invention is useful is to cut out face's nucleus in the three-dimensional data that can from the beginning take on automatically, exactly.

Description of drawings

Fig. 1 is the process flow diagram that face area of the present invention extracts;

Fig. 2 is the data and the corresponding surface reconstruction result of three-dimensional head shoulder of the present invention;

Fig. 3 is a Gaussian curvature result of calculation of the present invention;

Fig. 4 is the result of calculation of connection sheet of the present invention;

Fig. 5 is that face area of the present invention extracts the result;

Embodiment:

Face area extracts automatically in the data of head shoulder

As shown in Figure 1, concrete steps are as follows:

1, surface reconstruction

The purpose of surface reconstruction is the information of reconstructed surface from the information that the three-dimensional coordinate point set comprises, and our method adopts the thought of subdivision mesh, and the reconstruct triangular mesh mainly is divided into three steps:

(1) topological structure of the former curved surface of estimation.This step is at first determined an initial estimation of model geometric, makes up the grid larger, that number of vertices is more, with the preliminary topology of determining model.

(2) grid optimization.Adopt the grid optimization technology to reduce summit and triangle number that previous step makes up grid, whole process by optimize energy function with balance summit, triangle number and and the degree of former surface fitting between relation.The free variable of energy function comprises the position on number of vertices, annexation and summit in the optimizing process.

(3) piecewise smooth surface optimization.The work in this step itself also is an optimizing process based on the optimization result in second step.It changes the quantity and the position of summit quantity, annexation, position and sharp features.The optimization in this step also realizes automatic detection and the recovery to sharp features.As Fig. 2, the curved surface after the reconstruct.

2, the calculating of Gaussian curvature

On continuous curve surface, establish the border of simply connected region D on the curved surface

Be piecewise smooth closed curve, promptly $\∂D=C_1\∪C_2\·\·\·\∪C_n$ , each C _iBe smooth, establish α _iBe

The exterior angle, summit, Gaussian curvature adopts formula (1) to calculate.

$\∫{\∫}_D\mathrm{KdA}+{\∫}_{\∂D}{k}_g\mathrm{ds}+\mathrm{\Σ}{\mathrm{\α}}_i=2\mathrm{\π}---\left(1\right)$

Wherein K is the Gaussian curvature of arbitrfary point, k _gIt is the geodesic curvature on border.

The data of the three-dimensional head shoulder after curved surface reconstruct, its data manifestation mode is a 3D grid, its main element comprises the set of three-dimensional coordinate point and the annexation of the set mid point represented with triangle, comprises limit and triangle surface.A vertex v and its adjacent vertex collection in the given grid

,, make α to i=0...n-1 _i=∠ v _iVv _{(i+1) %n}Be two continuous limit e of vertex v _i=vv _i, e _I+1=vv _{(i+1) %n}The angle of opening, % is a modulo operator, (1) formula can be converted into (2) formula under the discrete case.

$\underset{A}{\∫\∫}\mathrm{KdA}=2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i---\left(2\right)$

Wherein A is the leg-of-mutton total area of adjacent vertex v, and in the small neighbourhood of v, K keeps approximate constant, can be used as the estimation of the Gaussian curvature of vertex v:

$K=\frac{2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i}{\frac{1}{3}A}---\left(3\right)$

Press the characteristic information that the Gaussian curvature of (3) calculating has been expressed grid surface preferably, partial results is shown in figure (3), and red area represents that curvature is less, and blue region curvature is bigger.

3, face area extracts automatically

The result of calculation that Gaussian curvature has been arranged, we finish the automatic extraction of human face region by following three steps.

(1) calculating of connection sheet

From figure (3) as seen, the neighboring area of human face region has higher curvature value, and zone line has lower curvature value, and we are by setting a threshold value, and the zones of different of raw data is separated.Have benefited from the topology information that the data of the three-dimensional head shoulder behind the curve reestablishing comprise, our connection sheet growing method utilizes the syntople in the topology information to carry out regional continuation.Make M=(V _M, K _M, C _M) people's face grid of expression trigonometric ratio, V wherein _M={ v _iBe the vertex set of grid, K _MThe topology information of expression grid, C _M={ c _iRepresent the set of the Gaussian curvature value of each summit i correspondence, it is as follows to be communicated with the sheet computing method:

I is provided with a label t for each summit _i, be made as 0, initial sheet number l=0, the initialization curvature threshold th of being communicated with of order _c

II is from V _MT of middle selection _iThe vertex v of=0 correspondence _i, v=v is set _i, t _i=1, and with v _iJoin and be communicated with sheet set P ₁In;

III is to adjacent vertex collection N (each adjacent vertex v v) of vertex v _kIf, c _k＜th _cAnd t _k=0, with v _kAdd set P ₁

IV makes that vertex v is P ₁In each untreated summit, the label t=1 on this summit is set, get back to the III step;

L=l+1 gets back to the II step, up to the summit that can not find a t=0.

We do not safeguard topological relation the computation process of connection sheet, promptly do not keep the triangle of grid, only preserve the point set that respectively is communicated with sheet, and the reconstruct curved surface calculates to simplify once more after the human face region extraction finishes.Fig. 4 is the connection sheet result after cutting apart.

(2) rough face area connection sheet is chosen

The computation process that is communicated with sheet is divided into a plurality of coordinate points set P with three-dimensional the data of takeing on ₁... P _n, as seen from Figure 4, the front face zone can condense together preferably, just as one in a plurality of connection sheets.In most cases, human face region is that all are communicated with area maximum in the sheet (counting at most); But, if growing into one, neck, shoulder and chest be communicated with sheet, then the latter is wideer, bigger than human face region.Therefore, we at first select the connection sheet of two maximums, according to the priori that is communicated with plate shape, therefrom select real human face region by certain rule then, and are as follows:

To any one given connection sheet coordinate points set P _m, at first calculate TR _m(upper right point), BL _m(lower-left point) and Span _m, as follows:

${\mathrm{TR}}_m(x,y,z)={(\max \left(p_i\left(x\right)\right),\max \left(p_i\left(y\right)\right),\max p_i\left(z\right)))}_{p_i\∈P_m}---\left(4\right)$

${\mathrm{BL}}_m(x,y,z)={(\min \left(p_i\left(x\right)\right),\min \left(p_i\left(y\right)\right),\min \left(p_i\left(z\right)\right))}_{p_i\∈P_m}---\left(5\right)$

Span _m＝‖TR _m-BL _m‖ (6)

From P ₁... P _nIn select two of the Span maximum and be communicated with sheets, be designated as P _M1, P _M2, mutually Span should be arranged _M1, Span _M2, set a threshold value th _b, use following three rules to select real human face region:

1) if Span _M1＜th _b, Span _M2＜th _b, P _M1, P _M2The middle bigger zone of Span is chosen as the front face zone;

2) if Span _M1, Span _M2Among both one less than thb, another is greater than th _b, P _M1, P _M2The middle bigger zone of Span is chosen as the front face zone;

3) if Span _M1〉=th _b, Span _M2〉=th _b, P _M1, P _M2The middle less zone of Span is chosen as the front face zone.

Threshold value th _bValue obtain by training, we are taken as 180.

(3) face area accurately extracts

Because the error that discrete curvature is calculated, the human face region of choosing roughly is irregular, and has some holes, the also main region just of extraction; But,, can obtain comparatively smooth dough sheet by adjusting once more because the connection sheet that extracts has been the core patch in front face zone.

Suppose that P ' is people's face sheet point set of selecting roughly, find the solution a center of gravity of P ' collection, be the centre of sphere then with the center of gravity, before cutting apart, once cut again on the original data set P that the center of gravity that radius is got P ' collection is concentrated distance apart from the center of gravity solstics to P ' with a ball.By such adjustment again, a comparatively smooth human face region cutting has just been finished.

Test findings of the present invention is tested on FRGC v1.0 storehouse, and this storehouse comprises the data model of 943 original head shoulders, and method of the present invention all obtains good cutting effect on all models.Fig. 5 is the part of test results synoptic diagram.

The working time of the inventive method, the sampling rate with raw data had much relations, selected 50 models at each sampling density section, calculated average working time, the result is as shown in table 1, the computing platform of experiment is a PC, Pentium IV 2.4GHz processor and 512M DDR internal memory.By table 1 as seen, the most of the time consumes in curve reestablishing, and curvature is calculated and face area extracts about 25% the time that takies.Overall operational performance can satisfy demands of applications.

Consume the averaging time of the automatic extractive technique of table 1 face area

Claims (7)

1, extract the method for face area in a kind of three-dimensional scanning original data of from the beginning shoulder automatically, its step is as follows:

(1), the surface reconstruction of three-dimensional face: adopt the method for subdivision mesh, to the original piecewise smooth triangular mesh curved surface of three-dimensional head shoulder data construct;

(2), the calculating of Gaussian curvature: the discrete Gaussian curvature of on the triangular mesh data that surface reconstruction obtains, calculating each summit;

(3), human face region extracts:

Be communicated with the calculating of sheet: the Gaussian curvature based on step (2) is calculated becomes a plurality of connection sheets by certain threshold value with the mesh segmentation that obtains in the step (1);

Rough human face region is selected: from the connection sheet of cutting apart, select to comprise the rough selection result of the connection sheet of people's face nucleus maximum as human face region by certain rule;

The accurate extraction of human face region:, cut out accurate face nucleus by the connection sheet of rough selection and original three-dimensional head shoulder data.

2, extract the method for face area in the three-dimensional scanning original data of from the beginning shoulder according to claim 1 automatically, it is characterized in that: the method for the subdivision mesh described in the step (1) comprises that network topology estimation, grid optimization and piecewise smooth surface optimize three steps.

3, extract the method for face area in the three-dimensional scanning original data of from the beginning shoulder according to claim 1 automatically, it is characterized in that: the computing method of the Gaussian curvature that adopts in the step (2) are to one of the calculating of Gaussian curvature on the continuous curve surface formula approximate, that employing is following:

$\underset{A}{\∫\∫}\mathrm{KdA}=2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i---\left(1\right)$

$K=\frac{2\mathrm{\π}-\underset{i=0}{\overset{n-1}{\mathrm{\Σ}}}{\mathrm{\α}}_i}{\frac{1}{3}A}---\left(2\right)$

α _iBe the exterior angle in abutting connection with tri patch on a summit, A is in abutting connection with the tri patch total area.

4, extract the method for face area in the three-dimensional scanning original data of from the beginning shoulder according to claim 1 automatically, it is characterized in that: when in the step (3) the triangular mesh curved surface that obtains in the step (1) being divided into a plurality of connection sheet, carry out the expansion of connected relation according to 1 ring neighborhood of grid.

5, extract the method for face area in the three-dimensional scanning original data of from the beginning shoulder according to claim 1 automatically, it is characterized in that: in the operation of step (3), what described threshold value was got is the mean-Gaussian curvature of all grid vertexes.

6, the automatic method of extracting face area in the three-dimensional scanning original data of from the beginning takeing on according to claim 1 is characterized in that: the rough selective rule of face area need calculate three amounts to each connection sheet that connection sheet in the step (3) calculates in the step (3): upper right some TR _m, lower-left point BL _mAnd Span _m, it is as follows that it calculates used formula:

${\mathrm{TR}}_m(x,y,z)={(\max \left(p_i\left(x\right)\right),\max \left(p_i\left(y\right)\right),\max p_i\left(z\right)))}_{p_i\∈P_m}---\left(3\right)$

${\mathrm{BL}}_m(x,y,z)={(\min \left(p_i\left(x\right)\right),\min \left(p_i\left(y\right)\right),\min \left(p_i\left(z\right)\right))}_{p_i\∈P_m}---\left(4\right)$

Span _m＝‖TR _m-BL _m‖ (5)

P wherein _mBe that step (3) is communicated with any one the connection sheet that obtains in the sheet calculating, p _iIt is any one summit on this connection sheet;

Rough selective rule: at first select two to be communicated with sheet P _M1And P _M2, feasible corresponding Span _M1And Span _M2Be maximum and second largest in all connection sheets; Setting threshold th _b=180, if Span _M1〉=th _b, Span _M2〉=th _b, then select P _M2As face area, otherwise select P _M1As face area.

7, automatically extract the method for face area in the three-dimensional scanning original data of from the beginning shoulder according to claim 1, it is characterized in that: used cutting radius was the center of gravity of the rough connection sheet the selected distance apart from the center of gravity solstics to the set when face area accurately extracted in the step (3).

Images