KR20040036831A - System and method for compositing three dimension scan face model and recording medium having program for three dimension scan face model composition function - Google Patents

Abstract

PURPOSE: A system for mixing 3-dimensional scanned face models and a method thereof and a recording medium with a program including a function to mix the 3-dimensional scanned face models, are provided to construct a face model database by performing conversion of a cylinder coordinate system and mapping for 3-dimensional face scan data, and to mix the 3-dimensional face models based on statistics using the face model database. CONSTITUTION: A data corrector performs a correction operation to unify a data structure of a mesh and a polygon for multiple 3-dimensional face scan data. A face model database(22) classifies and stores the 3-dimensional face model data corrected by the data corrector. An average model generator(24) generates an average face model for the stored multiple face model data. A probability distribution extractor(26) calculates a difference between the average face model and each face model to generate a probability distribution value for the face models. A statistical model generator(30) generates a face model statistically mixed from data of the average face model and each face model, based on a model characteristic parameter of a user inputted from an external. A model mixer(32) performs a mixing operation for a new face model through morphing and caricature for data of the statistical face model and each face model.

Description

System and method for compositing three dimension scan face model and recording medium having program for three dimension scan face model composition function}

The present invention relates to a recording medium containing a synthesis system of a 3D scan face model, a method thereof, and a program having a synthesis function of a 3D scan face model, and more particularly, to correcting scan data of an arbitrary 3D face model. 3D scan face modeling system, method, and 3D scan for constructing database of face model and synthesizing new 3D face model that can be stochastic based on face model data of face model database The present invention relates to a recording medium containing a program having a synthesis function of a face model.

In general, in the process of modeling a human face using a computer, a high level of expertise and time-consuming manual labor are required in order not to create an unnatural face that is difficult to exist in reality. The difficulty in facial animation processing is to distinguish between the problem of finding and defining feature points that correspond to each other among many facial models, and the distinction between faces that can exist in the real world and those that do not exist.

That is, in the generation and morphing of the face model, the correspondence of the feature points between the face models is very important. In order to find a correspondence relationship between the same points between the face models, a point such as the tip of the face or the tip of the eye is usually used. In the case where the feature points are displayed and the feature points are matched, the feature points are caught within a range of about 50 to 300, but in order to extract more accurate feature points, manual operation for a long time must be preceded.

In addition, in order to enable accurate face model generation and morphing, all feature points must be accurately aligned and each correspondence must be defined. This requires relying on the operator's empirical knowledge and requires a great deal of manual work. There is nothing else. That is, in the current face model generation technique, only automatic matching techniques for visually distinct feature points such as an eye tip or a lip tip are implemented, and a lot of manual work is required for the rest of the process.

In addition, another problem in the creation of the face model is to distinguish between the natural face and the unnatural face that exist in the real world. To solve this problem, the operator's empirical knowledge is absolutely working, but the three-dimensional face In order to manipulate various features of the face using the model and create a visually natural effect, the work requires a lot of time and a considerable aesthetic sense of the worker.

In order to solve the above problems, recently, the scan data of a plurality of three-dimensional faces using an automatic correspondence method applying an optical flow algorithm and a bootstrapping algorithm, the appearance (Geometry) and color ( Colors are mapped to vertices by 1: 1 point-to-point, and linearly combining the vertices of each model to create a Morphable Face Model with a new appearance and face color.

The morphable face model generation method combines non-linear equations and probabilities and combines a morphable model into a two-dimensional scanned face image to reconstruct it in three dimensions.

However, this conventional morphable face model generation technique requires a large number of face models scanned in a cylinder coordinate system in three dimensions, but in order to obtain such a face model, a number of manual operations correspond to each of the scanned face models. Due to the introduction of nonlinear equations and matching methods, the algorithm for the synthesis of face models is complicated and a large amount of calculations have to be performed.

In addition, in the conventional face modeling process, there is a high possibility that an unnatural face shape that is difficult to exist in the real world is calculated, and morphing or compositing becomes difficult due to the lack of correspondence points between the face meshes. There is a disadvantage that the manual work due to the increase, and the modeling result is not consistently calculated because the results of the modeling is dependent on the individual judgment criteria or work ability of the worker.

Accordingly, the present invention has been made to solve the above-mentioned conventional problems, and an object thereof is to construct a face model database by transforming and mapping a three-dimensional face scan data into a cylinder coordinate system, and using the face model database for statistics. The present invention provides a recording medium containing a synthesis system of a 3D scan face model and a method thereof, and a program having a synthesis function of a 3D scan face model.

Another object of the present invention is to generate a face model capable of morphing three-dimensional scan to create infinitely new face models of various types through statistical combination processing of the model by morphing between models and exaggeration of features The present invention provides a recording medium containing a synthesis system of a face model, a method thereof, and a program having a synthesis function of a three-dimensional scan face model.

1 is a process chart illustrating a face data processing process for constructing a database of a 3D scan face model to which the present invention is applied;

FIG. 2 is a diagram illustrating three-dimensional face scan data input to construct a database of a three-dimensional scan face model; FIG.

3 is a diagram illustrating a state in which mesh data of a 3D face model is initially aligned;

4A and 4B are views illustrating voxel extraction and color assignment states of aligned mesh data by way of example;

5A to 5C are views illustrating a state of reconstructing a mesh and extraction of scan lines and cylinder coordinates of 3D face scan data;

6A and 6B exemplarily illustrate a state in which 1: 1 mapping of 3D scan mesh data to a standard reference face model is performed;

7 is a view showing the configuration of a synthesis system of a three-dimensional scan face model according to the present invention;

8 is a diagram illustrating an example of a configuration of a model synthesizing unit illustrated in FIG. 7;

9 is a diagram illustrating another example of the configuration of the model synthesizing unit illustrated in FIG. 7;

10A and 10B are flowcharts for explaining an operation of a method for synthesizing a 3D scan face model according to the present invention;

11 is a diagram illustrating a state in which each 3D face model is automatically 1: 1 mapped according to an exemplary embodiment of the present invention.

12A to 12C are views illustrating a state in which a statistical mean model is generated for a three-dimensional face model according to an exemplary embodiment of the present invention.

FIG. 13 is a diagram illustrating probability distribution states of appearance and face color of a three-dimensional face model; FIG.

14 is a diagram illustrating face models that can be generated stochasticly from a database of three-dimensional face models;

15 is a view showing a three-dimensional morphing result between different face data;

16A to 16C are views illustrating results of performing a feature process of a face model according to an exemplary embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

20: data mapping section, 22: face model database,

24: average model generation unit, 26: probability distribution extraction unit,

30: statistical model generation unit, 32: model synthesis unit.

According to the system of the present invention for achieving the above object, data correction means for performing a correction for unifying the data structure of the mesh and polygon for a plurality of three-dimensional face scan data, and the correction by the data correction means A face model database in which 3D face model data is stored for each classification, an average model generation means for generating an average face model for a plurality of face model data stored in the face model database, and an average face model generated by the average model generation means A probability distribution extracting means for generating a probability distribution value for the face model by calculating a difference between the face model and each face model, and statistically comparing the mean model data with each face model data based on the user's model feature parameters received from the outside. Statistical model generating means for generating a combined face model, and the statistics It provides del generating means a statistical face model and each face model 3D synthesis system of the scanning face model, characterized in that consisting of a model synthesizing means for synthesizing process the new face models through morphing and caricature of the data generated through.

In order to achieve the above object, according to the method of the present invention, receiving a plurality of three-dimensional face scan data to perform a correction process for unifying the data structure of the mesh and polygon, and morphing through the correction process Constructing a face model database for each category of face model data, generating an average face model for a plurality of 3D face models constructed in the face model database, and a difference value between the average face model and each face model Calculating a probability distribution by calculating a value; calculating a difference value between average face model data and each face model data according to a feature parameter input by a user, and generating a statistically combined face model; and Create a new face model using the combined face model data and the average face model data. It provides a method for synthesizing the three-dimensional face model scan, characterized in that comprising the step of processing property.

According to the recording medium in which the program of the present invention is recorded to achieve the above object, a function of receiving a plurality of three-dimensional face scan data and performing a correction process for unifying the data structure of a mesh and a polygon, and performing the correction process A function of constructing a face model database for each classification of the face model data that can be morphed through, A function of generating an average face model for each of a plurality of three-dimensional face models constructed in the face model database, The average face model and each face The probability distribution is calculated by calculating the difference between appearance and face color with the model, and the statistically combined face by calculating the difference between the average face model data and each face model data according to the feature parameters input by the user. A function of generating a model, and the statistically combined face model data The present invention provides a recording medium containing a program having a synthesis function of a three-dimensional scan face model, including a function of synthesizing a new face model using average face model data.

Hereinafter, the present invention configured as described above will be described in detail with reference to the accompanying drawings.

That is, FIG. 1 is a flowchart illustrating a face data processing process for constructing a database of a 3D scan face model to which the present invention is applied.

As shown in FIG. 1, a data processing process for constructing a database for various face data scanned in three dimensions includes a mesh data alignment process (2), a voxel extraction process (4), and a cylinder coordinate system sampling process (6). ), And the face data mapping process 10.

In the mesh data alignment process (2), the phases of the different meshes are aligned so that the scanned 3D face scan data from the outside has the same data structure. The slope of the x, y, and z axes is corrected so that the height of all face models is equally corrected based on the length of the limit position of each face model (ie, the starting point of the neckline from the forehead). The correction is performed to delete unnecessary parts such as ears and neck, leaving only necessary face parts.

As illustrated in FIG. 2, the three-dimensional face scan data is synthesized by combining the scan data scanned from the mesh, the image, and the three-dimensional scanner, and input as the texture-mapped three-dimensional face data.

As shown in FIG. 3, the plurality of different three-dimensional face scan data shows the result of initial alignment through the mesh data initial alignment process (2).

In addition, the voxel extraction process (4) is a data structure of each face model because the data structure, such as the mesh structure and the number of polygons of the face model initially aligned through the mesh data alignment process (2) is different for each model This is to create a single unified structure, which maps the surface components of each triangle constituting the mesh of the face model to a regular voxel grid of constant resolution, extracting all the vertices constituting the surface of the face model, and extracts the texture image. To calculate the color value of each vertex.

That is, in the voxel extracting step 4, as shown in FIG. 4A, a voxel containing surface components is defined as a surface voxel by mapping to a voxel of volume data having a size of 256 × 256 × 256. As shown in the above, the color values of the surface components are assigned.

Next, in the cylinder coordinate system sampling step 6, the scan data of the three-dimensional face is reconstructed using a regular volume grid and sampled by the cylinder coordinate system. Here, the cylinder coordinate system sampling process 6 converts the surface voxels into one continuous line on each voxel surface of the face model and converts them into scan lines, as shown in FIG. As shown above, the intersection of the outermost scan line that meets while rotating at a certain angle on each voxel plane is centered on the line passing through the center of the face model, and the color value at that point is stored and converted into a cylindrical coordinate system. Done.

In addition, in the cylinder coordinate system sampling process 6, as shown in FIG. 5C, since the three-dimensional face data obtained through the cylinder sampling is reconstructed into a regular mesh structure of a rectangular shape, each face model data obtained through the scan is uniform. It has a data structure and can be converted into the appearance of an image unfolded in two dimensions (ie image (h, Φ) = (x, y, z) + (R, G, B)). Work can be easily implemented in two dimensions.

In the face data mapping process 10, mesh data obtained through the cylinder coordinate system sampling process 6 is manually mapped 1: 1 with a preset standard reference face model 8, as shown in FIG. 6A. Since the data stored in the image (image (h, Φ)) at the same position in each face model data must store the appearance (x, y, z) and color values (R, G, B) of the same part. As shown in FIG. 6B, elements that determine the features of the face, that is, eyes, eyebrows, noses, and lips using feature lines indicating the positions of the eyes, noses, and mouths, correspond to the feature lines and the feature lines. The linear interpolation is applied to the rest of the parts.

Next, FIG. 7 is a diagram illustrating a configuration of a synthesis system of a 3D scan face model according to the present invention.

As shown in FIG. 7, the synthesis system of the 3D scan face model according to the present invention includes a data mapping unit 20, a face model database 22, an average model generator 24, and a probability distribution extractor 26. ), A statistical model generation unit 30, and a model synthesis unit 32.

The data mapping unit 20 is to automatically 1: 1 map the face model data manually mapped through the face data mapping process 10 shown in FIG. 1 using a nonlinear equation, which is a manual mapping. If only the feature lines are used during operation, it is difficult to accurately match the contours of the eyes and mouth. Therefore, the nonlinear equations (LMA) for the eye and mouth regions are used using the color values (R, G, B). In this case, the mapping operation is performed again so that the difference in color values between the face models is minimized. As a result of the automatic 1: 1 mapping operation, data as shown in FIG. 11 can be calculated.

The face model database 22 stores a plurality of face model data having a uniform data value calculated as a result of automatically 1: 1 mapping through the data mapping unit 20. To make it possible to create an unlimited number of models, hundreds of male and female face model data are stored by gender and age.

The average model generator 24 uses the geometry and face color information for each part of the face model data, which is classified and stored in the face model database 22 according to gender and age, to calculate arithmetic mean values. By generating the average face model for each category, the average face model for each category is calculated.

Here, the average model generator 24 calculates an average face model by gender for the male face model and the female face model, respectively, as shown in FIGS. 12A and 12B, and by classification, as shown in FIG. 12C. (E.g., gender) an average face model is used to generate an overall three-dimensional average face model (i.e., male / female / total average face model from left in FIG. 12C).

In addition, the probability distribution extractor 26 calculates a difference value of appearance and face color between the average model and each face model calculated by the average model generator 24 in the form of a probability distribution to extract features of each classification. As shown in FIG. 13, it is possible to calculate a probability distribution of appearance and face color for each face model and generate a face model according to the aspect of the probability distribution, and from the face model database 22, When combining faces, it is possible to express numerically whether or not the probability of generating a realistic face.

The statistical model generator 30 statistically generates a random face model according to a probability distribution of the face model extracted from the probability distribution extractor 26 based on user input data 28 inputted by gender and age. It is to.

Here, the statistical model generation unit 30 receives the user's model feature parameters such as gender and age as the user input data 28, and each face model data whose 1: 1 correspondence is calculated based on the input parameters. In addition to calculating the difference between the appearance of the vertex unit and the face color value using the overall average model data, statistically various source face models are generated by statistically combining the calculated values.

The model synthesizing unit 32 is for synthesizing a new face model through mutual morphing and caricature processing for the source face model calculated in a 1: 1 correspondence generated by the statistical model generating unit 30.

Here, an example of the configuration of the model synthesizing unit 32 includes a target model selecting unit 36 and a target model interpolation processing unit 40 as shown in FIG. 8. In order to generate a new face model from the source face model 34, a target model is received and selected, and a target model is selected. The target model interpolation processor 40 sets the selected target model according to a preset interpolation parameter. It is to synthesize new face model by interpolating appearance and face color.

Accordingly, the three-dimensional morphing result between the target model selecting unit 36 of the model synthesizing unit 32 and the different face model data through the target model interpolation processing unit 40 is shown in FIG. 15.

In addition, another example of the configuration of the model synthesizing unit 32 is composed of a feature element extracting unit 46, an average calculating unit 48, and a data deformation processing unit 50, as shown in FIG. The feature element extractor 46 receives the face model data 42 and the average model data 44 correspondingly calculated in a one-to-one correspondence, and generates facial features to generate a caricature according to the setting input of the feature model. The elements are extracted, and the average calculating unit 48 calculates a difference value between the average model and the remaining face models except for the specific face model for which the feature is to be calculated, and obtains an average, thereby determining the specific residuals in the average model. Numerical data reflecting the features of the face model is extracted.

That is, in the mean calculation unit 48, for example, when there is a male face model and a female face model, the numerical data representing the overall characteristics of the male face model in the overall average model is a value obtained by sequentially subtracting the female face models one by one. It can be obtained by averaging.

In addition, the data deformation processing unit 50 exaggerates the characteristics of a specific face model using the numerical data obtained from the average calculating unit 48, which continuously adds numerical data to the average model to gradually increase the average model. As a result, as shown in Figs. 16A, 16B, and 16C, exaggeration of masculine features, exaggeration of feminine features, and personally existing features can be made. Can be exaggerated.

Next, the operation of the present invention made as described above will be described in detail with reference to the flowcharts of FIGS. 10A and 10B.

First, three-dimensional face data scanned through a three-dimensional scanner and subjected to correction processing through a mesh data alignment process (2), a voxel extraction process (4), a cylinder coordinate system sampling process (6), and a face data mapping process (10). Is input (step S10), the data mapping unit 20 automatically applies the nonlinear equations so that the face color value (color) between the face models for each of the three-dimensional face data is minimized. (Step S11).

In this state, when the mapping operation of the three-dimensional face data through the data mapping unit 20 is performed, each of the three-dimensional face data can be morphed into a face model database 22 by dividing it by gender and age. It is stored (step S12).

Next, the average model generation unit 24 classifies a plurality of three-dimensional face model data stored separately by sex and age (step S13), and shapes that are already stored for each classified face model. ) By calculating the arithmetic mean value for each classification using the data of the face and the color of the face, and the average face model of each category (eg, the average face model of the man and the average face model of the woman) and the overall average face model (eg, of the man and the woman). An overall average face model) is generated (step S14).

In addition, the probability distribution extractor 26 extracts a feature for each classification by using the difference between the appearance and the face color of the face model for each category by the average model, and the difference between the appearance and the face color. Probability distributions are calculated by using the method, and a face model is generated according to the probability distribution (step S15).

Accordingly, the statistical model generator 30 receives the user's model feature parameters such as gender and age as the user input data 28 (step S16), and calculates a 1: 1 correspondence based on the input parameters. The difference between the appearance of the vertex unit and the face color value is calculated by using each face model data and the total average model data (step S17), while a new face model having infinitely diverse shapes is statistically combined with the difference values. It is possible to be generated (step S18).

Next, the model synthesizing unit 32 determines whether or not an operation for performing morphing between the three-dimensional face model data calculated to have a 1: 1 correspondence is performed (step S19).

As a result of the determination, when it is determined by the operator that the manipulation for mutual morphing between the three-dimensional face model data is being performed, the target model selecting unit 36 of the model synthesizing unit 32 performs the statistical model generating unit 30. The target face model 34 receives data from the source face model 34 generated from the target model according to the setting input of the target model (step S20).

In this state, the target model interpolation processing unit 40 performs a process for interpolating the appearance of the face and the face color according to the preset interpolation parameter data 38 to synthesize a new face model (step S21).

On the other hand, if it is determined that the morphing of the face data is not performed according to the determination result of step S19, the feature element extracting unit 46 of the model synthesizing unit 32 and the respective face model data 42 The feature model of the face model data is calculated based on the input of the average model data 44 and the setting input of the model to be characterized (step S22).

If it is determined that the calculation of the feature element for the face model data to be characterized by the feature element extractor 46 is completed (YES in step S23), the average calculation unit 48 features the feature by the feature element extractor 46. The average value is calculated by calculating the difference between the average model and the other models except for the model for which the feature is to be obtained (step S24).

In this state, the data transformation processing unit 50 can exaggerate the feature for the face model by continuously adding numerical data based on the average value obtained through the average calculating unit 48 to the average model (step S25). ).

It is a matter of course that the present invention having the above-described embodiments can be variously modified and implemented without departing from the technical spirit of the present invention without departing from the embodiments.

As described above, in order to generate 3D face scan data into a morphable face model, the sample is converted into cylinder sampling data and mapped to 1: 1 and generated as a morphable face model using the face model data. By allowing morphing and feature exaggeration between each face model, it is possible to create new face models infinitely by statistically combining the faces existing in the real world through a simpler operation. It has the effect that it can be widely applied to the fields of modeling, animation, and avatar for commercial use of faces.

Claims (13)

Data correction means for performing correction to unify data structures of meshes and polygons for a plurality of three-dimensional face scan data; A face model database in which three-dimensional face model data corrected by the data correction means is stored for each classification; Average model generation means for generating an average face model for a plurality of face model data stored in the face model database; A probability distribution extracting means for generating a probability distribution value for the face model by calculating a difference between the average face model and each face model generated by the mean model generating means; Statistical model generating means for generating a statistically combined face model of the average model data and each face model data based on the model feature parameters of the user received from the outside; And a model synthesizing means for synthesizing the new face model through the statistical face model generated by the statistical model generating means and morphing and caricature of each face model data. The method of claim 1, wherein the data correction means performs alignment processing of mesh data through tilt correction, height correction, and deletion correction of 3D face scan data, and maps volume data of a face model to voxels to determine surface components. Performs voxel extraction that assigns color values to the surface voxel, performs sampling processing of the cylinder coordinate system that converts continuous lines of the surface voxel into scan lines, and converts them into cylindrical coordinates, 3. A system for synthesizing a three-dimensional scan face model, characterized in that a mapping process is performed on a face reference and a face data to be mapped in a 1: 1 correspondence with each detail. The method of claim 1, wherein the means for generating the average model is configured to generate an average face model for each of the plurality of face model data stored in the face model database, and generate an overall average face model of the face model data. Synthesis system of 3D scan face model. The method of claim 1, wherein the means for generating the average model is configured to generate an average face model by calculating an arithmetic mean value for each classification of the geometry and the color information included in the face model data stored in the face model database. Synthesis system of a three-dimensional scan face model, characterized in that made. The method of claim 1, wherein the probability distribution extracting means extracts a feature for each classification using a difference value between the mean face model and each face model and appearance and face color information, and extracts a probability distribution using the difference between appearance and face color. 3D scanning face model synthesis system characterized in that it is configured to calculate. 2. The apparatus of claim 1, wherein the face model synthesizing means comprises: a target model selecting unit for selecting a target model for generating a new face from among the face models generated from the statistical model generating unit, and applying specific interpolation parameter data to the target model; And a target model interpolation processing unit for interpolating the appearance of the face and the color of the face. The apparatus of claim 1, wherein the face model synthesizing means receives the average face model and each face model data, and extracts feature elements of the face according to a setting input of the feature model. The average calculation unit calculates an average value by calculating a difference value with the remaining face model except for, and a data deformation processing unit for applying the average value to the face model to exaggerate the feature of the three-dimensional scan face model Synthesis system. Receiving a plurality of three-dimensional face scan data and performing correction processing to unify data structures of meshes and polygons; Constructing a face model database for each category of face model data that can be morphed through the correction process; Generating an average face model for a plurality of 3D face models constructed in the face model database; Calculating a probability distribution by calculating a difference value between the average face model and each face model, Generating a statistically combined face model by calculating a difference between the average face model data and each face model data according to a feature parameter input by the user; And synthesizing a new face model using the statistically combined face model data and average face model data. The method of claim 8, wherein the receiving of the plurality of three-dimensional face scan data and performing a correction process for unifying data structures of meshes and polygons comprises: Performing the alignment process of the mesh data through the tilt correction, the height correction and the deletion correction of the three-dimensional face scan data; Performing a voxel extraction process of mapping volume data of the face model to voxels and assigning color values to the surface voxels of the surface components; Performing a sampling process of the cylinder coordinate system converting the continuous line of the surface voxel into a scan line and converting it into a cylindrical coordinate system; A method of synthesizing a three-dimensional scan face model, comprising the step of performing a mapping process of the face data for mapping the mesh data of the face model to one-to-one correspondence with a specific standard reference face model for each detail part. . The method of claim 8, wherein the generating of the average face model for the plurality of 3D face models built in the face model database comprises: Classifying each face model stored for each classification in the face model database; Generating an average face model by classification by obtaining an average value of the appearance and face color information of each face model; And generating a total mean face model with respect to the overall face model. The method of claim 8, wherein the synthesizing of the new face model using the statistically combined face model data and average face model data comprises: Setting a target model according to a setting input of a target model for statistically combined face model data; And synthesizing a new face model by interpolating the appearance and face color of the face model by applying a specific interpolation parameter to the target model. The method of claim 8, wherein the synthesizing of the new face model using the statistically combined face model data and average face model data comprises: Extracting features of the face model data based on the average face model according to the setting input of the feature model; And calculating an average of the difference values based on the extracted features, and exaggerating the features of the face model by the average values. A function of receiving a plurality of three-dimensional face scan data and performing a correction process to unify data structures of meshes and polygons; A function of building a face model database for each classification of the face model data that can be morphed through the correction process; A function of generating an average face model for each classification of a plurality of 3D face models constructed in the face model database; Calculating a probability distribution by calculating a difference between the average face model and each face model in appearance and face color; A function of generating a statistically combined face model by calculating difference values between average face model data and each face model data according to feature parameters input by a user; And a program for synthesizing a three-dimensional scan face model, comprising: synthesizing a new face model using the statistically combined face model data and average face model data.