JP2004506276A - Three-dimensional face modeling system and modeling method - Google Patents

Abstract

A front face, a side face, and a back face of a face are corrected on a screen on which a user displays a single face front image to produce a three-dimensional face shape that can be reproduced as an accurate three-dimensional stereoscopic image.
The operation method of the three-dimensional face modeling system provides a standard model of a virtual three-dimensional face expressed in the form of a polygon mesh and a front image of a target model to be modeled so as to be superimposed on a screen on which the standard model is displayed. . The standard model is deformed so that the superimposed standard model and the target model front image match. Based on the deformed standard model, a face front texture in a form obtained by correcting the front image of the target model is generated. The same pixels as the face color and the hair color are extracted from the generated frontal texture of the face, and the extracted pixels are added to the side and rear face textures of the face to edit the head shape. A three-dimensional face shape is restored by performing texture mapping on a standard model in which the front, side, and rear face textures of the face have been deformed.

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to three-dimensional face modeling in which a whole face is formed in a three-dimensional shape using one face front image. More specifically, the front, side, and rear faces of a face are corrected on a screen displayed by a user. The present invention relates to a three-dimensional face modeling that can reproduce an accurate three-dimensional stereoscopic image.
[0002]
The most important part for three-dimensional modeling of the human body is the face. The face not only occupies the largest specific gravity in shaping a person's identity, but also contains many important components in a small area. Such small changes in the components can make the face look totally different.
[0003]
Also, if an animation effect is provided by adding an additional element such as a three-dimensional body to a three-dimensional face, an avatar (avatar) having the same identity as the actual one in the virtual reality world (VR) can appear. Become. In other words, in order to form a virtual reality society with a true avatar, one's own three-dimensional face modeling must be implemented.
[0004]
In general, three-dimensional modeling of a face includes three-dimensional modeling using a three-dimensional scanning device, and photographs taken at other angles using stereo vision or the like. And three-dimensional modeling by combining two-dimensional information corresponding to a common area.
[0005]
[Problems to be solved by the invention]
The method of 3D modeling a face using scanning equipment can completely restore not only geometric models but also texture information such as complexion, but it is difficult to commercialize it to ordinary people because it uses expensive equipment. There is a problem.
[0006]
In addition, the method of restoring two-dimensional information such as stereo vision in three dimensions has a problem that photographs taken at various angles must be provided so that a common part of the provided photograph exists. .
[0007]
SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a system for modeling a three-dimensional face using only one face photograph taken in front.
[0008]
Another object of the present invention is to provide a method for three-dimensional face modeling using only one face photograph taken in front.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, a method of operating a three-dimensional face modeling system according to the present invention includes providing a virtual three-dimensional face represented by a polygon mesh as a standard model; Providing the model to be superimposed on the screen on which the model is displayed; deforming the standard model so that the front image of the superimposed standard model matches the front image of the target model; target model based on the deformed standard model Extracting the same pixels as the face color and hair color of the front image of the target model, slightly inside the boundary line between the front and side in the front image of FIG. Applying the same complexion and hair color to the side and back texture of the face and editing the hairstyle; and , And texture mapping the front, side, and rear textures of the face to restore a three-dimensional face shape.
[0010]
Preferably, the deformation step of the standard model adjusts the head circumference adjustment line and the horizontal and vertical adjustment lines of the standard model so that the contour of the standard model matches the front image of the target model, and linearly positions the entire vertex. A global transformation step for correcting and a local transformation step for modifying feature points and non-feature points of the standard model so that the global transformation of the standard model and the specific part of the target model front image coincide with each other, and correcting the specific part in detail. And
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram of a three-dimensional face modeling system according to the present invention. FIG. 2 is a flowchart of three-dimensional face modeling according to the present invention. FIG. 3 shows a virtual three-dimensional standard model provided by the present invention on a screen. FIG. 4 is an embodiment in which a front image of a target model provided by the present invention is superimposed on a standard model. FIG. 5 is a front image of a target model obtained by adjusting feature points of the standard model according to the present invention. FIG. 6 is an example of generating a front face texture according to the present invention, and FIG. 7 is an example of editing a rear face texture according to the present invention.
[0012]
FIGS. 3 to 7 showing the present embodiment are displayed on a screen, and the screen is composed of four small screens which are divided into upper and lower rows on the left and right sides. The upper right part of the screen deforms the standard model on the side, the lower left part of the screen edits the side and rear texture of the face, and the lower right part displays the three-dimensional restored 3D face three-dimensionally. A screen that can be rotated and compared is provided.
[0013]
Further, the polygon mesh in the present invention is a triangular net shape connecting a plurality of points of 500 to 1,000 or more with respect to the entire face, and expresses the size and shape of each entire face.
[0014]
The three-dimensional face modeling system of the present invention provides a virtual three-dimensional standard model providing module 102 for providing a three-dimensional virtual face as a standard model in the form of a polygon mesh, and a target model front image providing module for providing a face front image to be modeled. 104, a standard model transformation module 106 that superimposes the target model front image on the standard model and then deforms the standard model so as to match the target model, and a face that modifies the target model front image to generate a face front texture A front side texture generation module 112, face side and back side texture editing modules 114 and 116 for producing face side and back side textures as well as the face color and hair color of the face front side texture, and a front, side and back side texture of the produced face To the standard model of deformation And a texture mapping module 118 to ring.
[0015]
Preferably, the virtual three-dimensional standard model providing module 102 provides a virtual standard model three-dimensionally manufactured based on a general face as shown in FIG. The standard model is provided as a geometric three-dimensional image in which the entire face is a collection of polygon meshes 10 having a triangular shape. In addition, the standard model provided on the screen includes a head circumference adjustment line 2 for adjusting the horizontal / vertical ratio of the entire face in order to match the front image 1 of the target model, and a horizontal adjustment line for matching a specific part of the face. An adjustment line 4, a vertical adjustment line 6 and a feature point 8 are included.
[0016]
In the target model front image providing module 104, as shown in FIG. 4, the front image 1 of the target model obtained by scanning the front face photograph of the target model for three-dimensional production is superimposed on the screen on which the standard model is displayed. provide. Therefore, the overall size and the position difference of the main part between the various adjustment lines 2, 4, 6, and 8 of the standard model and the front image 1 of the actual target model can be easily understood.
[0017]
The standard model transformation module 106 largely includes a global transformation module 108 and a local transformation module 110, and transforms the standard model so that the front image 1 of the standard model and the target model match.
[0018]
At this time, as shown in FIG. 5, the global transformation module 108 compares the front image 1 of the target model superimposed on the screen with the standard model, and sets the standard model so that the standard model matches the front image 1. The head adjustment line 2, the horizontal adjustment line 4, the vertical adjustment line 6, and the like are adjusted.
[0019]
In the local transformation module 110, the global transformation module 108 generally matches the front image 1 of the target model with the outer shape of the standard model, and then specifies the face of the standard model such as eyebrows, eyes, nose, mouth, and ears. Adjustment is made such that a predetermined feature point 8 for the part matches the front image 1 of the target model. Further, such a feature point 8 is linked to a change in the feature point 8 and is accompanied by a change in a non-feature point expressing the bending of the entire face.
[0020]
The face front texture generation module 112 matches the front image 1 of the standard model and the target model, and then, based on the position of the head circumference adjustment line adjusted by the global transformation module 108, as shown in FIG. A number of pixels similar to the face color and hair color of the target model front image are extracted slightly inside the boundary line between the front and side of the image, and a face front texture that is a modified version of the target model front image is generated.
[0021]
The side and rear face texture editing modules 114 and 116 generate and edit the side and rear face textures using the generated front face textures. At the time of initial generation, the size of the side and back face textures of the face are the same as the size of the front face texture, and the composition is the face front texture, with the corrected right square area on the left of the face back face texture based on the eye position correction line. The left square area is to the right of the back face texture. The center portion is filled with both sides linearly corrected. After the generation of the side and rear surface textures, the editing of the textures is corrected using the head-shaped adjustment point 12, as shown in FIG.
[0022]
The texture mapping module 118 texture maps the generated front, side, and back textures of the face to a standard model of deformation. The front face texture is texture mapped at about 120 ° from the front of the face, and the texture coordinates are obtained by projecting each vertex corresponding to the front of the shape on the x and y planes. In the case of the side surface and the back surface texture, texture mapping is performed on an area that is not mapped by the face front texture. However, unlike the face front texture, texture coordinates are obtained by cylindrical mapping. In the cylindrical mapping, when the geometric shape is viewed from above the head, the side and rear surface textures are cylindrically formed from the side to the rear with the center thereof as an axis, and texture mapping is performed.
[0023]
Hereinafter, referring to FIG. 2, a detailed description will be given of a process of three-dimensional face modeling according to the present invention.
First, a three-dimensional virtual standard model is provided to be displayed on a screen (step S202). As shown in FIG. 3, the standard model is displayed on the screen on the front and side surfaces, respectively, and the provided standard model represents the entire face by a triangular polygon mesh 10 and defines the geometric shape of the entire face. Embody. The standard model has a head adjustment line 2 for adjusting the horizontal / vertical ratio of the face, a horizontal adjustment line 4 for adjusting the positions of the eyes, the nose and the mouth, and a vertical line for adjusting the positions of the eyes and the nose. It includes an adjustment line 6 and a feature point 8 schematically representing the position of a specific part of the face.
[0024]
After providing the standard model, various color and black-and-white front images 1 obtained by scanning and storing a frontal photograph of the face of the target model are provided so as to be superimposed on a screen on which the standard model is displayed as shown in FIG. S204). Therefore, the provided front face photograph is superimposed so as to be compared with the standard model constituted by the polygon mesh 10.
[0025]
If the front image 1 of the target model and the standard model are displayed on the screen, the user globally deforms the standard model (step S206). The global deformation compares the front image 1 with the standard model, and adjusts the head circumference adjustment line 2, the horizontal adjustment line 4, and the vertical adjustment line 6 of the standard model at a rate corresponding to the corresponding front image 1 respectively. The head adjustment line 2 adjusts the horizontal and vertical ratio of the face with respect to the front image 1. Of the three horizontal lines, the upper horizontal adjustment line is at the eye position, the middle horizontal adjustment line is at the nose end, The lower horizontal adjustment line adjusts to match the mouth position and groups the overall vertex. The three vertical lines are adjusted to match the positions of both eyes and the nose. Further, the position of the entire vertex of the standard model is linearly corrected in accordance with the amount of change according to the position of the deformed adjustment line.
[0026]
After adjusting the ratio of the entire face of the standard model by the global transformation, the specific part is corrected in detail by the local transformation (step S208). The local deformation matches the predetermined feature point 8 of each part of the standard model with the front image 1. The feature point 8 schematically represents the size and position of a specific part of the face such as the face outline, eyebrows, eyes, nose, mouth, and ears. Indexes are linked to points. At this time, in the deformation of the feature point 8, the front feature point 8 is for x and y in the x, y, and z coordinates, and is related to the movement for the z coordinate in the case of the side face.
[0027]
In the global deformation and the local deformation, a user can selectively use a symmetry mode and an asymmetric mode. The symmetry mode utilizes the fact that a human face is left / right symmetric. If one of the left and right adjustment lines or feature points moves, the corresponding adjustment line or feature point 8 on the other side is automatically set. Move to correspond. Alternatively, the respective adjustment lines 2, 6 or feature points 8 can be adjusted using the asymmetric mode.
[0028]
Based on the standard model deformed by such global deformation and local deformation, a face front texture in a form in which the face image of the target model is corrected is generated. As shown in FIG. 6, using the position of the adjustment line of the global deformation in the front image 1 of the target model, the image processing operation corrects the outer region with the face color and the hair color slightly inside the boundary line between the front and side by image processing work. To generate a face front texture (step S210).
[0029]
After generating the front face texture, the side and rear face textures are generated and edited (step S212). First, the initial configuration at the time of generation is such that the pixels in the corrected right-sided square region are on the left side of the side and rear-surface textures, and the pixels in the left-sided square region are on the right side, based on the correction line for the eye position in the front face texture. The center portion is filled with both sides linearly corrected. At the time of texture mapping, a user edits a region where the hair is inconvenient in a portion where the front texture is connected to the side and rear textures while checking the side and rear textures of the corresponding portion.
[0030]
The head shape of the side and rear textures of the face is corrected at the head shape adjustment point 12 as shown in FIG. If the head shape adjustment point 12 determines the head shape of the side and back of the face and forms a form with the adjustment point 12, the upper part of the line is the hair color and the lower part is the face color with the line formed from that point as the center. Perform image processing. Here, the user can use the symmetric mode as well as the global deformation and the local deformation, so that the change of the side and rear surface textures on one side is similarly applied to the other side.
[0031]
When the front, side, and rear textures of the face are completed, a texture mapping of about 120 ° from the front of the face is performed as a front texture of the face in the geometric shape of the three-dimensional face (step S214). The texture coordinates are obtained by projecting each vertex corresponding to the front of the geometric shape on the x, y plane. In the case of the face side and rear face textures, texture mapping is performed on an area that is not mapped by the face front texture. However, unlike the face front texture, texture coordinates are obtained by cylindrical mapping. When the geometric shape of the whole face texture is viewed from above the head, the face back surface texture is set up in a cylindrical shape from the side to the back with the center as an axis, and this is texture mapped.
[0032]
As described above, the preferred embodiment of the present invention has been described with respect to the three-dimensional facial modeling system. However, a configuration that can be easily changed by a person having ordinary knowledge in the technical field to which the present invention belongs is described in the appended claims. Are included in the technical idea described in the above.
[0033]
【The invention's effect】
According to the configuration of the present invention, the present invention can easily realize the entire face in three dimensions only with one frontal face photograph.
[0034]
In addition, the present invention can provide a complete three-dimensional stereoscopic image by embodying the complexion and hair color along with the overall facial shape.
[0035]
In addition, since the present invention can realize a three-dimensional stereoscopic image with one frontal face photograph, it can be applied to a virtual reality world by embodying its own avatar.
[Brief description of the drawings]
FIG.
1 is a schematic block diagram of a three-dimensional face modeling system according to the present invention.
FIG. 2
FIG. 4 is a flowchart illustrating a three-dimensional face modeling according to the present invention;
FIG. 3
5 is an example showing a virtual three-dimensional standard model provided by the present invention on a screen.
FIG. 4
It is an example in which a target model front image provided by the present invention is superimposed on a standard model.
FIG. 5
9 is an embodiment of adjusting the feature points of the standard model to match the target model front image according to the present invention.
FIG. 6
7 is an example in which the face color and hair color of the face front texture are reconstructed according to the present invention.
FIG. 7
9 is an example of editing a back face texture according to the present invention.
[Explanation of symbols]
1 Front image of target model 2 Head adjustment line 4 Horizontal adjustment line 6 Vertical adjustment line 8 Feature point 10 Polygon mesh 12 Head adjustment point

Claims (11)

A virtual three-dimensional standard model providing module for providing a three-dimensional virtual face with a polygon mesh standard model;
A target model front image providing module for providing a front image of the face to be modeled;
A standard model transformation module that superimposes the front image of the target model on the standard model, and then deforms the standard model to match the front image of the target model;
A face front texture generation module for generating a front face texture of a corrected face for the front image of the target model;
A face side and back face texture editing module for producing face side and back face textures as well as the face color and hair color of the front texture; and
A three-dimensional face modeling system, comprising: a texture mapping module for performing texture mapping of front, side and rear textures of the manufactured face on a standard model of deformation. The face side and back side texture editing module may provide a symmetry mode in which, when a head shape on one of the left side and the right side of the standard model is deformed, the corresponding head shape on the other side is automatically changed. The three-dimensional face modeling system according to claim 1, wherein the three-dimensional face modeling system performs. The virtual three-dimensional standard model has a head circumference adjustment line for matching the horizontal and vertical ratios of the entire face as compared with the front image of the target model;
A horizontal adjustment line for matching the horizontal positions of the eyes, nose and mouth as compared with the front image of the target model;
A vertical adjustment line for matching the vertical positions of both eyes and the nose as compared with the front image of the target model; and
The three-dimensional face modeling system according to claim 1, further comprising a feature point for matching a specific part of the standard model with a front image of the target model. A global deformation module that matches the contour of the standard model by associating the head circumference adjustment line, the horizontal adjustment line, and the vertical adjustment line of the standard model with the front image of the target model; and
3. The three-dimensional face modeling according to claim 1, further comprising a local deformation module that matches a specific part of the standard model by matching a characteristic part of the deformed standard model with a front image of the target model. system. The global transformation module and the local transformation module are configured such that, when one of the left and right adjustment lines and the feature point of the standard model moves, the corresponding adjustment line or the feature point on the other side automatically changes (symmetry mode). The three-dimensional face modeling system according to claim 4, wherein Providing a three-dimensional virtual face as a standard model of a geometrical form expressed by a polygon mesh;
Providing a front image of the target model to be superimposed on a screen on which the standard model is displayed;
Deforming the standard model so that the front image of the superimposed standard model matches the front image of the target model;
Based on the deformed standard model, in the front image of the target model, the same pixels as the face color and hair color of the front image of the target model are extracted slightly inside the boundary between the front and side surfaces, and the face front texture is extracted. Generating;
Applying the same complexion and hair color as the complexion and hair color of the generated frontal texture to the side and rear face textures, and editing the hairstyle;
A method for modeling a three-dimensional face, comprising the steps of texture mapping the front, side, and rear textures of the face to restore a three-dimensional face shape. The step of deforming the standard model includes adjusting a head circumference adjustment line, a horizontal adjustment line, and a vertical adjustment line of the standard model so that a contour of the standard model matches a front image of the target model, and vertex (vertex). A) a global transformation step that linearly corrects the overall position;
A local modification step of modifying feature points and non-feature points of the standard model so that the adjusted and corrected standard model and a specific part of the front image of the target model coincide with each other, thereby correcting the specific part in detail. The method of claim 6, further comprising: The method according to claim 11, wherein in the step of generating the frontal face texture, the extracted pixels of the face color are image-processed in a radial direction about the position of the mouth, and the pixels of the hair color are processed in the radial direction about the forehead. 7. The three-dimensional face modeling method according to 6. Giving a face color and a hair color to the back face texture is such that, based on the correction line for the eye position in the front face texture, the corrected right square area enters the left of the face back texture, and the left square area corresponds to the face. The three-dimensional face modeling method according to claim 6, wherein the right side of the back surface texture is corrected linearly on both sides of the center part. 7. The texture mapping method according to claim 6, wherein the texture mapping is to perform texture mapping of about 120 [deg.] Of the front of the face in the case of the front face texture, and to perform cylindrical mapping in the case of the side and rear face textures of the face. 3D face modeling method. The correction of the hairstyle on the side and the back of the face is performed by adding a hairstyle adjustment point by the hairstyle on the side and the back of the face, and then giving a hair color above the line around the line formed by the hairstyle adjustment point. The three-dimensional face modeling method according to claim 6, wherein a complexion is provided below the line.

Images