KR101759188B1 - the automatic 3D modeliing method using 2D facial image - Google Patents

Abstract

The present invention relates to a method for generating a 3D model from a 2D image, and more particularly, to a method for automatically generating a 3D model from a single two-dimensional frontal face photograph using a technique capable of easily generating a 3D model .
A method for automatically generating a 3D model from a 2D face image, the method comprising:
(102) converting a color image to a gray image, detecting (103) a face from the gray image transformed from the step, detecting (104) eye concealing from the detected face, A method comprising: executing a filter (105), detecting an edge in a gray image (106), separating foreground and background from edge information detected in the step (110), distributing a foreground vertex Step 111: Computing the front and rear three-dimensional vertices 107 from the distributed vertices, creating a rear partial surface 108, and calculating the forward vertex calculated in the step 107 in the step A step 109 of re-setting the forward partial vertex using the information extracted in steps 103, 104 and 105, and a step 109 of generating a forward partial surface using the vertex information calculated in the step 109 And a step (112) of creating a final 3D model using the information generated through the steps (113) and all the steps.
In addition, the present invention may also include calculating (107) the forward and backward three-dimensional vertices from the distributed vertices,
Calculating the center coordinates (centerX, centerY) of the image in a horizontal line, each of the virtual and image-space coordinates (x, y) for calculating vertex coordinates _{V (V x, V y,} V z) of the three-dimensional model corresponding to the on And the 3D model is automatically generated from the 2D face image.
Further, the present invention is characterized in that the step 109 of resetting the forward partial vertex comprises:
V _z = V _z + ΒI and that (x, y) / G Formula reset operation to the _max from the 2D face image according to claim it provides a method for automatically generating a 3D model.

Description

A method for automatically generating a 3D model from a 2D facial image is disclosed.

The present invention relates to a method for generating a 3D model from a 2D image, and more particularly, to a method for automatically generating a 3D model from a single two-dimensional frontal face photograph using a technique capable of easily generating a 3D model .

3D models of faces are in demand in various fields such as computer games and animation production, and more demand has been demanded with the recent spread of 3D printers.

The 3D models for the face were created using skilled professionals and specialized software tools or created using a scanner.

Recently, researches for generating 3D models easily are being actively conducted by the general public.

There are many ways to model the face of the general public in three dimensions.

One of them is a method of directly modeling a three-dimensional face with scanned data by scanning the face of a specific person in three dimensions, and the other is a method of photographing a face of a specific person at various angles, There is a method of modeling a three-dimensional face of a specific person by modifying a general face model of a shape.

In the former case, it is possible to obtain fairly accurate 3-D face modeling for a specific person's face by using specialized equipment such as a 3-D scanner to obtain information about the bending of a specific person's face and the actual skin color. But this is expensive

Equipment and 3D face modeling should be scanned by going to a specific place where a 3D scanner is located so that 3D face modeling can be performed without inconvenience.

In order to solve the above-described problems, a method for generating a three-dimensional face model using a two-dimensional face front image, hereinafter referred to as " A step of setting a control point on a two-dimensional image, a step of transforming a three-dimensional basic model made in advance to coincide with a two-dimensional image using the control point, a step of mapping a texture of the two- A method of generating a three-dimensional face model using a two-dimensional face front image including steps ".

[0002] Recently, the number of patent applications related to the automatic three-dimensional modeling and system capable of generating a three-dimensional model from photographs or other images containing the above-described prior art has been increased.

The conventional technique for generating a three-dimensional face generates a three-dimensional face using a point cloud composed of all 3D points corresponding to a face and texture information corresponding to the point cloud.

That is, since the number of 3D points required to generate a three-dimensional face is large, the time is delayed in the process of generating a three-dimensional face, that is, the number of 3D points required to generate a three- The present invention provides a method of automatically generating a 3D model from a 2D face image that significantly reduces the time delay.

In addition, since the conventional technology can include only one face of a single image, there is a problem that a plurality of images taken at various viewpoints are required to generate a three-dimensional face. And a method for automatically generating a 3D model from a 2D face image that generates a 3D face corresponding to various viewpoints with a single image.

In order to solve the above problems and needs,

A method for automatically generating a 3D model from a 2D face image,

(102) converting a color image to a gray image, detecting (103) a face from the gray image transformed from the step, detecting (104) eye concealing from the detected face, A method comprising: executing a filter (105), detecting an edge in a gray image (106), separating foreground and background from edge information detected in the step (110), distributing a foreground vertex Step 111: Computing the front and back three-dimensional vertices from the distributed vertices 107, creating a rear partial surface 108, and calculating the forward vertex calculated in the step 107 in the step A step 109 of re-setting the forward partial vertex using the information extracted in steps 103, 104 and 105, and a step 109 of generating a forward partial surface using the vertex information calculated in the step 109 And a step (112) of creating a final 3D model using the information generated through the steps (113) and all the steps.

The present invention also includes a step 107 of calculating front and back three-dimensional vertices from the distributed vertices,

Calculating the center coordinates (centerX, centerY) of the image in a horizontal line, each of the virtual and image-space coordinates (x, y) for calculating vertex coordinates _{V (V x, V y,} V z) of the three-dimensional model corresponding to the on The 3D model is automatically generated from the 2D face image.

Further, the present invention is characterized in that the step 109 of resetting the forward partial vertex comprises:

V _z = V _z + ΒI and that (x, y) / G Formula reset operation to the _max from the 2D face image according to claim it provides a method for automatically generating a 3D model.

The method of automatically generating a 3D model from a 2D facial image according to the present invention creates an effect that a 3D modeling can be easily performed by a non-expert child and a general public.

1 is a block diagram according to an embodiment of the present invention.
FIG. 2A is a result of image processing in which a color-input image is converted into a gray image according to an embodiment of the present invention. FIG.
FIG. 2B illustrates a result of edge detection image processing according to an exemplary embodiment of the present invention.
FIG. 2C shows a result of image processing in which a face is detected in a foreground image according to an embodiment of the present invention, and eye, nose, and mouth are detected.
FIG. 2D shows a result of the image processing showing that the eyebrows and the nose portion are corrected and corrected using the equations (13) to (16) in the step of resetting the vertex of the front part.
[Figure 3a] is a result when the eyebrows and the nose area are not processed.
[Fig. 3B] is a result of treating eyebrows and nose portions more naturally.
FIG. 4 is a diagram showing a Haar-like feature. FIG.
5 is a view showing the use of an interpupillary distance (IPD) between the centers of two eyes detected in an image as the size of a face in an image is varied according to the distance and zoom between the camera and the subject.
6 is a diagram illustrating a principle of arranging vertices at regular intervals on an image separated by foreground and generating faces using vertices;

BRIEF DESCRIPTION OF THE DRAWINGS FIG.

A method for automatically generating a 3D model from a 2D face image, the method comprising:

(102) converting a color image to a gray image, detecting (103) a face from the gray image transformed from the step, detecting (104) eye concealing from the detected face, A method comprising: executing a filter (105), detecting an edge in a gray image (106), separating foreground and background from edge information detected in the step (110), distributing a foreground vertex Step 111: Computing the front and back three-dimensional vertices from the distributed vertices 107, creating a rear partial surface 108, and calculating the forward vertex calculated in the step 107 in the step A step 109 of re-setting the forward partial vertex using the information extracted in steps 103, 104 and 105, and a step 109 of generating a forward partial surface using the vertex information calculated in the step 109 And a step (112) of creating a final 3D model using the information generated through the steps (113) and all the steps.

A block diagram according to an embodiment of the present invention is shown in FIG.

In the present invention, the conversion of a 3D model from an input image starts with converting a color-input image into a gray image as shown in FIG. 2A.

The input image of the present invention is an image input in 2D and is input as an image of a face.

The present invention performs step 102 of transforming the color image into a gray image.

The modified gray image may be processed by a low-pass filter, or may be used for edge detection or edge detection.

The present invention performs a step (103) of detecting a face from the converted gray image.

As a method of detecting the face of the present invention, a face pattern recognition algorithm using an AdaBoost learning algorithm using a Haar-like feature is used.

[Figure 4] is a diagram showing a Haar-like feature.

The Haar-like feature sets the area of the neighboring rectangle in the search area window to obtain the sum of image brightness in the area and obtain the difference of the values.

This difference is used to distinguish the image area. For example, the human eye area has a darker pattern than the normal area of the ball, so the face is detected using this feature.

The Boosting algorithm is a teacher-learning machine learning method that creates a strong-learning machine by combining a weak-learner. In the detected face region, the positions of eyes and nose are found using the same technique. Since the detection is attempted only within the detected face area, the calculation speed is relatively fast.

The present invention performs step 104 of detecting eyes, nose, and mouth from the face detected in the above step.

The steps of detecting the eyes, nose, and mouth in the above can be performed as follows.

As shown in FIG. 5, the size of the face in the image is used as a face parameter, IPD (interpupillary distance) between the center of the two eyes detected in the image according to the distance and zoom between the camera and the subject.

The position of the eyebrow is determined based on this length.

The position of the eyebrow sets the region of interest (ROI) in the upper part of the eye and detects darker areas than the surrounding area.

The eyebrow area is segmented by calculating the histogram in the eyebrow ROI area and calculating the binarization threshold from the histogram.

The eye, nose and mouth positions are calculated by the same technique in the detected face region.

The present invention performs step 105 of performing a low pass filter on the gray image.

Since the original gray image has a lot of noise, a gray image using a low-pass filter is used. The low-pass filter uses a Gaussian filter.

Therefore, the present invention performs the above-described low-pass filter step to extract a gray image from which a large amount of noise is removed.

The present invention performs a step (106) of detecting an edge in a gray image after a step (103) of detecting a face from the gray image.

As shown in FIG. 2B, the step of detecting an edge of the present invention detects an edge through a line of bright and dark outline in the gray image.

The present invention performs step (106) of detecting the edges described above and then separating the foreground and background from the edge information detected in the step (110).

In the present invention, the position of a vertex of a model space is set in order to generate a three-dimensional model.

FIG. 6 illustrates the principle of arranging vertices at regular intervals on an image separated by foreground and generating faces using vertices.

The position of the vertex is calculated by using the x and y axes of the image coordinates as the x and y axes of the model space, and the z axes according to the relative positions of the x and y axes.

As described above, in order to detect only a required face part, the present invention needs a segmentation operation of an image and also needs to set a position of a vertex.

The process of separating the foreground and the background in the edge detection of the present invention is performed by assuming a virtual horizontal line in the face image, that is, in a line where a line starting from one edge extends to another edge, It can be performed by considering the position as the foreground part.

In the present invention, edge detection is performed as shown in FIG. 2B on the assumption that the background portion of the image uses a simple image as in the ID photograph, thereby separating the foreground and the background.

The present invention performs a step 110 of separating the foreground and background from each other and distributing a predetermined number of vertices to the segmented foreground image area.

The step of distributing the image region vertex is performed by setting a virtual horizontal line at regular intervals in the foreground image region and setting a predetermined number of vertex points for each assigned horizontal line. The resolution of the 3D model is determined by the number of the vertex points.

The present invention performs step (107) of calculating front and back three-dimensional vertices from the distributed vertices after step (111) of distributing the above-mentioned vertices.

The calculation of each of the horizontal line three-dimensional vertex positions is calculated as follows.

First, the center coordinates (centerX, centerY) of the image are calculated in each imaginary horizontal line. The vertex coordinates V (V _{x ,} V _y , V _z ) of the three-dimensional model corresponding to the coordinates (x, y) on the image space are calculated as follows.

V _x = S _x (x _y -center X)

V _y = S _y (center _Y - y) (Equation 2)

V _z = S _z r (y) cos? _X (Equation 3)

θ _x = α'θ ² _t (Equation 4)

? _t =? (centerX-x) / 2r (y)

In the above equation, θ _x is limited to the [-π, π] region.

Also, S _x , S _y , and S _z are the scaling factors and r (y) is the length of the horizontal axis image in the foreground of the image plane.

The vertex coordinates of the front part of the model are V (V _{x ,} V _y , V _z ) and the vertex coordinates of the rear part are V (V _{x ,} V _y , -V _z ).

In addition to the vertices given to the image plane, two vertices, V _n = (V _{nx ,} V _ny , V _nz ) and V _s = (V _{sx ,} V _sy , V _sz ) are calculated as follows.

Y _max represents the maximum point of the y coordinate, and y _min represents the minimum point of the y coordinate.

The present invention can perform the step 107 of calculating the front and back three-dimensional vertices and then generating the front and back faces when the vertex coordinate calculation is completed.

The present invention first performs step 108 of creating a backward (backward) surface.

The generation of the front and back faces is generated by a predetermined rule using neighboring vertices. The normal vectors of the front face and the face face are opposite to each other.

When the above steps are completed, a model having a structure in which the front direction and the back direction are symmetric is created.

The present invention performs step 108 of creating a backward plane as described above, and then the next step is to reset 109 the vertices such that the forward and backward directions are different.

That is, it performs a step of resetting the vertex of the front part in order to complete the eye conception and shape of the generated model.

The basic principle of the vertex reset of the present invention is to use the foreground image as a hight map. V _z of the front part vertex The components are reset as follows.

V _z = V _z + ΒI (x, y) / G max (Expression 12)

In the above equation,? Is a scaling factor, I (x, y) is a grayscale value on an image plane, and _Gmax is a maximum value of a grayscale value.

Β is the above-mentioned may be input to a predetermined value as the maximum value of gray level value, the scaling factor and the gray level value I (x, y) and G _max on the image plane.

That is, the vertical position of the vertex is modified according to the tone value of the image, and the front and back surfaces are set differently.

Since the original gray image has a lot of noise, a gray image using a low-pass filter is used.

The present invention uses a gray image using the low pass filter obtained through the step 105 of executing a low pass filter on the gray image as described above.

As mentioned above, it is better to use a Gaussian filter for the low-pass filter.

In the present invention, the model completed in the forward vertex resetting step has many problems because it uses the gray scale value of the image as a height map.

Especially the eyebrow area is depressed and the shape of the nose part is much different from the actual part.

Therefore, the face is detected from the foreground image, and the eye, nose, and mouth are detected and the partial correction is performed based on the detected face.

As shown in FIG. 2C, the method of detecting a face in the foreground image and detecting the eyes, nose, and mouth is as described above.

That is, as described above, the facial pattern recognition algorithm using the AdaBoost learning algorithm using Haar-like feature is used as the face detection method of the present invention.

The Haar-like feature sets the area of the neighboring rectangle in the search area window to obtain the sum of image brightness in the area and obtain the difference of the values.

This difference is used to distinguish the image area. For example, the human eye area has a darker pattern than the normal area of the ball, so the face is detected using this feature.

The Boosting algorithm is a teacher-learning machine learning method that creates a strong-learning machine by combining a weak-learner. In the detected face region, the positions of eyes and nose are found using the same technique. Since the detection is attempted only within the detected face area, the calculation speed is relatively fast.

The interpupillary distance (IPD) between the centers of the two eyes detected above is used as a face parameter.

The position of the eyebrow is determined based on this length.

The position of the eyebrow sets the region of interest (ROI) in the upper part of the eye and detects darker areas than the surrounding area.

The eyebrow area is segmented by calculating the histogram in the eyebrow ROI area and calculating the binarization threshold from the histogram.

The eyebrow area is depressed in the 3D model because the gray level value is low.

V _z = V _z +? r _e (x) cos (?) (Expression 13)

? _y =? (centerY-y) / 2r (x)

r _e (x) denotes the length of the eyebrow segment region in the y-axis direction at the x-position in the eyebrow region, and y is the scaling coefficient.

centerY means the center position of the eyebone segment region on the y axis. The vertex was modified to restore the undulating eyebrows naturally.

If the vertex of the nose area is calculated only by the tone value, it is not natural, and the correction position is detected and an arbitrary reset process is performed. Calculate the center between the left and right eyes, find the position of the nose, and select the correct position between the two points.

V _z = V _z +? S _n (x) n (x) cos (? _n ) (Expression 15)

? _n =? (centerX-x) / 2n (x)

Where n (x) is the base length of the nose at x, S _n (x) is the scaling factor and centerX is the center value of the nose. n (x) and S _n (x) should be statistically optimal values according to race and gender age.

FIG. 2 shows the results according to the embodiment of the present invention.

That is, [Figure 2a] converts a front photograph of a person into a monochrome (gray) image,

FIG. 2B shows an edge image extracted from a monochrome (gray) image,

FIG. 2C shows an image processing of eye, eyebrow, nose, and IPD centers as shown in (c)

[Fig. 2 (d)] is a diagram showing that the eyebrows and the nose portions are corrected and corrected by using the equations (13) to (16).

FIG. 3A shows that eyebrows and nose portions are depressed when eyebrows and nose regions are not processed, and a very awkward model is created.

[Figure 3b] shows a result of treating eyebrows and nose portions more naturally.

(A) V236, (b) V5018, (c) V46958, and (d) v226094 in FIG. 3B represent the number of vertices and show the result of generating the 3D model.

Therefore, the larger the number of vertices, the more sophisticated 3D model creation results are shown.

The present invention performs step 109 of resetting the vertices described above and then generating 113 the front partial surface using the vertex information calculated in step 109 of resetting the vertex.

The present invention performs a step 112 of creating a final 3D model using the information generated through all the steps described above.

The present invention provides a method for automatically generating a 3D model from a 2D face image composed of the above-described structure and function.

INDUSTRIAL APPLICABILITY The present invention is an extremely useful invention for an industry that produces, manufactures, and distributes software or hardware for modeling in three dimensions.

In particular, the present invention is a particularly useful invention for such a universal modeling industry as the modeling industry and 3D printer for the general public, especially for younger students, are becoming widespread in the home.

Claims (3)

A method for automatically generating a 3D model from a 2D face image,
Converting the color image into a gray image (102)
Performing a step (103) of detecting a face from the converted gray image from the step,
As a method of performing the face detection, a face pattern recognition algorithm using an AdaBoost learning algorithm using a Haar-like feature is used,
Detecting (104) eye coincidence on the detected face,
Performing (105) a low pass filter on the gray image,
Detecting (106) an edge in the gray image,
(110) separating foreground and background from the edge information detected in the step,
Distributing a foreground vertex to a separate foreground image (111),
Computing (107) front and back three-dimensional vertices from the distributed vertices,
Creating (108) a rear partial surface,
A step (109) of resetting the front partial vertex using the information extracted in the steps (103, 104, and 105) from the front vertex calculated in the step (107)
A step 113 of generating a front partial face using the vertex information calculated in the step 109 and a step 112 of creating a final 3D model using the information generated through all the steps A method for automatically generating a 3D model from an image.
The method according to claim 1,
Calculating (107) the forward and backward three-dimensional vertices from the distributed vertices,
Calculating the center coordinates (centerX, centerY) of the image in a horizontal line, each of the virtual and image-space coordinates (x, y) for calculating vertex coordinates _{V (V x, V y,} V z) of the three-dimensional model corresponding to the on The method of claim 1, wherein the 3D model is generated from a 2D face image.
3. The method according to any one of claims 1 to 2,
The step 109 of resetting the forward partial vertex comprises:
V _z = V _z + ΒI method for automatically generating a 3D model from the 2D face image characterized in that the (x, y) / G Formula reset operation to the _max.

Images