JP2003030684A - Face three-dimensional computer graphic generation method and device, face three-dimensional computer graphic generation program and storage medium storing face three-dimensional computer graphic generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate the generation of face three-dimensional CG based on the appearance information of an object. SOLUTION: In this invention, by setting shape control candidate points at a fixed interval on the surface of a basic face three-dimensional shape and changing the arrangement of the shape control candidate points on the basis of the three-dimensional arrangement information of face detailed structure feature points and the three-dimensional arrangement information of the shape control candidate points, the surface of the basic face three-dimensional shape is changed and a detailed face three-dimensional shape is restored.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face three-dimensional CG generation method and device, a face three-dimensional CG generation program, and a face three.
The present invention relates to a storage medium that stores a three-dimensional CG generation program.
The present invention relates to a face three-dimensional CG generation method and device, a face three-dimensional CG generation program, and a storage medium storing the face three-dimensional CG generation program, which are capable of being expressed in three-dimensional CG based on the appearance information of the person.

2. Description of the Related Art Conventionally, "3D MeNow" (http://www.bio
virtual.com/.2001.06.01) and "Face surface model for animation production" (Komatsu, "Face surface model for animation production", IPSJ Transactions, Vol.3
0, No. 5, pp. 633-641, 1989), there is a method of generating a three-dimensional character having a shape similar to an actual head by using a captured face image.

In the conventional three-dimensional character generation method, a basic three-dimensional shape model of the head is prepared in advance, and the details of the basic model are changed based on the personal facial features obtained from the photographed face image. The mainstream is to customize a three-dimensional shape model and then attach a processed image to a captured image to the customized model.

There has also been proposed a method in which a curved surface that passes through all face feature points obtained from a face image is generated without using a basic three-dimensional shape model to obtain a face shape.

[0004]

However, in the above-mentioned conventional method, since the detail is expressed, the three-dimensional shape model becomes complicated, and it is necessary to extract a large number of facial feature points from the image. Therefore, the calculation process required for automatic generation increases, and the time and effort of the user's own operation increases,
Such a problem also occurs.

Further, since the processed image is attached to the photographed image, it is suitable for restoration faithful to the actual head shape, but is not suitable for an abstract expression such as emphasizing a characteristic face portion. .

The present invention has been made in view of the above points, and stores a face three-dimensional CG generation method and device, a face three-dimensional CG generation program, and a face three-dimensional CG generation program, which reflect the appearance information of the subject. The purpose is to provide a storage medium.

[0007]

FIG. 1 is a diagram for explaining the principle of the present invention.

According to the present invention (Claim 1), a three-dimensional CG of a face of a subject in a virtual space is calculated based on appearance information of the subject.
In the face three-dimensional CG generation method represented by, the external contours of the side face and the front face and the edges of each face part area are extracted from the photographing information obtained by photographing the target person's head from two directions, the front and the side (step. 1) Obtain two-dimensional arrangement information of face feature points on front and side photographed images (step 2), and calculate three-dimensional arrangement information of face feature points based on the two-dimensional arrangement information of face feature points. (Step 3), the face feature points are classified into face basic structure feature points and face detailed structure feature points (step 4),
Based on the three-dimensional arrangement information of the face basic structure feature points, the three-dimensional structures of the head skeleton, the nose, and the mouth are respectively restored by the three-dimensional geometrical figures represented by the surface composed of the elliptic curve, the straight line, and each line (step 5). ), The restored head skeleton, nose, and mouth are integrated to restore the basic face three-dimensional shape (step 6), and on the surface of the basic face three-dimensional shape,
Shape control candidate points are set at regular intervals (step 7), and three-dimensional arrangement information of face detailed structure feature points and shape control candidate points 3 are set.
By changing the arrangement of the shape control candidate points based on the three-dimensional arrangement information, the surface of the basic face three-dimensional shape is changed to restore the detailed face three-dimensional shape (step 8) and the two-dimensional face feature points are calculated. Based on the placement information, the skin, eyebrows, eyes, and mouth are each approximated by a plurality of simple two-dimensional geometric figures, and the two-dimensional geometric figures are integrated to restore the two-dimensional structure of the skin, eyebrows, eyes, and mouth. Then, (step 9), the restored skin, eyebrows, and eyes are attached to the surface of the restored detailed face three-dimensional shape (step 10).

According to the present invention (claim 2), after the basic face three-dimensional shape is restored, changes are made to the three-dimensional geometric figure, and the detailed face three-dimensional shape and the skin, eyebrows and eyes are changed according to the result. , Gives a change in the reconstruction of the two-dimensional structure of the mouth.

FIG. 2 is a block diagram showing the principle of the present invention.

The present invention (Claim 3) is connected to a photographing device for photographing a subject directly or via a network, and the face of the subject in the virtual space is based on the appearance information of the subject. A face three-dimensional CG generation device represented by a three-dimensional CG, wherein the outlines of the side face and the front face and the edges of each face part region are obtained based on the photographing information obtained by photographing the head of the subject from two directions, the front and the side. Is extracted, two-dimensional arrangement information of the face feature points on the front and side photographed images is acquired, and 2 of the face feature points is acquired.
Based on the three-dimensional arrangement information, the three-dimensional arrangement information of the face feature points is calculated, and the face feature points are classified into a face basic structure feature point and a face detailed structure feature point, and a face basic structure feature. Face part three-dimensional shape restoring means for respectively restoring the three-dimensional structure of the head skeleton, nose, and mouth with a three-dimensional geometrical figure represented by a surface composed of an elliptic curve, a straight line, and each line based on the three-dimensional arrangement information of points. 130 and the restored head skeleton,
A face basic three-dimensional shape restoring unit 140 that restores a basic face three-dimensional shape by integrating the nose and mouth, and shape control candidate points are set at regular intervals on the surface of the basic face three-dimensional shape, By changing the arrangement of the shape control candidate points based on the three-dimensional arrangement information of the detailed structure feature points and the three-dimensional arrangement information of the shape control candidate points, the surface of the basic face three-dimensional shape is changed, and the detailed face 3 Based on the face detail three-dimensional shape restoring means 150 for restoring the three-dimensional shape and the two-dimensional arrangement information of the facial feature points, the skin, the eyebrows, the eyes, and the mouth are approximated by a plurality of simple two-dimensional geometric figures, and two-dimensional A two-dimensional structure restoring unit 160 that restores the two-dimensional structure of the skin, eyebrows, eyes, and mouth by integrating the geometric figures, and the surface of the detailed face three-dimensional shape restored by the face detailed three-dimensional shape restoring unit, Two
And a sticking unit 170 for sticking the skin, eyebrows, and eyes restored by the three-dimensional structure restoring unit.

According to the present invention (claim 4), after the basic face three-dimensional shape is restored, changes are made to the three-dimensional geometric figure, and the detailed face three-dimensional shape and the skin, eyebrows and eyes are changed according to the result. , And further includes modification means for modifying the reconstruction of the two-dimensional structure of the mouth.

The present invention (Claim 5) is connected to a photographing device for photographing a target person directly or via a network, and determines the face of the target person in a virtual space based on the appearance information of the target person. A face three-dimensional CG generation program to be executed by a face three-dimensional CG generation device expressed by three-dimensional CG, which has a program for executing the face three-dimensional CG generation method according to claim 1 or 2.

The present invention (Claim 6) is connected to a photographing device for photographing a target person directly or via a network, and the face of the target person in a virtual space is based on appearance information of the target person. A storage medium storing a face three-dimensional CG generation program to be executed by a face three-dimensional CG generation device expressed by three-dimensional CG, the program causing the face three-dimensional CG generation method according to claim 1 or 2 to be executed. To store.

As described above, according to the present invention, one or more computer terminals (face three-dimensional CG generation device) and one or more image capturing devices such as cameras connected to the computer terminals are included. The user performs simple input to the captured image obtained by the image capturing device to automatically extract the minimum facial feature amount necessary for individual identification, and a simple geometric figure based on the facial feature amount is used to extract the head. The basic three-dimensional shape of the face composed of partial facial parts is restored, the detailed three-dimensional shape of the face is restored by controlling the curved surface, and the characteristic facial parts among the facial parts are further emphasized. Therefore, it becomes possible to automatically generate a face three-dimensional CG with distinctive personality by changing the location, scale, shape, etc. of the face part.

The conventional method does not restore the appearance information faithfully and finely, but a stepwise restoration method such as a face basic three-dimensional shape restoration by an approximate geometric figure and a face detailed three-dimensional shape restoration by curved surface control, and , The basic three-dimensional shape of the face is different in that the expression is made more distinctive by adding changes in scale and shape.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION First, an apparatus and system configuration for carrying out the present invention will be described.

FIG. 3 shows the configuration of a face three-dimensional CG generation system according to an embodiment of the present invention.

In FIG. 1, the user terminal 1 and one or a plurality of photographing devices 2 are directly connected to each other, or the network 3 is connected.
Connected through.

The user terminal 1 is a personal computer equipped with a mouse, a keyboard, etc., and is connected to the photographing device 2 and the network 3 as required. Alternatively, a camera-installed multimedia personal computer having the photographing device 2 built therein may be used.

After transmitting the operation request input from the user to the photographing device 2, the user terminal 1 receives the photographing information and saves it in the storage device in its own device.

When an operation request is input from the user, the user terminal 1 displays the photographing information by the output means, and according to the designated point information input from the next moving input device such as a mouse operated by the user. , After performing face part two-dimensional layout information automatic extraction processing and then face three-dimensional shape restoration processing,
The output means draws the result in three-dimensional CG. When the user terminal device 1 draws a three-dimensional CG by the output means and if there is an editing request from the user, the user terminal device 1 changes the three-dimensional shape of the face in accordance with the input from the mouse or keyboard operated by the user. Redraw.

The photographing device 2 comprises a video camera, a scanner, etc., and if necessary, the user terminal 1 and the network 3
It is connected to the.

The photographing device 2 transmits a photographing request from the user terminal 1, photographs the target person 4, and then transmits photographing information to the user terminal 1.

FIG. 3 shows the configuration of the user terminal according to the embodiment of the present invention.

The user terminal 1 has an input / output interface 1
10, feature point extraction unit 120, face part three-dimensional shape restoration unit 1
30, a face basic three-dimensional shape restoration unit 140, a face detailed three-dimensional shape restoration unit 150, a texture creation unit 160, a texture pasting unit 170, and an emphasis / change unit 180.

The input / output interface 110 inputs / outputs data to / from the photographing device 2, an input device such as a mouse and a keyboard, and an output device such as a display.

The feature point extraction unit 120 extracts the outlines of the side face and the front face and the area of each face part from the photographing information of the head of the subject input from the photographing device 2 from the two directions of the front and the side. Edges of the face feature points are acquired, two-dimensional arrangement information of face feature points on the front and side captured images is acquired, and three-dimensional arrangement information of the face feature points is calculated based on the two-dimensional arrangement information. Are classified into face basic structure feature points and face detailed feature points.

The face part three-dimensional shape restoring unit 130 uses a three-dimensional geometric figure represented by a surface consisting of an elliptic curve, a straight line, and each line based on the three-dimensional arrangement information of the face basic structure feature points to form the head skeleton and nose. , The three-dimensional structure of the mouth is restored, and the restored head skeleton, nose, and mouth are integrated to restore the basic face three-dimensional shape.

The face basic three-dimensional shape restoration unit 140 restores the basic face three-dimensional shape by integrating the head skeleton, the nose and the mouth restored by the face part three-dimensional shape restoration unit 130.

The face detail three-dimensional shape restoring unit 150, the three-dimensional arrangement information of the face detail structure feature points, and the three-dimensional arrangement of the shape control candidate points set at regular intervals on the surface of the dull face three-dimensional shape. By changing the arrangement of the shape control candidate points based on the information, the surface of the basic face three-dimensional shape is changed to restore the detailed face three-dimensional shape.

The texture creating unit 160 uses the two facial feature points.
By approximating each of the skin, eyebrows, eyes, and mouth with a plurality of simple two-dimensional geometric figures based on the three-dimensional arrangement information and integrating the two-dimensional geometric figures, the two-dimensional structure of the skin, eyebrows, eyes, and mouth is obtained. Restore.

The texture attaching section 170 attaches the skin, eyebrows, and eyes restored by the texture creating section 160 to the surface of the detailed face three-dimensional shape restored by the detailed face three-dimensional shape restoring section 150.

The emphasizing / changing unit 180 restores the basic face three-dimensional shape by the face part three-dimensional shape restoring unit 130, then changes the three-dimensional geometric figure, and according to the result, the detailed face three-dimensional shape is changed. Change the shape and the reconstruction of the two-dimensional structure of the skin, eyebrows, eyes, and mouth to more prominently express the three-dimensional facial features of the subject.

Next, the operation of the above configuration will be described.

FIG. 5 is a flowchart of the operation in the embodiment of the present invention.

Step 101) The side face and the front face of the subject 4 are photographed using the photographing device 2 and the photographing information is transmitted to the user terminal 1.

Step 102) The input / output interface 110 of the user terminal 1 receives the photographing information from the photographing device 2 and acquires the designation of the input point from the mouse.

Step 103) The feature point extraction unit 120 of the user terminal 1 extracts the outer contours of the side face and the front face and the edges of each face part area from the photographed image acquired via the input / output interface unit 110. , The two-dimensional arrangement information of the facial feature points on the front and side captured images is acquired,
Three-dimensional arrangement information of face feature points is calculated based on the dimension arrangement information, and the face feature points are classified into face basic structure feature points and face detailed structure feature points.

Step 104) In the face part three-dimensional shape restoration unit 130 of the user terminal 1, based on the three-dimensional arrangement information of the face basic feature points obtained in step 103, a face composed of an elliptic curve, a straight line and each line is used. The head skeleton, the nose, and the three-dimensional structure of the mouth are reconstructed by the represented three-dimensional geometric figures.

Step 105) The face basic three-dimensional shape restoring unit 140 restores the basic face three-dimensional shape by integrating the head skeleton, the nose and the mouth restored in step 104.

Step 106) The face detail three-dimensional shape restoration unit 150 sets shape control candidate points at regular intervals on the surface of the basic face three-dimensional shape, and sets the face detail structure feature points to three.
By changing the arrangement of the shape control candidate points based on the three-dimensional arrangement information of the shape control candidate points and the three-dimensional arrangement information, the surface of the basic face three-dimensional shape is changed to restore the detailed face three-dimensional shape.

Step 107) Texture creating section 16
0, the skin based on the facial feature point two-dimensional arrangement information,
By approximating each of the eyebrows, eyes, and mouth with a plurality of simple two-dimensional geometric figures and integrating the two-dimensional geometric figures, skin,
Restore the two-dimensional structure of eyebrows, eyes, and mouth.

Step 108) Texture attaching section 17
0 pastes the restored skin, eyebrows, and eyes on the surface of the restored detailed face three-dimensional shape.

Step 109) Emphasis / change unit 180
Determines whether or not to emphasize a characteristic face part among the above face parts. If it is emphasized, the process proceeds to step 110, and if not emphasized, the process ends.

Step 110) If the image is emphasized by the user's operation, the process proceeds to step 111, and if not, the process proceeds to step 112.

Step 111) Emphasis / change unit 180
Changes the size by the user and moves to step 104.

Step 112) The emphasizing / changing unit 180 makes a comparison with the average face, and shifts to step 104.

[0049]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 6 shows an example of photographing a side face according to one embodiment of the present invention, and FIG. 7 shows an example of photographing a front face according to one embodiment of the present invention.

(1) First, the user inputs a photographing request to the user terminal 1 with a mouse, a keyboard or the like.

(2) As a result, the user terminal 1 transmits the photographing request to the photographing device 2. Shooting equipment 2
Takes a side face 5 and a front face 6 of the subject 4 as shown in FIGS. 6 and 7. Here, when the number of the photographing device 2 is one, the subject 4 or the photographing is separately performed while moving the photographing device 2. However, when the number of the photographing devices 2 is two, as shown in FIG. It may be installed in the direction of each side and front, and the images may be taken simultaneously.

(3) Next, facial feature points are acquired from the photographed information by the following method, for example.

Photographing information of the side face 5 and the front face 6 displayed on the output screen of the user terminal 1 includes the eyes of the side face and the upper and lower end points of the vertical axis of the head as shown in FIG. 4 points, both eyes on the front face, and 3 points on the mouth are designated as input points. According to the input information, the three points of the side face 5, the front face 6, both eyes, and the mouth are designated as input points. According to the input information, the face outlines of the side face 5 and the front face 6,
Edge information of face parts is automatically extracted, and fixed point groups (eyebrows, eyes, top of nose, top of lips, center of lips, top of lower lip, center of chin, which are present on outer edges of side faces are automatically extracted as shown in FIG. 10. , Chin tip) and fixed points existing on the outer edge of the front face 6 (eyebrows, eyes,
Fixed points existing on the top of the nose, the bottom of the nose, the top of the upper lip, the center of the lip, the top of the lower lip, the center of the chin, the tip of the chin, and the edges of the face part area (upper and lower left and right end points of the eyebrow and eye areas, the nose and mouth area) (Each of the left and right end points of) is acquired as the two-dimensional arrangement information of the facial feature points 7. The face feature points 7 are classified into a face basic structure feature point 8 and a face detailed structure feature point 9, respectively, as shown in FIGS.

(4) Next, the three-dimensional shape of the head skeleton, nose, and mouth is restored by the following method, for example.

FIG. 13 shows an example of restoring the three-dimensional structure of the head skeleton according to one embodiment of the present invention.

The three-dimensional shape of the head skeleton 10 is horizontally sliced and divided, and the slice position is determined by the three-dimensional arrangement information of the face basic structure feature points 8 and the contour shape at each slice position. Is drawn by a straight line passing through the face basic structure feature point 8 and an elliptic curve, as shown in FIG. Further, the cross section at each slice position is approximated by a semi-ellipse whose horizontal axis is the long axis and short axis is the horizontal axis of each of the side face 25 and the front face 26 as shown in FIG. The three-dimensional shape of the head skeleton 10 is a figure that complements the surface formed by the straight line and the elliptic curve at these sliced positions.

FIG. 15 shows an example of restoring the three-dimensional structure of the mouth according to one embodiment of the present invention.

The three-dimensional shape of the nose 11 is shown in FIG. 15 based on the three-dimensional arrangement information of the face basic structure feature points 8 constituting the nose part.
As shown in, a contour shape including a straight line and an elliptic curve is defined, and a figure that complements the surface formed by the straight line and the elliptic curve is the three-dimensional shape of the nose 11.

FIG. 16 shows an example of restoring the three-dimensional structure of the mouth according to the embodiment of the present invention.

The three-dimensional shape of the mouth 12 is also shown in FIG. 1 based on the three-dimensional arrangement information of the face basic structure feature points 8 constituting the mouth part.
As shown in FIG. 6, a contour shape consisting of a straight line and an elliptic curve is defined, and a figure that complements the surface formed by the straight line and the elliptic curve is drawn.
3D shape.

(5) Next, the face basic three-dimensional shape and the face detailed three-dimensional shape are restored by the following method, for example.

FIG. 17 shows an example of a face basic three-dimensional shape and shape control candidate points according to one embodiment of the present invention, and FIG. 18 shows a three-dimensional array of face detailed structure feature points according to one embodiment of the present invention. Here is an example.

By integrating the three-dimensional shapes of the head skeleton 10, the nose 11, and the mouth 12, the basic face three-dimensional shape 13 is restored, and as shown in FIG. The existing shape control candidate points 15 are determined. Of these shape control points 15, the point closest to the face detailed structure feature point 9 shown in FIG. 18 is defined as a shape control point 16, and its three-dimensional coordinates are moved to the three-dimensional position of the face detail structure feature point. Let Further, the shape control candidate points 15 other than the shape control point 16
Is changed to a shape approximate point 17 by changing the position according to the three-dimensional distance from the face detailed structure feature point 9. FIG. 19 shows an example of changing the position of the shape control candidate point according to the embodiment of the present invention. here,
To determine the changed position of the approximate shape point 17, the three-dimensional B-
It is also possible to use an approximation method using a sub-line curve or a Bezier curve. A face passing through the face basic structure feature point 8, the shape control point 16, and the shape approximation point 17 is defined as a detailed face shape 14. FIG. 20 shows an example of restoring the detailed three-dimensional shape of the face.

(6) Next, the two-dimensional textures of the skin, eyebrows and eyes are created by the following method.

FIG. 21 shows an example of generating a two-dimensional texture according to an embodiment of the present invention.

The skin, eyebrows, and eyes cover the surface of the three-dimensional shape.
As shown in FIG. 21, the skin part is defined as a uniform average color, and the eyes and eyebrows are defined by elliptic curves as shown in FIG. Two-dimensional texture 1 created
8 is attached to the surface of the face detailed three-dimensional shape 14 and is represented as a face three-dimensional CG 19 as shown in FIG.

(17) Finally, the individuality is emphasized by the following method.

FIG. 23 shows an example of the arrangement of facial feature points of average values according to an embodiment of the present invention.

The emphasis of individuality is based on a comparison with an average facial feature point 20 obtained by averaging the three-dimensional arrangement information of a plurality of target facial feature points 7 as shown in FIG. 23, or by the user himself. Head skeleton 1 that forms the basic three-dimensional shape 13 of the face
This is performed by changing the shapes of 0, the nose 11, and the nose 12 to more prominently express the three-dimensional facial features of the subject 4. For example, the nose 12 is emphasized by enlarging or reducing the scale of the nose 12. FIG. 24 shows an example of highlighting the enlarged nose 21 and the reduced nose 22 of the nose 12.

Although the above embodiment has been described based on the operation shown in FIG. 5, the operation shown in FIG. 5 is constructed as a program and installed in the CPU of the computer used as the user terminal, or It can also be distributed via a network.

Further, the constructed program is connected to a computer used as a user terminal, a hard disk device, a floppy (registered trademark) disk, a CD-
The present invention can be easily realized by storing it in a portable storage medium such as a ROM and installing it when carrying out the present invention.

The present invention is not limited to the above embodiment, but various modifications and applications are possible within the scope of the claims.

[0074]

As described above, according to the present invention, the shape control candidate points are set at regular intervals on the surface of the basic face three-dimensional shape, and the three-dimensional arrangement information of the face detailed structure feature points and the shape control candidate points are set. By changing the arrangement of the shape control candidate points based on the three-dimensional arrangement information of No. 3, the surface of the basic face three-dimensional shape is changed, and the detailed face three-dimensional shape is restored. A face-based three-dimensional CG can be simply and automatically generated.

Further, by emphasizing the facial features, it is possible to realize the creation of a face three-dimensional CG with more distinctiveness.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a system configuration diagram according to an embodiment of the present invention.

FIG. 4 is a configuration diagram of a user terminal according to the embodiment of the present invention.

FIG. 5 is a flowchart of the operation in the embodiment of the present invention.

FIG. 6 is an example of photographing a side face according to an embodiment of the present invention.

FIG. 7 is an example of photographing a front face according to an embodiment of the present invention.

FIG. 8 is an example of simultaneous photographing of a side face and a front face according to an embodiment of the present invention.

FIG. 9 is an example of input point designation by a user according to an embodiment of the present invention.

FIG. 10 is an example of extracting facial feature points according to an embodiment of the present invention.

FIG. 11 is an example of facial basic structure feature points according to an embodiment of the present invention.

FIG. 12 is an example of face detailed structure feature points according to an embodiment of the present invention.

FIG. 13 is an example of restoring a three-dimensional structure of a head skeleton according to an embodiment of the present invention.

FIG. 14 is a diagram for explaining an approximation of a semi-ellipse using face width information according to an embodiment of the present invention.

FIG. 15 is an example of restoring the three-dimensional structure of the nose according to the embodiment of the present invention.

FIG. 16 is an example of restoring the three-dimensional structure of the mouth according to the embodiment of the present invention.

FIG. 17 is an example of a basic face three-dimensional shape and shape control candidate points according to an embodiment of the present invention.

FIG. 18: 3 of the vertices for solving the face detailed structure according to the embodiment of the present invention
It is an example of dimension arrangement.

FIG. 19 is an example of changing the position of the shape control candidate point according to the embodiment of the present invention.

FIG. 20 is an example of restoration of a face detailed three-dimensional shape according to an embodiment of the present invention.

FIG. 21 is an example of generating a two-dimensional texture according to an embodiment of the present invention.

FIG. 22 is an example of generating a three-dimensional face CG according to an embodiment of the present invention.

FIG. 23 is an example of arranging facial feature points of an average face according to an embodiment of the present invention.

FIG. 24 is an example of an emphasized expression of the nose according to the embodiment of the present invention.

[Explanation of symbols]

1 User Terminal 2 Imaging Device 3 Network 4 Target 5 Side Face 6 Front Face 7 Face Feature Point 8 Basic Basic Structure Feature Point 9 Face Detailed Structure Feature Point 10 Head Skeleton 11 Nose 12 Mouth 14 Face Detail 3D Shape 15 Shape Control candidate point 16 Shape control point 17 Shape approximation point 18 Two-dimensional texture 19 Face three-dimensional CG 20 Average face feature point 21 Expanded nose 22 Reduced nose 25 Side face 26 Front face 110 Input / output interface 120 Feature point extraction means, feature Point extraction unit 130 Face part 3D shape restoration unit, face part 3D shape restoration unit 140 Face basic 3D shape restoration unit, Face basic 3D shape restoration unit 150 Face detail 3D shape restoration unit, Face detail 3D shape restoration Part 160 two-dimensional structure restoring means, texture creating part 170 pasting means, texture pasting part 180 highlighting / changing part

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 2F065 AA53 BB05 CC16 FF04 JJ03                       JJ26 QQ31 SS02                 5B050 AA10 BA08 BA09 BA12 EA05                       EA12 EA13 EA19 EA24 EA28                       FA02                 5B057 AA20 CA12 CB13 CC02 CD14                       CE08 CH01 DA07 DA16 DB03                       DC05 DC16                 5L096 AA09 BA20 CA05 DA01 FA06

Claims (6)

[Claims] 1. A face three-dimensional CG generation method for expressing a face of a subject in a virtual space by three-dimensional computer graphics (hereinafter, referred to as CG) based on appearance information of the subject. Two-dimensional facial feature points on the front and side captured images are extracted by extracting the outer contours of the side face and front face and the edges of each face part area from the captured information obtained by capturing the head of the person from two directions, front and side. Obtaining arrangement information, calculating three-dimensional arrangement information of the face feature points based on the two-dimensional arrangement information of the face feature points, and defining the face feature points as a face basic structure feature point and a face detailed structure feature point. 3 is represented by a surface composed of an elliptic curve, a straight line, and each line based on the three-dimensional arrangement information of the face basic structure feature points.
The three-dimensional structures of the head skeleton, nose, and mouth are respectively restored by three-dimensional geometric figures, and the restored head skeleton, nose, and mouth are integrated to restore a basic face three-dimensional shape. Shape control candidate points are set at regular intervals on the surface of the three-dimensional shape, and the shape control candidate points are arranged based on the three-dimensional arrangement information of the face detailed structure feature points and the shape control candidate points. By changing the surface of the basic face three-dimensional shape to restore the detailed face three-dimensional shape, and based on the two-dimensional arrangement information of the facial feature points, skin, eyebrows,
By approximating each of the eyes and mouth with a plurality of simple two-dimensional geometric figures and integrating the two-dimensional geometric figures, skin, eyebrows,
A face three-dimensional CG characterized by restoring the two-dimensional structure of eyes and mouth and sticking the restored skin, eyebrows and eyes on the surface of the restored detailed face three-dimensional shape
Generation method. 2. After restoring the three-dimensional shape of the basic face, 3
The face three-dimensional CG generation according to claim 1, wherein a change is applied to the three-dimensional geometric figure, and the detailed face three-dimensional shape and the two-dimensional structure of the skin, eyebrows, eyes, and mouth are changed according to the result. Method. 3. A face which is connected to a photographing device for photographing a target person directly or via a network and expresses the face of the target person in a virtual space by a three-dimensional CG based on appearance information of the target person. A three-dimensional CG generation device, which extracts outer contours of a side face and a front face and edges of each face part region from photographing information obtained by photographing a subject's head from two directions, the front and the side. The two-dimensional arrangement information of the face feature points on the captured image of is acquired, the three-dimensional arrangement information of the face feature points is calculated based on the two-dimensional arrangement information of the face feature points, and Structure feature points, feature point extracting means for classifying face detailed structure feature points, and 3D arrangement information of the face basic structure feature points, which is represented by a surface composed of an elliptic curve, a straight line, and each line 3
A three-dimensional shape of the face, a nose, and a mouth, which are respectively restored by a three-dimensional geometric figure, and a face part three-dimensional shape restoration means, and the restored head skeleton, nose, and mouth are integrated to form a three-dimensional basic face. Face basic three-dimensional shape restoring means for restoring the face detailed three-dimensional shape, and set shape control candidate points at regular intervals on the surface of the basic face three-dimensional shape, and three-dimensional arrangement information of the face detailed structure feature points and shape control candidate points. The surface of the basic face three-dimensional shape is changed by changing the arrangement of the shape control candidate points based on the three-dimensional arrangement information of the detailed face 3
A face detail three-dimensional shape restoring means for restoring a three-dimensional shape, and skin, eyebrows, based on the two-dimensional arrangement information of the face feature points,
By approximating each of the eyes and mouth with a plurality of simple two-dimensional geometric figures and integrating the two-dimensional geometric figures, skin, eyebrows,
Two-dimensional structure restoring means for restoring the two-dimensional structure of eyes and mouth, and skin restored by the two-dimensional structure restoring means on the surface of the detailed face three-dimensional shape restored by the face detailed three-dimensional shape restoring means , A face three-dimensional CG generation device comprising: an eyebrow and a sticking means for sticking eyes. 4. After the basic face three-dimensional shape is restored, 3
4. The changing means for changing the three-dimensional geometric figure and changing the three-dimensional shape of the detailed face and the two-dimensional structure of the skin, eyebrows, eyes and mouth according to the change. Face 3D CG generator. 5. A face that is connected to a photographing device for photographing a target person directly or via a network and expresses the face of the target person in a virtual space by three-dimensional CG based on appearance information of the target person. Face 3D C to be executed by the 3D CG generator
A face generation three-dimensional CG according to claim 1 or 2, which is a G generation program.
A face three-dimensional CG generation program having a program for executing the generation method. 6. A face which is connected to a photographing device for photographing a target person directly or via a network and which expresses the face of the target person in a virtual space by a three-dimensional CG based on appearance information of the target person. Face 3D C to be executed by the 3D CG generator
A storage medium storing a G generation program, the face three-dimensional CG according to claim 1 or claim 2.
A storage medium storing a face three-dimensional CG generation program, which stores a program for executing the generation method.