JP2005346604A - Face image expression change processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: it is necessary to move or deform each portion of a face, so that processing is large. <P>SOLUTION: This face image expression change processor has: an expression change template data storage means; a face area detection means extracting face area data from a static image; a polygon data generation means obtaining characteristic point coordinate data on an eye center or the like from the face area data and expression change template data to determine a polygon dividing line segment, and generating a plurality of pieces of polygon data wherein a rectangular plane is divided according to the polygon dividing line segment; a polygon vertex perpendicular movement means moving a vertex of the polygon data perpendicularly to the plane; a polygon data association means associating each portion of static image data to each the plane of the polygon data; and an image data generation means performing a rotation process to a polygon to generate image data. Thereby, the face image expression change processor performs an expression change to a face image with a small calculation amount. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an apparatus for dynamically changing a facial expression in a still image and displaying it on a display device with simple means.

In order to change the facial expression in a still image, for example, as disclosed in Japanese Patent Application Laid-Open No. 11-167626 “Image Processing Method, Recording Medium, and Image Processing Device”, facial eyes, eyebrows, nose, mouth, etc. Is identified as region information, and facial expression changes are obtained by moving, transforming, and interpolating each region of the still image data according to the facial expression to be changed.
JP 11-167626 A

  In order to move, transform, and interpolate each facial region on still image data, the amount of calculation in the CPU is large, so that the required processing time cannot be satisfied with devices with low CPU processing capacity. Have the potential to cause

  There is also a means to reduce the amount of calculation in the CPU by processing the movement, deformation and interpolation for each part region of the face using a graphics arithmetic unit, but to change the expression dynamically Since it is necessary to repeatedly calculate the amount of movement and the amount of deformation for each region of the face, it takes time and effort.

  In order to solve the above-described problems, a facial image expression processing apparatus according to the present invention includes a 3D graphic operation processing unit that performs 3D graphic operation processing, a still image data storage unit that stores still image data, and an expression change. Expression change template data storage means for storing template data, face area detection means for extracting face area data from the still image, face ends, eye center, mouth center, etc. from the face area data and the expression change template data Polygon data generating means for obtaining feature point coordinate data, thereby determining polygon dividing line segments and generating a plurality of polygon data by dividing a rectangular plane according to the polygon dividing line segments, and dividing the polygon according to the expression change template data The vertex of the polygon data on the line segment is perpendicular to the plane A moving polygon vertex vertical moving means, a polygon data associating means for associating a portion of the still image data with texture data and corresponding to each plane of the polygon data, and a 3D graphic calculation processing means for light source processing and expression change Image data generating means for generating image data by executing a rotation process for giving an inclination to a polygon obtained according to template data, and display means for displaying the image data generated by the image data generating means It is said.

  With the above configuration, it is possible to perform facial expression changes on a still face image with a smaller amount of calculation in the CPU, and it is possible to avoid a situation where the processing time required due to insufficient processing capacity of the CPU cannot be satisfied.

  In addition, it is possible to avoid the necessity of repeatedly performing calculations such as the movement amount and deformation amount for each part region of the face.

  According to the present invention, it is not necessary to move, transform, and interpolate each part of the face on the still image data, and the face in the still image that uses the illusion effect of human eyes with a smaller amount of calculation in the CPU. It is possible to obtain an image in which the facial expression is changed.

(Configuration of the embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram of a facial image expression change processing device according to an embodiment of the present invention.

  In the facial image expression change processing device 100, the 3D graphic calculation processing means 101 performs three-dimensional graphic calculation processing.

  Still image data storage means 102 stores still image data including a person (face). The expression change template data storage means 103 stores a plurality of expression change template data including a laughed face, an angry face, a sad face, etc., one of which is selected by the user. The face area detection unit 104 reads still image data from the still image data storage unit and extracts face area data. The polygon data generation means 105 obtains feature point coordinate data such as the face ends, the eye center, the mouth center, etc. from the face area data and the expression change template data selected by the user, thereby determining polygon dividing line segments, Is generated in accordance with the polygon dividing line segment.

  The polygon data associating means 106 uses the still image data as texture data, and associates the portion with each plane of the polygon data. The polygon vertex vertical moving means 107 moves the vertex of the polygon data on the polygon dividing line segment in the direction perpendicular to the plane according to the expression change template data. The image data generation means 108 uses the 3D graphic calculation processing means 101 to generate image data by executing light source processing and rotation processing for giving an inclination to the polygon obtained according to the expression change template data. The display unit 109 displays the image data generated by the image data generation unit.

(Operation of the embodiment)
About the system comprised as mentioned above, the operation | movement is demonstrated using the flowchart (step S201-S213) of FIG.

  First, facial expression change template data to be used is determined from a plurality of facial expression change template data (step S201). The expression change template data includes the following information. 1) An arithmetic expression for generating feature point coordinates from face area data or an identifier for designating a program. 2) An arithmetic expression for generating a polygon dividing line segment from the feature point coordinates, or an identifier for designating a program. 3) An arithmetic expression for generating a movement destination coordinate of a polygon vertex on a polygon dividing line segment or an identifier for designating a program. 4) An arithmetic expression for generating light source information data or an identifier for designating a program. 5) An arithmetic expression for generating an axis for giving an inclination to a polygon or an identifier for designating a program.

  Next, face area data including the entire face area, eye area, mouth area and the like as shown in FIG. 3 is obtained from the still image data by the face area detecting means 104 (step S202).

  Next, according to the expression change template data obtained from step S201 by the polygon data generation means 105, feature point coordinate data including feature points such as both face ends, eye centers and mouth centers from the face area data obtained from step S202. Is obtained (step S203). Note that it is worth using the both ends of the eye, the center of the eyebrows, and the ends of the eyebrows as the feature point coordinate data.

  Next, according to the expression change template data obtained from step S201, a plurality of polygon dividing line segments are determined from the feature point coordinates obtained from step S203 (step S204). The polygon dividing line segment obtained by this step is substantially parallel to the center line that bisects the face vertically.

  Next, a plurality of polygon data is generated by dividing a rectangular plane similar to the still image data according to the polygon dividing line segment obtained in step S204 (step S205).

  Next, the polygon data associating means 106 uses the still image data as texture data, and associates the portion with each plane of the polygon data obtained in step S205 (step S206).

  Next, the vertex of the polygon data obtained from step S205 existing at the coordinates on the polygon dividing line obtained from step S204 is obtained by the polygon vertex vertical moving means 107 according to the expression change template data obtained from step S201. It moves in the direction perpendicular to the plane (step S207). FIG. 4 shows the polygon data divided by the polygon dividing line segments passing through both the face ends, the eye center, and the mouth center, and the vertexes of the polygon data on the polygon dividing line segments passing through the eye center are moved toward the image. An example is shown.

  Next, the image data generation means 108 sets a normal vector for calculating the light reflection angle at each vertex of each polygon data obtained in step S207 (step S208).

  Next, viewpoint coordinates and line-of-sight direction data are set (step S209).

  Next, light source information data is set according to the expression change template data obtained from step S201 (step S210).

  Next, in accordance with the expression change template data obtained in step S201, a rotation axis for giving inclination to the polygon is obtained, and the polygon data obtained in step S207 is inclined along the rotation axis with respect to the viewing direction. The coordinate conversion is performed by the 3D graphics calculation processing means 101 so as to hold the image (step S211). 5 and 6 show an example in which the polygon data is tilted to an angle that forms an obtuse angle and an acute angle with respect to the line-of-sight direction around a straight line that is included in the polygons at the left and right ends and bisects the polygon. ing. FIG. 5 shows a smiling expression and a bright image due to the positional relationship of the light sources. FIG. 6 shows a sad expression and a dark image due to the positional relationship of the light sources. Note that the same effect can be obtained by rotating the viewpoint coordinate and line-of-sight direction data and the light source information along the rotation axis without rotating the polygon data.

  Next, 3D graphics is input using the texture data obtained from step S206, the viewpoint coordinates and line-of-sight direction data obtained from step S209, the light source information data obtained from step S210, and the coordinate-converted polygon data obtained from step S211. Image data is generated by the arithmetic processing unit (step S212).

  Finally, the display unit 109 displays the image data on the display device (step S213).

  After executing Step S201 to Step S210, the facial expression and the brightness of the image are changed by repeating only Step S211 to Step S213 while continuously changing the inclination of the polygon data in Step S211 little by little. It is possible to change it.

(Effect of embodiment)
As described above, according to the facial image expression change processing device of the present embodiment, a still image including a face is mapped to a planar polygon in 3D space, and the eyes and mouth of the face obtained by the face area detection unit 104 are mapped. Fold the plane polygon along a parallel polygon dividing line that passes through the center coordinate point, and then tilt the plane polygon about the straight line that bisects the polygon up and down. It is not necessary to move, transform, and interpolate each part of the face on the still image data, using the illusion effect of human eyes with a smaller amount of calculation. It is possible to obtain an image in which the facial expression in the still image is changed.

  Further, the obtained facial image after the expression change gives an unnatural impression, and by using it for a slide show or the like, an image effect effect that gives a strong impression is given. The effect is said.

  The face image facial expression change processing device according to the present invention can obtain an image in which the facial expression is changed with a small amount of processing, and thus is useful for a still image viewer (slide show) function in a mobile device or the like.

Configuration diagram of a facial image expression change processing device in an embodiment flowchart The figure which shows the face area data detected by the face area detection means 104 Polygon dividing line diagram (vertical movement of vertex data) Polygon rotation diagram (obtuse angle) Polygon rotation diagram (acute angle)

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Face image facial expression change processing apparatus 101 3D graphic calculation processing means 102 Still image data storage means 103 Expression change template data storage means 104 Face area detection means 105 Polygon data generation means 106 Polygon data association means 107 Polygon vertex vertical movement means 108 Images Data generation means 109 Display means

Claims (1)

A facial image expression change processing device for dynamically changing and displaying a facial expression in a still image,
3D graphic operation processing means for performing 3D graphic operation processing;
Still image data storage means for storing still image data;
Facial expression change template data storage means for storing facial expression change template data;
Face area detecting means for extracting face area data from the still image;
Obtaining feature point coordinate data such as both ends of the face, center of eyes, center of mouth from the face area data and the expression change template data, thereby determining polygon dividing line segments, and dividing a rectangular plane according to the polygon dividing line segments Polygon data generating means for generating polygon data of
A polygon vertex vertical moving means for moving the vertex of the polygon data on the polygon dividing line segment in a direction perpendicular to a plane according to the expression change template data;
Polygon data associating means for associating still image data as texture data and associating the portion with each plane of the polygon data;
Image data generating means for generating image data by executing rotation processing for giving tilt to a polygon obtained according to light source processing or the expression change template data by the 3D graphic calculation processing means;
A facial image expression processing apparatus comprising: display means for displaying the image data generated by the image data generation means.

Images