JP2009009296A - Face image deformation method and face image deformation device implementing this method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image deformation method and device, allowing acquisition of a face-lifted image fit for a customer desiring facelift with easy operation. <P>SOLUTION: A face image is input to an image input part 11. A characteristic point and a characteristic line that are characteristics are extracted from the input face image by a face characteristic extraction part 30. A face characteristic movement amount determination part 31 determines a movement amount of the extracted face characteristic based on learning data stored in a learning data storage part 31a. A deformed face image creation part 32 moves the face characteristic in the face image based on the movement amount determined by the face characteristic movement amount determination part 31, generates a deformed face image that is a face after the facelift, and presents it to a user. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face image deformation technique for deforming a face image based on learning data and creating an appropriate shaped image.

  At present, there is a great interest in public cosmetic surgery, and treatments such as shaping a part of the face are being actively performed. However, although the details of shaping are determined based on a thorough discussion before shaping, the face after shaping cannot be confirmed before shaping, so the customer is still uneasy.

  In order to solve this problem, a system that virtually generates a face image after shaping and presents it to the customer (Patent Document 1), or a system that presents a shaping policy on the screen and deepens the understanding of the face after shaping ( There are known techniques such as Patent Document 2) and a method of using morphing to bring a face close to a favorite face (Patent Document 3).

JP 2002-109555 (paragraph number 0012-0013, FIG. 15) JP 2005-62960 (paragraph number 0005-0008, FIG. 3) Japanese Patent Laid-Open No. 11-185048 (paragraph number 0006, FIG. 4)

  In this way, there are various techniques for performing virtual shaping and presenting it to the customer. However, in the technique of Patent Document 1, it is difficult to perform intuitive operations because it is necessary to input numerically how to deform facial parts (eyes, eyebrows, nose, mouth, contour, etc.). In the technique of Patent Document 2, a guideline for beautifying can be obtained intuitively, but the face after shaping cannot be confirmed. Although the technique of Patent Document 3 can create a shaped image that is close to a favorite person for each face part, many trials and errors are required to obtain a comprehensively beautiful shaped image. An object of the present invention is to provide a face image deformation method and apparatus capable of obtaining a shaped image suitable for a customer who desires shaping with a simple operation.

The face image deformation method of the present invention is extracted by an image input step for inputting a face image, a face feature extraction step for extracting feature points and feature lines as face features from the face image, and the face feature extraction step. A facial feature movement amount determination step for determining the movement amount of the facial feature based on learning data; and an image obtained by moving the facial feature in the facial image based on the movement amount determined by the facial feature movement amount determination step. A deformed face image creation step is generated.
According to this method, since the movement amount of the facial feature related to the deformation is determined based on the past deformation data accumulated in the learning data, the user can see how beautiful his / her face is deformed. It is possible to cope with an unknown case. Furthermore, since the deformed face image is generated and presented to the user, the face after shaping can be confirmed.

  In the face image deformation method of the present invention, the facial feature movement amount determination step includes a movement amount of the facial feature based on a movement amount of the learning data having a facial feature similar to the facial feature extracted by the facial feature extraction step. To decide. According to this method, learning data having a facial feature similar to the user's facial feature is extracted, and the movement amount of the user's facial feature is determined based on the movement amount of the facial feature corresponding to the learning data. Since the deformation of the user's face is determined based on the deformation performed in the past by a person with a face similar to that of the user's face, it is possible to obtain an appropriate deformation for the user's facial features.

  In the face image deformation method of the present invention, the facial feature movement amount determination step uses a neural network that receives the facial feature extracted by the facial feature extraction step and outputs the facial feature movement amount. And This makes it possible to easily obtain a deformed face image appropriate for the user based on past learning data.

  In the face image deformation method of the present invention, the learning data includes a plurality of clusters, and includes a policy specifying step that specifies a deformation policy of the face image, and the face feature movement amount determination step includes the policy specifying step. The cluster selected based on the policy specified by is used as the learning data. Thereby, for example, when it is desired to change the impression of the face, learning data corresponding to the impression can be used, and it is possible to easily obtain a deformed facial image that gives a desired impression.

  The face image deformation method of the present invention includes a face part specifying step for specifying a face part to be deformed, and the face feature moving amount determining step determines only the moving amount of the face feature related to the specified face part. Thereby, it is possible to cope with a case where only a part of the face is desired to be deformed.

  In the face image deformation method of the present invention, the face image input to the image input step includes two or more images having different viewpoints, and the face feature extraction step extracts the position of the face feature in a three-dimensional space, The facial feature movement amount determination step determines the movement amount of the facial feature in the three-dimensional space. As a result, facial features can be captured three-dimensionally, and it is possible to cope with cases where it is difficult to confirm deformation in two dimensions, such as the nose. Furthermore, by using a plurality of viewpoint images, a three-dimensional deformed face image can be generated, and the face after shaping can be confirmed from various angles.

Further, in the present invention, a program for causing a computer to execute the face image deformation method described above and a medium on which the program is recorded are also subject to rights.
Further, in the present invention, a face image deformation device incorporating the above-described face image deformation method is also subject to rights, and such a face image deformation device includes an image input unit for inputting a face image, and a face from the face image. A facial feature extraction unit that extracts feature points and feature lines as features; a facial feature movement amount determination unit that determines a movement amount of the facial feature extracted by the facial feature extraction unit based on learning data; and the facial feature Based on the amount of movement determined by the amount-of-movement determining unit, a deformed face image creating unit that generates an image obtained by moving the facial feature in the face image is configured. Naturally, such a face image deformation device can obtain all the effects described in the face image deformation method described above.

[First Embodiment]
An embodiment of a method and apparatus for deforming a face image according to the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram of a face image deformation apparatus using a face image deformation method according to the present invention. This face image deformation apparatus includes a controller 1 that controls the entire apparatus, a camera 21 that inputs a user's face image to the controller 1, various semiconductor memories on which face images are recorded, and media that acquires face image data from a CD-R. A reader 22, a scanner 23 that acquires face image data from a photograph, an input device 24 that performs input for operating the controller 1, and a monitor 25 that displays various information are connected. As the input device 24, a keyboard 24a, a mouse 24b, or the like can be used, and a trackball or a touch panel installed on the monitor 25 can also be used. In this embodiment, a general-purpose personal computer functions as the controller 1. In this case, the controller 1 can be made to function as a server, and a face image can be transmitted from a client terminal through a network (not shown), and a deformed face image can be generated and returned.

  The controller 1 of this face image deforming apparatus has a function unit for performing various operations of the face image deforming apparatus by using a CPU as a core member by hardware and / or software. As shown in FIG. 2, functional units particularly relevant to the present invention include an image input unit 11 that captures a face image read by a camera 21, a media reader 22, and a scanner 23, and is captured by the image input unit 11. A GUI for generating a control command from a memory 12 for holding a face image, creation of a graphic operation screen including various windows and various operation buttons, and user operation input (via the keyboard 24a, mouse 24b, etc.) through the graphic operation screen (Graphical User Interface) GUI section 13, face image deformation section 14 that creates a deformed image of the face image stored in the memory 12, display of the deformed face image created by the face image deformation section 14, and GUI section A video for generating a video signal for displaying the GUI screen generated by the monitor 13 on the monitor 25 Such as the control section 15, and the like.

  The face image deformation unit 14 extracts feature points and feature lines (hereinafter referred to as face features) that are the basis of face deformation from the face image data held in the memory 12, and based on the learning data. The facial feature movement amount determination unit 31 determines the movement amount of the facial feature extracted by the facial feature extraction unit 30, and the facial feature movement amount determination unit 31 determines the facial feature of the face image held in the memory 12. A deformed face image creation unit 32 is provided that creates a deformed face image that has been moved by the amount of movement. In addition, the face feature movement amount determination unit 31 includes a learning data storage unit 31a that stores learning data.

  Normally, the face image deformation unit 14 is configured by software, but may be configured by a combination of software and hardware, or may be configured only by hardware combining logic and the like.

  Next, the operation of the deformed face image generation method according to the present invention will be described with reference to the flowchart of FIG. First, a user's face image is input from the camera 21, media reader 22 or scanner 23 to the image input unit 11, and the input face image is subjected to pre-processing such as face size normalization and color correction. Both the face images before and after the preprocessing are accumulated in the memory 12 (# 01). At this time, the size of the face is normalized so that the binocular distance becomes a predetermined distance, and the enlargement / reduction ratio at the time of normalization is also stored in the memory 12 at the same time. By normalizing in this way, the amount of movement of the facial features can be obtained without being influenced by the size of each person's face. Further, when the input face image has a low resolution, it may be converted into a high resolution image by the technique disclosed in Japanese Patent Application Laid-Open No. 2006-222493 and used instead of the original image. With this configuration, as long as a plurality of low-resolution images can be obtained, the low-resolution camera 21 can be used, which contributes to cost reduction of the apparatus. Furthermore, even when the low-resolution camera 21 is used, the quality of the image after deformation can be maintained.

  When there is a face part desired to be deformed (hereinafter referred to as a deformed face part), the user who has input the face image into the face image deforming apparatus designates the deformed face part through the GUI configured on the monitor 25. can do. By performing this designation, for example, it is possible to respond to a request to change only the “eye”. At this time, the GUI unit 13 displays a check box together with the name of each face part as a GUI, and allows the user to check the check box corresponding to the deformed face part, and directly designates the deformed face part using the mouse 24b. Which face part is designated is acquired by using a method such as The method for specifying the deformed face part is not limited to this, and other methods may be used. Also, multiple face parts can be specified, and for face parts that are paired, such as “eye”, if one is specified, the other is also specified. It is possible to improve the convenience for the user.

  The GUI unit 13 notifies the face image deformation unit 14 of the deformed face part designated by the user. If the user does not specify a deformed face part, the process proceeds assuming that a predetermined deformed face part is specified.

  In the face image deformation unit 14, the face feature extraction unit 30 extracts the face features shown in FIG. 4 (# 02). The facial feature extraction unit 30 can use a general-purpose facial feature extraction program. Further, the GUI unit 13 presents the position of the facial feature extracted by the facial feature extraction unit 30 to the user, and the user uses the mouse 24b or the like. It is also possible to correct the face feature position by operating. Further, the face feature position can be directly designated by the user operating the mouse 24b or the like through the GUI unit 13 without using the face feature extraction unit 30.

  In this embodiment, the facial features shown in FIG. 4 are the left and right eyebrows PRB1 and PLB1, the left and right eyebrows PRB2 and PLB2, the left and right eye corners PRE1 and PLE1, the left and right eye heads PRE2, PLE2, the nose PRN, PCN and PLN, Left and right mouth corners PRL and PLL, three points PRF on the face contour, PCF and PLF, left and right eyebrow contours LRBU, LRBL, LLBU and LLBL, left and right eye contours LREU, LREL, LLEU and LLEL, lip contour Lines LLU and LLL, and contour line LF. The face feature is not limited to this, and other feature points and feature lines may be used as the face feature. In the present embodiment, the feature line uses a method of vectorizing a curve. However, the feature line may be approximated by a function, and a coefficient may be used as a face feature, or another method may be used.

  Next, the facial feature movement amount determination unit 31 determines the movement amount of the facial feature extracted by the facial feature extraction unit 30 by a method described later (# 03). At this time, if the user designates a deformed facial part, the movement amount of the facial feature related to the designated facial part is obtained, but the present invention is not limited to this, and the movement amount of all the facial features is obtained. A configuration may be adopted in which only the facial features related to the deformed face part designated by the user are moved to generate a deformed face image.

  The deformed face image creation unit 32 moves the face feature extracted by the face feature extraction unit 30 by the movement amount determined by the face feature movement amount determination unit 31 with respect to the face image held in the memory 12. Then, a deformed face image is created (# 04). At this time, the movement amount of the facial feature of the face image before normalization can be calculated by multiplying the movement amount by the reciprocal of the enlargement / reduction ratio at the time of normalization held in the memory 12.

Next, the operation of the face feature movement amount determination unit 31 will be described using the flowchart of FIG. The learning data storage unit 31a accumulates the facial features of a large number of pre-deformation face images and their movement amounts before and after the deformation, and these are used by the facial feature movement amount determination unit 31 to determine the movement amount of the facial features. It functions as a dictionary. This dictionary represents each pre-deformation face image as a point in a multidimensional space (hereinafter referred to as a feature space) spanned by facial features, and is composed of a point group and a variance covariance matrix Σ of the point group. Yes. That is, when the number of images registered in the dictionary N, the number of dimensions of the feature space and r, each of the modified pre-face image corresponds to a point d _i in the feature space, the dictionary point group {d _i | i = 1, 2,... N} and the variance-covariance matrix Σ∈R ^r × ^r . Of course, the above-mentioned normalized face image is also used when creating the dictionary.

First, initialize the minimum distance L _min and the nearest point d _near to (# 11), representing the facial features extracted by the facial feature extraction unit 30 as a point P in the feature space (# 12). The method of setting the point P in the feature space from the facial feature is performed by arranging the feature points and the vector elements constituting the feature line in a predetermined order, but is not limited to this. You may use the method.

Next, the distance between the point P and the point d _i in the dictionary is calculated (# 13). The distance L of the point P and the point d _i can be expressed by _{L = (P-d i)} T Σ + (P-d i). Here, T represents a transpose of a matrix, and + represents a pseudo inverse matrix. The above expression of distance L is a modification of the expression of Mahalanobis distance representing the distance between clusters and points. When the Mahalanobis distance is used, weighting is applied at the time of distance calculation according to the variance value of the point group {d _i } on each eigenvector axis, so distance calculation according to the distribution feature of the point group {d _i } is possible. ing. In this embodiment, the point-to-point distance L is obtained based on this concept. The distance calculation is not limited to this, and other distances such as Euclidean distance may be used.

When the distance L between the point P and the point d _i is smaller than the minimum distance L _min at that time (Yes branch of # 14), the nearest point d _near = point d _i and L _min = L are set (# 15). . This processing is performed for all points d _i, the movement amount with respect to the point P of the movement amount of the corresponding face features d _near at the time (Yes branch of # 16) which distance calculation is completed for all points d _i (# 17). In this way, by determining the amount of movement of the facial feature, it is possible to perform a transformation equivalent to the transformation performed by another person having a facial feature similar to the user who has input the facial image. In other words, even if the user does not clearly know how to transform his / her face to make it beautiful, it is possible to use the deformation performed by a person with a similar face from the deformation pattern accumulated in the past. Therefore, it is possible to easily perform face image deformation by a deformation method suitable for itself.

The method for determining the amount of movement based on the distance from the dictionary is not limited to the above method, and the distance between the point P and all d _i is obtained, a predetermined number is selected from the closest ones, and the corresponding values are determined. It is possible to use a method such as determining the average value of the movement amount as the movement amount, or determining the average of the movement amount corresponding to d _i within a predetermined distance as the movement amount. You can decide.

  At this time, if a deformed face part is specified, when the feature points and feature lines are vectorized, only the necessary feature points and feature lines of the face parts are used for vectorization. When calculating the distance, the dimension of the feature space is changed as necessary.

  Furthermore, it is also possible to specify what kind of facial appearance the user wants to become through the GUI unit 13. In this case, the learning data is divided into a plurality of clusters for each impression, and by using the cluster corresponding to the impression selected by the user as the learning data, the user can easily make a face that gives a desired impression. The amount of movement of feature points and feature lines can be obtained. The impression of appearance is, for example, cute, intelligent, youthful, etc. By using such a configuration, the user can know the result of a person who has a facial appearance similar to that of his / her face deformed to give a certain impression to his / her face.

  The face feature movement amount determination unit 31 can also be realized using a neural network. When a neural network is used for the face feature movement amount determination unit 31, the learning data storage unit 31a stores parameters for constructing the neural network shown in FIG. The neural network in FIG. 6 has a configuration in which the input layer is a facial feature and the output layer is the amount of movement of each facial feature. However, the present invention is not limited to this configuration. Neural networks of other configurations may be used, such as associating a network with each face part or a combination of a plurality of face parts. Further, a neural network having the above-described configuration may be prepared, and the neural network to be used may be switched depending on whether or not a deformed face part is designated by the user. With this configuration, when all facial features are processed by a single neural network, the transformation takes into account the overall balance. When a neural network for each facial part is used, the transformation is specialized for individual facial parts. When a plurality of face parts are processed by a single neural network, it is possible to perform deformation according to various situations such as deformation considering the balance between face parts.

  The face feature movement amount determination unit 31 sets the face feature extracted by the face feature extraction unit 30 for the input layer of the neural network in FIG. 6, and uses the output to the output layer as the movement amount of the face feature. Also in this case, if the user designates a deformed face part through the GUI unit 13, only the movement amount for the face feature related to the designated face part is passed to the next deformed face image creation unit 32. Even when a neural network is used, a configuration may be employed in which a neural network is created for each facial impression and the neural network is switched according to the facial impression desired by the user.

Next, an operation in the case where the “eye” is deformed in the deformed face image creation unit 32 will be described using the flowchart of FIG. The same applies to the case where other face parts are deformed, so the explanation of the deformation of other face parts is omitted. First, the feature points PRE1, PRE2, PLE1, PLE2, and feature lines LREU, LREL, LLEU, LLEL are moved using the movement amount determined by the face feature movement amount determination unit 31 (# 21). Next, a point group {q _i | i = 1, 2... M} constituting the face part (“eye”) defined by the face feature after movement is extracted, and the point q _i in the image before deformation is extracted. The corresponding point p _i is obtained (# 22), and the pixel value of p _i is set as the pixel value of q _i (# 23). coordinate values of q _i is an integer, coordinate values of the corresponding p _i, since in general not an integer, obtains a pixel value of p _i by interpolation using peripheral pixel values of p _i, it Let q _{i be} the pixel value. By repeating this until all the q _i pixel values are obtained (Yes branch of # 24), a deformed face image is created. At this time, a region in image before deformation is not included in the "eyes" in a region and {q _i} is the image after deformation pixel value becomes undefined. For such pixels, pixel values are determined by using surrounding skin color pixels (# 25).

FIG. 8 shows a case where the left eye is deformed. In this modification, the eye is enlarged by moving the feature line LLEU to LLEU ′. At this time, a region surrounded by the feature lines LLEU ′ and LLEL is {q _i }. From the movement of the facial features, a point corresponding to the point q _i in the pre-deformation image is obtained, the pixel value of the point is obtained, and the pixel value is set as the pixel value of the point q _i . In this example, since all the eye regions before deformation belong to {q _i }, there is no pixel with an indefinite pixel value, but when the position of the corner of the eye is changed, a pixel with an indefinite pixel value is present. These pixel values are determined using peripheral skin color pixels and the like.

  The deformed face image created by the deformed face image creating unit 32 is displayed on the monitor 25 through the video control unit 15. At this time, the user may be able to change the face feature position through the GUI unit 13. Further, a button or the like for selecting whether or not the user likes the deformed face image can be configured on the GUI, and when the user likes it, the facial feature of the user and the amount of movement are stored in the learning data storage unit 31a. The learning data storage unit 31a adds the notified face feature and its movement amount to the learning data, and performs relearning. The re-learning may be performed each time new data is added, or may be performed at a predetermined timing such as when a predetermined number of data is added. By using such a configuration, the learning data can be enhanced each time the user uses the face image deformation device.

[Second Embodiment]
A second embodiment of a facial image deformation method and apparatus according to the present invention will be described with reference to the drawings. The second embodiment according to the present invention includes a three-dimensional coordinate calculation unit 31b in addition to the functions of the first embodiment. In the second embodiment of the present invention, data capable of calculating three-dimensional coordinates is input to the image input unit 11. As data for which three-dimensional coordinates can be calculated, for example, a multi-viewpoint image represented by a stereo camera, a normal face image, a set of three-dimensional data by a range finder, or the like can be used. Instead, other information may be used. Preferably, an image obtained by a trinocular camera that can generate a multi-viewpoint image and has a small processing amount is used. In this embodiment, a trinocular camera is used. FIG. 10 shows an arrangement example of the trinocular camera. In this example, the cameras 21a, 21b, and 21c are all parallel stereo cameras whose optical axes are parallel, but may be converging stereo cameras. When a converging stereo camera is used, the amount of computation increases compared to a parallel stereo camera, but the range of viewpoint movement when generating a multi-viewpoint image can be widened. Moreover, although the base line length between the camera 21a and the camera 21b and the base line length between the camera 21b and the camera 21c are made to coincide, it is not limited to this arrangement, and other arrangements may be used. Although the horizontal position of the camera 21b is different from that of the other cameras, the horizontal positions of all the cameras may be the same. By using such a three-lens camera, it is possible to specify the three-dimensional position of the facial feature and confirm the deformed face image by moving the viewpoint.

  The operation of the second embodiment of the present invention will be described below using the flowchart of FIG. Three face images photographed by the trinocular camera are input to the image input unit 11 (# 31), pre-processed, and stored in the memory 12. The face feature extraction unit 30 extracts face features from each of the three images stored in the memory 12 (# 32). In the face feature movement amount determination unit 31, the three-dimensional coordinate calculation unit 31b calculates the three-dimensional coordinates of each face feature (# 33). In this embodiment, since a three-lens camera is used for the input image, three-dimensional coordinates can be calculated by a method similar to a three-dimensional measurement method based on triangulation using a normal stereo camera. In three-dimensional measurement using a stereo camera, it is generally known that the longer the baseline length, the higher the distance accuracy. Therefore, when the three-dimensional coordinates are calculated, images acquired from the camera 21a and the camera 21c are used. Prioritize the correspondence between the two. The image acquired by the camera 21b is used only when the feature points and the feature lines cannot be matched between the images acquired from the camera 21a and the camera 21c.

  The three-dimensional coordinate calculation unit 31b generates a user's three-dimensional face model from the three-dimensional coordinates of the face feature. The three-dimensional face model may be generated directly from the input image, or the basic face model is stored in advance, and the basic face model is deformed using the obtained three-dimensional coordinates of the facial features to obtain the user's face. You may create a matching face model. Thus, by creating a deformed face model, a three-dimensional deformed face image can be easily created.

  Next, using the learning data stored in the learning data storage unit 31a, the three-dimensional movement amount of the facial feature is determined by the above method. In the present embodiment, all the learning data stored in the learning data storage unit 31a is also three-dimensionally deformed data.

  The deformed face image creation unit 32 moves the face features of the three-dimensional face model created by the three-dimensional coordinate calculation unit 31b based on the movement amount determined by the face feature movement amount determination unit 31 (# 34). The user's face image held in the memory 12 is rendered in the later model to generate a three-dimensional face image (# 35). In this case, the monitor 25 can use a normal two-dimensional monitor, a three-dimensional monitor such as a lenticular method or a parallax barrier method, and the video control unit 15 can select a three-dimensional face according to the connected monitor 25. The image format is converted and displayed on the monitor 25. At the same time, the GUI unit 13 provides a GUI that can be operated so that the face image displayed on the monitor 25 can be viewed from different viewpoints, and the user operates the mouse 24b or the like to instruct a change of the viewpoint. 13 notifies the video control unit 15 of the viewpoint change, and the video control unit 15 notified of the viewpoint change generates a display image corresponding to the viewpoint change.

  In the present embodiment, by capturing and deforming a facial feature at a three-dimensional position, the transformation of the facial feature that is difficult to confirm in a two-dimensional image typified by the height of the nose is also expressed. Is possible. Furthermore, by using a multi-viewpoint image such as a three-lens camera, when the user confirms the deformed face, images viewed from various viewpoints can be provided, and the face image after deformation can be understood more. It becomes possible.

  In the description of the above-described embodiment, an example in which the face image deformation method according to the present invention is incorporated as a program in a general-purpose computer has been described. However, using dedicated hardware, only hardware or a combination of hardware and logic. A configuration combining software may be used.

External view of apparatus using face image deformation method according to the present invention The functional block diagram which shows the functional element of the face image deformation | transformation method by this invention The flowchart which shows the flow of the process in the 1st Embodiment of this invention. The figure which shows the feature point and feature line of the face in embodiment of this invention The flowchart which determines the moving amount | distance of the feature point and feature line in embodiment of this invention The figure which shows the neural network used for the determination of the movement amount of the feature point and feature line in embodiment of this invention The flowchart which shows the flow of the movement of the feature point and feature line in embodiment of this invention The figure which shows the movement of the feature point and feature point in embodiment of this invention The figure which shows the flow of a process in the 2nd Embodiment of this invention. The figure which shows arrangement | positioning of the camera in the 2nd Embodiment of this invention.

Explanation of symbols

1: Controller 11: Image input unit 12: Memory 13: GUI unit 14: Face image transformation unit 15: Video control unit 21: Camera 22: Media reader 23: Scanner 24: Input device 24a: Keyboard 24b: Mouse 25: Monitor 30 : Face feature extraction unit 31: face feature movement amount determination unit 31 a: learning data storage unit 31 b: three-dimensional coordinate calculation unit 32: deformed face image creation unit

Claims (7)

An image input step for inputting a face image;
A facial feature extraction step of extracting feature points and feature lines that are facial features from the facial image;
A facial feature movement amount determination step for determining a movement amount of the facial feature extracted by the facial feature extraction step based on a statistical learning rule;
A face image deformation method comprising: a deformed face image creation step of generating an image obtained by moving the face feature in the face image based on the amount of movement determined by the face feature movement amount determination step.   The facial feature movement amount determination step determines the movement amount of the facial feature based on a statistical learning rule defined by a facial feature similar to the facial feature extracted by the facial feature extraction step. Item 2. The face image deformation method according to Item 1.   The facial feature movement amount determination step uses a neural network that receives the facial feature extracted by the facial feature extraction step and outputs the facial feature movement amount as a statistical learning rule. The face image deformation method according to claim 1. The learning data used for the statistical learning rule is composed of a plurality of clusters,
A policy designating step of designating a policy of deformation of the face image;
The said face feature movement amount determination step uses the said cluster selected based on the said policy designated by the said policy designation | designated step as said learning data. Face image deformation method. A face part specifying step for specifying a face part to be deformed is provided.
5. The face image deformation method according to claim 1, wherein the facial feature movement amount determination step determines only a movement amount of a facial feature related to the designated facial part. 6. The face image input in the face image input step includes two or more images having different viewpoints,
The facial feature extraction step extracts a facial feature position in a three-dimensional space;
The face image deformation method according to claim 1, wherein the face feature movement amount determination step determines a movement amount of the face feature in a three-dimensional space. An image input unit for inputting a face image;
A facial feature extraction unit that extracts feature points and feature lines that are facial features from the facial image;
A facial feature movement amount determination unit that determines a movement amount of the facial feature extracted by the facial feature extraction unit based on a statistical learning rule;
A face image deformation apparatus comprising: a deformed face image creation unit that generates an image obtained by moving the face feature in the face image based on the amount of movement determined by the face feature movement amount determination unit.

Images