WO2017006615A1

WO2017006615A1 - Aging prediction system, aging prediction method, and aging prediction program

Info

Publication number: WO2017006615A1
Application number: PCT/JP2016/063227
Authority: WO
Inventors: 永田　毅; 和敏松崎; 秀正前川; 和彦今泉
Original assignee: みずほ情報総研株式会社; 警察庁科学警察研究所長が代表する日本国
Priority date: 2015-07-09
Filing date: 2016-04-27
Publication date: 2017-01-12
Also published as: CN107408290A; KR20170115591A; KR101968437B1

Abstract

Provided is an aging prediction system, comprising: a model storage unit which stores shape age models for predicting age-related changes in face shapes, texture age models for predicting age-related changes in facial surface textures, and three-dimensional predict models for predicting three-dimensional data from two-dimensional images; and a control unit which predicts aging. The control unit is configured to execute a predict process of generating an aging face model, and of outputting the generated aging face model to an output unit.

Description

Aging prediction system, aging prediction method, and aging prediction program

The present invention relates to an aging prediction system, an aging prediction method, and an aging prediction program for performing an aging simulation of a face image.

In beauty, a system for predicting changes in the shape of a face or body after aging has been studied (for example, see Patent Document 1). In the technique described in Patent Document 1, a polygon mesh is constructed from an image acquired by scanning a face, the polygon mesh is re-parameterized, and a base mesh and a displacement image are calculated. This displacement image is divided into a plurality of tiles, and the statistical value of each tile is measured. The displacement image is deformed by changing the statistical value, and the deformed displacement image is combined with the base mesh to synthesize a new face.

JP 2007-265396 A

However, when performing an aging simulation on a face image, it is not always possible to obtain 3D face data, and an aging simulation must be performed using a 2D face image of a face taken from various angles. There is also. In this case, it is necessary to consider the direction (angle) at which the face of the two-dimensional face image was captured. Also, the face has feature points that are anatomically the same position. Therefore, if a face image is handled as a simple texture image without considering anatomical feature points, it is difficult to efficiently execute an accurate aging simulation.

An object of the present invention is to provide an aging prediction system, an aging prediction method, and an aging prediction program for efficiently and accurately performing an aging simulation of a face image.

In one aspect, an aging prediction system that solves the above problems includes a shape aging model that predicts changes due to aging of the face shape, a texture aging model that predicts changes due to aging of the texture of the face, A model storage unit that stores a three-dimensional prediction model that predicts three-dimensional data from a two-dimensional image, and a control unit that is configured to be connected to an input unit and an output unit and that predicts aging. The control unit acquires a prediction target image from the input unit, extracts a feature point of the prediction target image, estimates a face direction in the prediction target image using the extracted feature point, and Generating first three-dimensional data based on the predicted prediction model and the estimated face orientation; generating second three-dimensional data from the first three-dimensional data using the shape aging model; Applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data to generate an aging texture, and for the second three-dimensional data, An aging texture is synthesized to generate an aging face model, and a prediction process for outputting the generated aging face model to the output unit is executed. Thereby, since an aging face model is produced | generated using a shape aging model and a texture aging model, it can carry out more precisely than aging only a shape or only a texture. Furthermore, even if the face of the prediction target image is not photographed from the front, the face direction in the prediction target image can be estimated by learning and efficiently aged.

In one embodiment, the control unit obtains a face orientation angle designated for output from the input unit, and uses the generated aging face model to generate a two-dimensional face image of the face orientation angle. It may be further configured to generate and output the generated two-dimensional face image to the output unit. As a result, an image of an aging face with a designated face orientation can be output.

In one embodiment, the two-dimensional image is a first two-dimensional image, and the control unit generates a second two-dimensional image based on the acquired three-dimensional face sample data, and the second 2D image A feature point is specified in a three-dimensional image, and the feature point is used to generate normalized sample data obtained by normalizing the three-dimensional face sample data. Using the normalized sample data, the shape aging model and the It may be further configured to generate a texture aging model and execute a learning process for storing the generated shape aging model and the generated texture aging model in the model storage unit. Thereby, an aging model can be generated based on actual sample data.

In one embodiment, the learning process may include generating the three-dimensional prediction model using the normalized sample data and storing it in the model storage unit. Thereby, a three-dimensional prediction model can be generated based on actual sample data.

In one embodiment, the model storage unit stores a first texture aging model calculated using principal component analysis and a second texture aging model calculated using wavelet transform. The control unit performs a first wavelet coefficient obtained by performing a wavelet transform on an image obtained by applying a first texture aging model to the first two-dimensional image, and a first wavelet coefficient for the first two-dimensional image. The texture aging model to be applied may be further specified according to a result of comparison with the second wavelet coefficient to which the texture aging model of 2 is applied. This makes it possible to use the second texture aging model that is predicted by using existing stains, wrinkles, and the like, so that a more appropriate aging model can be generated.

In another aspect, a shape aging model that predicts changes in facial shape due to aging, a texture aging model that predicts changes in facial texture due to aging, and 3D data from 2D images Method for predicting aging by using an aging prediction system including a model storage unit storing a three-dimensional prediction model, and a control unit configured to be connected to an input unit and an output unit I will provide a. The control unit acquires a prediction target image from the input unit, extracts a feature point of the prediction target image, estimates a face direction in the prediction target image using the extracted feature point, and Generating first three-dimensional data based on the predicted prediction model and the estimated face orientation; generating second three-dimensional data from the first three-dimensional data using the shape aging model; Applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data to generate an aging texture, and for the second three-dimensional data, A prediction process for generating an aging face model by synthesizing aging textures and outputting the generated aging face model to the output unit is executed.

In yet another aspect, a shape aging model that predicts changes due to aging of the face shape, a texture aging model that predicts changes due to aging of the texture of the face, and 3D data from 2D images are predicted. A program for predicting aging by using an aging prediction system including a model storage unit storing a three-dimensional prediction model, and a control unit configured to be connected to an input unit and an output unit A non-transitory computer-readable storage medium is provided. When the program is executed by the aging prediction system, the control unit acquires a prediction target image from the input unit, extracts a feature point of the prediction target image, and uses the extracted feature point to perform the prediction Estimating the face orientation in the target image, generating first three-dimensional data based on the three-dimensional prediction model and the estimated face orientation, and using the shape aging model, the first three-dimensional Generating second three-dimensional data from the data, applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data, generating an aging texture, Synthesizing the aging texture with the second three-dimensional data to generate an aging face model, and executing a prediction process for outputting the generated aging face model to the output unit .

Explanatory drawing of the aging prediction system of this embodiment. The flowchart of the whole data generation process used for this embodiment. The flowchart of the process sequence of the learning pre-processing of this embodiment. The flowchart of the process sequence of the conversion process to the cylindrical coordinate system of this embodiment. It is explanatory drawing explaining the conversion process to the cylindrical coordinate system of this embodiment, Comprising: (a) is an image with a missing part, (b) is 2 which made the radius and cylindrical direction angle in a cylindrical coordinate system 2 axes. A three-dimensional face image, (c) is a two-dimensional face image with the cylinder height and the circumferential angle in the cylindrical coordinate system as two axes. Explanatory drawing explaining the face feature point of the face feature point extraction process of this embodiment. The flowchart of the process sequence of the normalization process of the face feature point of this embodiment. It is explanatory drawing explaining the normalization process by the multiple regression analysis of this embodiment, Comprising: (a) is a flowchart of a processing procedure, (b) is explanatory drawing of the inclination correction | amendment around an X-axis and a Y-axis, (c) is Explanatory drawing of the inclination correction | amendment around Z-axis. It is explanatory drawing explaining the homology modeling process of the shape of this embodiment, Comprising: (a) is a flowchart of the process sequence of the homology modeling process of a shape, (b) A mesh model. It is explanatory drawing explaining the homologous modeling process of the texture of this embodiment, (a) is a flowchart of the process sequence of the homologous modeling process of a texture, (b) is the texture transformed into the average coordinate of each mesh model Average face showing average. It is explanatory drawing explaining the homologous modeling process of the shape of this embodiment, Comprising: (a) is an image normalization mesh model, (b) is the average face produced | generated from the texture of the calculated average value. The flowchart of the process sequence of the angle learning process of the two-dimensional face image of this embodiment. Explanatory drawing explaining the variable selection algorithm in the machine learning process of this embodiment. Explanatory drawing explaining the variable update process algorithm in the machine learning process of this embodiment. The flowchart of the process sequence of the learning process of the three-dimensional conversion of this embodiment. The flowchart of the process sequence of the prediction verification process of the three-dimensional conversion of this embodiment. It is explanatory drawing explaining the verification result of the three-dimensional conversion of this embodiment, (a) is input data, (b) is a correct answer, (c) is predicted only by a two-dimensional feature point, (d) is 2 Face image when predicted with dimension feature points and image. The flowchart of the process sequence of the learning process of texture aging using the principal component analysis of this embodiment. The flowchart of the process sequence of the prediction verification process of the texture aging of this embodiment. Explanatory drawing explaining each main component in the prediction verification of the texture aging of this embodiment. Explanatory drawing explaining the image reconfigure | reconstructed by changing the upper limit of a main component in this embodiment. It is explanatory drawing explaining the aging process of the texture using the wavelet transformation of this embodiment, (a) is a flowchart of a processing procedure, (b) is the image which visualized and displayed the wavelet coefficient. The flowchart of the process sequence of the learning process of the three-dimensional shape aging of this embodiment. The flowchart of the process sequence of the aging prediction process of this embodiment. The flowchart of the process sequence of the aging prediction process of the texture of this embodiment. It is explanatory drawing explaining the mask area | region used for the aging prediction process of this embodiment, Comprising: (a) shows a shape prediction mask area | region, (b) shows a texture prediction mask area | region. It is the image in the aging prediction process of this embodiment, Comprising: (a) is input data, (b) is aging prediction after 10 years, (c) is the image which predicted aging after 15 years. .

Hereinafter, an embodiment embodying the aging prediction system, the aging prediction method, and the aging prediction program will be described with reference to FIGS. In this embodiment, the aging of the face due to aging is learned using the three-dimensional face data before and after aging. An aging simulation for predicting an aging face image is performed using the photographed two-dimensional face image.

<Configuration of aging prediction system>
As shown in FIG. 1, in this embodiment, an aging prediction system 20 is used. An input unit 10 and an output unit 15 are connected to the aging prediction system 20. The input unit 10 is a means for inputting various types of information, and includes a keyboard, a pointing device, an input interface for acquiring data from a recording medium, and the like. The output unit 15 is a means for outputting various types of information, and includes a display or the like.

The aging prediction system 20 is a computer system for performing aging prediction processing. The aging prediction system 20 includes a control unit 21, an aging data storage unit 22, a snapshot data storage unit 23, and a model storage unit 25.

The control unit 21 includes control means (CPU, RAM, ROM, etc.), and processes to be described later (a learning management stage, a first learning stage, a second learning stage, a third learning stage, a principal component analysis stage, a machine learning stage, (Aging management stage, first treatment stage, second treatment stage, third treatment stage, etc.). By executing the aging prediction program for that purpose, the control unit 21 has a learning management unit 210, a first learning unit 211, a second learning unit 212, a third learning unit 213, a principal component analysis unit 214a, and a machine learning unit. 214b, an aging manager 215, a first processor 216, a second processor 217, and a third processor 218.

The learning management unit 210 executes a process of learning the secular change of the face in the aging simulation. As will be described later, the learning management unit 210 stores centroid calculating means for calculating the centroid position of both eyes using face feature points. Furthermore, the learning management unit 210 creates in advance data related to a generic model (basic model) used in a homologous modeling process for shapes to be described later, and stores the data in a memory. This generic model is a model related to the face that represents the general characteristics of Japanese people. In the present embodiment, a generic model having 10741 mesh points and 21256 polygons (triangles) is used. This generic model includes mesh points corresponding to face feature points, and identification information for specifying each face feature point is set.

In addition, the learning management unit 210 stores a predetermined texture average calculation rule for calculating the average of the coordinates of each vertex of the normalized mesh model from the coordinates of the face feature points in the texture homology modeling process described later. ing.

Further, the learning management unit 210 includes a learning memory for recording a cylindrical coordinate system image, a cylindrical coordinate system coordinate, and a homologous model used for learning.
The first learning unit 211 executes a first learning process for creating a model for predicting three-dimensional face data represented by a three-dimensional cylindrical coordinate system from two-dimensional face data (face image).

The 2nd learning part 212 performs the 2nd learning process which produces the model for predicting the change by aging about the texture (texture) of a face image.
The 3rd learning part 213 performs the 3rd learning process which produces the model for predicting the change by aging about face shape.

The principal component analysis unit 214a performs principal component analysis processing in response to instructions from the learning management unit 210 and the learning units 211 to 213.
The machine learning unit 214b performs a process of calculating an explanatory variable (a feature amount used at the time of prediction) using a dependent variable (a prediction target feature amount) according to an instruction from the learning units 211 to 213.

The aging management unit 215 executes processing for generating a face image after aging using a two-dimensional face image. The aging management unit 215 acquires a two-dimensional face image to be predicted, and uses the first to third processing units 216 to 218 to perform an aging simulation of the texture and shape.

The first processing unit 216 executes processing for generating three-dimensional face data represented by a three-dimensional cylindrical coordinate system from the processing target two-dimensional face image.
The second processing unit 217 executes a process of predicting a change due to aging for the texture (texture) of the face image. In the present embodiment, the texture aging process is executed using a texture aging model using principal component analysis and a texture aging model using wavelet (WAVELET) transformation. The second processing unit 217 stores the weighting coefficient w used for this processing in the memory. The weight coefficient w is a value for determining which model is to be emphasized when using a model using principal component analysis and a model using wavelet transform. In the present embodiment, “1” is used as the weighting coefficient w.

The 3rd process part 218 performs the process which estimates the change by aging about a face shape using 3D face data.
Further, the aging data storage unit 22 stores three-dimensional face data (aging) before and after aging (10 years in this embodiment) for a predetermined number of learning subjects (samples used for learning). Data) is recorded. By using this aging data, changes before and after aging can be grasped. In this embodiment, data for about 170 people is used as the secular change data.

In the snapshot data storage unit 23, three-dimensional face data (snapshot data) obtained by photographing more learning subjects is recorded. This snapshot data is taken only once, and no aging data is recorded in the snapshot data storage unit 23. In this embodiment, data for about 870 people is used as snapshot data.

It is assumed that three-dimensional data is recorded in the data format (XYZ coordinates) other than the cylindrical coordinate system in the aging data storage unit 22 and the snapshot data storage unit 23 of the present embodiment.

The model storage unit 25 stores a model (an algorithm for calculating a result variable using an explanatory variable) used when performing various predictions in an aging simulation. In the present embodiment, data related to an angle prediction model, a three-dimensional prediction model, a texture aging model, and a shape aging model are stored.

In the angle prediction model data area, model data for predicting the direction (angle with respect to the face front) where the face of the two-dimensional face image to be processed is photographed is stored. The angle prediction model data is calculated and recorded by the first learning unit 211.

In the three-dimensional prediction model data area, model data for converting a front-facing two-dimensional face image into three-dimensional face data is stored. The three-dimensional prediction model data is calculated and recorded by the first learning unit 211.

In the texture aging model data area, model data for predicting the texture after aging is stored for the facial texture. The texture aging model data is calculated and recorded by the second learning unit 212. In the present embodiment, texture aging model data using principal component analysis and texture aging model data using wavelet transform are stored.

In the shape aging model data area, model data for predicting the shape after aging is stored for the face shape. This shape aging model data is calculated and recorded by the third learning unit 213.

<Model generation>
Next, the outline of the generation process will be described with reference to FIG.
First, the control unit 21 of the aging prediction system 20 generates the cylindrical coordinate system image D2 and the cylindrical coordinate system coordinate data D3 using the three-dimensional face data D1 stored in the aging data storage unit 22. The cylindrical coordinate system image D2 is two-dimensional image data created by projecting three-dimensional face data onto a cylindrical coordinate system and interpolating into a “900 × 900” equidistant mesh. The cylindrical coordinate system coordinate data D3 is data relating to the three-dimensional coordinates of each point of the “900 × 900” image generated by projecting the three-dimensional face data onto the cylindrical coordinate system.

Next, the control unit 21 generates face feature point data D4 using the cylindrical coordinate system image D2 and the cylindrical coordinate system coordinate data D3. The face feature point data D4 is data relating to the coordinates of the face feature points in the cylindrical coordinate system image D2. Details of the facial feature points will be described later.

The control unit 21 uses the cylindrical coordinate system coordinate data D3 to normalize the face feature point data D4 and the cylindrical coordinate system image D2. A normalized cylindrical coordinate system image D5, normalized face feature point data D6, and three-dimensional mesh data D7 (homology model) are generated. Details of this normalization processing will be described later. Here, the homologous model is three-dimensional coordinate data in which three-dimensional data about a face is expressed by a mesh and converted so that the vertices of corresponding meshes in different data are anatomically the same position.

Further, the control unit 21 generates a two-dimensional face image D8 photographed from an arbitrary angle using the normalized cylindrical coordinate system image D5 and the three-dimensional mesh data D7 (homology model).
The control unit 21 changes the 2D face image to the 3D face data using the 2D face image D8 photographed from an arbitrary angle, the normalized face feature point data D6, and the 3D mesh data D7 (homology model). The first learning process for the conversion is performed. Details of the first learning process will be described later.

Furthermore, the control part 21 performs the 2nd learning process about texture aging using the normalized cylindrical coordinate system image D5. Details of the second learning process will be described later.
Moreover, the control part 21 performs the 3rd learning process about three-dimensional shape aging using the three-dimensional mesh data D7 (homology model). Details of the third learning process will be described later.

<Pre-learning process>
Next, the pre-learning process performed before the first to third learning processes described above will be described with reference to FIGS. This process is performed individually for each three-dimensional data recorded in the aging data storage unit 22 and the snapshot data storage unit 23.

As shown in FIG. 3, first, the control unit 21 executes conversion processing into a cylindrical coordinate system (step S1-1). Details of this processing will be described later with reference to FIGS.
Next, the control unit 21 executes face feature point extraction processing (step S1-2). Details of this processing will be described later with reference to FIG.

Next, the control unit 21 executes normalization processing of face feature points (step S1-3). Details of this processing will be described later with reference to FIGS.
Next, the control unit 21 executes homology modeling processing (step S1-4). Here, a face shape homology model and a texture homology model are generated. Details of these processes will be described later with reference to FIGS. 9 and 10.

Next, the control unit 21 executes a process for generating a normalized cylindrical coordinate system image (step S1-5). Details of this processing will be described later with reference to FIG.
<Conversion processing to cylindrical coordinate system>
Next, the conversion process to the cylindrical coordinate system (step S1-1) will be described with reference to FIGS.

First, the learning management unit 210 of the control unit 21 performs a missing portion interpolation process (step S2-1). Specifically, the learning management unit 210 checks whether there is a missing part in the three-dimensional face data. When the missing part is detected, the learning management unit 210 performs interpolation of the missing part using the peripheral information of the missing part. Specifically, the missing portion is compensated using a known interpolation method based on a predetermined range of data adjacent to the periphery of the missing portion.

For example, assume that the 3D face data shown in FIG. In FIG. 5B, the three-dimensional face data is represented by a cylindrical coordinate system having a radius and a cylindrical direction angle in the cylindrical coordinate system as two axes. In this three-dimensional face data, some data around the chin and ears are missing. The images of these portions are generated by interpolation processing using surrounding images and are made up for.

Next, the learning management unit 210 of the control unit 21 executes a cylindrical coordinate system image generation process (step S2-2). Specifically, the learning management unit 210 projects the three-dimensional face data that has undergone the missing portion interpolation processing onto a cylindrical coordinate system (two-dimensional mapping). The learning management unit 210 interpolates the projected face image data into a “900 × 900” equally spaced mesh to generate a two-dimensional face image in a cylindrical coordinate system. The learning management unit 210 records the generated two-dimensional face image in the learning memory as a cylindrical coordinate system image D2.

FIG. 5C is a two-dimensional face image in which the three-dimensional face data projected onto the cylindrical coordinate system is represented by two axes (cylinder height and circumferential angle).
Next, the learning management unit 210 of the control unit 21 executes cylindrical coordinate system coordinate generation processing (step S2-3). Specifically, the learning management unit 210 projects each coordinate (X, Y, Z) of the three-dimensional face data onto the cylindrical coordinate system, and for each point of the “900 × 900” image described above, Generate coordinate data (radial direction, angle, height). The learning management unit 210 records the generated two-dimensional face image in the learning memory as cylindrical coordinate system coordinate data D3.

<Face feature point extraction processing>
Next, face feature point extraction processing (step S1-2) will be described with reference to FIG. Here, the face feature point is a characteristic position (for example, the outermost point of the eyebrows, the innermost point of the eyebrows) in the face parts (eyebrows, eyes, nose, mouth, ears, cheeks, lower jaw, etc.) constituting the face , Mouth corner point, etc.). In the present embodiment, 33 face feature points are used. The face feature points may be arbitrarily added / deleted by the user. In this case, processing described later is performed using the added / deleted face feature points.

FIG. 6 shows the facial feature points (32) used in this embodiment with numbers. The feature point number “33” is calculated as “the midpoint of a straight line connecting the center of gravity of the feature points of both eyes and the nose vertex” using the other feature points.

The learning management unit 210 of the control unit 21 specifies face feature points from the generated cylindrical coordinate system coordinates, and calculates the position (coordinates). In the present embodiment, automatic extraction is performed using a well-known AAM (Active Appearance Models) method used for facial expression tracking, facial recognition, and the like. In this AAM method, a target object (here, a face) is modeled with a finite number of vertices, and this model is fitted to an input image to extract feature points of the target object.

The learning management unit 210 displays a face image in which the extracted face feature points are associated with the extraction position on the output unit 15. In this case, the position of each face feature point is movably arranged.
Here, the person in charge confirms the position of the facial feature point on the facial image displayed on the output unit 15 and corrects it if necessary. When face feature point confirmation or correction completion input is performed on the face image, the learning management unit 210 associates the cylindrical coordinate system coordinates of each face feature point with the number of each face feature point. D4 is generated and stored in the learning memory.

<Normalization processing of facial feature points>
Next, the facial feature point normalization process (step S1-3) will be described with reference to FIG.
First, the learning management unit 210 of the control unit 21 executes normalization processing by multiple regression analysis using the extracted face feature point data (step S3-1). Here, rotation on the XYZ axes is obtained from the feature points of the face by multiple regression analysis, and the face orientation is matched. The size of the face is normalized so that the distance between the centers of the eyes becomes a predetermined value (64 mm in this embodiment). Details of this processing will be described later with reference to FIG.

Next, the learning management unit 210 of the control unit 21 executes an average feature point calculation process (step S3-2). Specifically, the learning management unit 210 calculates the average coordinates of each feature point using the coordinates of 33 face feature point data for each learning target person. Thereby, the coordinates of each feature point in the average face of the learning subject are calculated.

Next, the learning management unit 210 of the control unit 21 executes normalization processing by procrustes analysis (step S3-3). Specifically, the learning management unit 210 uses the least square method to minimize the sum of the square distance between the average coordinates calculated in step S3-2 and each facial feature point. Move, rotate, and resize. In this case, 25 points (feature point numbers 1 to 6, 10, 14 to 22, 24 in FIG. 6) excluding the tragus points (feature point numbers 7 and 13), mandibular corner points (feature point numbers 8 and 12), and the like. To 27 and 28 to 32) are used. As a result, the facial feature points are adjusted to be close to the average face.

<Normalization processing by multiple regression analysis>
Next, normalization processing (step S3-1) by multiple regression analysis will be described with reference to FIG. By this processing, the face orientation is matched and the face size is normalized.

First, the learning management unit 210 of the control unit 21 executes the process of specifying the center of gravity of the feature points of both eyes (step S4-1). Specifically, the learning management unit 210 identifies facial feature points related to eyes in the facial feature point data. Next, the learning management unit 210 calculates the position of the center of gravity of both eyes using the center of gravity calculating means for the coordinates of the extracted face feature points. The learning management unit 210 specifies the calculated position of the center of gravity of both eyes as the origin.

Next, the learning management unit 210 of the control unit 21 executes an inclination correction process around the X axis and the Y axis (step S4-2). Specifically, the learning management unit 210 sets the Z coordinate as an objective variable and the X and Y coordinates as explanatory variables for a set of face feature points excluding the face outline and nose vertex with the position of the center of gravity of both eyes as the origin. Multiple regression analysis.

Here, as shown in FIG. 8B, a regression plane RPS is calculated by multiple regression analysis. The learning management unit 210 rotates the regression plane RPS about the X and Y axes so that the calculated normal line NV of the regression plane RPS is parallel to the Z axis.

Next, the learning management unit 210 of the control unit 21 executes an inclination correction process around the Z axis (step S4-3). Specifically, the learning management unit 210 uses a set of face feature points for calculating a face centerline. In this embodiment, a set of the center of gravity of the eyes, the apex of the nose, the lower end of the nose, the upper end of the upper lip, the lower end of the lower lip, and the chin tip coordinates are used as the facial feature points.

As shown in FIG. 8C, a regression line RL is calculated for this set using the Y coordinate as an objective variable and the X coordinate as an explanatory variable. The learning management unit 210 rotates around the Z axis so that the calculated slope of the regression line RL is parallel to the Y axis.

Next, the learning management unit 210 of the control unit 21 executes scaling processing (step S4-4). Specifically, the learning management unit 210 performs enlargement or reduction so that the distance between the centers of the eyes is 64 mm.

<Shape homology modeling process>
Next, the shape homology modeling process in the homology modeling process (step S1-4) will be described with reference to FIG.

First, the learning management unit 210 of the control unit 21 executes a process of matching facial feature point coordinates (step S5-1). Specifically, the learning management unit 210 uses the generic model mesh point identification information stored in the memory to match the coordinates of each normalized facial feature point with the identified facial feature point of the mesh point.

Next, the learning management unit 210 of the control unit 21 executes a shape fitting process (step S5-2). Specifically, the learning management unit 210 matches the shape of each facial part in the generic model in which each facial feature point is matched with the normalized shape of each facial part.

Next, the learning management unit 210 of the control unit 21 executes a triangulation process (step S5-3). Specifically, the learning management unit 210 calculates the coordinates of each mesh point corresponding to each polygon (triangle) of the generic model in the normalized shape of each face part matched with the shape of the generic model ( The shape homology model) is stored in the learning memory.

As shown in FIG. 9B, the number of mesh points can be reduced by using a mesh model in which the mesh around the face parts such as eyes, nose and mouth is made fine and the other area meshes are widened. it can. Note that the forehead portion is deleted because the presence of bangs adversely affects the statistical processing.

<Texture homology modeling>
Next, the texture homology modeling process will be described with reference to FIG.
First, the learning management unit 210 of the control unit 21 executes an average calculation process for the coordinates of each vertex of the normalized mesh model (step S6-1). Specifically, the learning management unit 210 calculates the average coordinates of each mesh point (vertex) from the normalized coordinates of each face feature point using a texture average calculation rule stored in advance.

Next, the learning management unit 210 of the control unit 21 executes a process of transforming the texture on the cylindrical coordinate system two-dimensional polygon into an averaged two-dimensional polygon (step S6-2). Specifically, the learning management unit 210 transforms the texture (pixel information) on the polygon of each two-dimensional face data in the cylindrical coordinate system into the average coordinates calculated in step S6-1, and the texture at the deformed average coordinates. Is stored in the learning memory.

The learning management unit 210 of the control unit 21 calculates the texture average at each average coordinate, thereby calculating a texture homology model and stores it in the learning memory.
FIG. 10B shows an average face obtained by averaging textures transformed into average coordinates of each mesh model.

<Generation processing of normalized cylindrical coordinate system image>
Next, a process for generating a normalized cylindrical coordinate system image (step S1-5) will be described.
The cylindrical coordinate system image generated in step S2-2 cannot be analyzed as it is because the position of the face parts (eyes, nose, mouth, etc.) differs for each data. Therefore, the cylindrical coordinate system image is normalized so that the positions of the face parts of each data are aligned.

Here, the image normalization mesh model used for normalization will be described. This image normalization mesh model uses 33 face feature points and pastes the mesh in a grid pattern on a cylindrical coordinate system.

Here, as shown in FIG. 11 (a), a mesh model having 5588 mesh points and 10862 polygons (triangles) is used.
The learning management unit 210 of the control unit 21 calculates the average value of the image normalized mesh model and the average value of the texture of each polygon for all data.

As a result, as shown in FIG. 11B, an average face is generated from the calculated texture of the average value.
When each polygon is deformed so that the input cylindrical coordinate system image matches the average mesh, the mesh constituting the face matches the average face mesh. Therefore, the cylindrical coordinate system image is normalized.

<An angle learning process for a two-dimensional face image>
Next, the angle learning process of the two-dimensional face image will be described with reference to FIG. In this process, a model for estimating an angle (face orientation) at which a face is photographed from a two-dimensional face image is generated. This angle learning process is executed using a homologous model of three-dimensional face data (a total of N) recorded in the aging data storage unit 22 and the snapshot data storage unit 23. In the three-dimensional face data recorded in the aging data storage unit 22, a homologous model of only the three-dimensional face data after aging is used.

First, the first learning unit 211 of the control unit 21 repeatedly executes the following processing for each predetermined processing target angle.
First, the first learning unit 211 of the control unit 21 performs a rotation process to a specified angle (step S7-1). Specifically, the first learning unit 211 rotates the three-dimensional homologous model to a predetermined target angle. The first learning unit 211 stores the rotation angle when rotated to the predetermined target angle in the learning memory.

Next, the first learning unit 211 of the control unit 21 executes a conversion process from the three-dimensional homologous model to the two-dimensional homologous model (step S7-2). Specifically, the learning management unit 210 generates a two-dimensional homology model by projecting the rotated three-dimensional homology model onto the XY plane.

Next, the first learning unit 211 of the control unit 21 executes a two-dimensional feature point specifying process (step S7-3). Specifically, the first learning unit 211 of the control unit 21 specifies coordinates corresponding to face feature points in the three-dimensional homology model in the calculated two-dimensional homology model. The first learning unit 211 stores the identified face feature points in the learning memory as two-dimensional feature points.

Next, the first learning unit 211 of the control unit 21 performs a process of excluding feature points hidden behind the face (step S7-4). Specifically, the first learning unit 211 includes a facial feature point on the photographing side (viewpoint side) and a facial feature point on the back side in the three-dimensional homologous model among the feature points of the two-dimensional homologous model. Identify The first learning unit 211 deletes the face feature points on the back side from the learning memory, and stores only the two-dimensional feature points on the photographing side.

The above processing is executed by repeating a loop for each angle to be processed.
The principal component analysis unit 214a of the control unit 21 executes the principal component analysis process using the two-dimensional feature points specified by the above-described repetition process (step S7-5). Specifically, the principal component analysis unit 214a performs principal component analysis on the two-dimensional feature points after rotation of each data i (i = 1 to N × number of processing target angles). In this case, the two-dimensional feature points are indicated as follows.

Next, the machine learning unit 214b of the control unit 21 executes machine learning processing (step S7-6). Specifically, the machine learning unit 214b uses “rotation angle (θ, ω)” as a dependent variable (feature to be predicted) and “two-dimensional all data” as an explanatory variable (feature used in prediction). "The principal component score of the feature point divided by the standard deviation" is used. The machine learning unit 214b executes machine learning processing using the dependent variable and the explanatory variable. The first learning unit 211 records the angle prediction model generated by the machine learning unit 214b in the model storage unit 25.

<Machine learning process>
Here, the machine learning process will be described with reference to FIGS. 13 and 14. In this machine learning process, another feature vector y (a feature amount used for prediction, which is an explanatory variable) is predicted from a certain feature vector x (a prediction target feature amount that is a dependent variable). In this case, a model for predicting y _{i, j} from x _{s (n), j is obtained} by learning the relationship between y and x using multiple regression analysis. Specifically, “a _{i, s (n)} ” and “b _i ” in the following equation are calculated.

When performing multiple regression analysis, it is important to select independent variables. If “y” and “x” having high correlation are selected in order, only those having high correlation between variables are selected, and there is a possibility that independence between variables cannot be maintained. Therefore, an algorithm for selecting a variable having high validity (t value) and independence (correlation coefficient is equal to or less than the maximum correlation coefficient Cmax) and high reliability (bic: Bayes information criterion) of the entire prediction model Is used. Hereinafter, in this embodiment, processing in an algorithm generally called “variable increase / decrease method” (“stepwise method”) will be described.

As shown in FIG. 13, first, the machine learning unit 214b of the control unit 21 executes an initial value setting process (step S8-1). Specifically, the machine learning unit 214b initializes a very large value to the minimum value (bic_min) of the Bayes information amount criterion stored in the memory, and resets the variable set (select_id) to an empty set.

Next, the machine learning unit 214b of the control unit 21 executes a setting process for the current Bayes information amount standard (step S8-2). Specifically, the machine learning unit 214b substitutes the minimum value (bic_min) based on the Bayes information amount into the minimum value (bic_min_pre) based on the current Bayes information amount.

主成分 Principal component number i to be processed is identified sequentially, and the following repeated processing is executed. “I” is a dimension number selected as a processing target. In this iterative process, it is determined whether or not the principal component number i to be processed is a component to be added to the variable set (addition target component).

Here, first, the machine learning unit 214b of the control unit 21 performs a determination process as to whether or not the minimum value of the correlation between the principal component number i and the variable set (select_id) is smaller than the maximum correlation coefficient Cmax (step). S8-3). Specifically, the machine learning unit 214b of the control unit 21 calculates a correlation coefficient between the principal component number i to be processed and the variable set (select_id) stored in the memory. The machine learning unit 214b compares the calculated correlation coefficient with the maximum correlation coefficient Cmax.

If the minimum correlation value is smaller than the maximum correlation coefficient Cmax (“YES” in step S8-3), the machine learning unit 214b of the control unit 21 adds an additional target component to the variable set (select_id). The multiple regression analysis is performed using the obtained variables, and the Bayes information criterion and the t value calculation process of the added variable are executed (step S8-4). Specifically, the machine learning unit 214b of the control unit 21 calculates a Bayesian information amount criterion by performing multiple regression analysis using a variable obtained by adding the principal component number i to be processed to the variable set stored in the memory. The t value is calculated by a known t test.

Next, the machine learning unit 214b of the control unit 21 performs a determination process as to whether or not the minimum value and t value of the Bayes information criterion satisfy the condition (step S8-5). Here, as the conditions, the calculated minimum value of the Bayes information amount criterion is larger than the current Bayes information amount criterion, and the t value is “2” or more.

Here, when the condition of the minimum value and t value of the Bayes information amount criterion is satisfied (in the case of “YES” in step S8-5), the machine learning unit 214b of the control unit 21 sets the minimum value of the Bayes information amount criterion and Update processing of the principal component number is executed (step S8-6). Specifically, the machine learning unit 214b of the control unit 21 substitutes the Bayes information criterion (bic) for the minimum value (bic_min) of the Bayes information criterion. Furthermore, the main component number i to be processed is stored as an additional target component (add_id).

On the other hand, when the minimum value of the correlation between the principal component number i and the variable set (select_id) is greater than or equal to the maximum correlation coefficient Cmax (“NO” in step S8-3), the machine learning unit 214b of the control unit 21 Steps S8-4 to S8-6 are skipped.

If either the minimum value of the Bayes information criterion or the t value does not satisfy the condition (“NO” in step S8-5), the machine learning unit 214b of the control unit 21 performs the process in step S8-6. To skip.

When the iterative process (steps S8-3 to S8-6) is completed for all principal component numbers i, the machine learning unit 214b of the control unit 21 determines whether or not the minimum value of the Bayes information amount criterion has been updated. Processing is executed (step S8-7). Specifically, the machine learning unit 214b determines whether or not the minimum value (bic_min) of the Bayes information amount reference matches the current minimum value (bic_min_pre) of the Bayes information amount reference set in step S8-2. Determine. The machine learning unit 214b determines that the minimum value of the Bayes information criterion is not updated when they match, and determines that the minimum value of the Bayes information criterion is updated when they do not match.

Here, when the minimum value of the Bayes information criterion is updated (in the case of “YES” in step S8-7), the machine learning unit 214b of the control unit 21 executes variable update processing (step S8-8). . Details of this variable update processing will be described with reference to FIG.

Next, the machine learning unit 214b of the control unit 21 executes a process for determining whether or not the variable update is successful (step S8-9). Specifically, the machine learning unit 214b determines the variable update success based on the flags (variable update success flag, variable update failure flag) recorded in the memory in the variable update process described later.

Here, when the variable update success flag is recorded in the memory and it is determined that the variable update is successful (in the case of “YES” in step S8-9), the machine learning unit 214b of the control unit 21 performs step S8— Steps 2 and after are repeated.

On the other hand, when the minimum value of the Bayes information criterion is not updated, or when the variable update failure flag is recorded and it is determined that the variable update is not successful ("NO" in steps S8-7 and S8-9) In this case, the machine learning unit 214b of the control unit 21 ends the machine learning process.

<Variable update processing>
Next, the variable update process (step S8-8) will be described with reference to FIG. In this process, it is determined whether or not the variable set including the addition target component is valid. When the invalid variable set is obtained by deleting the invalid variable, this variable set is specified as the explanatory variable.

First, the machine learning unit 214b executes a new variable set setting process (step S9-1). Specifically, the machine learning unit 214b adds a component to be added (add_id) to the variable set stored in the memory to generate a new variable set (select_id_new).

Next, the machine learning unit 214b repeats the following steps S9-2 to S9-4 in an infinite loop.
First, the machine learning unit 214b performs a multiple regression analysis using a new variable set (select_id_new), and executes a Bayes information criterion (bic) and t value calculation processing for all variables (step S9-2). . Specifically, the machine learning unit 214b calculates a Bayes information criterion by multiple regression analysis using a new variable set. Further, the t value is calculated by a known t test.

Next, the machine learning unit 214b determines whether or not the minimum t value among the t values of each variable of the new variable set is smaller than 2 (step S9-3).
Here, when the minimum t value is smaller than “2” (in the case of “YES” in step S9-3), the machine learning unit 214b executes the process of deleting the variable having the minimum t value from the new variable set. (Step S9-4). Specifically, the machine learning unit 214b deletes the variable for which the minimum t value has been calculated in the new variable set from the new variable set.

The processes after step S9-2 are repeatedly executed.
On the other hand, when the t values of the variables included in the new variable set are all 2 or more, and the minimum t value is 2 or more (in the case of “NO” in step S9-3), the machine learning unit 214b End this infinite loop. The machine learning unit 214b executes a process of determining whether or not the Bayes information amount criterion is smaller than the current minimum value of the Bayes information amount criterion (step S9-5). Specifically, the machine learning unit 214b determines whether or not the Bayes information amount criterion (bic) is smaller than the current minimum value (bic_min_pre) of the Bayes information amount criterion set in step S8-2.

Here, when the Bayes information criterion is smaller than the current minimum value of the Bayes information criterion (“YES” in Step S9-5), the machine learning unit 214b executes a variable update success process (Step S9). -6). Specifically, the machine learning unit 214b substitutes a new variable set (select_id_new) for the variable set (select_id). Further, the machine learning unit 214b records a variable update success flag in the memory.

On the other hand, when the Bayes information criterion is equal to or greater than the current minimum value of the Bayes information criterion (in the case of “NO” in Step S9-5), the machine learning unit 214b executes variable update failure processing (Step S9—). 7). Specifically, the machine learning unit 214b records a variable update failure flag in the memory.

<Learning process from 2D to 3D>
Next, a first learning process for converting a two-dimensional face image into three-dimensional face data will be described with reference to FIG. Here, when there is an arbitrary number of two-dimensional face images in the same scene, or when there is a two-dimensional face image in which a face is photographed from an arbitrary angle, learning to enable accurate prediction is performed. Do. In that case, the number of combinations becomes enormous, and the capacity of the database becomes a problem. Therefore, only the combinations specified in advance (for example, front and side surfaces) are learned, and in the case of other combinations, the model is learned each time. Create

First, the principal component analysis unit 214a of the control unit 21 performs a three-dimensional shape principal component analysis process in advance using a three-dimensional mesh (homology model) (step S10-1). Specifically, the principal component analysis unit 214a performs principal component analysis on the three-dimensional mesh points of each data. As a result, the two-dimensional mesh point is expressed by the following equation when expressed by an average value, a principal component score, and a principal component vector.

The first learning unit 211 of the control unit 21 executes the rotation process to the specified angle (step S10-2). Specifically, the first learning unit 211 displays a screen for designating the rotation angle on the output unit 15. Here, for example, the front (0 degree) and the side (90 degrees) are designated. When the rotation angle is input, the first learning unit 211 rotates the three-dimensional homologous model according to the input rotation angle.

Next, the first learning unit 211 of the control unit 21 executes a process for generating a two-dimensional homologous model from the three-dimensional homologous model (step S10-3). Specifically, the first learning unit 211 generates a two-dimensional homology model by projecting the rotated three-dimensional homology model onto a two-dimensional plane (XY plane).

Next, the first learning unit 211 of the control unit 21 executes a two-dimensional image generation process (step S10-4). Here, it is assumed that a gray two-dimensional homology model is used. Specifically, the first learning unit 211 generates a grayed image based on the luminance in each mesh of the generated two-dimensional homologous model.

Next, the principal component analysis unit 214a of the control unit 21 executes a principal component analysis process of the two-dimensional image (step S10-5). Specifically, the principal component analysis unit 214a performs principal component analysis on the two-dimensional image generated in step S10-4 and expresses it as follows.

Next, the first learning unit 211 of the control unit 21 executes a two-dimensional feature point specifying process (step S10-6). Specifically, the first learning unit 211 specifies the coordinates of the facial feature points in the calculated two-dimensional homology model, as in step S7-3. The first learning unit 211 stores the specified coordinates in the memory.

Next, the first learning unit 211 of the control unit 21 executes a feature point exclusion process that hides behind the face (step S10-7). Specifically, the first learning unit 211 deletes the face feature points hidden behind the face from the memory, as in step S7-4.

Next, the principal component analysis unit 214a of the control unit 21 executes a principal component analysis process of the two-dimensional feature points (step S10-8). Specifically, the principal component analysis unit 214a executes principal component analysis processing using the facial feature points stored in the memory, as in step S7-5.

The machine learning unit 214b of the control unit 21 executes machine learning processing in the same manner as in step S7-6 (step S10-9). Specifically, the machine learning unit 214b executes machine learning processing using the dependent variable and the explanatory variable. Here, “the principal component score of the three-dimensional mesh point divided by the standard deviation” is used as the dependent variable, and “the two-dimensional feature point of all data and the principal component score of the two-dimensional image of all data are used as explanatory variables. “Divided by standard deviation” is used. The first learning unit 211 records the three-dimensional prediction model generated by the machine learning unit 214b in the model storage unit 25.

<2D → 3D prediction verification>
Next, with reference to FIGS. 16 and 17, a model data verification process used for conversion from two-dimensional to three-dimensional calculated by the first learning will be described.

15, principal component analysis was performed on 33 two-dimensional feature points (coordinates), two-dimensional face images, and three-dimensional mesh points. In this principal component analysis, the number of principal components with a cumulative contribution rate of 95% was calculated. The number of principal components with a cumulative contribution rate of 95% was “29” for the two-dimensional feature point (66 dimensions) and “60” for the three-dimensional mesh point (32223 dimensions). Thus, although the number of dimensions is quite different, the number of principal components is in the same order. Since the two-dimensional face image (810000 dimensions) has a great individual difference, the number of principal components with a cumulative contribution rate of 95% increased to “226”.

In the correlation with the two-dimensional feature points, the maximum correlation coefficient is secured to approximately 0.2 or more for the principal components up to the 100th, but decreases to a value having little correlation in the principal components after the 200th. . On the other hand, in the correlation with the image, the correlation tends to be small as compared with the two-dimensional feature point, but the maximum correlation coefficient is secured to about 0.1 in the 200th and subsequent principal components.

This can be understood as follows. There are only 33 two-dimensional feature points, and a small principal component number (a component indicating a rough feature) has a high correlation, but a correlation with a large principal component number (a component indicating a fine shape) is low. On the other hand, an image has a relatively low correlation with a small principal component number, but a relatively high correlation with a large principal component number. In the present embodiment, a feature amount obtained by combining a two-dimensional feature point and an image is used as a feature point having a higher maximum correlation value. Based on the number of principal components (explanatory variables) selected at the time of learning, it can be seen that the number of selected principal components is larger when two-dimensional feature points and images are combined.

Here, the validity and explanatory power of the obtained prediction model will be described.
The F value is a parameter indicating the validity of the model. The t value is a parameter indicating the validity of each variable. Although it is considered that the F value and the t value are each “2” or more, it is found that a value of “2” or more is secured in any component.

The coefficient of determination is a parameter indicating the explanatory power of the model, and the value indicates the rate at which the model explains the prediction target data. Specifically, when the value is “1”, it is “all predictable”, and when the value is “0”, it indicates “not predicted at all”. In the case of only two-dimensional feature points, it has been found that the coefficient of determination is ensured to be approximately 50% or more for the main components up to the 40th, but less than 20% for the main components near the 100th. In the case of two-dimensional feature points and images, it was found that the coefficient of determination was ensured to be approximately 50% or more for the principal components up to the 50th, and exceeded 20% even for the principal component near the 100th. As a result, it can be seen that the accuracy in the case of using the two-dimensional feature point and the image is improved as compared with the case of only the two-dimensional feature point.

Next, in order to verify the validity of the prediction model data to be converted from 2D to 3D, 100 principal component scores P3 of 3D mesh points (the number of cumulative contributions 97.5%) are predicted. Perform multiple regression analysis. In this case, the maximum correlation coefficient Cmax is set to “0.15” as a variable selection criterion.

Next, using FIG. 16, the validity verification processing of the prediction model data to be converted from the two dimensions to the three dimensions is executed. Here, in the following processing, the processing target data j (j = 1 to n) is sequentially specified, and the following processing is repeated.

First, the first learning unit 211 of the control unit 21 executes a process of creating a prediction model machine-learned with the remaining [n-1] human data excluding the processing target data j (step S11-1). Specifically, the first learning unit 211 generates the three-dimensional conversion model by executing the first learning process described above using the data of [n−1] people.

Next, the first learning unit 211 of the control unit 21 performs an estimation process using a prediction model for the three-dimensional mesh point of the processing target data j (step S11-2). Specifically, the first learning unit 211 uses the processing target data j as input data and applies the generated three-dimensional conversion model to calculate a three-dimensional mesh point.

Next, the first learning unit 211 of the control unit 21 performs a comparison process between the three-dimensional data (correct answer) of the processing target data j and the estimated result (step S11-3). Specifically, the first learning unit 211 compares the three-dimensional face data generated in step S11-1 with the three-dimensional face data of the processing target data j. As a result of the comparison, the shift amount of each point of the three-dimensional mesh is recorded in the memory. In this case, the average prediction error of the principal component scores was less than “0.22”. Since the variance of the principal component score is normalized to “1”, it can be seen that the estimation can be performed with high accuracy. It has also been found that the prediction using only the two-dimensional feature points and the prediction using the two-dimensional image can improve accuracy.

Furthermore, the average prediction error of the 3D mesh points is “1.546391 mm” when only 2D feature points are used, and “1.477514 mm” when 2D feature points and 2D images are used. . In this case as well, it has been found that the prediction with only the two-dimensional feature points and the prediction with the two-dimensional image can improve accuracy.

FIG. 17A shows a two-dimensional face image (input data) before aging. FIG. 17B is a face image (correct answer) after 10 years of the face image shown in FIG.
FIG. 17C shows an aging face image predicted using only the two-dimensional feature points of the face image shown in FIG. FIG. 17D is an aging face image predicted using the two-dimensional feature points and images of the face image shown in FIG.

It can be confirmed that the prediction with the two-dimensional feature point and the two-dimensional image can be predicted with higher accuracy than the prediction with only the two-dimensional feature point.
<Texture aging learning process>
Next, the second learning process for texture aging is executed using FIG. In this second learning process, a texture aging process using principal component analysis and a texture aging process using wavelet transform are executed. Here, after explaining the texture aging process using principal component analysis, the texture aging process using wavelet transform will be described.

<Aging process of texture using principal component analysis>
Here, using the normalized cylindrical coordinate system image generated in step S1-5, a model for predicting the texture change due to aging in the three-dimensional face data is calculated by machine learning.

First, the control unit 21 executes principal component analysis processing of the normalized cylindrical coordinate system image (step S12-1). Specifically, the second learning unit 212 of the control unit 21 acquires aging data from the aging data storage unit 22 and snapshot data from the snapshot data storage unit 23. The principal component analysis unit 214a of the control unit 21 performs principal component analysis on a cylindrical coordinate system image (a cylindrical coordinate system image of acquired secular change data and snapshot data). In this case, the principal component analysis unit 214a determines the direction of the principal component vector using the snapshot data before (or after) the aging data, and uses the aging data to determine the principal component. Score is calculated. Here, each data is expressed as an average value, a principal component score, and a principal component vector as follows.

Here, “j” is an aging index, and “j” = 1 after aging and “j” = 0 before aging.

Next, the control unit 21 executes machine learning processing as in step S7-6 (step S12-2). Specifically, the machine learning unit 214b of the control unit 21 uses “an aging change difference vector of texture principal component scores normalized per unit year” as a dependent variable, and “pre-aging” as an explanatory variable. The “principal score of texture” is used. The second learning unit 212 records the texture aging model generated by the machine learning unit 214b in the model storage unit 25.

<Predictive verification of texture aging using principal component analysis>
Next, processing for converting to aging in a texture using principal component analysis will be described with reference to FIG.

The cumulative contribution ratio up to the 35th principal component calculated in texture aging using the principal component analysis shown in FIG. 18 exceeds 95%, and the contribution ratio of the 25th and subsequent principal components is 0.1%. Is less than. Each main component is shown in FIG. It can be seen that the higher the contribution ratio, the higher the frequency component.

FIG. 21 shows an image reconstructed by changing the upper limit of the principal component of two images for confirming the contribution of each principal component with the image. As a result, it can be seen that details such as spots and wrinkles cannot be reproduced unless all the main components are considered.

As shown in FIG. 19, the validity verification process of the texture aging prediction model data using this principal component analysis is executed. Here, the following processing is repeatedly executed for the processing target data j (j = 1 to n).

First, the control unit 21 executes a process for creating a prediction model that has been machine-learned with the data of the remaining [n−1] people excluding the data j (step S13-1). Specifically, the second learning unit 212 of the control unit 21 executes the second learning process in steps S12-1 to S12-2 using [n-1] data, thereby aging the texture. Generate a conversion model of.

Next, the control unit 21 executes an aging process using the prediction model using the pre-aging data of the data j (step S13-2). Specifically, the second learning unit 212 of the control unit 21 uses the data j before aging as input data, applies the generated conversion model to texture aging, and applies the three-dimensional mesh points after aging. Is calculated.

Next, the control unit 21 executes a comparison process between the post-aging data (correct answer) of the data j and the aging result (step S13-3). Specifically, the second learning unit 212 of the control unit 21 performs post-aging change of the three-dimensional face data after aging generated in step S13-2 and the data j stored in the aging data storage unit 22. And the error of each point of the three-dimensional mesh is calculated. In this case, the calculated error was found to be about 60% or less.

<Aging process for textures using wavelet transform>
Next, texture aging processing using wavelet transform will be described with reference to FIG. In the texture aging process using the principal component analysis described above, aging difference data is estimated. However, when there are already existing spots and wrinkles, aging using principal component analysis does not increase them. Therefore, in order to age using existing stains and wrinkles, aging change estimation using wavelet transform is performed.

Here, as shown in FIG. 22A, the control unit 21 performs a calculation process of an increase rate (wavelet coefficient Ri) due to aging of the wavelet component (step S14-1). Specifically, the second learning unit 212 of the control unit 21 extracts the aging data stored in the aging data storage unit 22. The second learning unit 212 calculates all the wavelet coefficients Ri of each two-dimensional image with the data number j for each wavelet coefficient number i (for each pixel). The second learning unit 212 sums up the calculated wavelet coefficients Ri (values for each pixel) of the calculated image data before aging. The second learning unit 212 sums up the calculated wavelet coefficients Ri (values for each pixel) of the image data after aging. By dividing the wavelet coefficient Ri (value per pixel) in the summed image after aging by the wavelet coefficient Ri (value per pixel) in the summed image before aging, the wavelet coefficient Ri of each data is calculated. Calculate the rate of change. In this case, when the wavelet coefficient Ri is less than 1, the second learning unit 212 calculates the rate of change as “1”.

In the expression shown in FIG. 22A, i represents each wavelet coefficient number, j represents a data number, a represents after aging, and b represents before aging.
As a result, the maximum value of the wavelet coefficient Ri was “5.407101”, and the average value of the wavelet coefficient Ri was “1.311112”. Note that FIG. 22B shows an image displayed by visualizing the wavelet coefficient Ri. In this image, black indicates a wavelet coefficient Ri having a minimum value of “1”, and the whiter the value, the larger the value. In addition, in FIG. 22B, the image shows the low frequency component as it goes to the upper left. Specifically, one-dimensional wavelet transform in the horizontal direction is performed for each row to separate the low-frequency component and the high-frequency component, and one-dimensional conversion in the vertical direction is performed on each column of the converted signal. The image which repeated is shown.

<3D shape aging learning process>
Next, a third learning process for aging the three-dimensional shape is executed using FIG. In this case, a model for predicting a shape change due to aging in a three-dimensional face image is calculated by machine learning using the homology model generated in the above-described shape homology modeling process. In this case, the maximum correlation coefficient Cmax between the selected variables is set to “0.15”.

Here, first, the control unit 21 executes the calculation process of the principal component score of the three-dimensional mesh (step S15-1). Specifically, the third learning unit 213 of the control unit 21 extracts the aging data stored in the aging data storage unit 22. Here, 144 pieces of aging data are extracted. The third learning unit 213 uses the principal component vector of the three-dimensional mesh generated in the principal component analysis of the three-dimensional mesh point in step S10-1 described above, and uses the three-dimensional mesh principal component score for the extracted secular change data. Is calculated.

Next, the control unit 21 executes machine learning processing as in step S7-6 (step S15-2). Specifically, the machine learning unit 214b of the control unit 21 executes machine learning processing using the dependent variable and the explanatory variable. Here, “the aging change difference vector of the three-dimensional mesh principal component score normalized per unit year” is used as the dependent variable, and “the principal component score of the three-dimensional mesh before aging” is used as the explanatory variable. The third learning unit 213 records the shape aging model generated by the machine learning unit 214b in the model storage unit 25.

The maximum value of the correlation coefficient between the aging change difference vector calculated in this way and the principal component score has a correlation of about “0.3”, and there is a constant correlation between the aging change and the principal component score. Therefore, it is appropriate to use for regression analysis. The number of selected variables is around 30, which is reasonable when compared with the number of secular change data used for calculation. Further, it is found that the F value is 10 or more, the t value is 2 or more, and the determination coefficient is almost 70% or more in any principal component, and it is understood that the accuracy of the model is high.

<Aging prediction process>
Next, the aging prediction process will be described with reference to FIG.
First, the control unit 21 executes feature point extraction processing (step S16-1). Specifically, the aging management unit 215 of the control unit 21 executes face feature point extraction processing from the processing target two-dimensional face image data in the same manner as in step S1-2.

Next, the control unit 21 executes face orientation extraction processing (step S16-2). Specifically, the first processing unit 216 of the control unit 21 uses the angle prediction model stored in the model storage unit 25 to specify the direction in which the face was photographed from the coordinates of the extracted face feature points, A two-dimensional face image is converted to the front direction.

Next, the control unit 21 executes a three-dimensional meshing process (step S16-3). Specifically, the aging management unit 215 of the control unit 21 generates a three-dimensional mesh for a front-facing two-dimensional face image using the three-dimensional prediction model stored in the model storage unit 25. .

The control unit 21 executes a process for generating a normalized cylindrical coordinate system image (step S16-4). Specifically, the aging management unit 215 of the control unit 21 uses the prediction model calculated in step S16-3 to create a two-dimensional mesh of the processing target two-dimensional face image. The aging management unit 215 creates an image in each polygon by affine transformation into a polygon in cylindrical coordinate system coordinates. Here, the image information in the polygon may be insufficient depending on the face orientation of the two-dimensional face image to be processed. In this case, the aging management unit 215 assumes that the left and right sides of the face are symmetric, and uses the center line of the left and right sides of the face as a symmetric line, and converts the polygons with insufficient image information into the polygons on the opposite left and right sides. Complement using image information.

Next, the control unit 21 executes an aging process for the three-dimensional mesh (step S16-5). Specifically, the third processing unit 218 of the control unit 21 inputs the three-dimensional mesh generated in step S16-3 to the shape aging model stored in the model storage unit 25, and ages the three-dimensional mesh. To do. In this case, the third processing unit 218 performs aging only on regions other than the shape prediction mask region.

Here, the white portion shown in FIG. 26A is used as the shape prediction mask region. These portions include cheeks and nose muscles, and are regions where there is little change in shape due to aging.
Next, the control unit 21 executes a texture aging process (step S16-6). Details of this processing will be described later with reference to FIG.

Next, the control unit 21 executes an aging three-dimensional face image generation process (step S16-7). Specifically, the third processing unit 218 of the control unit 21 synthesizes the aged images generated in steps S16-5 and S16-6, and generates an image in which the shape and texture are aged.

FIG. 27 (a) displays 30-year-old face image data. FIGS. 27B and 27C show images calculated by the control unit 21 using the face image data as input data and performing aging prediction after 10 years and 15 years later. Here, the conversion from the two-dimensional to the three-dimensional is not performed, but it can be seen that it is reasonably aged.

Next, the control unit 21 executes an aging two-dimensional face image generation process (step S16-8). Specifically, the aging management unit 215 of the control unit 21 rotates the generated aging three-dimensional face image so that the face direction specified in step S16-2 is oriented, and the two-dimensional face at that time Generate an image. The aging management unit 215 displays the generated two-dimensional face image on the display.

<Aging treatment of texture>
Next, the texture aging process (step S16-6) described above will be described with reference to FIGS.

First, the second processing unit 217 of the control unit 21 executes a wavelet transform process (step S17-1). Specifically, the second processing unit 217 calculates a wavelet coefficient R1i obtained by wavelet transforming the post-age image Ii using principal component analysis.

The following processing is repeated for the wavelet coefficient number i.
First, the second processing unit 217 of the control unit 21 compares the absolute values of the two wavelet coefficients, and executes a magnitude relation determination process (step S17-2). Here, the absolute value of the wavelet coefficient R1i based on the principal component analysis is compared with the value obtained by multiplying the absolute value of the wavelet coefficient R2i calculated by the texture aging process using the wavelet transform by the weighting coefficient w. It is determined whether or not the absolute value of the wavelet coefficient R1i is larger than a value obtained by multiplying the absolute value of the wavelet coefficient R2i by the weighting coefficient w.

Here, when the absolute value of the wavelet coefficient R1i is larger than the value obtained by multiplying the absolute value of the wavelet coefficient R2i by the weighting coefficient w (in the case of “YES” in step S17-2), the second processing unit 217 of the control unit 21. Executes a process of substituting the wavelet coefficient R1i into the wavelet coefficient R3i to be used (step S17-3).

On the other hand, when the absolute value of the wavelet coefficient R1i is equal to or smaller than the value obtained by multiplying the absolute value of the wavelet coefficient R2i by the weighting coefficient w (in the case of “NO” in step S17-2), the second processing unit 217 of the control unit 21 Then, a process of substituting the wavelet coefficient R2i into the wavelet coefficient R3i to be used is executed (step S17-4).

The above processing is repeated for the wavelet coefficient number i.
The second processing unit 217 of the control unit 21 executes wavelet inverse transformation processing of the wavelet coefficient R3 to be used (step S17-5).

Next, the second processing unit 217 of the control unit 21 performs a mask area reflection process (step S17-6). Specifically, the second processing unit 217 performs aging change only on regions other than the texture prediction mask region.

Here, the white portion shown in FIG. 26B is used as the texture prediction mask region. These portions include the eyes, nose, mouth, and the like, and are regions where there is little change in texture due to aging.

Through the above processing, an image in which the texture is aged is generated.
Therefore, according to the present embodiment, the following effects can be obtained.
(1) The control unit 21 of the present embodiment executes an aging management unit 215 that generates a face image after aging, and a second processing unit that performs a process of predicting a change due to aging for the texture (texture) of the face image. 217, a third processing unit 218 that predicts changes due to aging of the face shape. As a result, aging prediction is performed in consideration of aging of the face shape and changes due to aging of the texture of the face image, so that an aging face image can be generated more accurately.

(2) The learning management unit 210 of the aging prediction system 20 of the present embodiment uses the data recorded in the secular change data storage unit 22 and the snapshot data storage unit 23 to extract facial feature points, facial features A point normalization process and a homology modeling process are executed (steps S1-2 to S1-4). This makes it possible to generate a texture aging model and a shape aging model by using a plurality of actual sample data and sharing them in terms of anatomy.

(3) The control unit 21 of the present embodiment includes a first processing unit 216 that executes a process of generating three-dimensional face data from a processing target two-dimensional face image. Thereby, even if the orientation of the two-dimensional face image to be processed is not the front or the like, an aging face image can be generated with the face orientation at the specified angle.

(4) The first processing unit 216 of the aging prediction system 20 according to the present embodiment executes a two-dimensional face image angle learning process using a three-dimensional mesh (homology model). As a result, a three-dimensional prediction model for converting from two dimensions to three dimensions can be generated using actual sample data.

(5) The second processing unit 217 of the aging prediction system 20 of the present embodiment uses a texture aging model using principal component analysis and a texture aging model using wavelet (WAVELET) transformation, Execute texture aging processing. This makes it possible to more accurately generate an aged face image by using wavelet transformation that is aging using existing stains and wrinkles.

(6) The second processing unit 217 of the aging prediction system 20 of the present embodiment stores a weighting coefficient w that is a value that determines the importance of a model using principal component analysis and a model using wavelet transform. . Thereby, the weighting of the texture aging model by principal component analysis and the texture aging model by wavelet transform can be changed by changing the weighting coefficient w.

Moreover, you may change the said embodiment as follows.
The control unit 21 of the above embodiment uses the aging data stored in the aging data storage unit 22 and the snapshot data stored in the snapshot data storage unit 23 to use the texture aging model and the shape aging model. A model was generated. A texture aging model and a shape aging model may be generated for each attribute (for example, sex or age group) of secular change data or snapshot data. In this case, the control unit 21 uses the secular change data and the snapshot data having the same attribute to normalize the cylindrical coordinate system image D5, the normalized face feature point data D6, and the three-dimensional mesh data D7 ( Homology model). Using these, the control unit 21 generates a texture aging model and a shape aging model for each attribute, and stores them in the model storage unit 25 in association with each attribute information. In the aging prediction process, the control unit 21 acquires face attribute information included in this image together with the processing target two-dimensional face image data. The control part 21 performs an aging prediction process using the texture aging model and the shape aging model of the attribute corresponding to the acquired attribute information. Accordingly, more accurate face image data can be generated in consideration of the influence of texture and shape according to attributes such as sex and age group.

In the texture aging process, the control unit 21 of the above embodiment uses the wavelet coefficient R1i obtained by wavelet transforming the post-age image Ii using the principal component analysis, and the wavelet coefficient R2i of the texture aging model based on the wavelet transform. Was used alternatively. The texture aging process is not limited to the case where these two wavelet coefficients R1i and R2i are used alternatively, but the statistical values of these two wavelet coefficients R1i and R2i (for example, depending on the average value or attribute) (Combined value by ratio) may be used.

The control unit 21 of the above embodiment uses wavelet transform in the texture aging process. The aging process is not limited to wavelet transform. It is possible to use a method of deepening (emphasizing) the stains and wrinkles on the face texture. For example, the face texture may be aged using a known sharpening filter or Fourier transform.

-The control part 21 of the said embodiment produced | generated the aging model using the principal component analysis process. The analysis process used to generate the aging model is not limited to the principal component analysis process. Any analysis process can be used to identify variables that express individual differences. For example, an aging model can be generated using independent component analysis (ICA) or multidimensional scaling (MDS). You may do it.

In the above embodiment, the control unit 21 converts the two-dimensional face image to the front direction using the angle prediction model in the face direction estimation process (step S16-2) of the aging prediction process. The face direction estimation process (step S16-2) of the aging prediction process is not limited to the method using the angle prediction model. For example, the direction in which the face was photographed in the image may be specified using a known procrustes analysis.

In the above embodiment, the control unit 21 performs the first learning process for the conversion from the two-dimensional face image to the three-dimensional face data, the second learning process for the texture aging, and the third learning process for the three-dimensional shape aging. In the three learning processes, a machine learning process was executed. In this case, the machine learning unit 214b of the control unit 21 calculates an explanatory variable (a feature amount used at the time of prediction) using a dependent variable (a prediction target feature amount) by multiple regression analysis. Machine learning processing executed by the machine learning unit 214b of the control unit 21 is not limited to learning processing using multiple regression analysis, and other analysis methods may be used. For example, a coupling learning process, a learning process based on PLS regression (Partial Last Squares Regression), a learning process based on Support Vector Regression (Support Vector Regression; SVR), or the like may be performed.

Here, the coupling learning process will be described.
In this process, the machine learning unit 214b generates a single row vector (one-dimensional vector) by combining the “prediction target feature amount” and “feature amount used during prediction” of each sample data. For example, the principal component coefficient of “rotation angle (θ, ω)” is used as the “prediction target feature amount” in the first learning process, and “two-dimensional feature point of all data is used as the“ feature amount used during prediction ”. Principal component coefficients of “the principal component score divided by the standard deviation” are used.

The machine learning unit 214b generates a data matrix of combined patch vectors in which the generated one-dimensional vectors are arranged in the column direction for each sample data. A principal component analysis of the data matrix is performed to generate a principal component vector matrix. This principal component vector matrix is a matrix in which the row vectors are arranged in the order of principal components that change drastically between the “prediction target feature value” and the “feature value used for prediction”.

Next, the machine learning unit 214b executes orthogonalization processing of the principal component vector matrix. Here, the “feature value used for prediction” matrix is orthogonalized by the Gram Schmid method. In this case, the “prediction target feature amount” is converted by multiplying the orthogonalization coefficient in the “feature amount used during prediction”.

The machine learning unit 214b generates using the orthogonalized “features to be used for prediction” (matrix Di, j) and the “prediction target feature values” (matrix Ei, j) converted accordingly. The predicted model is recorded in the model storage unit 25.

In the prediction process, the control unit 21 calculates the coefficient bi representing the weight of the principal component by inner product of the input data Xj and the matrix Di, j stored in the model storage unit 25. Next, the control unit 21 reconstructs prediction data Yj, which is a prediction vector, using the coefficient bi and the matrix Ei, j. Thereby, the control part 21 can calculate the prediction data Yj based on the input data Xj.

By using this coupling learning process, it is possible to calculate prediction data in consideration of the overall balance while preferentially using components that change drastically and have a large influence.
Next, a learning process based on PLS regression will be described.

This PLS regression uses the covariance w _i of independent variables (features to be predicted) and explanatory variables (features used at the time of prediction), and adds multiple regression analysis to the variables in descending order of their correlation. By doing so, a regression coefficient matrix is calculated. Specifically, the following processes [1] to [4] are repeated until the intersection determination error is minimized. Here, the intersection determination error is an error between the prediction result and the prediction target when the sample data is divided into the prediction target and the input data, the prediction target is predicted using the input data.

(1) machine learning unit 214b calculates an independent variable (prediction target features), the covariance matrix of explanatory variables (features to be used for prediction) (correlation matrix) W _i. Here, the covariance matrix W _i of the independent variable and explanatory variables is calculated by the following equation.

W _i = X _i ^T Y _i / || X _i ^T Y _i ||
In this equation, T means a transposed matrix.
[2] Next, the machine learning unit 214b projects the independent variable X _i onto the space of the covariance w _i and calculates the score matrix ti.

[3] Next, the machine learning unit 214b executes an explanatory variable update process. Specifically, the control unit 21 calculates a regression coefficient matrix C that predicts the explanatory variable Y _{i + 1} from the score matrix t _i . The control unit 21 once deletes the information used for the regression, and specifies the remaining explanatory variables c _{i + 1} . In this case, the control unit 21 uses “c _i = Y _i ^T t _i (t _i ^T t _i ) ⁻¹ and Y _{i + 1} = Y _i −t _i c _i ^T ”.

[4] Next, the machine learning unit 214b executes an independent variable update process. Specifically, similarly to the update processing of the explanatory variables, the machine learning unit 214b calculates a regression coefficient matrix that predicts the independent variable from the score matrix, deletes information used for the regression once, and the remaining independent variables Is calculated. In this case, the control unit 21 uses “p _i = X _i ^T t _i (t _i ^T t _i ) ⁻¹ and X _{i + 1} = X _i −t _i p _i ^T ”.

[5] The machine learning unit 214b determines whether or not the intersection determination error is minimum. Specifically, first, the machine learning unit 214b assumes that a part of the learning data (for example, 1/4 of the entire learning data) is a prediction target, and uses the data excluding these prediction targets as input data. Then, using the explanatory variable Y _{i + 1} calculated in the process [4] and the independent variable X _{i + 1} calculated in the process [4], an error from the prediction target is calculated.

Here, when the calculated error is equal to or less than the error from the prediction target in the previous (Y _i , X _i ) process, the control unit 21 determines that the intersection determination error is not the minimum. In this case, the machine learning unit 214b uses the explanatory variable Y _{i + 1} as Y _i and the independent variable X _{i + 1} as X _i and repeatedly executes the processing [1] and subsequent steps.

On the other hand, when the calculated error becomes larger than the error with the prediction target in the previous process (Y _i , X _i ), the machine learning unit 214b performs the calculation calculated up to the previous process [1]. The covariance matrix W is generated by arranging the variances w _i in the horizontal direction. Further, the control unit 21 generates a matrix C by arranging the regression coefficients c _i calculated in the process [3] in the horizontal direction using the covariance matrix W, and uses the covariance matrix W in the process [4]. The calculated regression coefficient p _i is arranged in the horizontal direction to generate a matrix P. The control unit 21 calculates by substituting the regression coefficient matrix B into B = W (P ^T W) ⁻¹ C ^T. A prediction model generated using the regression coefficient matrix B is recorded in the model storage unit 25.

In the prediction process, the control unit 21 performs prediction using the input data Xj and the recorded regression coefficient matrix B.
By using this PLS regression analysis, it is possible to select more appropriate variables than in the multiple regression analysis, and therefore it is possible to make a more appropriate prediction.

Next, the learning process based on support vector regression analysis will be described. This support vector regression analysis is a nonlinear analysis and calculates a regression curve. Specifically, in the support vector regression analysis, the regression curve is calculated so that the sample data falls within the range (tube) of the regression curve ± predetermined distance w. The data that goes out of the tube is taken as penalty data ξ, and a curve that minimizes the following equation and a predetermined distance w are calculated.

Here, the adjustment constant C is a parameter for adjusting the allowable range of outliers. If this adjustment constant C is large, the allowable range becomes small. “Ξ ⁺ i” is “0” if the data i is in the tube, and is a value into which the distance from the tube is substituted if it is above the tube. “Ξ ⁻ i” is “0” if the data i is in the tube, and if it is below the tube, the distance to the tube is substituted.

学習 By using a learning process based on this support vector regression analysis, it can be applied to nonlinear (curve) regression. Therefore, the shape of the function can be set more freely than in the linear regression analysis of the multiple regression analysis or the PLS regression analysis of the above embodiment.

Claims

A shape aging model that predicts changes in facial shape due to aging,
A texture aging model that predicts changes in facial texture due to aging,
A model storage unit storing a three-dimensional prediction model for predicting three-dimensional data from a two-dimensional image;
An aging prediction system comprising a control unit configured to be connected to an input unit and an output unit and predicting aging,
The control unit is
Obtain a prediction target image from the input unit,
Extracting feature points of the prediction target image;
Using the extracted feature points, estimate the face direction in the prediction target image,
Generating first three-dimensional data based on the three-dimensional prediction model and the estimated face orientation;
Using the shape aging model, generating second three-dimensional data from the first three-dimensional data,
Applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data to generate an aging texture,
Synthesizing the aging texture with respect to the second three-dimensional data to generate an aging face model,
An aging prediction system configured to execute a prediction process of outputting the generated aging face model to the output unit.
The controller is
From the input unit, obtain the face orientation angle specified for output,
Using the generated aging face model, generate a two-dimensional face image of the face orientation angle,
The aging prediction system according to claim 1, further configured to output the generated two-dimensional face image to the output unit.
The two-dimensional image is a first two-dimensional image;
The controller is
A second 2D image is generated based on the acquired 3D face sample data,
Identifying feature points in the second two-dimensional image;
Using the feature points, generate normalized sample data obtained by normalizing the three-dimensional face sample data,
Using the normalized sample data, generating the shape aging model and the texture aging model,
The aging prediction system according to claim 1 or 2, further configured to execute a learning process of storing the generated shape aging model and the generated texture aging model in the model storage unit.
The aging prediction system according to claim 3, wherein the learning process includes generating the three-dimensional prediction model using the normalized sample data and storing the generated three-dimensional prediction model in the model storage unit.
The model storage unit stores a first texture aging model calculated using principal component analysis and a second texture aging model calculated using wavelet transform,
The control unit performs a first wavelet coefficient obtained by performing a wavelet transform on an image obtained by applying a first texture aging model to the first two-dimensional image, and a first wavelet coefficient for the first two-dimensional image. The aging according to claim 3 or 4, further configured to identify the texture aging model to be applied according to a result of comparison with a second wavelet coefficient to which the texture aging model of 2 is applied. Prediction system.
A shape aging model that predicts changes in facial shape due to aging,
A texture aging model that predicts changes in facial texture due to aging,
A model storage unit storing a three-dimensional prediction model for predicting three-dimensional data from a two-dimensional image;
A method for predicting aging by using an aging prediction system including a control unit configured to be connected to an input unit and an output unit,
The control unit is
Obtain a prediction target image from the input unit,
Extracting feature points of the prediction target image;
Using the extracted feature points, estimate the face direction in the prediction target image,
Generating first three-dimensional data based on the three-dimensional prediction model and the estimated face orientation;
Using the shape aging model, generating second three-dimensional data from the first three-dimensional data,
Applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data to generate an aging texture,
Synthesizing the aging texture with respect to the second three-dimensional data to generate an aging face model,
The aging prediction method which performs the prediction process which outputs the produced | generated said aging face model to the said output part.
A shape aging model that predicts changes in facial shape due to aging,
A texture aging model that predicts changes in facial texture due to aging,
A model storage unit storing a three-dimensional prediction model for predicting three-dimensional data from a two-dimensional image;
A non-transitory computer-readable storage medium for storing a program for predicting aging using an aging prediction system including an input unit and a control unit configured to be connected to an output unit. When the program is executed by the aging prediction system,
The control unit is
Obtain a prediction target image from the input unit,
Extracting feature points of the prediction target image;
Using the extracted feature points, estimate the face direction in the prediction target image,
Generating first three-dimensional data based on the three-dimensional prediction model and the estimated face orientation;
Using the shape aging model, generating second three-dimensional data from the first three-dimensional data,
Applying the texture aging model to the two-dimensional image generated based on the first three-dimensional data to generate an aging texture,
Synthesizing the aging texture with respect to the second three-dimensional data to generate an aging face model,
A medium that executes a prediction process of outputting the generated aging face model to the output unit.