JP2023111654A - Program, apparatus and method for statistically analyzing posture based on skeleton from skin model - Google Patents

Abstract

To provide a program, or the like, for statistically analyzing a skin model formed by including a plurality of joint points in a polygon mesh formed of multiple vertices.SOLUTION: A program causes a computer to function as: axial angle vector extraction means which extracts axial angle vectors with respect to all joint points each having a joint coordinate system, in a skin model (human model); and a statistical learning engine which constructs a statistical learning model so as to receive, in training stage, axial angle vectors of each joint point in multiple skin models serving as a training data group, and output component variables. The axial angle vector extraction means converts a rotation matrix R of the joint points to an axial angle vector r. The axial angle vector r is represented by a combination [e,θ] of a rotation axis e as a direction vector starting with a joint point and an axial rotation angle θ.SELECTED DRAWING: Figure 2

Description

The present invention relates to techniques for statistically analyzing skin models. In particular, it is suitable for a three-dimensional human body model as a skin model.

There is a three-dimensional body scanner technique for acquiring shape data of a human body (see, for example, Non-Patent Document 1). According to this technology, a three-dimensional human body model is measured by non-contact optical surveying. This human body model is expressed as a "skin" in which polygons made up of a plurality of vertices are configured in a mesh form. The number of vertices can also be measured at an ultra-high density of, for example, about 1.5 million points.

2. Description of the Related Art Conventionally, there is a technique for easily generating a three-dimensional model from only measured values and reducing the data volume by dimensional compression (see Patent Document 1, for example). According to this technique, using a training data group in which measured values of n dimensions corresponding to a plurality of measurement locations are associated with each three-dimensional model, three A dimensional model can be generated. Specifically, from a plurality of three-dimensional models of the teacher data group, a statistical learning engine that outputs dimensionally compressed component variables with the number of dimensions m and constructs a statistical learning model, and a measured value with the number of dimensions n , and a correlation learning engine that constructs a correlation learning model with component variables of the number of dimensions m. Also, using a correlation learning engine, an encoding unit that generates a component variable of the number of dimensions m from the measured value of the number of dimensions n of one body as target data, and a statistical learning engine is used to generate a component variable of the number of dimensions m and a decoding unit for generating a three-dimensional model from.

Japanese Patent No. 6424309

“smart & try”, Wacoal Corporation, [online], [searched on November 20, 2021], Internet <URL: https://www.wacoal.jp/smart_try/> "Ergonomics", [online], [searched on November 20, 2021], Internet <URL: https://www.cc.miyazaki-u.ac.jp/mitarai/room707/ergo002.html> Tomohiko Mukai, Katsuaki Kawachi, Yoichiro Miyake, "Mathematics and Systems of Character Animation, -Body Motion Generation and Artificial Intelligence in 3D Games-", [online], [Searched on November 20, 2021], Internet < URL: https://www.coronasha.co.jp/np/isbn/9784339029093/＞ "What is a Rotation Matrix (Direction Cosine Matrix)?", Sky Research Institute Blog, [online], [Searched on November 20, 2021], Internet <URL: https://www.sky-engin.jp /blog/rotation-matrix/>

According to the technique described in Patent Literature 1, the three-dimensional model expresses each vertex in three dimensions (x, y, z) of a world coordinate system (Global coordinate system). For example, if the number of vertices for each object is N=1,500,000, the three-dimensional model of one object is represented by a 3N (=4,500,000)-dimensional vector. Then, for example, about 6000 3D models can be input to the statistical learning engine to obtain component variables. The statistical learning engine is specifically based on Principal Component Analysis.

In order to obtain a three-dimensional human body model, a person must actually enter the three-dimensional body scanner and take an image. In the case of the world coordinate system, even for the same person, different three-dimensional models will be acquired under the influence of the person's posture.
The posture differs according to the positions of the joint points. For example, even if the person is exactly the same, the human body model will be acquired as a human body model that is greatly different, especially in the leg portion, when both legs are open and when both legs are closed. Moreover, when measuring with a three-dimensional scanner, it is very difficult to force all real persons to have the same posture.
That is, when the human body is measured by a three-dimensional scanner, the data is a mixture of the "body shape" based on the length and circumference of the human body and the "posture" based on the positions of the joints of the human body.

On the other hand, the inventors of the present application considered whether it would be possible to extract only the "posture" by separating the "body shape" from the three-dimensional model of the human body measured by the three-dimensional scanner. I thought that it might be possible to obtain only the "posture" of the person without depending on the "body shape".

For example, measurement values based on "body shape" include "length" such as height and crotch length, and "circumference" such as waist circumference and thigh diameter. Here, the length is based on the skeletal length of the human body, and the circumference is based on the flesh around the bone. Originally, the length and circumference based on the "body shape" of the human body are not affected by the "posture".
On the other hand, the "posture" changes depending on the positions of the joint points of the skeleton of the person. Even for people with the same body shape, their external impressions are greatly different depending on changes in their postures, such as leaning forward, leaning backward, stooping, and rolling shoulders.

On the other hand, the inventors of the present application considered whether it would be possible to statistically analyze a plurality of three-dimensional models from the data of only the "posture based on the skeleton" from which the "body shape" is separated. In particular, I thought that it would be an element for analyzing the "appearance", which is the external impression that is created by the posture of a person.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a program, apparatus, and method for generating a statistical shape space capable of analyzing a posture based on a skeleton from a skin model.

According to the present invention, there is provided a program for operating a computer installed in a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices,
A joint coordinate system is set for each joint point,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
In the training stage, the computer functions as a statistical learning engine that constructs a statistical learning model by inputting the axis angle vectors of each joint point in multiple skin models that serve as training data groups and outputting the component variables. and

According to another embodiment of the program of the present invention,
Preferably, the axis angle vector extracting means causes the computer to transform the rotation matrix R of all joint points into axis angle vectors r.

According to another embodiment of the program of the present invention,
The axis angle vector extracting means functions the computer to express the axis angle vector r by combining {e, θ} a direction vector starting from the joint point with a rotation axis e and a rotation angle θ about the axis. It is also preferable to let

According to another embodiment of the program of the present invention,
It is also preferable to cause the computer to further function as joint point estimation means for estimating the positions of joint points from one polygon mesh.

According to another embodiment of the program of the present invention,
The joint point estimation means is a joint point learning engine,
In the training stage, multiple polygon meshes of the training data group are input, and a joint point learning model is constructed so as to output the positions of the joint points,
In the estimation step, it is also preferable to input the polygon mesh to be estimated and operate the computer to output the positions of the joint points.

According to another embodiment of the program of the present invention,
As a first estimation step,
The statistical learning engine also preferably operates a computer to input axis angle vectors in a single polygon mesh to be estimated and to output component variables.

According to another embodiment of the program of the present invention,
A skeleton is constructed by connecting joint points with bones by parent-child relationship,
The joint coordinate system for each joint point has the joint point as a parent and the direction toward the child joint point as the z-axis, the bending direction of the joint point as the x-axis, and the direction perpendicular to the x-axis and the z-axis as the y-axis. It is also preferable to have the computer function so that it is set as

According to another embodiment of the program of the present invention,
A skin model is a human body model,
Skeleton is the skeleton of the human body,
Bones are bones between articulation points,
It is also preferable to have the computer function so that the axis angle vectors of all joint points are poses relative to the skeleton in the human body model.

According to another embodiment of the program of the present invention,
The statistical learning engine is also preferably computer functioning based on Principal Component Analysis or AutoEncoder.

According to the present invention, there is provided an analysis apparatus for statistically analyzing a skin model having a plurality of joint points in a polygon mesh composed of a plurality of vertices,
A joint coordinate system is set for each joint point,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
and a statistical learning engine that constructs a statistical learning model so as to input axis angle vectors of each joint point in a plurality of skin models as a training data group and output component variables as a training stage. .

According to the present invention, there is provided an analysis method for a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices, comprising:
A joint coordinate system is set for each joint point,
The device
a first step of extracting axis angle vectors for all joint points in the skin model;
a second step of constructing a statistical learning model by a statistical learning engine, as a training step, by inputting axis angle vectors of each joint point in a plurality of skin models as a training data group and outputting component variables; characterized by executing

According to the program, device, and method of the present invention, a statistical shape space capable of analyzing a posture based on a skeleton can be generated from a skin model.

FIG. 4 is an explanatory diagram showing the relationship between joint points and vertices; 1 is a basic functional configuration diagram of an analyzer in the present invention; FIG. FIG. 4 is an explanatory diagram showing a joint coordinate system of joint points, a rotation angle, and a rotation axis according to the present invention; FIG. 4 is an explanatory diagram showing a vector space of a three-dimensional model in a statistical learning engine; FIG. 4 is an explanatory diagram showing a statistical shape space in the statistical learning engine; A simple code for principal component analysis in a statistical learning engine. FIG. 3 is a functional configuration diagram of an analysis device further including a joint point estimator and a skeleton deformation unit;

BEST MODE FOR CARRYING OUT THE INVENTION Below, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an explanatory diagram showing the relationship between joint points and vertices.

According to embodiments of the present invention, terms are defined as follows.
"Skin": Polygon mesh with multiple vertices
(For example, a polygon mesh of a human body photographed with a 3D scanner)
"Skin model" : A model that has multiple joint points in the skin "Joint point" : A joint that transforms the skin (for example, a joint of the human body)
"Bone" : Line segments connecting joint points (e.g. bones of the human body)
"Skeleton" : A structure in which joint points are connected with bones by parent-child relationships
(e.g. human skeleton)
"Skin Weight": How much influence each joint point has on each vertex of the skin

According to FIG. 1, the human body is represented by a polygon mesh with 1.5 million vertices, for example. Also, the joint points are, for example, the following 15 points.
"cervical spine"
"central chest"
"Left shoulder""Leftelbow""Leftwrist"
"Rightshoulder""Rightelbow""Rightwrist"
"Centralwaist"
"Lefthip""Leftknee""Leftankle"
"Righthip""Rightknee""Rightankle"
It should be noted that the change in coordinates due to posture is considered to be the result of rotational motion about the joint point.

FIG. 2 is a basic functional configuration diagram of the analysis device in the present invention.

The analysis device 1 statistically analyzes a skin model having multiple joint points in a polygon mesh composed of multiple vertices. Statistical shape space is generated from the skin model to enable analysis of "posture based on skeleton".
According to FIG. 2, the analysis device 1 has an axis angle vector extractor 11 and a statistical learning engine 12 . These functional components are implemented by executing a program that causes a computer installed in the analyzer to function. In addition, the flow of processing of these functional components can also be understood as an analysis method.

The analysis device 1 prepares many skin models as teacher data for statistical learning. As described above, the skin model is a skin with multiple joint points. Joint points are connected by bones in a parent-child relationship. When the skin model is a human body, it is a polygon mesh of the human body with a skeleton that connects joint points. That is, human body skin models obtained from an unspecified number of persons are prepared.
It should be noted that the joint points and the skeleton assumed in the present invention are not anatomical of the human body, but are merely basic axes for motion.

[Axis angle vector extraction unit 11]
The axis angle vector extraction unit 11 extracts "axis angle vectors" for all joint points in the skin model.

FIG. 3 is an explanatory diagram showing a joint coordinate system of joint points, a rotation angle, and a rotation axis in the present invention.

<Position vector for each joint point>
Joint points are expressed with reference to a three-dimensional world coordinate system. The world coordinate system is basically a coordinate system in which the positive direction of the y-axis is the vertically upward direction and the xz plane is the horizontal plane. Then, the position vector for each joint point is expressed as follows.
Joint point position vector: t = (tx ty tz)

<Joint coordinate system for each joint point>
A joint coordinate system is set for each joint point as follows.
z-axis: direction toward the child joint point with the joint point as the parent
x-axis: bending direction of the joint point
y-axis: direction orthogonal to the x-axis and z-axis The skeleton of the human body is expressed as a tree structure connecting joint points.

According to FIG. 3, the joint coordinate system of the shoulder joint point and the joint coordinate system of the elbow joint point are represented. The z-axis of the joint coordinate system of the shoulder joint point is set in the direction toward the elbow joint point, which is the child joint point of the shoulder joint point. Also, the x-axis of the joint coordinate system of the shoulder joint point is set in the bending direction of the shoulder joint point. The y-axis of the joint coordinate system of the shoulder joint point is set in a direction orthogonal to the x-axis and the z-axis.

尚、この表記は、行ベクトルで、行列が右から作用する場合の表現である。列ベクトルで、行列が左から作用する場合は、回転行列や同次変換行列を転置する。 <Axis angle vector for each joint point>
The axis angle vector extraction unit 11 calculates a homogeneous coordinate transformation matrix A consisting of the rotation matrix R and the position t of each joint point.
A homogeneous coordinate system is a notation in which all coordinate transformations, including translation, are matrix operations (see, for example, Non-Patent Document 3). This expresses a vector representing the position and direction in a three-dimensional space as a four-dimensional vector to which homogeneous coordinate components are added.
Base vectors of the joint coordinate system of each joint point: r _x , r _y , r _z
(Unit vectors are orthogonal to each other (orthonormal basis))

Note that this notation is a row vector representation when the matrix operates from the right. Transposes a rotation matrix or homogeneous transformation matrix if it is a column vector and the matrix operates from the left.

The axis angle vector r is represented by a combination {e, θ} of a direction vector starting from the joint point as a rotation axis e and a rotation angle θ about the axis. This is called a rotation representation around an arbitrary axis (axis-angle representation). Since the rotation axis e is represented by a three-dimensional vector and the rotation angle θ is represented by one value, the axis angle vector is represented by four parameters. At this time, it can be represented by three parameters by including the rotation angle .theta. However, since {e, θ} and {−e, −θ} are the same rotation, the condition is a non-negative constraint that handles only positive value rotations.
Axis angle vector: r=θe=(θex, θey, θez)
Axis of rotation (unit vector): e = (ex, ey, ez) = r/|r|
Rotation angle: θ=|r|

Then, the axis angle vector extraction unit 11 transforms the rotation matrix R of all joint points into axis angle vectors r.
Axis angle vector: r=log(R)
Rotation matrix of each joint point: R=exp(r)
The conversion between the rotation matrix R and the axis angle vector r is based on Rodriguez's rotation formula (see, for example, Non-Patent Documents 3 and 4).

The rotation of the joint points in the skeleton is generally represented by Euler angles. Euler angles are represented by three axes (rotation around the x axis, rotation around the y axis, and rotation around the z axis). However, when complementing two or more rotations, the Euler angles cause a gimbal lock state, which limits the poses that can be expressed (see Non-Patent Document 3, for example). Therefore, the present invention expresses the rotation of the articulation point as an "axis angle vector".

[Statistical learning engine 12]
The statistical learning engine 12, as a training phase, functions as follows.
<Training stage>
As a training stage, the statistical learning engine 12 constructs a statistical learning model so as to input axis angle vectors of each joint point in a plurality of skin models that serve as teacher data groups and output component variables.

FIG. 4 is an explanatory diagram showing the vector space of the three-dimensional model in the statistical learning engine.

According to FIG. 4, as training data, for example, 6000 human bodies having various body shapes are assumed.
According to FIG. 4A, the number of joint points D=15 for each three-dimensional model of the human body, and each joint point is represented by three parameters (θe). A single three-dimensional model is represented by a 3D (for example, 3×15=45) dimensional vector, which is an axis angle vector for each D=15 joint point.
According to FIG. 4B, each human body in the three-dimensional model is represented by one point in the three-dimensional space.

FIG. 5 is an explanatory diagram showing the statistical shape space in the statistical learning engine.
FIG. 6 is a simple code representing principal component analysis in the statistical learning engine.

The statistical learning engine 12 may specifically be based on Principal Component Analysis.
By "principal component analysis", from 6000 points in the correlated 3D space, the principal components (component variables) of dimensions (for example, 45 dimensions) that are mutually uncorrelated and well represent the overall variation are derived. The variance of the first principal component is maximized, and the subsequent principal components are selected so as to maximize the variance under the constraint that they are uncorrelated with the previously determined principal components. By maximizing the variance of the principal components, the principal components are given as much explanatory power as possible for changes in observations. The principal axes that give the principal components are the orthogonal basis of the group of 6000 points in the 3D space. The orthogonality of the principal axes is derived from the fact that the principal axes are the eigenvectors of the covariance matrix and the covariance matrix is a real symmetric matrix.

The statistical learning engine 12 constructs a statistical learning model that projects the 3D space onto a statistical shape space (for example, 45 dimensions) having dimensions of component variables based on principal component analysis.
According to the present invention, each three-dimensional model in a 3D (=45) dimensional space is projected onto, for example, a 45-dimensional (component variable) space. The transformation that gives the principal components is represented by the singular value decomposition of a matrix consisting of a set of observed values.
X=U*Σ* ^VT
X: matrix of 6000 points in 3D dimensional space (6000 rows x 3D columns)
U: n(6000)×n(6000) square matrix (an orthogonal matrix of n-dimensional unit vectors)
Σ: n(6000)×p(3D) rectangular diagonal matrix (diagonal elements are singular values of X)
V: p(3D)×p(3D) square matrix (orthogonal matrix of p-dimensional unit vectors)
Here, a linear transformation by matrix V gives the principal components of X.
V: Matrix representing transformation from 3D dimensional space to statistical shape (45 dimensional) space
V ^-1 : Matrix representing the transformation from statistical shape (45-dimensional) space to 3D space Note that the superscript -1 of the matrix is not a symbol indicating the inverse matrix, but the transformation of the vector defined by the matrix , is used as an abstract symbol meaning its inverse transformation. Here, V ⁻¹ is equal to the transpose of V, V ^T .

As is clear from FIG. 6, by linear transformation using the matrix V or V ⁻¹ , correspondence can be established between the 3D dimensional space and the statistical shape space for each human body in the 3D model.
s=x*V
x=s*V ^-1
s: vector in statistical shape space
x: vector in 3D space
V: Statistical Learning Model The origin in the statistical shape space is the average posture (rotation of joint points) of all human body models trained as teacher data. In other words, the further away from the origin that is the average, the more characteristic the posture at that point.

Statistical learning engine 12 may also be based on an AutoEncoder.
An autoencoder is a type of neural network that outputs abstract concepts (feature values) from the amount of information in input data. Specifically, the input data is dimensionally compressed in the hidden layer and returned to the original size at the time of output.
Similar to principal component analysis, the autoencoder also derives 45-dimensional component variables that are mutually uncorrelated and best represent the overall variability from 6000 points in a correlated 3D-dimensional space.

According to FIG. 2 described above, the statistical learning engine 12 functions as follows as two stages of estimation.
<Estimation stage as an encoder>
The statistical learning engine 12 inputs axis angle vectors of all joint points in one polygon mesh to be estimated, and outputs component variables.
The component variable is, for example, a 45-dimensional vector, and is the principal component of the "attitude (rotation of all joint points)" of the skin model (polygon mesh + skeleton) to be estimated.
<Estimation stage as a decoder>
The statistical learning engine 12 inputs component variables to be estimated, and outputs "axis angle vectors of all joint points" in one skin model.
For example, by inputting component variables of a 45-dimensional vector to be estimated, the axis angle vectors of all joint points of one body can be output. The axis angle vectors of all joint points represent the pose of the human body.

Here, the features of the present invention will be described again in detail.
According to the prior art, skinning generally expresses poses and movements as rotations of human body parts about joint points. The skinning process is also called "Skeleton Subspace Deformation", and describes deformation using a coordinate system for each joint point in the skeleton. Deform the polygon mesh by displacing its skeleton. Also, according to general animation applications, the settings for skeleton creation are manually operated by the operator.
In contrast, the present invention extracts only the axis angle vector for each joint point in the skeleton and statistically analyzes it separately from the deformation of the 3D polygon mesh. Such technology does not exist as an existing technology. According to the present invention, the decomposition of the axis angle vectors of the joint points is achieved by extracting the axis angle vectors as intermediate data without calculating the skinning algorithm to the end, and training it with the statistical learning engine.

<Posture determination application>
For example, clothing is designed based on the size (length of the skeleton) and circumference of a person, but the posture (rotation of joint points) that suits the clothing is not taken into consideration. On the other hand, users who purchase clothes also consider their size and body shape, but they do not consider their posture at all. Nor does it consider how far the user's own posture is from the average or ideal posture.
As a posture determination application, various uses are assumed, and the difference in posture between different human bodies can be numerically acquired. Pose differences can be recognized as distances between points, for example in a statistical shape space based on principal component analysis.

FIG. 7 is a functional configuration diagram of an analysis device further including a joint point estimation unit and a skeleton deformation unit.

According to FIG. 7, a joint point estimator 13 and a skeleton deformer 14 are further provided as compared with FIG. These functional components are also realized by executing a program that causes a computer installed in the analyzer to function. In addition, the flow of processing of these functional components can also be understood as an analysis method.

When the human body is photographed by an optical scanner, the three-dimensional human body model becomes a skin (polygon mesh). That is, a skin does not contain a skeleton (framework) like a skin model. For this purpose, according to FIG. 7, before the axis angle vector extraction unit 11, joint points are extracted from the skin, and a skeleton is estimated in which the joint points are connected by bones according to the parent-child relationship.

[Joint point estimation unit 13]
The joint point estimation unit 13 estimates the positions of joint points from one polygon mesh.
The joint point estimator 13 may be a joint point learning engine and functions as follows.
(Training stage) Input multiple polygon meshes (explanatory variables) of the teacher data group, and build a joint point learning model so as to output joint point positions (objective variables).
(Estimation Stage) A polygon mesh to be estimated (explanatory variable) is input, and joint point positions (objective variable) are output.

As a joint point learning engine, a regression model that predicts motion capture joint points from polygon mesh vertex positions can also be used for joint point estimation. For the training of the regression model, it is assumed that "the shape of the human body is approximately convex around the joints". A convex shape is one in which a line segment connecting any two points does not go outside the polygon (the interior angle is 180° or less).
Of course, instead of machine learning such as a regression model, joint positions may be calculated directly from a mesh by designating mesh vertices corresponding to mark points appearing in frame images based on motion capture.

Alternatively, joint positions may be estimated from motion capture data. For example, like VICON (registered trademark), joint positions may be estimated from each frame of motion capture. For example, displacements of joint positions based on walking motions can be estimated from each motion-captured frame in which walking motions of many people are photographed.

As a feature of the present invention, the posture can be analyzed only from the skin (polygon mesh) that does not include the skeleton. This makes it possible regardless of the models of various three-dimensional scanners available on the market and the shooting environment. That is, even for polygon meshes captured by various three-dimensional scanners, as long as they are homologously modeled, joint positions can be estimated and associated with the statistical shape space. In this way, by separating only the posture from the skin of the human body, it is possible to uniformly handle different types of three-dimensional scanners and imaging environments.

[Skeleton deformation part 14]
The skeleton deformation unit 14 regenerates one skeleton from the axis angle vectors of all the joint points of one body.
Since only the axis angle vector is output from the statistical learning engine 12, a skeleton template is prepared in advance. The template is made to correspond to the axis angle vector (rotation axis, rotation angle) output from the statistical learning engine 12, and the skeleton model of the template is deformed (rotation of the joint points).
In this way, the posture based on the skeleton can be visually recognized by the skeleton deformation section 14 .

As described in detail above, according to the program, apparatus, and method of the present invention, a statistical shape space capable of analyzing a posture based on a skeleton can be generated from a skin model.

For the various embodiments of the present invention described above, various changes, modifications and omissions within the spirit and scope of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

REFERENCE SIGNS LIST 1 analyzer 11 axis angle vector extractor 12 statistical learning engine 13 joint point estimator 14 skeleton deformation unit

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a program, apparatus and method for generating a statistical shape space capable of analyzing posture based on a skeleton from a skin model.

According to the present invention, there is provided a program for operating a computer installed in a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices,
A joint coordinate system is set for each joint point,
A skeleton is constructed by connecting joint points with bones by parent-child relationship,
The axis angle vector of the bone at every articulation point represents the pose relative to the skeleton,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
In the training stage, the computer functions as a statistical learning engine that constructs a statistical learning model by inputting the axis angle vectors of each joint point in multiple skin models that serve as training data groups and outputting the component variables. and

According to another embodiment of the program of the present invention,
It is also preferable that the shaft angle vector extracting means cause the computer to express the shaft angle vector r by combining the direction vector starting from the joint point with the rotation axis e and the rotation angle θ about the axis. .

According to another embodiment of the program of the present invention,
The joint coordinate system for each joint point has the joint point as a parent and the direction toward the child joint point as the z-axis, the bending direction of the joint point as the x-axis, and the direction perpendicular to the x-axis and the z-axis as the y-axis. It is also preferable to have the computer function so that it is set as

According to another embodiment of the program of the present invention,
A skin model is a human body model,
Skeleton is the skeleton of the human body,
Bones are bones between articulation points
It is also preferable to have the computer function as

According to the present invention, there is provided an analysis apparatus for statistically analyzing a skin model having a plurality of joint points in a polygon mesh composed of a plurality of vertices,
A joint coordinate system is set for each joint point,
A skeleton is constructed by connecting joint points with bones by parent-child relationship,
The axis angle vector of the bone at every articulation point represents the pose relative to the skeleton,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
and a statistical learning engine that constructs a statistical learning model so as to input axis angle vectors of each joint point in a plurality of skin models as a training data group and output component variables as a training stage. .

According to the present invention, there is provided an analysis method for a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices, comprising:
A joint coordinate system is set for each joint point,
A skeleton is constructed by connecting joint points with bones by parent-child relationship,
The axis angle vector of the bone at every articulation point represents the pose relative to the skeleton,
The device
a first step of extracting axis angle vectors for all joint points in the skin model;
a second step of constructing a statistical learning model by a statistical learning engine, as a training step, by inputting axis angle vectors of each joint point in a plurality of skin models as a training data group and outputting component variables; characterized by executing

According to another embodiment of the program of the present invention,
It is also preferable that the shaft angle vector extracting means cause the computer to express the shaft angle vector r by combining the rotation axis e of the direction vector starting from the joint point and the rotation angle θ about the axis.

The axis angle vector r is represented by a combination {e, θ} of the rotation axis e of the direction vector starting from the joint point and the rotation angle θ about the axis. This is called a rotation representation around an arbitrary axis (axis-angle representation). Since the rotation axis e is represented by a three-dimensional vector and the rotation angle θ is represented by one value, the axis angle vector is represented by four parameters. At this time, it can be represented by three parameters by including the rotation angle .theta. However, since {e, θ} and {−e, −θ} are the same rotation, the condition is a non-negative constraint that handles only positive value rotations.
Axis angle vector: r=θe=(θex, θey, θez)
Axis of rotation (unit vector): e = (ex, ey, ez) = r/|r|
Rotation angle: θ=|r|

Claims (11)

A program for operating a computer installed in a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices,
A joint coordinate system is set for each joint point,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
In the training stage, the computer functions as a statistical learning engine that constructs a statistical learning model by inputting the axis angle vectors of each joint point in multiple skin models that serve as training data groups and outputting the component variables. A program to 2. The program according to claim 1, wherein the axis angle vector extracting means causes the computer to function to convert the rotation matrix R of each joint point into the axis angle vector r. The axis angle vector extracting means functions the computer to express the axis angle vector r by combining {e, θ} a direction vector starting from the joint point with a rotation axis e and a rotation angle θ about the axis. 3. The program according to claim 2, wherein the program causes 4. The program according to any one of claims 1 to 3, further causing the computer to function as joint point estimation means for estimating positions of joint points from one polygon mesh. The joint point estimation means is a joint point learning engine,
In the training stage, multiple polygon meshes of the training data group are input, and a joint point learning model is constructed so as to output the positions of the joint points,
5. The program according to claim 4, wherein, in the estimation stage, a polygon mesh to be estimated is input and the computer is caused to output positions of joint points. As a first estimation step,
6. The statistical learning engine according to any one of claims 1 to 5, wherein the statistical learning engine inputs an axis angle vector in one polygon mesh to be estimated, and causes a computer to function to output a component variable. program. A skeleton is constructed by connecting joint points with bones by parent-child relationship,
The joint coordinate system for each joint point has the joint point as a parent and the direction toward the child joint point as the z-axis, the bending direction of the joint point as the x-axis, and the direction perpendicular to the x-axis and the z-axis as the y-axis. 7. A program according to any one of claims 1 to 6, characterized in that it causes a computer to function so as to be set as A skin model is a human body model,
Skeleton is the skeleton of the human body,
Bones are bones between articulation points,
8. The program according to claim 7, causing the computer to function such that the axis angle vectors of all joint points are postures with respect to the skeleton of the human body model. 9. A program according to any one of claims 1 to 8, characterized in that the statistical learning engine makes the computer behave as if it is based on Principal Component Analysis or AutoEncoder. An analysis device for statistically analyzing a skin model having multiple joint points in a polygon mesh composed of multiple vertices,
A joint coordinate system is set for each joint point,
axis angle vector extracting means for extracting axis angle vectors for all joint points in the skin model;
and a statistical learning engine that constructs a statistical learning model so as to input axis angle vectors of each joint point in a plurality of skin models as a training data group and output component variables as a training stage. Analysis equipment. An analysis method for a device for statistically analyzing a skin model in which a plurality of joint points are inherent in a polygon mesh composed of a plurality of vertices, comprising:
A joint coordinate system is set for each joint point,
The device
a first step of extracting axis angle vectors for all joint points in the skin model;
a second step of constructing a statistical learning model by a statistical learning engine, as a training step, by inputting axis angle vectors of each joint point in a plurality of skin models as a training data group and outputting component variables; A method of analysis characterized by performing

Images