KR102449987B1 - Method for automatically setup joints to create facial animation of 3d face model and computer program thereof - Google Patents

Abstract

A method for automatically setting up target joints in a 3D target model is disclosed. The method can automatically set up joints of a 3D target model by matching the specifications of a texture image and texture projection method of the 3D target model to the specifications of a texture image and texture projection method of a 3D standard model, respectively, even if the geometry of the 3D standard model is not the same as the geometry of the 3D target model, and skin a target skeleton including the joints and the 3D target model by copying weights of a texture image of the 3D standard model and using the weights as weights of a texture image of the 3D target model.

Description

Method for automatic joint setup for creating facial animation of 3D face model and its computer program

The present invention relates to a joint automatic setup method, in particular, when the geometry of a 3D standard model is different from that of the 3D target model, a method for automatically setting up joints of the 3D target model, an object corresponding to the set of joints It relates to a method for skinning a skeleton and the 3D target model, and a computer program stored in a storage medium for performing these methods.

In general, a method (or method) of creating a 3D facial animation can be divided into a brand shape method (or method) and a joint-based method (or method).

The brand shaping method is a method of creating facial expressions and facial animations by creating tens to hundreds of basic facial expressions and combining the basic facial expressions by a method such as linear interpolation. The brand shaping method is often used to produce content that requires high-quality images even if it takes a long time to produce, such as a movie or animation.

1A to 1C are diagrams for explaining a conventional joint-based method. FIG. 1A is a 3D face model, FIG. 1B is a skeleton including joints having a hierarchical structure, and FIG. 1C is a 3D face model in which the joints are set up. indicates

In the joint-based method, as shown in Fig. 1c, joints for generating facial expressions are made inside the 3D face model, the joints are connected with the mesh of the 3D face model, and then the joints are moved How to make a facial expression. The joint-based method is used in fields where real-time processing is important, such as games, rather than fields where high-quality images are important because the control of facial expressions is intuitive and the amount of data required to create facial animations is small.

Therefore, the joint-based method is mainly used for content creation using game engines and the like. The joint-based method is achieved through a total of two steps consisting of positioning the joints inside the 3D face model, and skinning the meshes with the joints located inside the 3D face model. Whenever the 3D face model is changed, the operator must manually position the joints inside the 3D face model.

Patent Document 1: Republic of Korea Patent Publication - 10-1686028 (December 13, 2016 Announcement) Patent Document 2: Republic of Korea Patent Publication- 10-1102778 (Announced on January 05, 2012) Patent Document 3: Republic of Korea Patent Publication- 10-0896065 (Announced on May 07, 2009)

When the geometry of the 3D standard model and the geometry of the 3D target model are different from each other, the vertices of the 3D target model (or target object) can be found using the vertex IDs of the vertices of the 3D standard model (or standard object). none.

In addition, in order to make the geometric structure of the 3D standard model and the geometric structure of the 3D target model identical, a process of making the geometric structure of the 3D standard model and the geometric structure of the 3D target model identical is further required.

In addition, when modeling the 3D target model by fixing the geometrical structure of the 3D target model to the geometrical structure of the 3D standard model, the artist's work freedom may be impaired.

The technical problem to be achieved by the present invention is devised to solve the problems described above, and even if the geometric structure of the 3D standard model is not the same as that of the 3D target model, the specification and texture of the texture image of the 3D target model A method of automatically setting up the joints of the 3D target model by matching each projection method to the specification of the texture image of the 3D standard model and each of the texture projection methods, and copying the weights of the texture image of the 3D standard model as it is to copy the 3D To provide a method for skinning the 3D target model with the target skeleton including the joints using weights of the texture image of the target model, and a computer program stored in a storage medium for performing these methods.

According to an embodiment of the present invention, a computer program stored in a computer-readable storage medium coupled to a computer includes the set-up first joints, and receiving a 3D standard model formed by mapping a first texture image to a 3D model; , after the 3D standard model is received, receiving a 3D target model having a geometric structure different from that of the 3D standard model and including a second texture image; a second to be applied to the 3D target model Calculating a first minimum bounding box (MBB) for the 3D standard model and a second MBB for the 3D target model to obtain joints; generating a target skeleton corresponding to a set of joints; locating the target skeleton in the 3D target model using the first MBB and the second MBB; and a projection method of the second texture image. converting the first texture image in the same way as in the projection method, and determining the position of at least one end joint included in the second joints positioned in the 3D target model with the texture information of the 3D standard model and the 3D target adjusting using texture information of a model; adjusting the position of at least one intermediate joint connected to the at least one end joint using position information of the at least one end joint whose position is adjusted; After the automatic setup of the second joints is completed, skinning the target skeleton and the 3D target model using the texture information of the 3D standard model, the skinning information, and the texture information of the 3D target model is performed, The first texture image is a texture image for expressing the texture of the 3D standard model, and the second texture image is a texture image for expressing the texture of the 3D target model.

According to an embodiment of the present invention, a method for automatically setting up target joints in a 3D target model using a computer includes the set up standard joints, and receiving a 3D standard model formed by mapping a first texture image to the 3D model after receiving the 3D standard model, receiving the 3D target model including a second texture image while having a geometric structure different from that of the 3D standard model; applying to the 3D target model To obtain the target joints to be performed, calculating a first MBB (minimum bounding box) for the 3D standard model and a second MBB for the 3D target model, and copying a standard skeleton corresponding to the set of standard joints generating a target skeleton corresponding to the set of target joints; positioning the target skeleton in the 3D target model using the first MBB and the second MBB; and setting the standard of the second texture image as the second Converting the same as the standard of the first texture image, and converting the texture projection method of the second texture image to be the same as the texture projection method of the first texture image; adjusting the position of the included at least one end joint by using the texture information of the 3D standard model and the texture information of the 3D target model; Positioning the position of at least one intermediate joint connected to the at least one end joint adjusting using the adjusted position information of the at least one end joint; and after automatic setup of the target joints is completed, texture information of the 3D standard model, skinning information, and texture of the 3D target model skinning the target skeleton and the 3D target model using information, wherein the first texture image is a texture image for expressing a texture of the 3D standard model, and the second texture image is the 3D target model to express the texture of This is a texture image for

The method according to an embodiment of the present invention and the computer program stored in the storage medium for performing the method, even if the geometry of the 3D standard model is not the same as the geometry of the 3D object model, the specification of the texture image of the 3D object model. and texture projection method to match the specification and texture projection of the texture image of the 3D standard model, so that the joints of the 3D target model can be automatically set up, as well as the weights of the texture image of the 3D standard model. There is an effect of being able to skin the 3D target model and the target skeleton including the joints by copying and using the texture images of the 3D target model as weights.

Therefore, the method according to an embodiment of the present invention and the computer program stored in the storage medium for performing the method do not need to make the geometry of the 3D standard model the same as the geometry of the 3D target model, so to make them the same It does not require any additional process for modeling, and it does not impair the freedom of work of the artist who is working on modeling.

A detailed description of each drawing is provided in order to more fully understand the drawings recited in the Detailed Description of the Invention.
1A to 1C are diagrams for explaining a conventional joint-based method. FIG. 1A is a 3D face model, FIG. 1B is a skeleton including joints having a hierarchical structure, and FIG. 1C is a 3D face model in which the joints are set up. indicates
2 is a method for automatically setting up joints of the 3D target model when the geometry of the 3D standard model is not the same as that of the 3D target model according to the present invention, and a target skeleton corresponding to the set of joints; It is a flowchart for explaining a method of skinning the 3D target model.
3A to 3C show a 3D face standard model, a 3D mesh face standard model, and a 3D face standard model including set-up joints for explaining the present invention.
4 shows a skeleton including joints and bones having a hierarchical structure for explaining the present invention.
5 is a face texture image for explaining the present invention.
6 is a 3D face model including a texture image for explaining the present invention.
7 is a diagram illustrating a method of determining a color value of a surface of a 3D face model using a texture image for explaining the present invention;
8 is an example of defining a face region for explaining the present invention.
9 is an example of defining a face region MBB for explaining the present invention.
10 is an example of defining a face region MBB determined by extracting feature points or lines of a face for explaining the present invention.
11 is an example of defining the facial region MBB value for explaining the present invention.
12 is an example for explaining the positions of the adjusted joints after adjusting the positions of the face root joint (face root joint) for explaining the present invention.
13 is an example of a texture image configured in a spherical projection method for explaining the present invention.
14 is an example of texture images generated using various projection methods for explaining the present invention.
15 is an example of adjusting the size of the standard 3D face model and the size of the 3D face target model to be the same for explaining the present invention.
16 shows the positions of the joints after adjusting the positions of the child joints of the face root joint for explaining the present invention.
17 is an example for explaining the change in the position of the intermediate joint according to the change in the position of the longitudinal joint for explaining the present invention.
18 is an example illustrating a change in position of intermediate joints according to a change in positions of a plurality of end joints for explaining the present invention.
19 is an example for explaining skinning of a 3D face model and jaw joints using weights for explaining the present invention.
20 is an example in which the weights of FIG. 19 are displayed on a texture image.
21 is an example of copying weights of a 3D standard face model to a 3D target face model for explaining the present invention.
22 is a block diagram of a computer system capable of performing the method shown in FIG.
23 is a block diagram of a computer system capable of performing the method shown in FIG.

2 is a method for automatically setting up joints of the 3D target model when the geometry of the 3D standard model is not the same as that of the 3D target model according to the present invention, and a target skeleton corresponding to the set of joints; It is a flowchart for explaining a method of skinning the 3D target model.

Automatically setting up joints in the 3D target face model according to an embodiment of the present invention (eg, using a computer program to automatically or programmatically set up a skeleton corresponding to a set of joints in the 3D target face model instead of manually ) process will be described in detail with reference to FIGS. 2 to 21 .

Joints used in the processes of automatically setting up a skeleton including joints in a 3D target face model have a hierarchical structure as shown in FIG. 1B, at least one root joint, and at least one connected to the at least one root joint an intermediate joint of , and at least one end joint connected to the at least one intermediate joint. Referring to FIGS. 1B and 1C , positions of each joint (brow_parent_right, face_top_parent, and skullbase) may be understood.

The process of automatically setting up a skeleton containing joints in a 3D target face model, coupled with a computer (or hardware), to automatically set up the joints to create an animation of a three dimensional (3D) model. , may be performed by a computer program (or application) stored in a computer-readable storage medium (eg, a memory device).

In this specification, a 3D face model is exemplarily shown and described as a 3D model, and facial animation is exemplarily shown and described as an example of animation. Since the present invention is not limited to the processes of automatically setting up joints and skinning processes for generating can be directly applied to

A computer (also referred to as a 'computing device' 120), computing system 100 or 100A, or processor 122) is stored in a computer-readable storage medium (eg, a non-transitory data storage medium). A device that can run computer programs.

3A to 3C show a 3D face standard model, a 3D mesh face standard model, and a 3D face standard model including set-up joints for explaining the present invention.

The computer program 124 receives the 3D standard model including the set-up first joints and the first texture image (S110). The 3D standard model may be a 3D face standard model, and the 3D standard model may be a model previously manually created by an operator.

The computer program 124 receives from an external device a 3D standard model (FIG. 3C) in which joints are already set up in a 3D mesh face model (FIG. 3B) corresponding to the 3D standard model (FIG. 3A) to enable facial animation (FIG. 3C). S110).

For example, the 3D standard model of FIG. 3C may be a model set up by the operator so that the facial animation can be best expressed according to the judgment of the operator who sets up the joints. There are many ways to set up the joints, and any of the above methods may be used.

4 shows a skeleton including joints and bones having a hierarchical structure for explaining the present invention. 4 includes a root joint, an intermediate joint, end joints, and bones included in joints having a hierarchical structure. Once the location of the corresponding joint (eg, root joint, intermediate joint, and/or longitudinal joint) is determined, the size and orientation of the bone is automatically determined. A set of joints is also called a frame or skeletal data.

5 is a face texture image for explaining the present invention, FIG. 6 is a 3D face model including a texture image for explaining the present invention, and FIG. 7 is a 3D face model using a texture image for explaining the present invention It is a figure explaining how to determine the color value of a surface,

A 3D face model (eg, a 3D standard face model or a 3D target face model) includes a texture image for expressing a face texture as shown in FIG. 5 in addition to face geometry data and skeletal data. For example, the texture image (FIG. 6(b)) is mapped to the 3D face model (FIG. 6(a)) as shown in FIG. 6 to complete the 3D face model (FIG. 6(c)) ) can be expressed.

Methods for mapping a 2D texture image to a 3D face model are as follows.

As shown in FIG. 7, when pixel coordinates on a texture image corresponding to each vertex are designated for all vertices constituting the 3D face model (FIG. 7(c)), the color of the corresponding vertex is the texture image It is determined by the pixel value of the corresponding pixel coordinates on the image.

The color values of the remaining surfaces constituting the 3D face model are determined as follows. If the pixel values of the pixel coordinates (UV) of the texture image corresponding to the vertices constituting the surface are interpolated, the corresponding color values are interpolated. A pixel value of pixel coordinates of a 2D texture image with respect to a position of a 3D surface is determined, and a pixel value of the corresponding pixel coordinate is determined as a color value of the corresponding 3D surface.

The color values corresponding to the vertices v1, v2, v3, and v4 of FIG. 7(a) are the pixel coordinates ((u1, v1), (u2, v2) of the texture image of FIG. 7(b) , (u3, v3), (u4, v4)) is applied as a color value.

This method is commonly referred to as texture mapping. For texture mapping, UV coordinate values are specified that indicate to which pixel in the 2D texture image the corresponding vertex of the 3D face model is connected (or mapped).

The computer program 124 receives the 3D target model including the second texture image (S115). The geometry of the 3D standard model is different from the geometry of the 3D target model. The 3D target model contains UV coordinates and texture images just like the 3D standard model. The 3D target model means a 3D model for which you want to automatically set up a skeleton including joints.

As described above, when the geometry of the 3D standard model is different from that of the 3D target model, the 3D target model (or target object) using the vertex IDs for the vertices of the 3D standard model (or standard object) could not find the corresponding vertex of In addition, in order to make the geometric structure of the 3D standard model and the geometric structure of the 3D target model identical, a process of making the geometric structure of the 3D standard model and the geometric structure of the 3D target model identical is further required. In addition, when modeling the 3D target model in the same way as the geometric structure of the 3D standard model, the artist's work freedom may be impaired.

The present invention has been devised to solve the problems described above.

The computer program 124 calculates the reference size and position of each of the 3D standard model and the 3D target model ( S120 ). This step S120 is a step of obtaining reference sizes and positions of the 3D standard model and the 3D target model, respectively, in order to obtain (or adjust) joints to be applied to the 3D target model.

The reference size and position may be referred to as feature values of the 3D face model used to determine a translator and a scale factor for adjusting positions of joints of the 3D face model. In the present invention, the reference size and position are obtained (or calculated) through the Minimum Bounding Box (MBB) for the face region of the 3D face model.

The computer program 124 obtains the reference size and position of each of the 3D standard model and the 3D target model by using the first MBB for the face region of the 3D standard model and the second MBB for the face region of the 3D target model (S120) ). The computer program 124 obtains a first MBB for the 3D standard model and a second MBB for the 3D target model in order to obtain second joints to be applied to the 3D target model ( S120 ).

8 is an example of defining a face region for explaining the present invention, FIG. 9 is an example of defining a face region MBB for explaining the present invention, and FIG. 10 is a facial feature point or line for explaining the present invention This is an example of defining the face region MBB determined by extraction.

The MBB for defining the face region can be varied as shown in FIGS. 7 (a) and (b), but in the present invention, as shown in FIG. 9 (a), the MBB extends vertically from the eyebrow to the jaw line. And left and right is defined as a space corresponding to a plane including both jaw lines, and the MBB is defined as a space corresponding to the plane from the bridge of the nose to the back of the head as shown in FIG.

The computer program 124 automatically detects facial feature points or lines (LNS, or the green line in FIG. 10) as shown in FIG. 10 to obtain the upper, lower, left, and right lines constituting the MBB to obtain the MBB. have. The computer program 124 may obtain the MBB by detecting the maximum vertex in the front direction and the minimum vertex in the rear direction in order to obtain front and rear lines constituting the MBB.

11 is an example of defining the facial region MBB value for explaining the present invention.

The computer program 124 may obtain a facial region MBB value as shown in FIG. 11 .

과

)로 표시되고, 3D 대상 모델의 얼굴 영역 MBB 크기(예컨대, 기준 크기)와 중심점(예컨대, 위치)은 배열들(

와

)로 표시된다. 각 배열(

,

,

, 및

, A로 총칭함)는 3차원 요소 값들(x, y, z)를 포함하고, 각 요소는 A[x], A[y], A[z]로 표시된다.The size (eg, reference size) and the center point (eg, position) of the face region MBB of the 3D standard model obtained with reference to FIGS. 8 to 10 are arranged in

class

), the face region MBB size (eg, reference size) and center point (eg, position) of the 3D target model are arranged in the arrays (

Wow

) is indicated. each array (

,

,

, and

, A) includes three-dimensional element values (x, y, z), and each element is represented by A [x], A [y], and A [z].

The computer program 124 copies the standard skeleton corresponding to the set of first joints included in the 3D standard model to generate a target skeleton corresponding to the set of second joints to be included in the 3D target model (S125), and the 3D standard The target skeleton is positioned in the 3D target model using the first MBB for the model and the second MBB for the 3D target model (S130).

,

,

, 및

)을 이용하여 3D 대상 모델의 위치와 크기에 맞게 상기 대상 뼈대를 변형하고, 변형된 대상 뼈대를 상기 3D 대상 모델에 위치시킨다(S130).The computer program 124 creates a target skeleton by copying the standard skeleton of the 3D standard model (that is, a set of first joints having a hierarchical structure) (S125), and the arrays obtained in step (S120) (

,

,

, and

) to deform the target skeleton to match the position and size of the 3D target model, and place the deformed target skeleton in the 3D target model (S130).

As exemplarily shown in FIGS. 1B and 4 , each of the standard skeleton and the target skeleton has a hierarchical structure. For example, when an upper joint (referred to as a 'parent joint') of a skeleton moves, lower joints (referred to as 'child joints') connected to the upper joint through bones also move.

The first operation performed in step S130 is to move the uppermost joint (hereinafter referred to as a 'face root joint') of the face skeleton having a hierarchical structure to approximately position the entire face skeleton in the 3D target model.

)는 수학식 1과 같이 계산된다. 이때

은 3D 표준 모델의 페이스 루트 조인트의 위치이다.The position of the face root joint of the target bone (

) is calculated as in Equation 1. At this time

is the position of the face root joint of the 3D standard model.

[Formula 1]

,

, 및

)의 4칙 연산은 원소들(x, y, z) 사이의 4칙 연산으로 수행된다. 여기서,

는 스케일링 벡터로 X-축, Y-축, 및 Z-축 각각의 스케일링 팩터를 나타낸다.

는 차이 벡터로, 3D 표준 모델의 중심점과 3D 표준 모델의 페이스 루트 조인트의 위치 사이의 거리(또는 차이)를 나타낸다.At this time, each array (

,

, and

) is performed as a fourth-order operation between elements (x, y, z). here,

denotes a scaling factor of each of the X-axis, Y-axis, and Z-axis as a scaling vector.

is a difference vector, indicating the distance (or difference) between the center point of the 3D standard model and the position of the face root joint of the 3D standard model.

12 is an example for explaining the positions of the adjusted joints after adjusting the positions of the face root joint (face root joint) for explaining the present invention.

When step S130 is performed by the computer program 124, as shown in FIG. 12, although the target skeleton is located on the face of the 3D target model, the size of the 3D target model and the size of the target bone are different from each other. Therefore, the first part PT1 of the target skeleton pierces the face of the 3D target model.

Accordingly, the computer program 124 adjusts the size of the target skeleton according to the size of the 3D target model by changing the positions of the child joints of the face root joint.

)를 수학식 2에 따라 결정한다. 여기서, 상대 위치는 해당 자식 조인트의 부모 조인트에 대한 상대적인 위치를 말한다.To this end, the computer program 124 determines the relative position ( ) of all child joints connected to the face root joint of the target bone.

) is determined according to Equation (2). Here, the relative position refers to the relative position of the corresponding child joint to the parent joint.

[Equation 2]

According to embodiments, if the computer program 124 further receives first projection information indicating a projection method for the first texture image of the 3D standard model, the computer program 124 uses the first projection information to generate the first 1 It is possible to determine (or know) which method the projection method for the texture image is.

The computer program 124 selects the same projection method as the projection method for the first texture image from among a plurality of projection methods using the first projection information on the first texture image, and uses the selected projection method to select the second projection method. The texture image may be converted (S135).

According to embodiments, the computer program 124 uses first projection information indicating a first projection method for the first texture image of the 3D standard model and second projection information indicating a second projection method for the second texture image of the 3D target model. If further projection information is received, the computer program 124 compares the first projection method with the second projection method based on the first projection information and the second projection information, and the first projection method and the second projection method If the two projection methods are the same, the second texture image projected by the second projection method is used as it is, and if the first projection method and the second projection method are different from each other, the second texture projected by the second projection method The image is converted to fit the first projection method (S135).

According to the above-described examples, the projection method of the second texture image of the 3D target model is matched to the projection method of the first texture image of the 3D standard model.

Examples of the plurality of projection methods may include a planar projection method, a cylindrical projection method, and a spherical projection method, as exemplarily shown in FIG. It is not limited. Accordingly, the computer program 124 may generate the second texture image in any one of a planar projection method, a cylindrical projection method, and a spherical projection method ( S135 ).

The computer program 124 makes the projection method of the second texture image of the 3D target model the same as the projection method of the first texture image of the 3D standard model.

Since there is no standard for a texture image standard or a method for arranging an unfolded vertex, a person who creates the texture image generally decides arbitrarily. Therefore, the composition of the texture image of the 3D standard model and the composition of the texture image of the 3D target model may be different. Here, the configuration includes a standard of a texture image or a vertex arrangement method.

Since the UV map values of the texture image become important criteria from the next step (S140), in this step (S135), the computer program 124 determines the specification of the texture image of the 3D standard model, the vertex arrangement method, and the texture image of the 3D target model. work to unify the specifications and vertex arrangement method.

A method of unifying the standard of two text images and the vertex arrangement method by the computer program 124 is as follows.

The computer program 124 makes the size of the texture image of the 3D target model equal to the size of the texture image of the 3D standard model.

The computer program 124 makes the projection method of the texture image of the 3D target model the same as the projection method of the texture image of the 3D standard model.

13 is an example of a texture image configured by a spherical projection method for explaining the present invention, FIG. 14 is an example of texture images generated using various projection methods for explaining the present invention, FIG. 15 is an example of adjusting the size of the standard 3D face model and the size of the 3D face target model to be the same for explaining the present invention.

For example, if the 3D standard model is mapped to the first texture image through the spherical projection method as shown in FIGS. 13(a) and 13(b), the 3D target model is also the spherical projection method should be mapped to the second texture image through

Referring to FIG. 14, the 3D standard model of FIG. 14(a) may be projected as an arbitrary texture image as shown in FIG. 14(b), and the 3D standard model of FIG. 14(c) is a flat projection. If the (planar projection) method is mapped to the first texture image, the 3D target model must also be mapped to the second texture image through the planar projection method.

In addition, if the 3D standard model is mapped to the first texture image through the cylindrical projection method as shown in FIG. 14(d), the 3D target model must also be mapped to the second texture image through the cylindrical projection method. do.

Also, if the 3D standard model is mapped to the first texture image through the spherical projection method as shown in FIG. 14E , the 3D target model must also be mapped to the second texture image through the spherical projection method.

), 즉 X-축, Y-축, 및 Z-축 각각의 스케일링 팩터로 조정(scale) 한다. 도 15의 (a)와 (c)는 3D 표준 모델을 나타내고, 도 15의 (b)는 원래의 3D 대상 모델을 나타내고, 도 15의 (d)는 스케일링 벡터(

)로 크기 조정된 3D 대상 모델을 나타낸다.Before projecting the 3D target model into the second texture image, the area of the 3D standard model projected as the second texture image is equal to the area of the 3D standard model projected as the first texture image, as shown in FIG. 15 As shown, the size of the 3D target face model is calculated using the scaling vector (

), that is, the X-axis, the Y-axis, and the Z-axis are scaled by each scaling factor. Fig. 15 (a) and (c) show the 3D standard model, Fig. 15 (b) shows the original 3D target model, Fig. 15 (d) shows the scaling vector (

) represents the scaled 3D target model.

The computer program 124 uses the texture information of the 3D standard model and the texture information of the 3D target model, through step S130 of at least one longitudinal joint included in the second joints located in the 3D target model. Adjust the position (S140).

16 shows the positions of the joints after adjusting the positions of the child joints of the face root joint for explaining the present invention.

Even if the previous step S135 is performed, since the target skeleton generated in step S125 is not made for the 3D target model, as shown in FIG. 16 , the second part PT2 of the target skeleton is the 3D It can slightly break through the target model.

Since the second part PT2 protruding in this way is a longitudinal joint located at the most distal end among the second joints included in the hierarchical structure constituting the target skeleton, the position of the longitudinal joint must be adjusted.

How the computer program 124 adjusts the positions of the end joints is as follows.

First, the computer program 124 finds the closest vertex among the longitudinal joints of the standard skeleton of the 3D standard model and the vertices of the 3D standard model.

라 하고, 3D 표준 모델의 버텍스 위치를

라고 했을 때, 두 점들(이를 '두 위치들'이라고도 한다.) 사이의 길이(이를 '거리'라고도 한다.)는

이므로, 컴퓨터 프로그램(124)은 대응되는 두 점들 사이의 길이가 가장 작은 버텍스들을 찾는다. 찾아진 버텍스들을

라고 했을 때, 종단 조인트의 위치(

)는 수학식 3에 따라 계산된다.position of the end joint

, and the vertex position of the 3D standard model is

Called , the length (also called 'distance') between two points (this is also called 'two positions') is

Therefore, the computer program 124 finds the vertices having the smallest length between the two corresponding points. found vertices.

, the position of the end joint (

) is calculated according to Equation (3).

[Equation 3]

는 찾아진 버텍스들(

)에 대응되는 3D 대상 모델에서의 버텍스들의 위치이다.here

is the found vertices (

) is the position of the vertices in the 3D target model corresponding to .

)는 다음과 같은 방법으로 찾을 수 있다.Here, the positions of the vertices in the 3D target model (

) can be found in the following way.

First, the computer program 124 finds and stores the vertex values of the vertices of the 3D target model surrounding the corresponding pixels for all pixels in the second texture image for the 3D target model given through pre-processing. Save it to your device.

)의 UV 맵 값을 알고 있기 때문에, 상기 UV 맵 값에 해당하는 제2텍스처 이미지의 화소를 둘러싼 버텍스들을 저장 장치에 저장된 버텍스 값들을 이용하여 계산한다.The computer program 124 determines the vertex positions (

), the vertices surrounding the pixel of the second texture image corresponding to the UV map value are calculated using the vertex values stored in the storage device.

)를 계산된 버텍스 값들을 보간하여 계산한다. 보간은 잘 알려진 무게 중심 좌표들(Barycentric Coordinates)을 이용하는 선형 보간식(linear interpolation equation)을 이용하여 수행할 수 있다. That is, the computer program 124 determines the vertex position (

) is calculated by interpolating the calculated vertex values. Interpolation may be performed using a linear interpolation equation using well-known barycentric coordinates.

The computer program 124 adjusts the position of the at least one intermediate joint connected to the at least one end joint by using the position information on the position of the at least one end joint whose position has been adjusted through step S140. (S145).

Until the previous step S140, the computer program 124 finally confirmed the positions of the top joint (face_root_joint) and the end joints with respect to the second joints of the 3D target model. In this step (S145), the computer program 124 finally determines the positions of the other joints located between the uppermost joint and the terminal joint.

Since the positioning of the intermediate joints is fine, the visual effect on the facial animation may be negligible compared to the positioning of other joints (eg, the uppermost joint and the longitudinal joints), but the computer program 124 uses the 3D standard model. The positions of the intermediate joints of the target skeleton are adjusted to make the target skeleton of the 3D target model as identical to the configuration of the standard skeleton as possible.

How the computer program 124 adjusts the positions of the intermediate joints of the target skeleton is as follows.

)를 바로 전단계(S140)에서 조정된 종단 조인트들 각각의 위치 변화량의 평균 만큼 변화시켜 결정한다.The computer program 124 displays a new position (

) is determined by changing the average of the position change amount of each of the longitudinal joints adjusted in the previous step (S140).

17 is an example illustrating a change in the position of an intermediate joint according to a change in the position of the end joint for explaining the present invention, and FIG. 18 is a change in the position of the intermediate joints according to a change in the position of a plurality of end joints for explaining the present invention is an example to explain

또는

i=1 또는 2)가 전 단계(S140)에서 위치 변화량(

또는

i=1 또는 2)만큼 변한다면, 중간 조인트의 새로운 위치(

)는 수학식 4와 같다.17 and 18, the end joint (

or

i = 1 or 2) is the amount of position change in the previous step (S140) (

or

if i = 1 or 2), the new position of the intermediate joint (

) is the same as in Equation 4.

[Equation 4]

In Fig. 17, CJ denotes the initial position of the longitudinal joint, D denotes the change amount, CJ' denotes the new position of the longitudinal joint changed by the change amount (D), P denotes the initial position of the intermediate joint, and P' denotes the It represents the new position of the intermediate joint changed by the change amount (D).

In FIG. 18, each of CJ ₁ and CJ ₂ represents the initial position of the longitudinal joint, D ₁ and D ₂ each represent the amount of change, and each of CJ' ₁ and CJ' ₂ is the end changed by each change amount (D ₁ and D ₂ ). The new position of the joint is indicated, P indicates the initial position of the intermediate joint, and P' indicates the new position of the intermediate joint changed by the average change amount (D). The average change D is the arithmetic mean of the variances D ₁ and D ₂ .

In the case of an intermediate joint without child end joints (eg, having intermediate joints as child joints), the computer program 124 may update the position of the intermediate joint or after the child joint is updated, the change as described above. Calculate the same as the change in the position of the longitudinal joint.

The computer program 124 automatically sets up the second joints for the second joints of the 3D object motel (eg, the automatic setup for the second joints to create a facial animation of the 3D face model (eg, the 3D object model)). ) is completed, the 3D target skeleton and the 3D target model are skinned using the texture information of the 3D standard model, the skinning information, and the texture information of the 3D target model (S150).

In the step of skinning the target skeleton and the 3D target model ( S150 ), weighting factors indicating the degree of influence between the joints constituting the target skeleton and vertices constituting the 3D target model are also referred to as 'influence values'. .) is the process of assigning (or assigning) to vertices.

19 is an example for explaining skinning of a 3D face model and jaw joints using weights for explaining the present invention.

For example, in FIG. 19 , the vertices WT of the 3D standard model affected by the jaw joint are painted white. That is, the color of the vertices WT of the 3D standard model affected by the jaw joint is white. The darker the color, the more the vertices of the 3D standard model are affected by the corresponding joints. Usually, the operation of coloring the vertices of the 3D standard model is performed in such a way that a person assigns a value between 0 and 1 to each of the vertices through a method such as painting on the 3D model.

20 is an example in which the weights of FIG. 19 are displayed on a texture image. Referring to FIG. 20 , weights for vertices of the 3D standard model may be displayed in a specific color (eg, green, GRN) in a specific portion of the second texture image.

In the present invention, the computer program 124 copies the weights (or weight information) painted on the first texture image of the 3D standard model as it is and uses them as weights (or weight information) of the 3D target model.

21 is an example of copying weights of a 3D standard face model to a 3D target face model for explaining the present invention. Referring to FIG. 21 , the computer program 124 provides weighting information (eg, weights) painted on a specific part (GRN) of the first texture image of the 3D standard model shown in FIG. 21 (a). , green information) is copied as it is, and the copied weight information is stored in pixels (eg, pixel data) of a specific part (GRN) of the second texture image of the 3D target model shown in FIG. 21 ( b ). to generate skinning information for the 3D target model.

The computer program 124 skins the target skeleton of the 3D target model and the 3D target model by using the texture information of the 3D standard model, the generated skinning information, and the texture information of the 3D target model (S150).

The computer program 124 generates the weights of the vertices of the 3D target model by linearly interpolating the weights stored in the pixels of the second texture image.

Figure 22 is a block diagram of a computer system capable of performing the method shown in Figure 2; Referring to FIG. 22 , the computer system 100 includes a 3D scanner 110 , a computing device 120 , and a data storage device 130 .

The 3D scanner 110 generates a 3D image IM corresponding to the 3D target model 134 , and transmits the 3D image IM to the computing device 120 .

The computing device 120 includes a processor 122 capable of executing the computer program 124 described herein, the processor 122 figuring out how to automatically set up joints for generating animations of a 3D target model. The executable computer program 124 may be executed. The computing device 120 may store the 3D standard model 132 and the 3D target model 134 in the data storage device 130 . The data storage device 130 may be a hard disk drive (HDD) or a solid state drive (SSD) included in the computing device 120 such as a PC. The data storage device 130 stores the computer program 124 , and the computer program 130 may be loaded into the processor 122 and executed by the processor 122 .

23 is a block diagram of a computer system capable of performing the method shown in FIG. Referring to FIG. 23 , the computer system 100A may include a computing device 120 , a data storage device 130 , and a network database 150 capable of exchanging data with the computing device 120 through a communication network. can

22 shows an embodiment in which the 3D image IM corresponding to the 3D target model 134 is provided from the 3D scanner 110 to the computer program 124 , but in FIG. 23 , the 3D target model 134 is a network database from 150 to a computer program 124 executing on processor 122 over a communications network.

Although the present invention has been described with reference to the embodiment shown in the drawings, which is merely exemplary, those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100, 100A: computer system
110: 3D scanner
120: computing device
122: processor
124: computer program
130: data storage device
150: network database

Claims (10)

In a computer program coupled to a computer and stored in a computer-readable storage medium,
The computer program stored in the storage medium,
Receiving a 3D standard model including the set-up first joints, the first texture image is mapped to the 3D model is formed;
after receiving the 3D standard model, receiving a 3D target model including a second texture image while having a geometric structure different from that of the 3D standard model;
calculating a first minimum bounding box (MBB) for the 3D standard model and a second MBB for the 3D target model to obtain second joints to be applied to the 3D target model;
generating a target skeleton corresponding to the set of second joints by copying a standard skeleton corresponding to the set of first joints;
locating the target skeleton in the 3D target model using the first MBB and the second MBB;
converting a projection method of the second texture image to be the same as a projection method of the first texture image;
adjusting a position of at least one longitudinal joint included in the second joints located in the 3D target model using the texture information of the 3D standard model and the texture information of the 3D target model;
adjusting a position of at least one intermediate joint connected to the at least one end joint using position information of the at least one end joint whose position is adjusted; and
After the automatic setup of the second joints is completed, the step of skinning the target skeleton and the 3D target model using the texture information of the 3D standard model, the skinning information, and the texture information of the 3D target model is performed, ,
The first texture image is a texture image for expressing the texture of the 3D standard model,
The second texture image is a computer program stored in a storage medium that is a texture image for expressing the texture of the 3D target model. According to claim 1, wherein the step of converting the projection method of the second texture image to be the same as the projection method of the first texture image,
adjusting the size of the second texture image so that the size of the second texture image is the same as the size of the first texture image; and
and adjusting the size-adjusted projection method of the second texture image to be the same as the projection method of the first texture image. According to claim 1,
A computer program stored in a storage medium, wherein each of the texture information of the 3D standard model and the texture information of the 3D target model is UV map values. The method of claim 1, wherein the skinning of the target skeleton and the 3D target model comprises:
copying weights of the first texture image;
generating the skinning information for the 3D target model by storing the copied weights in pixels of the second texture image; and
and skinning the target skeleton and the 3D target model using the texture information of the 3D standard model, the generated skinning information, and the texture information of the 3D target model. 5. The method of claim 4,
and generating the weights of the vertices of the 3D target model by linearly interpolating the weights stored in the pixels of the second texture image. A method for automatically setting up target joints in a 3D target model using a computer, the method comprising:
Receiving a 3D standard model including set-up standard joints and formed by mapping a first texture image to the 3D model;
after receiving the 3D standard model, receiving the 3D target model including a second texture image while having a geometric structure different from that of the 3D standard model;
calculating a first minimum bounding box (MBB) for the 3D standard model and a second MBB for the 3D target model to obtain the target joints to be applied to the 3D target model;
creating a target skeleton corresponding to the set of target joints by copying a standard skeleton corresponding to the set of standard joints;
locating the target skeleton in the 3D target model using the first MBB and the second MBB;
converting the standard of the second texture image to be the same as that of the first texture image, and converting the texture projection method of the second texture image to be the same as the texture projection method of the first texture image;
adjusting the positions of at least one longitudinal joint included in the target joints located in the 3D target model using the texture information of the 3D standard model and the texture information of the 3D target model;
adjusting a position of at least one intermediate joint connected to the at least one end joint using position information of the at least one end joint whose position is adjusted; and
After the automatic setup of the target joints is completed, skinning the target skeleton and the 3D target model using texture information of the 3D standard model, skinning information, and texture information of the 3D target model,
The first texture image is a texture image for expressing the texture of the 3D standard model,
The method of automatically setting up target joints in a 3D target model, wherein the second texture image is a texture image for expressing the texture of the 3D target model. 7. The method of claim 6,
The specification is the size,
The method for automatically setting up target joints in a 3D target model, wherein the texture projection method is any one of a planar projection method, a cylindrical projection method, and a spherical projection method. 7. The method of claim 6,
A method of automatically setting up target joints in a 3D target model, wherein each of the texture information of the 3D standard model and the texture information of the 3D target model is UV map values. The method of claim 6, wherein the skinning of the target skeleton and the 3D target model comprises:
copying weights of the first texture image;
generating the skinning information for the 3D target model by storing the copied weights in pixels of the second texture image; and
Automatically setting up target joints in a 3D target model, including skinning the target skeleton and the 3D target model using the texture information of the 3D standard model, the generated skinning information, and the texture information of the 3D target model Way. 10. The method of claim 9,
The method of automatically setting up target joints in a 3D target model, wherein the weights correspond to the color of the part of the first texture image.

Images