KR102655987B1 - Apparatus and method for generating 3d avatar - Google Patents

Abstract

An apparatus and method for creating a 3D avatar are disclosed. A 3D avatar creation method according to an embodiment of the present invention includes 3D scanning a user's human body using an image sensor, and 3D scanning the user's human body. Creating a 3D scanning model using the results; It includes matching the 3D scanning model with a pre-stored template avatar model and generating a 3D avatar based on a result of matching the 3D scanning model with the template avatar model.

Description

Apparatus and method for generating 3D avatar {APPARATUS AND METHOD FOR GENERATING 3D AVATAR}

The present invention relates to computer graphics and computer vision technology, and more specifically, to computer 3D scanning technology and computer 3D information modeling technology.

Recently, 3D scanning technology using inexpensive depth sensors (Consumer-level Depth Sensor) such as Microsoft Kinect or image sensors (RGB Camera) of mobile phones has become popular. In particular, 3D human body scanning technology can reproduce body shape information of the human body based on distance information that 2D images do not have, and obtain actual measurement information, providing real world It is a core technology for implementing Augmented Reality technology that reproduces a virtual human body in space. Based on 3D reconstructed information, it can be applied to clothing recommendation and virtual try-on technology that reflects the user's body type information, and exercise recommendation technology for each body type based on the acquired 3D body type information, and its utility is It is expected to gradually increase.

However, unlike obtaining image information, the task for a user to acquire 3D body shape information is not simple. In order to obtain the user's 3D scan information accurately and completely, the user must continuously check the intermediate scan results and scan the restoration object 360 degrees to ensure that there are no holes or gaps. In other words, in order to perform a 360-degree scan, a turntable must be used, or the scan performer must rotate the scan object 360 degrees and perform the scan from every corner while checking the intermediate scan results, which is necessary to obtain 3D body shape information of the general user. It is a major obstacle to obtaining. In addition, the resulting 3D surface mesh is not complete due to holes or gaps in the 3D scan result due to limitations of low-cost sensors, resulting in the difficulty of having to manually correct it separately after scanning.

Meanwhile, Korean Patent Publication No. 10-2011-0070056 “Method and device for creating user-customized high-quality 3D avatars” establishes a database by obtaining high-quality data from various people, establishes the avatar creation process to proceed quickly, and creates 3D avatars. A method and device for creating a user-customized, high-quality 3D avatar that quickly creates an avatar that resembles the user while maintaining high quality is disclosed by obtaining the minimum data of the user and using this DB data to transform.

The purpose of the present invention is to secure the completeness and accuracy of the user's 3D body shape and create a 3D avatar that reflects the user's body shape.

Additionally, the purpose of the present invention is to create a 3D avatar capable of animation that reflects the user's body shape even for an incomplete 3D scanning model.

Additionally, the purpose of the present invention is to measure body shape information consistently even when the user's body shape changes.

A method of generating a 3D avatar according to an embodiment of the present invention to achieve the above object includes 3D scanning the user's human body using an image sensor, and Creating a 3D scanning model using the results of 3D scanning the user's body; It includes matching the 3D scanning model with a pre-stored template avatar model and generating a 3D avatar based on a result of matching the 3D scanning model with the template avatar model.

At this time, in the matching step, the 3D scanning model and the template avatar model may be matched using information on corresponding points between the 3D scanning model and the template avatar model.

At this time, the matching step is a 6-degree-of-freedom rigid transformation that corrects the approximate translation, orientation, and scale of the 3D scanning model and the template avatar model. ) can be applied.

At this time, the corresponding point information may include information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point.

At this time, at least one predefined position in the human body part may include at least one position among the face part, hand part, and foot part of the human body.

At this time, the matching step may define corresponding points of the 3D scanning model and corresponding points of the template avatar model as a corresponding pair based on at least one predefined position in the human body part.

At this time, the matching step uses the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and the 3D scanning Rigid body transformation can be applied to the template avatar model by rotating the direction of the model.

At this time, the matching step may transform the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and skeleton information of the template avatar model.

At this time, the matching step involves transforming the posture of the template avatar model using the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model. The minimum distance to corresponding points may be calculated, and the posture of the template avatar model may be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

At this time, the matching step may calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model by adjusting the positions of individual nodes of the template avatar model through an affine transformation matrix.

In addition, a three-dimensional avatar generating device according to an embodiment of the present invention for achieving the above object includes one or more processors; comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are executed by the one or more processors, wherein the one or more processors execute the one or more programs to detect the user's body using an image sensor. 3D scanning, generating a 3D scanning model using the results of 3D scanning the user's human body, matching the 3D scanning model with a pre-stored template avatar model, and matching the 3D scanning model with the template avatar model. A 3D avatar can be created based on the results of matching the avatar model.

At this time, the one or more processors may match the 3D scanning model and the template avatar model using correspondence point information between the 3D scanning model and the template avatar model.

At this time, the one or more processors perform a 6-degree-of-freedom rigid transformation that corrects the approximate translation, orientation, and scale of the 3D scanning model and the template avatar model. ) can be applied.

At this time, the corresponding point information may include information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point.

At this time, at least one predefined position in the human body part may include at least one position among the face part, hand part, and foot part of the human body.

At this time, the one or more processors may define the corresponding points of the 3D scanning model and the corresponding points of the template avatar model as a corresponding pair based on at least one predefined position in the human body part.

At this time, the one or more processors use the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and the 3D scanning Rigid body transformation can be applied to the template avatar model by rotating the direction of the model.

At this time, the one or more processors may transform the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and skeleton information of the template avatar model.

At this time, the one or more processors use the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model to transform the posture of the template avatar model and change the corresponding point of the 3D scanning model. The minimum distance to corresponding points may be calculated, and the posture of the template avatar model may be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

At this time, the one or more processors may adjust the positions of individual nodes of the template avatar model through an affine transformation matrix to calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model.

The present invention can create a 3D avatar that reflects the user's body shape by ensuring the completeness and accuracy of the user's 3D body shape.

Additionally, the present invention can generate a 3D avatar capable of animation that reflects the user's body shape even from an incomplete 3D scanning model.

Additionally, the present invention can measure consistent body shape information even when the user's body shape changes.

Figure 1 is a block diagram showing a 3D avatar creation device according to an embodiment of the present invention.
FIG. 2 is a block diagram showing in detail an example of the model matching unit shown in FIG. 1.
Figure 3 is an operation flowchart showing a method for creating a 3D avatar according to an embodiment of the present invention.
FIG. 4 is a detailed operational flowchart illustrating an example of the model matching step shown in FIG. 3.
Figure 5 is a diagram showing a 3D avatar creation process according to an embodiment of the present invention.
Figure 6 is a diagram showing the model matching process according to an embodiment of the present invention.
Figures 7 to 10 are diagrams showing the model matching process of a 3D model according to an embodiment of the present invention.
Figure 11 is a diagram showing a computer system according to an embodiment of the present invention.

The present invention will be described in detail with reference to the attached drawings as follows. Here, repeated descriptions, known functions that may unnecessarily obscure the gist of the present invention, and detailed descriptions of configurations are omitted. Embodiments of the present invention are provided to more completely explain the present invention to those skilled in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

Throughout the specification, when a part “includes” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram showing a 3D avatar creation device according to an embodiment of the present invention. FIG. 2 is a block diagram showing in detail an example of the model matching unit shown in FIG. 1.

Referring to FIG. 1, the 3D avatar generating device according to an embodiment of the present invention may include a 3D scanning unit 110, a model matching unit 120, and an avatar generating unit 130.

The 3D scanning unit 110 may 3D scan the user's body using an image sensor and create a 3D scanning model using the result of 3D scanning the user's body.

At this time, the image sensor may correspond to a depth sensor or an RGB image sensor.

At this time, the 3D scanning unit 110 may perform 3D scanning of the user's human body using a depth image or a color image.

At this time, the 3D scanning unit 110 may perform 3D scanning using only depth images, only color images, or a combination of the two.

The user's 3D scanning model created in this way may be the result of scanning only a part of the human body, or it may be the complete result of scanning every corner of 360 degrees. Additionally, in the case of a 3D scanning model, there may be holes or gaps due to limitations of the sensor or algorithm.

The model matching unit 120 can match the 3D scanning model with a pre-stored template avatar model.

At this time, the template avatar model may correspond to a standard avatar model representing the human body. At this time, the template avatar model may include skeleton information and vertex weight information.

At this time, if the template avatar model does not include skeleton information and vertex weight information, the model matching unit 120 may update the skeleton information and vertex weight information through an automatic rigging algorithm.

At this time, the model matching unit 120 may match the 3D scanning model and the template avatar model using information on corresponding points between the 3D scanning model and the template avatar model.

At this time, the model matching unit 120 performs a 6-degree-of-freedom rigid body transformation (Translation, Orientation, and Scale) that matches the approximate translation, orientation, and scale of the 3D scanning model and the template avatar model. Rigid Transformation) can be applied.

At this time, the corresponding point information may include information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point.

At this time, at least one predefined position in the human body part may include at least one position among the face part, hand part, and foot part of the human body.

At this time, the model registration unit 120 may define the corresponding points of the 3D scanning model and the corresponding points of the template avatar model as a corresponding pair based on at least one predefined position in the human body part.

At this time, the model matching unit 120 can recognize the pose of the 3D scanning model and automatically search for corresponding points of the 3D scanning model through an algorithm.

At this time, the model registration unit 120 largely performs at least one of three processes: global rigid registration, skeleton-based human pose registration, and local non-rigid registration. You can do more than one.

Referring to FIG. 2, the model matching unit 120 may include a global rigid body matching unit 121, a skeleton matching unit 122, and a local non-rigid body matching unit 123.

The model matching unit 120 can transform the template avatar model through partial matching even if the user's 3D scanning model is incomplete or part of the model is missing due to limitations of the sensor or algorithm, so that it reflects the user's body type. You can create a 3D avatar.

At this time, the model registration unit 120 first matches the front part of the human body through partial registration with respect to the partially deficient 3D scanning model, and then the front part is registered and deformed (the back part is deformed due to missing information). It can be matched with the (untransformed) template avatar model, and additionally, the rear portion of the template avatar model can be transformed.

At this time, the model matching unit 120 can gradually transform the template avatar model through partial matching to increase the accuracy and completeness of creating a three-dimensional avatar that reflects the user's human body.

The global rigid body registration unit 121 uses the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and performs the 3D scanning The direction of the model can be rotated to match the template avatar model.

At this time, the global rigid body matching unit 121 can obtain the overall 6 degrees of freedom translation and rotation information and scale information of the template avatar model based on the 3D scanning model using the corresponding point information and roughly match them.

The skeleton matching unit 122 may transform the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and the skeleton information of the template avatar model.

To express minute changes in the skeletal structure of a 3D avatar, minute translation and rotation changes in each segment of the skeleton must be reflected. Additionally, when the same person is partially scanned multiple times, the shape of a person's pose may change over time, so a registration method that can adapt to changes in the human body's pose is needed.

For example, the skeleton matching unit 122 adds a second 3D scanning model generated through side scanning to the template avatar model modified based on the first 3D scanning model generated through scanning of the user's front side. When transforming the avatar again, if the poses are different, the pose of the template avatar model can be transformed using the skeleton information to match the 3D scanning model.

The template avatar model may include the correlation between each predefined skeleton and each node.

In other words, since the template avatar model is rigged, changes in the posture of the skeleton can be reflected as changes in the posture of the template avatar.

At this time, the most commonly used rigging technique for the template avatar model is Linear Blend Skinning. The rigging technique defines a weight value that represents the influence of each skeleton at each node, and each node has a movement component equal to this weight value along the movement of each skeleton, which can be expressed as Equation 1. there is.

[Equation 1]

In Equation 1, V _i is the position of one node of the template avatar model, N _b is the number of bones in the entire skeleton model, w _j is the weight value of the skeleton, and T _-j is the rigid body deformation matrix of the jth skeleton. And V _j ⁰ may mean the local coordinates of the jth skeleton of the corresponding node in the reference posture.

At this time, the skeleton matching unit 122 may optimize the joint angles and joint coordinates of the skeleton in order to match the posture of the template avatar model to be similar to the user's 3D scanning model.

At this time, the skeleton matching unit 122 can perform an operation to find the rigid body matrix of the joint that minimizes the sum of the values of the three error functions as shown in Equation 2.

[Equation 2]

In Equation 2, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the It represents the weight, and finally, E _s is a function that minimizes the sum of the rotation angles of the joint angles, and γ represents the weight.

The error function E _d to minimize the skeleton distance between the template avatar model and the user's 3D scanning model can be defined as Equation 3.

[Equation 3]

In Equation 3, the dist() function represents a function that calculates the minimum distance between the point skin (V _i ) deformed by the rigid body matrix of the current joint and the surface D of the user's 3D scanning model, and N _m template avatar You can see that the number of nodes is indicated.

At this time, since the corresponding points of the 3D scanning model input from the user may converge to the local minima, to prevent this, the error function E between the corresponding points of the template avatar model and the user's 3D scanning model _f can be defined as in Equation 4.

[Equation 4]

In Equation 4, F _M and F _D are the corresponding points defined by the user for each template avatar M and the user's 3D scan information D, and the point skin(F _M ) transformed by the rigid body matrix of the joint where F _M is F _D It can be calculated so that the distance from is minimal. Here, N _f represents the number of user-defined corresponding points.

And Equation 5 can be defined using quaternion, which means rotation of the joint angle, as an error function to maintain the shape of the template avatar model M.

[Equation 5]

In Equation 5, Nb is the number of bones, qi is the rotation of joint i, and it can be seen that it represents the objective function of minimizing the sum of the rotation angles of the joint angles.

That is, the skeleton matching unit 122 uses the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model, and changes the posture of the template avatar model to the corresponding point and the 3D scanning model. The minimum distance to the corresponding point is calculated, and the posture of the template avatar model can be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

The local non-rigid body matching unit 123 can calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model by adjusting the positions of individual nodes of the template avatar model through an affine transformation matrix. .

At this time, the local non-rigid body matching unit 123 may adjust the positions of individual nodes of the template avatar model M through an Affine Transformation matrix to minimize the distance from the user's 3D scan information D.

At this time, the local non-rigid matching unit 123 can define the error function as shown in Equation 6.

[Equation 6]

In Equation 6, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the weight. , and finally, it can be seen that E _s represents constraints to maintain the basic appearance of the template avatar.

At this time, the local non-rigid matching unit 123 may first define an error function E _d that minimizes the distance between the template avatar model M and the surface D of the user's 3D scanning model as shown in Equation 7.

[Equation 7]

In Equation 7, it can be seen that the dist() function represents a function that calculates the minimum distance between the node V _i of the template avatar transformed by the affine transformation matrix T _i and the user's 3D scan information surface D, It can be seen that N _m represents the number of template avatar nodes. Here, the distance can be determined by measuring the closest distance to the triangle mesh.

At this time, the local non-rigid matching unit 123 can define the error function of the corresponding point receiving the user input as shown in Equation 8.

[Equation 8]

In Equation 8, F _M and F _D are corresponding points defined by the user between each template avatar M and the user's 3D scan information D, and the distance between the point where F _M is transformed by the T _i affine matrix and F _D is the minimum. Make it possible. Here, N _f indicates the number of user-defined corresponding points.

Lastly, the local non-rigid matching unit 123 can define an error function that minimizes the difference in the transformation matrices of all edges as a constraint to maintain the shape of the template avatar model M.

If two nodes of an arbitrary edge e _i undergo similar transformations, the difference between the transformation matrices of the two nodes will converge to the 0 matrix. Using this, Equation 9 can be defined.

[Equation 9]

It can be seen that the matrix of two nodes of an arbitrary edge is expressed as T _i and T _j , where N _s represents the number of edges of the template avatar M.

The model matching unit 120 may be used as a global rigid body matching module, a skeleton-based pose matching module, and as a local non-rigid body matching module, depending on the usage, one module may be used independently, or two or more modules may be mixed together.

For example, if the translation/rotation/scale of the overall template avatar is set in advance to the 3D scan information, the model matching unit 120 does not perform global rigid body matching, but uses skeleton-based pose matching and local non-rigid body matching. The template avatar model and the 3D scanning model can also be matched.

Additionally, when the poses of the 3D scanning model and the template avatar model are the same, the model matching unit 120 may not perform skeleton-based pose matching but may only perform local non-rigid body matching.

Additionally, the model matching unit 120 may only perform matching functions selected by the user in accordance with the usage example.

The avatar creation unit 130 may generate a 3D avatar based on a result of matching the 3D scanning model and the template avatar model.

At this time, the 3D avatar may correspond to a refined, animation-capable avatar in which rigging information and vertex weight information are reflected by reflecting the user's body shape.

At this time, the avatar creation unit 130 can display an exercise prescription animation to the user using the generated 3D avatar, and can also be used as the user's character in a game, etc.

At this time, the 3D avatar can provide the user's body shape information using 3D surface information and skeleton information. For example, circumference information such as head circumference and chest circumference can be obtained through the position defined in the skeleton information of the 3D avatar and plane information with the skeleton direction as the normal, or the arm-leg ratio and waist can be obtained using the ratio of the skeleton. -You can obtain the hip ratio, etc., and use vertex information to provide volume information of the user's avatar. At this time, the body shape information of the 3D avatar can be automatically obtained based on the skeleton information of the 3D avatar, so body shape information for each user can be consistent and can be easily used for comparison according to the user's history.

At this time, the avatar creation unit 130 can create a 3D avatar that can gradually reflect the user's body shape even in the 3D scanning model of the mismatched human body part.

At this time, the avatar creation unit 130 can generate an avatar that reflects a part of the user's body shape based on the partial scan results of the human body even in a situation where 3D scanning as a whole is not possible.

At this time, the avatar creation unit 130 can modify the avatar based on the scan results of additional human body parts, thereby gradually improving the reflection of the user's body shape.

At this time, the avatar creation unit 130 may generate a refined mesh-based avatar that can be animated.

At this time, the avatar creation unit 130 may generate a 3D avatar reflecting the user's body type that can be animated despite the incomplete 3D scan result of the human body.

At this time, the avatar creation unit 130 detects incompleteness of the 3D scan result due to noise, holes, or limitations of the 3D scan algorithm of a low-cost (consumer-level) depth sensor or image sensor. ) You can overcome this and create a 3D avatar.

At this time, the avatar creation unit 130 can operate through animation even with incomplete 3D scan results and can generate a refined mesh-based 3D avatar that reflects the user's body shape.

At this time, the avatar creation unit 130 can automatically measure the volume, ratio, and circumference information of a consistent skeleton-based body shape for each user using the resulting avatar that reflects the user's three-dimensional body shape.

At this time, the avatar creation unit 130 may measure the volume of the user's body shape using surface mesh information of the generated 3D avatar.

At this time, the avatar creation unit 130 can measure the human body ratio using the skeleton information of the generated 3D avatar and the bounding box of the surface.

At this time, the avatar creation unit 130 can automatically measure the perimeter using the skeleton position information and surface information of the created 3D avatar.

Additionally, a 3D avatar creation device according to an embodiment of the present invention includes one or more processors; It includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are executed by the one or more processors, and the one or more processors can execute the one or more programs.

At this time, one or more processors may include a 3D scanning unit 110, a model matching unit 120, and an avatar generating unit 130, and one or more processors may include a 3D scanning unit 110 and a model matching unit ( The functions of 120) and the avatar creation unit 130 can be executed through one or more programs.

Figure 3 is an operation flowchart showing a method for creating a 3D avatar according to an embodiment of the present invention. FIG. 4 is a detailed operational flowchart illustrating an example of the model matching step shown in FIG. 3.

Referring to FIG. 3, the method for creating a 3D avatar according to an embodiment of the present invention may first perform a 3D scan (S210).

That is, in step S310, the user's body can be 3D scanned using an image sensor, and a 3D scanning model can be created using the result of the 3D scanning of the user's body.

At this time, the image sensor may correspond to a depth sensor or an RGB image sensor.

At this time, step S310 may perform 3D scanning of the user's human body using a depth image or a color image.

At this time, in step S310, 3D scanning may be performed using only the depth image, only the color image, or a combination of the two.

The user's 3D scanning model created in this way may be the result of scanning only a part of the human body, or it may be the complete result of scanning every corner of 360 degrees. Additionally, in the case of a 3D scanning model, there may be holes or gaps due to limitations of the sensor or algorithm.

Additionally, the 3D avatar generation method according to an embodiment of the present invention can perform model matching (S220).

That is, step S220 can match the 3D scanning model with a pre-stored template avatar model.

At this time, the template avatar model may correspond to a standard avatar model representing the human body. At this time, the template avatar model may include skeleton information and vertex weight information.

At this time, in step S220, if the template avatar model does not include skeleton information and vertex weight information, the skeleton information and vertex weight information can be updated through an automatic rigging algorithm.

At this time, step S220 may match the 3D scanning model and the template avatar model using information on corresponding points between the 3D scanning model and the template avatar model.

At this time, step S220 is a 6-degree-of-freedom rigid transformation that corrects the approximate translation, orientation, and scale of the 3D scanning model and the template avatar model. ) can be applied.

At this time, the corresponding point information may include information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point.

At this time, at least one predefined position in the human body part may include at least one position among the face part, hand part, and foot part of the human body.

At this time, in step S220, the corresponding point of the 3D scanning model and the corresponding point of the template avatar model may be defined as a corresponding pair based on at least one predefined position in the body part.

At this time, step S220 may recognize the pose of the 3D scanning model and automatically search for corresponding points of the 3D scanning model through an algorithm.

At this time, step S220 largely involves at least one of three processes: global rigid registration, skeleton-based human pose registration, and local non-rigid registration. can be performed.

At this time, in step S220, even if the user's 3D scanning model is incomplete or part of the model is missing due to limitations of the sensor or algorithm, the template avatar model can be transformed through partial registration, thereby changing the user's body shape. You can create a reflected 3D avatar.

At this time, in step S220, the front part of the human body is first registered through partial registration for the partially deficient 3D scanning model, and the front part is registered and deformed (the back part is not deformed due to missing information). (not shown) can be matched with the template avatar model, and additionally the rear portion of the template avatar model can be transformed.

At this time, step S220 can gradually transform the template avatar model through partial matching to increase the accuracy and completeness of creating a 3D avatar that reflects the user's human body.

Referring to FIG. 4, step S220 may perform global rigid body registration (S221), skeleton registration (S222), and local non-rigid body registration (S223).

Step S221 uses the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and to change the direction of the 3D scanning model. can be rotated to match the template avatar model.

At this time, in step S221, the overall 6 degrees of freedom translation and rotation information and scale information of the template avatar model can be obtained and roughly matched based on the 3D scanning model using the corresponding point information.

In step S222, the posture of the template avatar model may be transformed to correspond to the posture of the 3D avatar model using the corresponding point information and skeleton information of the template avatar model.

To express minute changes in the skeletal structure of a 3D avatar, minute translation and rotation changes in each segment of the skeleton must be reflected. Additionally, when the same person is partially scanned multiple times, the shape of a person's pose may change over time, so a registration method that can adapt to changes in the human body's pose is needed.

For example, in step S222, a template avatar model modified based on the first 3D scanning model generated by scanning the user's front part is additionally used with a second 3D scanning model generated by scanning the side part. When transforming the avatar again, if the poses are different, the pose of the template avatar model can be transformed using the skeleton information to match the 3D scanning model.

The template avatar model may include the correlation between each predefined skeleton and each node.

In other words, since the template avatar model is rigged, changes in the posture of the skeleton can be reflected as changes in the posture of the template avatar.

At this time, the most commonly used rigging technique for the template avatar model is Linear Blend Skinning. The rigging technique defines a weight value that represents the influence of each skeleton at each node, and each node has a movement component equal to this weight value along the movement of each skeleton, which can be expressed as Equation 1. there is.

In Equation 1, V _i is the position of one node of the template avatar model, N _b is the number of bones in the entire skeleton model, w _j is the weight value of the skeleton, and T _-j is the rigid body deformation matrix of the jth skeleton. And V _j ⁰ may mean the local coordinates of the jth skeleton of the corresponding node in the reference posture.

At this time, in step S222, optimization of the joint angles and joint coordinates of the skeleton may be performed to match the posture of the template avatar model similarly to the user's 3D scanning model.

At this time, step S222 can perform an operation to find the rigid body matrix of the joint that minimizes the sum of the values of the three error functions as shown in Equation 2.

In Equation 2, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the It represents the weight, and finally, E _s is a function that minimizes the sum of the rotation angles of the joint angles, and γ represents the weight.

The error function E _d to minimize the skeleton distance between the template avatar model and the user's 3D scanning model can be defined as Equation 3.

In Equation 3, the dist() function represents a function that calculates the minimum distance between the point skin (V _i ) deformed by the rigid body matrix of the current joint and the surface D of the user's 3D scanning model, and N _m template avatar You can see that the number of nodes is indicated.

At this time, since the corresponding points of the 3D scanning model input from the user may converge to the local minima, to prevent this, the error function E between the corresponding points of the template avatar model and the user's 3D scanning model _f can be defined as in Equation 4.

In Equation 4, F _M and F _D are the corresponding points defined by the user for each template avatar M and the user's 3D scan information D, and the point skin(F _M ) transformed by the rigid body matrix of the joint where F _M is F _D It can be calculated so that the distance from is minimal. Here, N _f represents the number of user-defined corresponding points.

And Equation 5 can be defined using quaternion, which means rotation of the joint angle, as an error function to maintain the shape of the template avatar model M.

In Equation 5, Nb is the number of bones, qi is the rotation of joint i, and it can be seen that it represents the objective function of minimizing the sum of the rotation angles of the joint angles.

That is, in step S222, the posture of the template avatar model is transformed using the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model, and the corresponding point of the 3D scanning model is changed. The minimum distance to can be calculated, and the posture of the template avatar model can be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

In step S223, the positions of individual nodes of the template avatar model can be adjusted through an affine transformation matrix to calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model.

At this time, in step S223, the distance from the user's 3D scan information D can be minimized by adjusting the positions of individual nodes of the template avatar model M through an Affine Transformation matrix.

At this time, step S223 can define the error function as shown in Equation 6.

In Equation 6, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the weight. , and finally, it can be seen that E _s represents constraints to maintain the basic appearance of the template avatar.

At this time, in step S223, an error function E _d that minimizes the distance between the template avatar model M and the surface D of the user's 3D scanning model can be defined as shown in Equation 7.

In Equation 7, it can be seen that the dist() function represents a function that calculates the minimum distance between the node V _i of the template avatar transformed by the affine transformation matrix T _i and the user's 3D scan information surface D, It can be seen that N _m represents the number of template avatar nodes. Here, the distance can be determined by measuring the closest distance to the triangle mesh.

At this time, in step S223, the error function of the corresponding point receiving the user input can be defined as shown in Equation 8.

In Equation 8, F _M and F _D are corresponding points defined by the user between each template avatar M and the user's 3D scan information D, and the distance between the point where F _M is transformed by the T _i affine matrix and F _D is the minimum. Make it possible. Here, N _f indicates the number of user-defined corresponding points.

Lastly, step S223 can define an error function that minimizes the difference in the transformation matrices of all edges as a constraint to maintain the shape of the template avatar model M.

If two nodes of an arbitrary edge e _i undergo similar transformations, the difference between the transformation matrices of the two nodes will converge to the 0 matrix. Using this, Equation 9 can be defined.

It can be seen that the matrix of two nodes of an arbitrary edge is expressed as T _i and T _j , where N _s represents the number of edges of the template avatar M.

Additionally, in the case of global rigid body registration, skeleton-based pose matching, and local non-rigid body registration, step S220 may be used as a single step independently or as a mixture of two or more steps depending on the usage example.

For example, in step S220, if the Translation/Rotation/Scale of the overall template avatar is matched with the 3D scan information in advance, global rigid body registration is not performed, and skeleton-based pose matching and local non-rigid body registration are also used to match the template. The avatar model and the 3D scanning model can be matched.

Additionally, when the poses of the 3D scanning model and the template avatar model are the same, step S220 may not perform skeleton-based pose matching but only local non-rigid body matching.

Additionally, step S220 may only perform the matching function selected by the user according to the usage example.

Additionally, the 3D avatar creation method according to an embodiment of the present invention can generate a 3D avatar (S230).

That is, step S230 may generate a 3D avatar based on a result of matching the 3D scanning model and the template avatar model.

At this time, the 3D avatar may correspond to a refined, animation-capable avatar in which rigging information and vertex weight information are reflected by reflecting the user's body shape.

At this time, in step S230, an exercise prescription animation can be shown to the user using the generated 3D avatar, and can also be used as the user's character in a game, etc.

At this time, the 3D avatar can provide the user's body shape information using 3D surface information and skeleton information. For example, circumference information such as head circumference and chest circumference can be obtained through the position defined in the 3D avatar's skeleton information and plane information with the skeleton direction as the normal, or the arm-leg ratio and waist can be obtained using the skeleton's ratio. -You can obtain the hip ratio, etc., and use vertex information to provide volume information of the user's avatar. At this time, the body shape information of the 3D avatar can be automatically obtained based on the skeleton information of the 3D avatar, so body shape information for each user can be consistent and can be easily used for comparison according to the user's history.

At this time, in step S230, a 3D avatar that can gradually reflect the user's body shape can be created even in the 3D scanning model of the mismatched human body part.

At this time, step S230 can generate an avatar reflecting a part of the user's body shape based on the partial scan results of the human body even in a situation where 3D scanning is impossible.

At this time, in step S230, the avatar can be modified based on the scan results of additional human body parts, thereby gradually improving the reflection of the user's body shape.

At this time, step S230 may generate a refined mesh-based animable avatar.

At this time, step S230 may generate a 3D avatar reflecting the user's body type that can be animated despite the incomplete 3D human body scan result.

At this time, step S230 overcomes the incompleteness of the 3D scan result due to noise, holes, or limitations of the 3D scan algorithm of a low-cost (consumer-level) depth sensor or image sensor. And you can create a 3D avatar.

At this time, in step S230, operation is possible through animation even with incomplete 3D scan results, and a refined mesh-based 3D avatar reflecting the user's body shape can be created.

At this time, in step S230, the volume, ratio, and circumference information of the consistent body shape of each user based on the skeleton can be automatically measured using the resulting avatar that reflects the user's three-dimensional body shape.

At this time, in step S230, the volume of the user's body shape can be measured using surface mesh information of the generated 3D avatar.

At this time, in step S230, the human body ratio can be measured using the skeleton information of the generated 3D avatar and the bounding box of the surface.

At this time, in step S230, the perimeter can be automatically measured using the skeleton position information and surface information of the generated 3D avatar.

Figure 5 is a diagram showing a 3D avatar creation process according to an embodiment of the present invention.

Referring to FIG. 5, in the 3D avatar creation process according to an embodiment of the present invention, the 3D avatar creation device may first perform a 3D scan.

That is, the 3D avatar creation device can 3D scan the user's human body using an image sensor and generate a 3D scanning model using the results of 3D scanning the user's human body.

At this time, the image sensor (3D Body Scanner) may correspond to a depth sensor or an RGB image sensor.

At this time, the 3D avatar creation device can perform 3D scanning of the user's human body using a depth image or color image (Depth or RGB image).

At this time, the 3D avatar creation device may perform 3D scanning using only depth images, only color images, or a combination of the two.

The user's 3D scanning model (Source 3D Scan) created in this way may be the result of scanning only a part of the human body, or it may be the complete result of scanning every corner of 360 degrees. Additionally, in the case of a 3D scanning model, there may be holes or gaps due to limitations of the sensor or algorithm.

The 3D avatar creation device can match the 3D scanning model with a pre-stored template avatar model (Correspondences, Registration) through a model matching unit (3D Correspondence Module, Registration Module).

At this time, the template avatar model may correspond to a standard avatar model representing the human body. At this time, the template avatar model may include skeleton information and vertex weight information.

At this time, if the template avatar model does not include skeleton information and vertex weight information, the 3D avatar creation device may update the skeleton information and vertex weight information through an automatic rigging algorithm.

At this time, the 3D avatar creation device may match the 3D scanning model and the template avatar model using information on corresponding points between the 3D scanning model and the template avatar model.

At this time, the 3D avatar creation device uses a 6-degree-of-freedom rigid body transformation that corrects the approximate translation, orientation, and scale of the 3D scanning model and the template avatar model. Transformation) can be applied.

At this time, the corresponding point information may include information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point.

At this time, at least one predefined position in the human body part may include at least one position among the face part, hand part, and foot part of the human body.

At this time, the 3D avatar creation device may define the corresponding points of the 3D scanning model and the corresponding points of the template avatar model as a corresponding pair based on at least one predefined position in the body part.

At this time, the 3D avatar creation device can recognize the pose of the 3D scanning model and automatically search for corresponding points of the 3D scanning model through an algorithm.

At this time, the 3D avatar creation device largely uses at least one of three processes: global rigid registration, skeleton-based human pose registration, and local non-rigid registration. You can do more than this.

The 3D avatar creation device uses the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and The direction can be rotated to match the template avatar model (Global translation, rotation, scale).

At this time, the 3D avatar creation device can use the corresponding point information to obtain the overall 6 degrees of freedom translation and rotation information and scale information of the template avatar model based on the 3D scanning model and roughly match them.

The 3D avatar creation device can transform the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and the skeleton information of the template avatar model (Skeleton based Human Pose Registration).

To express subtle changes in the skeletal structure of a 3D avatar, the subtle translation and rotation changes of each segment of the skeleton must be reflected (Skeleton Translation, Rotation). Additionally, when the same person is partially scanned multiple times, the shape of a person's pose may change over time, so a registration method that can adapt to changes in the human body's pose is needed.

For example, the 3D avatar creation device additionally uses a template avatar model modified based on the first 3D scanning model generated by scanning the user's front part, and a second 3D scanning model generated by scanning the side part. When transforming the avatar again, if the poses are different, the pose of the template avatar model can be transformed using the skeleton information to match the 3D scanning model.

The template avatar model may include the correlation between each predefined skeleton and each node.

In other words, since the template avatar model is rigged, changes in the posture of the skeleton can be reflected as changes in the posture of the template avatar.

At this time, the most commonly used rigging technique for the template avatar model is Linear Blend Skinning. The rigging technique defines a weight value that represents the influence of each skeleton at each node, and each node has a movement component equal to this weight value along the movement of each skeleton, which can be expressed as Equation 1. there is.

In Equation 1, V _i is the position of one node of the template avatar model, N _b is the number of bones in the entire skeleton model, w _j is the weight value of the skeleton, and T _-j is the rigid body deformation matrix of the jth skeleton. And V _j ⁰ may mean the local coordinates of the jth skeleton of the corresponding node in the reference posture.

At this time, the 3D avatar creation device may optimize the joint angles and joint coordinates of the skeleton to match the posture of the template avatar model similar to the user's 3D scanning model.

At this time, the 3D avatar creation device can perform an operation to find the rigid body matrix of the joints that minimizes the sum of the values of the three error functions as shown in Equation 2.

In Equation 2, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the It represents the weight, and finally, E _s is a function that minimizes the sum of the rotation angles of the joint angles, and γ represents the weight.

The error function E _d to minimize the skeleton distance between the template avatar model and the user's 3D scanning model can be defined as Equation 3.

In Equation 3, the dist() function represents a function that calculates the minimum distance between the point skin (V _i ) deformed by the rigid body matrix of the current joint and the surface D of the user's 3D scanning model, and N _m template avatar You can see that the number of nodes is indicated.

At this time, since the corresponding points of the 3D scanning model input from the user may converge to the local minima, to prevent this, the error function E between the corresponding points of the template avatar model and the user's 3D scanning model _f can be defined as in Equation 4.

In Equation 4, F _M and F _D are the corresponding points defined by the user for each template avatar M and the user's 3D scan information D, and the point skin(F _M ) transformed by the rigid body matrix of the joint where F _M is F _D It can be calculated so that the distance from is minimal. Here, N _f represents the number of user-defined corresponding points.

And Equation 5 can be defined using quaternion, which means rotation of the joint angle, as an error function to maintain the shape of the template avatar model M.

In Equation 5, Nb is the number of bones, qi is the rotation of joint i, and it can be seen that it represents the objective function of minimizing the sum of the rotation angles of the joint angles.

That is, the 3D avatar creation device uses the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model to change the posture of the template avatar model to the corresponding point of the 3D scanning model. The minimum distance to corresponding points may be calculated, and the posture of the template avatar model may be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

The 3D avatar creation device can calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model by adjusting the positions of individual nodes of the template avatar model through an Affine transformation matrix (Affine Transform per Vertex).

At this time, the 3D avatar creation device can adjust the positions of individual nodes of the template avatar model M through an Affine Transformation matrix to minimize the distance from the user's 3D scan information D.

At this time, the 3D avatar creation device can define the error function as shown in Equation 6.

In Equation 6, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the weight. , and finally, it can be seen that E _s represents constraints to maintain the basic appearance of the template avatar.

At this time, the 3D avatar creation device can first define an error function E _d that minimizes the distance between the template avatar model M and the surface D of the user's 3D scanning model, as shown in Equation 7.

In Equation 7, it can be seen that the dist() function represents a function that calculates the minimum distance between the node V _i of the template avatar transformed by the affine transformation matrix T _i and the user's 3D scan information surface D, It can be seen that N _m represents the number of template avatar nodes. Here, the distance can be determined by measuring the closest distance to the triangle mesh.

At this time, the 3D avatar creation device can define the error function of the corresponding point receiving the user input as shown in Equation 8.

In Equation 8, F _M and F _D are corresponding points defined by the user between each template avatar M and the user's 3D scan information D, and the distance between the point where F _M is transformed by the T _i affine matrix and F _D is the minimum. Make it possible. Here, N _f indicates the number of user-defined corresponding points.

Lastly, the 3D avatar creation device can define an error function that minimizes the difference in the transformation matrices of all edges as a constraint to maintain the shape of the template avatar model M.

If two nodes of an arbitrary edge e _i undergo similar transformations, the difference between the transformation matrices of the two nodes will converge to the 0 matrix. Using this, Equation 9 can be defined.

It can be seen that the matrix of two nodes of an arbitrary edge is expressed as T _i and T _j , where N _s represents the number of edges of the template avatar M.

Additionally, the 3D avatar creation device may use one step independently or a combination of two or more steps depending on the use case in the case of global rigid body registration, skeleton-based pose matching, and local non-rigid body registration.

For example, if the 3D avatar creation device matches the translation/rotation/scale of the overall template avatar with the 3D scan information in advance, it does not perform global rigid body registration, but also uses skeleton-based pose matching and local non-rigid body registration. The template avatar model and the 3D scanning model can be matched.

Additionally, when the poses of the 3D scanning model and the template avatar model are the same, step S220 may not perform skeleton-based pose matching but only local non-rigid body matching.

Additionally, the 3D avatar creation device may only perform the matching function for which the matching function has been appropriately selected by the user according to the usage example.

Additionally, the 3D avatar creation device may generate a 3D avatar (User Avatar) based on a result of matching the 3D scanning model and the template avatar model.

At this time, the 3D avatar may correspond to a refined, animation-capable avatar in which rigging information and vertex weight information are reflected by reflecting the user's body shape.

At this time, the 3D avatar creation device can display exercise prescription animations to the user using the generated 3D avatar, and can also be used as the user's character (motion information) in games, etc. (Animation Module).

At this time, the 3D avatar can provide the user's body shape information using 3D surface information and skeleton information (Body Measurement Module). For example, circumference information such as head circumference and chest circumference can be obtained through the position defined in the 3D avatar's skeleton information and plane information with the skeleton direction as the normal, or the arm-leg ratio and waist can be obtained using the skeleton's ratio. -You can obtain the hip ratio, etc., and use vertex information to provide volume information of the user's avatar. At this time, the body shape information of the 3D avatar can be automatically obtained based on the skeleton information of the 3D avatar, so body shape information for each user can be consistent and can be easily used for comparison according to the user's history ( Body Shape Information).

At this time, the 3D avatar creation device can generate a 3D avatar that can gradually reflect the user's body shape even in a 3D scanning model of a mismatched human body part.

At this time, the 3D avatar creation device can generate an avatar that reflects a part of the user's body shape based on the partial scan results of the human body even in situations where 3D scanning as a whole is not possible.

At this time, the 3D avatar creation device can transform the avatar based on the scan results of additional human body parts, thereby gradually improving the reflection of the user's body shape.

At this time, the 3D avatar creation device can generate a refined mesh-based avatar that can be animated.

At this time, the 3D avatar creation device can generate a 3D avatar reflecting the user's body type that can be animated despite the incomplete 3D scan results of the human body.

At this time, the 3D avatar creation device is a low-cost (consumer-level) depth sensor or image sensor that has noise, holes, or incompleteness of the 3D scan result due to limitations of the 3D scan algorithm. You can overcome this and create a 3D avatar.

At this time, the 3D avatar creation device can operate through animation even with incomplete 3D scan results and can generate a refined mesh-based 3D avatar that reflects the user's body type.

At this time, the 3D avatar creation device can automatically measure the volume, ratio, and circumference information of a consistent skeleton-based body shape for each user using the resulting avatar that reflects the user's 3D body shape.

At this time, the 3D avatar creation device may measure the volume of the user's body shape using surface mesh information of the generated 3D avatar.

At this time, the 3D avatar creation device can measure the human body proportions using the skeleton information of the generated 3D avatar and the bounding box of the surface.

At this time, the 3D avatar creation device can automatically measure the perimeter using the skeleton position information and surface information of the generated 3D avatar.

Figure 6 is a diagram showing the model matching process according to an embodiment of the present invention.

Referring to Figure 6, the 3D avatar creation device largely consists of three processes: Global Rigid Registration, Skeleton-based Human Pose Registration, and Local Non-rigid Registration. You can do at least one of the following.

The 3D avatar creation device uses the corresponding point information to move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and The direction can be rotated to match the template avatar model (Global translation, rotation, scale).

At this time, the 3D avatar creation device can use the corresponding point information to obtain the overall 6 degrees of freedom translation and rotation information and scale information of the template avatar model based on the 3D scanning model and roughly match them.

The 3D avatar creation device can transform the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and the skeleton information of the template avatar model (Skeleton based Human Pose Registration).

To express subtle changes in the skeletal structure of a 3D avatar, the subtle translation and rotation changes of each segment of the skeleton must be reflected (Skeleton Translation, Rotation). Additionally, when the same person is partially scanned multiple times, the shape of a person's pose may change over time, so a registration method that can adapt to changes in the human body's pose is needed.

For example, the 3D avatar creation device additionally uses a template avatar model modified based on the first 3D scanning model generated by scanning the user's front part, and a second 3D scanning model generated by scanning the side part. When transforming the avatar again, if the poses are different, the pose of the template avatar model can be transformed using the skeleton information to match the 3D scanning model.

The template avatar model may include the correlation between each predefined skeleton and each node.

In other words, since the template avatar model is rigged, changes in the posture of the skeleton can be reflected as changes in the posture of the template avatar.

At this time, the most commonly used rigging technique for the template avatar model is Linear Blend Skinning. The rigging technique defines a weight value that represents the influence of each skeleton at each node, and each node has a movement component equal to this weight value along the movement of each skeleton, which can be expressed as Equation 1. there is.

In Equation 1, V _i is the position of one node of the template avatar model, N _b is the number of bones in the entire skeleton model, w _j is the weight value of the skeleton, and T _-j is the rigid body deformation matrix of the jth skeleton. And V _j ⁰ may mean the local coordinates of the jth skeleton of the corresponding node in the reference posture.

At this time, the 3D avatar creation device may optimize the joint angles and joint coordinates of the skeleton to make the posture of the template avatar model similar to the user's 3D scanning model.

At this time, the 3D avatar creation device can perform an operation to find the rigid body matrix of the joints that minimizes the sum of the values of the three error functions as shown in Equation 2.

In Equation 2, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the It represents the weight, and finally, E _s is a function that minimizes the sum of the rotation angles of the joint angles, and γ represents the weight.

The error function E _d to minimize the skeleton distance between the template avatar model and the user's 3D scanning model can be defined as Equation 3.

In Equation 3, the dist() function represents a function that calculates the minimum distance between the point skin (V _i ) deformed by the rigid body matrix of the current joint and the surface D of the user's 3D scanning model, and N _m template avatar You can see that the number of nodes is indicated.

At this time, since the corresponding points of the 3D scanning model input from the user may converge to the local minima, to prevent this, the error function E between the corresponding points of the template avatar model and the user's 3D scanning model _f can be defined as in Equation 4.

In Equation 4, F _M and F _D are the corresponding points defined by the user for each template avatar M and the user's 3D scan information D, and the point skin(F _M ) transformed by the rigid body matrix of the joint where F _M is F _D It can be calculated so that the distance from is minimal. Here, N _f represents the number of user-defined corresponding points.

And Equation 5 can be defined using quaternion, which means rotation of the joint angle, as an error function to maintain the shape of the template avatar model M.

In Equation 5, Nb is the number of bones, qi is the rotation of joint i, and it can be seen that it represents the objective function of minimizing the sum of the rotation angles of the joint angles.

That is, the 3D avatar creation device uses the skeleton information to minimize the error between the position of the corresponding point of the template avatar model and the 3D scanning model to change the posture of the template avatar model to the corresponding point of the 3D scanning model. The minimum distance to corresponding points may be calculated, and the posture of the template avatar model may be modified to match the 3D scanning model based on the minimum distance between the corresponding points and the minimum distance between individual nodes of the template avatar model.

The 3D avatar creation device can calculate the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model by adjusting the positions of individual nodes of the template avatar model through an Affine transformation matrix (Affine Transform per Vertex).

At this time, the 3D avatar creation device can adjust the positions of individual nodes of the template avatar model M through an Affine Transformation matrix to minimize the distance from the user's 3D scan information D.

At this time, the 3D avatar creation device can define the error function as shown in Equation 6.

In Equation 6, E _f represents the distance error function between corresponding points, α represents the weight of the function, E _d is the error function to minimize the distance between the template avatar and the user's 3D scan information, and β is the weight. , and finally, it can be seen that E _s represents constraints to maintain the basic appearance of the template avatar.

At this time, the 3D avatar creation device can first define an error function E _d that minimizes the distance between the template avatar model M and the surface D of the user's 3D scanning model, as shown in Equation 7.

In Equation 7, it can be seen that the dist() function represents a function that calculates the minimum distance between the node V _i of the template avatar transformed by the affine transformation matrix T _i and the user's 3D scan information surface D, It can be seen that N _m represents the number of template avatar nodes. Here, the distance can be determined by measuring the closest distance to the triangle mesh.

At this time, the 3D avatar creation device can define the error function of the corresponding point receiving the user input as shown in Equation 8.

In Equation 8, F _M and F _D are corresponding points defined by the user between each template avatar M and the user's 3D scan information D, and the distance between the point where F _M is transformed by the T _i affine matrix and F _D is the minimum. Make it possible. Here, N _f indicates the number of user-defined corresponding points.

Lastly, the 3D avatar creation device can define an error function that minimizes the difference in the transformation matrices of all edges as a constraint to maintain the shape of the template avatar model M.

If two nodes of an arbitrary edge e _i undergo similar transformations, the difference between the transformation matrices of the two nodes will converge to the 0 matrix. Using this, Equation 9 can be defined.

It can be seen that the matrix of two nodes of an arbitrary edge is expressed as T _i and T _j , where N _s represents the number of edges of the template avatar M.

Additionally, the 3D avatar creation device may use one step independently or a combination of two or more steps depending on the use case in the case of global rigid body registration, skeleton-based pose matching, and local non-rigid body registration.

For example, if the 3D avatar creation device matches the translation/rotation/scale of the overall template avatar with the 3D scan information in advance, it does not perform global rigid body registration, but also uses skeleton-based pose matching and local non-rigid body registration. The template avatar model and the 3D scanning model can be matched.

Additionally, when the poses of the 3D scanning model and the template avatar model are the same, step S220 may not perform skeleton-based pose matching but only local non-rigid body matching.

Additionally, the 3D avatar creation device may only perform the matching function for which the matching function has been appropriately selected by the user according to the usage example.

Figures 7 to 10 are diagrams showing a model matching process between a 3D scanning model and a template avatar model according to an embodiment of the present invention.

Referring to Figures 7 and 8, the 3D model generating device generates 3 based on the connection line between the corresponding point of the pre-stored template avatar model 10 and the corresponding point of the input user's 3D scanning model 20, as shown in Figure 7. It can be seen that global rigid body registration is performed as shown in FIG. 8 by moving the position and rotating the direction of the dimensional scanning model 20 and matching it with the template avatar model 10.

Referring to FIGS. 8 and 9, the 3D avatar creation device uses skeleton information to minimize the error between the posture of the template avatar model 10 and the posture of the user's 3D scanning model 20, as shown in FIG. 8. Thus, it can be seen that the pose of the template avatar model 10 is transformed and skeleton-based pose matching is performed as shown in FIG. 9.

Referring to FIGS. 9 and 10 , the 3D avatar creation device creates a template avatar model ( It can be seen that local non-rigid registration as shown in Figure 10 is performed by minimizing the distance between the surface of the corresponding point of the template avatar model 10 and the corresponding point of the user's 3D scanning model 20 using the vertex information in 10). there is.

That is, as shown in FIGS. 7 to 10, the 3D avatar creation device converts the template avatar model 10 into the user's 3D scanning model 20 using global rigid body registration, skeleton-based pose registration, and local non-rigid body registration. ) can be seen to match.

Figure 11 is a diagram showing a computer system according to an embodiment of the present invention.

Referring to FIG. 11, the 3D avatar creation device according to an embodiment of the present invention may be implemented in a computer system 1100 such as a computer-readable recording medium. As shown in FIG. 11, the computer system 1100 includes one or more processors 1110, a memory 1130, a user interface input device 1140, and a user interface output device 1150 that communicate with each other through a bus 1120. and storage 1160. Additionally, the computer system 1100 may further include a network interface 1170 connected to the network 1180. The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or storage 1160. Memory 1130 and storage 1160 may be various types of volatile or non-volatile storage media. For example, memory may include ROM 1131 or RAM 1132.

Additionally, one or more processors 1110 may execute one or more programs.

At this time, one or more processors 1110 may include a 3D scanning unit 110, a model matching unit 120, and an avatar generating unit 130, and one or more processors 1110 may include a 3D scanning unit 110. ), the functions of the model matching unit 120 and the avatar creating unit 130 can be performed through one or more programs.

As described above, the 3D avatar generating device and method according to an embodiment of the present invention is not limited to the configuration and method of the embodiments described above, and the embodiments may be modified in various ways. All or part of each embodiment may be configured by selectively combining them.

10: Template avatar model 20: 3D scanning model
110: 3D scanning unit 120: model matching unit
121: Global rigid body matching unit 122: Skeleton matching unit
123: Local non-rigid body matching unit 130: Avatar creation unit
1100: computer system 1110: processor
1120: Bus 1130: Memory
1131: Romans 1132: Ram
1140: User interface input device
1150: User interface output device
1160: Storage 1170: Network Interface
1180: network

Claims (20)

In the 3D avatar creation method of the 3D avatar creation device,
3D scanning the user's human body using an image sensor and generating a 3D scanning model using the results of the 3D scanning of the user's body;
Matching the 3D scanning model with a pre-stored template avatar model; and
generating a 3D avatar based on a result of matching the 3D scanning model and the template avatar model;
Including,
The step of creating the 3D avatar is
If the 3D scanning model is incomplete, generating a 3D avatar reflecting the user's incomplete body shape based on an additional partial scan of the human body for the user's incomplete body shape,
The matching step is
Matching the 3D scanning model and the template avatar model using correspondence point information between the 3D scanning model and the template avatar model,
The above corresponding point information is
Contains information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point,
The matching step is
Transforming the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and the skeleton information of the template avatar model,
Modifying the posture of the template avatar model by additionally using a second 3D scanning model generated by scanning the side of the template avatar model modified based on the first 3D scanning model generated by scanning the front part of the user. A method of creating a 3D avatar, characterized in that, when the poses are different, the skeleton information is additionally used to modify the pose of the template avatar model and match it with the 3D scanning model. delete delete In claim 1,
At least one predefined location in the body part is
A method for creating a three-dimensional avatar, comprising at least one location among the face, hand, and foot of the human body. In claim 4,
The matching step is
A method for creating a 3D avatar, characterized in that defining corresponding points of the 3D scanning model and corresponding points of the template avatar model as a corresponding pair based on at least one predefined position in the human body part. In claim 1,
The matching step is
Using the corresponding point information, move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and rotate the direction of the 3D scanning model to display the template. A three-dimensional avatar creation method characterized by applying rigid body transformation to the avatar. In claim 6,
The matching step is
A 3D avatar creation method characterized by applying a 6-degree-of-freedom rigid body transformation that corrects the movement, direction, and scale of the 3D scanning model and the template avatar model. delete In claim 6,
The matching step is
Calculate the minimum distance between the corresponding point of the modified posture of the template avatar and the corresponding point of the 3D scanning model using the skeleton information to minimize the error between the position of the corresponding point of the template avatar and the 3D scanning model, and A 3D avatar creation method characterized by modifying the posture of the template avatar and matching it to the 3D scanning model based on the minimum distance between corresponding points and the minimum distance between individual nodes of the template avatar model. In claim 9,
The matching step is
A method for creating a 3D avatar, characterized in that the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model is calculated by adjusting the positions of individual nodes of the template avatar model through an affine transformation matrix. One or more processors;
Memory; and
contains one or more programs,
the one or more programs are stored in the memory and executed by the one or more processors,
The one or more processors execute the one or more programs,
3D scanning the user's body using an image sensor, creating a 3D scanning model using the results of 3D scanning the user's body, matching the 3D scanning model with a pre-stored template avatar model, Generating a 3D avatar based on a result of matching the 3D scanning model and the template avatar model,
The one or more processors
If the 3D scanning model is incomplete, generating a 3D avatar reflecting the user's incomplete body shape based on an additional partial scan of the human body for the user's incomplete body shape,
Matching the 3D scanning model and the template avatar model using correspondence point information between the 3D scanning model and the template avatar model,
The above corresponding point information is
Contains information defining at least one predefined position in the human body part of the 3D scanning model and the template avatar model as a corresponding point,
The one or more processors
Transforming the posture of the template avatar model to correspond to the posture of the 3D avatar model using the corresponding point information and the skeleton information of the template avatar model,
Modifying the posture of the template avatar model by additionally using a second 3D scanning model generated by scanning the side of the template avatar model modified based on the first 3D scanning model generated by scanning the front part of the user. 3D avatar creation device, characterized in that, when the poses are different, the skeleton information is additionally used to modify the posture of the template avatar model and match it with the 3D scanning model. delete delete In claim 11,
At least one predefined location in the body part is
A three-dimensional avatar creation device comprising at least one of the face, hand, and foot of the human body. In claim 14,
The one or more processors
A 3D avatar creation device characterized in that the corresponding point of the 3D scanning model and the corresponding point of the template avatar model are defined as a corresponding pair based on at least one predefined position in the human body part. In claim 11,
The one or more processors
Using the corresponding point information, move the position of the 3D scanning model based on the position of the corresponding point of the template avatar model and the position of the corresponding point of the 3D scanning model, and rotate the direction of the 3D scanning model to display the template. A three-dimensional avatar creation device characterized by applying rigid body transformation to the avatar. In claim 16,
The one or more processors
A 3D avatar creation device characterized by applying a 6-degree-of-freedom rigid body transformation that corrects the movement, direction, and scale of the 3D scanning model and the template avatar model. delete In claim 16,
The one or more processors
Calculate the minimum distance between the corresponding point of the modified posture of the template avatar and the corresponding point of the 3D scanning model using the skeleton information to minimize the error between the position of the corresponding point of the template avatar and the 3D scanning model, and A 3D avatar creation device, characterized in that the posture of the template avatar is modified based on the minimum distance between corresponding points and the minimum distance between individual nodes of the template avatar model and matched to the 3D scanning model. In claim 19,
The one or more processors
A 3D avatar creation device, characterized in that the minimum distance between the surface of the template avatar model and the surface of the 3D scanning model is calculated by adjusting the positions of individual nodes of the template avatar model through an affine transformation matrix.

Images