KR102068251B1 - Method of generating 3d volumetric data - Google Patents

Abstract

The present invention relates to a method for generating 3D volume data, the method comprising: generating a multilayer image, generating a type and volume information of a visible portion of an object based on the generated multilayer image, and generating the multi And generating type and volume information of an invisible portion of the object based on the layer image.

Description

3D volume data generation method {METHOD OF GENERATING 3D VOLUMETRIC DATA}

The present invention discloses a technical idea for generating three-dimensional volume data of an object that can be used as input data of recognizer learning for volume reconstruction of an object.

In everyday life, humans use both eyes, so even a low-resolution human body pose can be recognized well. However, the method of recognizing human body pose in computer vision systems is a difficult problem that is required in various fields but is not well solved.

Recently, research and development of a technology for controlling a user interface by sensing a user's body motion has been accelerated.

However, current motion sensing technology is not only able to recognize the visible part of the object, it is impossible to recognize the hidden part caused by occlusion, etc., and its application range is Human Object (Virtual Body Parts) and depth image (Depth Image). Is limited to).

Conventionally, body posture data of a 3D object model is generated by retargeting a 3D model by capturing motion of a body. However, 3D volume data reflecting the user's body type could not be generated.

3D volume data generation method according to an embodiment of the present invention comprises the steps of generating a multilayer image, generating the type and volume information of the visible portion of the object, based on the generated multilayer image, and the generation And generating type and volume information of an invisible portion of the object based on the multilayered image.

According to an embodiment of the present invention, a method of generating 3D volume data includes reading a multilayer image, performing segmentation on the same object part identification information from the read multimedia image, and performing the segmentation. Estimating a position corresponding to the same object part identification information according to segmentation, and estimating a skeleton model using the estimated position.

1 is a main flowchart illustrating a 3D volume data generation method according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating visible data and invisible data. FIG.
FIG. 3 is a view illustrating a depth image of a visible layer as an image of a posture of an object captured by a depth camera.
FIG. 4 is a diagram for explaining an embodiment of an image of object part identification information of a visible layer.
5 is a diagram for explaining an embodiment of a multilayer body part identification image / multilayer depth image.
6 is a view for explaining a method of generating a multilayer image based on ray casting.
FIG. 7 is a diagram illustrating a method of generating a multilayer image based on ray casting, according to a first embodiment of generating a multilayer image.
FIG. 8 is a diagram illustrating a Ray-Casting Based Training Data Synthesis block in a first embodiment.
FIG. 9 is a diagram illustrating a slicing based multilayer image generating method as a modification of the first embodiment of generating the multilayer image.
10 is a diagram illustrating a slicing-based multilayer image generation method.
FIG. 11 is a diagram illustrating an example 1100 in which an actor actually wears a color patch suit composed of color patches.
12 is a diagram for explaining a Ray-Triangle Intersection Method.
FIG. 13 illustrates a method for generating a multilayer image based on chroma key screens as a second embodiment of generating the multilayer image.
14 is a view for explaining an example of an object part identification information-RGB lookup table in a chroma key screen-based multilayer image generation method.
15 is a diagram for explaining two chroma key screen installation examples.
FIG. 16 is a diagram illustrating a method for generating a multilayer image based on an initial template, as a third embodiment of generating a multilayer image.
FIG. 17 is a diagram illustrating an object part lookup table according to an initial template.
FIG. 18 is a view for explaining an example of measuring each object part property of an object.
FIG. 19 is a view for explaining an embodiment of determining each object part property of an object.
FIG. 20 is a diagram illustrating an embodiment in which an initial template of a two-dimensional object part is matched on motion capture data.
FIG. 21 is a diagram illustrating an embodiment in which an initial template of a 3D object part is matched on motion capture data.
FIG. 22 illustrates an example of generating an image within a depth range when there is a depth camera input in FIG. 17.
23 is a flowchart illustrating a method of generating skeleton data in a multilayer image.
24 is a view for explaining an embodiment of estimating a skeleton model based on a multilayer image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In describing the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Terminology used herein is a term used to properly express a preferred embodiment of the present invention, which may vary depending on the intention of a user, an operator, or customs in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification. Like reference numerals in the drawings denote like elements.

1 is a main flowchart illustrating a 3D volume data generation method according to an embodiment of the present invention.

The 3D volume data generation method according to an embodiment of the present invention is to generate recognizer learning input data for volume reconstruction of an object, and generates visible data through step 101, and invisible data in step 102. Create

Specifically, the three-dimensional volume data generation method according to an embodiment of the present invention generates the type and volume information of the visible portion (invisible data) of the object, the type and volume of the invisible portion (invisible data) of the object Generate information.

To this end, the method for generating 3D volume data according to an embodiment of the present invention includes at least one multilayer image among a ray casting based multilayer image, a chroma key screen based multilayer image, and an initial template based multilayer image. And type and volume information for each of the visible and invisible portions of the object may be generated using the generated multilayer image.

FIG. 2 is a diagram illustrating visible data and invisible data. FIG.

In the present invention, the visible part of the object photographed by the image sensor 201 is defined as the visible object part 202, and the visible data is data generated by photographing the visible object part 202.

A hidden object part 203 is defined as a hidden object part 203 that is not visible behind or inside the object photographed by the image sensor 201, and the invisible data is data estimated from the visible data.

When capturing an object in the image sensor 201, the part in which the image is formed in the sensor plan is defined as a visible object part, and the occlusion is caused by self occlusion or another object of the object. The object area that occurs can be defined as a hidden object part.

For example, three surfaces viewed in the front view 202 of the cube may be defined as three visible object parts, and three surfaces viewed in the rear view 203 of the cube may be defined as three hidden object parts.

FIG. 3 is a diagram illustrating an image 300 of capturing the posture of an object, and illustrates a depth image of a visible layer as the image 300 capturing the posture of an object with a depth camera. do.

FIG. 4 is a diagram for explaining an embodiment of an image of object part identification information of a visible layer.

FIG. 4 illustrates Labeled Data including object parts ID information of an object present in depth data, which is an image of a posture of an object captured by a depth camera. Display an example.

In other words, in FIG. 4, as the object part ID image of the visible layer, different object IDs are assigned to each object part, thereby providing all objects. Identify the parts.

5 is a diagram for explaining an embodiment of a multilayer body part identification image / multilayer depth image.

In detail, the first image 510 of FIG. 5 illustrates an object part ID and a depth image of a visible layer.

The second through fifth images 520 illustrate the object part identification information and the depth image of the invisible layer.

6 is a view for explaining a method of generating a multilayer image based on ray casting.

Specifically, FIG. 6 illustrates a conceptual diagram of a method of generating a ray-cast based multi-layer image.

Ray-casting-based multi-layer image generation method generates a virtual ray 603 in the direction of a three-dimensional object 602 from the position of the camera 601, so that the ray 603 is three-dimensional Each time the object 602 passes through the surface of the object 602, the pixel of the touched surface is stored in the image.

At this time, the surface where Ray 603 first meets is stored as the visible layer 604, and the back surface that meets while passing through the 3D object 602 is stored as each invisible layer 605. Accordingly, not only the visible object part but also the invisible object part may be generated as a ray-casting based multi-layer image generating method.

FIG. 7 is a diagram illustrating a method of generating a multilayer image based on ray casting, according to a first embodiment of generating a multilayer image.

7 is a first embodiment of a method of generating a cast-based multilayer image.

In order to generate a ray casting-based multilayer, motion data 703 is generated by inverse kinematics (IK) processing 701 obtained after capturing the motion of an object.

In addition, in order to generate a ray casting-based multilayer, a three-dimensional object model composed of a combination of 3D meshes is generated by reflecting the actual body shape of the object separately from the inverse kinematics (IK) process 701 (702). .

Next, in order to generate a ray casting-based multilayer, the generated motion data and the generated three-dimensional object model are merged to generate a motion augmented 3D human model combined with motion. do.

In this case, retargeting may be performed on the generated 3D object model to match motion data to a 3D object model having a different body shape to generate a ray casting based multilayer (704).

Finally, a multilayer image may be generated as the ray casting based multilayer image generating method described with reference to FIG. 6 (705).

FIG. 8 illustrates a method of generating a ray casting based multilayer image according to a first embodiment.

The ray casting based multilayer image generation method, that is, the ray-casting based training data synthesis method, may first generate a ray casting based ray map (step 801).

Next, in the ray casting based multilayer image generation method, a point is selected from the generated ray map (step 802), and a plurality of vertices, for example, three vertices are generated from the generated three-dimensional object model (Human Model). Can be selected (step 803).

Next, the ray casting based multilayer image generating method may determine whether a point of a ray is located inside a plurality of vertices, for example, three vertices (step 804).

That is, the ray casting based multilayer image generating method may identify a relative position between the selected point and the plurality of vertices.

In other words, the ray casting-based multilayer image generation method selects one ray and selects one of a plurality of meshes constituting the 3D object model, for example, a plurality of meshes formed in the form of a polygon. In addition, it may be determined whether a ray intersects a plurality of vertices constituting one mesh and whether a point of a ray is located inside a triangle formed of a plurality of vertices, for example, three vertices.

By using this method, when a single point selected from a raymap and a plurality of vertices intersect, that is, when a point of the selected ray is located inside the three vertices, the multilayer image may be stored (step 805).

In addition, as a result of the determination in step 804, if it is not located inside, it is checked whether the vertex combination is the last (step 806). May be step 807. As a result of the determination in step 807, if all points of the ray map are examined, the generation of the multilayer image may be finished (step 808).

If it is determined in step 806 that the combination of the vertices is not the last one, the process branches to step 803 to select a plurality of new vertices from the 3D object model.

If it is determined in step 807 that all points of the raymap have not been examined, the process branches to step 802 to select a new point from the raymap.

To determine the intersection, the Ray-triangle Intersection Method may be used. Such a Ray-Triangle Intersection Method will be described in detail later with reference to FIG. 12.

9 is a diagram illustrating a slicing-based multilayer image generation method as a modified example of the first embodiment of generating the multilayer image.

The slicing-based multilayer image generating method generates motion data 903 by performing inverse kinematics (IK) processing 901 on data obtained after capturing the motion of an object.

In addition, in order to generate a slicing-based multilayer, a three-dimensional object model composed of a combination of three-dimensional meshes (3D meshes) is generated by reflecting the actual body shape of the object separately from the inverse kinematics (IK) process 901 (902).

Next, in order to generate a ray casting-based multilayer, the generated motion data and the generated three-dimensional object model are merged to generate a motion augmented 3D human model combined with motion. do.

In this case, retargeting may be performed on the generated 3D object model to match motion data with a 3D object model having a different body shape to generate a ray casting based multilayer (904).

Finally, a multilayer image may be generated as a slicing-based multilayer image generation method (905). The slicing-based multilayer image generation method will be described with reference to FIG. 10.

10 is a diagram illustrating a slicing-based multilayer image generation method.

As shown in FIG. 10, the slicing-based multilayer image generating method according to an embodiment of the present invention generates a motion augmented 3D human model (step 1001) or generates a primitive template 3D human model (step 1002).

Thereafter, all pixels (or voxels) of the model corresponding to the current depth value Z are searched using the generated Motion Augmented 3D Human Model or the generated Primitive Template 3D Human Model (step 1003). .

Next, the slicing-based multilayer image generation method according to an embodiment of the present invention generates a labeled image when there is a pixel corresponding to the current depth value Z among all the retrieved pixels (step 1004). In operation 1005, a depth image is generated together with the labeled image.

Next, the slicing-based multilayer image generating method according to an embodiment of the present invention increases the Z value by 1 (step 1006), and then determines whether Z is the last search and image generation process (step 1007). After the process is repeated until the end, image generation is terminated (step 1008).

FIG. 11 is a diagram for explaining an example 1100 in which an object actually wears a color patch suit composed of color patches.

The chroma key screen-based multilayer image generation method may generate a color camera image from a chroma key screen and a suit worn by a subject and to which different colors are applied for each object part. The number of color patches constituting the suit can be changed according to the object part.

To determine the intersection, the Ray-triangle Intersection Method can be used.

First, three vertices (p0, p1, p2) and Normal Vector n are defined as shown in Equation 1 as shown by reference numeral 1210 of FIG. 12.

[Equation 1]

n: normal of the plane

n = (p ₁ -p ₀ ) * (p ₂ -p ₀ )

n is a normal vector of planes consisting of p ₀ , p ₁ , and p ₂ vertices.

For reference, the normal vector n is perpendicular to the vector p ₂ -p ₀ in three-dimensional space and can be interpreted as perpendicular to the p ₁ -p ₀ .

Next, as shown by reference numeral 1220 or 1230, it may be determined whether the ray vector x passes through a plane composed of three vertices (p0, p1, p2). In this case, the normal vector n of the plane is also used as shown in [Equation 2].

[Equation 2]

(p ₁ -p ₀ ) × (xp ₀ ) · n≥0

(p ₂ -p ₁ ) × (xp ₁ ) · n≥0

(p ₀ -p ₂ ) × (xp ₂ ) · n≥0

For reference, reference numeral 1220 denotes an embodiment in which x, which is a ray vector, is located in the vertex, and reference numeral 1230 denotes an embodiment in which x is located outside the vertex, and is an embodiment for determining whether the ray vector is in or outside the vertex. .

CCW represents a direction, among which CCW ₀ , CCW ₁ , and CCW ₂ are directions toward x in the vertex, and indicate counterclockwise directions. In addition, CCW ₃ and CCW ₅ are the vertex toward x out, also counterclockwise and CCW ₄ is clockwise.

FIG. 13 illustrates a method for generating a multilayer image based on chroma key screens as a second embodiment of generating the multilayer image.

In order to generate a chroma key screen based multilayer image, a chroma key screen and a camera are installed (step 1301), and the actor may wear a suit having a different color for each object part (step 1302).

Subsequently, the method for generating a multilayer image based on chroma key screen may generate an RGB lookup table in which color patch information and object part identification information of the suit are recorded (step 1303), and generate a color camera image (step 1304). .

The RGB lookup table may refer to FIG. 14.

As shown in FIG. 14, the RGB lookup table 1400 is an example of installing two Chroma-Key Screens.

In the chroma key screen-based multilayer image generation method, the RGB lookup table 1400 associated with the object part identification information stores the color information (red, green, blue) of the color patch pixels (x, y), and Save the object part ID (Color Patch).

The input of the lookup table 1400 is color information RGB for a specific pixel of the color camera, and the output is object part identification information.

Referring back to FIG. 13, in the method for generating a multilayer image based on a chroma key screen, a color camera image may be generated from a chroma key screen and a suit 1100 worn by a subject and to which different colors are applied for each object part.

Next, the method for generating a multilayer image based on chroma key screen checks color information of a current position of the generated color camera image, compares the identified color information with the RGB lookup table, and corresponds to the color information. The object part identification information may be retrieved.

In detail, the method for generating a multilayer image based on chroma key screens reads the values of R, G, and B of the current position of the color image (step 1305), and determines whether the index is the same as the chroma key screen (step 1306).

When the identified color information is the same index as the chroma key screen, the chroma key screen-based multilayer image generation method stores the background image as a "background" in the current position of the labeled image.

Thereafter, the method for generating a multilayer image based on a chroma key screen may determine whether the pixel is the last pixel of the color camera image (step 1310), and generate the multilayer label image when the pixel is the last pixel (step 1311).

If the identified color information is different from the chroma key screen, the object part identification information corresponding to the color information is retrieved based on the RGB lookup table (step 1308) and stored (step 1309).

If it is determined in step 1310 that it is not the last pixel, the next position of the color image may be selected (step 1312), and the process may move to step 1305.

The multilayer image generating environment may be configured by installing a plurality of chroma key screens, a depth camera, and a color camera as shown in FIG. 15. In the same environment as that of FIG. 15, a multilayer image is generated using the method of FIG. 13.

In FIG. 15, two depth cameras 1501 and two color cameras 1502 are provided, respectively, but a plurality of chroma key screens, depth cameras, and color cameras may be installed.

In detail, the lookup table used in FIG. 13 may be generated from color patch information of a costume worn by an actor in an image captured by a color camera.

When the actor enters the environment of FIG. 15 for wearing the color patch costume and generates the multilayer data, the actor reads the red, green, and blue values of the current position (x, y) pixels of the color image. At this time, check whether the pixel is the same color information as the chroma key screen. If yes, the Labeled Image is saved as "background" at the current pixel position. If No, the object part identification information corresponding to RGB is retrieved from the lookup table. Labeled Image Stores object part identification information in the current location.

The number of color patches in the suit of the object is not limited.

If the current pixel is the last pixel of the input image, and if it is Yes, the creation of the Labeled Image is completed, and if No, it moves to the next pixel of the color image and continues creating the Labeled Image. At this time, a retro-reflective marker may be attached on the color patch suite to generate detailed motion data.

FIG. 16 is a diagram illustrating a method for generating a multilayer image based on an initial template, as a third embodiment of generating a multilayer image.

The initial template-based multilayer image generation method measures attribute values of each object part according to gender, age, and body type (step 1601).

Next, the initial template-based multilayer image generation method defines an initial template with the measured respective property values (step 1602), and generates an object part lookup table based on the defined initial template (step 1603). .

For example, an object part lookup table may be generated based on an initial template using the average value of gender, age, and body type as attributes.

The initial template-based multilayer image generating method prepares a motion capture system in parallel with generating the object part lookup table (step 1604), and generates motion capture data using the prepared motion capture system (step 1605). ).

For example, motion data may be generated using a plurality of IR cameras and a retro-reflective marker. At this time, the system for generating motion data is not limited to a specific type.

The initial template-based multilayer image generating method may receive the depth image from the depth camera in parallel with generating the object part lookup table and generating the motion capture data (step 1606).

Next, the initial template-based multilayer image generation method matches the initial template defined to each joint of the generated motion capture data based on the size and volume of the matched region based on the generated object part lookup table. (Step 1607).

Next, the initial template-based multilayer image generation method receives the depth image and generates an image within a selected depth range based on the input depth image (step 1608).

For example, the depth image may be generated from depth information recorded in the lookup table.

As another example, when the depth image and the motion data are acquired simultaneously by using the depth camera and the motion data generation system, the depth image is more accurately referred to the depth image range by referring to the size or volume of the matched region using the initial template. Can produce

Subsequently, in the initial template-based multilayer image generation method, a multilayer depth image may be generated from an image generated within the selected depth range (step 1609), and a multilayer labeled image may be generated (step 1610).

17 is a diagram illustrating an object part lookup table 1700 according to an initial template.

The object part lookup table 1700 stores attribute values measured for object parts according to gender, age, and body type. The object part lookup table 1700 may calculate and store an average and a standard deviation of attribute values of each object part.

18 is a view for explaining an example of measuring each object part property of the person 1800.

As shown in FIG. 18, each object part property may be calculated by measuring an attribute value of each object part according to gender, age, and body type to obtain an average and a standard deviation.

FIG. 19 is a diagram illustrating an embodiment of determining each object part property of a person 1900. As shown in FIG. 19, an attribute value measured for each object part may be determined in consideration of an object ratio.

FIG. 20 is a diagram illustrating an embodiment 2000 in which an initial template of a two-dimensional object part is matched on the motion capture data.

As shown in FIG. 20, the head is set to a circle, the shoulder is trapezoidal, the arms and torso, the leg is rectangular, the hand is semicircle, and the foot ID is different. The type of figure is not limited to the five types described above.

FIG. 21 illustrates an embodiment 2100 in which an initial template of a 3D object part is matched to motion capture data. Each object part is defined as an initial template by making a 3D body shape.

FIG. 22 is a diagram illustrating an example of generating an image within a depth range when there is a depth camera input in FIG. 17. In the range of Real Depth Data, FIG. An example of generating

23 is a flowchart illustrating a method of generating skeleton data in a multilayer image.

The method for generating skeleton data from the multilayer image reads the multilayer image (step 2301), and performs segmentation on the same object part identification information from the read multimedia image (step 2302).

According to the segmentation performed, a position corresponding to the same object part identification information is estimated (step 2303), and a skeleton model is estimated using the estimated position (step 2304).

24 is a diagram for explaining an embodiment of estimating a skeleton model 2400 based on a multilayer image.

23 and 21 is a method for generating a skeleton model can be used as a motion data capture system without a separate Retro-Reflective Marker.

As a result, according to the present invention, the output generated by the present invention may be used as input data of the recognizer learning for reconstructing the volume of the object, and may be utilized as data for recognizing the volume and posture of the object. In addition, 3D volume data of an object including a visible layer and an invisible layer may be generated.

3D volume data generation method according to an embodiment of the present invention is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

101: generation of visible data
102: Invisible data generation step

Claims (13)

Generating a multilayer image;
Generating type and volume information of the visible portion of the object based on the generated multilayer image; And
Generating type and volume information of an invisible portion of the object based on the generated multilayer image;
Including,
The generating of the multilayer image may include:
Generating motion data by performing motion capture of a subject;
Generating a 3D object model composed of a combination of 3D meshes to reflect the actual body shape of the object;
Performing retargeting to match the generated motion data to the generated three-dimensional object model; And
Generating a slicing-based multilayer image according to the retargeting result;
Generating the slicing-based multilayer image,
Generating a motion augmented 3D human model or a primitive template 3D human model based on the motion data and the 3D object model;
Generating a labeled image and a depth image when there is a pixel corresponding to a current depth value in the generated motion enhancement 3D object model or the initial template 3D object model; And
Generating the multilayer image using the generated labeled image and the depth image;
How to generate 3D volume data. The method of claim 1,
The generating of the multilayer image may include:
Generating motion data by performing motion capture of a subject;
Generating a 3D object model composed of a combination of 3D meshes to reflect the actual body shape of the object;
Performing retargeting to match the generated motion data to the generated three-dimensional object model; And
Generating a multilayer image based on the casting based on the retargeting result
3D volume data generation method comprising a. The method of claim 2,
The step of generating the multilayer image based on the ray casting,
Generating a ray map based on the ray casting;
Selecting a point from the generated ray map and selecting a plurality of vertices from the generated 3D object model; And
Determining whether to store the multilayer image based on a position of the selected point and the plurality of vertices.
3D volume data generation method comprising a. The method of claim 3,
The step of determining whether to store the multilayer image based on the position of the selected point and the plurality of vertices,
Checking whether the selected point is located within a range generated by the plurality of vertices
3D volume data generation method comprising a. delete delete The method of claim 1,
The generating of the multilayer image may include:
Generating a color camera image from a chroma key screen and a suit worn by a subject and having different colors applied to each body part;
Generating an RGB lookup table in which color patch information and object part identification information of the suit are recorded;
Checking color information of a current position of the generated color camera image, comparing the identified color information with the RGB lookup table, and retrieving object part identification information corresponding to the color information; And
Generating a multilayer image using the retrieved object part identification information;
3D volume data generation method comprising a. The method of claim 7, wherein
The searching of the object part identification information corresponding to the color information may include:
Checking the color information by reading R, G, and B values of the current position of the generated color camera image;
Determining whether the identified color information is an index such as a chroma key screen;
Retrieving and storing object part identification information corresponding to the color information based on the RGB lookup table when the identified color information is different from the chroma key screen.
3D volume data generation method comprising a. The method of claim 8,
The searching of the object part identification information corresponding to the color information may include:
If the identified color information is an index such as a chroma key screen, determining the checked color information as a background color
3D volume data generation method further comprising. The method of claim 1,
The generating of the multilayer image may include:
Measuring an attribute value of each of the object parts;
Defining an initial template with each measured attribute value, and generating an object part lookup table based on the defined initial template;
Generating motion capture data using the motion capture system;
Matching the defined initial template to each joint of the generated motion capture data; And
Receiving a depth camera image and generating an image within a predetermined depth range based on the input depth camera image;
3D volume data generation method comprising a. The method of claim 10,
Generating a multilayer depth image and a multilayer labeled image from the image within the selected depth range
3D volume data generation method further comprising. Generating a multilayer image;
Reading the multilayer image;
Performing segmentation on the same object part identification information from the read multimedia image;
Estimating a position corresponding to the same object part identification information according to the segmentation performed; And
Estimating a skeleton model using the estimated position
Including,
The generating of the multilayer image may include:
Generating a motion augmented 3D human model or a primitive template 3D human model based on the motion data and the 3D object model;
Generating a labeled image and a depth image when there is a pixel corresponding to a current depth value in the generated motion enhancement 3D object model or the initial template 3D object model; And
Generating the multilayer image using the generated labeled image and the depth image;
How to generate 3D volume data. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 1 to 4 and 7 to 12.

Images