JP2008176696A - Cg character animation creation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To intuitively and interactively operate a 3DCG character having a juncture structure of a joint largely different from the human body by a physical motion. <P>SOLUTION: A video rendered by use of a controllable characteristic amount extracted from a measurement value of a shape of the body of every moment as one or more pieces of attitude information (a bend of the joint or a size of a portion) of the 3DCG character is displayed in real time. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to a CG character animation creation device and a CG character animation creation method.

As a method of creating a motion of a CG character similar to a person in CG animation, a method of performing posture recognition by attaching a motion capture device to a human body and reconstructing it is common.
Jinxiang.ChaiandJessicaK.Hodgins, Performance Animation from Low-dimensional Control Signals, ACM Transactions on Graphics (TOG), Proceedings of SIGGRAPH 2005, No. 3, pp.686-696. Norman I. Badler, Michael J. Hollick, John P. Granieri: Real-Time Control of a Virtual Human Using Minimal Sensors. Presence 2 (1): 82-86 (1993) Semwal, S., Hightower, R., and Stansfield, S. 1998. Mapping algorithms for real-time control of an avatar using eight sensors.Presence 7, 1, 1-21. Yin, K., Pai, DK An interactive animation system.In Proc ACM SIGGRAPH / Eurographics Symposium on Computer Animation.pp.329-338. 2003.

However, as a method of creating a movement of a character whose shape is significantly different from that of the human body, it has traditionally been difficult to rely on frame-by-frame editing operations, such as changing a series of CG character movements with a GUI application using mouse input. It is the present condition that we do not get.

In the background art described above, CG animation creation requires a lot of time because it is created manually one frame at a time, and the creator needs specialized techniques such as editing equipment.

The apparatus according to the first aspect of the present invention extracts shape acquisition means for acquiring information on the position and shape of a user's body or a part thereof, and one or more amounts determined for each shape from the shape information as feature amounts. Feature amount extraction means, feature amount-posture conversion means for converting the feature amount into position / posture information representing the angle of the joint or part of the CG character, and surface shape forming means for forming the surface shape from the position / posture information The above object is achieved by providing rendering means for visualizing the surface shape to obtain an image and display means for displaying the image.

According to the above configuration, the movement of the CG character can be controlled in accordance with the change in the feature quantity indicating the movement of the whole body or part of the body of the user acquired by the feature quantity extraction unit. Here, the feature quantity-posture conversion means enables conversion of the CG character having a shape different from the skeleton of the human body into the three-dimensional position / posture information.

According to a second aspect of the present invention, the shape acquisition means includes a single or a plurality of photographing means arranged around the user, and a position shape estimation that estimates a position and a shape from an image obtained by the photographing means. Means may be provided.

According to the above configuration, the position and orientation of the user can be estimated without wearing a special device.

According to a third aspect of the present invention, the feature amount may be a position / orientation vector having the bending direction and magnitude and position of each joint as components.

With the above configuration, a CG character animation that reproduces the user's motion can be generated.

As described in claim 4, the feature quantity-posture conversion means is a temporary conversion means that represents the position and orientation information of the CG character using the temporary feature quantity (parameter), and which feature quantity each parameter is embodied as 2. A provisional feature correspondence table that describes, and an assigning device that substitutes one of the feature quantities into a parameter in each position and orientation information of the CG character according to the provisional feature correspondence table, and each means according to claim 1. In addition, provisional feature value correspondence table changing means for changing the contents of the temporary feature value correspondence table may be provided.

With the above configuration, it is possible to dynamically switch the assignment of the correspondence relationship to which feature amount the position / posture information of the part of the CG character corresponds. Here, the part of the CG character is a set of joints having position and orientation information controlled by the same parameters.

According to the CG character animation creation method of the present invention, it is possible to intuitively create a CG animation by directly transmitting a user's body motion to the character.

FIG. 4 is a diagram showing a usage scene of the embodiment of the present invention, and the user 2 operates the CG character 1 displayed on the display 5. This is common to the three embodiments described later. FIG. 1 shows a flow of processing for controlling the CG character 1 based on claim 2. The user 2 is photographed by the photographing means 11 and the video is sent to the position shape estimating means 6. The position shape estimation means 6 estimates the position shape information of the user 2 and sends it to the feature quantity extraction means 17. The feature quantity extraction unit 17 calculates a feature quantity from the position shape information and sends it to the feature quantity-posture conversion unit 7. The feature amount-posture conversion means 7 generates a character posture by converting the feature amount into position / posture information of the CG character 1 in accordance with a conversion formula arbitrarily determined in advance, and sends the character posture to the surface shape construction means 8. The surface shape forming means 8 determines the surface shape of the CG character 1 in the posture according to the surface shape of the CG character 1 determined in advance, and sends it to the rendering means 9. The rendering means 9 visualizes the surface shape and sends it to the display means 10. The display means 10 displays the video of the CG character 1.

FIG. 2 shows an implementation of the feature quantity / attitude conversion means 7 based on claim 4. The assigning device 19 substitutes the feature quantity into the parameter of the CG character 1 with reference to the correspondence relationship from the temporary feature quantity correspondence table 15 and sends it to the temporary conversion means 16. The temporary conversion means 16 calculates the position / orientation information of the CG character 1. On the other hand, the temporary feature quantity correspondence table changing unit 14 changes the contents of the temporary feature quantity correspondence table 15.

FIG. 3 shows the implementation of the provisional feature value correspondence table changing means 14. The image of the user 2 is captured by the imaging unit 21 and sent to the tracking point detection unit 12 and the display unit 20. The tracking point detection unit 12 detects a specific point on the body of the user 2 as a tracking point and sends it to the proximity part detection unit 13 and the display unit 20. The display means 20 displays the CG character 1 and the live user 2 and the tracking point, and the proximity part detection means 13 corresponds to the parameter of the part of the CG character 1 and the tracking point of the user 2 at a certain distance from each part. The feature quantities to be paired are registered in the temporary feature quantity correspondence table 15 and simultaneously, a highlight display command for the part is sent to the display means 20.

The first embodiment is based on the method described in claim 2. As an example of controlling the movement of the CG character 1 with the hand of the user 2's body, a method of mounting a marker on the fingertip of the user 2 and controlling with a feature value will be described.

FIG. 4 is a diagram showing an overview when the first embodiment is used. A video camera 3 (photographing means 11) for photographing the user 2, and a computer 4 for extracting feature amounts, converting position and orientation information to the CG character 1, determining the surface shape of the CG character 1, and visualizing the CG character 1. (Tracking point detection means 12, position shape estimation means 6, feature quantity extraction means 17, feature quantity-posture conversion means 7, surface shape construction means 8, rendering means 9), live video and CG character of user 2's hand A display 5 (display means 10) for displaying one video is required.

In the first embodiment, the position shape estimation unit 6 and the feature amount extraction unit 17 are simultaneously implemented by a moving rotation bending / stretching algorithm described below. The video camera 3 is disposed in front of the user 2 and photographs the user 2's hand. In the first embodiment, the user 2 wears red, green, and blue markers (22, 23, and 24) on the thumb, index finger, and middle finger of the hand to be photographed. The computer 4 obtains feature points corresponding to each part of the CG character 1 from the temporary feature value correspondence table 15.

In the first embodiment, the green marker 23 corresponds to the right arm of the CG character 1, the red marker 22 corresponds to the left arm of the CG character 1, and the blue marker 24 corresponds to both feet of the CG character 1.

The posture information of the CG character 1 is calculated according to the change of the feature amount by a conversion formula defined in advance for each part, and the joint of the part is bent. The live image of the marker and the image of the CG character 1 after the deformation visualized by rendering are displayed on the display 5. FIG. 5 is a diagram showing the state of the hand when the index finger is extended upward and the image of the CG character 1 at that time. Details of each of the above means will be described below.

The tracking point detection means 12 retrieves the colors of the red, green, and blue markers (22, 23, 24) from the video imaged by the video camera 3 and obtains the respective coordinates.

It is assumed that “the fingertip moves only on a line passing through the center of rotation of the hand and the angle between these lines does not change”. At this time, the moving rotation flexion / stretching algorithm, when given the initial value of the rotation center of the hand and the position of the three markers of red, green and blue in each frame, moves the hand as the position shape in each frame. L, hand rotation σ and finger bending a ₁ , a ₂ , a ₃ as feature quantities are calculated as follows.

When the position of the rotation center P of the hand in one frame and the three markers Q _i (i = 1,2,3) and the three markers Q _i ′ (i = 1,2,3) in the next frame are known The difference 1 = P′−P (difference for each coordinate component) between a ₁ , a ₂ , a _{3 in the} former frame and the rotation centers P ′ and P in the latter frame can be calculated as follows.

First, the relative position q _i '= Q _i ' −P 'of each marker from the rotation center P ′ of the next frame is expressed as a relative position q _i = Q _i −P from the rotation center of the marker of the current frame. Note that it looks like 1. (The unknown is a _i (i = 1,2,3), θ, l)

Assuming that θ << 0 and sin θ˜θ and cos θ˜1, Equation 1 is a linear equation and can be solved algebraically to obtain a _i (i = 1,2,3), θ and l. It is done. Further, at this time, the rotation center P ′ of the hand in the next frame can be calculated by P ′ = P + 1. Thus, by repeating the above procedure from the initial value of P, a _i (i = 1, 2, 3) of all frames can be obtained, and σ and L can be obtained by accumulating θ and l in each frame. .

In the feature quantity-posture conversion means 7, σ, L, a _i (i = 1, 2, 3) calculated by the moving rotation flexion / extension algorithm are expressed as joint angles (φ, ψ) (spherical coordinates) of the CG character 1. To position and orientation information. For example, with b as the reference length, conversion is performed as follows: φ = (π / 2) a _i / b, ψ = θ.

As a method of use of the first embodiment, the user 2 can watch the video of the CG character 1 displayed on the display 5 while keeping the angle between the fingers constant and bending the thumb, forefinger, and middle finger. The left arm and both feet can be operated.

The second embodiment is based on the method described in claim 3. As an example of controlling the movement of the CG character 1 by the hand of the user 2's body, a method for enabling a CG character operation by a data grab will be described. Computer 4 (shape acquisition means 18, feature quantity-attitude conversion means 7, which obtains the position and orientation of the data grab, converts the position and orientation information into the CG character 1, determines the surface shape of the CG character 1, and visualizes the CG character 1, Surface shape forming means 8, rendering means 9) and display 5 (display means 10) for displaying a CG character image are required.

According to the second embodiment, it is possible to obtain a realistic movement as if the puppet is virtually operated by the same manual operation as the actual puppet operation.

The third embodiment is based on the method described in claim 4. An example will be described in which a method of determining the corresponding parts of the user 2 and the CG character 1 by the user's 2 manual operation is described. A video camera 28 (imaging means 21) for photographing the user 2, a computer 4 (tracking point detection means 12, proximity part detection means 13) for detecting tracking points and associating tracking points with parts, and a user 2 A display 29 (display means 20) for displaying the image of the image, the tracking point, and the image of the CG character 1 is required.

In the third embodiment, the user 2 wears red, blue, green, and yellow markers (22, 23, 24, and 31) on the thumb, index finger, and middle finger of the hands to be photographed, and each of these markers is a _1. , A ₂ , a ₃ , θ tracking points (25, 26, 27, 30).

The computer 4 registers in the temporary feature correspondence table 15 a set of feature amounts corresponding to the tracking points (25, 26, 27, 30) of each color and corresponding to the tracking points overlapped with the parameters of the part of the CG character 1. At the same time, the part is highlighted on the display 29. As an example, FIG. 6 shows a temporary feature value correspondence table 15 in which the sets (α, σ) (β, a ₁ ) (γ, a ₂ ) and (η, a ₃ ) are registered.

The assigning device 19 calculates the position / orientation information of the CG character 1 by substituting the feature quantity described as a pair with the parameter in the provisional feature quantity correspondence table 15 into each parameter of the provisional conversion means 16. For example, when the temporary feature value correspondence table 15 is as shown in FIG. 6, σ is substituted for the parameter α in the _neck angle ψ _neck = α / π, and as a result, ψ _neck = σ / π.

FIG. 7 is a diagram illustrating a situation when the third embodiment is used. In the screen output to the display 29, the tracking point 26 of the green marker is overlapped with the left arm of the CG character 1 to specify the correspondence. Thereby, the left arm of the CG character 1 can be moved with the index finger. As described above, according to the third embodiment, the user 2 can specify the feature amount to move each part of the CG character 1 by an intuitive operation.

It is a figure which shows the flow of the process for controlling CG character 1. FIG. It is a figure which shows mounting of the feature-value-and-orientation conversion means 7. It is a figure which shows mounting of the temporary feature-value correspondence table change means. 1 is a diagram showing an overview of Example 1. FIG. It is the figure which showed the situation of the CG character 1 and the user 2. FIG. It is an example of the temporary feature amount correspondence table 15. It is the figure which showed the condition at the time of use of Example 3. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 CG character 2 User 3 Video camera 4 Computer 5 Display 6 Position shape estimation means 7 Feature-posture conversion means 8 Surface shape formation means 9 Rendering means 10 Display means 11 Imaging means 12 Tracking point detection means 13 Proximal part detection means 14 Temporary feature quantity correspondence table changing means 15 Temporary feature quantity correspondence table 16 Temporary conversion means 17 Feature quantity extraction means 18 Shape acquisition means 19 Substitution device 20 Display means 21 Imaging means 22 Red marker 23 Green marker 24 Blue marker 25 Red marker tracking point 26 Green marker tracking point 27 Blue marker tracking point 28 Video camera 29 Display 30 Yellow marker tracking point 31 Yellow marker

Claims (4)

Shape acquisition means for acquiring information on the position and shape of the user's body or part thereof, feature quantity extraction means for extracting one or more quantities determined for each shape from the shape information as feature quantities, and feature quantities Feature amount-posture conversion means for converting into position and orientation information representing the angle of the CG character's joint or part, surface shape forming means for forming the surface shape from the position and orientation information, and visualizing the surface shape to obtain an image A CG character animation creating apparatus comprising rendering means and display means for displaying video. 2. The shape acquisition means according to claim 1, comprising a single or a plurality of photographing means arranged around the user, and a position shape estimation means for estimating a position and a shape from an image obtained by the photographing means. Equipment. 2. The apparatus according to claim 1, wherein the feature amount is a position / orientation vector having components of a bending direction, a magnitude, and a position of each joint. The feature amount-posture conversion unit according to claim 1 is a temporary conversion unit that expresses the position and orientation information of the CG character using the temporary feature amount (parameter), and a temporary amount that describes each parameter as a feature amount. 2. A feature quantity correspondence table, and a substitution device for substituting one of the feature quantities into parameters in each position / orientation information of the CG character according to the provisional feature quantity correspondence table, and in addition to each means according to claim 1, An apparatus characterized by comprising provisional feature value correspondence table changing means for changing the contents of the feature value correspondence table.

Images