JP6431265B2 - Object motion generation method, program, and storage medium - Google Patents

Description

  The present invention relates to an object motion generation method, program, and storage medium in a three-dimensional space.

  There is a technology that supports production of animation and the like by controlling movement, rotation, deformation, and the like of a (virtual) object such as a character in a three-dimensional (3D) virtual space or a real space. In this case, a skeleton model (skeleton model) (FIG. 1A) in which a plurality of bones (bones) are connected by joints may be used.

  An object taken as an example in the present specification is constituted by a skeleton model connected by a bone 101, a bone 102, and the like as shown in FIG. Each part constituting the skeleton model is called a node, and each node can have a parent-child relationship. The state of node rotation (bending, twisting) can be expressed by the direction of the child node with respect to the parent node. In order to express the direction of the node, a coordinate system is defined for each node (not shown). A line segment connecting the origins of nodes in a parent-child relationship, or the node itself is a bone.

  In such a skeleton model used for controlling the posture (pose, motion, etc.) of a human body model, an animal, a structure, etc., the position, rotation (bending, bending) of each bone and joint shown in FIG. (Twist) etc. must be controlled as intended. In the skeleton model shown in FIG. 1A, each bone is connected by a joint, and the movement of each bone affects the movement of the other bone. Conventionally, various methods for modeling a skeleton and controlling the skeleton model have been proposed.

  For example, a method for easily controlling the pose or motion of a controlled model by defining a pose or motion in advance for the standard model and applying the pose or motion of the standard skeleton model to the controlled model Has been proposed (see, for example, Patent Document 1).

JP 2013-120395 A

  In this specification, the word “original character” is used as a standard model used to define a pose or motion. In addition, the word “current character” is used as a model to be controlled using the motion.

  FIG. 1A shows a specific skeleton model.

  FIG. 1B shows an example in which an original character 110 is configured by associating a skin composed of a plurality of polygons, for example, with a skeleton model.

  FIG. 2A shows the configuration of the bone of the current character as the control target model, and includes, for example, a bone 201 and a bone 202.

  FIG. 2B shows a current character 210 in which a plurality of polygons representing the body surface are associated with a plurality of bones.

  FIG. 3A shows a virtual plastic bottle 310 with an open stopper.

  FIG. 3B shows a pose in which the original character 320 holds the plastic bottle 330 with the right hand and drinks the beverage from the plastic bottle 330. In this case, the tip 331 of the plastic bottle and the mouth 321 of the original character 320 are in contact. By using a plurality of such poses (frames) and continuously moving the original character 320 and the plastic bottle 330 on the time axis, the motion of the original character 320 drinking the beverage of the plastic bottle 330 in the three-dimensional space. Can be made.

  FIG. 4A is a diagram showing a state where the pose of the original character in FIG. 3B is applied to the current character 420b. The current character 420b holds the plastic bottle 430b with the right hand 422. However, the mouth 421b of the current character 420b and the tip 431a of the plastic bottle are not in contact with each other and are separated.

  FIG. 4B is an enlarged view of a part of FIG. There is a gap between the mouth 421b and the tip 431b of the plastic bottle. The reason why such a deviation occurs is that the original character 320 and the current character 420b have different body shapes and structures. Therefore, it can be seen that even if the pose of the original character 320 is applied to the current character 420b as it is, the intended pose cannot be obtained and the pose may be unnatural. Therefore, even in a motion that is a sequence of a plurality of poses, even if the motion of the original character is simply applied, the motion of the current character may not be as intended.

  The object of the present invention is to eliminate such inconveniences. That is, generating appropriate motion data of the current character (second object) using the motion data of the original character when the original character (first object) and the object (linked object) move in relation to each other. With the goal.

According to this embodiment, a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation, The method corrects the first motion data that defines the motion of the first object so that the first object moves in relation to the linked object, so that the first object is different from the first object . Generating second motion data for the second object, having a first positional relationship between the first object and the linked object, wherein the second object and the linked object Adapting a second positional relationship to the first positional relationship, It is subjected.

  According to the embodiment, the motion data of the character can be easily generated and used.

It is a figure which shows the example of a skeleton model (skeleton model). It is a figure which shows the example of a current character (control object model). It is a figure which shows the pose which the original character is drinking the drink of a plastic bottle. It is the figure which applied the pose of the original character to the current character. It is the figure which showed the corrected pose of the current character. It is a figure which shows the attention point of an original character and a current character. It is the figure which showed the pose which the original character is sitting on the chair. It is the figure which applied the pose of the original character to the current character. It is the flowchart which showed the outline | summary of the process sequence of one Example. It is a detailed flowchart regarding correction | amendment of the pose of a current character. It is a flowchart which shows the determination process of the correction | amendment normal / reverse. It is a figure which shows the process which preserve | saves the direction of a hand and returns the direction of a hand after a movement process. It is the figure which showed the pose which blows a flute. It is the figure which showed the pose which opens the stopper of a plastic bottle. It is the figure which showed the pose of the character with a shovel.

  Note that in this specification, a pose can be regarded as a motion with a zero time length, so both motion data that defines motion and pose data that defines a pose are treated as motion data.

  In this specification, character words are mainly used. Characters are mainly used for virtual persons or creatures. However, the present invention does not exclude virtual objects such as general objects (for example, bicycles, automobiles, balls, etc.) other than virtual persons or living things. Therefore, it should be noted that virtual characters, objects, and the like used in this specification can be collectively referred to as virtual object words.

  In this specification, a skeleton model having bones is used as an example of a three-dimensional object. However, the present invention is not limited to a specific three-dimensional model. In other words, the embodiments described below may be applied to a three-dimensional model having no bone and a three-dimensional model using other than a bone.

  When creating a pose or motion of a character as described above, the position or orientation adjustment work may become complicated if entanglement with the object occurs. For example, when making a pose on a bicycle, it is necessary to align the saddle and hips, pedals and feet, and handles and hands.

  In addition, if a pose or motion created using a character of a certain body type is applied to a character of a different body type, the positional relationship of the parts may be inconsistent. For example, if the pose of “pressing the button of a mobile phone held with the left hand with the index finger of the right hand” is applied to a character having a different shoulder width or arm length, the position of the button may not match the position of the finger.

  The following embodiment can cope with such inconvenience. Examples will be described below.

[Create object with motion]
First, a correct motion is created using a standard body character (for example, the original character 320 in FIG. 3B). Here, “correct” means that the positional relationship between the object (for example, the plastic bottle 330 in FIG. 3B) and the character part (for example, the mouth 321 of the original character 320 in FIG. 3B) is correct ( In the case of this example, it means that the tip 331 of the plastic bottle is in contact with the mouth). In the example of the mobile phone, this means that the button and the index finger are aligned (not shown). The character 320 used here will be referred to as an “original character” (320).

  The character to which the motion data (and correction) of the original character 320 is applied is called a “current character” (420b).

The following information is added to the motion data created for the original character 320.
-Which part of the character should be the point of interest.
-Which part of the character should be the moving part.

  The point of interest and the moving part are used for motion correction described later.

  The point of interest is, for example, a point that exists in the character and refers to a point that should have a correct positional relationship with an object (eg, a plastic bottle). By associating the point of interest with the motion data, an appropriate point of interest can be used when the motion data is used. For the point of interest, for example, a unique name is preferably assigned in order to identify a plurality of points of interest existing in the character such as “left eye” and “mouth”. In addition, it is desirable to establish a uniform nomenclature for points of interest for multiple characters. If the point of interest has a uniform name, when the point of interest associated with the motion data of the original character is applied to another current character, the point of interest of the current character can be easily identified. .

  The moving part is, for example, a part of the current character 420b that is moved when correcting the pose of the current character 420b.

  With respect to the original character, the position of the point of interest (for example, the mouth 321 of the original character 320 in FIG. 3B) in the coordinate system of the object (for example, the plastic bottle 330 in FIG. 3B) is obtained. This is the “correct positional relationship”. When motion data is applied to the current character 420b, the point of interest (for example, the mouth 421b of the current character 420b in FIG. 4A) of the object (for example, the PET bottle 430b in FIG. 4A) in the coordinate system is displayed. Find the position. This is the “current positional relationship”.

    Then, the difference between the “correct positional relationship” and the “current positional relationship” is obtained. The moving part (the right hand 422 in FIG. 4A) is moved by this difference vector so that the positional relationship between the object and the point of interest of the current character is correct.

  FIG. 5 is a diagram illustrating a corrected pose of the current character 520.

  By moving the arm 522 that is the moving part, the positional relationship between the PET bottle 530 of the current character 520 and the mouth 521 of the current character 520 becomes correct, and as a result, the tip 531 of the PET bottle 530 moves to the mouth 521. It comes into contact. As can be seen from the above processing procedure, the object handling in this correction method only uses the position of the point of interest in the object coordinate system, and therefore does not require any information regarding the shape of the object. Each part of the arm 522 moves in a natural manner by IK (inverse kinematics). The present invention is not limited to the above processing procedure.

  FIG. 6A is a diagram illustrating an example of a point of interest of the original character 620. In the original character 620, an eye focus point 621, 622, a nose focus point 624, a mouth focus point 625, and an ear focus point 623 are set in advance.

  FIG. 6B is a diagram illustrating an example of a point of interest of the current character 630. In the current character 630, eye attention points 631, 632, nose attention point 634, mouth attention point 635, and ear attention point 633 are set in advance.

  FIG. 7A shows a pose in which the original character 720 is seated on the chair 750. The original character 720 and the chair 750 are posed so as to have a natural appearance.

  FIG. 7B shows a point of interest 731 set for the original character 721.

  FIG. 8A shows a state where the pose of the original character is applied to the current character 820a. It can be seen that the bottom of the current character 820a is away from the chair 850a and is in an unnatural state.

  FIG. 8B shows an example in which the pose of the current character 820b is corrected so that the positional relationship between the point of interest 821b and the chair 850b is correct. In this case, the moving parts are the waist and chest. When you move your hips and chest, you move from your hips to your left and right wrists, from your hips to your neck, all the way from your hips to your left and right ankles. In this case, it is not necessary to move from the left and right wrists and from the ankles.

  In the example of the mobile phone described above, the tip of the index finger of the right hand is the point of interest, and the right hand is the moving part (not shown). The motion data to which correction information is added in this way is referred to as “motion with correction information”.

  An object with one or more motions with correction information added to one object is called an “object with motion”.

[Use of objects with motion]
For example, the user can use an object with motion in the following procedure.
1) Select the character you want to use and the object with motion.
2) From the motion data that the object with motion has, select the one that you want to apply to the character. Then, the motion is applied to the character with correction (described later) applied to the character.

[Overview of motion compensation processing]
As described above, the motion of the character can be handled as a plurality of poses arranged in time series. Therefore, motion correction can be performed as correction for individual poses.

As already described, a character to which motion data (and correction) is applied is called a “current character”.
1) Apply the pose data to the original character to determine the positional relationship between the object and the point of interest.
2) Apply the pose data to the current character and find the positional relationship between the object and the point of interest.
3) Find the difference between the positional relationship 1) for the original character and the positional relationship 2) for the current character, and use this as the "displacement".
4) The amount “movement amount” that should move the moving part of the current character is obtained from the above displacement amount. By moving the moving part by the moving amount, the displacement amount is set to 0 (or minimum). IK (inverse kinematics) is used to move the moving part. That is, by moving the moving part of the current character by IK, the object or the point of interest moves, and the displacement amount becomes 0 (or minimum). Since IK is a technique well known to those skilled in the art, description thereof is omitted.

  With the above processing, the positional relationship between the object and the point of interest becomes correct (or close to the correct state).

The above method has the following advantages.
1. Creating poses and motions Instead of creating motions (poses) from scratch, you can easily obtain the desired motions (poses) by applying the motions (poses) that objects with motion have, and complicated positioning Is no longer necessary (or a motion (pause) that can ignore positioning is obtained).
2. Applying poses and motions Even if motion (pose) is applied to a character with a body type different from that of the original character, the positional relationship of the parts will not be inconsistent (or inconsistent enough to be ignored). Can be easily).
3. Detail of motion The feature of this correction method is that the position of the target point and the object are not aligned but the positional relationship between them is the same (or very close). Due to this feature, the details of the motion of the original character (subtle changes in position between the object and the point of interest) are reproduced (or easily reproduced) even after the motion is corrected.

[IK (inverse kinematics) used for moving parts]
IK (inverse kinematics) can be used to move parts, but there are “full body IK” and “partial IK” types of IK. In full-body IK, moving one location moves the whole body part. On the other hand, only a limited part moves in the partial IK. For example, in the case of the part IK from the shoulder to the wrist, only the tip from the right shoulder moves when the right hand is moved. Even with full body IK, by setting fixed points appropriately, the same behavior as partial IK can be achieved. For example, if the position of the right shoulder is fixed, only the tip of the right shoulder moves following the movement of the right hand, which is equivalent to the above-described example of the part IK.

  When performing corrections using multiple motion correction information in order, the part that is the point of interest in the previous motion correction information or the part to which the object is attached will not move due to the correction by the subsequent motion correction information. It is necessary to set the tracking range by IK appropriately.

[When attaching an object to a character]
There are cases where an object is attached to a character and cases where an object is not attached. The attached object moves together with the part to which it is attached, even if the character moves or changes its pose. An example of the presence / absence of attachment will be described in further detail in “Example of motion correction” described later.

[Direction of correction direction]
The direction in which the moving part should be moved is the opposite direction depending on the positional relationship with the part where the target point exists or the part to which the object is attached. The direction in which the moving part should be moved when the part where the point of interest exists follows the movement of the moving part is defined as the “positive” direction. On the other hand, when the part to which the object is attached follows the movement of the moving part, the direction in which the moving part should be moved is opposite, and this direction is defined as the “reverse” direction. Examples of forward and reverse movement directions are shown below.
-Positive direction: The above-mentioned “mobile phone” corresponds to this.
・ Reverse direction: “Pet bottle” already mentioned corresponds to this.

[Weighting correction amount]
Multiple moving parts can be specified in one motion correction information, and both the moving part that the point of interest follows and the moving part that the object attachment target part follows can be used (see the example "sword" below) ). The former moving part is moved in the positive direction. The latter moving part is moved in the reverse direction. In order to obtain a correct result (the positional relationship between the object and the point of interest is correct), it is necessary to reduce the amount of movement by multiplying each amount of movement by a coefficient. This coefficient will be referred to as a “weight value”. By setting the sum of the weight values to be 1, a correct correction result can be obtained.

[Movement assist part]
There are cases where the pose cannot be corrected with only the moving part. In an example “bicycle” described later, if the arm of the current character is short, the hand may not reach the handle even if the pose is corrected with the hand as the moving part. In such a case, it is desirable to further correct movement so that the hand can reach the handle by using movement of the chest or the like as a movement assisting part.

[Composition of point of interest]
The position of the point of interest is preferably set in advance for the character. There may be no point of interest at the appropriate location. In this case, the positions of a plurality of existing points of interest can be combined and substituted. This is referred to as a “composite point of interest”. The synthesis method may take the average of the coordinate values of the point of interest.

[Pre-calculation and storage of point of interest coordinates]
In 1) of the correction process “Overview of motion correction process”, a pose is applied to the original character to determine the positional relationship between the object and the point of interest. This can be replaced with the following process.
a) In advance, the position of the point of interest in the object coordinate system when the pose is applied to the original character for all frames of the motion is obtained and stored as data.
b) When executing the correction process, the position of the point of interest in the object coordinate system of the corresponding frame is read from the stored data and used.

  With the above processing, the processing load can be reduced because the processing for applying the pose to the original character becomes unnecessary when the correction processing is executed.

[Hide object]
You can use objects of size 0. Alternatively, no object can be displayed. By using this method for objects, the correction of this method can be applied to motions that do not use objects. For example, it is possible to correct the misalignment of both hands due to the palm pose.

[Change in correction factor over time]
On the timeline (time axis), the correction rate (degree of correction) may be changed for each motion (for each frame).

  On the timeline, when a motion that does not require correction is followed by a motion that requires correction, the correction is suddenly applied and the motion may become discontinuous. Therefore, for motions that require correction, by gradually increasing the value of the correction rate from 0, it is possible to avoid motion discontinuity when moving from a motion without correction to a motion with correction.

[Change in correction factor depending on position]
The correction factor may be changed according to the distance between the object and the point of interest.

  As a general tendency, correction is often unnecessary when the object is far from the point of interest. Therefore, when the distance between the object and the point of interest is greater than or equal to a predetermined value, the correction factor may be lowered as the distance increases. This may make the character motion more natural.

  FIG. 9 is a flowchart showing an outline of the processing procedure of one embodiment.

  In step S910, the motion of the character is applied to the original character and the current character.

  The following processing is performed for each character pose (for each frame).

  In step S920, it is checked whether all current character poses have been processed. If all poses (frames) have been processed, the process ends ("Yes"). If there is a pose (frame) to be processed (“No”), the process proceeds to step S925.

  In step S925, the original character and current character poses are sequentially extracted one by one.

  In step S930, it is checked whether the positional relationship between the target point of the original character and the object matches the positional relationship between the target point of the current character and the object. If the original character's target point is located at a given coordinate in the original character's object coordinate system, the current character's target point is the same in the current character's object coordinate system. It means that it is located at or near the coordinates. If this determination is “yes”, the process returns to step S920. If this determination is “NO”, the process proceeds to step S940.

  In step S940, in order to adapt the positional relationship between the point of interest of the current character and the object to the positional relationship between the point of interest of the original character and the object, the moving part of the current character is moved to correct the current character. Get pause data. When the movement assisting part is designated, the movement assisting part may be moved if necessary to obtain the corrected pose data of the current character. Then, the process returns to step S920.

  In step S950, motion data of the current character is generated from all corrected pose data of the current character.

  Thus, the corrected motion data of the current character is obtained, and the current character and the object can be moved using this motion data.

[Example of motion compensation]
An example of motion correction will be described below. One row in the table corresponds to one piece of “motion correction information”. Each motion correction information is applied in order from the top row. In the following motion correction example, it is assumed that a partial IK as shown in Table 1 is used.

[Example of using a mobile phone as an object]
For example, the following shows an example of pressing a cellular phone button held with the left hand with the index finger of the right hand. Assume that the mobile phone is attached to the left hand.

The motion of the current character is corrected so that the positional relationship between the button and the fingertip is correct.

[Example using a plastic bottle as an object (drinking beverage)]
As described above, an example is shown below in which a bottle held in the right hand is carried into the mouth and a drink is taken. Assume that a plastic bottle is attached to the right hand.

Correct so that the positional relationship between the PET bottle and the mouth is correct.

[Example using a sword as an object]
An example of holding a sword in front, swinging it up, and swinging it down is shown below. Assume that the sword is attached to the left hand.

Correct so that the positional relationship between the sword and the right hand is correct. In order to make the sword as close to the front as possible, it is desirable to make corrections by half with both hands as moving parts.

[Example using trombone as an object]
Assume that the trombone is played. You may bend and stretch the right arm and move the head and upper body. In this case, the object itself may have a motion (stretching and shrinking of the trombone moving part). Assume that the trombone is attached to the left hand.

Correct so that the positional relationship between the trombone and the mouth and the positional relationship between the trombone and the right hand are correct. In this motion correction, the order of correction is important. It is necessary to correct in the order of moving the "reverse" moving part first and then moving the "forward" moving part. If this correction order is reversed, the right hand position is first aligned with the trombone, and then the trombone position is aligned with the mouth. As a result, the trombone and right hand positions are shifted. In order to prevent this from happening, it is necessary to perform the correction sequence appropriately. Note that the correction order of the present invention is not limited to this order.

[Example using a bicycle as an object]
Row a bicycle. The object may have movement (pedal rotation, wheel rotation, etc.). This involves the entire movement of the bicycle.

Correction is performed so that the positional relationship between the bicycle and each point of interest is correct. If the right hand is corrected forward with the right hand as the moving part, but the right hand does not reach the handle, the chest may be moved further (upper body forward). The same applies to the left hand.

[Example using chair (large character) as an object]
The following shows how to create a motion from standing to sitting on a chair. It is assumed that the feet will reach the floor when sitting.

[Example of using a chair (small character) as an object]
A case where motion is generated from a standing state to sitting on a chair is shown below. Assume that your feet float off the floor when you sit down.

  First half: Put the right thigh on the chair.

Second half: Move the whole body.

[Example of using a door as an object]
Shows the action of holding the door knob with the right hand and opening the door to enter the room. The object has motion (door knob rotation, door opening / closing, etc.).

Correct so that the positional relationship between the door (door frame) and the right hand is correct. As a result, the position of the doorknob and the right hand match.

  The terms used in this specification are defined below.

Symbols used in this specification are defined as follows.
BO _poi : The bone of the original character's point of interest.
BC _poi : The bone where the point of interest rides on the current character.
M ' _bp : Matrix (4x4 matrix) representing the position and orientation of bone BO _poi in the world coordinate system.
M _bp : A matrix (4 × 4 matrix) representing the position and orientation of bone BC _poi in the world coordinate system.
U ' _poi : A vector (three-dimensional vector) representing the position of the point of _{interest in} the coordinate system of bone BO _poi .
U _poi : A vector (three-dimensional vector) representing the position of the point of _{interest in} the coordinate system of bone BC _poi .
V ' _poi : A vector (three-dimensional vector) representing the position of the point of _interest of the original character in the world coordinate system.
V _poi : A vector (three-dimensional vector) representing the position of the point of _interest of the current character in the world coordinate system.
BO _obj : The original character's bone to which the object is attached.
BC _obj : The bone to which the current character's object is attached.
M ' _bo : Matrix (4x4 matrix) representing the position and orientation of bone BO _obj in the world coordinate system.
M _bo : Matrix (4 × 4 matrix) representing the position and orientation of bone BC _obj in the world coordinate system.
L ' _obj : Bone BO Matrix representing the position and orientation of the object in the coordinate system of _obj (4x4 matrix).
L _obj : A matrix (4 × 4 matrix) representing the position and orientation of the object in the coordinate system of bone BC _obj .
M ' _obj : A matrix (4x4 matrix) representing the position and orientation of the original character object in the world coordinate system.
M _obj : A matrix (4x4 matrix) that represents the position and orientation of the object of the current character in the world coordinate system.
_{V'poi_obj} : A vector (three-dimensional vector) representing the position of the point of _interest of the original character in the object coordinate system.
K _r : Correction rate. Expresses the degree of correction and has a value between 0 and 1. 0 means no correction and 1 means complete correction.
K _s : Correction normal / reverse coefficient. Represents the correct or reverse of the correction direction and has a value of 1 or -1. -1 for forward direction, 1 for reverse direction.
K _w : Correction weight value. A value that adjusts the ratio of the amount of movement when moving both the object attachment destination part and the point of interest part.
M _hf : Matrix (3 × 3 matrix) representing the orientation of the hand (or foot) node in the world coordinate system.

[Correction process]
FIG. 10 shows a detailed flowchart regarding correction of the pose of the current character.
Apply a pose to the original character. This determines M ' _bp and M' _bo .
U ' _poi has a fixed value for each character, so use that value.
L ' _obj uses the value when motion was created using the original character.

In step S1010, the position and orientation of the original character object are determined:
M ' _{obj is obtained} by the following equation.
M ' _obj = L' _obj M ' _bo (1)
In step S1022, the position of the point of interest of the original character is determined:
_{V'poi is obtained} by the following equation.
V ' _poi = U' _poi M ' _bp (2)
In step S1024, the position of the point of interest of the current character is determined:
Apply a pose to the current character. This determines M _bp and M _bo .
U _poi has a fixed value for each character, so use that value.
L _obj uses the attached state of the object for the current character.
V _poi is _calculated by the following equation.
V _poi = U _poi M _bp (3)
In step S1026, the position / orientation of the object of the current character is obtained:
M _{obj is obtained} by the following equation.
M _obj = L _obj M _bo (4)
In step S1030, a position P _t where the target point should be in the object coordinate system is obtained.
P _t = K _r V ' _poi M' _obj ^-1 + (1-K _r ) V _poi M _obj ^-1 (5)
In step S1040, in the world coordinate system, determine the deviation V _d from the position to a target point.
V _d = V _poi -P _t M _obj (6)
In step S1050, it is determined whether the moving part is a hand or a foot. If “yes”, the process proceeds to step S1052. If “no”, the process proceeds to step S1054.

In step S1052, the orientation of the hand (or foot) node in the world coordinate system is acquired and set as M _hf .

That is, the direction (world coordinate system) of the hand (or foot) before movement is stored as M _hf .

In step S 1054, by the following equation, obtains the movement amount V _m of the mobile site.
V _m = K _s K _w V _d (7)
A method for determining whether the moving direction is normal or reverse required to determine the normal / reverse coefficient K _s will be described later with reference to FIG.

In step S1056, the moving part is moved by V _m .

  In step S1057, it is determined whether the moving part is a hand or a foot. If yes, go to step S1058. If “no”, the process proceeds to step S1060.

In step S1058, the orientation of the hand (or foot) node is returned to the original orientation M _hf .

FIG. 12 shows processing for storing the hand orientation (step S1052) and processing for returning the hand orientation after the movement processing (step S1058). FIG. 12A shows a state before the node 1210 of the hand 1200 is moved. FIG. 12B shows a state where the hand 1201 is moved using IK. It can be seen that the orientation of the hand 1201 is different from the orientation of the hand 1200. FIG. 12C shows a state in which the stored M _hf is read and the direction of the hand 1202 is returned to the same direction as the hand 1200. As described above, when the hand (or foot) is moved by the movement amount 1220 using IK, the direction (world coordinate system) of the moved hand (or foot) is changed. If the point of interest is set on the hand (or foot) and the direction of the hand changes, the amount of movement of the point of interest will be different from the amount of movement of the moving part (hand or foot), and correct correction cannot be performed. Similarly, when an object is attached to the hand (or foot), the amount of movement of the object is different from the amount of movement of the moving part (hand or foot), and correction cannot be performed correctly. Therefore, correct the correction by returning the direction of the hand (or foot) after moving.

In the present embodiment, the above-described processing using M _hf is performed on the hand or the foot, but it may be set so that this processing is performed on other parts. Whether or not to perform this processing can be determined in advance based on the behavior of the IK used for movement.
In step S1060, it is determined whether there is a movement assisting part. If “yes”, the process proceeds to S1070. If no, exit.

  In step S1070, counter = 0 is set to reset the counter.

In step S1082, the position of the point of interest of the current character is determined:
Apply a pose to the current character. This determines M _bp and M _bo .
U _poi has a fixed value for each character, so use that value.
L _obj uses the attached state of the object for the current character.
V _poi is _calculated by the following equation.
V _poi = U _poi M _bp
In step S1084, the position / orientation of the object of the current character is obtained:
M _{obj is obtained} by the following equation.
M _obj = L _obj M _bo
In step S1086, in the world coordinate system, determine the deviation V _d from the position to a target point.
V _d = V _poi -P _t M _obj
In step S1088, it is determined whether V _d is equal to or less than eps or the value of counter is equal to or greater than N. eps is an allowable value for alignment and is a predetermined value. The value of eps may be about 1/2000 of the character's height. N is the maximum number of movements of the movement assisting part and is a predetermined value. The value of N may be about 3.

  If “Yes” in the step S1088, the process ends. If “no”, the process proceeds to step S1090.

In step S1090, the movement amount V _m in the world coordinate system is obtained by the following equation.
V _m = K _s V _d
A method for determining whether the moving direction is normal or reverse required to determine the normal / reverse coefficient K _s will be described later with reference to FIG.

In step S1092, the moving part is moved by V _m .

In step S1094, counter is incremented by 1, and the process returns to step S1082 and step S1084.
(Process when object is not attached)
When the object is not attached (when the object is a chair, etc.), the following changes are added to the “correction process”.

BO _obj and BC _obj which are bones to which the object is attached do not exist. Therefore, there is no matrix M ′ _bo , M _bo , L ′ _obj , or L _obj that represents the position / orientation of these bones.

A predetermined value of M ′ _obj is _assigned for each motion. For example, the position / orientation of the chair is given to the motion of sitting on the chair. Use this value instead of finding M ' _obj in Equation (1).

The value of M _obj uses the position and orientation of the object when applying motion to the current character. Use this value instead of finding M _obj in Equation (4).
(Use of pre-calculation)
V ' _poi M' _obj ^{-1 in} equation (5) does not include information about the current character, so it can be calculated in advance.
V ' _{poi_obj} = V' _poi M ' _obj ^-1 (8)
_{V'poi_obj} means the position of the point of _interest of the original character in the object coordinate system. _{V'poi_obj} is pre-calculated and saved for all frames of motion. Use the saved _{V'poi_obj} when executing motion compensation.

As a result, the following processing and information are not required during correction execution.
・ Pose application for original characters.
-Calculation of equations (1) and (2).
・ Value of U ' _poi , L' _obj .
The above is the details of the pose correction flow.

  FIG. 11 is a flowchart illustrating a process for determining whether the moving direction is correct or not during correction.

  In step S1110, it is determined whether the object is attached to the character. If “no”, the process proceeds to step S1112. If yes, go to step S1114.

  In step S1112, the moving direction is determined to be positive.

  In step S1114, it is determined whether the point of interest (any of the nodes) is within the interlocking range of the moving part. If yes, go to step S1116. If “no”, the process proceeds to step S1118.

  In step S1116, the movement direction is determined to be positive.

  In step S1118, it is determined whether the attachment destination of the object is within the interlocking range of the moving part. If “yes”, the process proceeds to step S1120. If “no”, the process proceeds to step S1122.

  In step S1120, the direction of movement is determined to be opposite.

  In step S1122, it is determined that the combination is invalid. In this case, no correction is necessary.

  The above is the details of the process for determining whether the correction moving direction is normal or reverse.

[Other Examples]
The following embodiment using the processing described above will be described.

[Example using flute as an object]
FIG. 13 is a diagram showing a pose of blowing a flute. FIG. 13A shows a pose in which the original character 1300 is blowing the flute 1309. A flute 1309 is attached to the right hand 1302. The left hand 1303 is attached to the proper position of the flute. The mouth 1304 is in a proper position with respect to the flute outlet 1305.

  FIG. 13B shows a state in which the 1300 pose is directly applied to the current character 1310 without correction. Flute 1319 is attached to right hand 1312. The left hand 1313 is not in an appropriate positional relationship with the flute 1319. Also, the flute outlet (not shown) is far away from the mouth 1314 and is hidden in the current character 1310.

  FIG. 13C shows the pose of the current character 1320 that has been appropriately corrected based on the correction information in the following table.

Since the flute is being played, motion that moves the head and upper body may be added. Correct so that the positional relationship between the flute and the mouth and the positional relationship between the flute and the left hand are correct. In this motion correction, the order of correction is important. First, the “reverse” moving part (right hand 1322) is moved, and then the “forward” moving part (left hand 1323) is moved. If this order is reversed, the position of the left hand is first aligned with the flute, and then the position of the flute is aligned with the mouth. As a result, the positions of the flute and the left hand are shifted.

[Example using a plastic bottle as an object (turning the cap)]
FIG. 14 shows a pose for opening the bottle stopper. In FIG. 14A, a plastic bottle 1409 is attached to the right hand 1402 of the original character 1400. A left hand 1404 is attached to the stopper 1408 of the plastic bottle.

  FIG. 14B shows a case where the current character 1410 is applied without correcting the pose. It can be seen that the positions of the PET bottle stopper 1418 and the left hand 1414 are shifted.

  FIG. 14C shows an example in which the right hand 1422 of the current character 1420 is moved to correct the pose. The position of the stopper 1428 of the plastic bottle has moved and is aligned with the position of the left hand 1424. The correction process in this case is as shown in the following table.

FIG. 14D shows an example in which the left hand 1434 of the current character 1430 is moved to correct the pose. The left hand 1434 has moved and is aligned with the stopper 1438 of the plastic bottle. The correction process in this case is as shown in the following table.

It is also possible to correct the pose by moving both hands (not shown). The contents of the correction process in this case are as shown in the following table.

[Example using a shovel as an object]
FIG. 15 shows an example of correcting the pose of a character having a shovel. FIG. 15A shows a pose in which the original character 1500 holds the shovel 1509 with both hands. A shovel 1509 is attached to the right hand 1504. The left hand 1502 holds the middle position 1508 of the shovel handle.

  FIG. 15B shows a case where a pose is applied to the current character 1510 without correction. It can be seen that the position of the middle position 1518 of the handle of the shovel 1519 and the left hand 1512 are shifted.

  FIG. 15C shows a state in which the pose has been corrected so that the left hand 1522 of the current character 1520 is moved and the positional relationship with the intermediate position 1528 of the handle of the shovel 1529 is correct. The correction process in this case is as shown in the following table.

[Modification]
The method of the embodiment described above can be executed by a computer. Further, the method of the embodiment may be implemented as a program to be executed by a computer.

  The motion data in the above embodiment has a data structure associated with the object. A file having this data structure can be stored in a storage medium. An object is associated with one or more motion data to form a data structure. A file having this data structure can be stored in a storage medium. Data defining an object in a three-dimensional space and motion data may be stored in the same storage medium.

  Also, the point of interest of the original character can be stored in association with the motion data. As the focus point of the current character, a focus point corresponding to the focus point of the original character can be selected.

  In the above-described embodiments, the method steps may be executed simultaneously or in a different order as long as there is no contradiction.

  The above embodiments can be implemented as a hardware device.

  As already described, the original character, the current character, and the object may be expressed by the word of the object.

  Further, the object itself may move, rotate, and deform.

  In addition, the invention described in each claim is not limited to the control of the motion of the object in the virtual space, but can be used to control the motion of the actual object in the real space. When the present invention is applied in real space, a position sensor, a proximity sensor, a pattern recognition technique for a captured image, and the like may be used in combination.

  It is needless to say that the above embodiments do not limit the invention described in the claims but are treated as examples.

  In this specification, the original character, the object, and the current character are examples of the first object, the linked object, and the second object, respectively.

101: Bone 310: Object (plastic bottle)
320: Original character 420b: Current character

Claims (10)

A method in which a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation,
The method is
By correcting the first motion data defining the motion of the first object so that the first object moves in relation to the linked object, the second object having a body shape different from the first object is corrected. Generating second motion data for the first object, wherein the second object data has a first positional relationship between the first object and the linked object, and a second position between the second object and the linked object. Adapting a relationship to the first positional relationship,
Having a method. A method in which a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation,
The method is
Generating second motion data for the second object by correcting the first motion data defining the motion of the first object such that the first object moves relative to the associated object A first positional relationship between the predetermined part of the first object and the linked object, and a second positional relationship between the predetermined part of the second object and the linked object. , Moving a part different from the predetermined part of the second object to adapt to the first positional relationship,
Having a method. A method in which a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation,
The method is
Generating second motion data for the second object by correcting the first motion data defining the motion of the first object such that the first object moves relative to the associated object A first positional relationship between the first object and the linked object, wherein the second positional relationship between the second object and the linked object is changed to the first positional relationship. Adapt, step,
Have
The first positional relationship is a first relative position of a predetermined part of the first object in the coordinate system of the linked object,
The method in which the second positional relationship is a second relative position of a predetermined part of the second object in the coordinate system of the linked object. The step of generating the second motion data includes:
The moving part of the second object corresponding to the predetermined moving part of the first object is minimized so that the amount of displacement between the first relative position and the second relative position is minimized. Moving the position,
The method of claim 3 comprising: A method in which a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation,
The method is
Generating second motion data for the second object by correcting the first motion data defining the motion of the first object such that the first object moves relative to the associated object A first positional relationship between the first object and the linked object, wherein the second positional relationship between the second object and the linked object is changed to the first positional relationship. Have steps to adapt,
A method in which the linked object has a size of zero or is not displayed. A method in which a computer generates motion data of an object in a three-dimensional space, and the motion defined by the motion data includes at least one of movement, rotation, and deformation,
The method is
Generating second motion data for the second object by correcting the first motion data defining the motion of the first object such that the first object moves relative to the associated object Step,
The amount of displacement between the first relative position of the point of interest of the first object viewed from the linked object and the second relative position of the point of interest of the second object viewed from the linked object is minimal. Moving the position of the moving part of the second object corresponding to the predetermined moving part of the first object,
The predetermined moving part of the first object has a plurality of candidates, and is sequentially selected from the plurality of candidates based on a predetermined priority and used. The step of generating the second motion data includes:
Smoothing the motion data of the second object so that the motion of the second object when applying the second motion data is smooth on the time axis;
7. A method according to any one of claims 1-6. The point of interest of the first object is associated with the linked object in advance,
The method according to claim 6, wherein the point of interest of the second object is specified based on the point of interest of the first object associated in advance.   The method according to claim 1, wherein the first motion data is associated with the cooperation object in advance.   A program for causing a computer to execute the method according to any one of claims 1 to 9.