JPH0887609A - Image processor - Google Patents

Abstract

PURPOSE: To provide the image processor with which various operational modifications can be easily executed. CONSTITUTION: This device is provided with a three-dimensional joint position storage part 11 storing the three-dimensional coordinate position of the valid joint of an object corresponding to the operation in one cycle, shape data storage part 15 storing the shape data concerning the parts of the object corresponding to plural peculiar coordinate systems corresponding to the respective valid joints, and peculiar coordinate system deciding part 12 for deciding the peculiar coordinate system from the three-dimensional coordinate position. Besides, such an image processor 10 is equipped with a three-dimensional joint angle calculating part 13 for calculating a three-dimensional joint angle based on the peculiar coordinate system, joint angle displacing part 14 for displacing the three-dimensional joint angle, peculiar coordinate system generating part for generating the peculiar coordinate system based on the three-dimensional joint angle, and operation synthesizing part 16 for performing the allocation of the relevant shape data based on the peculiar coordinate system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus which can easily generate a natural motion of a human having many joints.

[0002]

2. Description of the Related Art In the case of expressing the motion of a human being or the like by computer graphics, image processing based on the key frame method or kinematics is generally performed. The key frame method is a method of expressing a motion by an important frame and its interpolation data, but it takes a very long time to determine a motion by a frame which becomes a key. The method based on kinematics can express motions fairly naturally, but the calculation processing time by the computer becomes enormous.

Therefore, as a method capable of more easily performing image processing, "a method for expressing motion of an articulated object and a computer graphics device" (Japanese Patent Publication No. 4-71078) has been proposed. This is to measure the motion of an articulated object (human) and to represent the motion by using a one-dimensional joint angle and a constant function for the joint angle.

[0004]

However, in such a conventional technique, since the joint angle is expressed in one dimension, only a planar motion can be expressed.
Since we do not consider the directionality of the parts between each joint,
There was a problem that it was not possible to express movements such as twisting.

Further, in the above-mentioned prior art, the processing time and the amount of data are reduced by converting a one-dimensional joint angle into a function using a constant function. However, there is a problem in that it is not possible to perform a motion according to the situation, such as a motion of a person, such as following the ball with a finger.

Therefore, according to the present invention, with respect to an object having a plurality of joints, the parts of the object corresponding to the joint and the adjacent joint can be uniquely applied.
It is an object of the present invention to provide an image processing device that can easily combine images in standard operation by using a unique coordinate system that can consider the orientation of an object in one direction.

It is another object of the present invention to provide an image processing apparatus capable of twisting, rotating, etc. by using a three-dimensional joint angle with respect to a joint.

It is another object of the present invention to provide an image processing apparatus that can easily perform various motion modifications by using a unique coordinate system for a joint and a three-dimensional joint angle for the joint.

Another object of the present invention is to provide an image processing apparatus capable of adding different operations depending on the situation to the basic operations expressed by functions.

[0010]

According to a first aspect of the present invention, when an object having a plurality of joints performs a predetermined motion, each frame for a standard motion in units of one cycle or a plurality of cycles of the motion, Three-dimensional joint position storage means for storing three-dimensional coordinate positions for each of the effective joints, and shape data for storing shape data relating to parts of the object corresponding to a plurality of unique coordinate systems for each of the effective joints. According to a storage unit, a proper coordinate system determination unit that determines a proper coordinate system for each of the effective joints based on the three-dimensional coordinate positions stored in the three-dimensional joint position storage unit, and a desired motion modification mode. A three-dimensional joint angle calculator for calculating three-dimensional joint angles for all or some of the effective joints based on the proper coordinate system determined by the proper coordinate system determining means. When, in accordance with aspects of the action modification,
Based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculating means, a joint angle displacing means for displacing the three-dimensional joint angles for all or some of the effective joints, and according to a mode of the motion modification. Based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculating means and / or the three-dimensional joint angle displaced by the joint angle displacing means, an intrinsic coordinate system for each of all or some of the effective joints is determined. Based on the unique coordinate system generating means to generate, and the unique coordinate system determined by the unique coordinate system determining means and / or the unique coordinate system generated by the unique coordinate system generating means, depending on the mode of the motion modification. An image processing apparatus comprising: a motion synthesis unit that assigns the corresponding shape data to each of the effective joints.

Next, the operation of the present invention will be described.

The three-dimensional joint position storage means, when an object having a plurality of joints performs a predetermined motion, for each frame of a standard motion in which one cycle of the motion or a plurality of cycles is used, each of the effective joints is stored. The three-dimensional coordinate position with respect to is stored. An effective joint is a joint that is used in image processing and serves as a reference for movement, and is all or a part of a plurality of joints of the object (hereinafter the same). The shape data storage means stores shape data regarding the part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints. The unique coordinate system is a three-dimensional coordinate system unique to one effective joint when attention is paid to one effective joint (the same applies hereinafter).
The proper coordinate system determination means determines a proper coordinate system for each of the effective joints based on the three-dimensional coordinate positions stored in the three-dimensional joint position storage means. The three-dimensional joint angle calculating means calculates a three-dimensional joint angle for each of all or some of the effective joints based on the proper coordinate system determined by the proper coordinate system determining means according to a desired mode of motion modification. . The three-dimensional joint angle is a three-dimensional angle between the unique coordinate system and the adjacent unique coordinate system (hereinafter the same). The joint angle displacement means determines a three-dimensional joint angle for each of all or some of the effective joints based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculation means according to the mode of motion modification. Displace. The unique coordinate system generation means calculates the 3 calculated by the three-dimensional joint angle calculation means according to the mode of motion modification.
An intrinsic coordinate system for each of all or some of the effective joints is generated based on the dimensional joint angle and / or the three-dimensional joint angle displaced by the joint angle displacement means. The motion synthesizing means is based on the eigencoordinate system determined by the eigencoordinate system determining means and / or the eigencoordinate system generated by the eigencoordinate system generating means according to the mode of motion modification, and the effective joint. The corresponding shape data is assigned to each of the above.

The eigencoordinate system determining means of the present invention determines a vector from the target joint, which is an effective joint for which the eigencoordinate system is determined, to the first joint adjacent to the effective joint, as a vector of the eigencoordinate system for the target joint. The first axis is a vector from the second joints adjacent to the target joint to the third joint, and the vector is a temporary vector, and the outer product of the temporary vector and the first axis is the proper coordinate system for the target joint. And the outer product of the first axis and the second axis may be the third axis of the proper coordinate system for the target joint.

Further, the proper coordinate system determining means of the present invention determines a vector from the target joint, which is an effective joint for which the proper coordinate system is determined, to the first joint adjacent to the effective joint, in the proper coordinate system for the target joint. A temporary vector is a composite vector of the vector from the target joint to the adjacent second joint and the first axis or its inverse vector as a temporary axis, and an outer product of the temporary vector and the first axis. May be the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis may be the third axis of the proper coordinate system for the target joint.

Further, the proper coordinate system determining means of the present invention determines the vector from the target joint, which is an effective joint for which the proper coordinate system is determined, to the first joint adjacent to the target joint, in the proper coordinate system for the target joint. One of the remaining coordinate axes other than the coordinate axis determined based on the vector from the second joint to the target joint in the proper coordinate system for the second joint adjacent to the target joint as the first axis. The outer product of the first axis may be the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis may be the third axis of the proper coordinate system of the target joint. .

Further, the joint angle displacement means of the present invention comprises:
Discrete cosine transform means for applying a discrete cosine transform to the three-dimensional joint angles for all or some of the effective joints and a transform coefficient for displacing a predetermined element included in each of the DCT coefficients The displacement means and the inverse discrete cosine transform means for applying the inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacement means may be provided. The predetermined element may be the energy of the frequency component and / or the selected order. Further, the inverse discrete cosine transform means is
The inverse discrete cosine transform applied to the CT coefficient or the displaced DCT coefficient may be limited.

Further, the proper coordinate system generating means of the present invention uses the three-dimensional joint angle with respect to the effective joint to rotate each of the three axes of the proper coordinate system with respect to the effective joint, thereby making the effective joint A proper coordinate system for adjacent effective joints may be generated.

According to a second aspect of the present invention, when an object having a plurality of joints performs a predetermined motion, it is unique to each effective joint for each frame for a standard motion in which one cycle or a plurality of cycles of the motion is a unit. Unique coordinate system storage means for storing a coordinate system, shape data storage means for storing shape data relating to a part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints, and desired motion modification According to this aspect, a three-dimensional joint angle calculating means for calculating a three-dimensional joint angle for all or some of the effective joints based on the proper coordinate system stored in the proper coordinate system storage means, and the motion modification. According to the aspect of 3), based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculation means, 3 for each of all or some of the effective joints
A joint angle displacement means for displacing the dimensional joint angle, and a three-dimensional joint angle calculated by the three-dimensional joint angle calculation means and / or a three-dimensional displaced by the joint angle displacement means according to the mode of motion modification. A unique coordinate system generating means for generating a unique coordinate system for each of all or some of the effective joints based on a joint angle, and a unique coordinate system stored in the unique coordinate system storage means according to the mode of motion modification. A motion synthesizing means for allocating the corresponding shape data to each of the effective joints based on the coordinate system and / or the unique coordinate system generated by the unique coordinate system generating means. The image processing device.

Next, the operation of the present invention will be described.

The unique coordinate system storage means, when an object having a plurality of joints performs a predetermined motion, is unique to each of the effective joints for each frame for a standard motion in which one cycle or a plurality of cycles of the motion is taken as a unit. Remembers the coordinate system. The shape data storage means stores shape data regarding the part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints. The three-dimensional joint angle calculation means calculates a three-dimensional joint angle for each of all or some of the effective joints based on the proper coordinate system stored in the proper coordinate system storage means according to a desired mode of motion modification. To do. The joint angle displacement means displaces the three-dimensional joint angle for each of all or some of the effective joints based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculation means according to the mode of motion modification. Let The proper coordinate system generation means is based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculation means and / or the three-dimensional joint angle displaced by the joint angle displacement means according to the mode of motion modification, A unique coordinate system is generated for each of all or some of the effective joints. The behavior synthesizing means is based on the proper coordinate system stored in the proper coordinate system storing means and / or the proper coordinate system generated by the proper coordinate system generating means according to the behavior modification mode. The corresponding shape data is assigned to each joint.

The proper coordinate system stored in the proper coordinate system storage means of the present invention is a vector from the target joint, which is an effective joint for which the proper coordinate system is determined, to the first joint adjacent to the target joint. The first axis of the proper coordinate system with respect to the target joint is set, and the vector from the second joint adjacent to the target joint to the third joint is set as the temporary vector, and the outer vector of the temporary vector and the first axis is set. May be the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis may be the third axis of the proper coordinate system for the target joint.

Further, the proper coordinate system stored in the proper coordinate system storage means of the present invention is such that a vector from the target joint, which is an effective joint for which the proper coordinate system is determined, to the first joint adjacent thereto is The first axis of the proper coordinate system with respect to the target joint is used as a temporary vector, and a composite vector of the vector from the target joint to the second joint adjacent thereto and the first axis or its inverse vector is used as the temporary vector. And an outer product of the first axis and a second axis of the proper coordinate system for the target joint, and an outer product of the first axis and the second axis as a third axis of the proper coordinate system for the target joint. Good.

Further, the proper coordinate system stored in the proper coordinate system storage means of the present invention is such that a vector from a target joint which becomes a joint for determining the proper coordinate system to a first joint adjacent to the target joint Of the proper coordinate system for the second joint adjacent to the target joint, which is the first axis of the proper coordinate system for the joint, the remaining coordinate axes other than the coordinate axis determined based on the vector from the second joint to the target joint The outer product of either one of the coordinate axes and the first axis is the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis is the third coordinate of the proper coordinate system for the target joint.
It may be an axis.

Further, the joint angle displacement means of the present invention comprises:
Discrete cosine transform means for applying a discrete cosine transform to the three-dimensional joint angles for all or some of the effective joints and a transform coefficient for displacing a predetermined element included in each of the DCT coefficients The displacement means and the inverse discrete cosine transform means for applying the inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacement means may be provided. The predetermined element may be the energy of the frequency component and / or the selected order. Further, the inverse discrete cosine transform means is
The inverse discrete cosine transform applied to the CT coefficient or the displaced DCT coefficient may be limited.

Further, the proper coordinate system generating means of the present invention uses the three-dimensional joint angle with respect to the effective joint to rotate each of the three axes of the proper coordinate system with respect to the effective joint, so that the effective joint is generated. A proper coordinate system for adjacent effective joints may be generated.

According to a third aspect of the present invention, when an object having a plurality of joints performs a predetermined motion, 3 cycles for each of the effective joints are set for each frame for a standard motion in a unit of one cycle or a plurality of cycles of the motion. A three-dimensional joint angle storage means for storing a three-dimensional joint angle; a shape data storage means for storing shape data on a part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints; A unique coordinate system determining unit that determines a unique coordinate system for each of the effective joints based on the unique coordinate system for the joint and the three-dimensional joint angle stored in the three-dimensional joint angle storage unit, and a desired motion modification mode. According to the three-dimensional joint angle storage means, the three-dimensional joint angles for all or some of the effective joints are displaced based on the three-dimensional joint angles stored in the three-dimensional joint angle storage means. Joint angle displacement means, and based on the three-dimensional joint angle stored in the three-dimensional joint angle storage means and / or the three-dimensional joint angle displaced by the joint angle displacement means, according to the mode of motion modification, A unique coordinate system generating means for generating a unique coordinate system for each of all or some of the effective joints, and the unique coordinate system determined by the unique coordinate system determining means according to the mode of motion modification and / or An image processing apparatus comprising: a motion synthesizing unit that assigns the corresponding shape data to each of the effective joints based on the unique coordinate system generated by the unique coordinate system generating unit.

Next, the operation of the present invention will be described.

The three-dimensional joint angle storage means, when an object having a plurality of joints performs a predetermined motion, for each frame of the standard motion in units of one cycle or a plurality of cycles of the motion, each effective joint is stored. The three-dimensional joint angle with respect to is stored. The shape data storage means stores shape data regarding the part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints. The proper coordinate system determining means determines, based on the proper coordinate system for the root joint and the three-dimensional joint angle stored in the three-dimensional joint angle storage means,
A unique coordinate system is determined for each of the effective joints. The joint angle displacement means, depending on the desired mode of motion modification,
Based on the three-dimensional joint angles stored in the three-dimensional joint angle storage means, the three-dimensional joint angles for all or some of the effective joints are displaced. The unique coordinate system generation means is based on the three-dimensional joint angle stored in the three-dimensional joint angle storage means and / or the three-dimensional joint angle displaced by the joint angle displacement means according to the mode of motion modification. To generate a unique coordinate system for each of all or some of the effective joints. The motion synthesizing means is based on the eigencoordinate system determined by the eigencoordinate system determining means and / or the eigencoordinate system generated by the eigencoordinate system generating means according to the mode of motion modification, and The corresponding shape data is assigned to each.

In the intrinsic coordinate system for the root joint of the present invention, a vector from the root joint to a first joint adjacent to the root joint is defined as a first axis of the intrinsic coordinate system for the root joint. A vector from the second joint adjacent to the root joint to the third joint is defined as a temporary vector, and an outer product of the temporary vector and the first axis is a second axis of an intrinsic coordinate system for the root joint. The outer product of the first axis and the second axis may be the third axis of the unique coordinate system with respect to the root joint.

Further, in the proper coordinate system for the root joint of the present invention, a vector extending from the root joint to a first joint adjacent thereto is defined as a first axis of the proper coordinate system for the root joint, and A composite vector of the vector from the joint of the root toward the second joint adjacent thereto and the first axis or its inverse vector is taken as a temporary vector, and the outer product of the temporary vector and the first axis is taken as the joint of the root. May be the second axis of the proper coordinate system, and the outer product of the first axis and the second axis may be the third axis of the proper coordinate system for the root joint.

Further, in the proper coordinate system for the root joint of the present invention, a vector from the joint of the root to a first joint adjacent thereto is defined as a first axis of the proper coordinate system for the joint of the root. Of the proper coordinate system for the second joint adjacent to the root joint, one of the remaining coordinate axes other than the coordinate axis determined based on the vector from the second joint to the root joint and the first axis. The outer product of and may be the second axis of the proper coordinate system for the root joint, and the outer product of the first axis and the second axis may be the third axis of the proper coordinate system for the root joint.

Further, the joint angle displacement means of the present invention comprises:
Discrete cosine transform means for applying a discrete cosine transform to the three-dimensional joint angles for all or some of the effective joints and a transform coefficient for displacing a predetermined element included in each of the DCT coefficients The displacement means and the inverse discrete cosine transform means for applying the inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacement means may be provided. The predetermined element may be the energy of the frequency component and / or the selected order. Further, the inverse discrete cosine transform means may limit the inverse discrete cosine transform applied to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacement means.

Further, the proper coordinate system generating means of the present invention uses the three-dimensional joint angle with respect to the effective joint to rotate each of the three axes of the proper coordinate system with respect to the effective joint, thereby making the effective joint A proper coordinate system for adjacent effective joints may be generated.

According to a fourth aspect of the present invention, in the case where a plurality of objects having an effective joint serving as a motion reference perform a predetermined motion, for each periodic or aperiodic motion, each frame,
Desired from the basic motion storage means that stores data relating to the three-dimensional joint angles for all or some of the effective joints as basic motion data, and a plurality of basic motion data stored in the basic motion storage means. Basic operation selecting means for selecting basic operation data, and operation modification parameters required when changing an operation configured by the basic operation data selected by the basic operation selecting means to an operation accompanied by a desired operation modification. Using the parameter setting means for setting the motion modification parameter and the motion modification parameter, the motion configured by the basic motion data selected by the basic motion selection means is changed to a motion accompanied by the desired motion modification, Basic motion modification generation means for generating motion modification data, and a motion different from the motion configured by the basic motion modification data. And additional motion selecting means for selecting a desired additional motion to be added to the motion constituted by the basic motion modification data, and additional motion generation means for generating the basic motion modification data and the additional motion data by the additional motion. , A behavior synthesizing unit for synthesizing the basic behavior modification data and the additional activity data to generate motion data related to an activity in which the activity configured by the basic activity modification data is added with the activity configured by the additional motion data An image processing apparatus comprising:

Next, the operation of the present invention will be described.

When an object having a plurality of effective joints performs a predetermined motion, the basic motion storage means stores all or a part of the effective joint for each frame with respect to the cyclical or aperiodic motion. Data relating to the three-dimensional joint angle for each is stored as basic motion data. The basic motion selection means selects desired basic motion data from the plurality of basic motion data stored in the basic motion storage means. The parameter setting means sets the operation composed of the basic operation data selected by the basic operation selecting means,
A motion modification parameter required when changing to a motion accompanied by a desired motion modification is set. The basic motion modification generation means uses the motion modification parameter to change the motion composed of the basic motion data selected by the basic motion selection means into a motion accompanied by the desired motion modification, and Generate qualified data. The additional action selection means selects a desired additional action that is different from the action configured by the basic action modification data and that is added to the action configured by the basic motion modification data. The additional action generation means generates additional action data by the basic action modification data and the additional action. The behavioral synthesizing unit synthesizes the basic behavioral modification data and the additional behavioral data to generate behavioral data related to an operation in which the behavioral behavior of the basic behavioral modification data is added to the behavior of the additional behavioral data. To do.

The basic motion modification generation means of the present invention uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means. When the basic motion data changed by is a cyclic motion, it is assumed that the basic motion data is provided with a cyclic motion generating means for generating the basic motion modification data by repeating the modified basic motion data for a desired period. Good.

Further, the basic motion modification generation means of the present invention uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means. When the basic motion data changed by is an aperiodic motion, the non-cyclic motion generating means may generate the basic motion modification data from the changed basic motion data without repeating. .

The basic motion modification generation means of the present invention uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means. When the basic motion data changed by is a cyclic motion, the basic motion modification means for generating the basic motion modification data by repeating the modified basic motion data for a desired period and the basic motion modification In the case where the basic motion data changed by the means is an aperiodic motion, a non-periodic motion generation means for generating the basic motion modification data from the changed basic motion data without repeating is provided. Good.

Further, the additional motion generating means of the present invention is
The coordinate input means for inputting coordinates to be moved corresponding to the additional motion selected by the additional motion selecting means, and the joint corresponding to the tip end of the effective joint selected by the additional motion selecting means as the target of the additional motion are the coordinates. And a coordinate tracking means for generating the additional operation data.

Further, the additional motion generating means of the present invention is
Reference addition operation storage means for storing, as reference addition operation data, data regarding three-dimensional joint angles for all or some of the effective joints for each predetermined addition operation, and the reference addition operation. The additional operation reading means for reading the reference additional operation data corresponding to the additional operation selected by the additional operation selecting means from the plurality of reference additional operation data stored in the storage means, and the additional operation selecting means for selecting Using the additional motion modification setting means for setting a motion modification parameter according to the additional motion and the motion modification parameter set by the additional motion modification setting means, the reference additional motion data read by the additional motion reading means is used. It may be modified to include additional action modification generation means for generating the additional action data.

[0042]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of an embodiment of an image processing apparatus according to claim 1 of the present invention. That is, the three-dimensional joint position storage unit 11 of the image processing apparatus 10 stores the three-dimensional coordinate position for each effective joint for each frame of one cycle of movement when a person performs a predetermined movement, for example. It is a storage unit. The predetermined motion is a motion that can be expressed by repeating one cycle of motion, and is, for example, various motions such as continuously performing vertical jumps, walking, running, and swimming by crawling. .
The predetermined motion is not limited to this, and includes, for example, a motion completed in one cycle such as a dive motion at the start of a crawl swimming race. The effective joints are a plurality of joints required when a predetermined motion is combined, and do not mean all joints that a human has. In the future, the above 3
The predetermined motion stored in the dimensional joint position storage means 11 is referred to as a standard motion. Further, this storage unit may be stored in either the internal storage device of the image processing apparatus 10 or the external storage device.

The input device 17 is an input device for inputting an instruction regarding an operation desired by the operator. Further, the control unit 18 is a control unit that performs control related to image processing according to an instruction input to the input device 17.

Here, the intrinsic coordinate system and the apparatus related thereto will be described. The proper coordinate system is a three-dimensional coordinate system unique to the effective joint with the effective joint as the origin. The characteristic of the proper coordinate system is that, if the proper coordinate system for the effective joint is obtained, the part of the corresponding object can be uniquely fitted between the effective joint and the adjacent effective joint. In the present embodiment, the proper coordinate system for the effective joint is set so that the corresponding human part is fitted between the effective joint and its child joint. The direction when fitting is the positive z-axis direction from the origin of the proper coordinate system for the effective joint. Another characteristic of the proper coordinate system is that if the proper coordinate system for the effective joint is determined, the end point of the part corresponding to the effective joint that is not the origin becomes the origin of the proper coordinate system for the child joint. is there. Details of determining the unique coordinate system will be described later. Therefore, the shape data storage unit 15 is a storage unit that stores shape data regarding a human body part in consideration of a plurality of unique coordinate systems for each effective joint. The unique coordinate system determination unit 12 determines the unique coordinate system for each effective joint.

Here, the three-dimensional joint angle and the apparatus relating thereto will be described. The three-dimensional joint angle with respect to the effective joint means the three-dimensional angle between the proper coordinate system for the effective joint and the proper coordinate system for the adjacent effective joint. In the present embodiment, the three-dimensional joint angle with respect to the effective joint means the three-dimensional angle between the proper coordinate system for the valid joint and the proper coordinate system for the parent joint. The characteristic of the three-dimensional joint angle is that motions such as twisting and rotating can be easily expressed. In addition, it is possible to easily add motion modification to the standard motion by displacing the 3D joint angle.
Is one. Motion modification is, for example, making a normal walking motion stride, straddle, fast walk, or slowly walk, and is related to the standard motion. A characterization that adds another action. The method for obtaining the three-dimensional joint angle will be described later. Therefore, the three-dimensional joint angle calculation unit 13 determines the proper coordinate system for the effective joint based on the proper coordinate system determined by the proper coordinate system determination unit 12 in accordance with the desired mode of motion modification based on the operator's instruction. And a three-dimensional joint angle for each effective joint is calculated using the proper coordinate system for the child joint. The joint angle displacement unit 14
The three-dimensional joint angle calculation unit 1 according to the mode of motion modification.
Based on the three-dimensional joint angle calculated in 3, the three-dimensional joint angle for each effective joint is displaced. The behavioral synthesis unit 16 includes a unique coordinate system generation unit (not shown). The proper coordinate system generation unit generates a proper coordinate system for each effective joint based on the three-dimensional joint angle displaced by the joint angle displacement unit 14 according to the mode of motion modification. Behavioral synthesis unit 16
Based on the unique coordinate system generated by the unique coordinate system generation unit and / or the unique coordinate system determined by the unique coordinate system determination unit 12, the shape data storage unit 15 stores shape data corresponding to each effective joint. The shape data thus read out is assigned to each effective joint.

Here, the three-dimensional joint angle calculation means 13 has three
A calculation target range that is a range of an effective joint for calculating a three-dimensional joint angle, a displacement target range that is a range of an effective joint for which the joint angle displacement unit 14 displaces a three-dimensional joint angle, and the unique coordinate system generation unit. The generation target range, which is the range of the effective joint that generates the system, is determined by the mode of the motion modification. For example, when a person modifies a standard motion of rotating 360 degrees, it is conceivable that the calculation target range, the displacement target range, and the generation target range are all effective joints. On the other hand, for example, in the case of modifying the standard action when moving only the right hand back and forth, the target effective joint is from the effective joint near the right shoulder to the fingertip of the right hand or the effective joint of the hand. There are cases. First, in the case of up to the fingertip, the calculation target range, the displacement target range, and the generation target range have the same range of effective joints as the respective targets, and are from the effective joint near the right shoulder to the fingertip. However, their coverage is not all of the effective joints. Further, in the case of the effective joint of the hand, the calculation target range and the generation target range are equal to each other, and the effective joint near the right shoulder to the fingertip. However, the displacement target range is from the effective joint near the right shoulder to the effective joint near the joint of the hand. The calculation target range, the displacement target range, and the generation target range are not all effective joints.

After all, the calculation target range, the displacement target range, and the generation target range are determined by the mode of motion modification. These ranges may be all of the effective joints at all times, or may be divided according to the mode of motion. In short, the behavioral synthesis unit 16 determines the effective joint based on the proper coordinate system generated by the proper coordinate system generation unit and / or the proper coordinate system determined by the proper coordinate system determination unit 12 according to the behavior modification mode. The shape data corresponding to each of the above may be assigned.

Claims 1, 9 and 1 of the present invention
“All or part of the effective joint” in 7 is the expression related to the above.

Next, the operation of this embodiment will be described. FIG. 2 is a diagram schematically showing the positions of effective joints when a person is running. The relationship between each parent and child of the effective joint is
Determined when the root joint is set. In the case of FIG. 2, the effective joint 21 of the waist is set as the root joint. The operation of this embodiment has a standard operation mode and an operation modification mode, which will be described below in order. (1) Standard operation mode The standard operation mode is a mode in which a predetermined operation is combined without modifying the operation. That is, the unique coordinate system determination unit 1
2 determines the proper coordinate system for each of the effective joints based on the three-dimensional coordinate positions stored in the three-dimensional joint position storage unit 11 in the order from the root joint to the parent and child. Here, the calculation target range, the displacement target range, and the generation target range that are determined by the desired mode of motion modification are all effective joints because of the standard motion mode. The three-dimensional joint angle calculation unit 13 uses the unique coordinate system determined by the unique coordinate system determination unit 12 according to the order from the root joint to the parent-child.
A three-dimensional joint angle for each effective joint is calculated. The joint angle displacement unit 14 does not displace the three-dimensional joint angle calculated by the three-dimensional joint angle calculation unit 13 in the standard operation mode. The proper coordinate system generation unit (not shown) included in the motion synthesis unit 16 uses each of the effective joints by using the proper coordinate system for the root joint and the three-dimensional joint angle input from the joint angle displacement unit 14. Generate an intrinsic coordinate system for. The behavioral synthesis unit 16 uses the unique coordinate system generated by the unique coordinate system generation unit, and in accordance with the order from the root joint to the parent and child, the corresponding shape stored in the shape data storage unit 15 for each effective joint. Allocate data.

By repeating the above processing in the order of frames for a desired period, a predetermined operation can be repeated in the standard operation mode.

In the standard operation mode, the processes in the three-dimensional joint angle calculation unit 13, the joint angle displacement unit 14, and the unique coordinate system generation unit are omitted, and the motion synthesis unit 16 is operated by the unique coordinate system determination unit 12 from the unique coordinate system determination unit 12. A process of inputting the unique coordinate system for each effective joint and applying the shape data stored in the shape data storage unit 15 to each effective joint may be performed. (2) Motion modification mode The motion modification mode is a mode in which a standard motion is motion modified and synthesized. That is, the unique coordinate system determination unit 12
Based on the three-dimensional coordinate position stored in the three-dimensional joint position storage unit 11, the proper coordinate system for each effective joint is determined in the order from the root joint to the parent and child. Here, the calculation target range determined by the desired behavior modification mode,
The displacement target range and the generation target range are all effective joints for simplicity. The three-dimensional joint angle calculation unit 13 calculates the three-dimensional joint angle for each effective joint in the order from the root joint to the parent and child using the proper coordinate system determined by the proper coordinate system determination unit 12. The joint angle displacement unit 14
A three-dimensional joint angle for each of the effective joints input from the three-dimensional joint angle calculation unit 13 is displaced using a predetermined function that is displaced according to the mode of motion modification. The proper coordinate system generation unit (not shown) included in the motion synthesis unit 16 uses the proper coordinate system for the root joint and the three-dimensional joint angle input from the joint angle displacement unit 14 to calculate the effective joint. Generate a unique coordinate system for each. Behavioral synthesis unit 16
Assigns the corresponding shape data stored in the shape data storage unit 15 to each of the effective joints in the order from the root joint to the parent and child using the proper coordinate system generated by the proper coordinate system generation unit. I do.

By repeating the above processing in the order of frames for a desired period, it is possible to repeat a predetermined operation in the operation modification mode.

Next, the method of obtaining the unique coordinate system by the unique coordinate system determination unit 12 will be described more specifically. The method for obtaining the three types of unique coordinate systems used in this embodiment will be described below. (1) 4 Joints Unique Coordinate System Determination Method FIG. 3 is a diagram showing how the 4 joints unique coordinate system is determined.
For example, it is used to determine the proper coordinate system of an effective joint such as a person's waist. In this case, since the hip joint, which is the target joint for which the proper coordinate system is to be determined, is the root joint, other effective joints are, as shown in FIG. And it becomes the third child joint. The procedure for determining this unique coordinate system is as follows.

1. The vector in the direction of the first child joint from the target joint is determined as the z axis.

2. The vector in the direction from the second child joint to the third child joint is copied to the target joint and is used as a temporary y'axis.

3. The value of the outer product of the z axis and the y ′ axis is determined as the x axis.

4. The value of the outer product of the z axis and the x axis is determined as the y axis. (2) Three-joint peculiar coordinate system determination method FIG. 4 is a diagram showing how the three-joint peculiar coordinate system is determined.
Here, the effective joint for which the proper coordinate system is to be determined is the target joint, the effective joint corresponding to the parent of the target joint is the parent joint (the proper coordinate system may not be determined), and the target joint The effective joint that hits the child is called the child joint.

FIG. 4A is a diagram showing how to determine the proper coordinate system of an effective joint of a human elbow or the like. The procedure for determining this unique coordinate system is as follows.

1. The vector in the direction from the target joint to the child joint is determined as the z axis.

2. A composite vector of the vector in the direction from the target joint to the parent joint and the z axis is taken as a temporary x'axis.

3. The value of the outer product of the z axis and the x'axis is determined as the y axis.

4. The value of the outer product of the z axis and the y axis is determined as the x axis.

FIG. 4B is a diagram showing how to determine the proper coordinate system of an effective joint such as a human knee. The procedure for determining this unique coordinate system is as follows.

1. The vector in the direction from the target joint to the child joint is determined as the z axis.

2. The inverse vector of the combined vector of the vector in the direction from the target joint to the parent joint and the z axis is taken as the temporary x ′ axis.

3. The value of the outer product of the z axis and the x'axis is determined as the y axis.

4. The value of the outer product of the z axis and the y axis is determined as the x axis.

From the above, the three-joint unique coordinate system is, for example,
It is suitable for obtaining the proper coordinate system of an effective joint such as a person's elbow or knee, and has a feature that the frontal direction of the person in the front-back direction can always be set to the positive x-axis. (3) Two-joint peculiar coordinate system determination method FIG. 5 is a diagram showing how the two-joint peculiar coordinate system is determined.
(A) is a figure which shows a mode that the x-axis is copied from the parent's proper coordinate system and the proper coordinate system of the target joint is determined, and (b) is a target joint by copying the y-axis from the parent's proper coordinate system. FIG. 6 is a diagram showing a state in which the proper coordinate system of is determined. In this case, the effective joint for which the proper coordinate system is to be determined is the target joint, the effective joint corresponding to the parent of the target joint is the parent joint (the proper coordinate system has already been determined), and the child of the target joint. The effective joint corresponding to is the child joint. The procedure for determining this unique coordinate system is as follows.

1. The vector in the direction from the target joint to the child joint is determined as the z ₁ axis.

2. The x ₀ axis (or y ₀ axis) of the parent joint is copied to the target joint and used as a temporary x ₀ 'axis (or y ₀ ' axis).

3. the value of the cross product of the z ₁ axis and x ₀ 'axis (or y _0' axis) is determined as y ₁ axis (or x ₁ axis).

4. The value of the outer product of the z ₁ axis and the y ₁ axis (or the x ₁ axis) is determined as the x ₁ axis (or the y ₁ axis).

FIG. 6 is a flowchart relating to the determination of these three types of unique coordinate systems.

In this way, the unique coordinate system for each effective joint can be determined. With this unique coordinate system, it is possible to set a coordinate system for each part such as the elbow and knee in consideration of the front-back direction of the person.

Next, regarding the method of obtaining the three-dimensional joint angle,
This will be described more specifically. FIG. 7 is a diagram showing how the three-dimensional joint angle is calculated. FIG. 7A shows a three-dimensional angle relationship between the proper coordinate system (x, y, z) for an arbitrary joint and the proper coordinate system (x ′, y ′, z ′) for a joint adjacent to the joint. It is the figure shown. An example of the calculation procedure of the three-dimensional joint angle with respect to an arbitrary joint will be described below with reference to FIG. 7. However, in the present embodiment, any joint is a parent joint, and the adjacent joint is its child joint.

1. As shown in FIG. 7A, the origins of the unique coordinate systems of the parent joint and the child joint are made to coincide with each other.

2. As shown in FIG. 7B, the angle (−θ _x ) between the coordinate axis and the z axis when the z ′ axis of the child joint is projected onto the yz plane of the proper coordinate system of the parent joint is obtained.

3. As shown in FIG. 7B, an angle (θ _y ) between the coordinate axis and the z axis when the z ′ axis of the child joint is projected on the zx plane of the unique coordinate system of the parent joint is obtained.

4. As shown in FIG. 7D, the x-axis of the unique coordinate system of the parent joint is rotated by an angle of −θ _x . When the proper coordinate system (x, y, z) is rotated by (θ _x , 0, 0), the coordinate system (x, y ″, z ″) is obtained.

5. As shown in FIG. 7E, the y axis of the unique coordinate system of the parent joint is further rotated by an angle of θ _y . When the coordinate system (x, y ″, z ″) is rotated by (θ _x , θ _y , 0), the coordinate system (x ″, y ″, z ″) is obtained.

6. (Θ _x , θ _y , 0) Coordinate axis z ″ after rotation
7 coincides with the z'axis of the coordinate system for the child joint, as shown in FIG. Therefore, rotate the coordinate axis z ",
Let θ _z be the angle at which the coordinate system (x ″, y ″, z ″) coincides with the unique coordinate system for the child joint.

7. (Θ _x , θ _y , θ _z ) is a three-dimensional joint angle with respect to an arbitrary joint, that is, an arbitrary parent joint.

FIG. 8 is a diagram showing that when the proper coordinate system for the parent joint shown in FIG. 7 is rotated in the zxy order, the proper coordinate system for the child joint is obtained. That is, by using the three-dimensional joint angle with respect to the joint and the proper coordinate system with respect to the joint, the proper coordinate system with respect to the child joint can be generated. In the present embodiment, the order of axis rotation is zxy, but the order is not necessarily limited to this.
The order may be xyz. In short, it is sufficient to rotate each of the three axes once.

Next, the predetermined function by which the joint angle displacement section 14 displaces the three-dimensional joint angle with respect to each arbitrary joint in accordance with the desired mode of motion modification will be described in more detail. The five types of displacement modes used in this embodiment are shown below. However, the j-axis is the x-axis of the intrinsic coordinate system, y
Either the axis or the z-axis. (1) Energy Displacement Mode FIG. 9 is a configuration diagram of the joint angle displacement unit 14 that displaces the three-dimensional joint angle with respect to each joint in the energy displacement mode. The discrete cosine transform unit 91 treats the three-dimensional joint angle for each joint as time series data according to (Equation 1), applies the discrete cosine transform to the time series data,
Then, each DCT coefficient is output.

[0086]

[Equation 1]

N: total number of data (total number of frames) k: order of DCT coefficient m: data number (frame number) θ _m (i, j): for the m-th frame, in the proper coordinate system for the i-th joint Three-dimensional joint angle with respect to j-axis z _k (i, j): k-th order DCT coefficient with respect to θ _m (i, j) The conversion coefficient displacement unit 92 includes an energy displacement unit 93, and uses a parameter a Discrete cosine transform unit 91
The energy of each DCT coefficient output by is displaced. The inverse discrete cosine transform unit 94 applies the inverse discrete cosine transform to each DCT coefficient whose energy is displaced by (Equation 2), and outputs a three-dimensional joint angle for each displaced joint.

[0088]

[Equation 2]

N: total number of data (total number of frames) k: order of DCT coefficient m: data number (frame number) a: motion modification setting parameter θ _m (i, j) in energy displacement mode: for m-th frame , Three-dimensional joint angle after displacement with respect to the j-axis of the i-th joint for the i-th joint According to this energy displacement mode, when the parameter a is increased and the inverse discrete cosine transform unit 94 performs the inverse transform, large motion data is obtained. . On the contrary, when the parameter a is reduced and the inverse discrete cosine transform unit 94 performs the inverse transform, the data of the small motion is obtained. Therefore, with this mode, for example, a motion modification mode in which a person walks with a large stride or with a small stride can be realized with one parameter with respect to a walking motion in the standard operation mode. (2) Order displacement mode FIG.
It is a block diagram of the joint angle displacement part 14 which performs displacement of a dimensional joint angle. The discrete cosine transform unit 91 similar to FIG. 9 treats the three-dimensional joint angles for each joint as time series data,
The discrete cosine transform is applied to the time series data, and each DCT coefficient is output. Conversion coefficient displacement unit 101
Includes an order selection unit 102 and uses the order M to displace each DCT coefficient output from the discrete cosine transform unit 91. The inverse discrete cosine transform unit 103 uses (Equation 3)
The inverse discrete cosine transform is applied to each displaced DCT coefficient, and the three-dimensional joint angle for each displaced joint is output.

[0090]

(Equation 3)

N: total number of data (number of frames) k: order of DCT coefficient M: operation modification setting parameter in order displacement mode m: data number (frame number) θ _m (i, j): for m-th frame, For M such that the three-dimensional joint angle N> M after displacement with respect to the j-axis of the unique coordinate system for the i-th joint, the smaller this value is, the less data expresses fine movements. On the other hand, when the size becomes large, the amount of data that expresses a detailed motion increases. Therefore,
With this mode, it is possible to realize a motion modification mode in which a delicate motion can be set with one parameter. (3) Motion Velocity Displacement Mode FIG. 11 is a configuration diagram of the joint angle displacement unit 14 that performs displacement of the three-dimensional joint angle with respect to each joint in the motion velocity displacement mode. Here, in order to describe the motion modification only by the motion velocity displacement mode, the transform coefficient displacement unit 111 outputs the input DCT coefficient without displacement. The discrete cosine transform unit 91 similar to FIG. 9 handles the three-dimensional joint angle for each joint as time series data, applies the discrete cosine transform to the time series data, and outputs each DCT coefficient. The transform coefficient displacement unit 111 outputs each input DCT coefficient without displacement. Inverse discrete cosine transform unit 11
2 includes a transform order limiting unit 113, and according to (Equation 4), an inverse discrete cosine transform is applied to each input DCT coefficient by using a parameter n and using only a required order among the inverse discrete cosine transforms. It is applied to output a three-dimensional joint angle for each joint whose order is limited.

[0092]

[Equation 4]

N: Total number of data (total number of frames) k: Order of DCT coefficient n: Speed setting parameter in operating speed displacement mode m: 0, 1, 2, ..., (N-1) / n θ _mn (i , j): Three-dimensional joint angle with respect to the j-axis of the unique coordinate system for the ith joint for the mnth frame. According to this movement speed displacement mode, a predetermined movement is executed at a movement speed n times that of the parameter n. Can be made. Therefore, with this mode, it is possible to realize a motion modification mode in which a fast motion or a slow motion can be set with respect to the standard motion mode with one parameter. (4) Energy / Order Compound Displacement Mode FIG. 12 is a configuration diagram of the joint angle displacement unit 14 that performs displacement of the three-dimensional joint angle with respect to each joint in the energy / order compound displacement mode. The discrete cosine transform unit 91 similar to FIG. 9 treats the three-dimensional joint angle for each joint as time series data, applies the discrete cosine transform to the time series data, and outputs each DCT coefficient. The conversion coefficient displacement unit 121 includes the energy displacement unit 93 of FIG.
The energy displacing unit 93 uses the parameter a to displace the energy of each DCT coefficient output from the discrete cosine transform unit 91, and includes the same circuit as the zero order selecting unit 102. Uses the order M to displace each DCT coefficient output by the discrete cosine transform unit 91. The inverse discrete cosine transform unit 122 uses (Equation 5)
Thus, the inverse discrete cosine transform is applied to each displaced DCT coefficient, and the three-dimensional joint angle for each displaced joint is output.

[0094]

(Equation 5)

N: total number of data (total number of frames) k: order of DCT coefficient m: data number (frame number) M: motion modification setting parameter in order displacement mode a: motion modification setting parameter θ _m in energy displacement mode i, j): For the m-th frame, the three-dimensional joint angle with respect to the j-axis of the intrinsic coordinate system for the i-th joint According to this motion modification mode, the motion magnitude is changed by the parameter a, By selecting the parameter M of, smooth movement can be realized.
In addition, it becomes possible to express a fine operation each time the parameter M is increased and higher-order components are added.
Therefore, in this mode, it is possible to realize the motion modification mode in which the size of the motion and the setting of the delicate motion can be performed at the same time with two parameters. (5) Energy / Order / Operation Speed Composite Displacement Mode FIG. 13 is a configuration diagram of the joint angle displacement unit 14 that displaces a three-dimensional joint angle with respect to each joint in the energy / order / operation speed composite displacement mode. The discrete cosine transform unit 91 similar to FIG. 9 handles the three-dimensional joint angle for each joint as time series data, applies the discrete cosine transform to the time series data, and outputs each DCT coefficient.
The conversion coefficient displacement unit 121 is the energy displacement unit 93 of FIG.
And a circuit similar to the order selection unit 102 of FIG. 10, the energy displacement unit 93 uses the parameter a,
The energy of each DCT coefficient output from the discrete cosine transform unit 91 is displaced, and the order selection unit 102 uses the order M to output each DC output from the discrete cosine transform unit 91.
Displace the T coefficient. The inverse discrete cosine transform unit 131
The conversion order limiting unit 132 is provided, and according to (Equation 6),
Using the parameter n, the inverse transform in which the transform order of the inverse discrete cosine transform is limited is applied to each of the displaced DCT coefficients, and the three-dimensional joint angle for each displaced joint is output.

[0096]

(Equation 6)

N: Total number of data (total number of frames) k: Order of DCT coefficient m: 0, 1, 2, ..., (N-1) / n M: Operation modification setting parameter in order displacement mode a: Energy displacement Motion modification setting parameter n in mode: Velocity setting parameter in motion speed displacement mode θ _mn (i, j): Three-dimensional joint angle of the mnth frame with respect to the j-axis of the proper coordinate system for the i-th joint This motion modification mode According to this, it is possible to set the operation speed with the parameter n, the magnitude of the operation with the parameter a, and the fine delicate operation with the parameter M. Therefore, in this mode, it is possible to realize an operation modification mode in which the operation speed, the size of the operation, and the fine and delicate operation can be simultaneously set with three parameters.

Next, the behavioral synthesis unit 1 in the present embodiment.
The operation of 6 will be described more specifically. It is assumed that the proper coordinate system for the root joint is set to an effective joint that does not change even by motion modification. In the present embodiment, the root joint is the hip joint 21 shown in FIG. The synthesis procedure by the behavioral synthesis unit 16 including the unique coordinate system generation unit is as follows.

1. Use the root joint as the parent joint (initial setting).

2. The origin of the proper coordinate system for the parent joint is applied to that joint.

3. Shape data corresponding to the joint is fitted along the positive z-axis from the origin of the intrinsic coordinate system.

4. The end point of the shape data other than the origin is set as the origin of the unique coordinate system for the child joint.

5. Using the three-dimensional joint angle with respect to the parent joint, the proper coordinate system for the parent joint is rotated in the zxy order, and this is used as the proper coordinate system for the child joint.

6. The child joint is made the parent joint for the next processing.

7. The above steps 2 to 6 are repeated until the processing for each effective joint for the frame is completed.

8. Based on the desired cycle,
Repeat 7.

Note that, for the sake of simplicity in the above description, the mode of motion modification is intended for all effective joints.

As described above, by using the unique coordinate system, the shape data can be uniquely applied, and by using the three-dimensional joint angle, the operations such as twisting and rotating can be performed. Alternatively, by processing the three-dimensional joint angle with a function, it is possible to easily add variously changing motions to the standard motion mode.

FIG. 14 is a block diagram of an embodiment of an image processing apparatus according to claim 10 of the present invention. That is, for example, when a person performs a predetermined motion, the proper coordinate system storage unit 141 stores the proper coordinate system for each of the effective joints for each frame for the standard motion in which one cycle of the motion is a unit. . The shape data storage unit 145 stores shape data regarding a human body part corresponding to a plurality of unique coordinate systems for each effective joint. The input device 147 inputs an instruction regarding an operation desired by the operator. Control unit 1
The control unit 48 controls the image processing according to the instruction input to the input device 147. Three-dimensional joint angle calculation unit 142
Is a three-dimensional joint angle for each of all or some of the effective joints based on the proper coordinate system stored in the proper coordinate system storage unit 141 according to the desired mode of motion modification according to the operator's instruction. calculate. The joint angle displacement section 143 is
The three-dimensional joint angle calculation unit 1 according to the mode of motion modification.
Based on the three-dimensional joint angles calculated by 42, the three-dimensional joint angles for all or some of the effective joints are displaced. The unique coordinate system generation unit 144 has a three-dimensional joint angle and / or a joint angle displacement unit 143 calculated by the three-dimensional joint angle calculation unit 142 according to the mode of motion modification.
A unique coordinate system for each of all or some of the effective joints is generated based on the three-dimensional joint angles displaced by. The behavioral synthesis unit 146 determines the unique coordinate system and / or the unique coordinate system stored in the unique coordinate system storage unit 141 according to the behavior modification mode.
Alternatively, based on the unique coordinate system generated by the unique coordinate system generation unit 144, the corresponding shape data is read from the shape data stored in the shape data storage unit 145,
Then, they are assigned to each of the corresponding effective joints.

FIG. 15 is a block diagram of an embodiment of an image processing apparatus according to claim 19 of the present invention. That is, 3
For example, when a person performs a predetermined motion, the dimensional joint angle storage unit 151 stores 3 for each of the proper coordinate system of the root joint and each effective joint for each frame for the standard motion in which one cycle of the motion is a unit. The dimensional joint angle is stored. The shape data storage unit 154 stores and holds shape data regarding a human body part corresponding to a plurality of unique coordinate systems for each effective joint. The input device 157 inputs an instruction regarding an operation desired by the operator. The control unit 158 controls the image processing according to the instruction input to the input device 157. The proper coordinate system determination unit 152 uses the proper coordinate system for the root joint and the three-dimensional joint angle stored in the three-dimensional joint angle storage unit 151, and determines the effective joints for each effective joint in the order from the root joint to the parent and child. Determine the intrinsic coordinate system. The joint angle displacement unit 153, based on the three-dimensional joint angle stored in the three-dimensional joint angle storage unit, determines all or some of the effective joints according to the desired mode of motion modification according to the operator's instruction. Displace the 3D joint angle for each. The unique coordinate system generation unit 154, according to the mode of motion modification, has the three-dimensional joint angle and / or joint angle displacement unit 153 stored in the three-dimensional joint angle storage unit 151.
A unique coordinate system for each of all or some of the effective joints is generated based on the three-dimensional joint angles displaced by.
The behavioral synthesis unit 155 determines the shape data based on the unique coordinate system determined by the unique coordinate system determination unit 152 and / or the unique coordinate system generated by the unique coordinate system generation unit 154 according to the behavior modification mode. The corresponding shape data is read out from the shape data stored in the storage unit 156, and is assigned to each corresponding effective joint.

In the above embodiment, the input device 1 is used.
The image processing corresponding to the desired motion is supposed to be performed based on the operator's instruction input by 7, 147 or 157, but not limited to this, it is described that a moving image such as an animation is generated. Image processing may be performed based on an instruction of the executed program.

Further, in the above embodiment, the eigencoordinate system and the three-dimensional joint angle are limited by the effective joint, but the invention is not limited to this. The eigencoordinate system and the three-dimensional joint angle are defined by the effective joint. It may be limited by a parent-child line which is a line segment connecting the parent joint or the child joint. In this case, the proper coordinate system for the parent-child line is synonymous with the proper coordinate system for the effective joint, and the three-dimensional joint angle with respect to the parent-child line is synonymous with the three-dimensional joint angle with respect to the effective joint.

FIG. 16 is a block diagram of an embodiment of an image processing apparatus according to claim 25 of the present invention. When an object having a plurality of “effective joints serving as a reference of motion” performs a predetermined motion, the basic motion storage unit 161 of the image processing apparatus 160 performs a periodic motion or an aperiodic motion for each frame. , A storage unit that stores functionalized three-dimensional joint angles for all or some of the effective joints as basic motion data. This storage unit may be stored in either the internal storage device of the image processing device 160 or an external storage device.

The input device 162 is an input device for inputting an instruction regarding an operation desired by the operator.

The basic operation selection section 163 has the input device 162.
Based on the instruction input by the basic operation storage unit 161.
The desired basic operation data is selected from the plurality of basic operation data stored in the above item. Basic operation storage unit 16
1 outputs the basic motion data selected by the basic motion selection unit 163 to the basic motion modification generation unit 165.

The parameter setting section 164 uses the input device 16
Based on the instruction input by 2, the action modification parameter required when changing the action composed of the basic action data selected by the basic action selecting means to the action accompanied by the action modification according to the instruction is set. It is something to set.

The basic action modification generation unit 165 is configured by the basic action data selected by the basic action selection unit 163 and input from the basic action storage unit 161 by using the action modification parameter set by the parameter setting unit 164. The motion is changed to a motion accompanied by motion modification desired by the operator, and basic motion modification data is generated.

The additional operation selecting section 166 uses the input device 162.
Based on the instruction input by the basic action modification generation unit 1
The operation is different from the operation configured by the basic action modification data generated by 65, and a desired additional action to be added to the operation configured by the basic motion modification data is selected.

The additional motion generating section 167 generates additional motion data by the basic motion modifying data generated by the basic motion modifying generating section 165 and the additional motion selected by the additional motion selecting section 166.

The behavior synthesizing device 168 synthesizes the basic behavior modification data generated by the basic behavior modification generation unit 165 and the additional motion data generated by the additional motion generation unit 167, and is configured by the basic motion modification data. The action data relating to the action in which the action constituted by the additional action data is added to the action to be generated is generated.

Next, the operation of this embodiment will be described.

By operating the skeleton model for a person shown in FIG. 2B, 3 for the effective joint is obtained.
The dimensional joint angle changes with time as shown in FIG. In this way, the motion can be expressed by the time change of the three-dimensional joint angle for each of the effective joints.
Therefore, the temporal change of the three-dimensional joint angle shown in FIG. 17 is approximated by a function applied to each of the X-axis rotation θx, the Y-axis rotation θy, and the Z-axis rotation θz. In this embodiment,
The discrete cosine transform represented by (Equation 7) is used for the approximation function. However, i corresponds to the x, y, and z axes, j corresponds to the number of frames, and k corresponds to the effective joint.

[0123]

(Equation 7)

FIG. 18 shows an example of function approximation of FIG. 17 by discrete cosine transform. As for the periodical motion such as walking, the motion for one cycle is functionalized. In addition, for non-periodic motions such as diving in swimming, the motion for one time is made into a function. In this way, basic motion data such as “walk” and “dive” that are approximated by the discrete cosine transform are stored in the basic motion storage unit 161. By the way, when a certain operation is modeled by DCT, DCT
The quality of the operation can be changed by the order of.
In general, energy concentrates on the low frequency components of the transform coefficients, as shown in FIG. Therefore, when registering the basic motion data in the database, if the high-frequency components that have little influence on the reconstruction of the basic motion are ignored, the database is configured with less data than when the time series data of the three-dimensional joint angle is registered as it is. can do.

The basic operation selection section 163 has the input device 162.
Based on the operator's instruction input by, the desired basic motion data is selected from the functionalized basic motion data stored in the basic motion storage unit 161. Although the five types of displacement modes relating to the predetermined function for displacing the three-dimensional joint angle used by the joint angle displacement device 14 in FIG. 1 have been described above, the parameter setting unit 164 can also use the same motion modification parameter. . The basic motion modification generation unit 165 uses the motion modification parameter set by the parameter setting unit 164 to perform an inverse discrete cosine transform on the basic motion data selected by the basic motion selection unit 163 and input from the basic motion storage unit 161. By applying it, a three-dimensional joint angle which is basic motion modification data is generated.

That is, the parameter setting unit 164 sets a motion modification parameter having a parameter a representing the magnitude of the motion and a parameter b representing the speed of the motion based on the operator's instruction input by the input device 162. Set. Then, the basic motion modification generation unit 165 uses the motion modification parameter to apply the inverse discrete cosine transform of (Equation 8) to the basic motion data selected by the basic motion selection unit 163.

[0127]

[Equation 8]

Here, with respect to the size of the motion by the parameter a, when a = 1.0, a motion having a size based on the basic motion data stored in the basic motion storage unit 161 is generated. Then, a larger motion is generated when a> 1.0, and a smaller motion is generated when 0 <a <1.0. Also, regarding the speed of the operation by the parameter b, when b = 1, the operation of the speed based on the basic operation data stored in the basic operation storage unit 161 is generated. Then, when b> 1.0, a faster operation is generated. However, the parameter b is an integer. In this way, operation modification can be easily performed by manipulating these two parameters. It should be noted that the basic motion data selected by the basic motion selection unit 163 is used as it is in the motion modification in the case of a = 1 and b = 1, but this is also handled as one aspect of the motion modification.

The additional operation selecting section 166 uses the input device 162.
The additional operation is selected based on the operator's instruction input by. This additional operation is performed by the basic operation modification generation unit 165.
The operation is different from the operation configured by the basic action modification data generated by, and is an action added to the configured action. The additional action generation unit 167 generates additional action data based on the basic action modification data generated by the basic action generation unit 165 and the additional action selected by the additional action selection unit 166. For example, if the basic motion is a "walk" motion, an additional motion of "waving" is added to it. The behavioral synthesis unit 168 synthesizes the basic behavioral modification data generated by the basic behavioral modification generation unit 165 and the additional motional data generated by the additional motional generation unit 167 into a motion constituted by the basic motional modification data. The operation data regarding the operation to which the operation configured by the additional operation data is added is generated.

In the present embodiment, the basic operation selecting section 163 stores the selected basic operation data in the basic operation storage section 1.
Although the configuration is such that 61 is output to the basic action modification generation unit 165, the present invention is not limited to this, and the basic action selection unit 163 is not limited to this.
May read the selected basic motion data from the basic motion storage unit 161 and output it to the basic motion modification generation unit 165.

Further, in the present embodiment, the input device 162 is used.
Although it has been stated that the motion modification parameter is set based on the instruction input by, the parameter setting unit 164 may be provided with an input unit for inputting the motion modification parameter desired by the operator. .

Further, in the present embodiment, the input device 162 is used.
Although it is assumed that the image processing corresponding to the desired motion is performed based on the operator's instruction input from, the instruction is not limited to this, and the instruction of the program described to generate a moving image such as animation is not limited to this. Based on this, image processing may be performed.

In addition, in the configuration of the present embodiment shown in FIG. 16, the additional action generation section 167 is the basic action modification generation section 1
The additional motion data is generated while taking into consideration the basic motion modification data generated by 65, and the motion synthesis unit 16
8 is configured to combine the basic action modification data and the additional action data, but the configuration is not limited to this, and the additional action generation unit 167 generates the additional action data without considering the basic action modification data. , And behavioral synthesis unit 1
68 may combine the basic motion modification data with the basic motion modification data and combine the additional motion data generated by the additional motion generation unit 167.

FIG. 20 is a block diagram showing another embodiment of the basic action modification generator 165 shown in FIG. The basic action modification unit 201 of the basic action modification generation unit 165a uses the action modification parameter set by the parameter setting unit 164 of FIG. 16, is selected by the basic action selection unit 163, and is input from the basic action storage unit 161. The inverse discrete cosine transform used by the basic motion modification generation unit 165 is applied to the basic motion data to calculate the three-dimensional joint angle for each effective joint. Periodic motion generation unit 202
When the motion processed by the basic motion changing unit 201 is a cyclic motion, the basic motion modification data is generated by repeating the calculated three-dimensional joint angle for the cycle for a desired period. In addition, the aperiodic motion generation unit 203
When the motion processed by the basic motion changing unit 201 is an aperiodic motion, the basic motion modification data is generated from the three-dimensional joint angle calculated without repeating.

FIG. 21 is a block diagram showing another embodiment of the additional action generator 167 shown in FIG. The coordinate input unit 211 of the additional action generation unit 167a inputs the coordinates that move in response to the additional action selected by the additional action selection unit 166 shown in FIG. The data relating to the coordinates to be moved is stored in the storage device in which the basic operation storage unit 161 is stored or in a storage device different from the storage device in association with the additional operation. The coordinate tracking unit 212
In consideration of the motion based on the basic motion data generated by the basic motion generation unit 165, the effective joint corresponding to the tip of the effective joint selected by the additional motion selection unit 166 is input by the coordinate input unit 211. The additional operation data is generated by following the coordinate set.
The coordinate tracking unit 212 uses inverse kinematics.

Here, a detailed calculation method of inverse kinematics will be described. An additional operation when a person follows the ball with the right hand will be described as an example. Elbow joint and shoulder joint 3
The dimension joint angles are (θ1x, θ1y, θ1z) and (θ1x
2x, θ2y, θ2z). Also, the length from the shoulder to the elbow
l1z, and the length from the elbow to the hand is l2z. At this time, the coordinate r of the position of the hand with reference to the coordinate system of the shoulder is (Equation 9)
Can be represented by However, o is the origin of the shoulder coordinate system. Further, Xθ, Yθ, Zθ, and L ₁ respectively represent a matrix of X-axis rotation, Y-axis rotation, Z-axis rotation, and parallel movement in the Z-axis direction.

[0137]

[Equation 9]

Since these four expressions are non-linear with respect to Θ, the amount of change in the position of the hand becomes linear with respect to the amount of change in Θ by the velocity decomposition method and is represented by (Equation 10). However,
J is called a Jacobian, and is a matrix that has the partial differential value of the function as an element.

[0139]

[Equation 10]

In general, since J is not regular, the transposed matrix J ^t of J is multiplied from the left to obtain regular J ^t J. J ^t J
If there is an inverse matrix in, δΘ can be obtained as in (Equation 11).

[0141]

[Equation 11]

Therefore, by inputting the movement amount δr of the ball as the movement amount of the hand, the change amount δΘ of the three-dimensional joint angle for each effective joint can be calculated, and the additional movement data can be generated. . As shown in FIG. 22, a motion of “walking” + “following the ball” can be realized.
In this way, by performing an action different from the basic action on some joints using inverse kinematics, it is possible to perform an action according to the situation. FIG. 23 is a flowchart showing the operation of the image processing apparatus of FIG. 16 provided with the additional operation generation unit 167a.

FIG. 24 is a block diagram showing another embodiment of the additional action generator 167 shown in FIG. The additional operation generation unit 167b is configured to include a plurality of coordinate input units 211-1 to 21-1.
1-n and a plurality of coordinate tracking units 212-1 to 212-n corresponding to them. The coordinate input unit and the coordinate tracking unit of each pair generate additional motion data in the same manner as in FIG. Thereby, for example, the additional motion data can be generated for the left and right arms and the left and right feet of a person. Assuming that each target coordinate is rk (xk, yk, zk), the process by the additional motion generation unit for the effective joint corresponding to each target coordinate is given by (Equation 12).

[0144]

[Equation 12]

Similar to FIG. 21, the amount of change in the three-dimensional joint angle for each effective joint is obtained from the target amount of movement. δrk is the amount of movement of the kth target coordinate (δx, δy, δ
z), Jk is the Jacobian of the matrix F (Θk), and Θk is (θk1, θk2, ..., θkN) t. In this way, it is possible to independently perform the target tracking operation for each effective joint. Therefore, different targets can be tracked by the right hand and the left hand while walking, and more complicated motions can be performed.

FIG. 25 is a block diagram showing another embodiment of the additional action generator 167 shown in FIG. The reference additional motion storage unit 241 of the additional motion generating unit 167c stores, as the standard additional motion data, the data regarding the three-dimensional joint angles for all or some of the effective joints for each predetermined additional motion for each frame. It is a thing. In the reference addition operation storage unit 241, for example, the movements of the arm, leg, head, etc. are functionalized and stored. The motion on the head is a motion such as "nod" or "shake the head". The motion of the arm is "waving" and the motion of the leg is "kicking". Similar to the basic motion storage unit 161 of FIG. 16, the reference additional motion storage unit 241 also stores conversion data obtained by applying the discrete cosine transform to the temporal change of the three-dimensional joint angle. The reference addition operation storage unit 241 is not limited to the storage device in which the basic operation storage unit 161 is stored, and may be stored in another storage device.

The additional operation reading unit 242 adds the reference operation corresponding to the additional operation selected by the additional operation selecting unit 166 shown in FIG. 16 from the plurality of reference additional operation data stored in the reference additional operation storage unit 241. The action data is read out and input to the additional action modification generation unit 244.

The additional action modification setting unit 243 sets the action modification parameter according to the additional action selected by the additional action selecting unit 166. Here, it is assumed that the same operation modification parameter as that of the parameter setting unit 164 shown in FIG. 16 is used.

The additional action modification generation unit 244 uses the action modification parameter set by the additional motion modification setting unit 243 in consideration of the basic motion modification data generated by the basic motion modification generation unit 165 shown in FIG. Then, the reference additional motion data read by the additional motion reading unit 242 is changed to generate the additional motion data.

That is, the additional motion modification generation unit 244 uses the motion modification parameter set by the additional motion modification setting unit 243 to generate the basic motion modification data in the standard additional motion data read by the additional motion read unit 242. Part 16
The inverse discrete cosine transform similar to that in 5 is applied to calculate the three-dimensional joint angle.

Next, the operation of the additional motion generating section 167c will be described by taking as an example the case where the additional motion of "waving" is added to the basic motion of "walking".
When the additional action selected by the additional action selection unit 166 is the action of “shaking a large hand”, the additional action reading unit 242 reads the action of “waving a hand” from the reference additional action storage unit 241. Additional motion modification setting unit 243
Sets the motion modification parameter according to the motion of the additional motion “large” selected by the additional motion selection unit 166. In this case, the parameter a is set to be larger than 1.0. The additional motion modification generation unit 244 uses the motion modification parameter representing the motion of “large” to change the read “waving hand” to generate the additional motion data that is the motion of “large motion”. To generate.

By setting the motion modification parameter b to be larger than 1.0, the motion modification of "fast" can be added to "waving".

In this way, "walk" as shown in FIG.
+ It is possible to realize an operation of “waving hands (largely)”. In this way, by making a meaningful partial operation an additional operation, it is possible to perform an operation suitable for the situation. Further, by changing the motion modification parameter for the additional motion data, it is possible to realize a large motion and a small motion. Further, FIG. 27 is a flowchart showing the operation of the image processing apparatus of FIG. 16 provided with the additional operation generating unit 167c.

FIG. 28 is a block diagram showing another embodiment of the additional action generator 167 shown in FIG. The additional operation generation unit 167d is capable of generating a plurality of additional operation data, and individual constituent elements are the same as those in FIG. The reference additional motion storage unit 271 stores, as reference additional motion data, data regarding a three-dimensional joint angle for all or some of the effective joints for each frame for a predetermined additional motion.

Additional operation reading units 242-1 to 242-n
Respectively reads the reference addition operation data corresponding to the addition operation selected by the addition operation selection unit 166 shown in FIG. 16 from the plurality of reference addition operation data stored in the reference addition operation storage unit 271 and adds the data. The motion modification generation units 244-1 to 244-n are made to input.

Additional action modification setting units 243-1 to 243-n
Are for setting motion modification parameters corresponding to the additional motions selected by the additional motion selection unit 166.

Additional action modification generators 244-1 to 244-n
Are respectively the basic action modification generation units 16 shown in FIG.
In consideration of the basic motion modification data generated by 5, the additional motion reading unit 2 uses the motion modification parameters set by the additional motion modification setting units 243-1 to 243-n.
The reference additional motion data read by 42-1 to 242-n is changed to generate the additional motion data.

According to the image processing apparatus of FIG. 16 equipped with the additional motion generating section 167d, it is possible to select a plurality of additional motions that can be added to the basic motion. The additional motion generation unit 167d sets motion modification parameters for each of the plurality of additional motions selected by the additional motion selection unit 166, and generates a plurality of additional motion data based on these parameters.

For example, it becomes possible to add the additional motions of "nodding" and "waving" to the basic motion "walking". At this time, by changing the motion modification parameter, it is possible to simultaneously realize a large nod, a fast nod, a large hand shake, and a fast hand shake. In this way, it is possible to generate more varied motions.

Further, by additionally using the method using the database and the method using the inverse kinematics as the additional operation, it is possible to realize more flexible operation generation. For example, it is possible to realize walking the ball with the left hand and waving while swinging the right hand. FIG. 29 is a flowchart showing the flow of processing when the method using the database and the method using the inverse kinematics are combined.

[0161]

As is apparent from the above description,
The image processing apparatus of the present invention can uniquely fit the corresponding part of the object between the effective joint and the adjacent effective joint, and can also take into account the unidirectional orientation of the object. By using the system, it is possible to easily synthesize images in standard operation.

Further, the image processing apparatus of the present invention has an effect that operations such as twisting and rotating can be easily performed by using the three-dimensional joint angle with respect to the effective joint.

Further, the image processing apparatus of the present invention has an effect that various motion modifications can be easily executed by using the proper coordinate system and the three-dimensional joint angle with respect to the effective joint.

Further, according to the image processing apparatus of the present invention, 3
By dividing the dimensional joint angle into basic motion data that has been converted into a function and additional motion data that can be added to it, it is possible to not only reduce the amount of motion data and calculation amount, but also generate various motion data according to the situation. Become.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an embodiment of an image processing apparatus according to claim 1 of the present invention.

FIG. 2 is a diagram schematically showing the positions of joints when a person is running.

FIG. 3 is a diagram showing how the four-joint unique coordinate system is determined.

FIG. 4 is a diagram showing a state of determining a unique coordinate system for three joints.

FIG. 5 is a diagram showing how the two-joint unique coordinate system is determined.

FIG. 6 is a flowchart regarding determination of a 2-joint unique coordinate system, a 3-joint unique coordinate system, and a 4-joint unique coordinate system.

FIG. 7 is a diagram showing a state of calculating a three-dimensional joint angle with respect to a joint.

FIG. 8 is a graph showing the unique coordinate system for the parent joint shown in FIG.
Diagram showing that when the axes are rotated in the order of the y-axis, the unique coordinate system for the child joint is obtained.

FIG. 9 shows 3 for a joint according to the energy displacement mode.
Configuration diagram of a joint angle displacement device 14 for performing displacement of a three-dimensional joint angle

FIG. 10 is a joint angle displacement device 1 for performing displacement of each three-dimensional joint angle with respect to each joint in the order displacement mode.
Configuration diagram of 4

FIG. 11 is a configuration diagram of a joint angle displacement device 14 that performs displacement of each three-dimensional joint angle with respect to each joint in a motion velocity displacement mode.

FIG. 12: 3 by the energy / order composite displacement mode
Configuration diagram of a joint angle displacement device 14 for performing displacement of a three-dimensional joint angle

FIG. 13 is a configuration diagram of a joint angle displacement device 14 that displaces a three-dimensional joint angle by an energy / order / movement speed composite displacement mode.

FIG. 14 is a configuration diagram of an embodiment of an image processing apparatus according to claim 10 of the present invention.

FIG. 15 is a configuration diagram of an embodiment of an image processing apparatus according to claim 19 of the present invention.

FIG. 16 is a configuration diagram of an embodiment of an image processing apparatus according to claim 25 of the present invention.

FIG. 17 is a diagram showing that the three-dimensional joint angle with respect to each of the effective joints changes temporally by operating the skeleton model for the person shown in FIG. 2B.

FIG. 18 is a diagram showing an example of function approximation of FIG. 17 by discrete cosine transform.

FIG. 19 is a diagram showing an example in which energy after DCT conversion is concentrated on low-frequency components of conversion coefficients.

FIG. 20 is a configuration diagram showing another embodiment of the basic action modification generation unit 165 shown in FIG.

FIG. 21 is a configuration diagram showing another embodiment of the additional action generation unit 167 shown in FIG.

FIG. 22 is a diagram showing an example of realizing an operation of “walking” + “following the ball”.

FIG. 23 is a flowchart showing the operation of the image processing apparatus of FIG. 16 including the additional operation generation unit 167a.

FIG. 24 is a configuration diagram showing another embodiment of the additional action generation unit 167 shown in FIG.

FIG. 25 is a configuration diagram showing another embodiment of the additional action generation unit 167 shown in FIG.

FIG. 26 is a diagram showing an example of realizing an operation of “walking” + “waving hands (largely)”.

FIG. 27 is a flowchart showing the operation of the image processing apparatus of FIG. 16 provided with the additional operation generation unit 167c.

28 is a configuration diagram showing another embodiment of the additional action generation unit 167 shown in FIG.

FIG. 29 is a flowchart showing a processing flow when a method using a database and a method using inverse kinematics are combined.

[Explanation of symbols]

 10 ... Image processing device 11 ... 3D joint position storage unit 12 ... Unique coordinate system determination unit 13 ... 3D joint angle calculation unit 14 ... 3D joint angle displacement unit 15 ... Shape data storage unit 16 ... Motion synthesis unit 17 ... Input Device 18 ... Control unit 140 ... Image processing device 141 ... Unique coordinate system storage unit 142 ... Three-dimensional joint angle calculation unit 143 ... Joint angle displacement unit 144 ... Unique coordinate system generation unit 145 ... Shape data storage unit 146 ... Motion synthesis unit 147 Input device 148 Control unit 150 Image processing device 151 Three-dimensional joint angle storage unit 152 Proper coordinate system determination unit 153 Joint angle displacement unit 154 Proper coordinate system generation unit 155 Behavior combination unit 156 Shape data storage Unit 157 ... Input device 158 ... Control unit 160 ... Image processing device 161 ... Basic operation storage unit 162 ... Input device 163 ... Basic operation selection unit 16 4 ... Parameter setting unit 165 ... Basic action modification generation unit 166 ... Additional action selection unit 167 ... Additional action generation unit 168 ... Behavior synthesis unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location 9365-5H G06F 15/62 322 M 9365-5H 340 K

Claims (30)

[Claims] 1. When an object having a plurality of joints performs a predetermined motion, all or some of the plurality of joints are included in each frame for a standard motion in which one cycle of the motion or a plurality of cycles is a unit. A three-dimensional joint position storage unit that stores a three-dimensional coordinate position for each effective joint, and a plurality of unique joint coordinate systems for each of the effective joints, which is a unique three-dimensional coordinate system for the effective joint. A unique coordinate system for each of the effective joints is determined based on the shape data storage unit that stores the shape data relating to the corresponding part of the object and the three-dimensional coordinate position stored by the three-dimensional joint position storage unit. And a unique coordinate system determining means for performing each of the effective joints based on the unique coordinate system determined by the unique coordinate system determining means according to a desired mode of motion modification. A three-dimensional joint angle calculating means for calculating a three-dimensional joint angle, which is a three-dimensional angle between the intrinsic coordinate system and an adjacent intrinsic coordinate system, and the three-dimensional joint according to the mode of motion modification. Joint angle displacement means for displacing the three-dimensional joint angles for all or some of the effective joints based on the three-dimensional joint angles calculated by the angle calculation means, and the three-dimensional joints according to the mode of motion modification. Unique coordinates for generating a unique coordinate system for all or some of the effective joints based on the three-dimensional joint angle calculated by the joint angle calculation means and / or the three-dimensional joint angle displaced by the joint angle displacement means A system generation means, and a fixed coordinate system generated by the proper coordinate system determined by the proper coordinate system determination means and / or the proper coordinate system generation means in accordance with the behavior modification mode. Based on the coordinate system, the effective in each of the joints, corresponding image processing apparatus characterized by comprising an operation combining means for performing the allocation of the shape data. 2. The proper coordinate system determining means determines a vector from a target joint, which is an effective joint for which the proper coordinate system is determined, to a first joint adjacent thereto, as a first axis of the proper coordinate system for the target joint. And a vector from the second joint adjacent to the target joint to the third joint is defined as a temporary vector, and an outer product of the temporary vector and the first axis is defined as a second coordinate of the unique coordinate system with respect to the target joint. An axis is defined as an axis, and an outer product of the first axis and the second axis is defined as a third axis of an intrinsic coordinate system with respect to the target joint.
The image processing device described. 3. The proper coordinate system determining means determines a vector from a target joint, which is an effective joint for which the proper coordinate system is determined, to a first joint adjacent to the target joint, as a first axis of the proper coordinate system for the target joint. And a composite vector of the vector from the target joint to the second joint adjacent thereto and the first axis or its inverse vector is a temporary vector, and an outer product of the temporary vector and the first axis is the target The second axis of the proper coordinate system for the joint, and the outer product of the first axis and the second axis is the third axis of the proper coordinate system for the target joint.
The image processing device described. 4. The proper coordinate system determining means determines a vector from a target joint, which is an effective joint for which the proper coordinate system is determined, to a first joint adjacent thereto, as a first axis of the proper coordinate system for the target joint. In the proper coordinate system for the second joint adjacent to the target joint, one of the remaining coordinate axes other than the coordinate axis determined based on the vector from the second joint to the target joint and the first joint An outer product with an axis is set as a second axis of the proper coordinate system for the target joint, and a cross product of the first axis and the second axis is set as a third axis of the proper coordinate system for the target joint. Item 1
The image processing device described. 5. The joint angle displacement means applies a discrete cosine transform to a three-dimensional joint angle for each of all or some of the effective joints, and outputs a DCT coefficient. Transform coefficient displacing means for displacing a predetermined element included in each of them, and inverse discrete cosine transform for applying an inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacing means The image processing apparatus according to claim 1, further comprising a conversion unit. 6. The image processing apparatus according to claim 5, wherein the predetermined element is energy of a frequency component and / or a selected order. 7. The inverse discrete cosine transforming means comprises:
The image processing apparatus according to claim 5, wherein the inverse discrete cosine transform applied to the CT coefficient or the displaced DCT coefficient is limited. 8. The unique coordinate system generation means uses the three-dimensional joint angle with respect to the effective joint to rotate each of the three axes of the unique coordinate system with respect to the effective joint so that the effective joint adjacent to the effective joint is rotated. The image processing apparatus according to claim 1, wherein a unique coordinate system for the joint is generated. 9. When an object having a plurality of joints performs a predetermined motion, effective joints that are all or a part of the plurality of motions for each frame for a standard motion in a unit of one cycle or a plurality of cycles of the motion. A unique coordinate system storage unit that stores a unique coordinate system, which is a three-dimensional coordinate system unique to the effective joint, and the object corresponding to a plurality of unique coordinate systems for each of the effective joints. Shape data storage means for storing shape data relating to the body part, and all or part of the effective joint based on the proper coordinate system stored in the proper coordinate system storage means according to a desired mode of motion modification. Three-dimensional joint angle calculation means for calculating a three-dimensional joint angle, which is a three-dimensional angle between the proper coordinate system and the adjacent proper coordinate system for each of them, and, according to the mode of motion modification, A joint angle displacing means for displacing the three-dimensional joint angles for all or some of the effective joints based on the three-dimensional joint angles calculated by the three-dimensional joint angle calculating means; Then, based on the three-dimensional joint angle calculated by the three-dimensional joint angle calculation means and / or the three-dimensional joint angle displaced by the joint angle displacement means, a proper coordinate system for all or some of the effective joints. And a unique coordinate system generated by the unique coordinate system generation means, and / or the unique coordinate system stored in the unique coordinate system storage means according to the mode of operation modification. An image processing apparatus comprising: a motion synthesizing unit that allocates the corresponding shape data to each of the effective joints based on the motion synthesizing unit. 10. The proper coordinate system stored in the proper coordinate system storage means is a vector that directs a vector from a target joint, which is an effective joint for which the proper coordinate system is determined, to a first joint adjacent to the target joint. Is a first axis of the proper coordinate system, and a vector from the second joint adjacent to the target joint to the third joint is a temporary vector, and an outer product of the temporary vector and the first axis is the target 10. The second axis of the unique coordinate system for the joint is determined, and the outer product of the first axis and the second axis is determined as the third axis of the unique coordinate system for the target joint. The image processing device described. 11. The eigencoordinate system stored in the eigencoordinate system storage means is a vector from a target joint, which is an effective joint for which the eigencoordinate system is determined, to a first joint adjacent thereto, Is a first axis of the proper coordinate system with respect to the first joint, a vector from the target joint to the second joint adjacent thereto and the first axis or the inverse vector thereof is a temporary vector, and the temporary vector and the first vector It is determined by setting the outer product of the first axis as the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis as the third axis of the proper coordinate system for the target joint. The image processing apparatus according to claim 9, wherein 12. The eigencoordinate system stored in the eigencoordinate system storage means is a vector for directing a vector from a target joint, which is a joint that determines the eigencoordinate system, toward a first joint adjacent thereto, to the target joint. The first axis of the coordinate system, and one of the other than the coordinate axis determined based on the vector from the second joint to the target joint in the proper coordinate system for the second joint adjacent to the target joint The outer product of the coordinate axes of the above and the first axis is the second axis of the proper coordinate system for the target joint, and the outer product of the first axis and the second axis is the third axis of the proper coordinate system for the target joint. The image processing device according to claim 9, wherein the image processing device is determined by 13. The joint angle displacement means applies a discrete cosine transform to the three-dimensional joint angles for all or some of the effective joints, and outputs a DCT coefficient. Transform coefficient displacing means for displacing a predetermined element included in each of them, and inverse discrete cosine transform for applying an inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacing means The image processing apparatus according to claim 9, further comprising a conversion unit. 14. The image processing apparatus according to claim 13, wherein the predetermined element is energy of a frequency component and / or a selected order. 15. The image processing apparatus according to claim 13, wherein the inverse discrete cosine transform means limits the inverse discrete cosine transform applied to the DCT coefficient or the displaced DCT coefficient. 16. The proper coordinate system generating means uses the three-dimensional joint angle with respect to the effective joint to rotate each of the three axes of the proper coordinate system with respect to the effective joint, thereby enabling the effective joint adjacent to the effective joint. The image processing apparatus according to claim 9, wherein a unique coordinate system for the joint is generated. 17. When an object having a plurality of joints performs a predetermined motion, all or some of the plurality of joints are included in each frame for a standard motion in which one cycle of the motion or a plurality of cycles is used as a unit. For each of the effective joints, the three-dimensional angle between the proper coordinate system which is the proper three-dimensional coordinate system for the effective joint and the adjacent proper coordinate system, 3
A three-dimensional joint angle storage unit that stores a three-dimensional joint angle; a shape data storage unit that stores shape data relating to a part of the object corresponding to a plurality of unique coordinate systems for each of the effective joints; A unique coordinate system determining unit that determines a unique coordinate system for each of the effective joints based on the unique coordinate system for the joint and the three-dimensional joint angle stored in the three-dimensional joint angle storage unit, and a desired motion modification mode. According to the three-dimensional joint angle storage means, the joint angle displacement means for displacing the three-dimensional joint angles for all or some of the effective joints based on the three-dimensional joint angles stored in the three-dimensional joint angle storage means; According to the aspect, the three-dimensional joint angle stored in the three-dimensional joint angle storage means and / or the three-dimensional joint angle displaced by the joint angle displacement means Based on the proper coordinate system generating means for generating a proper coordinate system for each of all or some of the effective joints, and the proper coordinate system determined by the proper coordinate system determining means according to the mode of motion modification, and / or Alternatively, the image processing apparatus is provided with a motion synthesizing unit that assigns the corresponding shape data to each of the effective joints based on the unique coordinate system generated by the unique coordinate system generating unit. 18. The unique coordinate system for the root joint is such that a vector from the root joint to a first joint adjacent to the root joint is a first axis of the unique coordinate system for the root joint. A vector from the second joint adjacent to the third joint to the third joint is a temporary vector, and an outer product of the temporary vector and the first axis is a second axis of an intrinsic coordinate system with respect to the root joint, 18. The image processing apparatus according to claim 17, wherein the outer product of the first axis and the second axis is determined as the third axis of the unique coordinate system for the joint of the root. 19. The proper coordinate system for the root joint has a vector from the joint of the root to a first joint adjacent thereto as a first axis of the proper coordinate system for the joint of the root. To a second joint adjacent thereto and a composite vector of the first axis or the inverse vector thereof is a temporary vector, and an outer product of the temporary vector and the first axis is a unique coordinate with respect to the root joint. 18. The image according to claim 17, wherein the second axis of the system is determined, and the outer product of the first axis and the second axis is determined as the third axis of the unique coordinate system with respect to the root joint. Processing equipment. 20. The intrinsic coordinate system for the root joint is a vector that extends from the root joint to a first joint adjacent to the root joint as a first axis of the intrinsic coordinate system for the root joint. Outer product of one of the remaining coordinate axes other than the coordinate axis determined based on the vector from the second joint to the root joint in the proper coordinate system for the second joint adjacent to Is the second axis of the proper coordinate system for the root joint, and the outer product of the first axis and the second axis is the third axis of the proper coordinate system for the root joint. The image processing device according to claim 17. 21. The joint angle displacement means applies a discrete cosine transform to a three-dimensional joint angle for each of all or some of the effective joints, and outputs a DCT coefficient. Transform coefficient displacing means for displacing a predetermined element included in each of them, and inverse discrete cosine transform for applying an inverse discrete cosine transform to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacing means The image processing apparatus according to claim 17, further comprising a conversion unit. 22. The image processing apparatus according to claim 21, wherein the predetermined element is energy of a frequency component and / or a selected order. 23. The inverse discrete cosine transform means limits the inverse discrete cosine transform applied to the DCT coefficient output by the discrete cosine transform means or the DCT coefficient displaced by the transform coefficient displacing means. The image processing device according to claim 21. 24. The unique coordinate system generation means rotates each of the three axes of the unique coordinate system with respect to the effective joint by using the three-dimensional joint angle with respect to the effective joint, so that the effective joint adjacent to the effective joint is rotated. The image processing apparatus according to claim 17, wherein a unique coordinate system for the joint is generated. 25. When an object having a plurality of effective joints serving as a reference of movements performs a predetermined movement, all or one of the effective joints is frame by frame with respect to the periodic or aperiodic movement. Basic motion storage means for storing data relating to three-dimensional joint angles for each part as basic motion data, and basic for selecting desired basic motion data from a plurality of basic motion data stored in the basic motion storage means. A motion selecting means, and a parameter setting means for setting a motion modification parameter required when changing a motion constituted by the basic motion data selected by the basic motion selecting means into a motion accompanied by a desired motion modification. , A motion constituted by the basic motion data selected by the basic motion selection means by using the motion modification parameter, the desired motion modification And a basic action modification generation means for generating basic action modification data, and an action different from the action configured by the basic action modification data and configured by the basic action modification data. Additional motion selection means for selecting a desired additional motion to be added to the basic motion modification data, additional motion generation means for generating the additional motion data by the additional motion, the basic motion modification data and the additional motion data And an action synthesizing unit that generates action data related to an action in which the action configured by the additional action data is added to the action configured by the basic action modification data. . 26. The basic motion modification generation means uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means modifies the basic motion modification means. When the basic motion data is a cyclic motion, the basic motion data is repeated for a desired period to generate the basic motion modification data. The image processing device according to claim 25. 27. The basic motion modification generation means uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means modifies the basic motion modification means. And a non-periodic motion generation means for generating the basic motion modification data from the changed basic motion data without repeating when the basic motion data is an aperiodic motion. Item 25. The image processing device. 28. The basic motion modification generation means uses the motion modification parameter to modify the basic motion data selected by the basic motion selection means, and the basic motion modification means modifies the basic motion modification means. When the basic motion data is a cyclic motion, the basic motion modification unit and the basic motion changing unit generate the basic motion modification data by repeating the modified basic motion data for a desired period of time. And a non-periodic motion generation means for generating the basic motion modification data from the changed basic motion data without repeating if the generated basic motion data is an aperiodic motion. The image processing apparatus according to claim 25. 29. The additional motion generating means includes coordinate input means for inputting coordinates that move corresponding to the additional motion selected by the additional motion selecting means, and the additional motion selected by the additional motion selecting means. 27. Coordinate tracking means for generating the additional motion data by causing a joint corresponding to a tip end of a target effective joint to follow the coordinates, 27.
27. The image processing device according to 27 or 28. 30. The additional motion generating means stores, as reference additional motion data, data regarding a three-dimensional joint angle for all or some of the effective joints for each frame for a predetermined additional motion. The reference addition operation storage means and the reference addition operation data corresponding to the addition operation selected by the addition operation selection means from the plurality of reference addition operation data stored in the reference addition operation storage means Means, an additional motion modification setting means for setting a motion modification parameter according to the additional motion selected by the additional motion selecting means, and the motion modification parameter set by the additional motion modification setting means. Additional operation for changing the reference additional operation data read by the reading means to generate the additional operation data The image processing apparatus according to claim 25, 26, 27 or 28, wherein in that a decorative generating means.