JP4997419B2 - Robot motion conversion system - Google Patents

Description

  The present invention relates to a robot motion conversion system, and in particular, for example, for a novel robot capable of converting a human expression motion into a robot motion and thereby causing the robot to perform a similar motion to the human motion. The present invention relates to a motion conversion system.

This type of expression motion conversion system such as a motion capture system mainly converts human motions (gestures) into 3D CG character motions as shown in Non-Patent Documents 1-3. It was a thing.
http://www.vicon.com/products/bodybuilder.html http://www.vicon.com/products/polygon.html http://www.crescentvideo.co.jp/vicon/mx/software/index.shtml

  Although many degrees of freedom are required to express human motions, the 3D CG (Computer Graphics) character as described above can be freely transformed, so that the 3D CG character can be used to capture human motions obtained by motion capture. There is no great difficulty in converting to the operation. However, it is impossible to accurately trace human movements with a robot having only a small degree of freedom, and there has conventionally been no useful motion conversion system for such robots.

  Therefore, a main object of the present invention is to provide a novel robot motion conversion system.

  Another object of the present invention is to provide a robot motion conversion system that enables a robot to reproduce a human motion as accurately as possible.

The invention according to claim 1 is a robot motion conversion system that converts a motion of a target object, which is a part expressing a human motion, into a control target motion of the robot corresponding to the target motion. Position data input means for inputting the position data of each part involved, one part among the human head, left hand, right hand and torso as the target coordinate system, and the remaining one part as the reference coordinate system Based on this, the human motion description means describing the time change of the position and orientation in the reference coordinate system in the reference coordinate system as a trajectory , and the trajectory of the human attention target described by the human motion description means as the trajectory of the robot control target. A robot motion conversion system including approximation means for approximation.

  In the invention of claim 1, for example, position data input means such as a motion capture (102) inputs position data of each part associated with human movement. The human motion description means (S5) composed of the motion conversion computer (104) of the embodiment converts the trajectory (Hp, Hr) of the target of human attention to the head, left hand, right hand and It is described by the relative positional relationship between two of the fuselage. Similarly, the approximating means (S13, S17) constituted by the motion conversion computer (104) of the embodiment takes the trajectory of the attention target of the human as two of the head, left hand, right hand, and torso of the robot's attention target. Approximate the trajectory described in relative positional relationship. Therefore, if the robot is controlled based on the approximated trajectory, the robot can reproduce the human motion.

  According to the first aspect of the present invention, it is possible to cause a robot having a much lower degree of freedom than that of a human to perform an operation similar to that of the human by simply inputting position data associated with the human operation.

The invention of claim 2 includes normalizing means for normalizing the locus described by the human action description means by individually normalizing the position and orientation, and the approximating means is a human being normalized by the normalizing means. The robot motion conversion system according to claim 1, wherein the target object trajectory is approximated to a robot control target trajectory.

  According to the invention of claim 2, in the embodiment, the normalization means (S7) comprising the motion conversion computer (104) normalizes the human motion trajectory, thereby eliminating the size difference between the human and the robot.

According to a third aspect of the present invention, the robot has a plurality of joints for controlling the position and orientation of the controlled object, and further sets control data for the plurality of joints based on the locus of the target of interest of the robot approximated by the approximating means. The robot motion conversion system according to claim 2, further comprising data setting means .
The invention according to claim 4 further comprises a judging means for judging whether or not an approximated trajectory can be realized within the joint angle range of the robot, and the approximating means is a target of human attention when the judging means makes a negative judgment. 4. The robot motion conversion system according to claim 3, wherein a joint angle for the locus of the control target that is maximally similar to the locus of interest is obtained by principal component analysis of the locus.

In the invention 請 Motomeko 4, since approximating the trajectory using only data required by the principal component analysis, with minimal similarity, it is possible to approximate the operation of the robot to the human operation.

The invention of claim 5, further comprising a control data setting means for setting the control data of the robot based on the trajectory of the object of interest of the robot approximated by approximating means, a motion conversion system for robot according to claim 1.

In the invention of claim 5 , for example, the control data setting means (S27) such as the motion conversion computer (104) of the embodiment sets control data such as the joint angle of the robot based on the obtained approximate locus. To do.

  According to the present invention, it is possible to automatically convert a human expression operation having many degrees of freedom into a robot expression operation having only a few degrees of freedom. Therefore, by simply inputting a human motion by, for example, motion capture, it is possible to reproduce the robot motion approximate to the human motion.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  A motion conversion system 100 according to an embodiment of the present invention shown in FIG. 1 is a system for causing a robot 10 to reproduce a human (not shown) motion detected by a motion capture 102 by a motion conversion computer 104. That is, the motion conversion computer 104 converts the control data of each axis of the robot 10 to reproduce the motion based on the three-dimensional data of each position related to the human motion acquired by the motion capture 102.

  However, the robot database 106 shown in FIG. 1 sets or registers in advance the types of robots to which the motion conversion system 100 of the embodiment can be applied and the axis arrangement of each robot. Note that the shaft arrangement may be registered in the form of a skeleton diagram as shown in FIG.

  In this embodiment, human motion is converted into motion of a humanoid robot (hereinafter simply referred to as “robot”) 10 having an appearance as shown in FIG. 2 as an example. The robot 10 will be described.

  FIG. 3 is a skeleton diagram representing the robot 10 as a model. Referring to FIG. 3, the robot 10 of this embodiment includes two recumbent arms 12 and 14 that are spaced apart in the vertical direction or the longitudinal direction, and between the upper and lower recumbent surfaces 12 and 14, 16 is formed.

  One yaw shaft joint 18 that forms the degree of freedom of the chest is provided on the upper portion of the body 16, and one pitch shaft joint 20 that forms the degree of freedom of the waist is provided below the body 16. . As is well known, the yaw axis is a vertical rotation axis (Z axis), the pitch axis is one of two axes perpendicular to the yaw axis, for example, the X axis, and will be described later. The roll axis is the other of the two axes perpendicular to the yaw axis, for example, the Y axis. The X axis (pitch axis) is a rotation axis extending in a direction parallel to the paper surface of FIG. 2, and the Y axis (roll axis) is a rotation axis extending in a direction perpendicular to the paper surface of FIG.

  Arms 22 </ b> R and 22 </ b> L are attached to both ends of the upper horizontal arm 12, respectively. These left and right arms 22R and 22L have the same configuration. When it is necessary to distinguish between right and left, “R” indicating right or “L” representing left is added, and when it is not necessary to distinguish, “R” or “L” is not added. In this way, duplicate descriptions are omitted as much as possible. The same applies to the feet described later.

The arm 22 includes a roll shaft joint 24 that forms a degree of freedom of the shoulder, and is attached to the upper lateral reed 12 by the shoulder joint 24 so as to be rotatable around the roll shaft. In front of the shoulder joint 24, one pitch shaft joint 26, one yaw shaft joint 28, and one roll shaft joint 30 are connected in series. Therefore, in the robot 10 of this embodiment, the arm 22 has 4 degrees of freedom and 8 degrees of freedom on the left and right.

  In addition, legs 32R and 32L are attached to both ends of the lower lying heel 14, respectively. The foot 32 is attached to the lower recumbent joint 14 by one roll shaft joint 34 that forms a degree of freedom of the hip joint. In the lower end direction from the hip joint 32, three pitch shaft joints 36, 38 and 40, which are continuous, And a series connection of one roll shaft joint 42 and one yaw shaft joint 44 is attached. Therefore, in the case of the robot 10 of this embodiment, the foot 32 has 6 degrees of freedom and 12 degrees of freedom on the left and right.

  The electrical configuration of the robot 10 is shown in the block diagram of FIG. As shown in FIG. 4, the robot 10 includes a microcomputer or CPU (robot computer) 48 for overall control. The robot computer 48 is connected to a memory 52, a motor control board 54, and a bus 50 through a bus 50. A sensor input / output board 56 is connected.

  Although not shown, the memory 52 includes a ROM and a RAM, in which a control program and data for the robot 10 are written in advance. The RAM can be used as a temporary storage memory and a working memory.

  To the sensor input / output board 56, one biaxial acceleration sensor 58 provided on the body 16 of FIG. 3 and a joint angle sensor 60 are connected. The joint angle sensor 60 is shown as one block, but actually includes 22 joint angle sensors that are individually provided for the 22 joints described above and detect the respective joint angles. The biaxial acceleration sensor 58 can detect acceleration with two axes in the front-rear direction and the left-right direction of the robot 10. Using the output of the biaxial acceleration sensor 58, the robot computer 48 controls the bipedal walking, reaction rise, and the like of the robot 10.

The motor control board 54 is composed of, for example, a DSP (Digital Signal Processor),
Each arm, each leg, chest and waist joint shaft motor 62 is controlled. In FIG. 4, the joint shaft motor 62 is shown as one block for simplification of the drawing, but in reality, one motor (servo motor) is provided for each of the 22 joints described above. It includes 22 motors that control the joint angle.

  The robot 10 to which the motion conversion system 100 of FIG. 1 embodiment can be applied has a considerably large number of 22 degrees of freedom as described above, but it still does not reach the degree of freedom that humans have. The robot 10 cannot imitate human movements as they are. On the other hand, the robot to which the present invention can be applied is not limited to the robot 10 described above, and there is a robot with a lower degree of freedom among such robots. Therefore, in this embodiment, the human gesture (expression operation) is described by the relationship of the four parts of the head, the left hand, the right hand, and the torso, so that it can be applied to a robot with less flexibility.

  The fact that most human movements (gestures) can be described only by the relationship between the four parts of the head, left hand, right hand, and torso is a finding that the inventors have acquired by analyzing, for example, a gesture dictionary. The present invention has been made based on such knowledge.

  In the first step S1 of the flowchart of FIG. 5, the behavior conversion computer 104 (FIG. 1) displays the first setting screen 108 shown in FIG. This setting screen 108 is a GUI (Graphical User Interface) for allowing the user of the computer 104 to perform input and selection operations, similarly to the second setting screen 110 shown in FIG.

  The first setting screen 108 is a GUI for setting a human motion that the user desires to convert to a robot motion and two coordinate systems describing the motion. For this purpose, the setting screen 108 is provided with an operation input unit 112, a reference coordinate system input unit 114, and a target coordinate system input unit 116. The operation input unit 112 can selectively input an operation to be converted, for example, “waving hand”, “looking right”, “looking left”, “nodding”, in a so-called pull-down menu format. The attention coordinate system indicates which one of the four coordinate systems of the head, the left hand, the right hand, and the trunk is to be expressed in order to express the motion. The coordinate system of the target or control target) is selected. However, it is not possible to specify the position in the target coordinate system alone, it can be described only in the relative relationship with the position in the reference coordinate system. This is called a reference coordinate system. However, the two coordinate systems are both orthogonal coordinate systems having three axes X, Y, and Z in this embodiment. In the embodiment, as described above, since the human motion is simplified so as to describe the relationship between the head, left hand, right hand, and torso, these two coordinate system input units 114 and 116 are both Any one of the four coordinate systems of the head, the left hand, the right hand, and the torso is selectively input.

For example, if the action of “waving hand” is selected, the attention object or the control object is the right hand or the left hand. Therefore, although “right hand” or “left hand” is set as the attention coordinate system, since the coordinate system that is the reference for it is only “body”, “body” is selected as the reference coordinate system. For example, when the operation “nodding” is selected, the coordinate system of interest (control) is naturally “head”, and the reference coordinate system for it is “body”.

  Then, in step S3, it is determined whether or not all the movements and coordinate systems have been selected or set. If “NO”, the process waits for input, and if “YES”, the process proceeds to next step S5.

  In step S5, the human motion set by the motion input unit 112 of the setting screen 108 is converted into temporal changes in the position Hp and the direction Hr of the target coordinate system in the reference coordinate system. Therefore, the locus of the target region of human attention can be expressed by the position Hp (t) and the direction Hr (t), and this step S5 functions as a human action description means.

  If the reference coordinate system and the target coordinate system are set as shown in FIG. 7, the position Hp is the position of the origin of the target coordinate system viewed from the origin of the reference coordinate system, and the three axes X, The position xt, yt, zt for each time t (t = 1, 2,..., N) of Y and Z is expressed as shown in Equation 1. Further, the direction Hr is described as shown in Equation 1, with three angles φt, θt, and γt indicating how the direction of the reference coordinate system is changed to become the direction of the target coordinate system for each time t. Is done. However, as an example, the angle φ is, for example, an angle with respect to the vertical rotation axis (Z axis), the angle θ is an angle with respect to one of the two axes (X axis) perpendicular to the vertical axis, and the angle γ is It is an angle with respect to the other axis (Y axis).

  Both the position Hp and the direction Hr in Equation 1 are time series of three-dimensional vectors. In step S 7, the position Hp and the direction Hr are individually normalized to “1” as shown in Equation 2 in order to match the human size and the robot size. That is, step S7 functions as normalization means.

  dp0 represents the maximum movement amount. However, max (xt) in Equation 2 means that it is the maximum value of x among the positions for each time up to t1-tn. Similarly, min (xt) is t1-tn. It means that it is the minimum value of x in the position for each time until. The same applies to the y value and the z value. The reason why the maximum angle dr0 = π is set is that the human joint does not bend by 180 ° or more, so that it falls within that range. Furthermore, Ml is the maximum distance between the origins of the human reference coordinate system and the coordinate system of interest, so to speak, the range that the hand can reach when the human reaches the maximum (that is, the length of the hand). Represent.

  The normalization method may be a method using standard deviation, for example, in addition to Equation 2.

  Subsequently, in step S9, the second setting screen 110 shown in FIG. 8 is displayed. This setting screen 110 is a GUI for setting or inputting a robot to simulate human motion, a reference coordinate system to be set in the robot, and a target coordinate system. For this purpose, a robot input unit 118 and a skeleton display unit 120 are formed on the setting screen 110. The robot input unit 118 can selectively input the robot to which the conversion operation is applied, for example, in the form of a pull-down menu, for example, from “Roboby M” (product name), “Roboby IV” (product name),. . In the robot database 106 of FIG. 1 described above, the axis arrangement, that is, the skeleton is registered for each type (type) of such robot, so that the skeleton display unit 120 reads out from the robot database 106. Display a skeleton diagram. Then, the user or operator sets the attention coordinate system and the reference coordinate system for the robot to be simulated by indicating the position on the skeleton displayed there with a pointing device such as a mouse or a touch panel. To do.

  In step S11, two coordinate systems on these robots are set and wait until the routine. If they are set, "YES" is determined and the process proceeds to the next step S13.

  In step S13, the normalized human motion, that is, the trajectory He (= Hpe, Hre) of the human attention coordinate system is converted into the robot motion, that is, the trajectory Re ( = Rpe, Rre) (Rp is position, Rr is direction). That is, step S13 functions as an approximation means.

  In Equation 3, Rpt is the position of the attention coordinate system for each time, Rpt * is the center of gravity of Rpt, and f and g are the positions of the robot's attention coordinate system based on the joint angle of the robot (relative to the reference coordinate system). This is a function to convert to (coordinates), and is derived from the robot axis arrangement. In this way, the human motion, ie, the trajectory He of interest, is approximated to the attention of the robot, ie, the trajectory Re.

In step S15, when α is a positive real number that is sufficiently small, joint angles (φ _1t , φ _2t ,... Φ _nt ) (t = 1, 2, ... tn) is determined.

  The determination of “YES” in step S15 means that the approximation of the human motion He to the robot motion Re in step S13 was correct. However, “NO” in step S15 means that the solution of Equation 4 does not exist. Therefore, in this case, it is necessary to determine the joint angle of the robot by another method.

In step S17, the human motion trajectory He is subjected to principal component analysis.
(A) First, let A be the covariance matrix of He and correspond to the eigenvalues (λi (i = 1,... 6), | λ1 |> | λ2 |>...> | λ6 |) of the covariance matrix A. The eigenvector (ψi (i = 1,... 6)) is obtained.
(B) Next, in all orders Ni (1 ≦ Ni ≦ 6), an angle (φ _{1t, Ni} , φ _{2t, Ni} ,... Φ _{nt, Ni} ) that minimizes Formula 5 and a minimum value εNi are obtained. .

(C) Then, the maximum order Ni satisfying Equation 6 is selected, and the angles (φ _{1t, Ni} , φ _{2t, Ni} ,..., Φ _{nt, Ni} ) are set as angles when converted into robot motion. That is, the joint angle for the robot motion Re at the order N that is maximally similar to the human motion He is obtained.

  In this way, when the approximation cannot be performed well in step S13, in step S17, the joint for the robot motion Re that is similar as much as possible is left through the principal component analysis of the human motion He. Determine the angle. Therefore, this step S17 can also be called an approximation means.

Whether the approximation is possible in step S13 or the approximation as possible in step S17, whether or not the approximated locus Re is in the work area of the robot representing the joint movable range, that is, according to the approximation. It is checked whether the robot can operate at the joint angle φ. To that end, the computer 104 may be simulated by substituting the joint angle φ calculated by reading the axis arrangement of the robot from the robot database 106. In step S21, the simulation process is displayed as an animation using a CG (Computer Graphics) robot, and if there is a place beyond the movable range, for example, highlighting is displayed to alert the user or operator.

  Then, in step S23, an appropriate adjustment GUI (not shown) is displayed, and if there is a problem, for example, if there is a highlight display in step S21, it is possible to make partial adjustment using the GUI. Good. For example, the highlight display is turned off by dragging the problematic part of the CG robot.

  However, if the partial adjustment does not fit, “NO” is determined in step S25, and the process returns to step S9, for example, to reselect the attention coordinate system and / or the reference coordinate system. Steps S9 to S23 are repeated. For example, if it was a human's action of shaking the right hand, but it could not be approximated as a robot's right hand movement, it could be approximated by changing the robot's control target (target object) to the left hand There may be.

  After step S25 is completed, in step S27, the joint angle φ obtained in step S13 or S17 functioning as an approximation means and adjusted in step S23 as necessary is set as robot control data. Human motion can be reproduced at a specific part (attention target or control target).

  Specifically, in order to reproduce the operation of “waving the left hand” with the robot 10 shown in FIG. 2, the target of attention is the palm of the left hand, and the pitch axis joint 26L, the yaw axis joint 28L, and the roll axis shown in FIG. What is necessary is just to obtain | require each joint angle of the joint 30L by step S13 or step S17 as joint angle (phi) for controlling the palm of the left hand.

FIG. 1 is a block diagram showing the configuration of a behavior conversion system according to an embodiment of the present invention. FIG. 2 is an external view showing an example of a robot applicable to this embodiment. FIG. 3 is an illustrative view showing an axis arrangement (skeleton) of the robot shown in FIG. FIG. 4 is a block diagram showing an electrical configuration of the robot shown in FIG. FIG. 5 is a flowchart showing the operation of this embodiment. FIG. 6 is an illustrative view showing one example of a first setting screen. FIG. 7 is an illustrative view illustrating the relationship between the reference coordinate system and the target coordinate system, and Hp and Hr. FIG. 8 is an illustrative view showing one example of a second setting screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Robot 100 ... Motion conversion system 102 ... Motion capture 104 ... Motion conversion computer 106 ... Robot database 108 ... First setting screen 110 ... Second setting screen

Claims (5)

A robot motion conversion system that converts a motion of a target object that represents a human motion into a control target motion of a robot corresponding to the target target,
Position data input means for inputting position data of each part accompanying human movement,
Based on the position data, one part of the human head, left hand, right hand, and torso is used as a target coordinate system, and the remaining one part is used as a reference coordinate system in the reference coordinate system in the reference coordinate system. A robot comprising human motion description means for describing a temporal change in position and orientation as a trajectory , and approximating means for approximating the trajectory of the target of attention described by the human motion description means to the trajectory of the control target of the robot Motion conversion system. Normalizing means for normalizing the locus described by the human motion description means by normalizing the position and the orientation individually ;
The robot motion conversion system according to claim 1, wherein the approximating unit approximates the trajectory of the target of human attention normalized by the normalizing unit to the trajectory of the control target of the robot. The robot has a plurality of joints for controlling the position and orientation of the controlled object , and
The robot motion conversion system according to claim 2, further comprising control data setting means for setting control data for the plurality of joints based on the locus of attention of the robot approximated by the approximating means . A judgment means for judging whether or not the approximate locus can be realized within a joint angle range of the robot;
The approximating means performs a principal component analysis of the trajectory of the human attention object when the determination means makes a negative determination, thereby obtaining a control object trajectory that is maximally similar to the trajectory of the attention object. The robot motion conversion system according to claim 3, wherein the joint angle is obtained . Further comprising motion conversion system for robot according to claim 1, wherein the control data setting means for the setting control data for the robot based on the trajectory of the object of interest of the robot approximated by said approximating means.

Images