JP4539291B2 - Robot apparatus and operation control method thereof - Google Patents

Description

  The present invention relates to an autonomously operable robot apparatus and an operation control method thereof.

  A mechanical device that performs an action similar to that of a human (living body) using an electrical or magnetic action is called a “robot”. Robots have begun to spread in Japan since the late 1960s, but many of them are industrial robots such as manipulators and transfer robots for the purpose of automating and unmanned production work in factories. Met.

  Recently, practical robots that support life as a human partner, that is, support human activities in various situations in daily life such as the living environment, have been developed. Unlike industrial robots, such practical robots have the ability to learn how to adapt themselves to humans with different personalities or to various environments in various aspects of the human living environment. For example, it was designed based on the body mechanism and motion of a “pet-type” robot that imitates the body mechanism and movement of a quadruped animal such as a dog or cat, or a human who walks upright on two legs. Robotic devices such as “humanoid” or “humanoid” robots are already in practical use. Since these robot devices can perform various operations with an emphasis on entertainment performance as compared with industrial robots, they may be referred to as entertainment robots. Some of these robot apparatuses autonomously operate.

  By the way, as one of the most important issues in such an entertainment robot, there is a search for a mechanism that does not make the user bored. However, since the mechanism of the phenomenon that humans get tired of has not yet been elucidated, it is difficult to prepare a mechanism that does not completely get bored, so a method that creates many mechanisms that the user can be interested in is taken in advance .

  Here, as an example of such a mechanism, Patent Document 1 and Non-Patent Document 1 disclose a technique for causing a robot device to perform an operation synchronized with a user's voice and operation. However, since the techniques described in Patent Document 1 and Non-Patent Document 1 cannot change the tuning relationship based on a mechanism built in in advance, they are gradually bored when used for human interaction. There was a possibility. On the other hand, the present inventors have proposed a technique in Non-Patent Document 2 that realizes closer interaction between a robot apparatus and a human by repeating imitation of a user's motion and its modulation.

JP 2001-246174 A Masato Ito, Jun Tani, “On-line Imitative Interaction with a Humanoid Robot Using a Mirror Neuron Model”, IEEE International Conference on Robotics and Automation (2004, in press) Fumihide Tanaka, Hirotaka Suzuki, “Dance Interaction with QRIO: A Case Study for Non-Boring Interaction by using an Entertainment Ensemble Model”, 13th IEEE International Workshop on Robot and Human Interactive Communication (2004, in press)

  However, since the modulation method in Non-Patent Document 2 is basically based on the design of the programmer, it is suitable for long-term interaction such as predictable or completely random. It wasn't.

  According to recent basic research, it has been reported that chaotic behavior, which is not predictable or random, one of the classes existing in between, gives comfort to humans (supervised by Toshimitsu Takeshi , "Fluctuation Science 1-9", Morikita Publishing, etc.). Therefore, it is conceivable for the programmer to design the chaos and incorporate it into the movement pattern of the robot device, for example, but even if chaos is incorporated, it matches the specific physicality (physical characteristics) of the robot device. If it is not, it will not function effectively.

  The present invention has been proposed in view of such a conventional situation, and it is possible to express chaotic motion based on one's own physicality without causing the user or other interaction object to get bored. An object of the present invention is to provide a simple robot apparatus and its operation control method.

In order to achieve the above-described object, a robot apparatus according to the present invention is a robot apparatus that can operate autonomously, and the above-mentioned other units per unit time via an imaging unit that images the operation of another operation subject. Based on the data input by the motion input unit, the motion input unit that inputs the motion subject image, and the motion vector derived from the motion of the other motion subject and the motion vector derived from its own motion are mixed. Motion mixing means for generating a motion vector, block area extraction means for extracting information on a rectangular block area surrounding a portion indicating motion in the image of the other motion subject based on the generated motion vector, based on the above block area, it includes a motion generating means for generating an operation that express, an instruction input means for inputting an external instruction, the instruction from the instruction input means When given, the operation mixing means for mixing by adjusting the mixing ratio of the motion vectors derived from the operation of the motion vector and the own derived from the operation of the other main operation.

In order to achieve the above-described object, an operation control method for a robot apparatus according to the present invention is an operation control method for a robot apparatus that can operate autonomously, and is an imaging unit that images the operation of another operation subject. A motion input step for inputting the image of the other motion subject for each unit time via a motion vector, and a motion vector derived from the motion of the other motion subject based on the data input in the motion input step and itself A rectangular block region surrounding a portion indicating a motion in the image of the other motion subject based on the motion vector, and a motion mixing step for generating a motion vector mixed with the motion vector derived from the motion Bei the block area extracting step of extracting information on, based on the block area, and a motion generation step of generating an operation of expressing, an instruction input step of inputting an instruction from the outside In the above operation the mixing step, when the instruction in the instruction input step is provided, mixed with the motion vector which the robot apparatus is derived from the operation of the motion vector and the own derived from the operation of the other operation subject Adjust the ratio.

  According to the robot apparatus and the motion control method thereof according to the present invention, a motion in which a motion vector derived only from the motion of another motion subject such as a user and a motion vector derived only from its own motion are mixed in a desired ratio. Since a vector is generated and a motion is generated based on the motion vector, a chaotic motion that does not make the user more tired can be expressed based on his / her physicality.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is applied to a bipedal walking type robot apparatus having an emotion model in which emotions such as instinct and emotion are modeled. As will be described later, this robotic device is capable of expressing chaotic motion that does not make the user more bored based on his / her physicality when imitating the user's motion. In the following, first, the configuration of such a robot apparatus will be described, and then an operation control method for causing the robot apparatus to exhibit a chaotic operation will be described in detail.

(1) Configuration of Robot Device First, the configuration of the robot device in the present embodiment will be described. As shown in FIG. 1, the robot apparatus 1 according to the present embodiment includes a head unit 3 connected to a predetermined position of the trunk unit 2, two left and right arm units 4 </ b> R / L, The leg units 5R / L are connected to each other (provided that R and L are suffixes indicating right and left, respectively, and the same applies hereinafter).

  The joint degree-of-freedom configuration of the robot apparatus 1 is schematically shown in FIG. The neck joint that supports the head unit 3 has three degrees of freedom: a neck joint yaw axis 101, a neck joint pitch axis 102, and a neck joint roll axis 103.

  Each arm unit 4R / L constituting the upper limb includes a shoulder joint pitch axis 107, a shoulder joint roll axis 108, an upper arm yaw axis 109, an elbow joint pitch axis 110, a forearm yaw axis 111, and a wrist. A joint pitch axis 112, a wrist joint roll axis 113, and a hand part 114 are included. The hand portion 114 is actually a multi-joint / multi-degree-of-freedom structure including a plurality of fingers. However, since the operation of the hand 114 has little contribution or influence on the posture control or walking control of the robot apparatus 1, it is assumed in this specification that the degree of freedom is zero. Therefore, each arm unit 4R / L is configured with seven degrees of freedom.

  The trunk unit 2 has three degrees of freedom: a trunk pitch axis 104, a trunk roll axis 105, and a trunk yaw axis 106.

  Each leg unit 5R / L constituting the lower limb includes a hip joint yaw axis 115, a hip joint pitch axis 116, a hip joint roll axis 117, a knee joint pitch axis 118, an ankle joint pitch axis 119, and an ankle joint. A roll shaft 120 and a foot 121 are included. In the present specification, the intersection of the hip joint pitch axis 116 and the hip joint roll axis 117 defines the hip joint position of the robot apparatus 1. Although the foot part of the human body is actually a structure including a multi-joint / multi-degree-of-freedom sole, the foot part 121 of the robot apparatus 1 has zero degrees of freedom. Accordingly, each leg unit 5R / L is configured with six degrees of freedom.

  In summary, the robot apparatus 1 as a whole has a total of 3 + 7 × 2 + 3 + 6 × 2 = 32 degrees of freedom. However, the robot device 1 for entertainment is not necessarily limited to 32 degrees of freedom. Needless to say, the degree of freedom, that is, the number of joints, can be increased or decreased as appropriate in accordance with design / production constraints or required specifications.

  Each degree of freedom of the robot apparatus 1 as described above is actually implemented using an actuator. It is preferable that the actuator be small and light in light of demands such as eliminating the appearance of extra bulges on the appearance and approximating the shape of a human body, and performing posture control on an unstable structure such as biped walking. .

  FIG. 3 schematically shows a control system configuration of the robot apparatus 1. As shown in FIG. 3, the control system is a motion that controls the whole body cooperative motion of the robot apparatus 1 such as driving the actuator 350 and the actuator 350 and the thought control module 200 that dynamically controls emotion judgment and emotional expression in response to user input. And a control module 300.

  The thought control module 200 includes a CPU (Central Processing Unit) 211, a RAM (Random Access Memory) 212, a ROM (Read Only Memory) 213, and an external storage device (hard disk) that execute arithmetic processing related to emotion judgment and emotion expression. A drive or the like) 214 and is an independent drive type information processing apparatus capable of performing self-contained processing in a module.

  The thought control module 200 determines the current emotion and intention of the robot device 1 according to stimuli from the outside such as image data input from the image input device 251 and sound data input from the sound input device 252. Here, the image input device 251 includes, for example, one CCD (Charge Coupled Device) camera on each side, and the audio input device 252 includes, for example, a plurality of microphones. Moreover, the thought control module 200 can output a voice via a voice output device 253 provided with a speaker.

  In addition, the thought control module 200 issues a command to the motion control module 300 to execute an action or action sequence based on decision making, that is, exercise of the limbs.

  One motion control module 300 includes a CPU 311 for controlling the whole body cooperative motion of the robot apparatus 1, a RAM 312, a ROM 313, and an external storage device (hard disk drive, etc.) 314, and performs self-contained processing within the module. It is an independent drive type information processing apparatus that can be performed. The external storage device 314 can store, for example, walking patterns calculated offline, target ZMP trajectories, and other action plans. Here, the ZMP is a point on the floor where the moment due to the floor reaction force during walking is zero, and the ZMP trajectory is, for example, a trajectory in which the ZMP moves during the walking operation period of the robot apparatus 1. Means. Regarding the concept of ZMP and the point where ZMP is applied to the stability criteria for walking robots, “LEGGED LOCOMOTION ROBOTS” written by Miomir Vukobratovic (“Walking Robot and Artificial Feet” written by Ichiro Kato (Nikkan Kogyo Shimbun)) It is described in.

  The motion control module 300 includes an actuator 350 that realizes the degrees of freedom of joints distributed throughout the body of the robot apparatus 1 shown in FIG. 2, a posture sensor 351 that measures the posture and inclination of the trunk unit 2, and left and right soles Various devices such as ground check sensors 352 and 353 for detecting the leaving or landing of the vehicle and a power supply control device 354 for managing the power supply of the battery or the like are connected via a bus interface (I / F) 301. Here, the posture sensor 351 is configured by, for example, a combination of an acceleration sensor and a gyro sensor, and the grounding confirmation sensors 352 and 353 are configured by proximity sensors, micro switches, or the like.

  The thought control module 200 and the motion control module 300 are constructed on a common platform, and are interconnected via bus interfaces 201 and 301.

  The motion control module 300 controls the whole body cooperative motion by each actuator 350 in order to embody the action instructed from the thought control module 200. That is, the CPU 311 extracts an operation pattern corresponding to the action instructed from the thought control module 200 from the external storage device 314 or internally generates an operation pattern. Then, the CPU 311 sets a foot movement, a ZMP trajectory, a trunk movement, an upper limb movement, a waist horizontal position, a height, and the like according to the specified movement pattern, and commands that instruct the movement according to these setting contents. The value is transferred to each actuator 350.

  In addition, the CPU 311 detects the posture and inclination of the trunk unit 2 of the robot apparatus 1 from the output signal of the posture sensor 351, and each leg unit 5R / L is caused to move freely by the output signals of the ground contact confirmation sensors 352 and 353. Alternatively, the whole body cooperative movement of the robot apparatus 1 can be adaptively controlled by detecting whether the robot is standing or standing.

  Further, the CPU 311 controls the posture and operation of the robot apparatus 1 so that the ZMP position is always directed to the center of the ZMP stable region.

  Furthermore, the motion control module 300 returns to the thought control module 200 the level of behavior as intended as determined by the thought control module 200, that is, the processing status.

  In this way, the robot apparatus 1 can determine its own and surrounding conditions based on the control program and act autonomously.

(2) Operation Control Method of Robot Device Next, an operation control method for causing the above-described robot device 1 to exhibit a chaotic operation will be described. As described above, the robot apparatus 1 is capable of expressing a chaotic motion that does not make the user more bored based on his / her physical property when imitating the user's motion. Of the functional block configuration of the robot apparatus 1, FIG. 4 shows a portion necessary for expressing this chaotic operation.

  As shown in FIG. 4, the robot apparatus 1 includes a camera image input device 10, a motion feature amount extractor 11, a motion generator 15, a motion rule base storage device 16, and an actuator 17. The motion feature quantity extractor 11 includes a self-motion calculator 12, a motion mixer 13, and a BR extractor 14.

  The camera image input device 10 corresponds to the image input device 251 in FIG. 3, and inputs a user frame image for each unit time, and collects a difference image or a motion vector for each pixel and distance information for each pixel. Are generated for each frame. Then, the camera image input unit 10 supplies the generated difference image or the motion vector for each pixel and the distance image to the motion feature amount extractor 11 together with the frame number.

  The motion feature amount extractor 11 generates a motion feature amount for each frame from the difference image or the motion vector for each pixel and the distance image supplied from the camera image input device 10, and sends the motion feature amount to the motion generator 15. Supply. Hereinafter, the processing in each block in the motion feature quantity extractor 11 will be specifically described.

  Here, the robot apparatus 1 imitates the user's operation. However, since the CCD camera provided in the head unit 3 moves by its own operation, the camera can be operated only by its own operation even when there is no user operation. Equivalent motion is generated on the image. This is a phenomenon known as a self-motion (ego-motion) problem. When the robot apparatus 1 wants to know only a user's motion, it is necessary to obtain a motion vector from which the influence of its own motion is subtracted.

  Therefore, the self-motion calculator 12 monitors all joint angle information of the robot apparatus 1 every unit time, and based on the time series information of all joint angles, a specific point (for example, the center of gravity of the robot apparatus 1) is used as the origin. Then, the motion vector of the head unit 3 is dynamically calculated. The motion vector can be calculated according to a known method. By correcting the motion vector of the input image with the motion vector derived only from the own motion calculated in this way, for example, a motion vector derived purely from the user motion can be obtained.

  The motion mixer 13 corrects the input image based on a motion vector derived only from its own motion supplied from the self-motion calculator 12. At this time, the motion mixer 13 does not generate a motion vector derived only from the user's motion by subtracting a motion vector derived only from the motion itself from the motion vector of the input image, but derives only from the motion itself. A motion vector in which a motion vector to be generated and a motion vector derived only from the user's motion are mixed in a desired ratio is generated. Specifically, the motion vector M ′ is obtained from the motion vector M supplied from the camera image input device 10 and the image plane projection v of the motion vector supplied from the self-motion calculator 12 according to the following equation (1). calculate. Here, in Expression (1), I is a unit matrix having the same size as the matrix M, and α and β are parameters that the robot apparatus 1 sets as appropriate. A method for setting the parameters α and β will be described later.

M ′ = α × M−β × v · I (1)
In the above example, for simplicity, it is assumed that the image plane projection v of the motion vector supplied from the self-motion calculator 12 is the same in all components of the matrix. It is also possible to calculate an accurate motion vector projection for each pixel in the matrix by further adding the physical characteristics of the robot apparatus 1 (the size of the robot apparatus 1, the angle of view of the CCD camera, etc.). Thereby, for example, even when the head unit 3 rotates in a complicated manner, the motion vector of the input image can be corrected more accurately.

  As shown in FIG. 5, the BR extractor 14 extracts a rectangular region called a block region (BR) surrounding the motion region, and supplies information related to the block region to the motion generator 15. Here, the information regarding the block area specifically includes the center coordinates (nx, ny) of the block area, the vertical / horizontal value vector (sx, sy) of the block area, and the center of gravity of the robot apparatus 1 to the actual movement target. This is a vector (nx, ny, sx, sy, d) consisting of five elements with the distance d combined.

  The center coordinates and vertical / horizontal value vectors of the block area are obtained as relative positions in the frame, but are converted into actual three-dimensional positions and sizes with the center of gravity of the robot apparatus 1 as the origin by combining with the distance image, These values may be used as the vector elements described above.

The motion generator 15 maps the information regarding the block area supplied from the BR extractor 14 to its own joint angle control information. Specifically, the motion generator 15 is a motion rule base previously determined by the programmer from the motion rule base storage 16, for example, the pitch angle of the shoulder joint pitch axis = γ1 × nx.
Roll angle of shoulder joint roll axis = γ2 × sx
Pitch angle of trunk pitch axis = γ3 × d
Is obtained, and joint angle control information is obtained using this. Here, γ1, γ2, and γ3 are predetermined parameters. The motion generator 15 supplies the joint angle control information obtained in this way to the actuator 17 and drives it to express it as a motion.

  Next, a method for setting the parameters α and β in the motion mixer 13 will be described.

  When the parameters α and β in the above equation (1) are set to α = β = 1.0, the motion vector M ′ is derived from Mv · I, that is, only the user's motion. At 15, the motion is generated based solely on the user's motion. This corresponds to the robot device 1 being drawn into the user's motion and exhibiting a mimicking motion. Here, when the user's movement is regular, for example, when the user shakes his / her hand in front of the robot apparatus 1 at a constant left / right period, the center coordinate nx at the time t in the block region is set as the x coordinate, and at the time t + 1. A trajectory in which the center coordinate nx is plotted as the y coordinate is a periodic attractor as shown in FIG.

  Further, when the parameters α and β in the equation (1) are α = 1 and β = 0, the motion generator 15 generates a motion based on not only the user's motion but also its own motion. It becomes a behavior. In the same manner as described above, when the user shakes his / her hand in front of the robot apparatus 1 at a fixed period, the trajectory has chaotic properties as shown in FIG. This is because even if the user's movement is regular, it is combined with its own movement, and randomness is added in a bootstrap manner, resulting in a complicated trajectory that is difficult to predict in the long run. This can be explained as being generated based on the physicality (physical characteristics) of the robot apparatus 1.

  Furthermore, by adjusting the parameters α and β in the motion mixer 13, it is possible to freely control the influence of its own operation, thereby controlling the strength of chaoticity in the operation output. . Specifically, for example, a user who is an object of interaction is instructed by voice or a button to get tired, and when the degree of tiredness is high, a more complicated operation with strong chaos can be expressed.

  As described above, according to the robot apparatus 1 and the motion control method thereof in the present embodiment, the motion vector derived only from its own motion and the motion vector derived only from the user's motion are mixed in a desired ratio. Since a motion vector is generated and a motion is generated based on the motion vector or the like, a chaotic motion that does not make the user more tired can be expressed based on his / her physicality.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, in the above-described embodiment, it has been described that the joint angle control information is obtained based on the operation rule base read from the operation rule base storage unit 16, but the present invention is not limited to this, and may be used in the image processing field. For example, the position of the user's head and both left and right hands is obtained using basic template matching and color histogram information used, and this position information is converted into joint angle control information of the robot apparatus 1 itself using inverse kinematics. It doesn't matter.

  In the above-described embodiment, a camera image is used as input information. However, the present invention is not limited to this, and another modal such as an acoustic signal may be used.

  In the above-described embodiment, a user is assumed as a communication target. However, the present invention is not limited to this, and other moving subjects such as non-living animals such as robot devices and animals may be set as communication targets.

It is a perspective view which shows the external appearance of the robot apparatus in this Embodiment. 3 is a block diagram schematically showing a functional configuration of the robot apparatus. FIG. It is a block diagram which shows the structure of the control unit of the robot apparatus in detail. It is a figure which shows a part required in order to generate chaotic behavior among the functional block structures of the robot apparatus. It is a figure which shows the block area | region extracted with BR extractor of the robot apparatus. It is a figure which shows the trajectory in the case of producing | generating an operation | movement based only on a user's operation | movement. It is a figure which shows the trajectory in the case of producing | generating an operation | movement based on a user and own operation | movement.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Robot apparatus, 10 Camera image input device, 11 Motion feature-value extractor, 12 Self-motion calculator, 13 Motion mixer, 14 BR extractor, 15 Action generator, 16 Action rule base memory | storage device, 17 Actuator

Claims (4)

A robot device that can operate autonomously,
An operation input means for inputting the image of the other operation subject per unit time via an imaging means for imaging the operation of the other operation subject;
Based on the data input by the motion input means, motion mixing means for generating a motion vector in which a motion vector derived from the motion of the other motion subject and a motion vector derived from its own motion are mixed;
Block area extraction means for extracting information on a rectangular block area surrounding a portion indicating the movement in the image of the other operation subject based on the generated motion vector;
Based on the block region, action generating means for generating an action to be expressed,
An instruction input means for inputting an instruction from the outside,
When an instruction is given from the instruction input means , the motion mixing means adjusts and mixes the mixing ratio of the motion vector derived from the motion of the other motion subject and the motion vector derived from the own motion. Yes Robotic device. The block area extracting means outputs said other operation the block center coordinates and vertical and horizontal value vector region of the subject, as well as information about the block area including the elements of the distance to the other operation subject,
The robot apparatus according to claim 1, wherein the motion generation unit generates a motion to be expressed based on information related to the block region from the block region extraction unit. An operation control method for an autonomously operable robotic device,
An operation input step of inputting the image of the other operation subject per unit time via an imaging means for imaging the operation of the other operation subject;
Based on the data input in the motion input step, a motion mixing step of generating a motion vector in which a motion vector derived from the motion of the other motion subject and a motion vector derived from its own motion are mixed,
A block region extraction step for extracting information on a rectangular block region surrounding a portion indicating the motion in the image of the other operation subject based on the generated motion vector;
Based on the block area, an action generation step for generating an action to be expressed,
An instruction input process for inputting instructions from the outside,
In the motion mixing step, when an instruction in the instruction input step is given, the robot apparatus mixes a motion vector derived from the motion of the other motion subject and a motion vector derived from the motion of the robot device. A method for controlling the operation of a robot apparatus. In the block area extracting step, information on a block area including a center coordinate and a vertical / horizontal value vector of a rectangular block area surrounding the motion area of the other action subject, and an element of a distance to the other action subject is output. ,
The robot apparatus motion control method according to claim 3, wherein in the motion generation step, a motion to be expressed is generated based on information on the block region from the block region extraction step.

Images