JP4774964B2 - Robot equipment - Google Patents

Description

  The present invention relates to a robot apparatus having a plurality of joint degrees of freedom, and more particularly to a robot apparatus having two arms together with movable legs.

  More particularly, the present invention relates to a robot apparatus that grips and transports an object using two arms, and in particular, grips using two arms while considering the position and posture of an object to be moved or transported. The present invention relates to a robot apparatus.

  Conventional robots, as represented by a six-axis manipulator, have been mainly used for agency and support for various human tasks in industrial activities and production activities. The robot in this case is premised on operating in a structured environment such as a factory, and basically does not perform complex physical interaction with people or the environment.

  On the other hand, recently, the use of the partner type, that is, “symbiosis” with human beings or “entertainment” has been attracting attention. This type of robot is expected to have not only the environment and things but also advanced physical interaction capabilities such as contact with people. Partner-type robots are not fixed to a predetermined place like a 6-axis manipulator, but are generally so-called mobile robots that autonomously move to provide various services. A representative example of a partner robot is a humanoid. For example, there has been proposed a robot apparatus that includes a moving means including movable legs and can generate a stable motion pattern that transitions between grounding and non-grounding in real time (for example, see Patent Document 1). ).

  Humanoids are capable of various physical interactions with people and the environment using their limbs. For example, a moving operation such as walking and running can be performed by alternately repeating a single-leg support period and a both-leg support period using the left and right movable legs. Leg-based movement is unstable compared to crawler robots and makes posture control and walking control difficult, but is superior in that it can realize flexible walking and running operations such as climbing stairs and climbing obstacles. Yes. Recently, research and development on the structure of legged mobile robots and their stable walking control has progressed, and expectations for practical application are also increasing.

  In addition, the humanoid can hold an object using a hand or an arm, and can carry the object in combination with a legged operation.

  An object can be grasped using a plurality of fingers provided at the hand of a single-arm manipulator. For example, there are two or more forefinger, middle finger, ring finger, little finger that have joints and are arranged in parallel, a thumb that is placed so as to face these fingers, and a joint of two or more fingers except the thumb It consists of a flexible member pulled to bend, and the power generated from one drive source is transmitted to two or more flexible members via one or more moving pulleys, and each of the flexible members is transmitted. A multi-fingered robot hand configured to generate a gripping force on two or more fingers by towing the gripper performs a gripping operation by bringing two or more fingers and thumbs into contact with a gripping object including objects of different diameters. (For example, see Patent Document 2).

  On the other hand, if the two arms of the robot are used, it is possible to grip and carry a long and heavy object with an excessive torque applied to the wrist joint portion with a single arm.

  For example, proposals have been made for robots that perform dual-arm coordination (see, for example, Patent Document 3). In this dual-arm robot, teaching point data is taught to the master arm M, a passing point is determined from the data by interpolation calculation, and passing point data to be moved by the master arm is transmitted to the slave arm S. Based on the data and data representing the relative position / posture relationship of both arms, the slave arm S is determined to move, and each arm is moved to the point to be moved, and interpolation calculation is performed. By repeating the steps of determining the point to which the master arm M should move, transmitting the data at that point, calculating the point to which the slave arm S should move, and moving both arms, both arms perform the target point coordinated operation. Subsequently, operations such as gripping and transporting articles are performed.

  In addition, if the left and right arm tips are used to grip the vicinity of both ends of the object, the gripped state does not need to be so rigid, and the structure of the hand can be simplified. For example, there has been proposed a walking robot that wears a fingertip portion of a hand and carries an article in cooperation with a human while walking on two legs (see, for example, Patent Document 4).

  However, many of the double-arm robots that use left and right arms to carry an object carry many objects near both ends on the palms of the left and right hands. In such a case, when the object is lowered at the destination, the installation surface when placing the object and the contact surface gripped by the palm are the same. This causes a problem of dropping the object by the height of. In addition, when grasping an object, it is necessary to presuppose that there is a gap for inserting a fingertip on the bottom surface of the object in advance.

  Moreover, the degree of freedom of joints of the robot is more limited than that of a human skeleton. There is no problem in moving and doing other work at the same height from the first grasp of the object, but it has enough joint freedom to lift an object placed on the floor and install it in a high place It's hard to say.

JP 2004-167676 A JP 2003-145474 A JP-A-6-71580 JP 2005-131718 A

  An object of the present invention is to provide an excellent robot apparatus having a double arm with a movable leg.

  A further object of the present invention is to provide an excellent robot apparatus capable of suitably holding and transporting a long and heavy object by using two arms cooperatively.

  A further object of the present invention is to provide an excellent robot apparatus that can be suitably gripped by using two arms while taking into consideration the position and posture of an object to be moved or transported.

  The present invention has been made in consideration of the above-described problems, and includes a robot including at least a body, left and right double arms connected to the body, and a control unit that controls joint driving of each unit including the double arms. The double arms include joint degrees of freedom around the roll, pitch, and yaw axes of the shoulder, joint degrees of freedom around the pitch axis of the elbow, and around the wrist roll and yaw axes. A robot apparatus comprising a state-of-the-art pitch axis joint having a degree of freedom of joint and a degree of freedom around a pitch axis provided in a portion close to the hand.

  A humanoid, which is a typical example of a partner type robot, includes a moving means and arms composed of movable legs, and an application in which an object is gripped and transported to a desired installation location can be considered. In particular, if a robot has two arms, a single arm can hold and carry a long and heavy object that causes excessive torque to be applied to the wrist joint. However, many of the conventional double-arm robots carry many objects near both ends of the palms of the left and right hands. In such a case, when the object is lowered at the destination, the installation surface when placing the object and the contact surface gripped by the palm are the same, so in the work of releasing the object being gripped, This is a problem because the object is dropped by the height of the finger.

  On the other hand, the robot apparatus according to the present invention can hold the object by holding the left and right opposing side surfaces of the object with the two arms, so that it can be easily lifted from the floor surface and the installation surface is opened. Therefore, it is easy to release the gripping state of the object at the installation location (that is, easy to place).

  In order to do this, the robot device recognizes the object to be grasped from the captured image of a stereo camera or the like, and the pitch axes of the most advanced pitch axis joints of the two arms are almost in a straight line. And, the center of gravity of the object to be grasped may be placed on the straight line, and the two arms may be grasped by gradually narrowing the width of the object to be grasped and sandwiched between the hands.

  For example, when the most advanced pitch axis is provided in the palm, in response to an object position / posture change request while holding an object with two arms, the other joints of the arm part are fixed, Only the most advanced pitch axis can be driven to change the posture of the grasped object. In other words, if the object is gripped so that the left and right palm pitch axes are on a straight line, the posture changes as if the object has a rotation axis only by the rotational drive of the palm pitch axis. The posture of the grasped object can be changed without controlling the posture of the other six axes. Also, by providing the most advanced pitch axis on the inner side of the palm as an end effector, the center of gravity position of the gripping object and the rotation axis can be easily matched and stable gripping becomes possible.

  Further, when the required position / posture change of the gripping object is performed, it is assumed that the movable range of the most advanced pitch axis is out of the range or self-interference occurs. Such a case is dealt with by finding an inverse kinematics solution for the other joints of the arm to obtain the required position and orientation of the grasped object while keeping the most advanced pitch axis at the current position. be able to.

  Or, instead of driving only the most advanced pitch axis, the other joints of the arm can be freely used with the most advanced pitch axis as a redundant axis according to the movement demand of the object while holding the object with two arms. It is also possible to perform a double-arm gripping operation that moves while maintaining the posture of the gripping object using the degree.

  In such a case, it is preferable to determine the position of the redundant axis so that the margin of the inverse kinematics solution of each joint other than the redundant axis of the arm portion is maximized. As a result, it is possible to realize a double-arm operation in which it is difficult for the inverse kinematics calculation to be solved during the transportation and movement of the object (that is, the operation is difficult to fail).

  As described above, the robot apparatus according to the present invention changes the posture of the gripping object by driving only the leading-edge pitch axis of the two arms, or uses the leading-edge pitch axis as a redundant axis to change the posture of the object. It can be transported and moved. Depending on the height or depth direction of the installation location of the object, it may be difficult to place the object on the ground contact surface immediately before gripping, but the rotation of the leading-edge pitch axis to make the installation surface possible The gripping state can be released by appropriately changing the posture of the gripping object by driving.

  In addition, since the robot apparatus can be gripped with the double arms in various ways using the state-of-the-art pitch axis, when the object on the floor is gripped with the double arms, the installation location of the object is high. Alternatively, the object may be gripped after the most advanced pitch axis is rotationally driven so that the posture of the hand is easy to obtain an installation surface determined according to the direction of the depth or depth.

  In addition, the robot apparatus according to the present invention can further include a fall detection unit that detects the fall of the object being gripped using the two arms.

  For example, it is possible to detect the fall of the grasped object based on on / off of a touch sensor provided on the palm of the double arm.

  Also, a 6-axis force sensor is provided on the wrist of the double arm, the output value of the sensor immediately after gripping the object is stored, and the fall of the gripped object is detected when the sensor output value is lower than immediately after gripping. be able to. When the value in the Y-axis direction of the 6-axis sensor at the wrist is below the threshold value, it is checked whether a large force has been applied in the X or Z direction immediately before the Y-axis value further decreases. Assume that the gripping object has fallen downward when not applied, and determine that the gripping object may have been forcibly dropped if a large force is applied in a direction different from the direction in which the palm grips the object Can do.

  In addition, the distance between both palms is obtained from the joint angle calculated from the potentiometer of the two arms, and the fall of the grasped object is detected when the palm distance becomes smaller than the width obtained from the model information of the grasped object. be able to.

  In addition, when the object position recognized by the captured image of the stereo camera greatly deviates from the palms of the two arms, or when observation becomes impossible, it is possible to detect the fall of the grasped object.

  In addition, in the case of a robot apparatus that includes movable legs and uses ZMP as a standard for posture stability control, a sole sensor that detects a load received from the floor surface on the ground surface is attached. The weight obtained from the sole sensor is held immediately after gripping the object, and the fall of the gripped object can be detected when the value calculated from the weight measured during gripping falls below a certain value. .

  Also, when the robot apparatus has a plurality of microphones whose directivities are adjusted for the purpose of voice recognition or the like, if a loud sound is detected from below the robot arm, it is recognized that there is a possibility of falling. be able to.

  Further, the robot apparatus may activate a predetermined coping process such as searching for the dropped object on the floor surface in response to the detection of the fall of the gripped object.

  Also, the robot apparatus can confirm the cancellation of the double-arm gripping state using the drop detection means as described above when the gripping object is placed at a desired installation location.

  In addition, the robot apparatus further includes the degree of freedom of the trunk and movable legs, and the object can be transported and moved by the whole body cooperative operation including the two arms. In this case, first, an inverse kinematics operation is performed using only two arms to search for a solution that can obtain the position and orientation of a desired object. Then, when a solution using only two arms is not found, a solution that can obtain the position and posture of a desired object by performing inverse kinematics calculation by adding the degree of freedom of the trunk to the two arms. Search for. Also in this case, when a solution is not found, a solution that can obtain the position and posture of a desired object is obtained by performing an inverse kinematics calculation by adding a degree of freedom to the legs.

  According to such a processing procedure, when the position and orientation of the next object can be obtained using only two arms, a solution is obtained using only two arms, and the number of degrees of freedom to be used is suppressed as much as possible. A desired motion such as stacking objects using arms can be realized at a lower cost.

  ADVANTAGE OF THE INVENTION According to this invention, the outstanding robot apparatus which can hold | grip and convey a long and heavy object suitably by using a double arm cooperatively can be provided.

  In addition, according to the present invention, it is possible to provide an excellent robot apparatus that can be suitably gripped using two arms while considering the position and posture of an object to be moved or transported.

  In addition, according to the present invention, there is provided an excellent robot apparatus that can hold and carry a long and heavy object such that an excessive torque is applied to the wrist joint portion with a single arm using a double arm. be able to.

  Further, according to the present invention, the degree of freedom of the height of the double arms, the trunk and the waist can be obtained by holding the object using the two arms and releasing the object while appropriately changing the installation surface of the object before the holding. It is possible to provide an excellent robot apparatus capable of transporting and moving an object over a wide range regardless of the limitation.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Robot device configuration:
FIG. 1 shows an external configuration of a robot apparatus according to an embodiment of the present invention. The robot apparatus shown in the figure is a so-called humanoid robot, and the upper body is connected to the pelvis through two legs as a moving means and a hip joint. The head is connected to the upper body through two limbs and a neck joint.

  The left and right legs have a total of 6 degrees of freedom, 3 degrees of freedom for the hip joint, 1 degree of freedom for the knee joint, and 2 degrees of freedom for the ankle joint. The left and right arms have a total of 7 degrees of freedom: 3 degrees of freedom for the shoulder joint, 1 degree of freedom for the elbow joint, 2 degrees of freedom for the wrist joint and 1 degree of freedom for the palm joint (or 3 degrees of freedom for the wrist joint). ing. Both the neck joint and the waist joint are assumed to have three degrees of freedom around the X, Y, and Z axes. The finger attached to the hand also has a degree of freedom of joint, but since it is not directly related to the gist of the present invention, the description is omitted below.

  FIG. 2 shows a configuration example of a connection topology for connecting the joint driving actuators of each part in the robot apparatus shown in FIG. FIG. 3 shows a joint degree of freedom model of the robot apparatus shown in FIG.

  The robot apparatus has three-axis waist joint actuators a1, a2, and a3 and three-axis neck joint actuators a16, a17, and a18, and these are serially connected to the control unit. Each joint actuator receives the position control target value via a serial cable and transmits the current output torque, joint angle, and joint angular velocity to the control unit.

  In addition, the robot apparatus has three-axis shoulder joint actuators a4, a5, a6, one-axis elbow joint actuator a7, two-axis wrist joint actuators a8, a9, and one-axis palm joint actuator a31 on the left arm. These are serially connected to the control unit. Similarly, the right arm portion has three-axis shoulder joint actuators a10, a11, a12, one-axis elbow joint actuator a13, two-axis wrist joint actuators a14, a15, and one-axis palm joint actuator a32. Is serially connected to the control unit.

  Further, the robot apparatus has three-axis hip joint actuators a19, a20, a21, a one-axis knee joint actuator a22, and two-axis ankle joint actuators a23, a24 on the left leg, which are serially connected to the control unit. It is connected. Similarly, the right leg has triaxial shoulder joint actuators a25, a26, a27, a monoaxial elbow joint actuator a28, and biaxial ankle joint actuators a29, a30, which are serially connected to the control unit. Has been.

  The actuators a1 to a30 used in each joint are composed of, for example, a DC brushless motor, a speed reducer, and a position sensor that detects the rotational position of the output shaft of the speed reducer, and rotate according to a position control target value given from the outside. While driving, the current output torque, joint angle, and joint angular velocity are output. This type of joint actuator is described in, for example, Japanese Patent Application Laid-Open No. 2004-181613 already assigned to the present applicant.

As can be seen from FIG. 3, the biped walking mobile robot can be expressed as an open link tree structure based on the pelvis B. In the case of a mobile robot, the robot can move freely in the world space and change its posture. Therefore, Euler angles α = (α, β, γ) ^T of the pelvis B and its position p ₀ (p _0x , p _0y , p _0z ) ^T are introduced as state variables for representing the state of the entire robot. Then, since the robot can be handled as a link structure formed by connecting a plurality of rigid bodies, the motion equation of the robot can be expressed in the form of the following expression (1).

Here, q is a generalized variable for the entire robot, and is composed of a vector in which the base posture α and position p ₀ as the robot's motion state are arranged and a vector θ that can be obtained by arranging all joint values as the current state of the actuator. Is done. That is, q corresponds to a dynamic model of a robot as a link structure, and is expressed in the form of the following equation (2).

In the above formula (1), τ is a generalized force corresponding to the generalized variable q, and components corresponding to α and p ₀ are ₀ (because there is no actuator that generates force), and θ The component corresponding to is equal to the generated force of the actuator that drives each joint. H represents the inertia matrix of the entire robot, and b represents gravity, Coriolis force, and the like. f is an external force. J is a Jacobian that associates the speed generated in the direction of the external force with the generalized speed that is the time derivative of the generalized variable q, and is expressed by the following equation (3).

  FIG. 4 schematically shows a control system configuration for realizing a whole body cooperative operation such as walking with a movable leg of a robot apparatus or grasping or carrying an object using two arms. As shown in the figure, the robot apparatus 1 includes a control unit 20 that performs overall control of driving of each joint actuator and other data processing, an input / output unit 40, a drive unit 50, and a power supply unit 60. Is done. Hereinafter, each part will be described.

  The input / output unit 40 is a camera 15 corresponding to the eyes of the robot apparatus 1 as an input unit, a microphone 16 corresponding to an ear, and a touch sensor 18 that is disposed on a part such as a head or a back to detect a user's contact. Or other various sensors corresponding to the five senses. Further, as an output unit, a speaker 17 corresponding to the mouth or an LED indicator (eye lamp) 19 for forming a facial expression by a combination of blinking and lighting timing is provided. These output units can express user feedback from the robot apparatus 1 in a format other than a mechanical motion pattern such as a leg or the like, such as sound or blinking of a lamp. Moreover, the sound source direction can be detected by mounting the seven microphones on the head with the directivity adjusted.

  Here, the camera 15 is configured as a stereo camera by two cameras having different viewpoints (projection centers) maintained in a predetermined positional relationship, and the distance between each point in the captured image and the projection center is set to three angles. It can be measured by the principle of surveying.

  The drive unit 50 is a functional block that realizes the body operation of the robot apparatus 1 in accordance with a predetermined motion pattern commanded by the control unit 20, and is a control target during motion execution. The drive unit 50 is a functional module for realizing a degree of freedom in each joint of the robot apparatus 1 and is configured by a plurality of drive units provided for each axis such as a roll, a pitch, and a yaw in each joint. Each drive unit includes a motor 51 and a speed reducer (not shown) that rotate around a predetermined axis, an encoder 52 that detects the rotational position of the motor 51, and the rotational position and rotation of the motor 51 based on the output of the encoder 52. It is comprised by the combination of the driver 53 which controls a speed adaptively.

  The power supply unit 60 is a functional module that supplies power to each electric circuit in the robot apparatus 1. The rechargeable battery 61 is configured, for example, in the form of a “battery pack” in which a plurality of lithium ion secondary battery cells are packaged in a cartridge type. The charge / discharge control unit 62 grasps the remaining capacity of the battery 61 by measuring the terminal voltage of the battery 61, the amount of charging / discharging current, the ambient temperature of the battery 61, and the like, and determines the charging start timing and end timing. . The charging start / end timing determined by the charge / discharge control unit 62 is notified to the control unit 20 and serves as a trigger for the robot apparatus 1 to start and end the charging operation.

  The control unit 20 corresponds to a “brain”, and is mounted on, for example, the body head or the trunk of the robot apparatus 1. FIG. 5 illustrates the configuration of the control unit 20 in more detail. As shown in the figure, the control unit 20 has a configuration in which a CPU 21 as a main controller is connected to a memory, other circuit components, and peripheral devices via a bus. The bus 27 is a common signal transmission path including a data bus, an address bus, a control bus, and the like. Each device on the bus 27 is assigned a unique address (memory address or I / O address). The CPU 21 can communicate with a specific device on the bus 28 by specifying an address.

  The RAM 22 is a writable memory composed of a volatile memory such as a DRAM, and is used for loading a program code to be executed by the CPU 21 and for temporarily storing work data by the execution program. The ROM 23 is a read-only memory that permanently stores programs and data. Examples of the program code stored in the ROM 23 include a self-diagnosis test program that is executed when the robot apparatus 1 is powered on, and an operation control program that defines the operation of the robot apparatus 1. The nonvolatile memory 24 is composed of an electrically erasable / rewritable memory element such as an EEPROM, for example, and stores data to be updated sequentially such as an encryption key, other security information, and a device control program to be installed after shipment. Used to hold in a non-volatile manner.

  The interface 25 is a device for interconnecting with devices outside the control unit 20 and enabling data exchange. The interface 25 performs data input / output with the camera 15, the microphone 16, and the speaker 17, for example. The interface 25 inputs and outputs data and commands to and from the drivers 53-1.

  The interface 25 is a serial interface such as RS-232C, a parallel interface such as IEEE1284, a USB interface, an IEEE1394 interface, a SCSI interface, a card slot that accepts a PC card or a memory stick, and the like. A general-purpose interface for connecting peripheral devices may be provided, and programs and data may be moved between externally connected external devices.

  Further, the control unit 20 includes a wireless communication interface 26, a network interface card 27, and the like, via a proximity wireless data communication such as Bluetooth, a wireless network such as IEEE 802.11b, or a wide area network such as the Internet. Thus, data communication can be performed with the external host device.

Double-arm freedom configuration:
FIG. 6 specifically shows the configuration of the degree of freedom of the right arm part of the robot apparatus shown in FIG. Since the two arms are bilaterally symmetric, the illustration of the degree of freedom configuration of the left arm portion is omitted here.

  The shoulder portion has joint degrees of freedom of rotation about the roll, pitch, and yaw axes, and corresponding shoulder joint actuators a10, a11, and a12 are disposed. Further, the elbow part has an elbow pitch axis rotation degree of freedom rotating around the pitch axis, and an elbow pitch axis joint actuator a13 corresponding to this is disposed. In addition, wrist joint actuators a14 and a15 having joint degrees of freedom of rotation about the roll and yaw axes are disposed on the wrist. Further, the palm has a degree of freedom of joint rotation around the pitch axis with respect to the hand portion, and a palm joint pitch axis actuator a32 corresponding thereto is disposed. The palm joint constitutes the most advanced pitch axis freedom of the arm.

  FIG. 7 shows a modification of the degree of freedom configuration of the right arm portion of the robot apparatus shown in FIG. The difference from FIG. 6 is that the wrist is given a degree of joint freedom around the pitch axis at the wrist, not at the palm. In this case, the actuator a32 is used for rotational driving of the wrist joint pitch axis instead of the palm joint pitch axis. It is done. The wrist joint constitutes the most advanced pitch axis freedom for the arm.

  As shown in FIG. 6, the left and right arms of the robot apparatus are end effectors as shown in FIG. 6 in addition to the shoulder roll, pitch, yaw joint, elbow pitch joint, wrist roll and yaw joint. By providing the palm with a joint degree of freedom of the pitch axis, the following two operations can be realized in an application of grasping an object with two arms.

(1) The posture of the grasped object is changed by driving only the palm pitch axis while fixing the six axes of the arm.
(2) Movement / transportation with the palm pitch axis as the redundant axis and the other six-axis degrees of freedom of the arm portion, and the posture of the grasped object fixed.

FIG. 8 to FIG. 10 show the two arms of the robot apparatus shown in FIG. 1 in the state where the six arms other than the palm pitch axis are fixed and the hand moves with respect to the wrist. The state in which the palm pitch axis is driven so as to be horizontal, downward, and upward is shown.

  Here, when the robot apparatus holds the cube so that the pair of side surfaces of the cube facing each other are used as contact surfaces with the left and right palms and is sandwiched between the two arms, when the above operation is performed, FIG. As shown in FIG. 13, the cube rotates around the palm pitch axis. In other words, if the object is gripped so that the left and right palm pitch axes are on a straight line, the posture changes as if the object has a rotation axis only by the rotational drive of the palm pitch axis. The posture of the grasped object can be changed without controlling the posture of the other six axes.

  At this time, if gripping is performed so that the center of gravity of the object is on or near the rotation axis where the left and right palm pitch axes coincide, the posture of the object can be changed to the palm pitch without substantially changing the position of the center of gravity of the object. It can be changed only by rotating the shaft.

  Further, by providing the most advanced pitch axis of the arm portion inside the palm, that is, in the end effector as shown in FIG. 6, it is easy to make the center of gravity position of the gripping object coincide with the rotation axis, and stable gripping becomes possible.

  For example, when an end effector (hand) position r of an arm having seven joint degrees of freedom of q = (q1, q2, q3, q4, q5, q6, q7) is designated, the rotation command to each joint axis is r is obtained from inverse kinematics calculation. There is a relationship represented by the following formula (4) between q and r. Where J is Jacobian.

  If q7 is the pitch axis inside the end effector, if you want to change the posture of the object only once after holding it so that the center of gravity of the object is on or near the rotation axis where the left and right palm pitch axes match, Since only q7 needs to be rotated, it is not necessary to repeat the inverse kinematics operation.

  On the other hand, as shown in FIG. 7, when the most advanced pitch axis of the arm is outside the end effector such as the wrist, or when the most advanced joint axis is directed in a direction other than the pitch axis, FIG. When the gripping object is rotated as shown in FIG. 13, the end effector position changes with the rotation. For this reason, in order to change only the posture of the grasped object, it is necessary to solve the inverse kinematics calculation of the arm part and change the joint angle of the entire arm part.

  Of course, as shown in FIG. 6, even when the degree of freedom at the tip of the arm is the pitch axis inside the end effector, when the posture of the gripped object is changed and the pitch axis moves out of the movable range, The inverse kinematics calculation may be solved again using the other six axes of the arm so as to obtain a desired position / posture.

  FIG. 14 is a flowchart showing a processing procedure for changing the posture of the grasped object when the object is grasped so that the center of gravity of the object is on or near the rotation axis where the left and right palm pitch axes coincide. Show.

  Assume that the robot apparatus grips an object so that the left and right palm pitch axes are on one straight line with the two-arm palm as an initial posture. If there is no request for changing the position / orientation of the object, the apparatus stands by while maintaining the current position and orientation of the object (step S8).

  Here, if there is a request for changing the position / posture of the object (Yes in step S1), it is determined whether or not the requested position / posture can be realized only by the rotational drive of the palm pitch (step S2).

  If the requested position / posture can be realized only by rotational driving of the palm pitch (Yes in step S2), the palm pitch axis is changed by the requested amount of rotation (step S3). Rotational driving using only the palm pitch axis is extremely advantageous in terms of calculation cost compared to the case of performing the inverse kinematics calculation of the other six axes.

  Next, it is determined whether or not the palm pitch axis is within the movable range (step S4). If the palm pitch axis is within the movable range, a new joint angle is set (step S7), and the process returns to step S1.

  On the other hand, when the requested position / posture cannot be realized only by the rotational drive of the palm pitch (No in Step S2), and when the palm pitch axis is changed by the requested amount of rotation, it is out of the movable range (Step S4). No), the palm pitch axis holds the current value, and the other six axes are solved by inverse kinematics calculation (step S5).

  Then, it is checked whether or not each of the obtained 6-axis joints is within the movable range, and whether or not self-interference that collides with the arm portion will occur (step S6). If the 6-axis joint is within the range of motion and no self-interference occurs, a new joint angle is set (step S7), and the process returns to step S1. When any joint axis is out of the movable range or causes self-interference, the current position and posture of the object are maintained (step S8), and the process returns to step S1.

Movement / transportation with the palm pitch axis as the redundant axis and the posture of the gripped object fixed. FIGS. 15 to 16 show the two arms of the robot apparatus shown in FIG. It shows how the six axes from the shoulder to the wrist are changed.

  This is a solution to the inverse kinematics of the six axes from the shoulder to the wrist by providing the most advanced palm pitch axis, which is the seventh axis, in order to bring the gripped object into a specific position and posture. It means that the space has expanded. As shown in FIG. 42, if the solution of the IK of the six axes of the arms excluding the redundant axis is within the movable range of the redundant axis, the posture of the end effector on the seven axes can be maintained. The 6-axis IK solution space is expanded. As a result, in the arm portion consisting of six axes from the shoulder portion to the wrist portion, even if the destruction is not found by the inverse kinematics operation (or the solution does not converge by the predetermined number of recursive operations), It becomes easier to obtain a solution by driving.

  For example, when the robot device grips an object with two arms and lifts the object while maintaining its posture, the inverse kinematics calculation is first performed using a smaller number of joint degrees of freedom among the seven axes. In this case, the number of degrees of freedom of joints to be used when the solution derivation becomes difficult can be increased at a low cost. In addition, in the inverse kinematics solution space for the same position and posture of the grasped object (ie, end effector), the end effector (ie, the pitch axis of the palm or wrist) is the redundant axis, and the other six axis inverse kinematics solutions By determining the position of the redundant axis so that the margin is maximized (ie, there is a margin in the movable range), it is difficult to find the inverse kinematics calculation during the transportation and movement of the object (ie, the operation fails) It is possible to realize a double-arm operation.

  FIG. 17 shows a processing procedure in the form of a flowchart for the robot apparatus to carry the grasped object using the two arms while maximizing the solution space of the inverse kinematics operation. In the following, it is assumed that the robot apparatus has joints around the pitch axis of the palm or wrist as the most advanced degree of freedom of both arms, and this is set as a redundant axis.

  It is assumed that the robot apparatus holds an object and a request for changing the position / orientation of the object is generated. In this case, it is necessary to change the position / posture of the two arms in order to realize the position / posture of the object.

  In this case, first, a double-arm inverse kinematics solution capable of realizing the change of the position / posture of the object without moving the redundant axis is obtained (step S11). Here, for convenience of explanation, it is assumed that a situation where an inverse kinematics solution cannot be obtained is not assumed.

  Next, regarding the six axes other than the redundant axis, when the redundant axis is moved in the plus direction in the dual-arm posture, it is divided into the joint group A far from the movable angle limit and the opposite joint group B (step S12).

  Here, when all the other six axes are only group A or group B (or when concentrated on one joint group) (step S13), depending on the next object position / posture change command, the joint The axis tends to reach the movable angle limit, and the possibility that the operation will fail increases. Therefore, the redundant axis is moved so that one of the joints of the A group or the B group becomes the other joint group, and an attempt is made to expand the solution space of the inverse kinematics calculation (step S14). Then, a double-arm inverse kinematics solution is obtained so that the position / posture P of the object after the change of the position / posture of the grasped object can be maintained.

  Next, one joint that is closest to the movable angle limit is selected from each of the A group and the B group (step S15), and it is determined which is closer to the movable angle limit (step S16).

  Here, when the group A has a joint axis closer to the movable angle limit, the redundant axis is moved in the plus direction to equalize the movable range between the group A and the group B, A double-arm inverse kinematics solution that can maintain the post-change position / posture P is obtained (step S17), and the process returns to step S15.

  In addition, when the group B has a joint axis closer to the movable angle limit, the redundant axis is moved in the minus direction to equalize the movable range between the group A and the group B, and then the position / posture of the gripped object Obtains a double-arm inverse kinematics solution that can maintain the changed position / posture P (step S18), and returns to step S15.

  Further, when the allowances up to the movable angle limit are approximately equal between the A group and the B group, the entire processing routine is terminated.

Conveying objects using two arms Here, we take a concrete example where a robot carries scattered cubes one by one and piles them up in a specific collection location, and the object is transported using the two arms of a robotic device. Will be described.

  It is assumed that the surface of each cube is painted with a different color so that the robot apparatus can identify individual cubes by visual or other means. It is assumed that how to place objects, that is, the order in which cubes are stacked is determined in advance (see FIG. 18). Therefore, the height at which the cubes are stacked differs depending on the order of placement.

  As shown in FIG. 6 and FIG. 7, when the robot arm has two degrees of freedom of joint around the pitch axis at a portion close to the end effector, various methods can be applied to a cube placed on the floor surface. Can be gripped by using two arms (see FIG. 19). Further, since a pair of opposing side surfaces are held as contact surfaces instead of the bottom surface of the cube, it is easy to lift from the floor surface and the installation surface is open, so that it can be easily placed at the installation location.

  On the other hand, the height of the cubes to be stacked differs depending on the order of placement (see FIG. 20). Therefore, the position of the hand that makes it easy to place at the specified height is selected, and each cube on the floor surface is selected. It is necessary to grip the cube (see FIG. 21).

  When the cube is moved in a certain height or depth direction, the bottom surface of the cube on the floor surface before gripping, that is, the floor surface can be used as the installation surface at the destination (see FIG. 22). In addition, even if the installation surface at the destination is a height or depth that cannot be reached with just two arms, the movement of the cube can be achieved by transporting the whole body in a coordinated manner, taking into consideration the posture of the trunk and the height of the waist. Is possible.

  On the other hand, there is a limit to the posture that can be formed by the movement of the double arm, so when moving the cube to a higher or farther position in the depth direction, the bottom surface of the cube on the floor surface before gripping, i.e. In some cases, the installation surface cannot be used as it is (see FIG. 23). In addition, there is a limit to the movement of the cube even if transport is performed with the whole body emphasized, taking into account the posture of the trunk and the height of the waist.

  The robot apparatus according to the present embodiment has a degree of freedom of joint around the pitch axis in a portion where the double arms are close to the end effector, and as shown in FIG. Therefore, by moving in consideration of the installation surface at the destination, it can be moved and transported with the above problems solved.

  If it is a cube, as shown in FIG. 24, all the side surfaces T1-T6 can be used as an installation surface. When the dual-arm gripping surfaces are the right surface T5 and the left surface T6, the front surface T1 that is not grounded immediately before gripping is used, and the degree of joint freedom around the pitch axis is used near the end effector at the destination. The posture of the cube can be changed to be the installation surface. As a result, even if the cube cannot be moved without changing the installation surface, it can be moved by changing the installation surface as shown in FIG.

  FIG. 26 shows a processing procedure for determining the possibility of movement in consideration of the allowable posture of the gripping target object in the form of a flowchart.

  When the gripping target object T is moved from the position P1 to P2, it is first checked whether or not the posture of the object T at the position P2 is designated (step S21).

  If the posture at the position P2 of the movement destination of the object T is not specified, whether the object T can be moved from the position P1 to P2 without changing the posture of the object T, for example, the height of the installation surface at the position P2 Judgment is made by comparing the degrees of freedom of the arms and the height of the trunk and waist (step S22).

  When the object T can be moved from the position P1 to P2 without changing the posture of the object T, it is determined that the object is movable, and this processing routine is ended.

  Further, when the posture of the object T cannot be moved from the position P1 to P2 without changing the posture, subsequently, it is checked whether or not there is a candidate other than the posture at the current position P1 in the posture at the position P2 of the object T. (Step S23). For example, as described above, it is determined whether or not the installation surface of the object T can be changed and placed at the position P2. When the installation surface of the object T can be changed and placed at the position P2, the posture with the surface as the installation surface is output as the posture candidate of the object T. Further, when the posture candidate cannot be determined, it is determined that the movement is impossible, and this processing routine is ended.

  Then, it is checked whether or not the object T can be moved according to the posture specified in advance at the position P2 for the object T or the posture candidate determined in step S23 (step S24). When there is something that can be moved in the designated posture or among the posture candidates, it is determined that the movement is possible, and this entire processing routine is ended. Further, when the designated posture cannot be moved or there is no posture that can be moved, it is determined that the movement is impossible, and the entire processing routine is ended.

  In the following, an operation for the robot apparatus to hold the object with two arms and move / carry as shown in FIGS. 22 and 25 will be described. However, it is assumed that the robot apparatus knows how to stack the gripping target objects, how to grip the gripping target object, model information of the gripping target object, and model information of the robot itself (see FIG. 27). See The model information of the gripping target object includes information known about the gripping target object such as shape, size, weight, and surface material (friction coefficient). The model information of the robot itself includes information on the range of the double-armed 7-axis inverse kinematics solution and the degrees of freedom of the trunk and waist height.

  FIG. 28 shows a processing procedure from a state where the robot apparatus is not gripping the gripping target object to a state where the robot apparatus can grip and walk, in the form of a flowchart.

  First, the grip target object and the stack destination of the grip target object are recognized by a method such as image recognition of a camera video (step S31).

  Next, the method of stacking the gripping target objects is searched (step S32). In order to load the gripping target object to the target position and orientation, the following is determined first.

・ Where to grasp the object to be grasped.
・ In what kind of posture should you hold both palms?
・ How much gripping pressure is used for gripping?
-Trunk posture and waist height when taking and placing.

  Where to grasp the object to be grasped can be determined according to the processing procedure shown in FIG.

  Next, approaching while walking so as to face the gripping target object (step S33), it stands at a position where both palms can be brought to a position and orientation where the gripping target object can be gripped.

  Next, a situation where the two arms can grip the object to be gripped is set (step S34). Specifically, the following operation is performed in consideration of the grounding state of the object to be grasped and the grounding state of the robot itself. Here, the grounding state of the object to be grasped and the grounding state of the robot apparatus itself are considered.

・ Expand the gap between both palms beyond the width of the object to be grasped.
・ Raise (or bend) the trunk.
・ Raise (or lower) your waist.
・ Open your finger.
・ Take both palms to a position and posture where you can hold the handset.

  Next, the object to be grasped is grasped (step S35). Specifically, the space between both palms is narrowed from the situation in which the object to be grasped can be grasped with two arms in the previous step S34 to the width of the object to be grasped. Then, the movement control of the two arms is performed so that both palms are narrowed until a gripping pressure necessary for transporting (lifting) the object to be gripped is applied. Further, fine adjustment is performed so that the gripping surfaces of both palms are in close contact with the object surface to be gripped. When sufficient gripping pressure is reached, lock with your finger.

  Next, the gripping state continuation process is started (step S36). In other words, double arm force control or gain of each joint actuator so that the grasped object is kept fixed between both palms by parallel processing with other motion control such as double arm motion and walking using legs. Take control.

  Along with the gripping of the object to be gripped, a drop detection process of the gripped object is started (step S37), and it is continuously monitored whether it is dropped by the parallel processing. Details of the fall detection method and the handling method when an object falls will be described later.

  In addition, it is confirmed whether the grasped object can be grasped by parallel processing until the grasped object is released (that is, released at a desired installation position) (step S38). For example, if the hand is lifted by a minute height (so that the object to be grasped does not roll greatly even if it is removed), and it is caught by the drop detection, it is determined that the object cannot be grasped correctly.

  Then, the position and orientation of the robot apparatus itself and the object to be grasped are set in a state where there is no hindrance for grasping and walking. Specifically, an operation is performed such as returning the position / posture of the trunk or waist changed so as to be able to be gripped in step S34 to the original state, and returning the gripping target object to the state.

  Also, if there is a request to change the position and orientation of the object until the object to be grasped is released (that is, released at the desired installation position), the double-arm inverse kinematics calculation is performed according to the processing procedure shown in FIG. It works to maximize the solution space.

  In step S31, the object to be grasped can be recognized by performing image processing on the image of the stereo camera mounted on the head of the robot apparatus. FIG. 29 shows the processing procedure at that time in the form of a flowchart. However, it is assumed that the object to be grasped is a cube and its model information (see FIG. 27) is already known.

  When the captured image of the gripping target object is shown in FIG. 30, first, a color image is subjected to a labeling process with a color threshold value to obtain a labeling image from which the gripping target object region is extracted as shown in FIG. Step S41).

  Then, after performing dilation and contraction processing on the labeling image to remove noise (step S42), the edge extraction processing is applied to extract only the edge portion of the labeling image (step S43). FIG. 32 shows an edge image obtained from the labeling image shown in FIG.

  Next, the edge image is subjected to a Hough transform (step S44) and an inverse Hough transform (step S45) to detect a straight line related to the edge. FIG. 33 shows a straight line obtained from the edge image shown in FIG.

  Next, an effective line segment is searched from the obtained straight line group (step S46), and an effective line segment and a point are detected by adding model information of a known cube (step S47). FIG. 34 shows the vertices of a cube detected from the straight line group obtained by the Hough transform.

  Then, by matching the effective point and stereo vision information, the point is converted into a point having three-dimensional information, and the position and inclination of the cube are obtained from the three-dimensional information. The vertex positions in the edge image that are not on the effective line segment are interpolated based on the cube model information and recognized as a cube (step S48).

  In step S33, the object recognition result is used to approach the gripping target object. In order to hold a cube with two arms, it is necessary to be parallel to the side surfaces of the cube to be contacted with both palms. In other words, when the front surface of the robot apparatus is brought close to one side surface of the cube, the side surfaces at both ends face each other substantially parallel to both palms.

  FIG. 35 shows how the robot apparatus approaches a cube. In the illustrated example, among the four side surfaces A to D orthogonal to the floor surface, A is closest to the robot apparatus, so that it walks and approaches a cube so as to face A. As a result, B and D which are both ends of A become gripping surfaces by both palms.

  In addition, in the two-arm gripping operation with two arms, the robot apparatus starts the fall detection process of the gripped object in step S37 together with the gripping target object, and continues to monitor whether or not it falls in parallel processing. For this drop detection processing, the robot apparatus can use at least one of the following sensors or a combination of two or more.

(1) Palm touch sensor (2) Wrist six-axis force sensor (3) Joint angle (4) Image recognition of object to be grasped by camera (5) Sole sensor (6) Sound

  As shown in FIG. 36, a switch for detecting contact with an object (external world) is provided on the palm, and drop detection can be performed based on the on / off output of the switch. When the gripping object falls, the switch is released and the switch is turned off.

  When drop detection is performed by providing a wrist sensor with a six-axis force sensor, the output value of the sensor immediately after gripping an object is stored in step S35. When the sensor output value falls below n% (n is an arbitrary threshold value) immediately after gripping during the fall detection process, it is determined that the object cannot be pinched with sufficient force with both palms, that is, it has fallen. Can do. Here, it is assumed that only the component perpendicular to the palm, that is, the grip surface, is used from the 6-axis force sensor output. Further, by using the sum of the sensor values of both palms as the sensor value immediately after gripping, the difference in gripping pressure when the posture of the object is changed can be taken into account.

  Further, the distance between both palms can be obtained from the joint angle calculated from the potentiometers of the two arms of the robot apparatus. When the interval between the palms becomes smaller than the width obtained from the model information of the object to be grasped, it is assumed that the object is no longer sandwiched, so it can be determined that the object has fallen.

  Further, the robot apparatus may observe the object to be grasped with a camera periodically, for example, during the object grasping operation. Then, when the position of the object when processed by image recognition deviates greatly from both palms, or when it can no longer be observed, it can be determined that the object has fallen.

  In addition, when the robot device performs walking or other stance work using ZMP (Zero Moment Point) as a stability criterion, the pitch axis and roll axis moments are zero inside the support polygon formed by the ground contact point and the road surface. In many cases, a sole sensor is provided to obtain a point. In such a case, the weight obtained from the sole sensor is held immediately after gripping the object, and when the value calculated from the weight measured during gripping falls below a certain value, the object is no longer indicated. Therefore, it can be determined that the object has fallen.

  In addition, as shown in FIG. 4, the robot apparatus according to this embodiment includes a microphone. For example, the direction of the sound source can be detected by adjusting the directivity and mounting seven microphones on the head. If a loud sound is detected from below the robot arm using these seven microphones, it can be recognized that there is a possibility of falling. However, it is difficult to conclude that the object is falling only by sensing the sound because there may be causes other than the fall. For example, observing the direction in which the sound was produced with a camera image (see FIG. 37), Preferably combined with other sensor outputs.

  FIG. 38 shows a processing procedure in the form of a flowchart for the robot apparatus to perform the drop detection based on the sensor detection result described above.

  When a loud sound is generated below the palm (step S51), when the grasped object is no longer visible at the palm position from the camera image (step S52), when the weight measured by the sole sensor is equal to or less than the threshold value (step S52) In step S53), when the palm interval calculated from the joint angles of the two arms is equal to or smaller than the threshold value (step S54), or when the palm touch sensor is turned off (step S55), an object is moved from both palms. It is presumed that it has left and dropped below (ie, normal fall) (step S58).

  Further, when the value in the Y-axis direction (or the direction perpendicular to the gripping surface) of the 6-axis sensor at the wrist is equal to or less than the threshold value (step S56), the X-direction or Z-axis immediately before the Y-axis value decreases. It is checked whether or not a large force is applied in the direction (step S57). Here, when a particularly large force is not applied, it can be estimated that the gripped object has dropped downward (that is, a normal drop) as described above (step S58). On the other hand, if a large force is applied in a direction different from the direction in which the palm grips the object, the gripped object may have been forcibly dropped (step S59).

  When the robot apparatus detects the fall of the grasped object in this way, the countermeasure processing is subsequently started. FIG. 39 shows the procedure of the handling process when the gripping object is dropped in the form of a flowchart.

  First, it is determined whether or not the detected fall is a normal fall (step S61). If it is a normal fall, the object which fell on the ground centering on the downward direction of a palm is searched (step S62). In addition, when it is not a normal drop, that is, when the grasped object is forcibly dropped, the drop direction and the drop position are estimated based on the Z-axis and X-axis forces detected immediately before dropping, and the object is searched from that point. Is started (step S63).

  If the fallen object can be found (Yes in step S64), the object is approached and grasped according to the same procedure as in steps S33 to S35 (step S65).

  On the other hand, when an object cannot be found at the estimated fall position (No in step S64), the object is searched over the entire ground (step S66).

  When the robot apparatus grips the cube with two arms and takes a walking state in accordance with the processing procedure shown in FIG. 28, the robot apparatus then shifts to a processing operation for transporting the gripped cube to a predetermined installation position. To do. The carrying work is realized by the whole body cooperative movement of the robot apparatus.

  The motion data of the robot apparatus is configured in the form of posture data arranged in time series, for example. Then, the inverse kinematics calculation is executed to calculate the rotation angle, angular velocity, angular acceleration, etc. of each joint from the posture data, and the process of sending the instruction value to the actuator is repeated every predetermined control cycle, so that the motion is Realize. FIG. 40 shows a processing procedure for the robot apparatus to carry the grasped object by the whole body cooperative operation in the form of a flowchart. By performing this processing procedure for each control cycle, the object can be transported while changing the position and orientation of the object to be grasped.

  First, inverse kinematics calculation is performed using only two arms, and a solution that can obtain the position and orientation of a desired object is searched (step S71). Here, according to the processing procedure shown in FIG. 17, a solution that maximizes the solution space of the double-arm inverse kinematics operation is obtained.

  When the desired position / orientation of the object is obtained (step S72), it is assumed that the processing of the control cycle is successful. In addition, when the position and orientation of the desired object cannot be obtained, the inverse kinematics operation is performed by adding degrees of freedom of the trunk and waist to the two arms, and a solution that can obtain the position and orientation of the desired object is searched. (Step S73).

  When the desired position and orientation of the object are obtained (step S74), it is assumed that the processing of the control cycle is successful. In addition, when the desired object position and orientation cannot be obtained, the inverse kinematics calculation is performed by adding the freedom degree of the hips (and legs) to the two arms and the trunk, and the desired object position and orientation are obtained. A solution that can be obtained is searched (step S75).

  Then, when the desired position and orientation of the object are obtained (step S76), the processing of the control cycle is assumed to be successful, and when the solution is not obtained, a value indicating that the control cycle has failed is returned.

  According to the processing procedure shown in FIG. 40, when the position and orientation of the next object can be obtained using only two arms, a solution is obtained using only two arms, and the number of degrees of freedom to be used is suppressed as much as possible. The desired object stacking motion is realized at a lower cost.

  When the robot apparatus transports the cube to a desired installation position, the robot apparatus stacks the cube to a target position, releases the cube, and retreats from the stacked position. FIG. 41 shows a processing procedure in the form of a flowchart until the robot apparatus stacks objects at a target position and finishes the work.

  First, the object to be grasped is transported (changed) to a height and posture that can be stacked (step S81). Specifically, the trunk is raised (or bent), the waist is raised (or lowered), and the posture of the object to be grasped is changed to a posture for accumulation (see FIGS. 25 and 26). An operation such as changing to a height at which the position can be stacked is performed. Here, the height is a height that does not hinder the user when he / she approaches the place to be piled up. Further, as the posture of the two arms, for example, as shown in FIG. 17, an inverse kinematics solution space maximization process is adopted, and a posture that does not hinder the loading is selected.

  Next, the robot walks to a position where a gripping target object can be stacked at a predetermined location (step S82).

  Next, in consideration of the installation state of the gripping target object and the grounding state of the robot apparatus itself, the gripping target object is transported to the place where it is stacked while being finely adjusted (step S83).

  When the gripping target objects are stacked, the gripping state by the two arms is released (step S84). Specifically, the gripping state continuation process started in step S36 is stopped, the fingers of both palms are opened, and the gap between both palms is expanded beyond the width of the gripping target object.

  Then, the release of the gripping state is confirmed (step S85). For this confirmation processing, the above-described gripping object detection processing such as palm touch sensor off can be applied. That is, by continuing the drop detection process activated in step S37 even after the gripping state continuing process is stopped, it can be detected that the gripping target object has been reliably released from the double-armed gripping state.

  Next, the object is retreated from the place where the object is loaded (step S86). At this time, the two arms are compared so that there is no influence on the stacked object, and the influence on the stacked object is walked to the contents and evacuated from the place.

  Then, the robot apparatus is returned to a predetermined posture (step S87), and the entire processing routine is ended.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention.

  The gist of the present invention is not necessarily limited to a product called a “robot”. That is, if it is a mechanical device or other general mobile device that performs an action similar to that of a human using two arms using electric or magnetic action, other industries such as toys, etc. The present invention can be similarly applied to products belonging to the field.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

FIG. 1 is a diagram showing an external configuration of a robot apparatus according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration example of a connection topology for connecting the joint driving actuators of each unit in the robot apparatus illustrated in FIG. 1. FIG. 3 is a diagram showing a configuration of the degree of freedom of the robot apparatus shown in FIG. FIG. 4 is a diagram schematically showing a control system configuration for realizing a whole body cooperative operation such as walking with a movable leg of the robot apparatus or grasping or carrying an object using two arms. FIG. 5 is a diagram showing the configuration of the control unit 20 in more detail. FIG. 6 is a diagram specifically showing the configuration of the degree of freedom of the right arm portion of the robot apparatus shown in FIG. FIG. 7 is a view showing a modification of the configuration of the degree of freedom of the right arm portion of the robot apparatus shown in FIG. FIG. 8A shows a state where the palm pitch axis is driven so that the tip of the robot apparatus shown in FIG. 1 is horizontal with respect to the wrist with six axes other than the palm pitch axis fixed. It is a figure. FIG. 8B shows a state in which the palm pitch axis is driven so that the hand is level with respect to the wrist with the two arms of the robot apparatus shown in FIG. 1 fixed with six axes other than the palm pitch axis. It is a figure. FIG. 9A shows a state in which the palm pitch axis is driven so that the tip of the robot apparatus shown in FIG. 1 is directed downward with respect to the wrist while the six arms other than the palm pitch axis are fixed. It is a figure. FIG. 9B shows a state in which the palm pitch axis is driven so that the hand is directed downward with respect to the wrist with the six arms other than the palm pitch axis fixed with the two arms of the robot apparatus shown in FIG. 1. It is a figure. FIG. 10A shows a state in which the palm pitch axis is driven so that the tip of the robot apparatus shown in FIG. 1 is directed upward with respect to the wrist while the six arms other than the palm pitch axis are fixed. It is a figure. FIG. 10B shows a state in which the palm pitch axis is driven so that the tip of the robot apparatus shown in FIG. 1 is directed upward with respect to the wrist in a state where six axes other than the palm pitch axis are fixed. It is a figure. FIG. 11 is a diagram illustrating a state in which the posture of the grasped object is changed by driving the palm pitch axis while the six arms other than the palm pitch axis are fixed with the two arms of the robot apparatus illustrated in FIG. 1. is there. FIG. 12 is a diagram showing a state in which the posture of the grasped object is changed by driving the palm pitch axis with the two arms of the robot apparatus shown in FIG. 1 fixed with six axes other than the palm pitch axis. is there. FIG. 13 is a diagram showing a state in which the posture of the grasped object is changed by driving the palm pitch axis with the two arms of the robot apparatus shown in FIG. 1 fixed with six axes other than the palm pitch axis. is there. FIG. 14 is a flowchart showing a processing procedure for changing the posture of the grasped object when the object is grasped so that the center of gravity of the object comes on or near the rotation axis where the left and right palm pitch axes coincide. . FIG. 15A is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while fixing the position and posture of the hand as an end effector. FIG. 15B is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while fixing the position and posture of the hand as an end effector. FIG. 15C is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while fixing the position / posture of the hand as an end effector. FIG. 15D is a diagram illustrating a state in which the six arms from the shoulder to the wrist are changed while the position and posture of the hand as an end effector are fixed in the two arms of the robot apparatus illustrated in FIG. 1. FIG. 16A is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while fixing the position and posture of the hand as an end effector. FIG. 16B is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while the position and posture of the hand as an end effector are fixed. FIG. 16C is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while the position and posture of the hand as an end effector are fixed. FIG. 16D is a diagram showing a state in which the six arms from the shoulder to the wrist are changed with the two arms of the robot apparatus shown in FIG. 1 while fixing the position / posture of the hand as an end effector. FIG. 17 is a flowchart showing a processing procedure for the robot apparatus to carry the grasped object using the two arms while maximizing the solution space of the inverse kinematics operation. FIG. 18 is a diagram for explaining a procedure in which the robot apparatus stacks a plurality of cubes. FIG. 19 shows that a robot apparatus having joint degrees of freedom around the pitch axis at a position where the two arms are close to the end effector grips a cube placed on the floor surface using the two arms in various ways. It is the figure which showed a mode. FIG. 20 is a diagram illustrating a state in which the heights of cubes when stacked are different depending on the order of placing. FIG. 21 is a diagram illustrating a state in which a hand position that makes it easy to place at a specified height is selected and each cube on the floor surface is gripped. FIG. 22 is a diagram for explaining that when a cube is moved in a certain height or depth direction, the bottom surface of the cube on the floor surface is used as it is as the installation surface at the movement destination before gripping. FIG. 23 is a diagram showing a state in which the bottom surface of the cube on the floor surface cannot be used as the installation surface at the moving destination as it is due to the limit of the posture that can be formed by the double-arm operation. FIG. 24 is a diagram illustrating a state in which all side surfaces T1 to T6 of the cube can be set as installation surfaces. FIG. 25 is a diagram illustrating a state in which the robot apparatus can move the cube by changing the installation surface. FIG. 26 is a flowchart illustrating a processing procedure for determining the possibility of movement in consideration of the allowable posture of the gripping target object. FIG. 27 is a diagram illustrating information that is grasped when the robot apparatus performs an operation of gripping, moving, and transporting an object with two arms. FIG. 28 is a flowchart showing a processing procedure from when the robot apparatus is not gripping the gripping target object to when the robot apparatus is gripped and can be walked. FIG. 29 is a flowchart illustrating a processing procedure for recognizing a gripping target object by performing image processing on a video image of a stereo camera mounted on the head of the robot apparatus. FIG. 30 is a diagram illustrating a captured image of a gripping target object. FIG. 31 is a diagram illustrating a labeling image obtained from the captured image of the gripping target object illustrated in FIG. 30. FIG. 32 is a diagram showing an edge image extracted from the labeling image shown in FIG. FIG. 33 is a diagram showing a straight line obtained from the edge image shown in FIG. FIG. 34 is a diagram showing the vertices of a cube detected from the straight line group. FIG. 35 is a diagram illustrating a state in which the robot apparatus approaches a cube. FIG. 36 is a diagram illustrating a state in which a switch is provided on the palm so that contact with an object (the outside world) can be recognized. FIG. 37 is a diagram illustrating a state in which the robot apparatus observes the direction in which the sound is generated with a camera image. FIG. 38 is a flowchart showing a processing procedure for the robot apparatus to perform drop detection based on the sensor detection result. FIG. 39 is a flowchart showing the procedure of the coping process when the gripping object is dropped. FIG. 40 is a flowchart showing a processing procedure for the robot apparatus to carry the gripped object by the whole body cooperative operation. FIG. 41 is a flowchart showing a processing procedure until the robot apparatus stacks objects at a target position and finishes the work. FIG. 42 is a diagram illustrating a state in which the posture of the end effector on the seven axes is maintained if the solution of the six axes IK of the arms excluding the redundant axes is within the movable range of the redundant axes.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Robot apparatus 15 ... CCD camera 16 ... Microphone 17 ... Speaker 18 ... Touch sensor 19 ... LED indicator 20 ... Control part 21 ... CPU
22 ... RAM
23 ... ROM
24 ... Nonvolatile memory 25 ... Interface 26 ... Wireless communication interface 27 ... Network interface card 28 ... Bus 29 ... Keyboard 40 ... Input / output unit 50 ... Drive unit 51 ... Motor 52 ... Encoder 53 ... Driver

Claims (11)

At least with the torso,
The degree of joint freedom around the shoulder roll, pitch, and yaw axes, the degree of joint freedom around the elbow pitch axis, the degree of joint freedom around the wrist roll and yaw axes, The torso having a leading-edge pitch axis joint having a degree of freedom around the provided pitch axis, the leading-edge pitch axis joint being disposed either on the inner side of the palm of the hand or on the wrist The left and right arms connected to
Object recognition means for recognizing the position and orientation of the object to be grasped;
A control unit for controlling joint driving of each unit including the two arms;
With
The control unit is configured so that the pitch arms of the most advanced pitch axis joints of each of the two arms are substantially in a straight line, and the center of gravity of the object to be grasped is on the straight line, so that the two arms are positioned on the object to be grasped. Gradually narrow the width and hold it between your left and right hands.
A robot apparatus characterized by that. The control unit drives only the most advanced pitch axis while fixing other joints of the arm according to the position and orientation change request of the object while holding the object with two arms, Change the posture of the gripping object,
The robot apparatus according to claim 1. When the required position / orientation change of the gripping object is performed, the control unit is requested while keeping the most advanced pitch axis at the current position when the movable range of the most advanced pitch axis is deviated or self-interference occurs. Find inverse kinematics solutions for other joints of the arm to obtain the position and orientation of the gripping object
The robot apparatus according to claim 2 . In response to an object movement request while holding an object with two arms, the control unit uses the most advanced pitch axis as a redundant axis and uses the other joint degrees of freedom of the arm to hold the posture of the object Keep moving,
The robot apparatus according to claim 1 . The control unit, when moving an object having a plurality of side surfaces as installation surfaces, allows the installation surface to have an installation surface corresponding to the height or depth direction of the installation site of the object. Changing the posture of the grasped object by rotational driving of the most advanced pitch axis;
The robot apparatus according to claim 1 . It further comprises a fall detection means for detecting the fall of the object gripped using the two arms.
The robot apparatus according to claim 1 . A six-axis force sensor is provided on the wrist of the double arm;
Before Symbol drop detecting means may store the output value of the sensor immediately after the object is gripped, the output value of the sensor detects a drop of the gripping object by falls below immediately gripped,
The robot apparatus according to claim 6 . The drop detection means detects the fall of the gripped object when the object position recognized by the object recognition means is greatly deviated from the palms of the two arms, or when it can no longer be observed.
The robot apparatus according to claim 6 . The drop detection means checks whether a large force is applied in the X direction or the Z direction immediately before the Y axis value decreases when the value in the Y axis direction of the 6-axis sensor at the wrist is below the threshold value. However, when a large force is not applied, it is estimated that the grasped object has fallen downward. Judge that there is,
The robot apparatus according to claim 7 . The control unit confirms the release of the double-arm gripping state using the drop detection means when installing the gripping object at a desired installation location.
The robot apparatus according to claim 6 . It further comprises the flexibility of the trunk and movable legs,
When carrying the controller while changing the position and orientation of the object to be gripped,
First, the inverse kinematics calculation is performed using only the two arms to search for a solution that can obtain the position and posture of the desired object,
When a solution using only the two arms is not found, a solution that can obtain the position and posture of a desired object by performing inverse kinematics calculation by adding the degree of freedom of the trunk to the two arms. Search and
When a solution is not found, search for a solution that can obtain the position and orientation of a desired object by performing inverse kinematics calculation with a degree of freedom added to the legs.
The robot apparatus according to claim 1 .

Images