JP2006130580A - Method for gripping arbitrarily-shaped object by robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for gripping an arbitrarily-shaped object by a robot having a visual sensor and capable of properly gripping the arbitrarily-shaped object by a robot hand. <P>SOLUTION: An object to be gripped is allocated to a prescribed simple shape on the basis of image information acquired with the visual sensor (a step S1). The size and direction of the simple shape are calculated (a step S3). A gripping posture of the robot hand is set from the direction and size of the simple shape corresponding to a type of shape (steps S11, 21, and 31). A wrist position of the robot hand is calculated from the calculated gripping posture (a step S13). A target posture of an arm and that of a body are calculated by inverse kinetic analysis (a step S15). The target postures are obtained by controlling a motor (a step S17). Then, the gripping operation is executed (a step S19). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method of gripping an object having an arbitrary shape by a robot, and more particularly to a method of gripping an object by recognizing the shape of an object using a visual sensor.

  In an industrial robot or the like, usually, the object to be grasped and the processing content are limited, and the shape of the robot hand and the grasping control according to the object are performed. Furthermore, there is a robot hand using a visual sensor as a technique for enlarging a gripping object from a single item to a plurality of items and performing processing corresponding to each (see, for example, Patent Documents 1 and 2).

The technique of Patent Document 1 aims to improve the operation speed of the hand by making the finger center axis of the hand coincide with the optical axis of the optical distance sensor and the visual sensor so that the coordinate conversion calculation is unnecessary. The technique of Patent Document 2 is to place an imaging means for imaging a workpiece, which is an object, on a robot hand, calculate the position and shape of the workpiece based on the captured image, and control the gripping posture of the hand. .
JP-A-6-31666 JP 2003-94367 A

  These techniques are based on the premise that the object to be grasped has a grasping portion that is assumed to be grasped by the robot hand, or has a form suitable for grasping by the robot hand. However, when the application range of the robot hand is further expanded, it is necessary to perform an operation of appropriately grasping various objects having arbitrary shapes.

  Therefore, an object of the present invention is to provide a method for gripping an arbitrarily shaped object by a robot that has a visual sensor and can appropriately grasp an arbitrarily shaped object by a robot hand.

  In order to solve the above problems, a method for gripping an arbitrarily shaped object by a robot according to the present invention is a method for grasping an arbitrarily shaped object by a robot having a visual sensor and a robot hand having multiple fingers and multiple joints, 1) a step of obtaining main axis direction, size, position, and posture information of the target object by image recognition from the image of the target object acquired by the visual sensor; and (2) the obtained main axis direction, size, position, and posture information. A step of setting a target posture of the robot hand based on the start of gripping, (3) a step of adjusting the target posture according to the size of the object, and (4) a wrist position of the hand based on the center position of the object. A step of calculating, (5) a step of calculating a target posture of the robot arm and body by an inverse kinematics method, and (6) a step of operating each joint of the robot based on the obtained target posture. And wherein the are.

  The target posture of the robot's hand is set according to the principal axis direction of the object, and the target posture of the arm and body is calculated back by the inverse kinematics method so that this target posture can be obtained. To move each finger of the robot to the grip start point to perform the grip operation.

  According to the present invention, the target posture of the hand is set based on the image of the target object acquired by the visual sensor, and control is performed so as to obtain the target posture. Therefore, the target object is wrapped and held in an appropriate posture. be able to. For this reason, even if there is an error in the position measurement of the target object by the visual sensor, or when the target object is applied to a simple shape and there is a large difference between the fitted shape and the original shape, It can be gripped, and an arbitrarily shaped object can be gripped appropriately.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the description, the same reference numerals are given to the same components in the drawings as much as possible, and duplicate descriptions are omitted.

  FIG. 1 is a diagram showing a configuration of a robot hand 1 to which a method of gripping an arbitrarily shaped object by a robot according to the present invention is applied. FIG. 2 shows a control system thereof, that is, an arbitrary shaped object by a robot according to the present invention. It is a block block diagram which shows the control system which performs the holding | grip method. FIG. 3 is a schematic configuration diagram of the robot 100 including the robot hand 1.

  The robot hand 1 according to the present embodiment includes four fingers: a thumb 10, an index finger 11, a middle finger 12, and a ring finger 13. The thumb 10 is a four-degree-of-freedom link system in which a motor 14 is arranged for each of four joints, and a six-axis force sensor 15 is arranged at the root. The six-axis force sensor 15 is a sensor that can independently detect a force in a direction along each of the three axial directions orthogonal to each other and a force around each axial direction. The other three fingers 11 to 13 are configured such that the motor 14 is arranged for each of the three joints excluding the most advanced joint, and the most advanced joint is configured to interlock with an adjacent joint (interlocking joint 17). Therefore, each of the four joints constitutes a link system having three degrees of freedom. The degree of freedom of the entire robot hand 1 is 13. A 6-axis force sensor 15 is arranged at the base of each finger 11 to 13 like the thumb 10. The tips 10t to 13t of these fingers are composed of curved elastic bodies. Further, an encoder potentiometer 16 for detecting the bending angle of the joint is disposed at each joint.

  As shown in FIG. 3, the robot hand 1 is attached to the body 8 of the robot 100 by an arm 7. The body 8 has arms 7 on the left and right, and a camera eye 3 as a visual sensor on the head. The arm 7 is also provided with a motor and an encoder potentiometer so that the robot hand 1 can be moved to a desired position.

  The control system is mainly configured by a control ECU 2 including a RAM, a ROM, a CPU, and the like, and recognizes the shape and position of an object by image recognition from an output image of a camera eye 3 that captures an image of the grasped object. An image recognizing unit 20, a gripping posture calculating unit 21 that calculates a gripping position / griping posture based on the recognition result, and a hand control unit 22 that controls the movement of the robot hand 1. The 6-axis force sensor 15 and the output signals of the encoder potentiometer 16 are input to the hand control unit 22, and the movement of each motor 14 is controlled by the motor driver 4.

  The control system is not limited to such a configuration, and the control unit that controls the movement of the robot hand 1 so as to have the instructed posture, and the image recognition device and the gripping posture calculation unit may be separated. . Further, it may be divided by hardware or software.

  Next, a control method for gripping a gripping object by the robot hand 1 will be specifically described. FIG. 4 is a main flowchart of the control process.

  First, based on the image information input from the camera eye 3, the image recognition unit 20 applies the gripping object to one of the predetermined simple shapes by image recognition (step S1). For example, fitting (approximation) to any one of a rectangular parallelepiped (including a cube), a cylinder, and an ellipsoid (including a perfect sphere) is performed. This fitting can be performed by template matching, feature extraction, correlation calculation, or the like. If it is applied to the simple shape, the size and orientation of the applied simple shape are obtained (step S3).

  In step S5, the type of the applied simple shape is determined, and the process branches to steps S11, S21, and S31 according to the determined simple shape.

  In step S11, the target posture determination process of the hand 1 is performed when the fitted simple shape is a rectangular parallelepiped. A flowchart of specific processing is shown in FIG. As shown in FIG. 6, a robot coordinate system (a coordinate system fixed to the body 8 of the robot 100) is represented by Σr, and a local coordinate system of the robot hand 1 is represented by Σh. Further, the normal directions of three adjacent surfaces of the fitted rectangular parallelepiped 90 are V1, V2, and V3.

  The normal direction of the two surfaces having the longest distance among the two opposing surfaces is the main axis (step S110), and the gripping direction is determined from the main axis direction (step S111). Specifically, gripping from the side is selected when the main axis direction substantially coincides with the Zr direction, and gripping from above is selected when positioned in the horizontal plane.

ここで、（Ｘ_ｈｘ，Ｘ_ｈｙ，Ｘ_ｈｚ）はロボット座標系から見たロボットハンド１の局部座標系のＸ軸方向ベクトルＸｈであり、（Ｖ_１ｘ，Ｖ_１ｙ，Ｖ_１ｚ）はロボット座標系から見た法線ベクトルＶ１を示す。Ｙｈ、Ｚｈ、Ｖ２、Ｖ３についても同様である。 When the gripping direction is determined, the process branches according to the gripping direction (step S112). In the case of gripping from above, the process moves to step S113 to determine the hand posture. If the matrix representing the hand posture is Rh, the posture in this case is represented by the following equation (1).

Here, (X _hx , X _hy , X _hz ) is the X-axis direction vector Xh of the local coordinate system of the robot hand 1 viewed from the robot coordinate system, and (V _1x , V _1y , V _1z ) is the robot coordinate system. The normal vector V1 seen from FIG. The same applies to Yh, Zh, V2, and V3.

  That is, in the case of upward gripping, as shown in FIG. 6A, the vertically upward normal vector V1 and the Y axis of the hand coordinate system (direction perpendicular to the palm of the hand 1) are parallel in opposite directions. To be located. Then, the Z axis of the hand coordinate system (the extending direction of the indicating finger 11, the middle finger 12, and the ring finger 13 when the hand 1 is extended) is made to coincide with the normal vector V3, and the remaining X axis is made to coincide with the normal vector V2.

On the other hand, in the case of gripping from the side, the process proceeds to step S114 to determine the hand posture. The posture in this case is expressed by the following equation (2).

  That is, in the case of side gripping, as shown in FIG. 6B, the normal vector V1 (vertically upward) that coincides with the principal axis direction and the X axis of the hand coordinate system are matched. Then, the Y axis of the hand coordinate system is matched with the normal vector V2, and the remaining Z axis is matched with the normal vector V3. As described above, the posture of the robot hand 1 when grasping when the fitted simple shape is a rectangular parallelepiped is obtained.

  In step S21, the target posture determination process of the hand 1 is performed when the fitted simple shape is a cylinder. A flowchart of specific processing is shown in FIG. Here, the principal axis direction of the cylinder 91 is set to V0 as shown in FIG. The robot coordinate system and the hand coordinate system are the same as in the case of the rectangular parallelepiped grip shown in FIG.

  First, the gripping direction is determined based on the ratio of the main axis direction of the cylinder 91 to the length of the diameter (step S211). For example, when the length of the cylinder 91 in the main axis direction is sufficiently long and the thickness is in a certain range and the side wall of the cylinder 91 can be gripped, gripping from the side of the cylinder 91 is performed. In this case, gripping from the end surface of the cylinder 91 is selected.

ここで、（Ｖ_０ｘ，Ｖ_０ｙ，Ｖ_０ｚ）はロボット座標系から見た主軸ベクトルＶ０を表している。次に、指先方向つまりＺ軸方向ベクトルｚｈを求める（ステップＳ２１４）。図９（ａ）に示されるように、まず、円柱９１の上端面を含む平面Ｐを規定すると、この平面は、この上端面の中心Ｃの座標を（ｘ_ｃ，ｙ_ｃ，ｚ_ｃ）とすると、次式（４）で表せる。

ロボットの肩の中心Ａから鉛直線を下ろし、その線が平面Ｐと交わる点をＢ点とすると、ロボット座標系におけるＡ点とＢ点のｘ、ｙ座標は一致する（ｘ_Ａ＝ｘ_Ｂ、ｙ_Ａ＝ｙ_Ｂ）。一方、Ｂ点は平面Ｐ上に存在するから、そのｚ座標ｚ_Ｂは、（４）式より

で表せる。このＢ点とＣ点を結ぶ方向の単位ベクトルをｚｈに設定する。このｚｈの座標は、以下の（６）式から算出される。

When the gripping direction is determined, the process branches according to the gripping direction (step S212). In the case of gripping from the end face, the process proceeds to step S213, and first, as shown in FIG. 8A, the Yh-axis direction of the hand coordinate system is made to coincide with the principal axis direction V0.

Here, (V _0x , V _0y , V _0z ) represents the principal axis vector V 0 viewed from the robot coordinate system. Next, the fingertip direction, that is, the Z-axis direction vector zh is obtained (step S214). As shown in FIG. 9A, first, when a plane P including the upper end surface of the cylinder 91 is defined, the plane has coordinates (x _c , y _c , z _c ) of the center C of the upper end surface. Then, it can be expressed by the following formula (4).

If a vertical line is dropped from the center A of the robot's shoulder and the point where the line intersects the plane P is B point, the x and y coordinates of point A and point B in the robot coordinate system match (x _A = x _B , y _A = y _B ). On the other hand, since the point B exists on the plane P, the z coordinate z _B is obtained from the equation (4).

It can be expressed as A unit vector in the direction connecting the points B and C is set to zh. The coordinates of zh are calculated from the following equation (6).

The remaining unit vector xh in the X-axis direction is calculated by the following equation (7) as an outer product of the vector yh and the vector zh (step S215). Thereby, the posture of the hand 1 is obtained.

  When the diameter of the cylinder 91 is small, the height of the middle finger 12 facing the thumb 10 when contacting the object, as shown in FIG. 10A, according to the posture of the hand 1 obtained by the above-described method. Since they substantially coincide with each other, the gripping can be surely performed. However, when the diameter of the cylinder 91 is large, when the distance between the thumb 10 and the middle finger 12 of the hand 1 is widened so that an object can be sandwiched in the posture of the hand 1 obtained by the above-described method, At the time when 10 contacts, the difference between the height of the thumb 10 and the height of the middle finger 12 becomes large, and the middle finger 12 does not contact the cylinder 91. For this reason, gripping fails.

ここで、^ｓＴ_４は、ハンド座標系Σｓから指先の座標系Σ４への４×４斉次変換行列であり、関節角度θ_１〜θ_４の関数である。これを母指１０についても行う。 Therefore, the diameter d of the cylinder is compared with a predetermined threshold value dth (step S216). When the diameter d of the cylinder is less than the threshold value dth, it is determined that the grip can be performed in the obtained posture, and the posture setting process is terminated. On the other hand, when the diameter d of the cylinder is equal to or larger than the threshold value dth, the hand posture adjustment process is performed. This is performed by rotating the wrist of the hand 1 so that the line segment connecting the thumb 10 and the middle finger 12 is parallel to the top surface of the cylinder 91 (see FIG. 11). Specifically, first, the position E of the tip of the thumb 10 and the position F of the tip of the middle finger 12 are calculated by a forward kinematic method (step S217). Here, the case of the middle finger 12 will be described as an example, but the same applies to the case of the thumb 10. As shown in FIG. 12, the link coordinate system of each joint of the middle finger 12 is set. If each joint angle of the finger is known, the position of the fingertip j can be calculated by the following equation (8) by the forward kinematic method.

Here, ^s T ₄ is a 4 × 4 homogeneous transformation matrix from the hand coordinate system Σs to the fingertip coordinate system Σ4, and is a function of the joint angles θ _{1 to} θ ₄ . This is also performed for the thumb 10.

An adjustment angle θadj is obtained from the position E of the tip of the thumb 10 and the position F of the tip of the middle finger 12 (step S218). Here, the position coordinate of the tip position E of the thumb 10 in the robot coordinate system is (x _E , y _E , z _E ), and the position coordinate of the tip position F of the middle finger 12 in the robot coordinate system is (x _F , y _F , Z _F ), the adjustment angle θadj shown in FIG. 11 is calculated by the following equation (9).

The robot hand 1 is rotated around the xh axis about the point C by the obtained adjustment angle θadj to obtain a new hand position (step S219). The new hand position Rh ^new is calculated by the following equations (10.1) (10.2).

次に、指先方向つまりＺ軸方向ベクトルｚｈを求める（ステップＳ２２２）。まず、円柱９１の中心Ｃ点を含む平面Ｐを図９（ｂ）に示されるように、規定する。この平面は、上述の式（４）で表せる。 On the other hand, in the case of gripping from the side, the process proceeds to step S221, and first, as shown in FIG. 8B, the Xh-axis direction of the hand coordinate system is matched with the main-axis direction V0.

Next, the fingertip direction, that is, the Z-axis direction vector zh is obtained (step S222). First, a plane P including the center C point of the cylinder 91 is defined as shown in FIG. This plane can be expressed by equation (4) above.

  On the other hand, in the case of gripping from the side, it is necessary to dispose the hand 1 outside the center of the object. Therefore, the point A is shifted by the distance D from the center of the shoulder of the robot 100 toward the body 8. Then, a vertical line is dropped from the shifted point A, and a point where the line intersects the plane P is defined as a point B. A unit vector in the direction connecting the points b and C is set to zh. This zh can be obtained from equations (5) and (6).

The remaining unit vector yh in the Y-axis direction is calculated by the following equation (12) as an outer product of the vector zh and the vector xh (step S223). Thereby, the posture of the hand 1 is obtained.

  In step S31, the target posture of the hand 1 is determined when the fitted simple shape is an elliptical sphere. A flowchart of specific processing is shown in FIG. First, as shown in FIG. 14, the yh axis is set vertically downward, that is, opposite to the yr axis (step S311). Next, zh is set (step S312). The setting method of zh is based on the equations (4) to (6) as in the case of the cylinder end face instruction described above. In this case, point C is the vertex of an elliptic sphere. The unit vector xh in the X-axis direction is calculated by the above equation (7) as an outer product of the vector yh and the vector zh (step S313). Thereby, the posture of the hand 1 is obtained.

  In the case of an elliptical sphere, if the diameter is large, there is a possibility that the hand 1 cannot be grasped well with the posture of the hand 1 obtained here. Therefore, in steps S314 to S317, the hand posture is adjusted by the same method as in steps S216 to S219 described above. Here, step S317 differs from step S219 only in that the robot hand 1 is rotated about the center of the sphere around the xh axis. Accordingly, an appropriate gripping posture can be obtained even in the case of an elliptical sphere.

When the posture of the robot hand 1 is set in steps S11, 31, 41, the process proceeds to step S13, and the wrist position of the robot hand 1 is calculated. The center coordinate point in the enveloping grip is C (x _C , y _C , z _C ). This is the center point of the surface facing the robot hand 1 when gripping the cuboid, the center of the end surface when gripping the cylinder, and the elliptical sphere when gripping the cylinder laterally. If you want to be at the center. If the position coordinate of the wrist position w in the robot coordinate system Σr is w (x _w , y _w , z _w ), the wrist position coordinate w is obtained from Expression (13).

  From the obtained wrist position coordinates w and the hand posture, the target postures of the arm 7 and the trunk 8 are obtained by inverse kinematic analysis (step S15). A target gripping posture is obtained by controlling each motor 14 according to the obtained target posture (step S17). Then, gripping is performed by controlling the operation of the motor 14 so that the gripping force calculated from the output of the six-axis force sensor 15 becomes a target value (step S19). After gripping, the program shifts to another program, and a predetermined operation such as movement of the object and posture change is executed.

  Thus, since the principal axis direction of the object (target object) is obtained and the target posture of the robot hand 1 is directly determined therefrom, it is not necessary to preliminarily define a gripping point on the object. Since the hand 1 posture is finely adjusted according to the size of the object, the object can be reliably and symmetrically gripped. In addition, by setting an appropriate posture, it is possible to perform enveloping and gripping in which an object and a finger are brought into contact with each other at multiple points, thereby enabling more reliable gripping.

  Even when the applied simple shape is different from the actual shape of the object, the gripping point is not disposed on the object surface, and the robot hand 1 is disposed at a position for wrapping the gripping point. The number of cases in which gripping fails is reduced. Therefore, there is an advantage that even an object having a relatively complicated shape can be reliably gripped. In addition, since it is not necessary to set a gripping point, it is possible to reduce the size of the program for performing posture setting and to perform setting at high speed. For this reason, the responsiveness of the robot hand 1 is also improved.

  Furthermore, even if there is a slight difference between the fitted shape and the actual shape, the gripping force can be absorbed by controlling the gripping force so that it becomes the target value, so the accuracy of the fitting is improved. It is not necessary to have high accuracy, and the object can be gripped appropriately.

  Here, the method of determining the gripping point after applying the object to any one of a rectangular parallelepiped, an elliptical sphere, and a cylinder has been described, but the simple shape to apply is not limited to these three, and these three are It is not essential.

  Also, the number of fingers is not limited to four, and may be two, three, or five or more if there are two or more opposing fingers. Further, the combination of opposing fingers is not limited to the one-finger-finger combination as described above, and may be a combination of two fingers-2 fingers, two fingers-3 fingers, or the like.

  The gripping posture setting method is not limited to the above-described processing routine, and may be set as appropriate according to the arrangement of the fingers of the robot hand and the movable range thereof.

It is a figure which shows the structure of the robot hand to which the holding method of the arbitrarily-shaped object by the robot which concerns on this invention is applied. It is a block block diagram which shows the control system which performs the holding | grip method of the arbitrarily-shaped object by the robot which concerns on this invention. It is a schematic block diagram of the robot 100 which has the robot hand of FIG. It is a main flowchart of the control processing of the holding | grip method which concerns on this invention. It is a flowchart which shows the hand attitude | position determination process in the case of a rectangular parallelepiped. It is a figure which shows the hand attitude | position in the case of holding | griping a rectangular parallelepiped. It is a flowchart which shows the hand attitude | position determination process in the case of a cylinder. It is a figure which shows the hand attitude | position in the case of hold | gripping a cylinder. It is a figure explaining the setting method of the z-axis direction vector zh in the case of hold | gripping a cylinder. It is a figure explaining the difference in the gripping point by the diameter at the time of gripping a cylinder from the top. It is a figure explaining the adjustment method of the inclination of a hand. It is a figure explaining the link coordinate system of a finger. It is a flowchart which shows the hand posture determination process in the case of an elliptical sphere. It is a figure which shows the hand attitude | position in the case of holding | gripping an elliptical sphere.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Robot hand, 3 ... Camera eye, 4 ... Motor driver, 7 ... Arm, 8 ... Torso, 9 ... Target object, 10 ... Thumb, 11 ... Index finger, 12 ... Middle finger, 13 ... Ring finger, 14 ... Motor, 15 ... 6-axis force sensor, 16 ... Encoder potentiometer, 17 ... Interlocking joint, 20 ... Image recognition unit, 21 ... Grasping posture calculation unit, 22 ... Hand control unit, 100 ... Robot, Σr ... Robot coordinate system, Σs ... Hand Coordinate system.

Claims (1)

A method of gripping an arbitrarily shaped object by a robot having a visual sensor and a robot hand having multiple fingers and multiple joints,
Obtaining the principal axis direction and size, position, and orientation information of the target object by image recognition from the image of the target object acquired by the visual sensor;
Setting a target posture of the robot hand at the start of gripping based on the obtained spindle orientation, size, position, and posture information;
Adjusting the target posture according to the size of the object;
Calculating the wrist position of the hand based on the object center position;
A process of calculating the target posture of the robot arm and body using the inverse kinematics method;
And a step of operating each joint of the robot on the basis of the obtained target posture.

Images