JP6578671B2 - ROBOT, ROBOT CONTROL METHOD, AND ROBOT CONTROL DEVICE - Google Patents

Description

  The present invention relates to a robot, a robot control method, and a robot control apparatus.

  When the robot performs assembly work, if the position and orientation of the object being gripped deviate from the expected position and orientation, the assembly work is often adversely affected. As a means for detecting the deviation, it is effective to capture the object being grasped by the imaging unit and calculate the position / orientation relationship between the grasping grasping part and the grasped object. Since the deviation between the system and the imaging unit coordinate system differs depending on the location, the deviation amount is not uniform, and there is a possibility that the expected accuracy is not achieved with one correction amount. Therefore, there is a possibility that the gripped object may be slightly displaced even after the gripped object is gripped and the gripped object cannot be assembled.

  For example, a tool position correction method for an articulated robot that stores the amount of positional deviation in a database and uses the value during operation is disclosed (for example, see Patent Document 1).

JP 2006-82171 A

  However, in the tool position correction method described in Patent Document 1, when the amount of displacement is measured, the gripping unit is visible from the imaging unit. However, depending on the arrangement of the imaging unit such as a double-arm robot, this is not necessarily the case. There is a risk that the situation will not always be correct.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

  [Application Example 1] A robot according to this application example is a robot having a gripping unit, and is different from the first imaging information of the gripping unit that does not grip the gripping object at the first point and the first point. Operating based on second imaging information of the gripping part that does not grip the gripping object at a second point and third imaging information of the gripping object gripped by the gripping part at the first point. Features.

  According to this application example, the relationship between the gripping part and the gripped object gripped by the gripping part is determined at each of the positions of the first point (for example, the confirmation position) and the second point (for example, the target position). Since the imaging information is calculated, it is possible to reduce deviation during assembly (target position). Therefore, it is possible to provide a robot that enables highly accurate assembly.

  Application Example 2 In the robot according to the application example, it is preferable that the first imaging information, the second imaging information, and the third imaging information are position and orientation information.

  According to this application example, since each piece of imaging information is position / orientation information, it is possible to easily confirm the position / orientation between the grasping unit and the grasped object.

  Application Example 3 The robot according to the application example compares the first imaging information, the second imaging information, and the third imaging information with each expected value that is set in advance, and determines the result of the comparison. Accordingly, it is preferable to further include a fourth control unit that calculates each correction amount for correcting the shift amount.

  According to this application example, each piece of imaging information is corrected at the positions of the first point and the second point, so that it is possible to reduce deviation during assembly.

  Application Example 4 The robot according to the application example described above includes an imaging unit, the gripping unit is arranged in a first direction and a second direction, and a tip part that grips the gripped object; A plurality of claw portions having a base end portion disposed at a position in a third direction orthogonal to each of the first direction and the second direction with respect to a distal end portion, and at the first point, As for the posture of the gripping part imaged by the imaging part, it is preferable that the imaging part is arranged at a position in the third direction on the tip side of the claw part.

  According to this application example, it is possible to show the back surface of the gripped object (the surface that does not overlap the gripping part or the claw part) in the imaging direction of the imaging unit. Thereby, there is no shielding object between the imaging unit and the gripping object, and the gripping object itself is less likely to be hidden. As a result, the contour shape on the back side of the grasped object can be imaged.

  Application Example 5 In the robot according to the application example described above, it is preferable that a plurality of the imaging units are provided.

  According to this application example, by having a plurality of imaging units, it is possible to increase the resolution of an image by the imaging unit, and to create an accurate image. Thereby, a position and orientation with high positional accuracy can be obtained.

  Application Example 6 In the robot according to the application example, it is preferable that the imaging unit is a stereo camera.

  According to this application example, there is no shielding object between the imaging unit and the gripping object, and the gripping object itself is less likely to be hidden.

  Application Example 7 In the robot according to the application example described above, it is preferable that the second imaging information uses a shape of the grip portion or an image of a marker provided on the grip portion.

  According to this application example, it is possible to easily calculate the second imaging information.

  Application Example 8 A robot control method according to this application example is a robot control method for controlling a robot having a gripping unit, and the first imaging information of the gripping unit that does not grip a gripping object at a first point. Second imaging information of the gripping part that does not grip the gripping object at a second point different from the first point, and third imaging of the gripping object gripped by the gripping part at the first point The robot is controlled based on the information.

  According to this application example, the imaging information is calculated based on the positions of the first point and the second point regarding the relationship between the gripping part and the gripped object gripped by the gripping part. The deviation can be reduced. Therefore, it is possible to provide a robot control method for controlling a robot that enables highly accurate assembly.

  Application Example 9 In the robot control method according to the application example described above, the first imaging information, the second imaging information, and the third imaging information are compared with preset expected values, and the comparison is performed. It is preferable to further calculate each correction amount for correcting the shift amount according to the result.

  According to this application example, each piece of imaging information is corrected at the positions of the first point and the second point, so that it is possible to reduce deviation during assembly.

  Application Example 10 A robot control device according to this application example is a robot control device that controls a robot having a gripping portion, and the first imaging information of the gripping portion that does not grip a gripping object at a first point. And second imaging information of the gripper that does not grip the gripping object at a second point different from the first point, and a third of the gripped object gripped by the gripping part at the first point The robot is controlled based on the imaging information.

  According to this application example, the imaging information is calculated based on the positions of the first point and the second point regarding the relationship between the gripping part and the gripped object gripped by the gripping part. The deviation can be reduced. Therefore, it is possible to provide a robot control apparatus that controls a robot that enables highly accurate assembly.

  Application Example 11 The robot control device according to the application example compares the first imaging information, the second imaging information, and the third imaging information with preset expected values, and performs the comparison. It is preferable to further include a fourth control unit that calculates each correction amount for correcting the shift amount according to the result.

  According to this application example, each piece of imaging information is corrected at the positions of the first point and the second point, so that it is possible to reduce deviation during assembly.

The figure which shows an example of the utilization condition of the robot which concerns on 1st Embodiment. The figure which shows an example of the hardware constitutions of the control apparatus which concerns on 1st Embodiment. The figure which shows an example of a function structure of the control apparatus which concerns on 1st Embodiment. The figure explaining an example of the holding method of the object by the hand which concerns on 1st Embodiment. The figure which shows the hand in the confirmation position which concerns on 1st Embodiment. The figure which shows the hand in the target position which concerns on 1st Embodiment. The figure which shows the hand in the confirmation position which concerns on 1st Embodiment. The flowchart which shows an example of the flow of the pre-processing performed by the robot control part of the robot which concerns on 1st Embodiment. The flowchart which shows an example of the flow of the assembly | attachment process performed by the robot control part of the robot which concerns on 1st Embodiment. The figure which shows the position and orientation of the hand which concerns on 1st Embodiment. (A) is a figure which shows the target position of a hand, (B) is a figure which shows correction | amendment of the position and orientation of a hand. The figure which shows the movement position of the hand which concerns on 2nd Embodiment. The flowchart which shows an example of the flow of the assembly | attachment process performed by the robot control part of the robot which concerns on 2nd Embodiment.

(First embodiment)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, embodiments of the invention will be described with reference to the drawings. Note that the drawings to be used are appropriately enlarged or reduced so that the part to be described can be recognized.
FIG. 1 is a diagram illustrating an example of a usage status of a robot according to the present embodiment.
The robot 2 according to the present embodiment includes a camera (imaging unit) 10 and a control device 12. The imaging unit 10 is mounted on the robot 2. The camera 10 images the grasped object (gripping object) OBJ (see FIG. 7). The camera 10 captures an image by changing the position and orientation of the object OBJ gripped by the hand (grip unit) HND1.

  A plurality of cameras 10 may be provided. According to this, by having a plurality of cameras 10, the resolution of the image by the cameras 10 can be increased, and an image with high accuracy can be created. Thereby, a position and orientation (three-dimensional position and orientation) with high position accuracy can be obtained.

  The camera 10 is a camera including, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, which is an image sensor that converts collected light into an electrical signal. The camera 10 is, for example, a stereo camera composed of two cameras, but may be composed of three or more cameras, or one camera captures a two-dimensional image. It is good. The camera is movable and can be moved vertically and horizontally. According to this, there is no shielding object between the camera 10 and the object OBJ, and it is less likely that the object OBJ is hidden by the hand HND1 itself.

  The camera 10 is communicably connected to the control device 12 by a cable, for example. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB (Universal Serial Bus), for example. Note that the camera 10 and the control device 12 may be connected by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). The object OBJ is previously installed on the installation surface M. The “installation surface M” is, for example, a surface on a table. The camera 10 is installed at a position where the object OBJ can be imaged. The camera 10 images the object OBJ and outputs a captured image of the captured object OBJ to the control device 12 through communication.

  For example, as shown in FIG. 1, the robot 2 includes a hand HND1, a hand HND2, a force sensor 14, an arm part ARM1, an arm part ARM2, and a plurality of actuators (not shown). It is a double-armed robot equipped on the arm. The base axis of the robot has a rotation axis and rotates. Each arm of the robot 2 is a 6-axis vertical articulated type, and one arm performs an operation with 6 axes of freedom by an operation in which a support base, an arm part ARM1, and a gripping part HND1 are linked by an actuator. The other arm is a support base, the arm part ARM2, and the hand HND2 can be operated in a 6-axis degree of freedom by an operation in which the actuators cooperate with each other. Each arm of the robot 2 may operate with 5 degrees of freedom (5 axes) or less, or may operate with 7 degrees of freedom (7 axes) or more. Hereinafter, the operation of the robot 2 performed by the arm including the hand HND1 and the arm unit ARM1 will be described. However, the same operation may be performed by the arm including the hand HND2 and the arm unit ARM2. The hand HND1 holds the object OBJ. The “hand HND1” is an example of the “gripping part” in the claims. The robot 2 is communicably connected to the control device 12 by a cable, for example. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The robot 2 and the control device 12 may be connected by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). The robot 2 shown in FIG. 1 is a dual-arm robot, but may be implemented as a single-arm robot.

  The hand HND1 of the robot 2 includes a claw portion 52 that can hold or clamp the object OBJ (see FIG. 4). The force sensor 14 is provided between the hand HND1 of the robot 2 and the arm part ARM1, and detects a force and a moment acting on the hand HND1. The force sensor 14 outputs information indicating the detected force and moment to the control device 12 by communication. Information indicating the force and moment detected by the force sensor 14 is used, for example, for impedance control of the robot 2 by the robot control unit 16.

  The robot 2 acquires a control signal based on the three-dimensional position and orientation of the object OBJ from the control device 12, and performs a predetermined operation on the object OBJ based on the acquired control signal. The predetermined work refers to, for example, holding the object OBJ with the hand HND1 of the robot 2 and moving the gripped object OBJ from the currently installed position to another position, For example, assembling the device.

  The camera 10 may be installed other than the hand HND1 that holds the object OBJ. According to this, when the object OBJ is gripped, it is possible to avoid that only a partial shape of the object OBJ can be captured due to the positional relationship of the object OBJ with the camera 10.

  The control device 12 performs image processing on an image captured by the camera 10. The control device 12 calculates the position and orientation of the object OBJ. The control device 12 controls the robot 2 to perform a predetermined work. More specifically, the control device 12 derives the three-dimensional position and orientation of the object OBJ based on the captured image of the object OBJ captured by the camera 10. Then, the control device 12 causes the robot 2 to grip the object OBJ with the hand HND1 based on the derived three-dimensional position and posture of the object OBJ. Thereafter, the control device 12 controls the robot 2 so as to perform a predetermined operation on the grasped object OBJ.

Next, the hardware configuration of the control device 12 will be described with reference to FIG.
FIG. 2 is a diagram illustrating an example of a hardware configuration of the control device 12 according to the present embodiment. The control device 12 includes, for example, a CPU (Central Processing Unit) 20, a storage unit 22, an input reception unit 24, and a communication unit 26, and communicates with the camera 10 and the like via the communication unit 26. These components are connected to each other via a bus Bus so that they can communicate with each other. The CPU 20 executes various programs stored in the storage unit 22. The storage unit 22 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), and the like. Various information, images, and programs processed by the control device 12 are stored. The storage unit 22 may be an external storage device connected via a digital input / output port such as a USB instead of the one built in the control device 12.

  The input receiving unit 24 is, for example, a keyboard, mouse, touch pad, or other input device. The input receiving unit 24 may function as a display unit, and may be configured as a touch panel. The communication unit 26 includes, for example, a digital input / output port such as USB, an Ethernet (registered trademark) port, and the like.

Next, the functional configuration of the control device 12 will be described with reference to FIG.
FIG. 3 is a diagram illustrating an example of a functional configuration of the control device 12 according to the present embodiment. The control device 12 includes, for example, an image acquisition unit 30, a three-dimensional position / orientation derivation unit 32, and a robot control unit 16. Some or all of these functional units are realized by, for example, the CPU 20 executing various programs stored in the storage unit 22. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

  The image acquisition unit 30 acquires a captured image captured by the camera 10 and outputs the acquired captured image to the three-dimensional position / orientation deriving unit 32. Note that the image acquisition unit 30 may store the acquired captured image in the storage unit 22, and the three-dimensional position and orientation deriving unit 32 may read the captured image from the storage unit 22. The three-dimensional position / orientation deriving unit 32 derives the three-dimensional position and orientation of the object OBJ based on the captured image acquired from the image acquisition unit 30. Then, the 3D position / orientation deriving unit 32 outputs the derived 3D position and orientation of the object OBJ to the robot control unit 16.

  Based on the three-dimensional position and orientation of the object OBJ acquired from the three-dimensional position and orientation deriving unit 32, the robot control unit 16 controls the robot 2 so that the robot 2 grips the object OBJ with the hand HND1.

The robot control unit 16 includes registration means, gripping means, moving means, correction means, and assembling means.
The registration unit registers the target position of the hand HND1 so that the position and orientation of the object OBJ is assembled to the assembly surface MS (see FIG. 10A).
The gripping means grips the object OBJ with the hand HND1.
The moving means moves the contour shape of the object OBJ to a position where it can be imaged by the camera 10 while holding the object OBJ (see FIG. 11).
The correcting unit corrects the target position of the hand HND1 so that the position and orientation of the object OBJ are assembled to the assembly surface MS (see FIG. 10B).
The assembling means assembles the object OBJ on the assembling surface MS (see FIG. 10B).

The robot control unit 16 includes a first control unit, a second control unit, a third control unit, and a fourth control unit.
The first control unit uses the image of the hand HND1 that is captured by the camera 10 and does not hold the object OBJ at the confirmation position (first point) set on the movement path of the hand HND1 (first position). Imaging information) is calculated.
The second control unit uses the image of the hand HND1 that is captured by the camera 10 and does not hold the object OBJ at the target position (second point) set to a position different from the confirmation position on the movement path. (Second imaging information) is calculated.
The third control unit calculates position / orientation information (third imaging information) using the image of the object OBJ captured by the camera 10 at the confirmation position and held by the hand HND1.

  The fourth control unit compares each position and orientation information with each expected value set in advance, and calculates each correction amount for correcting the shift amount according to the comparison result. According to this, since each position and orientation information is corrected by the confirmation position and the target position, it is possible to reduce deviation during assembly. Note that the correction amount database may be acquired from the movement route (operating range) of the hand HND1. According to this, the error is reduced, and the assembly can be performed in consideration of the gripping error even after gripping the object OBJ.

  FIG. 4 is a diagram for explaining an example of a method of gripping the object OBJ by the hand HND1 according to the present embodiment. 4 is a cross-sectional view of the positional relationship between the hand HND1 and the object OBJ installed on the installation surface M as viewed from the side with respect to the installation surface M. 4 shows the positional relationship between the hand HND1 and the object OBJ installed on the installation surface M directly above the installation surface M (for example, the surface side on which the object OBJ is installed). It is the figure seen from. The installation surface M need not be a surface orthogonal to the vertical direction like the table surface as long as the object OBJ is installed, and may be a wall surface, for example.

As shown in FIG. 4, the hand HND1 is arranged side by side in the first direction (x) and the second direction (y), and the tip portion 54 that grips the object OBJ, and the first direction with respect to the tip portion 54 A plurality of claw portions 52 having a base end portion 56 disposed at a position in a third direction (z) orthogonal to each of (x) and the second direction (y) is provided.
Here, a method of gripping the object OBJ by the hand HND1 will be described with reference to FIG.

  First, the robot control unit 16 moves the hand HND1 of the robot 2 to the position shown in the lower diagram of FIG. 4 on the installation surface M based on the three-dimensional position and posture of the object OBJ. Here, the coordinate axes in the upper and lower stage diagrams of FIG. 4 are coordinate axes for indicating the positional relationship between the installation surface M and the object OBJ, and are not coordinate axes on the robot coordinate system or the captured image. When the object OBJ is installed on the installation surface M, the position of the hand HND1 shown in the lower diagram of FIG. 4 is directly above the object OBJ. Then, the robot control unit 16 moves the hand HND1 in a direction (z direction) indicated by an arrow in the upper diagram of FIG. 4 to move the object OBJ to a place where the hand HND1 can be gripped.

  The robot control unit 16 controls the robot 2 to perform a predetermined operation after the hand HND1 grips the object OBJ.

(Advance preparation)
FIG. 5 is a diagram showing the hand HND1 at the confirmation position according to the present embodiment. FIG. 6 is a diagram illustrating the hand HND1 at the target position according to the present embodiment.
First, the hand HND1 is moved to a confirmation position where the object OBJ to be grasped is not grasped, and the position / orientation information (first imaging information) of the hand HND1 is acquired. Thereby, the position and orientation information of the hand HND1 at the position where the object OBJ to be grasped is confirmed can be acquired. The position / orientation information is, for example, three-dimensional position / orientation information. In the confirmation of the position and orientation information of the hand HND1, the position and orientation of the palm of the hand HND1 are acquired as shown in FIG. The position and orientation information of the hand HND1 may be confirmed by measuring the part indicated by the arrow A in FIG. The position where the hand HND1 is moved may be, for example, on a straight line connecting an intersection of optical axes of the camera 10 to be described later and a base line between the cameras 10. It is possible to know the relationship between the position and orientation information of the hand HND1 obtained by forward kinematics and the position and orientation information of the hand HND1 in the camera coordinate system.

  Next, the hand HND1 is moved to a target position near the assembly surface MS, and position and orientation information (second imaging information) of the hand HND1 is acquired. The position and orientation information of the hand HND1 may be confirmed by measuring the part indicated by the arrow B in FIG. In addition, at the assembly position, it may be difficult to observe the hand HND1 from the stereo camera on the head due to arm shielding or the like. Therefore, as shown in FIG. 6, a jig is used so that a part of the jig 34 can be observed from the camera 10. For example, the marker 36 is attached to a part of the jig 34 and the marker 36 is measured by the camera 10. In this case, a straight line is calculated from the two markers 36 of the jig 34. By holding the jig 34 in two directions and finding the intersection of two straight lines, it is set as the position of the hand HND1. Since the surface can be formed from two straight lines, the direction of the surface can be the palm direction of the hand HND1. Similarly to the above, it is possible to know the relationship between the position and orientation information of the hand HND1 obtained by forward kinematics and the position and orientation information of the hand HND1 in the camera coordinate system. According to this, it is possible to easily calculate the position and orientation information of the hand HND1.

(During actual operation)
FIG. 7 is a diagram showing the hand HND1 at the confirmation position according to the present embodiment.
While holding the object OBJ, the hand HND1 is moved to a confirmation position for checking the object OBJ, and position / orientation information (third imaging information) of the gripping object OBJ is acquired. Thereby, in the camera coordinate system, the relationship between the position and orientation information of the hand HND1 and the position and orientation information of the grasped object OBJ can be acquired. In the confirmation of the position and orientation information of the hand HND1, as shown in FIG. 7, the palm position and orientation of the hand HND1 are acquired.

  Further, the position and orientation information is corrected using the relationship with the position and orientation information of the hand HND1 obtained by forward kinematics obtained in advance preparation. It is assumed that there is robot repeatability.

  Next, the hand of the hand HND1 is moved to the target position. The movement amount is set in consideration of the correction amount. The moved position is a position and orientation in consideration of the correction amount calculated in advance preparation.

  Note that, at the confirmation position, the posture of the hand HND1 imaged by the camera 10 is preferably such that the camera 10 is disposed at a position in the third direction (z) on the distal end portion 54 side of the claw portion 52. According to this, it is possible to show the back surface of the object OBJ (the surface that does not overlap the hand HND1 or the claw portion 52) in the imaging direction of the camera 10. Thereby, there is no obstruction between the camera 10 and the object OBJ, and it is less likely that the object OBJ is hidden by the hand HND1 itself. As a result, the contour shape on the back side of the object OBJ can be imaged.

(Example)
As an embodiment, an operation of assembling a screw fastening plate into a screw fastening base will be described.
FIG. 8 is a flowchart illustrating an example of a flow of pre-processing executed by the robot control unit of the robot according to the present embodiment. FIG. 9 is a flowchart illustrating an example of the flow of the assembly process executed by the robot control unit of the robot according to the present embodiment. FIG. 10 is a diagram illustrating the position and orientation of the hand HND1 according to the present embodiment. FIG. 10A is a diagram illustrating a target position of the hand HND1, and FIG. 10B is a diagram illustrating correction of the position and orientation of the hand HND1. The object OBJ is a screw tightening plate, and the assembly surface MS is an assembly surface of the screw tightening base. W represents world coordinates, and T represents coordinate conversion.

  Hereinafter, a process performed when the robot control unit 16 assembles the object OBJ on the robot 2 on the assembly surface MS will be described with reference to FIG. In the following description, it is assumed that the hand HND1 of the robot 2 is moved to the position just above the object OBJ by the robot control unit 16.

  First, as shown in FIG. 8, in step S10, the hand HND1 is moved to a confirmation position for confirming the object OBJ to be grasped.

Next, in step S20, position and orientation information (first imaging information) of the confirmation position (first point) of the hand HND1 is acquired. FIG. 10B shows the work as ^W _{THND1 ′} . According to this, since the first imaging information is position and orientation information, the position and orientation of the hand HND1 can be easily confirmed.

  Next, in step S30, the position and orientation information of the confirmation position of the hand HND1 is registered as the prior position and orientation 1.

  Next, in step S40, the hand HND1 is moved near the target position where the object OBJ to be grasped is assembled.

Next, in step S50, position and orientation information (second imaging information) of the target position (second point) of the hand HND1 is acquired. FIG. 10B shows the work as ^W _THND1 . According to this, since the second imaging information is position and orientation information, the position and orientation of the hand HND1 can be easily confirmed.

  Next, in step S60, the position and orientation information of the target position of the hand HND1 is registered as the prior position and orientation 2.

  Hereinafter, a process performed when the robot control unit 16 assembles the object OBJ on the robot 2 on the assembly surface MS will be described with reference to FIG. 9. In the following description, it is assumed that the hand HND1 of the robot 2 is moved to the position just above the object OBJ by the robot control unit 16.

First, as shown in FIG. 9, in step S110, registration of the object OBJ is performed. Specifically, the position and orientation of the object OBJ with respect to the assembly surface MS is designated. FIG. 10B shows the work as ^MS T _OBJ . The position and orientation of the assembly surface MS at the world coordinate W is designated. In FIG. 10 (B), shows the work as ^W T _MS. The position and orientation of the object OBJ at the world coordinates W are calculated. For example, it is possible to represent the work as _{^{W T OBJ = W T MS MS}} T OBJ. The position and orientation information of the confirmation position of the object OBJ is acquired. For example, FIG. 10B shows the work as ^W T _{OBJ ′} .

  Next, in step S120, the object OBJ is gripped.

  Next, in step S130, the hand HND1 is moved to a confirmation position for confirming the grasped object OBJ.

  Next, in step S140, position / orientation information (third imaging information) of the grasped object OBJ is acquired. According to this, since the third imaging information is the position and orientation information, the position and orientation of the object OBJ can be easily confirmed.

Next, in step S150, the position and orientation of the hand HND1 are corrected using the prior position and orientation 1 of the hand HND1. The correction amount of the confirmation position of the hand HND1 is calculated using the prior position and orientation 1 of the hand HND1. FIG. 10B shows the operation as T _{HND1 ′} . The confirmation position of the hand HND1 is corrected. For example, the operation can be expressed as T _{HND1 ′} ^W T _{HND1 ′} .

  Next, in step S160, the relationship between the position and orientation of the hand HND1 and the position and orientation of the grasped object OBJ is calculated.

Next, in step S170, the position of the hand HND1 that moves the grasped object OBJ to the target position is calculated. A conversion for moving the object OBJ to the target position is calculated. For example, it is possible ^{to _{W T OBJ' = T W T OBJ}} , as ^{_{T = W T OBJ' W T OBJ}} -1 representing that task. The target position of the hand HND1 is calculated. For example, it is possible to represent the work as ^{_{W T HND1 = TT HND1' W T}} HND1'.

Next, in step S180, the position and orientation of the hand HND1 are corrected using the prior position and orientation 2 of the hand HND1. The correction amount of the target position of the hand HND1 is calculated using the prior position and orientation 2 of the hand HND1. FIG. 10B shows the work as _THND1 . The target position of the hand HND1 is corrected. For example, the work can be expressed as T _HND1 ^W T _HND1 .

  Next, in step S190, the hand HND1 is moved to the corrected target position.

  Next, in step S200, an assembling operation is performed. According to this, the screw fastening plate can be moved to a position closer to the ideal.

  As described above, the robot 2 according to the present embodiment calculates the position and orientation information of the relationship between the hand HND1 and the object OBJ held by the hand HND1 based on the confirmation position and the target position. , It is possible to reduce the deviation during assembly. Therefore, it is possible to provide the robot 2, the control method for the robot 2, and the control device 12 for the robot 2 that enable highly accurate assembly.

(Second Embodiment)
The robot 2 of the present embodiment is different from the first embodiment in that the object OBJ that has been gripped is imaged by temporarily changing the position and orientation of the object OBJ to the imaging region of the camera 10. Hereinafter, the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted or simplified here.

FIG. 11 is a diagram illustrating a movement position of the hand HND1 according to the present embodiment.
The plurality of cameras 10 may be composed of two cameras 10. The object OBJ held by the hand HND1 may be on a vertical line 44 that is lowered on a straight line 48 connecting the positions of the two cameras 10 from the intersection 42 of the optical axes 40 of the two cameras 10.

  Further, the center of gravity 46 of the object OBJ gripped by the hand HND1 may be on a vertical line 44 drawn on a straight line 48 connecting the positions of the two cameras 10 from the intersection 42 of the optical axes 40 of the two cameras 10. Good.

  Further, the opening / closing direction of the claw portion 52 of the hand HND1 that holds the object OBJ is the normal direction of the plane 50 including the intersection 42 of the optical axes 40 of the two cameras 10 and the positions of the two cameras 10. May be. According to these, there is no shielding object between the camera 10 and the object OBJ, and it is less likely that the object OBJ is hidden by the hand HND1 itself.

Hereinafter, with reference to FIG. 12, a process performed by the robot control unit 16 when the object OBJ is assembled to the robot 2 on the assembly surface MS will be described.
FIG. 12 is a flowchart showing an example of the flow of the assembling process executed by the robot control unit 16 of the robot 2 according to this embodiment.

  First, as shown in FIG. 12, in step S210, as shown in FIG. 10A, the target position of the hand HND1 is stored in the storage unit 22 so as to be in a position / posture for assembling the object OBJ on the assembling surface MS as a registration process. Register with.

  Next, in step S <b> 220, data for detecting the object OBJ and the installation surface M is registered in the storage unit 22. Alternatively, the three-dimensional position and orientation of the object OBJ with respect to the installation surface M is registered in the storage unit 22.

  Next, in step S230, the installation surface M and the three-dimensional position and orientation of the object OBJ are acquired from the storage unit 22.

  Next, in step S240, the object OBJ is gripped by the hand HND1 as a gripping process. Then, the position and orientation at which the grasped object OBJ can be observed with the camera 10 are calculated.

  Next, in step S250, the object OBJ is moved to a position where it can be imaged by the camera 10 while the object OBJ is held as a moving process. Then, the gripping object OBJ is moved in front of the camera 10 to detect the three-dimensional position and orientation of the gripping object OBJ.

  Next, in step S260, the three-dimensional position and orientation of the object OBJ are acquired from the camera 10.

  Next, in step S270, the position and orientation of the hand HND1 holding the object OBJ is acquired from the storage unit 22.

  Next, in step S280, the relationship (the deviation from the set value) between the three-dimensional position and orientation of the object OBJ and the hand HND1 is acquired.

  Next, in step S290, the three-dimensional position and orientation of the assembly surface MS are acquired from the storage unit 22.

  Next, in step S300, the target position of the hand HND1 is corrected so that the position and orientation of the object OBJ are assembled to the assembly surface MS as a correction process. A target position of the hand HND1 in which the deviation from the set value is corrected is calculated. The position and orientation of the hand HND1 are changed so that the grasped object OBJ becomes the target position.

  Next, in step S310, the object OBJ is assembled to the assembly surface MS as an assembly process. By correcting the position and orientation of the hand HND1 with respect to the inclination (displacement) of the object OBJ with respect to the hand HND1, as shown in FIG. 10B, the inclination of the object OBJ with respect to the assembly surface MS is minimized. Can do. And it ends.

  As described above, the robot 2 according to the present embodiment images the grasped object OBJ once in the imaging area of the camera 10 while changing the position and orientation of the object OBJ. Thereby, when photographing, it is possible to capture an image with the amount of the object OBJ hidden by the hand HND1 of the robot 2 as small as possible. Then, by converting the camera coordinates of the object OBJ into world coordinates, the positional relationship between the object OBJ and the hand HND1 can be grasped. As a result, when the robot 2 is gripping the object OBJ, it is possible to correct the positional deviation between the object OBJ and the hand HND1 and to perform assembly with little error.

  When the relationship between the position and orientation of the hand HND1 and the camera 10 and the line-of-sight direction of the camera 10 are known, in order to check the state of the object OBJ held by the robot, the line of sight of the camera 10 is entered. The object OBJ may be moved. At this time, it is desirable that there are as few shields as possible between the camera 10 and the object OBJ, and it is preferable that the hand HND1 itself hides the object OBJ.

Further, by comparing the relationship between the detected position and orientation of the object OBJ and the position and orientation of the hand HND1, it is possible to detect whether or not the relationship deviates from the expected relationship registered in the database.
In addition, since the imaging unit can also be moved by rotating the head and waist, the object can be imaged without taking an unreasonable posture so that the position and posture of the object can be accurately acquired. Become.

In addition, embodiment is not limited above, It can also implement with the following forms.
In the embodiment described above, the opening / closing direction of the claw portion 52 of the hand HND1, the intersection 42 of the optical axis 40 of the imaging unit 10, and the normal line of the plane 50 including the two camera 10 positions are not limited to coincide with each other. For example, depending on the gripping state, the angle between the two may be approximately 90 degrees.

  Further, since the shape of the detected object OBJ is known, the relationship between the three-dimensional position and orientation (deviation from the set value) between the object OBJ and the hand HND1 may be acquired from a partial image of the object OBJ.

  Furthermore, the intersection 42 of the optical axis 40 of the camera 10 may be calculated by the midpoint method or the like because the optical axes 40 rarely intersect each other.

  In addition, when combined with the visual servo, the visual servo operation start position is stabilized, so high-precision assembly can be expected.

  The robot 2, the control method of the robot 2, and the control device 12 of the robot 2 have been described based on the illustrated embodiment. However, the present invention is not limited to this, and the configuration of each part is the same. It can be replaced with any configuration having the above function. In addition, any other component may be added to the present invention.

  DESCRIPTION OF SYMBOLS 2 ... Robot 10 ... Camera (imaging part) 12 ... Control apparatus 14 ... Force sensor 16 ... Robot control part 20 ... CPU 22 ... Memory | storage part 24 ... Input reception part 26 ... Communication part 30 ... Image acquisition part 32 ... Three-dimensional position and orientation Lead-out part 34 ... Jig 36 ... Marker 40 ... Optical axis 42 ... Intersection 44 ... Vertical 46 ... Center of gravity 48 ... Linear 50 ... Plane 52 ... Claw part 54 ... Tip part 56 ... Base end part ARM1, ARM2 ... Arm part Bus ... Bus HND1, HND2 ... hand (gripping part) M ... installation surface MS ... assembly surface OBJ ... object (gripping object).

Claims (8)

A robot having a gripping portion,
First imaging information of the gripper that does not grip the gripping object at the first point;
Second imaging information of the gripping part that does not grip the gripping object at a second point different from the first point;
Third imaging information of the grasped object grasped by the grasping unit at the first point;
It operates based on the,
A fourth control unit that compares the first imaging information, the second imaging information, and the third imaging information with a preset expected value and calculates a correction amount that corrects the shift amount according to the comparison result. robot, characterized in that it comprises a. The robot according to claim 1, wherein
The robot according to claim 1, wherein the first imaging information, the second imaging information, and the third imaging information are position and orientation information. The robot according to claim 1 or 2 ,
Having an imaging unit,
The grip portion is arranged side by side in a first direction and a second direction, and a distal end portion that grips the gripped object and a first portion that is orthogonal to the first direction and the second direction with respect to the distal end portion. A plurality of claw portions having a base end portion disposed at a position in three directions;
In the first point, the posture of the gripping part imaged by the imaging part is characterized in that the imaging part is arranged at a position in the third direction on the tip side of the claw part. robot. The robot according to claim 3 , wherein
A robot characterized in that a plurality of the imaging units are provided. The robot according to claim 3 or 4 ,
The robot according to claim 1, wherein the imaging unit is a stereo camera. The robot according to any one of claims 1 to 5 ,
The robot according to claim 2, wherein the second imaging information uses a shape of the gripper or an image of a marker provided on the gripper. A robot control method for controlling a robot having a gripping part,
First imaging information of the gripper that does not grip the gripping object at the first point;
Second imaging information of the gripping part that does not grip the gripping object at a second point different from the first point;
Third imaging information of the grasped object grasped by the grasping unit at the first point;
To control the robot based on the,
Comparing the first imaging information, the second imaging information, and the third imaging information with preset expected values, and calculating a correction amount for correcting a shift amount according to a comparison result. To control the robot. A robot control device for controlling a robot having a gripping portion,
First imaging information of the gripper that does not grip the gripping object at the first point;
Second imaging information of the gripping part that does not grip the gripping object at a second point different from the first point;
Third imaging information of the grasped object grasped by the grasping unit at the first point;
To control the robot based on the,
A fourth control unit that compares the first imaging information, the second imaging information, and the third imaging information with a preset expected value and calculates a correction amount that corrects the shift amount according to the comparison result. control apparatus for a robot, characterized in that it comprises a.

Images