JP2017052073A - Robot system, robot and robot control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot system that allows a hand to grip an object in a suitable hand state when causing the hand to grip the object.SOLUTION: A robot system comprises: a robot comprising a hand; and a robot control device for determining a hand state based on a position and a posture of an object, error information of the position and the posture and the hand state.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a robot system, a robot, and a robot control apparatus.

  Research and development have been conducted on techniques for causing a robot to hold an object based on a captured image obtained by imaging the object.

  In this regard, a grip pattern with a high gripping quality score, which is a scalar value indicating how firmly the target can be gripped by the robot, is selected from grip patterns that are patterns that the robot grips the target. There is a known method (see Patent Document 1).

JP2013-144355A

  However, the above-mentioned Patent Document 1 does not show a specific method for selecting a grip pattern suitable for the state of the robot hand, and grips an object using the grip pattern suitable for the state of the robot hand. It was sometimes difficult.

In order to solve at least one of the above problems, an embodiment of the present invention provides a robot including a hand, the position and orientation of an object, error information on the position and orientation, and the state of the hand. A robot control device that determines the state of the robot.
With this configuration, the robot system determines the hand state based on the position and posture of the object, error information on the position and posture, and the hand state. Accordingly, the robot system can cause the hand to hold the object according to the state of the hand suitable for causing the hand to hold the object.

In another aspect of the present invention, in the robot system, a configuration may be used in which the robot control device detects the position and orientation of the object from the image.
With this configuration, the robot system detects the position and orientation of the object from the image. As a result, the robot system can cause the hand to hold the object according to the hand state determined based on the position and orientation of the object detected from the image.

According to another aspect of the present invention, in the robot system, the robot control device evaluates the hand state based on the position and orientation of the object, the error information, and the hand state. A configuration may be used in which the state of the hand is determined according to.
With this configuration, the robot system determines the hand state according to the evaluation value for evaluating the hand state based on the position and orientation of the object, error information, and the hand state. Thereby, the robot system 1 can make a hand hold | grip a target object with the state of the hand determined according to the evaluation value which evaluates the state of a hand.

According to another aspect of the present invention, in the robot system, the evaluation value is a change in a relative position and posture of the hand and the object when the object is gripped by the hand. A configuration that is a value indicating difficulty may be used.
With this configuration, when the object is gripped by the hand, the robot system determines the state of the hand according to the evaluation value, which is a value indicating the difficulty of changing the relative position and posture between the hand and the object. To decide. Thereby, when the object is gripped by the hand, the robot system determines the hand determined according to the evaluation value, which is a value indicating the difficulty of changing the relative position and posture between the hand and the object. The object can be gripped depending on the state.

In another aspect of the present invention, in the robot system, the evaluation value includes a value indicating difficulty in translation of the object relative to the hand when the object is gripped by the hand. A configuration may be used.
With this configuration, when the object is gripped by the hand, the robot system determines the state of the hand based on an evaluation value including a value indicating the difficulty of translation of the object with respect to the hand. Thus, when the object is grasped by the hand, the robot system grasps the object according to the state of the hand determined according to the evaluation value including the value indicating the difficulty of translation of the object with respect to the hand. be able to.

In another aspect of the present invention, in the robot system, the evaluation value includes a value indicating difficulty in rotation of the object relative to the hand when the object is gripped by the hand. A configuration may be used.
With this configuration, when the object is gripped by the hand, the robot system determines the state of the hand based on an evaluation value including a value indicating the difficulty of rotating the object with respect to the hand. Thus, when the object is grasped by the hand, the robot system grasps the object according to the state of the hand determined based on the evaluation value including the value indicating the difficulty of rotation of the object with respect to the hand. be able to.

According to another aspect of the present invention, in the robot system, a configuration in which the state of the hand includes a relative position and posture of the hand and the target object may be used.
With this configuration, the robot system determines the state of the hand based on the position and orientation of the object, error information on the position and orientation, and the state of the hand including the relative position and orientation of the hand and the object. To do. Thereby, the robot system can hold the object according to the hand state determined based on the hand state including the relative position and posture between the hand and the object.

Further, according to another aspect of the present invention, in the robot system, a configuration may be used in which the state of the hand includes an angle between gripping parts included in the hand.
With this configuration, the robot system determines the state of the hand based on the position and posture of the object, error information on the position and posture, and the state of the hand including the angle between the gripping parts included in the hand. Thereby, the robot system can hold the object according to the state of the hand determined based on the state of the hand including the angle between the holding parts included in the hand.

According to another aspect of the present invention, in the robot system, the error information may be information indicating a probability distribution.
With this configuration, the robot system determines the hand state based on the position and orientation of the object, error information that is information indicating the probability distribution, and the hand state. As a result, the robot system can grip the target object according to the hand state determined based on the position and orientation of the target object, information indicating the probability distribution, and the hand state.

According to another aspect of the present invention, in the robot system, a configuration may be used in which the error information is information indicating an error range.
With this configuration, the robot system determines the state of the hand based on the position and orientation of the object, error information that is information indicating an error range, and the state of the hand. As a result, the robot system can hold the target object according to the hand state determined based on the position and orientation of the target object, the information indicating the error range, and the hand state.

Another aspect of the present invention is a robot that includes a hand and determines the state of the hand based on the position and orientation of an object, error information on the position and orientation, and the state of the hand.
With this configuration, the robot determines the hand state based on the position and posture of the target object, error information on the position and posture, and the hand state. Accordingly, the robot can cause the hand to hold the object according to the state of the hand suitable for causing the hand to hold the object.

Another aspect of the present invention is a robot control device that determines the state of the hand based on the position and posture of the object, error information on the position and posture, and the state of the hand included in the robot.
With this configuration, the robot control apparatus determines the hand state based on the position and posture of the object, error information on the position and posture, and the hand state. Thereby, the robot control apparatus can make a hand hold | grip a target object with the state of a hand suitable when making a hand hold | grip a target object.

  As described above, the robot system, the robot, and the robot control device determine the hand state based on the position and posture of the object, the error information on the position and posture, and the hand state. Thereby, the robot system, the robot, and the robot control apparatus can cause the hand to hold the object according to the state of the hand suitable for causing the hand to hold the object.

It is a block diagram which shows an example of the robot 20 which concerns on this embodiment. 2 is a diagram illustrating an example of a hardware configuration of a robot control device 30. FIG. 2 is a diagram illustrating an example of a functional configuration of a robot control device 30. FIG. It is a flowchart which shows an example of the flow of the process which the control part performs. It is a figure for demonstrating the state of 1st end effector E1. It is the figure which showed each of the target object position and posture visually. FIG. 6 is a diagram illustrating the maximum vertical drag that can be supported by each gripping part of the vertical drag applied from the object O to each gripping part included in the first end effector E1. FIG. 8 is a diagram in which a friction cone is drawn with respect to arrows representing the vertical drags f1 to f3 shown in FIG. 7. It is a figure which shows an example of the convex hull which the grip quality evaluation value calculation part 46 produces | generates. It is the figure which illustrated the normal force and frictional force which are added from the target object O with respect to each holding part with which the 1st end effector E1 is equipped.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram illustrating an example of a robot 20 according to the present embodiment.
First, the configuration of the robot 20 will be described.

The robot 20 is a double arm robot including a first arm and a second arm. The double-arm robot is a robot having two arms (arms) such as the first arm and the second arm in this example. The robot 20 may be a single arm robot instead of the double arm robot. A single arm robot is a robot provided with one arm. For example, the single-arm robot includes one of the first arm and the second arm in this example.
The robot 20 includes a first imaging unit 21, a second imaging unit 22, a third imaging unit 23, a fourth imaging unit 24, and a robot control device 30.

  The first arm includes a first end effector E1, a first manipulator M1, a first force detector 11, and a first imaging unit 21.

  The first end effector E1 includes two or more gripping parts capable of gripping an object and a plurality of actuators (not shown). Hereinafter, as an example, a case will be described in which the first end effector E1 includes three gripping parts including a gripping part C11, a gripping part C12, and a gripping part C13.

  Each of the plurality of actuators included in the first end effector E1 is connected to the robot controller 30 via a cable so as to be communicable. Accordingly, the actuator operates each of the gripping part C11, the gripping part C12, and the gripping part C13 based on the control signal acquired from the robot control device 30. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB (Universal Serial Bus), for example. The actuator may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The first manipulator M1 is a seven-axis vertical articulated manipulator including seven joints, a first imaging unit 21, and a plurality of actuators (not shown). Accordingly, the first arm performs a motion with seven degrees of freedom by the operation of the support base, the first end effector E1, and the first manipulator M1 coordinated by the actuator. The first arm may be configured to operate with a degree of freedom of 6 axes or less, or may be configured to operate with a degree of freedom of 8 axes or more.

  When the first arm operates with a degree of freedom of seven axes, the posture that the first arm can take is increased as compared with a case where the first arm operates with a degree of freedom of six axes or less. Thereby, for example, the first arm can be smoothly operated, and interference with an object existing around the first arm can be easily avoided. In addition, when the first arm operates with a degree of freedom of 7 axes, the control of the first arm is easy and requires a smaller amount of calculation than when the first arm operates with a degree of freedom of 8 axes or more. For this reason, it is desirable for the first arm to operate with seven axes of freedom.

  The plurality of actuators included in the first manipulator M1 are each connected to the robot control device 30 via a cable so as to be communicable. Accordingly, the actuator operates the first manipulator M1 based on the control signal acquired from the robot control device 30. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. Note that some or all of the actuators may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The first force detector 11 is provided between the first end effector E1 and the first manipulator M1. The 1st force detection part 11 detects the value which shows the magnitude | size of the force and moment which acted on the 1st end effector E1. The first force detection unit 11 outputs first force detection information including a value indicating the magnitude of the detected force or moment as an output value to the robot control device 30 through communication. The first force detection information is used for control based on the first force detection information of the first arm by the robot control device 30. The control based on the first force detection information is, for example, compliance control such as impedance control. The first force detector 11 may be another sensor that detects a value indicating the magnitude of the force or moment applied to the first end effector E1, such as a torque sensor.

  The first force detection unit 11 is connected to the robot control device 30 via a cable so as to be communicable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The first force detection unit 11 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The first imaging unit 21 is a camera including, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, which is an imaging element that converts collected light into an electrical signal. In this example, the first imaging unit 21 is provided in a part of the first manipulator M1. For this reason, the 1st imaging part 21 moves according to a motion of the 1st arm. In addition, the range in which the first imaging unit 21 can capture an image changes according to the movement of the first arm. The first imaging unit 21 may capture a still image in the range or a moving image in the range.

  The first imaging unit 21 is communicably connected to the robot control device 30 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The first imaging unit 21 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second arm includes a second end effector E2, a second manipulator M2, a second force detection unit 12, and a second imaging unit 22.

  The second end effector E2 includes two or more gripping portions that can grip an object and a plurality of actuators (not shown). Hereinafter, as an example, a case will be described in which the second end effector E2 includes three gripping parts including a gripping part C21, a gripping part C22, and a gripping part C23.

  Each of the plurality of actuators provided in the second end effector E2 is connected to the robot control device 30 through a cable so as to be communicable. Accordingly, the actuator operates each of the gripping part C21, the gripping part C22, and the gripping part C23 based on the control signal acquired from the robot control device 30. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The actuator may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second manipulator M2 is a seven-axis vertical articulated manipulator including seven joints, a second imaging unit 22, and a plurality of actuators (not shown). Accordingly, the second arm performs a motion with seven degrees of freedom by the operation of the support base, the second end effector E2, and the second manipulator M2 in cooperation with the actuator. The second arm preferably operates with seven axes of freedom for the same reason that it is desirable for the first arm to operate with seven axes of freedom. The second arm may be configured to operate with a degree of freedom of 6 axes or less, or may be configured to operate with a degree of freedom of 8 axes or more.

  The plurality of actuators provided in the second manipulator M2 are each connected to the robot control device 30 through a cable so as to be communicable. Thus, the actuator operates the second manipulator M2 based on the control signal acquired from the robot control device 30. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. Note that some or all of the actuators may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second force detector 12 is provided between the second end effector E2 and the second manipulator M2. The 2nd force detection part 12 detects the value which shows the magnitude | size of the force and moment which acted on 2nd end effector E2. The second force detection unit 12 outputs second force detection information including a value indicating the magnitude of the detected force or moment as an output value to the robot control device 30 through communication. The second force detection information is used for control based on the second force detection information of the second arm by the robot control device 30. The control based on the second force detection information is compliance control such as impedance control, for example. The second force detection unit 12 may be another sensor that detects a value indicating the magnitude of the force or moment applied to the second end effector E2 such as a torque sensor.

  The 2nd force detection part 12 is connected with the robot control apparatus 30 by the cable so that communication is possible. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The second force detection unit 12 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second imaging unit 22 is, for example, a camera provided with a CCD, a CMOS, or the like that is an imaging element that converts the collected light into an electrical signal. In this example, the second imaging unit 22 is provided in a part of the second manipulator M2. For this reason, the second imaging unit 22 moves in accordance with the movement of the second arm. Further, the range that can be imaged by the second imaging unit 22 changes according to the movement of the second arm. The second imaging unit 22 may capture a still image in the range or a moving image in the range.

  The second imaging unit 22 is connected to the robot control device 30 via a cable so as to be communicable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The second imaging unit 22 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The third imaging unit 23 is, for example, a camera that includes a CCD, a CMOS, or the like that is an imaging element that converts collected light into an electrical signal. The 3rd imaging part 23 is installed in the position which can image the imaging range which is a range including the target object O mounted on the upper surface of the workbench TB. The third imaging unit 23 may capture a still image in the imaging range or a moving image in the imaging range.

  The third imaging unit 23 is connected to the robot control device 30 via a cable so as to be communicable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The third imaging unit 23 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The fourth imaging unit 24 is, for example, a camera that includes a CCD, a CMOS, or the like that is an imaging element that converts collected light into an electrical signal. The fourth image pickup unit 24 is installed at a position where the third image pickup unit 23 can pick up an image pickup range that can be picked up by the third image pickup unit 23 in a stereo image pickup position. The fourth imaging unit 24 may capture a still image in the imaging range or a moving image in the imaging range.

  Further, the fourth imaging unit 24 is connected to the robot control device 30 through a cable so as to be communicable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The fourth imaging unit 24 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  Each of these functional units included in the robot 20 described above acquires a control signal from the robot control device 30 built in the robot 20 in this example. Then, each functional unit performs an operation based on the acquired control signal. The robot 20 may have a configuration (robot system) controlled by the robot control device 30 installed outside, instead of the configuration including the robot control device 30.

  The robot control device 30 operates the robot 20 by transmitting a control signal to the robot 20. As a result, the robot control device 30 causes the robot 20 to perform a predetermined operation.

Hereinafter, predetermined work performed by the robot 20 will be described.
The work table TB illustrated in FIG. 1 is, for example, a table. An object O is placed on the upper surface of the work table TB. The work table TB may be another object as long as it is an object such as a floor surface on which the object O can be placed instead of the table.

  The object O is, for example, an industrial part or member such as a plate, screw, or bolt that is assembled to a product. Below, the case where the target object O is a hexagonal prism-shaped object as shown in FIG. 1 is demonstrated as an example. The object O may be an industrial product instead of an industrial part or member, or may be another object such as a daily necessities or a living body. Further, the shape of the object O may be another shape instead of the hexagonal column shape.

  For example, the robot 20 holds the object O placed on the upper surface of the work table TB by the first end effector E1 as a predetermined work. Then, the robot 20 supplies (arranges) the grasped object O to the supply area A on the upper surface of the work table TB shown in FIG. The predetermined work may be another work. Further, the robot 20 may be configured to hold the object O with the second end effector E2 instead of the structure to hold the object O with the first end effector E1 in a predetermined work. The structure which hold | grips the target object O by both E1 and 2nd end effector E2 may be sufficient. For example, when the robot 20 grips the object O by the second end effector E2 in a predetermined work, the robot 20 performs the same operation as when the first end effector E1 is replaced with the second end effector E2 in the following description. I do.

Hereinafter, an outline of processing performed by the robot control device 30 to cause the robot 20 to perform a predetermined operation will be described.
The robot control device 30 determines the state of the first end effector E1 based on the position and posture of the object O, error information on the position and posture, and the state of the first end effector E1. As an example of this, in this embodiment, the robot control device 30 determines the gripping quality based on the position and orientation of the object O detected from the captured image, error information on the position and orientation, and the state of the first end effector E1. The state of the first end effector E1 is determined according to the evaluation value. Note that the robot control device 30 is configured to acquire information indicating the position and orientation of the object O from another device or a storage medium instead of the configuration to detect the position and orientation of the object O from the captured image, Other configurations, such as a configuration for acquiring information indicating the position and orientation of the object O input from, may be used.

  The hand is a part of the robot 20 that can grip an object, that is, one or both of the first end effector E1 and the second end effector E2. In this example, in order to explain the case where the object O is gripped by the first end effector E1, the hand is the first end effector E1, and the state of the hand is the state of the first end effector E1. It is. In this example, the state of the first end effector E1 is represented by the relative position and posture of the first end effector E1 with respect to the object O.

  The error information on the position and orientation of the object O is information indicating an error distribution (probability distribution) when the position and orientation of the object O detected from the captured image are used as the center value. In other words, the error information is information indicating an error range. For example, the error information is information indicating a normal distribution. Note that the error information may be information indicating another probability distribution such as information indicating a probability distribution reproducing the result of the experiment for detecting the object O, instead of the information indicating the normal distribution. The captured image is an example of an image.

  The processing performed by the robot control device 30 will be described more specifically. The robot control device 30 includes the position and orientation of the object O detected from the captured image, error information on the position and orientation, and the first end effector E1. For each state of the first end effector E1, based on the state, when the first end effector E1 grips the object O, the state of the first end effector E1 (ie, the first end effector E1 and the object O A grip quality evaluation value, which is a value obtained by evaluating the difficulty of changing (relative position and posture) (grip quality), is calculated.

  The robot control device 30 determines the state of the first end effector E1 suitable for causing the first end effector E1 to grip the object O based on the calculated gripping quality evaluation value. As a result, the robot controller 30 determines the state of the first end effector E1 determined according to the gripping quality evaluation value, that is, the first end effector E1 suitable for causing the first end effector E1 to grip the object O. The object O can be gripped by the robot 20 depending on the state. The robot control apparatus 30 causes the robot 20 to grip the object O according to the state of the first end effector E1 suitable for causing the first end effector E1 to grip the object O, and moves the object O to the feeding area A. The robot 20 is fed.

  In the following, the robot control device 30 performs the first measurement according to the grip quality evaluation value based on the position and posture of the object O detected from the captured image, error information on the position and posture, and the state of the first end effector E1. The process for determining the state of the one end effector E1 will be described in detail.

  In the following, for the sake of simplicity, the position and orientation of the object O detected by the robot control device 30 from the captured image are the position and orientation of the object O in the two-dimensional robot coordinate system. A case will be described in which the state of the first end effector E1 determined by 30 according to the gripping quality evaluation value is the state of the first end effector E1 in the two-dimensional robot coordinate system.

  In the following, for the sake of simplicity of explanation, the two coordinate axes of the two-dimensional robot coordinate system are horizontal with respect to the ground and are oriented in directions orthogonal to each other, and the work table on which the object O is placed. The case where the upper surface of TB is horizontal with respect to the ground will be described. The robot controller 30 detects the position and orientation of the object O in the three-dimensional robot coordinate system from the captured image, and determines the state of the first end effector E1 in the three-dimensional robot coordinate system according to the grip quality evaluation value. The structure to determine may be sufficient. The grip quality evaluation value is an example of an evaluation value for evaluating the hand state.

  Next, the hardware configuration of the robot control device 30 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a hardware configuration of the robot control device 30. The robot control device 30 includes, for example, a CPU (Central Processing Unit) 31, a storage unit 32, an input receiving unit 33, a communication unit 34, and a display unit 35. Further, the robot control device 30 communicates with the robot 20 via the communication unit 34. These components are connected to each other via a bus Bus so that they can communicate with each other.

The CPU 31 executes various programs stored in the storage unit 32.
The storage unit 32 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), and a random access memory (RAM). The storage unit 32 may be an external storage device connected via a digital input / output port such as a USB instead of the one built in the robot control device 30.

  The storage unit 32 stores various information processed by the robot control device 30, images, programs, error information, information indicating the position of the material supply area A, and the like. Below, the case where the memory | storage part 32 has memorize | stored beforehand the error information and the information which shows the position of the material supply area | region A as an example is demonstrated. The storage unit 32 may be configured to store either one or both of error information and information indicating the position of the material supply area A based on an operation received from the user.

The input receiving unit 33 is, for example, a teaching pendant provided with a keyboard, a mouse, a touch pad, or the like, or other input device. The input receiving unit 33 may be configured integrally with the display unit 35 as a touch panel.
The communication unit 34 includes, for example, a digital input / output port such as USB, an Ethernet (registered trademark) port, and the like.
The display unit 35 is, for example, a liquid crystal display panel or an organic EL (ElectroLuminescence) display panel.

  Next, the functional configuration of the robot control device 30 will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of a functional configuration of the robot control device 30. The robot control device 30 includes a storage unit 32 and a control unit 36.

  The control unit 36 controls the entire robot control device 30. The control unit 36 includes a force detection information acquisition unit 40, an imaging control unit 41, an image acquisition unit 42, a position and orientation detection unit 44, a grip quality evaluation value calculation unit 46, a hand state determination unit 47, and a robot control. Part 48 is provided. These functional units included in the control unit 36 are realized, for example, when the CPU 31 executes various programs stored in the storage unit 32. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

  The force detection information acquisition unit 40 acquires, from the first force detection unit 11, first force detection information including a value indicating the magnitude of the force or moment detected by the first force detection unit 11 as an output value. The force detection information acquisition unit 40 acquires second force detection information from the second force detection unit 12 including a value indicating the magnitude of the force or moment detected by the second force detection unit 12 as an output value.

  The imaging control unit 41 includes an imaging range including the object O placed on the upper surface of the work table TB, the first imaging unit 21, the second imaging unit 22, the third imaging unit 23, and the fourth imaging unit. Some or all of the 24 are imaged. Hereinafter, as an example, the case where the imaging control unit 41 causes the third imaging unit 23 to image the imaging range will be described. Note that the robot control device 30 detects the position and orientation of the object O in the three-dimensional robot coordinate system from the captured image, and determines the state of the first end effector E1 in the three-dimensional robot coordinate system according to the grip quality evaluation value. In the case of the configuration to be determined, the imaging control unit 41 applies to at least two imaging units among the first imaging unit 21, the second imaging unit 22, the third imaging unit 23, and the fourth imaging unit 24. Stereo imaging of the imaging range.

The image acquisition unit 42 acquires the captured image captured by the third imaging unit 23 from the third imaging unit 23.
The position / orientation detection unit 44 detects the position and orientation of the object O from the captured image based on the captured image acquired by the image acquisition unit 42.

  The gripping quality evaluation value calculation unit 46 reads error information from the storage unit 32. In addition, the gripping quality evaluation value calculation unit 46 is based on the read error information, the position and orientation of the object O detected by the position and orientation detection unit 44 from the captured image, and the state of the first end effector E1. A grip quality evaluation value is calculated for each state of one end effector E1.

  Based on the grip quality evaluation value calculated by the grip quality evaluation value calculation unit 46, the hand state determination unit 47 sets the state of the first end effector E1 that maximizes the grip quality evaluation value to the first end effector E1. The state of the first end effector E1 suitable for gripping O is determined. The hand state determination unit 47 sets the state of the first end effector E1 that maximizes the grip quality evaluation value as the state of the first end effector E1 suitable for causing the first end effector E1 to grip the object O. Instead of the configuration to be determined, the state of the first end effector E1 having a value other than the maximum gripping quality evaluation value, such as the state of the first end effector E1 having the gripping quality evaluation value of a predetermined threshold, is set to the first end. The configuration may be such that the state of the first end effector E1 suitable for causing the effector E1 to grip the object O is determined.

  The robot control unit 48 causes the robot 20 to grip the object O according to the state of the first end effector E1 determined by the hand state determination unit 47, and causes the robot 20 to supply the object O to the material supply area A.

Hereinafter, the process performed by the control unit 36 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of the flow of processing performed by the control unit 36.
The imaging control unit 41 causes the third imaging unit 23 to capture an imaging range including the object O placed on the upper surface of the workbench TB (step S100). Next, the image acquisition unit 42 acquires the captured image captured by the third imaging unit 23 in step S100 from the third imaging unit 23 (step S110). Next, the position and orientation detection unit 44 detects the position and orientation of the object O from the captured image as the detected position and orientation based on the captured image acquired by the image acquisition unit 42 in step S110 (step S120).

  Here, the process of step S120 will be described. For example, the position and orientation detection unit 44 detects the contour of the object O from the captured image based on the captured image acquired by the image acquisition unit 42 in step S110. The position / orientation detection unit 44 detects the position of the center of gravity of the detected contour as the position of the object O. Here, the position represented by each point in the captured image and the position in the two-dimensional robot coordinate system are associated in advance by calibration.

  In addition, the position / orientation detection unit 44 sets a two-dimensional local coordinate system representing the attitude of the object O with the position of the center of gravity of the detected contour as the origin as the object O. The directions of the two coordinate axes of the two-dimensional local coordinate system may be any direction on the captured image as long as they are orthogonal to each other. In this example, the position / orientation detection unit 44 sets the X-axis direction of the two-dimensional local coordinate system to the X-axis direction of the image sensor of the third imaging unit 23, and the Y-axis of the two-dimensional local coordinate system. The direction is set to the Y-axis direction of the image sensor of the third imaging unit 23. The position and orientation detection unit 44 specifies (detects) the directions of the two coordinate axes of the two-dimensional local coordinate system as the orientation of the object O. Note that the position / orientation detection unit 44 may have a configuration in which the orientation of the object O is represented by an angle of inclination from the reference with respect to one direction of the two coordinate axes of the two-dimensional local coordinate system. .

  After the detected position and orientation are detected in step S120, the gripping quality evaluation value calculation unit 46 repeats the process of step S125 for each state of the hand, that is, for each state of the first end effector E1 (step S123). Here, the state of the first end effector E1 will be described.

  FIG. 5 is a view for explaining the state of the first end effector E1. FIG. 5 shows the object O in the case where the position and orientation match the detected position and orientation. Further, FIG. 5 shows the first end effector E1 immediately before gripping the object O. However, in FIG. 5, in order to clearly show the relative positions and postures of the gripping part C11, the gripping part C12, the gripping part C13, and the object O included in the first end effector E1, Parts of the first end effector E1 other than the part C12 and the gripping part C13 are omitted. In FIG. 5, for simplification of the drawing, the gripping part C11, the gripping part C12, and the gripping part C13 are each drawn by a black circle, but the actual shape of each of these three gripping parts is a black circle. There is no reason.

  As described above, the state of the first end effector E1 is represented by the relative position and posture of the first end effector E1 and the object O. More specifically, in this example, the position of the first end effector E1 includes an auxiliary line extending in the movable direction CA1 of the gripping part C11 and an auxiliary line extending in the movable direction CA2 of the gripping part C12 shown in FIG. It is represented by a gripping position G that is the position of a point where the auxiliary line extending in the movable direction CA3 of the gripping part C13 intersects at one point. Note that the gripping position G is a position associated with the first end effector E1 such as TCP (Tool Center Point) instead of the position of the point where the one point intersects and moves together with the first end effector E1. Other positions may be used as long as they are positions.

  From the above, the relative position between the first end effector E1 and the object O is represented by the grip position G with respect to the position of the object O. In the following description, the relative position between the first end effector E1 and the object O is represented by a gripping position G drawn in the outline of the object O.

  In this example, the relative posture between the first end effector E1 and the object O is an angle between the X axis of the two-dimensional local coordinate system representing the posture of the object O and an auxiliary line extending in the movable direction CA1. φ, an angle θ1 between an auxiliary line extending in the movable direction CA1 and an auxiliary line extending in the movable direction CA2, and an angle θ2 between an auxiliary line extending in the movable direction CA1 and an auxiliary line extending in the movable direction CA3. Represented by two angles. In FIG. 5, the origin of the two-dimensional local coordinate system SO representing the posture of the object O is moved to the gripping position G in order to clearly show the angle φ.

  That is, the state of the first end effector E1 is represented by a state vector that is a vector having four values of the gripping position G, the angle φ, the angle θ1, and the angle θ2. The state of the first end effector E1 changes every time each component of the state vector is changed. The gripping quality evaluation value calculation unit 46 is based on the detected position and orientation detected by the position and orientation detection unit 44 in step S120 and a predetermined rule that the gripping quality evaluation value calculation unit 46 follows when each component of the state vector is changed. Thus, a plurality of vectors in which at least one of the components of the state vector is different from each other, that is, the states of the plurality of first end effectors E1 are calculated. Then, the gripping quality evaluation value calculation unit 46 performs the process of step S125 for each of the calculated states of the plurality of first end effectors E1.

  Here, a predetermined rule according to the case where the gripping quality evaluation value calculation unit 46 changes each component of the state vector will be described. When changing the gripping position G among the four components of the state vector, the gripping quality evaluation value calculation unit 46 divides the contour of the object O into square grids of a predetermined width, and sets the grid points of the square grids. The positions of the unselected grid points are selected randomly or according to a predetermined selection rule. Then, the gripping quality evaluation value calculation unit 46 changes the gripping position G to the position of the selected grid point.

  Further, when changing the angle φ of the four components of the state vector, the gripping quality evaluation value calculation unit 46 changes the angle from 0 ° to 360 ° for each first predetermined angle. The first predetermined angle is 1 °, for example. The first predetermined angle may be another angle. Further, when changing the angle θ1 of the four components of the state vector, the gripping quality evaluation value calculation unit 46 changes the angle from 0 ° to 360 ° every second predetermined angle. The second predetermined angle is 1 °, for example. The second predetermined angle may be another angle. Further, when changing the angle θ2 of the four components of the state vector, the gripping quality evaluation value calculation unit 46 changes the angle from 0 ° to 360 ° for each third predetermined angle. The third predetermined angle is 1 °, for example. The third predetermined angle may be another angle.

  Based on such a predetermined rule, in step S123, the grip quality evaluation value calculation unit 46 has a plurality of states in which at least one of the grip position G, the angle φ, the angle θ1, and the angle θ2 is different from each other. A vector is calculated, and the process of step S125 is performed for each state of the first end effector E1 represented by the calculated state vector.

  After the state of the first end effector E1 is selected in step S123, the gripping quality evaluation value calculation unit 46 calculates a gripping quality evaluation value based on the state (step S125). More specifically, the grip quality evaluation value calculation unit 46 reads error information from the storage unit 32, and based on the read error information, the detected position and orientation, and the state of the first end effector E1 selected in step S123. To calculate the grip quality evaluation value.

  Here, the process of step S125 will be described. The gripping quality evaluation value calculation unit 46 reads error information from the storage unit 32. Based on the read error information and the detected position / posture detected by the position / posture detection unit 44 in step S120, the gripping quality evaluation value calculation unit 46 calculates a plurality of errors of detected position / posture and having different sizes. calculate. Then, for each of the plurality of calculated errors, the gripping quality evaluation value calculation unit 46 calculates a position and posture obtained by adding the error to the detected position and posture as a calculated position and posture.

  For example, the grip quality evaluation value calculation unit 46 determines that the error distribution of the detected position and orientation detected by the position and orientation detection unit 44 in step S120 is a probability distribution indicated by the error information. In this example, since the error information is information indicating a normal distribution, the gripping quality evaluation value calculation unit 46 determines that the error distribution of the detected position and orientation is a normal distribution. Then, the gripping quality evaluation value calculation unit 46 has the detected position and orientation of ± 0.1 standard deviation, ± 0.3 standard deviation, and ± 0.5 standard deviation based on the detected position and orientation and the normal distribution. An error of a magnitude different from each other is calculated for a total of ± 0.7 standard deviation and ± 1 standard deviation. Then, for each of these errors, the gripping quality evaluation value calculation unit 46 calculates the position and orientation obtained by adding the error to the detected position and orientation as the calculated position and orientation. Note that the grip quality evaluation value calculation unit 46 may be configured to calculate the calculated position and orientation using other errors instead of these errors. Further, the gripping quality evaluation value calculation unit 46 may be configured to calculate an error of less than 10 instead of the configuration of calculating 10 errors, or may be configured to calculate 11 or more errors.

  In this example, the gripping quality evaluation value calculation unit 46 calculates the calculated position and orientation when both the position of the object O and the attitude of the object O are shifted due to errors when calculating the calculated position and orientation. To do. When calculating the calculated position and orientation, the gripping quality evaluation value calculation unit 46 calculates the calculated position and orientation when only the position of the object O is shifted due to an error, and the case where only the orientation of the object O is shifted due to an error. It may be configured to calculate either one or both of the calculated position and orientation. Hereinafter, for convenience of explanation, a plurality of calculated position / postures and detected position / postures will be collectively referred to as an object position / posture.

  FIG. 6 is a diagram visually showing each of the object positions and orientations. In FIG. 6, a contour CO drawn by the thickest line indicates the contour of the object O in the detected position and orientation included in the object position and orientation. Contours other than the contour CO indicate the contours of the object O in each of a plurality of calculated positions and orientations included in the object position and orientation. In FIG. 6, in order to clearly show the outline of the object O at each object position and orientation, the X axis and the Y axis with the position of the center of gravity of the object O as the origin are shown (in this example, the unit is a pixel) ). Further, the direction of the X axis coincides with the direction of the X axis of the two coordinate axes representing the posture of the object O identified by the position and posture detection unit 44 in step S120. Further, the direction of the Y axis coincides with the direction of the Y axis of the two coordinate axes representing the attitude of the object O specified by the position / orientation detection unit 44 in step S120. In addition, each of the plurality of contours shown in FIG. 6 is drawn slightly distorted to clearly show the overlapping contours. In addition, when the gripping quality evaluation value calculation unit 46 calculates a state vector representing the state of the first end effector E1, the gripping positions G shown in FIG. 6 are determined according to a predetermined rule in this example. A plurality of lattice points that can be changed.

  After calculating such a calculated position and orientation, the gripping quality evaluation value calculation unit 46 calculates the appearance probability of each object position and orientation based on the read error information. As described above, in this example, the error information is information indicating a normal distribution. As can be seen from the relationship between the detected position and orientation and the calculated position and orientation, the detected position and orientation are treated as the center value in the normal distribution. For this reason, the appearance probability of the detected position and orientation is maximized. In addition, the appearance probability of the calculated position and orientation obtained by adding an error of ± 1 standard deviation to the detected position and orientation is minimum. The thickness of the contour of the object O at each object position and orientation shown in FIG. 6 represents the appearance probability of the object position and orientation indicated by each contour.

  After calculating the appearance probability in this way, the gripping quality evaluation value calculation unit 46 grips the object O according to the state of the first end effector E1 selected in step S123 for each target object position and orientation. An evaluation value indicating the difficulty in changing the relative position and posture of the first end effector E1 and the object O is calculated. For example, if the detected position / posture is the first position / posture and the calculated position / posture of 10 is the second position / posture to the eleventh position / posture, the first position / posture is selected in step S123 for each of the first position / posture. An evaluation value indicating the difficulty in changing the relative position and posture of the first end effector E1 and the object O when the object O is gripped by the state of the one end effector E1 is calculated. Then, the gripping quality evaluation value calculation unit 46 calculates a gripping quality evaluation value based on the plurality of calculated evaluation values (11 evaluation values in this example). In this example, the gripping quality evaluation value calculation unit 46 calculates the sum of the evaluation values calculated for each calculated position and orientation as the gripping quality evaluation value.

  Here, the process of calculating the grip quality evaluation value will be described. The gripping quality evaluation value calculation unit 46 calculates a gripping quality evaluation value based on the following formula (1). In the following description, when A_B is described, B is a subscript attached after A. That is, A is a vector and B is a variable representing each component of A which is a vector. Further, when A_BC is described, it is assumed that B and C are subscripts attached after A. That is, A is a tensor, and B and C are variables representing each component of A that is a tensor. When B ^ A is described, B is described as a superscript attached to the front of A. A superscript in front of A is used as a label for distinguishing a plurality of A's.

  Q_ij indicates a grip quality evaluation value. i is a variable used as a label for distinguishing the relative posture between the first end effector E1 and the object O included in the state of the first end effector E1. j is a variable used as a label for distinguishing each of the object positions and orientations. N is the total number of object positions and postures, and is 11 in this example.

  P_j is the appearance probability of the object position / posture indicated by j. G_i indicates the relative posture of the first end effector E1 and the object O indicated by i. k is a label for distinguishing the relative positions of the first end effector E1 and the object O included in the state of the first end effector E1 (for example, each of the plurality of gripping positions G shown in FIG. 6). The variable to use. q_jk (G_i) holds the object O in the object position and posture indicated by j at the holding position G indicated by k when the relative posture G_i of the first end effector E1 and the object O indicated by i is realized. This is the evaluation value when

  As shown in the above equation (1), the gripping quality evaluation value Q_ik calculates an evaluation value q_jk (G_i) obtained by multiplying the appearance probability of the target object position / posture by the weight for each target object position / posture indicated by j. The average value obtained by dividing Σq_jk (G_i), which is the sum of all the calculated evaluation values q_jk (G_i) for each target position and orientation, by the total number N of target positions and orientations. Here, Σ is a sum symbol for j. The evaluation value q_jk (G_i) is the sum of two types of evaluation values in this example. The first of the two types of evaluation values is a translation evaluation value r_ijk (= r_jk (G_i)) indicating the difficulty in changing the relative position between the first end effector E1 and the object O (difficulty in translation). ). The second of the two types of evaluation values is a rotation evaluation value τ_ijk (τ_jk (G_i)) that represents the difficulty in changing the relative posture of the first end effector E1 and the object O (difficulty in rotation). It is. The grip quality evaluation value calculated by the translation evaluation value r_jk (G_i) is taken as the translation grip quality evaluation value 1 ^ Q_ik, and the grip quality evaluation value calculated by the rotation evaluation value τ_jk (G_i) is the rotation grip quality evaluation value 2 ^ Q_ik. Then, these are respectively expressed as the following expressions (2) and (3).

  The gripping quality evaluation value calculation unit 46 calculates a translation evaluation value r_ijk and a rotation evaluation value τ_ijk. The gripping quality evaluation value calculation unit 46 calculates a translational gripping quality evaluation value 1 ^ Q_ik based on the calculated translational evaluation value r_ijk, and calculates a rotational gripping quality evaluation value 2 ^ Q_ik based on the calculated rotational evaluation value τ_ijk. . Then, the gripping quality evaluation value calculation unit 46 calculates the above-described gripping quality evaluation value Q_ik as the sum of the translational gripping quality evaluation value 1 ^ Q_ik and the rotational gripping quality evaluation value 2 ^ Q_ik. The gripping quality evaluation value calculation unit 46 may be configured to calculate only one of the translation evaluation value r_ijk and the rotation evaluation value τ_ijk. In this case, the gripping quality evaluation value Q_ik is only one of the translational gripping quality evaluation value 1 ^ Q_ik and the rotating gripping quality evaluation value 2 ^ Q_ik.

  Hereinafter, a method for calculating the evaluation value q_jk (G_i) for each object position and orientation will be described. First, a process for calculating the translation evaluation value r_ijk for each object position and orientation will be described. When a force to translate the object O is applied to the object O grasped according to the state of the first end effector E1 selected in step S123, the grasping part C11, the grasping part C12, and the grasping part A normal force and a frictional force are applied from the object O to each of C13.

  For example, when the magnitude of the vertical drag applied from the object O to the gripping part C11 is larger than the maximum force that the gripping part C11 can apply to the object O, the object O is It translates with respect to the 1st end effector E1. Further, when the magnitude of the vertical drag applied from the object O to the gripping part C12 is larger than the maximum force that the gripping part C12 can apply to the object O, the object O is It translates with respect to the 1st end effector E1. Further, when the magnitude of the vertical drag applied from the object O to the gripping part C13 is larger than the maximum force that the gripping part C13 can apply to the object O, the object O is It translates with respect to the 1st end effector E1.

  In addition, when a large force is applied from the object O in the direction opposite to the direction of the frictional force from the object O to the gripping part C11 due to the maximum vertical drag that can be supported by the gripping part C11. The object O translates with respect to the first end effector E1. Further, when a large frictional force is applied from the object O to the gripping part C12 by the maximum vertical drag that can be supported by the gripping part C12, from the object O in a direction opposite to the direction of the frictional force. The object O translates with respect to the first end effector E1. In addition, when a large force is applied from the object O in the direction opposite to the direction of the frictional force, the frictional force applied from the object O to the gripping part C13 by the maximum vertical drag that can be supported by the gripping part C13. The object O translates with respect to the first end effector E1.

  Therefore, in this example, the gripping quality evaluation value calculation unit 46 has the maximum vertical drag that can be supported by each gripping part included in the first end effector E1, and each object from the object O by the vertical drag. A translation evaluation value r_ijk is calculated based on the frictional force applied to the grasped part. The maximum normal force that can be supported by each gripping part of the first end effector E1 is the state of the first end effector E1, the target position and posture, and the first force detection acquired from the first force detector 11. It is determined by control based on information.

  Hereinafter, the maximum vertical drag that can be supported by the gripping part C11 out of the vertical drags applied to the gripping part C11 from the object O will be referred to as a vertical drag f1. In the following description, the maximum vertical drag that the gripping part C12 can support the object O among the vertical drags applied to the gripping part C12 from the object O will be referred to as a vertical drag f2. In the following description, the maximum vertical drag that the gripping part C13 can support the object O among the vertical drags applied to the gripping part C13 from the object O will be referred to as a vertical drag f3.

  Here, a method of calculating the translation evaluation value r_ijk based on the vertical drag f1 to the vertical drag f3 illustrated in FIG. 7 will be described with reference to FIGS. FIG. 7 is a diagram illustrating the maximum vertical drag that can be supported by each gripping portion of the vertical drag applied from the object O to each gripping portion included in the first end effector E1. In FIG. 7, the magnitudes and directions of the vertical drags f1 to f3 are indicated by arrows, and the action points of the vertical drags f1 to f3 are made to coincide with the origin of the coordinate system shown in FIG. The vertical axis of the coordinate system shown in FIG. 7 is a coordinate axis Fy representing the magnitude of the Y-axis direction component of the force applied from the surface of the object O to another object. Also, the horizontal axis shown in FIG. 7 is a coordinate axis Fx representing the magnitude of the X-axis direction component of the force applied from the surface of the object O to another object.

  As shown in FIG. 8, a friction cone can be drawn with respect to the arrows representing the normal force f1 to the normal force f3 shown in FIG. FIG. 8 is a diagram in which a friction cone is drawn with respect to the arrows representing the normal force f1 to the normal force f3 shown in FIG. Here, the friction cone will be described by taking the normal force f1 shown in FIG. 8 as an example. As described above, when the object O is to be translated with respect to the first end effector E1, the object O is converted into the first end effector E1 by the vertical drag f1 to the vertical drag f3 shown in FIG. A frictional force is applied to each gripping part included in the.

  The frictional force applied from the object O to the grip portion C11 by the vertical drag f1 is applied in a direction along the plane HL perpendicular to the end point of the arrow representing the vertical drag f1. In this example, the surface HL is a line because it is considered that the force acting on the object O is applied only in the direction included in the two-dimensional plane.

  In addition, the direction in which the frictional force is applied from the object O to the grasping part C11 by the vertical drag f1 is determined by the direction of the force that tries to translate the object O. It can join in all directions along the surface HL. That is, in the example shown in FIG. 8, the two frictional forces that can be applied from the object O to the gripping part C11 by the vertical drag f1 are the frictional force f1f1 and the frictional force f1f2. As shown in FIG. 8, the end point of the arrow representing the friction force f1f1, the end point of the arrow representing the friction force f1f2, and the start point of the normal force f1 form a triangle. When the force acting on the object O is applied in the direction included in the three-dimensional space, the end point of each arrow representing the frictional force that can be applied from the object O to the gripping portion C11 by the vertical drag f1 and the vertical drag f1. The starting point of the arrow forms a triangular pyramid. The aforementioned friction cone is such a triangle or a triangular pyramid.

In FIG. 8, in order to clearly show the friction cone with respect to the normal force f1, the arrow f1r connecting the start point of the arrow indicating the normal force f1 to the end point of the arrow indicating the friction force f1f1, and the start point of the arrow indicating the normal force f1. An arrow f1l connecting the end point of the arrow representing the frictional force f1f2 is drawn. The angle between the arrow representing the normal force f1 and the arrow f1r and the angle between the arrow representing the normal force f1 and the arrow f1l are both the angle β. The angle β is calculated by β = tan ⁻¹ μ based on the geometrical relationship between the normal force f1 and the frictional force f1f1 or the frictional force f1f2 where the static friction coefficient of the surface of the object O is μ. Can do. As can be seen from this equation, the angle β is determined not only by the magnitude of the normal force f1 and the magnitude of the friction force f1f1 or the friction force f1f2, but only by the static friction coefficient μ.

  FIG. 8 shows an arrow f2r and an arrow f2l for clearly showing the friction cone with respect to the vertical drag f2 by a method similar to the method for drawing the arrow f1r and the arrow f1l for clearly showing the friction cone with respect to the vertical drag f1. In addition, an arrow f3r and an arrow f3l for clearly showing the friction cone with respect to the normal force f3 are drawn. Here, it is assumed that the static friction coefficient μ on the surface of the object O is constant throughout the surface of the object O, but instead, on a part of the surface of the object O, May be different.

  The volume of the convex hull, which is the smallest polygon that includes all of the end points of the arrows for clarifying the friction cone shown in FIG. 8 and the end points of the arrows representing the normal forces shown in FIG. The larger the value is, the more the first end effector E1 moves the object O so that the relative position between the first end effector E1 and the object O is not changed when a force that translates the object O is applied to the object O. Indicates that the ability to support is high. For this reason, it is considered that the volume of the convex hull has a characteristic that can be used as the translation evaluation value r_ijk. However, for example, when such a convex hull is elongated in a certain direction, the object O is directed from the object O to the first end effector E1 in a direction different from the direction although the volume of the convex hull is large. Therefore, the ability of the first end effector E1 to support the object O when the force is applied is low. For this reason, it is not preferable that the volume of the convex hull itself be the translation evaluation value r_ijk.

  Therefore, the volume of the convex hull and the amount that can represent the level of the ability of the first end effector E1 to support the object O in a specific direction from the origin of the coordinate system shown in FIG. The radius of the smallest inscribed circle that is inscribed can be considered.

  In order to calculate the radius of the smallest inscribed circle inscribed in the convex hull, the end point of each arrow for clarifying the friction cone shown in FIG. 8 and the arrow indicating each vertical drag shown in FIG. It is necessary to generate a convex hull based on the end point. Further, in order to generate a convex hull, the respective coordinates of the end point of each arrow for clarifying the friction cone shown in FIG. 8 and the end point of the arrow representing each vertical drag shown in FIG. 8 are calculated. There is a need to.

  For example, the coordinates (x1r, y1r) of the end point P1r of the arrow f1r for clarifying the friction cone with respect to the vertical drag f1 shown in FIG. 8 are the horizontal axis of the coordinate system shown in FIG. 8 and the arrow indicating the vertical drag f1. When the angle between them is α, it can be calculated from the geometric relationship between the arrow representing the normal force f1 and the arrow f1r using the following equations (4) to (8).

| F1r | = | f1 | / cos (β) (4)
x1r = | f1r | cos (α + β) (5)
y1r = | f1r | sin (α + β) (6)

Further, the coordinates (x1l, y1l) of the end point P1l of the arrow f1l for clarifying the friction cone with respect to the vertical drag f1 are based on the geometric relationship between the arrow representing the vertical drag f1, the arrow f1l, and the arrow f1l. These can be calculated using the following formulas (7) to (9).
| F1l | = | f1 | / cos (β) (7)
x1l = | f1l | cos (α−β) (8)
y1l = | f1l | sin (α−β) (9)

  The gripping quality evaluation value calculation unit 46 calculates the coordinates of the end point P1r and the end point P1l using the above formulas (4) to (9). In addition, the gripping quality evaluation value calculation unit 46 uses the same method as the method of calculating the coordinates of the end point P1r and the end point P1l, the end point coordinates of the arrow f2r and the arrow f2l, and the end point coordinates of the arrow f3r and the arrow f3l. And calculate. The gripping quality evaluation value calculation unit 46 generates the convex hull shown in FIG. 9 based on these calculated coordinates.

  FIG. 9 is a diagram illustrating an example of a convex hull generated by the gripping quality evaluation value calculation unit 46. The convex hull CH shown in FIG. 9 is a convex hull generated based on the end points of the six arrows for clarifying the friction cone shown in FIG. The gripping quality evaluation value calculation unit 46 generates a minimum inscribed circle IC inscribed in the convex hull from the origin of the coordinate system shown in FIG. 9 (that is, the coordinate system shown in FIG. 8). Then, the gripping quality evaluation value calculation unit 46 calculates the radius r1 of the generated inscribed circle IC as a translation evaluation value.

  In the example shown in FIG. 9, the radius r1, which is the distance from the origin of the coordinate system shown in FIG. 9 to the point where the inscribed circle IC touches the convex hull, connects the end point of the arrow f1r and the end point of the arrow f3l. This is the length of the perpendicular when a perpendicular is drawn from the origin of the coordinate system shown in FIG. Since the calculation of the length of such a perpendicular line can be calculated using elementary geometry based on the coordinates of the end point of the arrow f1r, the coordinates of the end point of the arrow f3l, and the coordinates of the origin point, description thereof is omitted. To do.

  In this way, the gripping quality evaluation value calculation unit 46 generates the convex hull generated based on the maximum vertical drag that can be supported by each gripping portion of the first end effector E1 and the friction cone against the vertical drag. Is calculated as the translation evaluation value r_ijk. The gripping quality evaluation value calculation unit 46 may be configured to calculate the volume of the convex hull as the translation evaluation value r_ijk, or may be configured to calculate another value based on the convex hull, It may be configured to calculate other values unrelated to the convex hull.

  Next, a process for calculating the rotation evaluation value τ_ijk will be described. The gripping quality evaluation value calculation unit 46 translates when a force to rotate the object O is applied to the object O held by the state of the first end effector E1 selected in step S123. Similarly to the case where the force is applied to the target object O, the normal force and the frictional force are applied from the target object O to each of the gripping part C11, the gripping part C12, and the gripping part C13.

  The frictional force supports the object O so as not to rotate when a force for rotating the object O is applied to the object O. More specifically, the rotational moment for rotating the object O by the frictional force and the rotational moment for rotating the object O by the force for rotating the object O are large. Are equal to each other and the directions are opposite to each other, the friction force supports the object O so as not to rotate.

  Therefore, in this example, the gripping quality evaluation value calculation unit 46 is configured so that the frictional force applied from the object O to each gripping part by the maximum normal force that can be supported by each gripping part included in the first end effector E1. The rotation evaluation value τ_ijk is calculated based on the rotation moment generated by the above.

  Here, a method for calculating the rotation evaluation value τ_ijk will be described with reference to FIG. FIG. 10 is a view exemplifying normal force and friction force applied from the object O to each gripping portion provided in the first end effector E1. The position of the gripping position G shown in FIG. 10 is determined by the component of the vector representing the state of the first end effector E1 selected in step S123. In FIG. 10, the object O gripped by the first end effector E1 exerts a force to rotate in the direction of the arrow RO shown in FIG. It is shown.

  In this example, when the rotation evaluation value τ_ijk is calculated, the vertical drag f1 is applied from the object O to the gripping part C11, the vertical drag f2 is applied from the object O to the gripping part C12, and the gripping part C13 is applied. Assume that the vertical force f3 is applied from the object O. For this reason, in FIG. 10, the frictional force applied from the object O to the gripping part C11 by the vertical drag f1 is indicated by the friction force μf1, and the frictional force applied from the object O to the gripping part C12 by the vertical drag f2 is rubbed. The frictional force indicated by the force μf2 and the frictional force applied from the object O to the gripping part C13 by the vertical drag force f3 is indicated by the frictional force μf3. Further, in FIG. 10, the shortest distance from the side of the six sides that form the contour of the object O to which the gripping part C11 is in contact to the gripping position G is indicated by a distance r4, and the contour of the target object O is formed. Of the six sides, the shortest distance from the side where the gripping part C12 is in contact to the gripping position G is indicated by the distance r5, and the side where the gripping part C13 is in contact among the six sides forming the outline of the object O The shortest distance from the gripping position G to the gripping position G is indicated by a distance r6.

  In this example, the grip quality evaluation value calculation unit 46 calculates the rotation evaluation value assuming that the rotation center of the object O is always the grip position G. Note that the rotation center of the object O may be the TCP of the first arm instead of the grip position G, or may be another position associated with the object O. The gripping quality evaluation value calculation unit 46 is a rotational moment generated by the frictional force μf1, and is a rotational moment μf1 × r4 that attempts to rotate the object O in the direction opposite to the direction of the arrow RO around the gripping position G. Is calculated. Further, the grip quality evaluation value calculation unit 46 is a rotation moment generated by the frictional force μf2, and the rotation moment μf2 that tries to rotate the object O in the direction opposite to the direction of the arrow RO with the gripping position G as the rotation center. Xr5 is calculated. The gripping quality evaluation value calculation unit 46 is a rotational moment generated by the frictional force μf3, and a rotational moment μf3 that attempts to rotate the object O in the direction opposite to the direction of the arrow RO around the gripping position G. Xr6 is calculated.

  The gripping quality evaluation value calculation unit 46 calculates the total of the calculated three rotation moments as the rotation evaluation value τ_ijk. In this way, the gripping quality evaluation value calculation unit 46 uses the friction force applied from the object O to each gripping part by the maximum vertical drag that can be supported by each gripping part included in the first end effector E1. A rotation evaluation value τ_ijk is calculated based on the generated rotation moment. The gripping quality evaluation value calculation unit 46 may be configured to calculate another value based on these three rotational moments as the rotational evaluation value τ_ijk, and a value based on another value different from these three rotational moments. May be calculated as the rotation evaluation value τ_ijk.

  After the translation evaluation value r_ijk and the rotation evaluation value τ_ijk are calculated as described above, the gripping quality evaluation value calculation unit 46 calculates the translational gripping quality evaluation value 1 ^ Q_ik based on the above equation (2). Based on the above equation (3), the rotary grip quality evaluation value 2 ^ Q_ik is calculated. Then, the gripping quality evaluation value calculation unit 46 calculates the sum of the calculated translational gripping quality evaluation value 1 ^ Q_ik and the rotational gripping quality evaluation value 2 ^ Q_ik as the gripping quality evaluation value Q_ik. That is, the grip quality evaluation value calculation unit 46 calculates the grip quality evaluation value Q_ik based on the following equation (10).

  Q_ik = 1 ^ Q_ik + 2 ^ Q_ik (10)

  Note that the gripping quality evaluation value calculation unit 46 uses the weight ω to make the relative gripping quality evaluation value 1 ^ Q_ik and the rotational gripping quality evaluation value 2 ^ Q_ik relative to each other as shown in the following equation (11). The structure which adjusts a magnitude | size may be sufficient.

  Q_ik = 1 ^ Q_ik + ω × 2 ^ Q_ik (11)

  The weight ω is a value between 0 and 1. Thereby, for example, the gripping quality evaluation value calculation unit 46 translates the gripping quality evaluation value 1 ^ Q_ik representing the difficulty of translation of the object O with respect to the first end effector E1, and the object O with respect to the first end effector E1. The rotational grip quality evaluation value 2 ^ Q_ik representing the difficulty of rotation can be adjusted to the same level.

  After calculating the grip quality evaluation value Q_ik for each state of the first end effector E1 selected in step S123 by repeating the process of step S125, the hand state determination unit 47 performs the plurality of grips calculated in step S125. The maximum grip quality evaluation value Q_ik is extracted from the quality evaluation value Q_ik. Then, the hand state determination unit 47 is suitable for causing the first end effector E1 to grip the object O with the state of the first end effector E1 used to calculate the extracted maximum grip quality evaluation value Q_ik. The state of the first end effector E1 is determined (step S130).

  Next, the robot control unit 48 causes the robot 20 to realize the state of the first end effector E1 determined in step S130, and causes the robot 20 to grip the object O. Then, the robot controller 48 feeds the object O gripped by the robot 20 to the feeding area A to the robot 20 (step S140), and ends the process.

  As described above, the robot control device 30 of the robot 20 in the embodiment includes the position and posture of the object O, error information on the position and posture, and the state of the hand (in this example, the first end effector E1). Based on the above, the hand state is determined. Accordingly, the robot control device 30 can cause the hand to hold the object O according to the state of the hand suitable for causing the hand to hold the object O.

  In addition, the robot control device 30 detects the position and orientation of the object O from an image (in this example, a captured image captured by the third imaging unit 23). Thereby, the robot controller 30 can cause the hand to hold the object O according to the state of the hand determined based on the position and orientation of the object O detected from the image.

  Further, the robot control apparatus 30 evaluates the state of the hand based on the position and posture of the object O, error information on the position and posture, and the state of the hand (for example, the translation evaluation described above). The state of the hand is determined according to at least one of a value, a rotation evaluation value, a translational grip quality evaluation value, and a rotation grip quality evaluation value. Accordingly, the robot control device 30 can cause the hand to hold the object O according to the state of the hand suitable for causing the hand to hold the object O.

  Further, the robot control device 30 determines whether the hand O is gripped by the hand according to an evaluation value that is a value indicating the difficulty of changing the relative position and posture of the hand and the object O. Determine the state. Thereby, the robot control apparatus 30 responds to the evaluation value which is a value indicating the difficulty of changing the relative position and posture of the hand and the object O when the object O is gripped by the hand. The object O can be held according to the determined hand state.

  In addition, the robot control device 30 determines an evaluation value including a value indicating the difficulty of translation of the object O with respect to the hand when the object O is gripped by the hand (for example, the translation evaluation value or the translation described above). The hand state is determined based on the gripping quality evaluation value. Thereby, when the object O is gripped by the hand, the robot controller 30 controls the object according to the state of the hand determined according to the evaluation value including the value indicating the difficulty of translation of the object O with respect to the hand. The object O can be gripped.

  In addition, the robot control device 30 is configured such that when the object O is gripped by the hand, an evaluation value including a value indicating the difficulty of rotation of the object O with respect to the hand (for example, the rotation evaluation value or the rotation described above). The hand state is determined based on the gripping quality evaluation value. Thereby, when the object O is gripped by the hand, the robot control device 30 performs the object according to the state of the hand determined based on the evaluation value including the value indicating the difficulty of rotation of the object O with respect to the hand. The object O can be gripped.

  The robot control device 30 determines the state of the hand based on the position and posture of the object O, error information on the position and posture, and the state of the hand including the relative position and posture between the hand and the object. decide. Thereby, the robot control apparatus 30 can hold the object O according to the hand state determined based on the hand state including the relative position and posture between the hand and the object O.

  In addition, the robot control device 30 includes the position and posture of the object O, error information on the position and posture, and the gripping parts included in the hand (in this example, the gripping part C11, the gripping part C12, and the gripping part C13). The hand state is determined based on the hand state including the angle. Thereby, the robot control apparatus 30 can hold | grip the target object O with the state of the hand determined based on the state of the hand including the angle between the holding | gripping parts with which a hand is equipped.

  In addition, the robot control device 30 determines the hand state based on the position and orientation of the object O, error information that is information indicating a probability distribution (normal distribution in this example), and the hand state. Thereby, the robot control apparatus 30 can hold the object O according to the hand state determined based on the position and orientation of the object O, the information indicating the probability distribution, and the hand state.

  Further, the robot control device 30 determines the hand state based on the position and orientation of the object O, error information that is information indicating an error range, and the hand state. Thereby, the robot control apparatus 30 can hold the target object O according to the hand state determined based on the position and posture of the target object O, the information indicating the error range, and the hand state.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, and the like are possible without departing from the gist of the present invention. May be.

  Further, a program for realizing the function of an arbitrary component in the apparatus described above (for example, the robot control apparatus 30 of the robot 20) is recorded on a computer-readable recording medium, and the program is read into a computer system. May be executed. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD (Compact Disk) -ROM, or a storage device such as a hard disk built in the computer system. . Furthermore, “computer-readable recording medium” means a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 11 ... 1st force detection part, 12 ... 2nd force detection part, 20 ... Robot, 21 ... 1st imaging part, 22 ... 2nd imaging part, 23 ... 3rd imaging part, 24 ... 4th imaging part, 30 ... Robot control device 31 ... CPU 32 ... storage unit 33 ... input receiving unit 34 ... communication unit 35 ... display unit 36 ... control unit 40 ... force detection information acquisition unit 41 ... imaging control unit 42 ... Image acquisition unit, 44 ... position and orientation detection unit, 46 ... gripping quality evaluation value calculation unit, 47 ... hand state determination unit, 48 ... robot control unit

Claims (12)

A robot with a hand,
A robot control device that determines the state of the hand based on the position and posture of the object, error information of the position and posture, and the state of the hand;
A robot system comprising: The robot controller detects the position and orientation of the object from an image;
The robot system according to claim 1. The robot control device determines the state of the hand according to an evaluation value for evaluating the state of the hand based on the position and orientation of the object, the error information, and the state of the hand;
The robot system according to claim 1 or 2. The evaluation value is a value indicating a difficulty in changing a relative position and posture between the hand and the object when the object is gripped by the hand.
The robot system according to claim 3. The evaluation value includes a value indicating difficulty in translation of the object relative to the hand when the object is gripped by the hand.
The robot system according to claim 4. The evaluation value includes a value indicating difficulty in rotation of the object relative to the hand when the object is gripped by the hand.
The robot system according to claim 4 or 5. The state of the hand includes a relative position and posture between the hand and the object.
The robot system according to any one of claims 1 to 6. The state of the hand includes an angle between gripping parts included in the hand,
The robot system according to any one of claims 1 to 7. The error information is information indicating a probability distribution.
The robot system according to any one of claims 1 to 8. The error information is information indicating an error range.
The robot system according to any one of claims 1 to 9. With a hand,
Determining the state of the hand according to an evaluation value based on the position and orientation of the object, error information on the position and orientation, and the state of the hand;
robot. Determining the state of the hand according to an evaluation value based on the position and posture of the object, error information on the position and posture, and the state of the hand included in the robot;
Robot control device.

Images