JP2019034386A - Robot control device, robot, robot system, and robot control method - Google Patents

Abstract

To provide a robot control device which can accurately detect an attention part of an object on the basis of a picked-up image picked-up by an imaging part.SOLUTION: A robot control device controls a robot comprising a first movable part moving an object. The robot control device moves an attention part of the object by the first movable part into a first depth range being a range from a focal position in a depth direction of an imaging part to a position farthest from the imaging part in the depth direction out of ranges in the depth direction of a depth of field of the imaging part, and causes the imaging part to pick-up a region including the attention part.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a robot control device, a robot, a robot system, and a robot control method.

  Research and development have been conducted on a technique for causing a robot to perform work related to an object based on a captured image obtained by imaging the object.

  In this regard, a robot is known that includes an imaging unit, causes the grasped object to approach the imaging unit, causes the imaging unit to image the object, and detects the position and orientation of the object based on the captured image captured by the imaging unit. (See Patent Document 1).

Japanese Patent Laying-Open No. 2015-226554

  However, in such a robot, an object other than the object may be imaged as the background of the object together with the object in the captured image captured by the imaging unit. As a result, the robot may not be able to accurately detect the target object from the captured image.

In order to solve at least one of the above-described problems, one aspect of the present invention is a robot control apparatus that controls a robot including a first movable unit that moves an object. The target portion is within a first depth range that is a range from a focal position in the depth direction of the imaging unit to a position farthest from the imaging unit in the depth direction in a range in the depth direction of the depth of field of the imaging unit. It is a robot control apparatus that moves the image to cause the imaging unit to image the region including the target portion.
With this configuration, the robot control device causes the first movable unit to move the target portion of the object from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit to the imaging unit in the depth direction. Is moved within a first depth range, which is a range from the farthest to the farthest position, and the imaging unit captures an area including the target portion. Thereby, the robot control apparatus can detect the attention part of the target object in the said imaging part with high precision based on the captured image imaged by the imaging part.

In another aspect of the present invention, a configuration in which the imaging unit is fixed so that the position and orientation of the imaging unit in a robot coordinate system do not move may be used in the robot control device.
With this configuration, the robot control device allows the first movable unit to fix the target portion of the object so that the position and orientation of the imaging unit in the robot coordinate system do not move. Is moved within a first depth range, which is a range from the focal position in the depth direction of the imaging unit to the position farthest from the imaging unit in the depth direction, and causes the imaging unit to capture an area including the target portion. As a result, the robot controller accurately determines the target portion of the object in the imaging unit based on the captured image captured by the imaging unit that is fixed so that the position and orientation of the imaging unit in the robot coordinate system do not move. Can be detected well.

In another aspect of the present invention, in the robot control device, a configuration in which the imaging unit is moved relative to the object by the second movable unit may be used.
With this configuration, the robot control apparatus moves the imaging unit relative to the object by the second movable unit. Accordingly, the robot control device can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit moved relative to the target by the second movable unit. Can do.

In another aspect of the present invention, in the robot control device, a configuration in which the imaging unit is a stereo camera may be used.
With this configuration, the robot control device has a range from the focal position in the depth direction of the image capturing unit to the position farthest from the image capturing unit in the depth direction of the range of the depth of field of the image capturing unit that is a stereo camera. The image is moved within a certain first depth range, and the imaging unit is caused to image a region including the target portion. Thereby, the robot control apparatus can detect the attention part of the target object in the said imaging part with high precision based on the captured image imaged by the imaging part which is a stereo camera.

In another aspect of the present invention, the robot control device may be configured to detect the portion of interest included in the captured image captured by the imaging unit.
With this configuration, the robot control device detects a target portion of the target object included in the captured image captured by the imaging unit. As a result, the robot control apparatus can cause the robot to accurately perform the work related to the target portion detected from the captured image.

In another aspect of the present invention, a configuration may be used in a robot control device that performs processing according to the portion of interest detected from the captured image.
With this configuration, the robot control apparatus performs processing according to the target portion detected from the captured image. Thereby, the robot control apparatus can perform the process according to the attention part detected from the captured image with high accuracy.

In another aspect of the present invention, the robot control device may be configured to perform processing related to the operation of the robot in accordance with the attention portion detected from the captured image.
With this configuration, the robot control apparatus performs a process related to the operation of the robot according to the target portion detected from the captured image. Thereby, the robot control apparatus can perform the process regarding the operation of the robot with high accuracy.

Moreover, the other aspect of this invention is equipped with the 1st movable part which moves a target object, The said attention part of the target object is made into the said depth direction of the depth of field of an imaging part by the said 1st movable part. A robot that moves within a first depth range, which is a range from a focal position in the depth direction of the imaging unit to a position farthest from the imaging unit in the depth direction, and causes the imaging unit to capture an area including the target portion It is.
With this configuration, the robot moves the target portion of the object from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit out of the range in the depth direction of the depth of field of the imaging unit by the first movable unit. It moves within the first depth range, which is a range up to a far position, and causes the imaging unit to image a region including the target portion. Accordingly, the robot can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit.

According to another aspect of the present invention, there is provided a robot system including a robot including a first movable unit that moves an object and a robot control device that controls the robot, wherein the robot control device includes the first control unit. A range from the focal position in the depth direction of the imaging unit to the farthest position from the imaging unit in the depth direction in the depth direction of the depth of field of the imaging unit by the movable part. This is a robot system that moves within a first depth range, and causes the imaging unit to image a region including the target portion.
With this configuration, the robot system allows the first movable unit to move the target portion of the object from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit from the imaging unit in the depth direction. It moves within the first depth range, which is the range up to the farthest position, and causes the imaging unit to image the region including the target portion. Accordingly, the robot system can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit.

According to another aspect of the present invention, there is provided a robot control method for controlling a robot including a first movable unit that moves an object, wherein the target portion of the object is captured by an imaging unit by the first movable unit. Move within the first depth range, which is the range from the focal position in the depth direction of the imaging unit to the position farthest from the imaging unit in the depth direction, in the depth direction of the depth of field, and A robot control method for imaging a region including the target portion.
With this configuration, the robot control method allows the first movable unit to move the target portion of the object from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit to the imaging unit in the depth direction. Is moved within a first depth range, which is a range from the farthest to the farthest position, and the imaging unit captures an area including the target portion. Accordingly, the robot system can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit.

According to another aspect of the present invention, there is provided a robot control device that controls a robot including a first movable unit that moves an object. The robot control device includes a processor, and the processor uses the first movable unit to control the object. The target portion is within a first depth range that is a range from a focal position in the depth direction of the imaging unit to a position farthest from the imaging unit in the depth direction in a range in the depth direction of the depth of field of the imaging unit. The robot control device is configured to execute a command to cause the imaging unit to image a region including the target portion.
With this configuration, the robot control device causes the first movable unit to move the target portion of the object from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit to the imaging unit in the depth direction. Is moved within a first depth range, which is a range from the farthest to the farthest position, and the imaging unit captures an area including the target portion. Thereby, the robot control apparatus can detect the attention part of the target object in the said imaging part with high precision based on the captured image imaged by the imaging part.

  As described above, in the robot control device, the robot, the robot system, and the robot control method, the first movable unit moves the target portion of the object in the depth direction of the imaging unit out of the range in the depth direction of the depth of field of the imaging unit. It moves within the first depth range, which is the range from the focal position to the farthest position from the imaging unit in the depth direction, and causes the imaging unit to image the region including the target portion. Accordingly, the robot control device, the robot, the robot system, and the robot control method can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit.

It is a figure showing an example of composition of robot system 1 concerning an embodiment. It is an example of the hardware constitutions of the robot control apparatus 30 in case the robot control apparatus 30 is comprised by several information processing apparatus. It is a figure which shows an example of the target object. In this example, the upper surface of the object O shown in FIG. 3 is represented as the target portion FO. 2 is a diagram illustrating an example of a functional configuration of a robot control device 30. FIG. It is a figure which shows an example of the flow of a process in which the robot control apparatus 30 moves 1st arm A1, in order to make the 3rd imaging part C3 image the target object O. FIG. It is a figure which shows an example of the target object O in the state in which an imaging position and an imaging posture and a position and attitude | position correspond. It is a block diagram which shows an example of the robot system 2 which concerns on the modification of embodiment.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Overview of the robot system>
First, an outline of the robot system 1 will be described. FIG. 1 is a diagram illustrating an example of a configuration of a robot system 1 according to the embodiment.

  The robot system 1 causes the imaging unit to image a target object. For example, the robot system 1 causes the robot to perform an operation related to the object based on the captured image obtained by the imaging unit capturing the object.

  Here, in a robot system X different from the robot system 1 (for example, a conventional robot system), an object other than the object is imaged in the captured image captured by the imaging unit as a background of the object together with the object. There is. As a result, the robot system X may not be able to accurately detect an object from the captured image.

  Therefore, the robot system 1 includes a robot including a first movable unit that moves the object, and the first movable unit captures the target portion of the target in the depth direction of the depth of field of the imaging unit. Is moved into a first depth range that is a range from the focal position in the depth direction to the position farthest from the imaging unit in the depth direction, and causes the imaging unit to image the region including the target portion. Here, the depth direction is a direction from the imaging element of the imaging unit to the lens of the imaging unit among the directions along the optical axis of the imaging unit. Thereby, the robot system 1 is a captured image in which the target portion of the object imaged by the imaging unit is captured without being lost, and a portion other than the target portion of the object imaged by the imaging unit. It is possible to acquire a picked-up image in which most of them are picked up. Here, in the captured image, the portion is blurred, and the smallest circle of confusion within a region where the portion of the captured image is captured is a predetermined diameter and allowable. This means that the diameter is larger than the allowable diameter which is the smallest diameter. In this case, it is difficult to detect any shape from the portion from the captured image. As a result, the robot system 1 suppresses erroneously detecting a portion other than the target portion in the captured image as the target portion based on the captured image captured by the imaging unit, and also in the captured image. The target portion can be detected with high accuracy.

  Below, the structure of the robot system 1 and the process which the robot system 1 moves a 1st movable part in order to make an imaging part image-capture a target object among the processes which the robot system 1 performs are demonstrated.

<Robot system configuration>
First, the configuration of the robot system 1 will be described.
FIG. 1 is a diagram illustrating an example of a configuration of a robot system 1 according to the embodiment. The robot system 1 includes a robot 20 which is an example of the above robot. The robot 20 includes a robot control device 30 that is an example of the robot control device described above. The robot system 1 may be configured to include other devices such as an imaging unit separate from the robot 20 in addition to the robot 20.

  The robot 20 is a double-arm robot including a first arm A1, a second arm A2, a support base B that supports the first arm A1 and the second arm A2, and a robot control device 30 inside the support base B. . The robot 20 may be a double-arm robot having three or more arms or a single-arm robot having one arm instead of the double-arm robot. The robot 20 may be another robot such as a SCARA (horizontal articulated) robot, a Cartesian coordinate robot, or a cylindrical robot. The orthogonal coordinate robot is, for example, a gantry robot.

  The support base B is divided into two parts along a direction orthogonal to the installation surface on which the robot 20 is installed. Note that the support base B may have a configuration that is not divided, or may be a configuration that is divided into three or more parts. The support base upper part B2 which is the part far from the installation surface among the two parts is rotatable with respect to the support base lower part B1 which is the part closer to the installation surface among the two parts. . That is, the robot 20 has a joint J0 that is a joint connecting the upper support base B2 and the lower support base B1. For example, the rotation surface when the support base upper portion B2 rotates is parallel to the installation surface. A robot control device 30 is provided inside the lower support base B1. Further, the support base upper part B2 can translate in the vertical direction with respect to the support base lower part B1. In this example, the upward direction is a direction from the support base lower part B1 toward the support base upper part B2 out of two directions orthogonal to the installation surface. Further, the downward direction is a direction opposite to the upward direction in this example. The rotating surface may be non-parallel to the installation surface. Further, the support base upper part B2 may be configured such that it cannot be translated in the vertical direction with respect to the support base lower part B1.

  The first arm A1 includes a first end effector E1 and a first manipulator M1.

  The first end effector E1 is an end effector that holds an object. In this example, the first end effector E1 includes a finger part, and holds the object by holding the object by the finger part. Alternatively, the first end effector E1 may be configured to hold the object by lifting the object by air suction, magnetic force, another jig, or the like. In this example, holding means that the object can be lifted.

  The first end effector E1 is communicably connected to the robot controller 30 via a cable. Thus, the first end effector E1 performs an operation based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB (Universal Serial Bus), for example. The first end effector E1 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The first manipulator M1 includes seven joints and a first imaging unit C1. Each of the seven joints includes an actuator (not shown). That is, the first arm A1 including the first manipulator M1 is a 7-axis vertical articulated arm. The first arm A1 performs an operation with seven degrees of freedom by a coordinated operation by the support base B, the first end effector E1, the first manipulator M1, and the actuators of the seven joints. The first arm A1 may be configured to operate with a degree of freedom of 6 axes or less, or may be configured to operate with a degree of freedom of 8 axes or more.

  When the first arm A1 operates with a degree of freedom of 7 axes, the posture that the first arm A1 can take is increased as compared with a case where the first arm A1 operates with a degree of freedom of 6 axes or less. Thereby, for example, the first arm A1 can be smoothly operated, and interference with an object existing around the first arm A1 can be easily avoided. In addition, when the first arm A1 operates with a degree of freedom of 7 axes, the control of the first arm A1 is easier and requires less calculation than when the first arm A1 operates with a degree of freedom of 8 axes or more. is there.

  Each of the seven actuators provided in the first manipulator M1 is communicably connected to the robot controller 30 via a cable. Accordingly, the actuator operates the first manipulator M1 based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB. Further, some or all of the seven actuators included in the first manipulator M1 may be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). Good.

  The first imaging unit C1 is a camera that includes, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like as an imaging element that converts the collected light into an electrical signal. In this example, the first imaging unit C1 is provided in a part of the first manipulator M1. Therefore, the first imaging unit C1 moves according to the movement of the first manipulator M1. Further, the range in which the first imaging unit C1 can capture an image changes according to the movement of the first arm A1. The first imaging unit C1 captures a two-dimensional image in the range. The first imaging unit C1 may be configured to capture a still image in the range, or may be configured to capture a moving image in the range. The first imaging unit C1 is an example of the imaging unit described above.

  The first imaging unit C1 is communicably connected to the robot control device 30 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The first imaging unit C1 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second arm A2 includes a second end effector E2 and a second manipulator M2.

  The second end effector E2 is an end effector that holds an object. In this example, the second end effector E2 includes a finger part, and holds the object by holding the object by the finger part. Alternatively, the second end effector E2 may be configured to hold the object by lifting the object by air suction, magnetic force, other jigs, or the like.

  The second end effector E2 is communicably connected to the robot controller 30 via a cable. Accordingly, the second end effector E2 performs an operation based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB. The second end effector E2 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The second manipulator M2 includes seven joints and a second imaging unit C2. Each of the seven joints includes an actuator (not shown). That is, the second arm A2 including the second manipulator M2 is a 7-axis vertical articulated arm. The second arm A2 performs an operation with seven degrees of freedom by a coordinated operation by the support base B, the second end effector E2, the second manipulator M2, and the actuators of the seven joints. The second arm A2 may be configured to operate with a degree of freedom of 6 axes or less, or may be configured to operate with a degree of freedom of 8 axes or more.

  When the second arm A2 operates with a degree of freedom of seven axes, the posture that the second arm A2 can take is increased as compared with a case where the second arm A2 operates with a degree of freedom of six axes or less. Thereby, for example, the second arm A2 can move smoothly and can easily avoid interference with an object existing around the second arm A2. In addition, when the second arm A2 operates with a degree of freedom of 7 axes, the control of the second arm A2 is easier and requires less calculation than when the second arm A2 operates with a degree of freedom of 8 axes or more. is there.

  Each of the seven actuators included in the second manipulator M2 is communicably connected to the robot controller 30 via a cable. Thus, the actuator operates the second manipulator M2 based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB. Further, some or all of the seven actuators included in the second manipulator M2 may be configured to be connected to the robot controller 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). Good.

  The second imaging unit C2 is a camera that includes, for example, a CCD, a CMOS, or the like as an imaging device that converts collected light into an electrical signal. In this example, the second imaging unit C2 is provided in a part of the second manipulator M2. Therefore, the second imaging unit C2 moves according to the movement of the second manipulator M2. In addition, the range in which the second imaging unit C2 can capture an image changes according to the movement of the second arm A2. The second imaging unit C2 captures a two-dimensional image in the range. The second imaging unit C2 may be configured to capture a still image in the range, or may be configured to capture a moving image in the range. The second imaging unit C2 is an example of the above imaging unit.

  The second imaging unit C2 is communicably connected to the robot control device 30 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The second imaging unit C2 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

The robot 20 includes a third imaging unit C3 and a fourth imaging unit C4.
The third imaging unit C3 is a camera provided with, for example, a CCD, a CMOS, or the like as an imaging device that converts collected light into an electrical signal. The third image capturing unit C3 is provided in a region where the fourth image capturing unit C4 can capture an image together with the fourth image capturing unit C4. The third imaging unit C3 is fixed so that the position and orientation of the third imaging unit C3 in the robot coordinate system do not move. The third imaging unit C3 is an example of the above imaging unit. The third imaging unit C3 is communicably connected to the robot control device 30 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The third imaging unit C3 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The fourth imaging unit C4 is a camera provided with, for example, a CCD, a CMOS, or the like as an imaging device that converts collected light into an electrical signal. The fourth imaging unit C4 is provided in a region where the third imaging unit C3 can capture an image, together with the third imaging unit C3, in a portion where stereo imaging is possible. The fourth imaging unit C4 is fixed so that the position and orientation of the fourth imaging unit C4 in the robot coordinate system do not move. The fourth imaging unit C4 is an example of the imaging unit described above. The fourth imaging unit C4 is connected to the robot controller 30 via a cable so as to be communicable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The fourth imaging unit C4 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  Here, the third imaging unit C3 and the fourth imaging unit C4 in this example are provided in the above-described support base upper part B2. For this reason, the range in which the third imaging unit C3 and the fourth imaging unit C4 can perform stereo imaging changes due to translation in the vertical direction of the support base upper part B2 with respect to the support base lower part B1.

  In addition, in the support base upper part B2 provided with the third image pickup part C3 and the fourth image pickup part C4, both the third image pickup part C3 and the fourth image pickup part C4 are rotated relative to the support base upper part B2. Thus, a tilt mechanism is provided that changes the direction of the optical axis of each of the third imaging unit C3 and the fourth imaging unit C4. The support base upper part B2 may be configured not to be provided with a tilt mechanism.

  In this example, each functional unit included in the robot 20 described above acquires a control signal from the robot control device 30 built in the robot 20. Then, each functional unit performs an operation based on the acquired control signal. Note that the robot 20 may be configured to be controlled by the robot control device 30 installed outside, instead of the configuration incorporating the robot control device 30. Further, the robot 20 may be configured not to include some or all of the first imaging unit C1, the second imaging unit C2, the third imaging unit C3, and the fourth imaging unit C4.

  The robot control device 30 includes at least one processor and at least one memory. In the robot control device 30, the at least one processor may be provided in one information processing device, or may be provided in a distributed manner in a plurality of information processing devices. Further, in the robot control device 30, the at least one memory may be configured to be provided in one information processing device, or may be configured to be distributed and provided in a plurality of information processing devices.

  In the example illustrated in FIG. 1, the robot control device 30 includes a processor 31 that is one processor provided in the information processing device PC1 and a memory 32 that is one memory provided in the information processing device PC1. ing. The information processing apparatus PC1 may be configured to include other processors in addition to the processor 31, or may be configured to include other memories in addition to the memory 32.

  The information processing apparatus PC1 is, for example, a workstation, a desktop PC (Personal Computer), a notebook PC, a tablet PC, a multi-function mobile phone terminal (smartphone), an electronic book reader with a communication function, a PDA (Personal Digital Assistant), or the like. .

  The processor 31 is, for example, a CPU (Central Processing Unit). The processor 31 may be another processor such as an FPGA (Field-Programmable Gate Array). The processor 31 executes various commands stored in a memory provided in the robot control device 30. The processor 31 also executes various commands stored in the memory of another device.

  The memory 32 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), and a random access memory (RAM). That is, the memory 32 includes a temporary storage device and a non-temporary storage device. Note that the memory 32 may be an external storage device connected by a digital input / output port such as a USB instead of the one built in the information processing apparatus PC1. The memory 32 stores various information processed by the processor 31 or a processor of another device, various commands (for example, programs, codes, etc.) executable by a computer such as commands 321 and 322, various images, and the like.

  Each of the command 321 and the command 322 is a part of a plurality of commands (that is, a plurality of commands that can be executed by a computer) executed by the processor 31 in order for the processor 31 to configure the robot control device 30.

In addition, the robot control device 30 includes a communication unit 34 as a hardware function unit in order to communicate with other devices.
The communication unit 34 includes a digital input / output port such as a USB, an Ethernet (registered trademark) port, and the like.

  Note that the robot control device 30 is replaced by an information processing device PC1 including one processor 31 and one memory 32, as shown in FIG. It may be configured. FIG. 2 is an example of a hardware configuration of the robot control device 30 when the robot control device 30 is configured by a plurality of information processing devices.

  The information processing apparatus PC2 illustrated in FIG. 2 includes a processor 41 that is one processor, a memory 42 that is one memory, and a communication unit 44. Note that the information processing apparatus PC2 may be configured to include other processors in addition to the processor 41. In addition to the memory 42, the information processing apparatus PC2 may be configured to include another memory.

  The information processing apparatus PC2 is, for example, a workstation, a desktop PC, a notebook PC, a tablet PC, a multi-function mobile phone terminal (smartphone), an electronic book reader with a communication function, a PDA, or the like. The information processing apparatus PC2 may be the teaching apparatus 40.

  The processor 41 is a CPU. The processor 41 may be another processor such as an FPGA. The processor 41 executes various commands stored in the memory 32. Further, the processor 41 executes various commands stored in the memory of another device.

  The memory 42 includes, for example, an HDD, SSD, EEPROM, ROM, RAM, and the like. That is, the memory 42 includes a temporary storage device and a non-temporary storage device. The memory 42 may be an external storage device connected by a digital input / output port such as a USB instead of the one built in the information processing apparatus PC2. The memory 42 stores various information processed by the processor 41 or the processor of another device, various commands (for example, programs, codes, etc.) executable by the computer, various images, and the like.

  The communication unit 44 includes a digital input / output port such as a USB, an Ethernet (registered trademark) port, and the like.

  The information processing apparatus PC3 illustrated in FIG. 2 includes a processor 51 that is one processor, a memory 52 that is one memory, and a communication unit 54. The information processing apparatus PC3 may be configured to include another processor in addition to the processor 51. In addition to the memory 52, the information processing apparatus PC3 may be configured to include another memory.

  The information processing apparatus PC3 is, for example, a workstation, a desktop PC, a notebook PC, a tablet PC, a multi-function mobile phone terminal (smartphone), an electronic book reader with a communication function, a PDA, or the like.

  The processor 51 is a CPU. The processor 51 may be another processor such as an FPGA. The processor 51 executes various commands stored in the memory 52. Further, the processor 51 executes various commands stored in the memory of another device.

  The memory 52 includes, for example, HDD, SSD, EEPROM, ROM, RAM, and the like. That is, the memory 52 includes a temporary storage device and a non-temporary storage device. Note that the memory 52 may be an external storage device connected by a digital input / output port such as a USB instead of the one built in the information processing apparatus PC3. The memory 52 stores various information processed by the processor 51 or the processor of another device, various commands (for example, programs, codes, etc.) executable by the computer, various images, and the like.

  The communication unit 54 includes a digital input / output port such as a USB, an Ethernet (registered trademark) port, and the like.

  In the example illustrated in FIG. 2, the information processing apparatus PC1 and the information processing apparatus PC2 are connected to be communicable with each other by wireless or wired communication. Further, in this example, the information processing device PC1 and the information processing device PC3 are connected to be communicable with each other by wireless or wired via a LAN (Local Area Network). In this example, the information processing device PC2 and the information processing device PC3 are connected to be communicable with each other by a wireless or wired connection via a LAN.

  In the example illustrated in FIG. 2, the robot control device 30 includes a processor 31, a processor 41, at least one of the processors 51, a memory 32, a memory 42, and at least one of the memories 52. Composed. When the robot control device 30 is configured by two or more of the processor 31, the processor 41, and the processor 51, the two or more processors constituting the robot control device 30 perform communication by the communication unit. In cooperation, the functions of the robot control device 30 are realized. In this case, the two or more processors execute processing based on the function of the robot control device 30 based on a command stored in at least one of the memory 32, the memory 42, and the memory 52.

  Note that, as in the example illustrated in FIG. 2, when the robot control device 30 includes a plurality of information processing devices, the teaching device 40 is connected to be communicable with at least one of the plurality of information processing devices.

  Returning to FIG. The robot control device 30 controls the robot 20. More specifically, the robot control device 30 causes the robot 20 to perform an operation related to the object O, which is an example of the object, according to an operation program stored in advance in the memory 32, for example. In the example shown in FIG. 1, the object O is held in advance by the first end effector E1. The object O may be configured to be held in advance by the second end effector E2, or may be configured to be stored in advance by both the first end effector E1 and the second end effector E2. . Hereinafter, as an example, a case where the object O is held in advance by the first end effector E1 as illustrated in FIG. 1 will be described. In this case, 1st arm A1 is an example of said 1st movable part. Note that the operation performed by the first arm A1 by the processing described below may be performed by the second arm A2. In this case, the second arm A2 is an example of the first movable part. Further, the operation performed by the first arm A1 by the processing described below may be performed by both the first arm A1 and the second arm A2. In this case, both the first arm A1 and the second arm A2 are examples of the first movable part.

  The object O is an industrial part or member to be assembled into a product. In the example illustrated in FIG. 1, the object O is represented as a rectangular parallelepiped object. The object O may be another object such as a daily necessities or a living body instead of an industrial part or member. Further, the shape of the object O may be another shape such as a cylindrical shape instead of the rectangular parallelepiped shape.

  Here, when the robot control device 30 causes the robot 20 to perform an operation related to the object O, the robot control apparatus 30 controls the robot 20 so that the target portion of the object O is one of the first imaging unit C1 to the fourth imaging unit C4. Part or all are imaged. Hereinafter, as an example, a case will be described in which the robot control device 30 causes the third imaging unit C3 to image the target portion. Therefore, the operation performed by the third imaging unit C3 by the process described below may be performed by any one of the first imaging unit C1, the second imaging unit C2, and the fourth imaging unit C4, and the first imaging unit C1. -It may be performed by two or more imaging units of the fourth imaging unit C4. Further, when the operation is performed by the two or more imaging units, stereo imaging may be performed by the two or more imaging units as the operation. That is, in this case, the imaging unit is a stereo camera.

  The target part of the object O is a part determined according to the work related to the object O in the part of the object O. More specifically, in the case where the robot 20 performs the work, the target portion is a portion that the robot control device 30 causes the third imaging unit C3 to image (ie, the user performs the first operation). 3 is a portion including a desired portion to be imaged by the imaging unit C3. In the example illustrated in FIG. 3, the target portion is one of the six surfaces of the object O. FIG. 3 is a diagram illustrating an example of the object O. In this example, the upper surface of the object O shown in FIG. 3 is represented as the target portion FO.

  When the robot controller 30 causes the third imaging unit C3 to capture the target portion FO of the target object O, the first end effector E1 converts the target portion FO of the target object O to the depth of field of the third imaging unit C3. Of the range in the depth direction, the third imaging unit C3 is moved within a first depth range that is a range from the focal position in the depth direction of the third imaging unit C3 to the farthest position from the third imaging unit C3 in the depth direction. The region including the target portion FO is imaged. Thereby, the robot control device 30 is a captured image in which the target portion FO of the object O among the objects imaged by the third imaging unit C3 is captured without being lost, and is captured by the third imaging unit C3. A captured image in which most of the object other than the target portion FO is blurred and captured can be acquired from the third imaging unit C3. As a result, the robot control device 30 suppresses erroneous detection of a portion other than the target portion FO in the captured image as the target portion FO based on the captured image captured by the third imaging unit C3. The target portion FO in the captured image can be detected with high accuracy.

  After causing the third imaging unit C3 to capture the target portion FO of the object O, the robot control device 30 detects the target portion FO from the captured image based on the captured image captured by the third imaging unit C3. . The robot control device 30 detects the target portion FO from the captured image, for example, by processing such as pattern matching. Then, the robot control device 30 performs processing according to the detected target portion FO. This process is a process related to the operation of the robot 20. Specifically, for example, the process corresponding to the target portion FO determines whether or not foreign matter (scratches, unintentionally formed unevenness, dust, etc.) is present on the surface of the target portion FO. It is processing. After determining whether or not there is a foreign object on the surface, the robot control device 30 places the object O in an area corresponding to the determination result. In this example, when it is determined that there is no foreign object on the surface, the robot control device 30 places the object O in a predetermined material supply area (not shown). On the other hand, when it determines with the foreign material existing on the said surface, the robot control apparatus 30 mounts the target object O in the material removal area | region which is determined beforehand and which is not shown in figure. Further, for example, when the object O is an electric driver and the target portion FO is the tip of the electric driver, the robot control device 30 supplies a screw to the tip as processing according to the detected target portion FO. Processing to determine whether or not the material is used. When the screw is not supplied to the tip, the robot controller 30 operates the robot 20 to supply the screw to the tip. On the other hand, when a screw is supplied to the tip, the robot 20 is operated, and the screw is fastened to a predetermined object by the electric screwdriver. In addition, for example, when the object O is a plate and the target portion FO is a marker such as a barcode provided on the plate, the robot control device 30 displays the information indicated by the marker detected as the target portion FO. Read from the marker. Then, the robot control device 30 performs processing based on the read information. Note that the process corresponding to the target portion FO detected by the robot control device 30 may be a process other than the process related to the operation of the robot 20. Further, the process related to the operation of the robot 20 may be another process.

<Functional configuration of robot controller>
Hereinafter, the functional configuration of the robot control device 30 will be described with reference to FIG. 4. FIG. 4 is a diagram illustrating an example of a functional configuration of the robot control device 30.

  The robot control device 30 includes a memory 32, a communication unit 34, and a control unit 36.

  The control unit 36 controls the entire robot control device 30. The control unit 36 includes an imaging control unit 361, an image acquisition unit 363, a robot control unit 365, and an image processing unit 367. These functional units included in the control unit 36 are realized, for example, when the processor 31 executes various commands such as commands 321 and 322 stored in the memory 32. Also, some or all of the functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

  The imaging control unit 361 causes the first imaging unit C1 to image the range that can be captured by the first imaging unit C1. In addition, the imaging control unit 361 causes the second imaging unit C2 to image a range that can be captured by the second imaging unit C2. In addition, the imaging control unit 361 causes the third imaging unit C3 to image a range that can be captured by the third imaging unit C3. In addition, the imaging control unit 361 causes the fourth imaging unit C4 to image a range that can be captured by the fourth imaging unit C4.

  The image acquisition unit 363 acquires the captured image captured by the first imaging unit C1 from the first imaging unit C1. Further, the image acquisition unit 363 acquires a captured image captured by the second imaging unit C2 from the second imaging unit C2. Further, the image acquisition unit 363 acquires the captured image captured by the third imaging unit C3 from the third imaging unit C3. Further, the image acquisition unit 363 acquires the captured image captured by the fourth imaging unit C4 from the fourth imaging unit C4.

  The robot control unit 365 operates the robot 20.

  The image processing unit 367 performs image processing based on the captured image acquired by the image acquisition unit 363. For example, when the target portion FO of the object O is captured in the captured image, the image processing unit 367 detects the target portion FO from the captured image.

<Process in which the robot control device moves the first arm to cause the imaging unit to image the object>
Hereinafter, a process in which the robot control device 30 moves the first arm A1 in order to cause the third imaging unit C3 to image the object O will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of a flow of processing in which the robot control device 30 moves the first arm A1 in order to cause the third imaging unit C3 to image the object O. In the flowchart shown in FIG. 5, the case where the first end effector E1 already holds the object O at the timing before the process of step S110 shown in FIG. 5 is executed will be described.

  The robot control unit 365 reads the imaging position / orientation information stored in advance in the memory 32 from the memory 32 (step S110). The imaging position / orientation information is information indicating an imaging position and an imaging attitude that are predetermined positions and attitudes. Further, the imaging position and the imaging posture are positions and postures at which the robot 20 matches the position and posture of the object O when the third imaging unit C3 images the object O. In this example, the position of the object O is represented by the position of the center of gravity of the object O. Further, the posture of the object O is represented by the direction in the robot coordinate system of each coordinate axis in the three-dimensional local coordinate system associated with the position. Note that a method in which the robot control unit 365 controls the robot 20 so that the position and orientation of the object O coincide with the imaging position and imaging orientation may be a known method or a method that will be developed in the future. Good.

  Here, an imaging position and an imaging posture will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of the object O in a state where the imaging position and the imaging posture are in the same position and orientation. In the example illustrated in FIG. 6, portions other than the third imaging unit C3 and the fourth imaging unit C4 are omitted from the robot 20 in order to prevent the diagram from becoming complicated. The arrow C3A illustrated in FIG. 6 is an arrow indicating a direction from the image sensor of the third imaging unit C3 toward the lens of the third imaging unit C3 among the directions along the optical axis of the third imaging unit C3. It is an arrow which shows the depth direction of the depth of field of the 3rd imaging part C3. A position P1 illustrated in FIG. 6 indicates a focal position in the depth direction of the third imaging unit C3. A range DF illustrated in FIG. 6 indicates a range in the depth direction of the depth of field of the third imaging unit C3. A position P11 illustrated in FIG. 6 is a position within the range DF and a position farthest from the third imaging unit C3 in the depth direction. That is, the position P11 indicates the position farthest from the third imaging unit C3 among the positions within the range in the depth direction of the depth of field of the third imaging unit C3. A range DF1 illustrated in FIG. 6 indicates a first depth range that is a range in the depth direction and is a range from the position P1 to the position P11. Further, a position P12 illustrated in FIG. 6 is a position within the range DF and the position closest to the third imaging unit C3 in the depth direction. That is, the position P12 indicates the position closest to the third imaging unit C3 among the positions within the range in the depth direction of the depth of field of the third imaging unit C3. A range DF2 illustrated in FIG. 6 indicates a second depth range that is a range in the depth direction and is a range from the position P1 to the position P12. In the example shown in FIG. 6, the ratio of the range DF1 that is the first depth range and the range DF2 that is the second depth range is the range DF1: range DF2 = 2: 1. It may be a ratio.

  In this example, the imaging position is a position where the third imaging unit C3 can image the object O when the position of the object O matches the imaging position among the positions of the object O. In addition, the imaging position is a position in which the entire target portion FO of the object O is included in the above-described range DF1 among the positions of the object O. In this case, the imaging position may be a position where a part of the target portion FO of the object O is included in the range DF1. The imaging position is preferably closer to the position P11 than to the position P1 in the depth direction of the depth of field of the third imaging unit C3. In addition, the imaging position may be a position where both the first end effector E1 and the object O do not interfere with other objects (for example, the third imaging unit C3) among the positions of the object O. desirable.

  Further, in this example, the imaging posture is determined by the third imaging unit C3 when the position and orientation of the object O coincides with the imaging position and orientation in the orientation of the object O. It is a posture capable of imaging the whole. Note that the imaging posture may be a posture in which, in this case, the third imaging unit C3 can capture a part of the target portion FO of the target object O among the postures of the target object O. In addition, the imaging posture may be a posture in which both the first end effector E1 and the target object O do not interfere with other objects (for example, the third imaging unit C3) among the postures of the target object O. desirable.

  When the position and orientation of the object O coincide with such an imaging position and orientation, the target portion FO of the object O is imaged without being lost by the third imaging unit C3. In this case, at least a part of an object other than the target object O (or an object other than the target portion FO) among objects that can be imaged by the third imaging unit C3 is captured by the third imaging unit C3. Thereby, the robot control apparatus 30 suppresses erroneously detecting a part other than the target part FO in the captured image as the target part FO based on the captured image captured by the third imaging unit C3. The target portion FO in the imaging unit can be detected with high accuracy. Thus, the target portion of the object O in the first depth range (the range DF1 in this example) of the range in the depth direction of the depth of field of a certain imaging unit (in this example, the third imaging unit C3). Hereinafter, the relative positional relationship between the imaging unit and the object O in the case where the FO is included will be described as an imaging positional relationship for convenience of explanation.

  In addition, when the robot control device 30 causes the third imaging unit C3 and the fourth imaging unit C4 to perform stereo imaging of the target portion FO of the object O, the focal positions of the third imaging unit C3 and the fourth imaging unit C4 are set. It is desirable that the focal positions in the depth direction of the depth of field substantially match when viewed from a direction orthogonal to the depth direction. In this case, it is desirable that at least a part of the first depth range of the ranges in the depth direction of the depth of field of the third imaging unit C3 and the fourth imaging unit C4 overlap each other.

  Returning to FIG. After the processing of step S110 is performed, the robot control unit 365 operates the first arm A1, and the position and orientation of the object O are captured by the imaging position and orientation indicated by the imaging position and orientation information read from the memory 32 in step S110. The imaging posture is matched (step S120).

  Next, the imaging control unit 361 causes the third imaging unit C3 to image the range that can be imaged by the third imaging unit C3 (step S130).

  Next, the image acquisition unit 363 acquires the captured image captured by the third imaging unit C3 in step S130 from the third imaging unit C3 (step S140).

  Next, based on the captured image acquired by the image acquisition unit 363 in step S140, the image processing unit 367 detects a target portion FO captured in the captured image (step S150). Here, in the captured image, an object other than the object O is blurred and captured. For this reason, in step S150, the image processing unit 367 suppresses erroneous detection of a part other than the target part FO in the captured image as the target part FO and accurately detects the target part FO in the captured image. can do.

  Next, the image processing unit 367 and the robot control unit 365 perform processing corresponding to the target portion FO detected in step S150 as predetermined processing (step S160). For example, the image processing unit 367 determines, based on the target portion FO, whether or not foreign matter (scratches, unintentionally formed unevenness, dust, etc.) is present on the surface of the actual target portion FO. To do. Then, the robot control unit 365 places the object O in an area corresponding to the determination result by the image processing unit 367. Note that the processing in step S160 may be configured such that all processing is performed by the robot control unit 365. Further, the process of step S160 may be configured such that the determination process by the image processing unit 367 is performed and the placement process of the object O by the robot control unit 365 is not performed. After the process of step S160 is performed, the robot control unit 365 ends the process.

  Note that the robot control device 30 controls the robot 20, and the first imaging unit C1 and the second imaging unit with respect to the object O that is installed (fixed) so that the position and orientation thereof do not change in the robot coordinate system. The configuration may be such that at least one of the imaging units of C2 is relatively moved so that the relative positional relationship between the object O and the imaging unit matches the above-described imaging positional relationship.

  The robot control device 30 controls the robot 20 and is an imaging unit installed (fixed) so as not to change its position and posture in the robot coordinate system. The configuration may be such that the object O is relatively moved so that the relative positional relationship between the object O and the imaging unit matches the imaging positional relationship.

  In addition, the robot control device 30 controls the robot 20 to move at least one of the second imaging unit C2 provided in the second arm A2 and the object O held by the first arm A1, and the second imaging unit. A configuration in which the relative positional relationship between C2 and the object O matches the above-described imaging positional relationship may be employed. In this case, the second arm A2 is an example of a second movable part.

  The robot control device 30 controls the robot 20 to move at least one of the third imaging unit C3 provided in the upper support base B2 and the object O held by the first arm A1, and The relative positional relationship with the third imaging unit C3 may be configured to match the above-described imaging positional relationship. In this case, the support base upper part B2 is an example of a second movable part.

  When the third imaging unit C3 has an autofocus function, the robot control device 30 matches the position and orientation of the object O with the imaging position and orientation, and then the target portion FO of the object O. After focusing on the area including, the position of the focus in the depth direction of the depth of field of the third imaging unit C3 may be moved in a direction opposite to the depth direction by a predetermined distance.

<Modification of Embodiment>
Hereinafter, a modified example of the embodiment will be described with reference to FIG. In addition, in the modification of embodiment, the same code | symbol is attached | subjected with respect to the structure part similar to embodiment, and description is abbreviate | omitted.

  FIG. 7 is a configuration diagram illustrating an example of a robot system 2 according to a modification of the embodiment. In a modification of the embodiment, the robot system 2 includes a robot 20a that is a single-arm robot, an imaging unit C5 that is separate from the robot 20a, and a robot control device 30 that is separate from the robot 20a. The robot control device 30 is connected to the robot 20a so that they can communicate with each other wirelessly or by wire. In the robot system 2, the robot control device 30 may be configured to be built in the robot 20a. In the robot system 2, the imaging unit C5 may be configured to be included in the robot 20a.

<Robot system configuration>
Hereinafter, the configuration of the robot system 2 will be described with reference to FIG.

  The robot 20a is a single-arm robot including a third arm A3 and a support base B3 that supports the third arm A3.

The third arm A3 includes a third end effector E3 and a third manipulator M3.
The third end effector E3 is an end effector that holds an object. In this example, the third end effector E3 includes a finger part, and holds the object by holding the object by the finger part. Alternatively, the third end effector E3 may be configured to hold the object by lifting the object by air suction, magnetic force, another jig, or the like. In the example illustrated in FIG. 7, the object O is held in advance by the third end effector E3 instead of the first end effector E1 of the robot 20 in the embodiment.

  The third end effector E3 is communicably connected to the robot controller 30 via a cable. Accordingly, the third end effector E3 performs an operation based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB. The third end effector E3 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  The third manipulator M3 includes six joints. Each of the six joints includes an actuator (not shown). That is, the third arm A3 including the third manipulator M3 is a 6-axis vertical articulated arm. Alternatively, the third arm A3 may be a six-axis horizontal articulated arm instead. The third arm A3 performs a motion with six degrees of freedom by a coordinated operation by the support base B3, the third end effector E3, the third manipulator M3, and the actuators of each of the six joints included in the third manipulator M3. Do. The third arm A3 may be configured to operate with a degree of freedom of 5 axes or less, or may be configured to operate with a degree of freedom of 7 axes or more.

  Each of the six actuators (provided at the joints) included in the third manipulator M3 is communicably connected to the robot controller 30 via a cable. Accordingly, the actuator operates the third manipulator M3 based on the control signal acquired from the robot control device 30. Note that wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. Further, some or all of the six actuators included in the third manipulator M3 may be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark). Good.

  The imaging unit C5 is a camera including, for example, a CCD, a CMOS, or the like as an imaging device that converts collected light into an electrical signal. In this example, the imaging unit C5 is installed at a position where at least a part of the movable range of the third arm A3 can be imaged. At this time, the imaging unit C5 is fixed so that the position and orientation of the imaging unit in the robot coordinate system do not move. In this example, the movable range is a range in which the third end effector E3 can move by the operation of the third manipulator M3. In addition, the said movable range may replace with this and the other range according to 3rd arm A3 may be sufficient. The imaging unit C5 captures a two-dimensional image in the range. Note that the imaging unit C5 may be configured to capture a still image in the range, or may be configured to capture a moving image in the range.

  The imaging unit C5 is communicably connected to the robot control device 30 via a cable. Wired communication via a cable is performed according to standards such as Ethernet (registered trademark) and USB, for example. The imaging unit C5 may be configured to be connected to the robot control device 30 by wireless communication performed according to a communication standard such as Wi-Fi (registered trademark).

  In the robot system 2, the robot control device 30 operates the third arm A3 of the robot 20a instead of the first arm A1 of the robot 20 in the embodiment. Further, the robot control device 30 causes the imaging unit C5 to capture an area including the target portion FO of the object O instead of the third imaging unit C3 of the robot 20. That is, the robot control device 30 moves the object O relative to the imaging unit C5 installed (fixed) by operating the third arm A3 so that the position and orientation thereof do not change in the robot coordinate system. The relative positional relationship between the object O and the imaging unit C5 is matched with the above-described imaging positional relationship. Accordingly, the robot control device 30 suppresses erroneous detection of a part other than the target part FO in the captured image as the target part FO based on the captured image captured by the imaging unit C5, and the imaging The target portion FO in the image can be detected with high accuracy.

  Note that the robot control device 30 controls the robot 20a and is held by the third end effector E3 with respect to the target object O (fixed) installed (fixed) so that the position and posture thereof do not change in the robot coordinate system. The imaging unit C5 may be relatively moved so that the relative positional relationship between the object O and the imaging unit C5 matches the imaging positional relationship.

  As described above, the robot control device 30 uses the first movable part (in the example described above, the first end effector E1, the second end effector E2, and the third end effector E3) to execute the target (the example described above). Then, the target portion of the object O) (the target portion FO in the example described above) is changed to the imaging unit (in the example described above, the first imaging unit C1, the second imaging unit C2, and the third imaging unit C3). , The focal position in the depth direction of the imaging unit (in the example described above, in the range DF in the example described above) in the depth direction of the depth of field of the fourth imaging unit C4 and imaging unit C5) Moving within a first depth range (range DF1 in the example described above) that is a range from the position P1) to the farthest position from the imaging unit in the depth direction, Thereby imaging an area including the target portion to the image portion. Thereby, the robot control apparatus 30 can detect the attention part of the target object in the said imaging part accurately based on the captured image imaged by the imaging part.

  In addition, the robot control device 30 is configured to capture the target portion of the target by the first movable unit so that the position and orientation of the imaging unit in the robot coordinate system do not move (in the example described above, imaging is performed). Part C5) is moved into a first depth range which is a range from a focal position in the depth direction of the imaging unit to a position farthest from the imaging unit in the depth direction in the range in the depth direction of the depth of field, The region including the part of interest is imaged. Thereby, the robot control device 30 determines the target portion of the target in the imaging unit based on the captured image captured by the imaging unit fixed so that the position and orientation of the imaging unit in the robot coordinate system do not move. It can be detected with high accuracy.

  Further, the robot control device 30 uses the second movable part (in this example, the first end effector E1 and the second end effector E2 which are not examples of the first movable part, the support base upper part B2). Move relative to the object. Accordingly, the robot control device can accurately detect the target portion of the target in the imaging unit based on the captured image captured by the imaging unit moved relative to the target by the second movable unit. Can do.

  The robot control device 30 is a range from a focal position in the depth direction of the imaging unit to a position farthest from the imaging unit in the depth direction in a range in the depth direction of the depth of field of the imaging unit which is a stereo camera. The image is moved within the first depth range, and the imaging unit is caused to image a region including the target portion. Thereby, the robot control apparatus 30 can detect the attention part of the target object in the said imaging part accurately based on the captured image imaged by the imaging part which is a stereo camera.

  Further, the robot control device 30 detects a target portion of the target object included in the captured image captured by the imaging unit. Thereby, the robot control device 30 can cause the robot (the robot 20 and the robot 20a in this example) to perform the work related to the target portion detected from the captured image with high accuracy.

  In addition, the robot control device 30 performs processing according to the target portion detected from the captured image. Thereby, the robot control apparatus 30 can perform the process according to the attention part detected from the captured image with high precision.

  Further, the robot control device 30 performs processing related to the operation of the robot in accordance with the target portion detected from the captured image. Thereby, the robot control apparatus 30 can perform the process regarding the operation of the robot with high accuracy.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, and the like are possible without departing from the gist of the present invention. May be.

  Further, a program for realizing the function of an arbitrary component in the above-described apparatus (for example, robot control apparatus 30) is recorded on a computer-readable recording medium, and the program is read into a computer system and executed. You may make it do. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD (Compact Disk) -ROM, or a storage device such as a hard disk built in the computer system. . Furthermore, “computer-readable recording medium” means a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1, 2 ... Robot system 20, 20a ... Robot, 30 ... Robot control apparatus 31, 41, 51 ... Processor, 32, 42, 52 ... Memory, 34, 44, 54 ... Communication part, 321, 322 ... Command, 361 ... Image pickup control unit, 363 ... Image acquisition unit, 365 ... Robot control unit, 367 ... Image processing unit, A1 ... First arm, A2 ... Second arm, A3 ... Third arm, B, B3 ... Support base, C1 ... 1st imaging part, C2 ... 2nd imaging part, C3 ... 3rd imaging part, C4 ... 4th imaging part, C5 ... Imaging part, E1 ... 1st end effector, E2 ... 2nd end effector, E3 ... 3rd End effector, M1 ... first manipulator, M2 ... second manipulator, M3 ... third manipulator, O ... object, PC1, PC2, PC3 ... information processing device

Claims (11)

A robot control apparatus for controlling a robot having a first movable part that moves an object,
Positioning the target portion of the object by the first movable unit farthest from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit Moving within a first depth range that is a range up to, causing the imaging unit to image a region including the portion of interest,
Robot control device. The imaging unit is fixed so that the position and orientation of the imaging unit in a robot coordinate system do not move.
The robot control apparatus according to claim 1. Moving the imaging unit relative to the object by a second movable unit;
The robot control apparatus according to claim 1. The imaging unit is a stereo camera.
The robot control apparatus according to any one of claims 1 to 3. Detecting the part of interest included in the captured image captured by the imaging unit;
The robot control apparatus according to any one of claims 1 to 4. Performing processing according to the target portion detected from the captured image;
The robot control apparatus according to claim 5. Depending on the portion of interest detected from the captured image, processing related to the operation of the robot is performed.
The robot control device according to claim 6. A first movable part for moving the object;
Positioning the target portion of the object by the first movable unit farthest from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit Moving within a first depth range that is a range up to, causing the imaging unit to image a region including the portion of interest,
robot. A robot system comprising a robot having a first movable part that moves an object, and a robot control device that controls the robot,
The robot controller is
Positioning the target portion of the object by the first movable unit farthest from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit Moving within a first depth range that is a range up to, causing the imaging unit to image a region including the portion of interest,
Robot system. A robot control method for controlling a robot having a first movable part that moves an object,
Positioning the target portion of the object by the first movable unit farthest from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit Moving within a first depth range that is a range up to, causing the imaging unit to image a region including the portion of interest,
Robot control method. A robot control apparatus for controlling a robot having a first movable part that moves an object,
With a processor,
The processor is
Positioning the target portion of the object by the first movable unit farthest from the imaging unit in the depth direction from the focal position in the depth direction of the imaging unit in the depth direction of the depth of field of the imaging unit It is configured to move within a first depth range that is a range up to and to execute a command to cause the imaging unit to image a region including the target portion.
Robot control device.

Images