WO2021124760A1 - ロボット、ロボットシステム及び制御方法 - Google Patents

ロボット、ロボットシステム及び制御方法 Download PDF

Info

Publication number
WO2021124760A1
WO2021124760A1 PCT/JP2020/042678 JP2020042678W WO2021124760A1 WO 2021124760 A1 WO2021124760 A1 WO 2021124760A1 JP 2020042678 W JP2020042678 W JP 2020042678W WO 2021124760 A1 WO2021124760 A1 WO 2021124760A1
Authority
WO
WIPO (PCT)
Prior art keywords
work
arm
camera
axis
swivel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/042678
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
昌寛 小川
入江　俊充
健一 小▲柳▼
智之 堀内
伊藤　雅人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yaskawa Electric Corp
Original Assignee
Yaskawa Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yaskawa Electric Corp filed Critical Yaskawa Electric Corp
Priority to CN202080079344.XA priority Critical patent/CN114728420B/zh
Priority to JP2021565378A priority patent/JP7399981B2/ja
Publication of WO2021124760A1 publication Critical patent/WO2021124760A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices

Definitions

  • This disclosure relates to robots, robot systems and control methods.
  • Patent Document 1 includes a work table for supporting the work, a robot arm for performing work on the work, and a camera installed above the work table, and the position and posture of the work are based on an image acquired from the camera.
  • a robot system that identifies is disclosed.
  • the present disclosure provides a robot that is effective in achieving both effective utilization of camera images for robot control and simplification of system configuration.
  • the robot according to one aspect of the present disclosure is fixed to a base, a swivel portion that swivels with respect to the base, an articulated arm that is connected to the swivel portion and changes the position and posture of a work tool, and a swivel portion.
  • the camera is equipped with a camera, and the range of movement of the tool by the articulated arm and the field of view of the camera overlap.
  • the robot system includes the robot, a command generation unit that generates an operation command based on an image captured by the camera of the work, and a first work by a tool on the work based on the operation command. It includes an arm control unit that controls an articulated arm to execute.
  • a control method includes a base, a swivel portion that swivels with respect to the base, and an articulated arm that is connected to the swivel portion and changes the position and orientation of a working tool.
  • An image of the work placed in the work area around the base of the robot is acquired from a camera provided in the swivel part, and an operation command is generated based on the image, and a tool is used based on the operation command. It includes controlling the articulated arm to perform the first task on the work.
  • a robot that is effective in achieving both effective utilization of camera images for robot control and simplification of system configuration is provided.
  • the robot system 1 shown in FIG. 1 is a system that executes a predetermined work on a work object (hereinafter, referred to as “work”) in a product production line.
  • work a work object
  • Specific examples of the work include, but are not limited to, pick-and-place work in which the work is picked up, transported to a predetermined transfer target position and posture, and dropped off.
  • Other examples of the work include a work of welding a work to another work, a work of fastening the work to another work with bolts or the like, a work of polishing the work, and the like.
  • the robot system 1 includes a robot 10 and a controller 100.
  • the robot 10 is a so-called vertical articulated industrial robot, and includes a base 11, a swivel portion 12, an articulated arm 20, actuators 71, 72, 73, 74, 75, 76, 77, and a tool 30. , And a camera 40.
  • the base 11 is fixed on a robot support 3 located in the vicinity of the work area 5.
  • the base 11 may be fixed on an AGV (Automated Guided Vehicle).
  • the swivel portion 12 is provided above the base portion 11.
  • the swivel portion 12 swivels with respect to the base portion 11. For example, the swivel portion 12 swivels around the vertical axis 51.
  • the articulated arm 20 is connected to the swivel portion 12 and changes the position and posture of the work tool 30.
  • the articulated arm 20 has 6 or more independent degrees of freedom.
  • the articulated arm 20 is a 6-axis serial link type arm, and has a first arm 21, a second arm 22, a third arm 23, and a tool holding portion 24.
  • the first arm 21 is connected to the swivel portion 12 and swings around the axis 52 passing through the connecting portion between the first arm 21 and the swivel portion 12.
  • the axis 52 (first axis) intersects (for example, orthogonally) the turning center axis (the axis 51) of the turning portion 12.
  • the intersection here also includes a twisting relationship like a so-called grade separation.
  • the first arm 21 has an arm base 25 and a swivel arm 26.
  • the arm base 25 is connected to the swivel portion 12 and is along an axis 53 that intersects (for example, is orthogonal to) the axis 52.
  • the swivel arm 26 is connected to the tip of the arm base 25, is along the axis 53, and swivels around the axis 53.
  • the second arm 22 is connected to the tip of the first arm 21, turns around the axis 53 (second axis) along the first arm 21, and intersects (for example, orthogonally) the axis 53 with the axis 54 (third). Swing around (axis).
  • the second arm 22 is connected to the tip of the swivel arm 26 and swings around an axis 54 passing through the connecting portion between the second arm 22 and the swivel arm 26. Further, the second arm 22 swivels around the axis 53 together with the swivel arm 26.
  • the second arm 22 has an arm base 27 and a swivel arm 28.
  • the arm base 27 is connected to the tip of the swivel arm 26 and is along an axis 55 that intersects (for example, is orthogonal to) the axis 54.
  • the swivel arm 28 is connected to the tip of the arm base 27, is along the axis 55, and swivels around the axis 55.
  • the third arm 23 is connected to the tip of the second arm 22, turns around the axis 55 (fourth axis) along the second arm 22, and intersects (for example, orthogonally) the axis 55 (for example, orthogonal) with the axis 56 (fifth). Swing around (axis).
  • the third arm 23 is connected to the tip of the swivel arm 28 and swings around an axis 56 passing through the connecting portion between the third arm 23 and the swivel arm 28. Further, the third arm 23 swivels around the axis 55 together with the swivel arm 28.
  • the third arm 23 is along an axis 57 that intersects (for example, is orthogonal to) the axis 56.
  • the tool holding portion 24 is provided at the tip of the third arm 23, holds the tool 30, and turns around the axis 57 (sixth axis) along the third arm 23.
  • the robot 10 includes a joint 61 that connects the base 11 and the swivel portion 12, a joint 62 that connects the swivel portion 12 and the arm base 25, and a joint 63 that connects the arm base 25 and the swivel arm 26.
  • a joint 64 that connects the swivel arm 26 and the arm base 27, a joint 65 that connects the arm base 27 and the swivel arm 28, a joint 66 that connects the swivel arm 28 and the third arm 23, and a third. It has a joint 67 that connects the arm 23 and the tool holding portion 24.
  • Actuators 71, 72, 73, 74, 75, 76, 77 include, for example, an electric motor and a speed reducer, and drive joints 61, 62, 63, 64, 65, 66, 67, respectively.
  • the actuator 71 is built in the base 11 (for example, housed in the outer shell 81 of the base 11), and the swivel portion 12 is swiveled around the axis 51.
  • the actuator 72 is built in the swivel portion 12 (for example, housed in the outer shell 82 of the swivel portion 12), and swings the arm base 25 around the axis 52.
  • the actuator 73 is built in the arm base 25 (for example, housed in the outer shell 83 of the arm base 25), and swivels the swivel arm 26 around the axis 53.
  • the actuator 74 is built in the swivel arm 26 (for example, housed in the outer shell 84 of the swivel arm 26), and swings the arm base 27 around the axis 54.
  • the actuator 75 is built in the arm base 27 (for example, housed in the outer shell 85 of the arm base 27), and swivels the swivel arm 28 around the axis 55.
  • the actuator 76 is built in the swivel arm 28 (for example, housed in the outer shell 86 of the swivel arm 28), and swings the third arm 23 around the axis 56.
  • the actuator 77 is built in the third arm 23 (for example, housed in the outer shell 87 of the third arm 23), and the tool holding portion 24 is swiveled around the axis 57.
  • the tool 30 acts on the work W in the work performed by the robot 10 on the work W.
  • Specific examples of the tool 30 include a suction nozzle that holds the work W by suction, a hand that holds the work W by grasping it, and the like.
  • Other examples of the tool 30 include welding torches, screw tightening tools (eg electric screwdrivers), polishing tools (eg grinders) and the like.
  • the camera 40 is fixed to the turning portion and photographs the work area 5.
  • the camera 40 has an image pickup device such as a CCD (Chaged-Coupled Devices) or CMOS (Complementary Metal-Oxide-Semiconductor) sensor, and an optical system for forming an image of the field of view 41 on the image pickup device.
  • the camera 40 may be a camera that acquires a two-dimensional image (for example, a color image or a monochrome image) showing the brightness, color tone, etc. of the imaging target portion for each pixel, or indicates the distance to the imaging target portion for each pixel. It may be a ToF (Time-of-Flight) camera that acquires a range image.
  • the camera 40 may be a camera that acquires both a two-dimensional image and a distance image.
  • the swivel coordinate system illustrated in FIG. 1 has an X-axis, a Y-axis, and a Z-axis.
  • the Z-axis is a coordinate axis that goes vertically upward.
  • the Y-axis is a coordinate axis along the axis 52.
  • the X-axis is a coordinate axis perpendicular to the Y-axis and the Z-axis.
  • the origin of the swivel coordinate system is located at the intersection of the axis 52 and the reference plane 43 (see FIG. 2) described later.
  • the positive direction of the X-axis is referred to as the front of the robot 10.
  • the camera coordinate system illustrated in FIG. 1 has a CX axis, a CY axis, and a CZ axis.
  • the CX axis is a coordinate axis along the optical axis 42 at the center of the field of view 41, and the direction away from the camera 40 is the positive direction.
  • the CZ axis is a coordinate axis whose positive direction is above the image imaged on the image sensor.
  • the CY axis is a coordinate axis perpendicular to the CX axis and the CZ axis.
  • the camera 40 may be built in the swivel portion 12 (for example, housed in the outer shell 82).
  • the camera 40 is arranged in front of the actuator 72 in the outer shell 82.
  • a window 82a for imaging is formed in the front portion of the outer shell 82, and the camera 40 captures an image obliquely downward in front of the swivel portion 12 through the window 82a.
  • the arrangement of the camera 40 in the turning portion 12 is set so that the movable range 31 of the tool 30 by the articulated arm 20 and the field of view 41 of the camera 40 overlap.
  • the arrangement of the camera 40 is set so that the movable range 31 and the field of view 41 overlap in the work area 5.
  • the camera 40 is fixed to the swivel portion 12 so that the optical axis 42 is along the reference surface 43 (see FIG. 2) that intersects (for example, is orthogonal to) the axis 52.
  • the camera 40 may be fixed to the swivel portion 12 so that the optical axis 42 faces diagonally downward on the reference surface 43.
  • the camera 40 is fixed to the swivel portion 12 so that the CX axis (optical axis 42) faces diagonally downward.
  • the CX axis is perpendicular to the Y axis, and the positive direction of the CX axis is toward the positive direction of the X axis and the negative direction of the Z axis.
  • the angle formed by the CX axis and the X axis around the Y axis is, for example, 30 to 60 degrees.
  • FIG. 2 is a front view of the robot 10 as viewed from the positive direction of the X-axis.
  • the movable range A1 of the first arm 21 may be separated from the reference surface 43 of the camera 40 in the direction along the Y axis (axis line 52).
  • the movable range A1 is offset in the negative direction of the Y axis with respect to the reference surface 43.
  • the camera 40 is located within the movable range of the first arm 21, so that the movable angle of the first arm 21 is sufficiently set. In order to secure it, it is necessary to arrange the axis 52 sufficiently above the camera 40. Therefore, the articulated arm 20 becomes large. On the other hand, if the arrangement height of the camera 40 is lowered in order to suppress the increase in size of the articulated arm 20, the imaging range by the camera 40 becomes narrow. As described above, it is difficult to achieve both space saving of the articulated arm 20 and widening of the imaging range of the camera 40.
  • the arrangement height of the camera 40 is increased without affecting the movable range of the first arm 21, and the camera 40 is imaged. It is possible to widen the range.
  • the camera 40 is located below the axis 52, but the camera 40 can also be located above the axis 52.
  • the movable range A2 of the second arm 22 may be separated from the first arm 21 in the direction along the axis 54. Further, the movable range A3 of the third arm 23 may be separated from the second arm 22 in the direction along the axis 56.
  • the direction in which the movable range A2 of the second arm 22 separates from the first arm 21 along the axis 54 is referred to as the “offset direction of the second arm 22”.
  • the direction in which the movable range A3 of the third arm 23 separates from the second arm 22 along the axis 56 is referred to as the “offset direction of the third arm 23”.
  • FIG. 2 shows a state in which the offset direction of the second arm 22 and the offset direction of the third arm 23 are both oriented in the positive direction of the Y axis.
  • the movable range A2 of the second arm 22 may overlap with the reference surface 43.
  • the movable range A3 of the third arm 23 may be separated from the reference surface 43.
  • the robot system 1 may further include a support portion 2 that supports the work W in the work area 5 around the base portion 11.
  • the upper surface 2a of the support portion 2 may be located below the swivel portion 12, or may be located below the base portion 11.
  • the controller 100 controls the robot 10. For example, the controller 100 acquires an image of the work W arranged in the work area 5 from the camera 40, generates an operation command based on the image, and works the first work by the tool 30 based on the operation command. It is configured to control the articulated arm 20 to perform for W and to perform.
  • the controller 100 has an image processing unit 112, a command generation unit 113, an arm control unit 114, and a turning control unit 111 as a functional configuration (hereinafter, referred to as “functional block”).
  • the image processing unit 112 calculates the position and orientation of the work W in the three-dimensional space based on the captured image of the work W (for example, the work W arranged on the support portion 2) by the camera 40. For example, the image processing unit 112 calculates the position and orientation of the work W in the camera coordinate system based on the shape and size of the work W in the captured image and the three-dimensional shape and size of the known work W, and the calculation result.
  • the position and orientation of the work W in the turning coordinate system are calculated by performing coordinate transformation on.
  • the robot 10 may be provided with two cameras 40 for stereoscopic viewing.
  • the image processing unit 112 may calculate the position and orientation of the work W in the camera coordinate system based on the images captured by the work W by the two cameras 40.
  • the command generation unit 113 generates an operation command based on an image captured by the camera 40 of the work W (for example, the work W arranged on the support unit 2).
  • the operation command includes, for example, a plurality of commands in a time series.
  • Each instruction includes at least the target position and orientation of the tool 30.
  • the command generation unit 113 generates an operation command based on the position and orientation of the work W calculated by the image processing unit 112.
  • the command generation unit 113 uses the tool 30 for holding the work W based on the position and orientation of the work W calculated by the image processing unit 112.
  • the target position and target posture (hereinafter referred to as “holding position and posture”) are calculated, and an operation command is calculated so as to displace the tool 30 from the current position and posture to the holding position and posture. Further, the command generation unit 113 calculates an operation command so as to displace the tool 30 holding the work W at the holding position and posture to a predetermined lifting position and posture.
  • the arm control unit 114 controls the articulated arm 20 so as to execute the first work by the tool 30 on the work W based on the operation command. For example, the arm control unit 114 calculates the motion angles of the joints 62, 63, 64, 65, 66, 67 by inverse kinematics calculation so as to move the tool 30 according to the motion command, and the joint 62, according to the calculated motion angles.
  • the actuators 72, 73, 74, 75, 76, 77 are controlled so as to operate 63, 64, 65, 66, 67.
  • the arm control unit 114 operates the tool 30 in conjunction with the displacement of the tool 30 by the articulated arm 20. For example, when the first work is the work of picking up the work W, the arm control unit 114 causes the tool 30 to hold the work W when the tool 30 is arranged at the holding position and posture.
  • the rotation control unit 111 rotates the rotation unit 12 so that the camera 40 faces the work W before the command generation unit 113 generates an operation command, and then rotates the first operation until the articulated arm 20 completes the first operation.
  • the unit 12 is stopped.
  • the controller 100 may further have a work monitoring unit 115.
  • the work monitoring unit 115 detects the irregular state in the first work based on the image captured by the camera 40 of the work W during the period when the articulated arm 20 is executing the first work.
  • specific examples of the irregular state include the displacement of the work W while the tool 30 is being moved to the position where the work W is held, and the work W by the tool 30. Poor retention of
  • the command generation unit 113 may modify the operation command based on the image captured by the camera 40 of the work W. For example, when the work monitoring unit 115 detects the displacement of the work W while the tool 30 is being moved to the position where the work W is held, the command generation unit 113 will perform the displacement after the displacement calculated by the image processing unit 112. The operation command is modified based on the position and orientation of the work W.
  • the arm control unit 114 releases the work W from being held by the tool 30, and then the position and orientation of the work W calculated by the image processing unit 112.
  • the command generation unit 113 corrects the operation command based on the above.
  • the arm control unit 114 causes the articulated arm 20 to continue the first operation based on the corrected operation command.
  • the command generation unit 113 performs a second operation based on an image captured by the camera 40 of another work W (for example, another work W on the support portion 2) during the period in which the articulated arm 20 is executing the first operation. Further commands may be generated.
  • the image processing unit 112 determines the position and orientation of the second work based on the image captured by the camera 40 of the other work W (second work) during the period when the articulated arm 20 is executing the first work. Is calculated.
  • the command generation unit 113 calculates the second operation command based on the calculation result of the position and orientation of the second work by the image processing unit 112.
  • the arm control unit 114 controls the articulated arm 20 so that after the articulated arm 20 executes the first work, the articulated arm 20 executes the second work by the tool 30 on the second work based on the second operation command. ..
  • the swivel control unit 111 swivels the swivel unit 12 with the articulated arm 20 picking up the work W by the first work, and the arm control unit 114 receives the work W.
  • the articulated arm 20 may be controlled so that the work W being picked up is arranged in the field of view of the camera 40 during at least a part of the period in which the rotation control unit 111 is rotating the rotation unit 12.
  • the controller 100 may further have a holding state monitoring unit 116.
  • the holding state monitoring unit 116 inspects the holding state of the work W by the tool 30 based on the image captured by the camera 40 of the work W being picked up during the period in which the turning control unit 111 is turning the turning unit 12.
  • each functional block described above is a component of the controller 100, the process executed by each functional block corresponds to the process executed by the controller 100.
  • FIG. 3 is a block diagram illustrating the hardware configuration of the controller 100.
  • the controller 100 has a circuit 190.
  • the circuit 190 includes one or more processors 191 and a memory 192, a storage 193, an image processing circuit 194, and a driver circuit 195.
  • the storage 193 has a computer-readable storage medium, such as a non-volatile semiconductor memory.
  • the storage 193 acquires an image of the work W arranged in the work area 5 from the camera 40, generates an operation command based on the image, and performs the first work by the tool 30 based on the operation command. It stores the control of the articulated arm 20 to be executed by the controller 100 and the program to be executed by the controller 100.
  • the memory 192 temporarily stores the program loaded from the storage medium of the storage 193 and the calculation result by the processor 191.
  • the processor 191 constitutes each functional block of the controller 100 by executing the above program in cooperation with the memory 192.
  • the image processing circuit 194 executes image processing in accordance with the request from the processor 191. Specific examples of the image processing include recognition of the shape and size of the work W in the captured image acquired from the camera 40.
  • the driver circuit 195 outputs drive power to the actuators 71, 72, 73, 74, 75, 76, 77 in accordance with a command from the processor 191.
  • circuit 190 is not necessarily limited to the one that configures each function by a program.
  • the circuit 190 may have at least a part of its functions configured by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) that integrates the logic circuit.
  • ASIC Application Specific Integrated Circuit
  • This procedure includes a base 11, a swivel portion 12 that swivels with respect to the base portion 11, and an articulated arm 20 that is connected to the swivel portion 12 and changes the position and orientation of the work tool 30.
  • An image of the work W arranged in the work area 5 around the base 11 is acquired from the camera 40 provided in the swivel portion 12, and an operation command is generated based on the image, and based on the operation command.
  • the articulated arm 20 is controlled so as to perform the first work by the tool 30 on the work W.
  • This procedure further generates a second operation command based on an image captured by the camera 40 of another work W (second work) while the articulated arm 20 is executing the first work on the work W.
  • the articulated arm 20 is controlled so that the second work by the tool 30 is executed for the second work based on the second operation command.
  • the controller 100 first executes steps S01, S02, S03, S04, S05, S06, S07, S08, S09, and S11.
  • step S01 the swivel control unit 111 swivels the swivel unit 12 so that the camera 40 faces the plurality of work Ws on the support unit 2, and stops the swivel unit 12 with the camera 40 facing the plurality of work Ws. Let me. After that, the turning control unit 111 keeps the turning unit 12 in the stopped state until the articulated arm 20 completes the pick-and-place work on the plurality of work Ws.
  • step S02 the image processing unit 112 acquires captured images of the plurality of work Ws from the camera 40.
  • step S03 the image processing unit 112 calculates the position and orientation of any work W (hereinafter referred to as “target work”) in the camera coordinate system based on the captured image acquired from the camera 40.
  • step S04 the image processing unit 112 performs coordinate transformation on the position and orientation in the camera coordinate system to calculate the position and orientation of the target work in the turning coordinate system.
  • step S05 the command generation unit 113 calculates an operation command for holding the target work based on the captured image of the target work (hereinafter, referred to as “holding command”). For example, the command generation unit 113 calculates the target position and posture (holding position and posture) of the tool 30 for holding the target work based on the position and posture of the target work calculated by the image processing unit 112.
  • step S06 the arm control unit 114 controls the articulated arm 20 so as to arrange the tool 30 at the holding position and posture based on the holding command.
  • step S07 the image processing unit 112 acquires the captured image of the target work from the camera 40.
  • step S08 the image processing unit 112 calculates the position and orientation of the target work in the camera coordinate system based on the captured image acquired from the camera 40.
  • step S09 the image processing unit 112 performs coordinate transformation on the position and orientation in the camera coordinate system to calculate the position and orientation of the target work in the turning coordinate system.
  • step S11 the work monitoring unit 115 confirms whether the position and posture of the target work are within the normal range. For example, in the work monitoring unit 115, the difference between the position and orientation of the target work calculated by the image processing unit 112 in step S04 and the position and orientation of the target work calculated by the image processing unit 112 in step S09 is within the permissible range. Check if.
  • step S11 If it is determined in step S11 that the position and orientation of the target work is not in the normal range, the controller 100 returns the process to step S03. As a result, the command generation unit 113 corrects the holding command based on the captured image of the target work.
  • the arm control unit 114 controls the articulated arm 20 so as to arrange the tool 30 at the corrected holding position and posture based on the corrected holding command.
  • step S11 When it is determined in step S11 that the position and orientation of the target work are within the normal range, the controller 100 executes steps S12, S13, S14, S15, S16, S17, and S18.
  • step S12 the arm control unit 114 causes the tool 30 to hold the work W.
  • step S13 the command generation unit 113 calculates an operation command (hereinafter referred to as “lift command”) so as to displace the tool 30 holding the target work to a predetermined lifting position and posture.
  • step S14 the arm control unit 114 controls the articulated arm 20 so as to arrange the tool 30 at the lifting position and posture based on the lifting command.
  • step S15 the image processing unit 112 acquires captured images of the plurality of work Ws from the camera 40.
  • step S16 the image processing unit 112 calculates the position and orientation of the target work in the camera coordinate system based on the captured image acquired from the camera 40.
  • step S17 the image processing unit 112 performs coordinate transformation on the position and orientation in the camera coordinate system to calculate the position and orientation of the target work in the turning coordinate system.
  • step S18 the work monitoring unit 115 confirms whether the position and posture of the target work are within the normal range. For example, the work monitoring unit 115 confirms whether the relative position and posture of the target work with respect to the tool 30 are within the normal range.
  • step S19 the controller 100 executes step S19.
  • step S19 the arm control unit 114 releases the holding of the work W by the tool 30.
  • the arm control unit 114 may release the holding of the work W by the tool 30 after controlling the articulated arm 20 so as to return the tool 30 from the lifting position and the posture to the holding position and the posture.
  • the controller 100 returns the process to step S03.
  • the command generation unit 113 corrects the holding command based on the captured image of the target work.
  • the arm control unit 114 controls the articulated arm 20 so as to arrange the tool 30 at the corrected holding position and posture based on the corrected holding command.
  • step S21 the command generation unit 113 commands an operation to displace the tool 30 to the target position and posture (hereinafter, referred to as “release target position and posture”) for arranging the target work at the transport target position and posture. (Hereinafter referred to as "transport command”) is calculated.
  • step S22 the arm control unit 114 controls the articulated arm 20 so as to start the displacement of the tool 30 toward the release target position and posture based on the transport command.
  • step S23 the command generation unit 113 confirms whether the undelivered work W remains among the plurality of work W.
  • step S24 the image processing unit 112 acquires a captured image of the unconveyed work W among the plurality of work Ws from the camera 40.
  • step S25 the image processing unit 112 selects the next target work (second work) from the undelivered work W based on the captured image acquired from the camera 40, and positions the next target work in the camera coordinate system. And calculate the posture.
  • step S26 the image processing unit 112 performs coordinate transformation on the position and orientation in the camera coordinate system to calculate the position and orientation of the next target work in the turning coordinate system.
  • step S27 the command generation unit 113 calculates the holding command (second operation command) of the next target work based on the captured image of the next target work. For example, the command generation unit 113 determines the target position and posture (holding position and posture) of the tool 30 for holding the next target work based on the position and posture of the next target work calculated by the image processing unit 112. calculate.
  • step S28 the arm control unit 114 waits for the tool 30 to reach the release target position and posture.
  • step S29 the arm control unit 114 releases the holding of the target work by the tool 30.
  • the controller 100 returns the process to step S06.
  • the pick-and-place work for the next target work is started based on the holding command (second operation command) calculated in step S27.
  • step S31 the arm control unit 114 waits for the tool 30 to reach the release target position and posture.
  • step S32 the arm control unit 114 releases the holding of the target work by the tool 30. This completes the pick-and-place work for the plurality of work Ws.
  • the robot 10 is made to perform the pick-up work for the work W on the support portion 2, then the swivel portion 12 is swiveled, and then the work W is transported to a predetermined transport target position and posture and dropped off.
  • the procedure to be executed is illustrated. This procedure involves turning the swivel portion 12 with the articulated arm 20 picking up the work W in the first operation, and picking up at least a part of the period during which the swivel control unit 111 swivels the swivel portion 12.
  • the articulated arm 20 is controlled so that the work W inside is arranged in the field of view of the camera 40, and the camera 40 of the work W being picked up during the period when the turning control unit 111 is turning the turning unit 12.
  • the procedure for causing the robot 10 to perform the pick-up operation for the work W is the same as the above-mentioned steps S01 to S19, and thus the description thereof will be omitted.
  • step S18 When it is determined in step S18 that the position and orientation of the target work are within the normal range, the controller 100 executes steps S41, S42, S43, S44, S45, S46, and S47 as shown in FIG.
  • step S41 the turning control unit 111 starts turning the turning unit 12 so that the camera 40 faces the transfer target position of the target work.
  • step S42 the arm control unit 114 controls the articulated arm 20 so that the target work being picked up is arranged in the field of view 41 of the camera 40.
  • step S43 the image processing unit 112 acquires captured images of the plurality of work Ws from the camera 40.
  • step S44 the image processing unit 112 calculates the position and orientation of the target work in the camera coordinate system based on the captured image acquired from the camera 40.
  • step S45 the image processing unit 112 performs coordinate transformation on the position and orientation in the camera coordinate system to calculate the position and orientation of the target work in the turning coordinate system.
  • step S46 the holding state monitoring unit 116 inspects the holding state of the work W by the tool 30 based on the position and posture of the target work calculated in step S45. For example, the holding state monitoring unit 116 calculates the relative position and posture of the work W with respect to the tool 30. If the work W at the time of completion of the pickup is located in the field of view 41 of the camera 40 and the captured image for inspection of the holding state of the work W by the tool 30 can be acquired, the step S42 can be omitted.
  • step S47 the turning control unit 111 confirms whether the turning unit 12 has turned until the camera 40 faces the transport target position of the target work. If it is determined in step S47 that the swivel portion 12 has not swiveled until the camera 40 faces the transport target position of the target work, the controller 100 returns the process to step S43. After that, the inspection of the holding state of the work W by the tool 30 is repeated until the camera 40 faces the transfer target position.
  • step S47 When it is determined in step S47 that the turning unit 12 has turned until the camera 40 faces the transfer target position of the target work, the controller 100 executes steps S48, S51, S52, S53, S54, S55, S56, and S57.
  • step S48 the turning control unit 111 stops the turning of the turning unit 12. After that, the turning control unit 111 keeps the turning unit 12 in the stopped state until the articulated arm 20 completes the work of transporting the target work to the transport target position and posture and dropping it off.
  • step S51 the arm control unit 114 controls the articulated arm 20 so that the target work being picked up is retracted from the field of view 41 of the camera 40.
  • step S52 the image processing unit 112 acquires the captured image of the transport target position from the camera 40. Even if the work W is in the field of view 41 of the camera 40, step S51 can be omitted as long as the captured image of the transport target position can be acquired.
  • step S53 the image processing unit 112 calculates the transport target position and the posture in the camera coordinate system based on the captured image acquired from the camera 40.
  • step S54 the image processing unit 112 performs coordinate transformation on the transport target position and posture in the camera coordinate system to calculate the transport target position and posture in the turning coordinate system.
  • step S55 the transport command is calculated so as to displace the tool 30 to the release target position and posture.
  • step S56 the arm control unit 114 controls the articulated arm 20 so as to displace the tool 30 to the release target position and posture based on the transport command.
  • step S57 the arm control unit 114 releases the holding of the work W by the tool 30. This completes the pick-and-place work of the target work.
  • the robot 10 is connected to the base 11, the swivel portion 12 that swivels with respect to the base 11, and the swivel portion 12, and the articulated arm 20 that changes the position and posture of the work tool 30.
  • the camera 40 fixed to the swivel portion 12 is provided, and the movable range 31 of the tool 30 by the articulated arm 20 and the field of view 41 of the camera 40 overlap.
  • a camera 40 is provided in the turning portion 12.
  • the camera 40 and the robot 10 are integrated, so that the system configuration can be simplified.
  • the movable range of the tool 30 by the articulated arm 20 and the field of view range of the camera 40 overlap.
  • an image including both the tool 30 and the work object (work W) by the tool 30 can be acquired from the camera 40. Therefore, it is easy to effectively utilize the image of the camera 40 for controlling the robot 10. Therefore, the robot 10 is effective in both effectively utilizing the camera image for robot control and simplifying the system configuration.
  • the articulated arm 20 may have 6 or more degrees of freedom. In this case, the position and posture of the tool 30 can be freely adjusted by the articulated arm 20 while acquiring a blur-free image from the camera 40.
  • the articulated arm 20 is connected to the swivel portion 12, and is connected to a first arm 21 that swings around an axis 52 that intersects the swivel center axis of the swivel portion 12, and a tip portion of the first arm 21, and is connected to the first arm.
  • a second arm 22 that swivels around the axis 53 along the axis 21 and swings around the axis 54 that intersects the axis 53, and a second arm 22 that is connected to the tip of the second arm 22 and around the axis 55 along the second arm 22.
  • the articulated arm 20 can be provided with 6 degrees of freedom with a simple configuration.
  • the first arm 21 and the optical axis 42 at the center of the field of view 41 are along the reference surface 43 intersecting the axis 52, and the movable range A1 of the first arm 21 is separated from the reference surface 43 in the direction along the axis 52. You may be. In this case, by removing the base end portion of the first arm 21 from the center of the visual field 41, it is difficult to form a blind spot by the articulated arm 20.
  • the robot system 1 executes the first work by the tool 30 on the work W based on the robot 10, the command generation unit 113 that generates an operation command based on the image captured by the camera 40 of the work W, and the operation command.
  • An arm control unit 114 that controls the articulated arm 20 is provided. In this case, the image of the camera 40 can be effectively used for controlling the robot 10.
  • the robot system 1 further includes a work monitoring unit 115 that detects an irregular state in the first work based on an image captured by the camera 40 of the work W during the period in which the articulated arm 20 is executing the first work. You may. In this case, the image of the camera 40 can be more effectively used for controlling the robot 10.
  • the command generation unit 113 corrects the operation command based on the image captured by the camera 40 of the work W, and the arm control unit 114 corrects the operation command based on the corrected operation command.
  • the first work may be continued on the articulated arm 20. In this case, the image of the camera 40 can be more effectively used for controlling the robot 10.
  • the command generation unit 113 further generates a second operation command based on an image captured by the camera 40 of another work W (second work) during the period in which the articulated arm 20 is executing the first work, and the arm
  • the control unit 114 may control the articulated arm 20 so as to execute the second work by the tool 30 on the second work based on the second operation command.
  • the work time for the first work and the second work can be shortened by acquiring the image of the second work during the execution of the first work.
  • the robot system 1 rotates the swivel unit 12 so that the camera 40 faces the work W before the command generation unit 113 generates an operation command, and then the swivel unit until the first work is completed by the articulated arm 20.
  • a turning control unit 111 for stopping the 12 may be further provided. In this case, a blur-free image can be acquired from the camera 40 even during the first operation. Therefore, the image of the camera 40 can be more effectively used for controlling the robot 10.
  • the first work is a work of picking up the work W
  • the swivel control unit 111 swivels the swivel unit 12 with the articulated arm 20 picking up the work W by the first work, and the arm control unit 114 swivels.
  • the articulated arm 20 may be controlled so that the work W being picked up is arranged in the field of view 41 of the camera 40 during at least a part of the period in which the control unit 111 is rotating the turning unit 12.
  • the image of the camera 40 is more effectively used for controlling the robot 10 by utilizing the fact that the rotation of the turning unit 12 does not affect the relative positional relationship between the work W during pickup and the camera 40. be able to.
  • the robot system 1 inspects the holding state of the work W by the tool 30 based on the image captured by the camera 40 of the work W being picked up during the period in which the turning control unit 111 is turning the turning unit 12.
  • a portion 116 may be further provided. In this case, the image acquired from the camera 40 during turning can be further effectively utilized.
  • the robot system 1 further includes an image processing unit 112 that calculates the position and orientation of the work W in the three-dimensional space based on the image captured by the work W by the camera 40, and the command generation unit 113 includes the image processing unit 112.
  • An operation command may be generated based on the calculated position and orientation of the work W.
  • the image from the camera 40 can be more effectively used for controlling the robot 10.
  • the swivel portion 12 may be provided on the base portion 11, and the camera 40 may be provided on the swivel portion 12 so that the optical axis 42 at the center of the field of view 41 faces diagonally downward. In this case, since the camera 40 looks at the work W from diagonally above, it is easy to acquire an image including both the tool 30 and the work W.
  • the robot system 1 further includes a support portion 2 that supports the work W around the base portion 11, and the upper surface 2a of the support portion 2 may be located below the swivel portion 12. In this case, it is easier to acquire an image including both the tool 30 and the work W.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)
PCT/JP2020/042678 2019-12-17 2020-11-16 ロボット、ロボットシステム及び制御方法 Ceased WO2021124760A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080079344.XA CN114728420B (zh) 2019-12-17 2020-11-16 机器人、机器人系统以及控制方法
JP2021565378A JP7399981B2 (ja) 2019-12-17 2020-11-16 ロボット、ロボットシステム及び制御方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019227488 2019-12-17
JP2019-227488 2019-12-17

Publications (1)

Publication Number Publication Date
WO2021124760A1 true WO2021124760A1 (ja) 2021-06-24

Family

ID=76477231

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/042678 Ceased WO2021124760A1 (ja) 2019-12-17 2020-11-16 ロボット、ロボットシステム及び制御方法

Country Status (3)

Country Link
JP (1) JP7399981B2 (https=)
CN (1) CN114728420B (https=)
WO (1) WO2021124760A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024029245A1 (ja) * 2022-08-02 2024-02-08 川崎重工業株式会社 ロボットシステム、ロボットおよびロボットシステムの制御方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06218684A (ja) * 1993-01-22 1994-08-09 Mitsui Eng & Shipbuild Co Ltd 操作型マニピュレータの教示装置および操作型マニピュレータによる自動作業方法
JPH09272081A (ja) * 1996-04-05 1997-10-21 Nippon Steel Corp 遠隔操作入力装置及び遠隔操作入力方法
JP2012228761A (ja) * 2011-04-27 2012-11-22 Yaskawa Electric Corp ロボットシステム及び被作業物の製造方法
JP2013078825A (ja) * 2011-10-04 2013-05-02 Yaskawa Electric Corp ロボット装置、ロボットシステムおよび被加工物の製造方法
US9688489B1 (en) * 2015-03-30 2017-06-27 X Development Llc Modular dock for facilities integration
JP2018171688A (ja) * 2017-03-31 2018-11-08 平田機工株式会社 移載方法および移載システム
JP2018187749A (ja) * 2017-05-11 2018-11-29 セイコーエプソン株式会社 ロボット
JP2019126866A (ja) * 2018-01-23 2019-08-01 トヨタ自動車株式会社 動作軌道生成装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104842362B (zh) * 2015-06-18 2017-04-05 厦门理工学院 一种机器人抓取物料包的方法和机器人抓取装置
CN108214486A (zh) * 2016-12-22 2018-06-29 精工爱普生株式会社 控制装置、机器人及机器人系统
CN206925844U (zh) * 2017-03-29 2018-01-26 天津农学院 机器人装置
JP7091777B2 (ja) * 2018-03-30 2022-06-28 株式会社安川電機 ロボットシステム及び制御方法
CN110170978A (zh) * 2019-03-08 2019-08-27 华东师范大学 一种全向机械臂危险勘测机器人

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06218684A (ja) * 1993-01-22 1994-08-09 Mitsui Eng & Shipbuild Co Ltd 操作型マニピュレータの教示装置および操作型マニピュレータによる自動作業方法
JPH09272081A (ja) * 1996-04-05 1997-10-21 Nippon Steel Corp 遠隔操作入力装置及び遠隔操作入力方法
JP2012228761A (ja) * 2011-04-27 2012-11-22 Yaskawa Electric Corp ロボットシステム及び被作業物の製造方法
JP2013078825A (ja) * 2011-10-04 2013-05-02 Yaskawa Electric Corp ロボット装置、ロボットシステムおよび被加工物の製造方法
US9688489B1 (en) * 2015-03-30 2017-06-27 X Development Llc Modular dock for facilities integration
JP2018171688A (ja) * 2017-03-31 2018-11-08 平田機工株式会社 移載方法および移載システム
JP2018187749A (ja) * 2017-05-11 2018-11-29 セイコーエプソン株式会社 ロボット
JP2019126866A (ja) * 2018-01-23 2019-08-01 トヨタ自動車株式会社 動作軌道生成装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024029245A1 (ja) * 2022-08-02 2024-02-08 川崎重工業株式会社 ロボットシステム、ロボットおよびロボットシステムの制御方法
EP4545256A4 (en) * 2022-08-02 2025-10-22 Kawasaki Heavy Ind Ltd ROBOT SYSTEM, ROBOT AND ROBOT SYSTEM CONTROL METHOD

Also Published As

Publication number Publication date
JPWO2021124760A1 (https=) 2021-06-24
CN114728420B (zh) 2024-12-20
CN114728420A (zh) 2022-07-08
JP7399981B2 (ja) 2023-12-18

Similar Documents

Publication Publication Date Title
US7657346B2 (en) Object picking system
JP6823008B2 (ja) バラ積みされたワークを取り出すロボットシステムおよびロボットシステムの制御方法
JP6228120B2 (ja) 多関節型ロボットを備えた作業機および電気部品装着機
JP5849403B2 (ja) ロボットコントローラー、ロボット、及び、ロボットシステム
US11466974B2 (en) Image capturing apparatus and machine tool
US20140277694A1 (en) Robot system and method for producing to-be-processed material
US20130085604A1 (en) Robot apparatus, robot system, and method for producing a to-be-processed material
JP7057841B2 (ja) ロボット制御システム及びロボット制御方法
US10328582B2 (en) Process system including robot that transfers workpiece to process machine
JP2013086184A (ja) ワーク取出システム、ロボット装置および被加工物の製造方法
JP2022126768A (ja) ロボット搭載移動装置
US20230390924A1 (en) Setting method using teaching operation
US12459131B2 (en) Picking system
US20190366494A1 (en) Wheel alignment adjustment system
JP7399981B2 (ja) ロボット、ロボットシステム及び制御方法
JP6898374B2 (ja) ロボット装置の動作を調整する動作調整装置およびロボット装置の動作を調整する動作調整方法
JP7358747B2 (ja) ロボットシステム
JP2021146445A (ja) 校正方法
JP7105223B2 (ja) ロボットシステム
JP6915085B2 (ja) 作業機および把持位置探索方法
US20220258353A1 (en) Calibration Method
JPS62226308A (ja) 視覚センサを有するロボツトの制御方式
JP7093881B1 (ja) システム、及び自動搬送車
US20260027726A1 (en) Robot system, robot, and method for controlling robot system
JP3577881B2 (ja) 移載方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20902809

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021565378

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20902809

Country of ref document: EP

Kind code of ref document: A1