TWI780887B - Robot system and picking method - Google Patents

Abstract

A robot system includes: a robot mechanism having a holding portion that holds a workpiece; a detection unit that detects a state of holding of the workpiece by the holding portion; a recognition unit that recognizes the number of workpieces held by the holding portion based on the detection result of the detection unit; and a control unit that, after having operated the robot mechanism to hold a workpiece from a box in which plural workpieces are placed, controls the movement of the robot mechanism such that the number of workpieces that have been held becomes a specified number in a case in which the number of workpieces recognized by the recognition unit is different from a predetermined specified number.

Description

Robot system and picking method

The disclosure relates to a robot system and a picking method.

In Patent Document 1 to Patent Document 3, a technology for identifying pickable workpieces based on photographic images of bulk workpieces and picking up the workpieces is disclosed.

[Prior art literature] [Patent Document]

Patent Document 1: Japanese Patent No. 5767464

Patent Document 2: Japanese Patent Laid-Open No. 2017-42859

Patent Document 3: Japanese Patent No. 5196156

However, in the prior art described above, especially in the case of small workpieces such as screws, metal workpieces, and glossy workpieces, it is difficult to reliably detect the position of the workpiece using an image sensor or a depth sensor. and posture, as a result, it is difficult to reliably perform pickup of a workpiece. In addition, due to the failure to estimate the gap with the surrounding disturbing objects, or the failure to calculate the possibility of gripping, the pickup may also fail. Furthermore, since picking up a workpiece will mostly cause the configuration of its surrounding workpieces to change, each It also takes time to detect the position and posture of the workpiece before the next pick-up operation.

The present disclosure was made in view of the above points, and an object thereof is to provide a robot system and a picking method capable of picking up a workpiece with high accuracy without having to detect the position and orientation of the workpiece every time.

The first disclosed aspect is a robot system, including: a robot mechanism, including a gripping part for gripping a workpiece; a detection part, which detects the gripping state of the workpiece by the gripping part; an identification part, based on the detection the detection result of the part to recognize the number of workpieces held by the gripping part; After the action, if the number of workpieces recognized by the recognition part is different from the specified number specified in advance, the operation of the robot mechanism is controlled so that the number of workpieces held quantity becomes the stated quantity.

In the first aspect, the specified number can be set to be 1.

In the first aspect, it may be configured that the holding part includes: a plurality of fingers; and a driving part that drives at least one finger while holding the workpiece with the plurality of fingers. to change the position of the at least one finger.

In the first aspect, the detecting unit may be configured as a pressure distribution sensor that is provided on the gripping surface of at least one finger and detects a pressure distribution of the gripping surface that is in contact with the workpiece. The identification unit identifies the number of workpieces based on the pressure distribution detected by the pressure distribution sensor.

In the first aspect, it may be configured that the pressure distribution sensor is disposed on the respective gripping surfaces of at least two fingers among the plurality of fingers.

In the first aspect, the recognition unit may include a learned model, and the learned model takes the pressure distribution as an input and determines the number of workpieces held by the gripping unit. Learning takes place as output.

In the first aspect, it may be configured that the learned model includes a neural network.

In the first aspect, it may be configured that the detection unit is an imaging unit that captures a state in which the workpiece is held by a plurality of fingers, and the identification unit is based on the photographed image captured by the imaging unit. Like, identify the number of the workpiece.

In the first aspect, the control unit may be configured to move at least one of the imaging unit and the gripping unit so that the camera gripped by the plurality of fingers can be photographed. The position of the workpiece is photographed.

In the first aspect, it may be configured that the detection unit is a force sensor, and when the workpiece is held by the plurality of fingers, the force sensor detects a force applied to the workpiece. For the force of the plurality of fingers, the recognition unit recognizes the number of the workpieces based on the total weight of the workpieces held by the force Calculate the amount of increase in the vertical direction component of the force detected by the sensory sensor based on the grip.

In the first aspect, the identifying unit may be configured to identify the number of the workpieces based on the total weight of the workpieces and the weight of each of the workpieces.

In the first aspect, it may be configured that, after being recognized by the recognition unit When the number of the workpieces differs from the specified number, the control unit controls the operation of the robot mechanism so as to re-grip the workpieces from the placement place.

In the first aspect, when the number of the workpieces recognized by the recognition unit is less than the predetermined number, the control unit may restart the work from the placement place. The manner in which the workpiece is held controls the operation of the robotic mechanism.

In the first aspect, it may be configured that, when the number of the workpieces recognized by the recognition unit is greater than the specified number, the control unit may The robot mechanism is controlled in such a manner that an external force is applied to at least a part of the workpiece being held so that the workpiece falls.

The second disclosed aspect is a picking method that detects a gripping state of a workpiece by a gripping portion for gripping the workpiece, and recognizes the object held by the gripping portion based on the detection result of the gripping state. For the number of workpieces, after the robot mechanism including the gripper performs an action of gripping the workpieces from a placement place where a plurality of the workpieces are placed, the number of the workpieces recognized and the number of the workpieces are determined. When the predetermined number is different, the operation of the robot mechanism is controlled so that the number of the workpieces to be gripped becomes the predetermined number.

According to the present disclosure, the workpiece can be picked up with high accuracy even if the position and orientation of the workpiece are not detected in advance every time.

10:Robot system

20: Robot Mechanism

22-1, 22-2: drive unit

24-1, 24-2: Pressure distribution sensor

30: Control device

30A:CPU

30B:ROM

30C: RAM

30D: Storage

30E: input part

30E1: keyboard

30E2: mouse

30F: Monitor

30G: CD drive

30H: communication interface

30I: busbar

32: Identification department

34: Control Department

40: Photography Department

50: boxes

AR: Robot Arm

F1, F2: fingers

H: robot hand

S100, S102, S104, S106, S108, S110, S112: steps

W: Workpiece

Figure 1 is a structural diagram of the robot system.

Fig. 2 is a functional block diagram of a control device.

Fig. 3 is a block diagram showing the hardware configuration of the control device.

FIG. 4 is a flowchart showing the flow of the pick-up process.

Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the drawings. In addition, the same reference symbol is attached|subjected to the same or equivalent structural element and part in each drawing. In addition, the dimensional ratios in the drawings are sometimes exaggerated for convenience of description, and may differ from actual ratios.

FIG. 1 is a configuration diagram of a robot system 10 according to the present embodiment. As shown in FIG. 1 , the robot system 10 includes a robot mechanism 20 , a control device 30 , and an imaging unit 40 . In the present embodiment, the robot system 10 functions as a pick-up device that picks up the workpiece W. As shown in FIG.

The robot mechanism 20 includes a robot arm AR, which is a mechanism part to be subjected to operational control when performing a picking operation, and a robot hand H, which is attached to the tip of the robot arm AR. The robot hand H holds the workpiece W from a box 50 which is an example of a placement place where a plurality of workpieces W are placed. The robot arm AR is an example of a gripping unit. In addition, in this embodiment, the case where the workpiece W is a relatively small part such as a screw or the like to the extent that a plurality of workpieces W can be grasped by the robot hand H is described as an example, but the workpiece W is not limited thereto.

The robot hand H includes a plurality of fingers, and is used as an example in this embodiment Two fingers F1 and F2 are included, but the number of fingers is not limited to two. In addition, in this embodiment, the finger F1 and the finger F2 are constituted by a plate-shaped member as an example, but the shapes of the finger F1 and the finger F2 are not limited thereto.

In addition, the robot hand H includes: a drive unit 22-1 that drives the workpiece W while being held by the fingers F1 and F2 to change the position of the finger F1; . In a state where the finger F2 is holding the workpiece W, driving is performed so that the position of the finger F2 is changed. In addition, in this embodiment, the case where the drive part is provided in both the finger F1 and the finger F2 is demonstrated, However, The drive part may be provided in either of the finger F1 and the finger F2.

In addition, a pressure distribution sensor 24 - 1 for detecting the pressure distribution of the gripping surface in contact with the workpiece W is provided on the gripping surface of the finger F1 . Similarly, a pressure distribution sensor 24 - 2 for detecting the pressure distribution of the gripping surface in contact with the workpiece W is provided on the gripping surface of the finger F2 . Here, the pressure distribution sensor 24 - 1 and the pressure distribution sensor 24 - 2 are examples of detection units that detect the gripping state of the workpiece W by the fingers F1 and F2 .

Furthermore, in this embodiment, the case where the pressure distribution sensor is provided on the gripping surfaces of both the finger F1 and the finger F2 has been described, but it is also possible to configure the pressure distribution sensor on any one of the finger F1 and the finger F2. The grip surface of the finger is provided with a pressure distribution sensor.

As an example, the robot mechanism 20 may use a vertical articulated robot with 6 degrees of freedom, a horizontal articulated robot, etc., but the degrees of freedom and types of robots are not limited to these.

The control device 30 controls the robot mechanism 20 . As shown in FIG. 2 , the control device 30 functionally includes an identification unit 32 and a control unit 34 .

Recognition unit 32 recognizes the number of workpieces W held by fingers F1 and F2 based on the detection results of pressure distribution sensors 24 - 1 and 24 - 2 , that is, pressure distributions. In the present embodiment, as an example, the recognition unit 32 uses the pressure distribution detected by the pressure distribution sensor 24-1 and the pressure distribution sensor 24-2 as an input and the number of workpieces W as an output. A learned learning completion model, such as a learning completion model using a neural network, identifies the number of workpieces W held by fingers F1 and F2.

After the control unit 34 causes the robot mechanism 20 to grasp the workpiece W from the box 50, if the number of workpieces recognized by the recognition unit 32 is different from the predetermined number specified in advance, the robot mechanism 20 is controlled. The operation is controlled so that the number of workpieces W to be held becomes a predetermined number. In addition, in this embodiment, the case where the designated number is 1 is demonstrated. That is, when the number of workpieces recognized by the recognition unit 32 is plural, the operation of the robot mechanism 20 is controlled so as to drop the workpiece W until the number of workpieces recognized by the recognition unit 32 becomes one. .

Specifically, the control unit 34 controls at least one of the drive unit 22-1 and the drive unit 22-2 so that the fingers F1, At least one of the fingers F2 moves. That is, at least one of the finger F1 and the finger F2 is moved to change the relative positions of the finger F1 and the finger F2. Thereby, the gripping state of the workpiece W gripped by the fingers F1 and F2 changes, and the workpiece W can be dropped.

The imaging unit 40 is installed at a position where the workpiece W in the box 50 can be photographed from above the box 50 , and outputs a captured image of the workpiece W in the box 50 according to an instruction from the control device 30 to the control device 30 .

Next is a block diagram showing the hardware configuration of the control device 30 .

As shown in Figure 3, the control device 30 has a central processing unit (Central Processing Unit, CPU) 30A, a read only memory (Read Only Memory, ROM) 30B, a random access memory (Random Access Memory, RAM) 30C, storage device (storage) 30D, input unit 30E, monitor (monitor) 30F, optical disk drive device 30G and communication interface 30H. The structures are communicatively connected to each other via a bus bar 30I.

In this embodiment, the pickup program is stored in the memory 30D. The CPU 30A is a central arithmetic processing unit, and executes various programs or controls various structures. That is, the CPU 30A reads the program from the memory 30D, and executes the program using the RAM 30C as a work area. The CPU 30A performs control of each of the above structures and various arithmetic processing according to the programs recorded in the memory 30D.

ROM 30B stores various programs and various data. The RAM 30C temporarily stores programs and data as a work area. The storage 30D includes a hard disk drive (Hard Disk Drive, HDD) or a solid state drive (Solid State Drive, SSD), and stores various programs and various data including an operating system.

The input unit 30E includes pointing devices such as a keyboard 30E1 and a mouse 30E2 , and is used for various inputs. monitor 30F such as liquid The crystal display displays various information such as the gripping state of the workpiece W. The monitor 30F may adopt a touch panel system and also function as the input unit 30E. The optical disk drive device 30G reads data stored in various recording media (Compact Disk Read Only Memory (CD-ROM), Blu-ray Disc, etc.), writes data to the recording medium, and the like.

The communication interface 30H is an interface for communicating with other devices, such as Ethernet (registered trademark), Fiber Distributed Data Interface (Fiber Distributed Data Interface, FDDI) or Wireless Fidelity (Wireless Fidelity, Wi-Fi) (registered trademark) and other standards.

Each functional structure of the control device 30 shown in FIG. 2 is achieved by the following: CPU 30A reads out the pickup program stored in memory 30D, expands and executes it in RAM 30C.

Next, the action of the robot system 10 will be described.

FIG. 4 is a flowchart showing the flow of picking processing performed by the robot system 10 . When the user operates the input unit 30E to instruct execution of the pick-up process, the CPU 30A reads the pick-up program from the memory 30D, expands and executes it in the RAM 30C, thereby executing the pick-up process.

In step S100 , the CPU 30A as the control unit 34 controls the robot mechanism 20 so that the robot hand H grasps the workpiece W in the box 50 . Specifically, for example, instructing the photographing unit 40 to take photographs to obtain photographed images of the workpiece W in the box 50, and confirming the existence of the workpiece W by performing image analysis on the acquired photographed images. place. At this time, there is no need to determine the workpiece to be held For the position and posture of W, it is only necessary to specify the place where the workpiece W exists. Then, the robot arm AR is controlled so that the robot hand H moves to a place where the workpiece W exists, and the driving unit 22-1 and the driving unit 22-2 are controlled so that the workpiece is grasped by the fingers F1 and F2. W. Alternatively, the operation of closing the fingers F1 and F2 at any position in the box 50 may be performed without taking pictures by the imaging unit 40, and as a result, the workpiece W may be grasped with a certain probability. . In particular, when a plurality of workpieces W remain in the box 50, the workpieces W can be grasped with a high probability even without specifying the location where the workpieces W exist in advance.

In step S102 , the CPU 30A as the recognition unit 32 acquires the pressure distributions of the grip surfaces of the fingers F1 and F2 from the pressure distribution sensors 24 - 1 and 24 - 2 , respectively. Then, the number of workpieces W gripped by the fingers F1 and F2 is recognized based on the acquired pressure distribution of the grip surfaces of the fingers F1 and F2 .

In step S104 , the CPU 30A as the control unit 34 determines whether or not the number of workpieces W recognized in step S102 is zero. Then, when the number of recognized workpieces W is not 0, that is, when at least one workpiece W is held, the process proceeds to step S106. On the other hand, when the number of recognized workpieces is 0, the process returns to step S100 to hold the workpiece W again.

In step S106 , the CPU 30A as the control unit 34 determines whether the number of workpieces W recognized in step S102 is a predetermined number, that is, whether the number of recognized workpieces W is one. And when the number of recognized workpiece|work W is 1, it transfers to step S108. On the other hand, when the number of recognized workpieces W is not 1, that is, when there are a plurality of objects, the process proceeds to step S110.

In step S108 , the CPU 30A as the control unit 34 controls the robot mechanism 20 so that the workpiece W held by the robot hand H is moved and placed on a predetermined place.

In step S110 , the CPU 30A as the control unit 34 controls at least one of the driving unit 22 - 1 and the driving unit 22 - 2 to move at least one of the finger F1 and the finger F2 . For example, the finger to be moved, the direction and the amount of movement may be determined in advance, or the finger to be moved, the direction and the amount of movement may be determined based on the pressure distribution on the gripping surfaces of the fingers F1 and F2 acquired in step S102. Thereby, the gripping state of the workpiece W gripped by the fingers F1 and F2 changes, and the workpiece W tends to drop.

In step S112 , the CPU 30A as the control unit 34 determines whether or not all the workpieces W in the box 50 have been picked up. That is, it is determined whether or not the inside of the box 50 is empty. Specifically, for example, image analysis is performed on the captured image captured by the imaging unit 40 to determine whether or not the workpiece W remains in the box 50 . And, when the workpiece W does not remain in the box 50, this routine (routine) is terminated. On the other hand, when the workpiece W remains in the box 50, the process returns to step S100 and the same process as above is repeated until all the workpieces W are picked up.

Thus, in this embodiment, instead of grasping the workpiece W after detecting the position and posture of the workpiece W to be grasped in advance, the workpiece W is first grasped, and then at least one of the fingers F1 and F2 is moved. The operation is performed until the number of workpieces W to be held reaches the specified number. Thereby, even if the position and orientation of the workpiece W are not detected in advance every time, the workpiece W can be picked up with high accuracy.

(Modification 1)

Next, Modification 1 of the present embodiment will be described.

When identifying the number of workpieces W held, for example, the number of workpieces W held may be identified based on the contact areas of the workpieces W on the gripping surfaces of the fingers F1 and F2 . For example, based on the respective pressure distributions detected by the pressure distribution sensor 24-1 and the pressure distribution sensor 24-2, the contact areas of the workpiece W on the gripping surfaces of the fingers F1 and F2 are respectively calculated. In addition, for example, the number of workpieces W held may be identified using tabular data or mathematical expressions representing the correspondence between each contact area and the number of workpieces held. For example, when the workpiece W is a sphere, since the contact area formed by one workpiece W pressing against the elastic grip surface is relatively stable, the number of workpieces W can be identified in the above manner.

(Modification 2)

Next, Modification 2 of this embodiment will be described.

In the case of recognizing the number of workpieces W being held, for example, the state of the workpiece W being held by the fingers F1 and F2 may be photographed by the imaging unit 40 as an example of the detection unit, and based on the imaging unit 40 The captured photographic image is used to identify the number of workpieces W being held. In this case, the control unit 34 can move the robot hand H so that the imaging unit 40 can photograph the workpiece W held by the fingers F1 and F2, that is, the workpiece W is not hidden by the fingers F1 and F2. The location is taken. In addition, it may be configured to further include a moving mechanism of the imaging unit 40 so that the imaging unit 40 moves to the position of the robot hand H. As shown in FIG. In addition, both the robot hand H and the imaging unit 40 may be moved.

(Modification 3)

Next, Modification 3 of the present embodiment will be described.

In the case of identifying the number of workpieces W being held, for example, it may be configured to include a six-axis force sensor as an example of a detection unit that detects whether the work W is held by the fingers F1 or F2. The force applied to the fingers F1 and F2 in the state of holding the workpiece W, so that the recognition unit 32 is based on the vertical direction component of the force detected by the force sensor. The total weight of W is used to identify the number of workpiece W. In this case, the number of workpieces W is identified based on, for example, the total weight of the workpieces W and the weight of each workpiece W. FIG. That is, the number of workpieces W may be calculated by dividing the total weight of the workpieces W calculated from the force detected by the force sensor by the weight of each workpiece W.

(Modification 4)

Next, Modification 4 of this embodiment will be described.

The control unit 34 may also control the operation of the robot mechanism 20 to reload from the box 50 when the number of workpieces W recognized by the recognition unit 32 is different from the specified number, for example, less than the specified number. Hold the workpiece W. That is, the robot mechanism 20 can also be controlled so that the workpiece W held temporarily can be returned to the box 50 by opening the fingers F1 and F2 on the box 50, and then the workpiece in the box 50 can be grasped again. W.

(Modification 5)

Next, Modification 5 of the present embodiment will be described.

The control unit 34 may also control the robot mechanism 20 so that the number of workpieces W recognized by the recognition unit 32 is greater than a specified number. F1, at least a part of the workpiece held by the finger F2 applies an external force to cause the workpiece to fall. For example, the robot mechanism 20 may be controlled so that the workpiece W collides with a rod-shaped fixed jig and the workpiece W falls. In addition, for example, an external force mechanism for applying an external force to the workpiece W by a rod-shaped member may be further provided so that the rod-shaped member collides with the workpiece W to cause the workpiece W to drop by controlling the external force mechanism.

In addition, in this embodiment, the pickup process performed by reading software (program) by CPU may be executed by various processors other than CPU. As the processor in this case, a field-programmable gate array (Field-Programmable Gate Array, FPGA) equal to a programmable logic device (Programmable Logic Device, PLD) whose circuit structure can be changed after manufacture, and an application-specific integrated circuit can be exemplified. Circuit (Application Specific Integrated Circuit, ASIC) and other processors that have a specially designed circuit structure in order to perform specific processing, that is, dedicated electrical circuits. In addition, the picking process may be performed by one of the various processors, or may be performed by a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware configuration of the various processors described above is more specifically an electrical circuit in which circuit elements such as semiconductor elements are combined.

In addition, in each of the above-described embodiments, the mode in which the pickup program is stored (installed) in the memory 30D or the ROM 30B has been described, but the present invention is not limited thereto. Programs can also be recorded on CD-ROM, Digital Versatile Disk Read Only Memory (DVD-ROM), and Universal Serial Bus (Universal Serial Bus, USB) memory and other recording media are provided. Also, the program may be downloaded from an external device via a network.

In addition, all the content of the indication of Japanese Patent Application No. 2020-151538 is incorporated in this specification by reference. In addition, all documents, patent applications, and technical specifications described in this specification are incorporated by reference to the same extent as if they were specifically and individually stated to be incorporated by reference. to this manual.

10:Robot system

20: Robot Mechanism

22-1, 22-2: drive unit

24-1, 24-2: Pressure distribution sensor

30: Control device

40: Photography Department

50: boxes

AR: Robot Arm

F1, F2: fingers

H: robot hand

W: Workpiece

Claims (6)

A robot system, comprising: a robot mechanism, including a gripping part for gripping a workpiece; a detection part, for detecting the gripping state of the workpiece by the gripping part; an identification part, based on the detection result of the detection part, for identifying the the number of workpieces held by the gripping unit; When the number of the workpieces recognized by the recognition unit is different from the predetermined number specified in advance, the operation of the robot mechanism is controlled so that the number of the workpieces held becomes the specified number; The gripping unit includes: a plurality of fingers; and a driving unit that drives at least one finger to change a position of the at least one finger in a state where the workpiece is gripped by the plurality of fingers; The detection unit is a plurality of pressure distribution sensors provided on the respective gripping surfaces of at least two fingers among the plurality of fingers and detecting a pressure distribution of the gripping surfaces in contact with the workpiece; The identification unit calculates a contact area of the workpiece on each gripping surface based on each of the pressure distributions detected by the plurality of pressure distribution sensors, and identifies an individual contact area of the workpiece based on each of the calculated contact areas. number. The robot system as claimed in item 1, wherein, The specified number is one. The robot system according to claim 1, wherein when the number of the workpieces recognized by the recognition unit is different from the specified number, the control unit re-grasps the workpiece from the placement place. The operation of the robot mechanism is controlled by the way of holding the workpiece. The robot system according to claim 3, wherein when the number of the workpieces recognized by the recognition unit is less than the specified number, the control unit re-grasps the workpiece from the placement place. The operation of the robot mechanism is controlled by the way of holding the workpiece. The robot system according to any one of claims 1 to 4, wherein when the number of the workpieces recognized by the recognition part is more than the specified number, the control part responds to the workpieces recognized by the recognition part. The robot mechanism is controlled such that an external force is applied to at least a part of the workpiece held by the gripping portion to cause the workpiece to fall. A picking method, wherein the gripping state of the workpiece by a gripping portion for gripping the workpiece is detected, based on the detection result of the gripping state, the number of workpieces gripped by the gripping portion is recognized, and the After causing the robot mechanism including the gripping part to grip the workpiece from a placement place where a plurality of the workpieces are placed, the number of the recognized workpieces is compared with the specified number specified in advance. Under different circumstances, the operation of the robot mechanism is controlled so that the number of the workpieces held becomes the specified number amount; the gripping portion includes: a plurality of fingers; and a driving portion that drives at least one finger to change the position of the at least one finger in a state where the workpiece is gripped by the plurality of fingers The detection is performed by a plurality of pressure distribution sensors that are provided on respective gripping surfaces of at least two fingers among the plurality of fingers and detect pressure distribution of the gripping surfaces that are in contact with the workpiece The detection performed; the identification is based on each pressure distribution detected by the plurality of pressure distribution sensors, calculating the contact area of the workpiece on each gripping surface, and based on the calculated each contact area, Identify the number of the workpieces.

Images