JP2016185573A - Robot system having function to correct target unloading passage - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot system capable of unloading objects one by one without interference with the objects surrounding each target object.SOLUTION: A robot system 10 comprises: a target-object selection unit 21 selecting a target object 31; a proximity state determination unit 22 determining whether another object 32 is disposed proximate to the target object 31; an avoidance vector determination unit 23 determining an avoidance vector to avoid interference with the object 32; an unloading passage correction unit 24 creating a corrected passage by correcting an unloading passage on the basis of the avoidance vector V; and an unloading operation execution unit 25 which, when it is determined that there is no object 32 disposed proximate to the target object 31, unloads the target object 31 in accordance with the unloading passage, and which, when it is determined that the object 32 is disposed proximate to the target object 31, unloads the target object 31 in accordance with the corrected passage.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a robot system configured to take out objects one by one.

  A robot system configured to detect information representing the position of an object with a visual sensor and control the robot based on the detected information to take out the object is known. Moreover, when taking out one object from a plurality of objects one by one, it is known to determine the order in which the objects are taken out in order to improve the efficiency of the taking-out process.

  In Patent Document 1, a visual sensor that acquires position information of a work conveyed by a conveyor, and an order in which a robot performs work on a plurality of works are arranged based on the position information of the work transmitted from the visual sensor. A handling device comprising a control means configured to be replaced is disclosed.

  In Patent Document 2, information indicating the type, position, direction, and the like of an object conveyed by a conveyor is detected by a visual sensor, and the transfer direction and passage route of each object are determined based on the detected information. In addition, a system having a function of selecting an object that can be transferred without touching another object is disclosed.

  In Patent Document 3, when a plurality of workpieces placed in the vertical direction are picked up by a robot, information on a portion where the workpieces are overlapped is acquired by a visual sensor, and the workpiece is picked up based on the acquired information. A robot system having the function of changing the method is disclosed.

JP 2009-297881 A JP 2005-536417 A JP 2005-305613 A Japanese Patent Laid-Open No. 60-193013 JP-A-8-062214

  By the way, when the objects are sequentially taken out by the robot, the positions of the surrounding objects may change due to the objects coming into contact with each other or the gripping device that grips the objects coming into contact with the surrounding objects. . In such a case, it is necessary to capture an image of the object again with a visual sensor and acquire the position information of the object after movement, which increases the cycle time.

  When an object is conveyed by a conveyor, the visual sensor may be installed upstream from the robot. In such a case, if the position of the object changes unexpectedly in the object extraction process by the robot, the position information of the object after movement cannot be obtained by the visual sensor, and there is a possibility that the object extraction may fail. is there.

  Therefore, there is a need for a robot system that can take out objects one by one without interfering with surrounding objects.

According to the first invention of the present application, information representing the positions of a plurality of objects placed on a plane is detected by a visual sensor, and the plurality of objects are taken out one by one by a robot based on the detected information. A robot system for taking out according to a path, based on information detected by the visual sensor, a target object selecting unit for selecting a target object to be taken out by the robot, and information detected by the visual sensor The proximity state determination unit that determines whether other objects other than the target object are arranged in proximity to the target object, and the proximity state determination unit determines that at least one object is Based on information detected by the visual sensor when it is determined that the target object is disposed in proximity to the target object, the robot performs the eye detection. An avoidance vector determination unit that determines an avoidance direction and an avoidance amount parallel to the plane as an avoidance vector so that interference with the at least one object does not occur when the object is taken out; and the avoidance vector determination Based on the avoidance vector determined by the unit, an extraction path correction unit that generates a correction path that corrects the extraction path so that interference with other objects does not occur, and the target state target by the proximity state determination unit When it is determined that there is no object arranged close to the object, the robot is controlled according to the extraction route to take out the target object, and at least one object is detected by the proximity state determination unit. When it is determined that the target object is arranged close to the target object, the target is determined according to the correction path created by the take-out path correction unit. Comprising a take-out operation execution section for taking out the elephant was a robotic system is provided.
According to a second invention, in the robot system according to the first invention, when the avoidance vector determination unit cannot determine an avoidance vector that prevents interference with another object, the target object selection is performed. The unit is configured to newly select another target other than the target target as the target target.
According to a third aspect, in the robot system according to the first or second aspect, the proximity state determination unit sets a plurality of proximity state determination regions around the target object, and the target object When the target object other than is overlapped with at least one of the proximity state determination regions, it is determined that the target object is disposed close to the target target object.
According to a fourth invention, in the robot system according to the first or second invention, the proximity state determination unit sets a plurality of proximity state determination regions around objects other than the target object, respectively. When the predetermined outer shape of the target object overlaps the proximity state determination region of the target object, it is configured to determine that the target object is arranged close to the target object.
According to a fifth aspect, in the robot system according to the first or second aspect, the proximity state determination unit determines in advance a distance between the target object and an object other than the target object. It is configured to determine that the object is disposed in proximity to the target object when the threshold value is smaller than the threshold value.
According to a sixth invention, in the robot system according to the first or second invention,
The proximity state determination unit generates a polygon corresponding to the outer shape of each object,
The distance between the side of the polygon corresponding to the outer shape of the target object and the vertex of the polygon corresponding to the outer shape of the object other than the target object, or the polygon corresponding to the outer shape of the target object And when the distance between the vertex of the polygon and the side of the polygon corresponding to the outer shape of the object other than the target object is smaller than a predetermined threshold, the object is arranged close to the target object Configured to be determined.
According to a seventh aspect, in the robot system according to the third or fourth aspect, the avoidance vector determination unit is determined by the proximity state determination unit that the target object is close to the target target object. In addition, the direction determined from the proximity state determination area toward the target object is determined as the avoidance direction.
According to an eighth aspect, in the robot system according to the fifth or sixth aspect, the avoidance vector determination unit is an object that is determined to be close to the target object by the proximity state determination unit. The direction from the target toward the target object is determined as the avoidance direction.
According to a ninth invention, in the robot system according to the seventh invention, the avoidance vector determination unit causes the proximity state determination unit to cause the target object to approach the target object in a plurality of proximity state determination regions. When it is determined that there is a temporary avoidance vector for each of the plurality of proximity state determination regions, the avoidance vector is determined based on the temporary avoidance vector.
According to a tenth aspect, in the robot system according to the eighth aspect, the avoidance vector determination unit is configured such that a plurality of objects are arranged close to the target object by the proximity state determination unit. When the determination is made, a temporary avoidance vector is determined for each of the plurality of objects, and the avoidance vector is determined based on the temporary avoidance vector.
According to an eleventh aspect, in the robot system according to the first or second aspect, the take-out path correction unit determines the position of at least the first passing point of the correction path by the avoidance vector determination unit. A determination is made using the avoidance vector.
According to a twelfth aspect, in the robot system according to the first or second aspect, the take-out path correction unit moves the position of the start point of the take-out path according to the avoidance vector determined by the avoidance vector determination section. Further, the position moved by a predetermined distance in the direction away from the plane is determined as the position of the first passing point of the correction path.

  These and other objects, features and advantages of the present invention will become more apparent by referring to the detailed description of the exemplary embodiments of the present invention shown in the accompanying drawings.

  According to the robot system of the present invention, when there is an object arranged close to the target object to be picked up by the robot, the take-out path for picking up the target object is corrected by the avoidance vector. . Thereby, the target object can be taken out without interfering with surrounding objects.

1 is an overall configuration diagram of a robot system according to an embodiment. It is a figure which shows the coordinate system utilized for control of a robot. It is a figure which shows the coordinate system utilized for control of a robot. It is a functional block diagram of a robot control device. It is a flowchart which shows the process performed in the robot system which concerns on one Embodiment. It is a top view which shows a target object and the surrounding object. It is a figure which shows the example of the proximity | contact state determination area | region set around a target target object. It is a top view which shows a target object and the surrounding object. It is a figure which shows the rectangle corresponding to the rectangle corresponding to a target object, and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a top view which shows a target object and the surrounding object. It is a figure which shows the correction | amendment path | route created according to an avoidance vector. It is a figure which shows the correction | amendment path | route created according to an avoidance vector. It is a figure which shows the correction | amendment path | route created according to an avoidance vector. It is a figure which shows the correction | amendment path | route created according to an avoidance vector. It is a whole block diagram of the robot system which concerns on another embodiment. It is a figure which shows the coordinate system utilized for control of a robot.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The components of the illustrated embodiment have been appropriately dimensioned to aid in understanding the present invention. The same reference numerals are used for the same or corresponding components.

  FIG. 1 is an overall configuration diagram of a robot system 10 according to an embodiment. The robot system 10 includes a robot 1, a robot control device 2 that controls the robot 1, and a visual sensor 4 that is installed above an object 3 that is sequentially taken out by the robot 1. The robot system 10 is configured to detect information representing the positions of a plurality of objects 3 by the visual sensor 4 and to extract the objects 3 one by one according to the extraction path by the robot 1 based on the detected information. Yes.

  The robot 1 includes a motor 13 that drives each of a plurality of joint axes, and a gripping device 11 that is attached to the end of the arm 12. In FIG. 1, only some motors 13 are visible. Each motor 13 is driven in accordance with a control command transmitted from the robot control device 2 so that the gripping device 11 has a desired position and posture. The gripping device 11 has a claw 14 that opens and closes in accordance with a control command transmitted from the robot control device 2, and the object 3 can be gripped by the claw 14 so as to be openable.

  Although the illustrated robot 1 is a vertical articulated robot, a robot having any other configuration that can execute transfer of an object may be used. For example, the robot system 10 may include a robot having a moving mechanism, a parallel link mechanism, or a linear motion mechanism.

  A coordinate system used for controlling the robot 1 will be described with reference to FIGS. 2A and 2B. In the robot 1, a robot coordinate system R0 that is a reference for the operation of the robot 1 and the measurement of the visual sensor 4 is defined in advance. In addition, an arm coordinate system T0 is defined at the distal end point of the arm 12 of the robot 1, that is, the attachment position of the gripping device. It is known that the positional relationship between the arm coordinate system T0 and the robot coordinate system R0 is obtained based on the mechanism parameters of the robot 1.

  In this specification, the “robot position” means the position of the tool tip point (TCP) RP set at the tip of the tool (gripping device 11) attached to the arm 12. As described above, the positional relationship between the arm coordinate system T0 and the robot coordinate system R0 is known. Therefore, if the position of the tool tip point RP in the arm coordinate system T0 is known, the position of the tool tip point RP in the robot coordinate system R0 can be obtained by calculation. The position of the tool tip point RP in the arm coordinate system T0 may be obtained by calculation from CAD data of the gripping device 11, or may be obtained by measurement with a three-dimensional measuring instrument. The tool tip point RP may be set at a position separated from the tip of the gripping device 11 as necessary.

  Referring to FIG. 1 again, the visual sensor 4 is fixed by a gantry 41 at a position where a plurality of objects 3 placed on the flat upper surface of the pedestal 5 can be detected. A broken line 43 represents the visual field of the camera of the visual sensor 4. In the present embodiment, the objects 3 are arranged so as not to overlap each other within the movable range of the robot 1. The visual sensor 4 may be provided at any position as long as the object 3 can be detected. The fixing means for fixing the visual sensor 4 is not limited to the illustrated example. In one embodiment, the visual sensor 4 may be attached to the end of the arm 12 of the robot 1.

  The visual sensor 4 is calibrated in advance with respect to the robot coordinate system R0. Therefore, the position of an arbitrary part in the image acquired by the visual sensor 4 can be converted into a position in the robot coordinate system R0 based on the calibration data stored in the nonvolatile memory of the image processing device 42. Although the calibration method is not particularly limited, for example, a known technique described in Patent Document 4 can be used.

  The image data acquired by the visual sensor 4 is input to the image processing device 42. The image processing device 42 processes the image data and detects the position of each object 3. As long as the two-dimensional position information of the target 3 can be detected from the image data, the image processing method for detecting the position of the target 3 is not limited to a specific method. For example, the image processing device 42 can detect the position of the object 3 by template matching using normalized correlation.

  Further, when a plurality of objects 3 are included in the image data acquired by the visual sensor 4, the image processing device 42 determines an extraction order of each object 3. In one embodiment, the image processing device 42 determines the extraction order so that the objects 3 are extracted in the order in which the correlation value in template matching is high.

  In another embodiment, the image processing device 42 determines the extraction order of the objects 3 according to the position of the object 3 on the image or the position of the object 3 in the robot coordinate system R0. For example, the extraction order may be determined so that the objects 3 are extracted in descending order of the X-axis value in the robot coordinate system R0.

  The result of the image processing by the image processing device 42 is transmitted to the robot control device 2. The image processing apparatus 42 may be a general-purpose computer or a device configured exclusively for image processing. Alternatively, the image processing device 42 may be built in the robot control device 2.

  The robot control device 2 is a digital computer having a known hardware configuration such as a CPU, ROM, RAM, and volatile memory. The robot control device 2 includes an interface for transmitting and receiving data and signals to and from an external device, and is connected to an input device or a display device, an external storage device, or the like as necessary.

  FIG. 3 is a functional block diagram of the robot control device 2. The robot controller 2 selects the target object 31 (see FIG. 5) to be taken out by the robot 1 in accordance with the take-out order, and takes out the target object 31 without interfering with other objects 32. To control. The robot control apparatus 2 according to the illustrated embodiment includes a target object selection unit 21, a proximity state determination unit 22, an avoidance vector determination unit 23, an extraction path correction unit 24, and an extraction operation execution unit 25. I have.

  The target object selection unit 21 selects a target object 31 to be taken out by the robot 1 based on information detected by the visual sensor 4. The target object 31 is an object to be taken out by the robot 1 among the objects 3. The target objects 31 are sequentially selected according to a predetermined extraction order. Therefore, the target object selection unit 21 selects the target object 31 according to the signal output from the image processing device 42 as a result of the image processing of the image data acquired by the visual sensor 4.

  Based on the information detected by the visual sensor 4, the proximity state determination unit 22 determines whether or not an object 32 other than the target object 31 is disposed in proximity to the target object 31.

  The avoidance vector determination unit 23 is based on information detected by the visual sensor 4 when the proximity state determination unit 22 determines that at least one object 32 is disposed in proximity to the target object 31. The avoidance vector is determined. The avoidance vector is determined so that interference with the object 32 adjacent to the target object 31 does not occur when the target object 31 is taken out by the robot 1. The avoidance vector is determined by the avoidance direction parallel to the plane on which the target object 31 and the other object 32 are placed, and the avoidance amount.

  Based on the avoidance vector determined by the avoidance vector determination unit 23, the extraction path correction unit 24 corrects the extraction path when the target object 31 is extracted so that interference with the other object 32 does not occur. Create a route.

  The take-out operation execution unit 25 controls the robot 1 to take out the target object 31 according to a predetermined route. According to the present embodiment, the take-out operation execution unit 25 is designated by a program when it is determined by the proximity state determination unit 22 that there is no other object 32 arranged close to the target object 31. The target object 31 is taken out according to the take-out route. A program for designating the take-out path is stored in the nonvolatile memory or the external storage device of the robot control device 2.

  On the other hand, when it is determined by the proximity state determination unit 22 that at least one object 32 is arranged close to the target object 31, the extraction operation execution unit 25 is created by the extraction path correction unit 24. The target object 31 is taken out according to the corrected path.

  FIG. 4 is a flowchart illustrating processing executed in the robot system 10 according to an embodiment. When the robot control device 2 receives a start signal for starting the process of taking out the object 3, the processes in steps S401 to S410 are started. In step S <b> 401, image data of a plurality of objects 3 placed on the pedestal 5 is acquired by the visual sensor 4. When only one object 3 is detected, it is not necessary to execute the processes of steps S402 to S410 described later. Therefore, in that case, the robot system 10 takes out the object 3 by the robot 1 in accordance with a predetermined program.

  When a plurality of objects 3 are detected, the process proceeds to step S402, and the image data acquired by the visual sensor 4 is processed by the image processing device 42, and the extraction order of the objects 3 is determined.

  In step S403, the target object selection unit 21 selects the target object 31 to be extracted by the robot 10 from the plurality of objects 3 according to the extraction order determined in step S402.

  In step S <b> 404, the proximity state determination unit 22 determines whether there is another object 32 that is close to the target object 31. If the determination in step S404 is negative, the robot 1 can take out the target object 31 without interfering with surrounding objects. Accordingly, in this case, the process proceeds to step S405, and the take-out operation execution unit 25 moves the robot 1 according to the take-out path defined by the predetermined operation program, and takes out the target object 31.

  On the other hand, when the determination in step S404 is affirmed, the process proceeds to step S406, and the avoidance vector determination unit 23 determines an avoidance vector.

  In step S407, it is determined whether an avoidance vector has been determined in step S406. When the determination in step S407 is affirmed, the process proceeds to step S408, and the extraction path correction unit 24 corrects the extraction path according to the avoidance vector determined in step S406, and creates a correction path. Next, the process proceeds to step S409, where the take-out operation execution unit 25 moves the robot 1 according to the correction path created by the take-out path correction unit 24 and takes out the target object 31.

  If the determination in step S407 is negative, that is, if the avoidance vector determination unit 23 cannot obtain an avoidance vector, the process proceeds to step S410. In step S410, the target object selection unit 21 newly selects the target object 31 from other objects 3 different from the target object 31 for which the avoidance vector has not been obtained. And the process after step S404 is performed about the newly selected target object 31. FIG.

According to the robot system according to the present embodiment, the following effects can be obtained.
When there is an object 32 arranged close to the target object 31, the take-out path of the target object 31 is corrected so that interference with the object 32 does not occur. Therefore, when the target object 31 is taken out, it is possible to prevent the position of the object 32 around the target object 31 from changing unexpectedly. Thereby, it is not necessary to re-execute the imaging of the object 3 by the visual sensor 4 and the image processing by the image processing device 42, and the cycle time can be shortened.

  If an avoidance vector cannot be determined for the target object 31, that is, if interference with the surrounding object 32 cannot be avoided with certainty, another target object is not taken out without taking out the target object 31. A new target object 31 is selected. Thus, since the target object 31 is automatically switched according to the arrangement state of the object 3, the burden on the operator is reduced. Furthermore, since the request | requirement with respect to the initial stage arrangement | positioning of the target object 3 is eased, work efficiency can be improved.

  Next, the function of the proximity state determination unit 22 will be described in more detail. In one embodiment, the proximity state determination unit 22 includes a plurality of proximity state determination regions 33a around the target object 31 based on the position information of the target object 31 on a two-dimensional image including the target object 31. Set ~ 33d. FIG. 5 is a plan view showing the target object 31 and the object 32 positioned around the target object 31.

  In the illustrated embodiment, rectangular proximity state determination regions 33 a to 33 d are set around the sides 31 a to 31 d of the target object 31. The widths W1 and W2 of the proximity state determination regions 33a to 33d are appropriately determined according to the gap to be secured when the target object 31 is taken out. As long as the proximity determination region is set around the target object 31, the number and shape thereof are not particularly limited. For example, the proximity state determination area 33d may be divided into two.

  When the object 32 overlaps at least one of the proximity state determination areas 33a to 33d (in the case of FIG. 5, the proximity state determination area 33d), the proximity state determination unit 22 determines that the target object 32 is the target object 31. It is determined that they are arranged close to each other. Specifically, the brightness or color of the pixels in the proximity state determination areas 33a to 33d is measured. And when it differs from the brightness or color of the pixel assumed when the target object 32 does not exist, it determines with the target object 32 overlapping the proximity | contact state determination area | region 33a-33d.

  Even when the difference in pixel brightness or color is not actually caused by the presence of the object 32, it is preferable to perform the avoidance operation in the same manner. This is because there is a possibility that a foreign object is mixed even if it is not another object 32.

  FIG. 6 shows an example of the proximity state determination areas 33 a to 33 h set for the circular target object 31. In the illustrated example, proximity state determination areas 33 a to 33 h divided every 45 degrees are set around the target object 31. However, the proximity state determination area divided into an arbitrary number at an arbitrary interval may be used.

  In another embodiment (not shown), when the outer shape of the target object 31 is formed by a curve, a rectangle circumscribing the outer shape of the target object 31 is obtained, and a proximity state determination region is set around the rectangle. Also good.

  In one embodiment, the proximity state determination unit 22 may be configured to determine the proximity state by numerical calculation. In that case, the outer shape of the object 3 and the outer shape of the proximity state determination area provided around the object 3 are set in advance. Then, it is calculated whether or not the outer shape of the target object 31 and the outer shape of the proximity state determination area set around the surrounding object 32 overlap. When the target object 31 overlaps with the proximity state determination region, the proximity state determination unit 22 determines that the object 32 is disposed in proximity to the target object 31.

  In the modified examples of these embodiments, the proximity state determination unit 22 is automatically generated around the target object 31 or the object 32 by calculation according to the distance to be separated from the outer edge of the target object 31. A state determination area may be used.

  In one embodiment in which the object is circular, the proximity state determination unit 22 determines that the surrounding state is determined when the distance between the center of the target object 31 and the center of the surrounding object 32 falls below a predetermined threshold. It may be configured to determine that the target object 32 is arranged close to the target target object 31.

  In FIG. 7, two target objects 32 located around the target object 31, that is, a first object 321 and a second object 322 are shown together with the target object 31. A circle 34 concentric with the target object 31 indicated by a broken line indicates the diameter of the target object 31 (= the diameter of the object 32) and the size G of the gap to be secured when the target object 31 is taken out. And a radius D1 having a size equal to the sum of. When the centers 321 c and 322 c of the first and second objects 321 and 322 are within the range of the circle 34, the proximity state determination unit 22 moves the objects 321 and 322 closer to the target object 31. It is determined that it is arranged.

  In the case of the example shown in FIG. 7, since the center 321 c of the first object 321 is within the range of the circle 34, the first object 321 is arranged close to the target object 31. It is determined. On the other hand, since the center 322 c of the second object 322 is located outside the range of the circle 34, it is determined that the second object 322 is not arranged close to the target object 31.

  In an embodiment in which the object is not circular, the proximity state determination unit 22 may determine the proximity state of the object based on a convex polygon circumscribing the outline of the object. In that case, when the distance between each side of the polygon corresponding to the target object 31 and each vertex of the polygon corresponding to the surrounding object 32 falls below a predetermined threshold, these target objects It is determined that the object 31 and the object 32 are close to each other.

  FIG. 8 shows a rectangle corresponding to the target object 31, that is, circumscribing the outline of the target object 31 and having sides 131a to 131d (hereinafter referred to as “target object shape 131”) and surrounding objects. A rectangle corresponding to the object 32, that is, a rectangle circumscribing the outline of the object 32 (hereinafter referred to as “object shape 132”) is shown.

  According to the present embodiment, the position of the foot P1 of the perpendicular L1 dropped from the vertex P22 of the object shape 132 to the side 131c of the target object shape 131 is obtained by calculation. Further, a distance D2 from the midpoint P2 of the side 131c of the target object shape to the foot P1 of the perpendicular L1 is obtained by calculation. When both the length of the perpendicular line L1 (the distance from the vertex P22 to the foot P1 of the perpendicular line L1) and the distance D2 are lower than the corresponding threshold values, the proximity state determination unit 22 determines that the object 32 is the target object. It is determined that it is arranged close to 31. Note that the threshold for the length of the perpendicular line L1 is determined according to the gap to be secured between the target object 31 and the surrounding object 32. The threshold for the distance D2 is set to be at least half the length of the side 131c of the target object shape.

  In another embodiment, when the foot P1 of the perpendicular L1 dropped from the vertex P22 of the object shape 132 to the side 131c of the target object shape 131 is not located on the line segment of the side 131c, The proximity state may be determined by comparing the distance between the vertex P22 and the end point P14 of 131c close to the vertex P22 (see FIG. 8) with a predetermined threshold value. In this case, when the foot P1 of the vertical line L1 is located on the line segment of the side 131c, the proximity state is determined by comparing the length of the vertical line L1 with the corresponding threshold value.

  The proximity state determination unit 22 performs the above-described processing on all sides 131a to 131d of the target object shape 131 and all vertices P21 to P24 of the object shape 132. Further, the proximity state determination unit 22 performs the above-described processing for all the sides 132 a to 132 d of the object shape 132 and all the vertices P <b> 11 to P <b> 14 of the target object shape 131.

  Next, the function of the avoidance vector determination unit 23 will be described in detail. In the example shown in FIG. 5, the proximity state determination area 33 d set along the side 31 d of the target object 31 and the object 32 overlap. The avoidance vector determination unit 23 determines a direction perpendicular to the side 31d of the target object 31 and toward the inside of the target object 31 as an avoidance direction. The avoidance vector determination unit 23 determines the width W1 of the proximity state determination region 33d as an avoidance amount. The avoidance vector V is determined according to the avoidance direction and the avoidance amount thus determined.

  With reference to FIG. 9, the aspect which determines the avoidance vector V in embodiment with the some target object 32 arrange | positioned in proximity to the target target object 31 is demonstrated. In the present embodiment, proximity state determination areas 33a to 33f are set around the hexagonal target object 31 having sides 31a to 31f, respectively. The widths of the proximity state determination areas 33a to 33f are determined according to a gap to be secured between the target object 31 and the surrounding object 32 when the target object 31 is taken out.

  In the example shown in FIG. 9, the first object 321, the second object 322, and the third object 323 overlap with the proximity state determination areas 33 c, 33 d, and 33 e, respectively. Therefore, the three objects 321, 322, and 323 are arranged close to the target object 31.

  The avoidance vector determination unit 23 determines a direction that is perpendicular to the three sides 31c, 31d, and 31e of the target object 31 and that faces the inside of the target object 31 as an avoidance direction of the avoidance vectors Vc, Vd, and Ve. To do. Further, the avoidance vector determination unit 23 determines the respective widths of the proximity state determination regions 33c, 33d, and 33e as the avoidance amounts of the avoidance vectors Vc, Vd, and Ve.

  When three avoidance vectors Vc, Vd, Ve are obtained in this way, the angle formed between each avoidance vector Vc, Vd, Ve is calculated, and two vectors Vc, Extract Ve.

  According to one embodiment, the starting points of the vectors Vc and Ve move so as to coincide with the center of gravity CG of the target object 31, pass through the end point of the vector Vc, and obtain a straight line L2 perpendicular to the vector Vc. Further, a straight line L3 that passes through the end point of the vector Ve and is perpendicular to the vector Ve is obtained. Then, a vector V starting from the center of gravity CG of the target object 31 and ending at the intersection of the straight lines L2 and L3 is obtained. The avoidance vector determination unit 23 determines the vector V thus obtained as an avoidance vector.

  In another embodiment, the avoidance vector determination unit 23 may simply determine the combined vector of the vectors Vc and Ve as the avoidance vector when the avoidance amount is sufficient.

  In one embodiment, the avoidance vector determination unit 23 may have a function of confirming whether or not the avoidance vector determined by the avoidance vector determination unit 23 is appropriate. The avoidance vector V is used to correct the take-out path in order to avoid contact with the object 32 arranged close to the target object 31. However, as a result of correcting the take-out route of the target object 31, there is a possibility of approaching another object around the target object 31 and possibly contacting.

  According to one embodiment, if the size of the avoidance vector, i.e., the avoidance amount exceeds a predetermined threshold, it is considered that there is a possibility of contact with another object if the take-out operation is performed according to the correction path. It may be considered that interference with an object is unavoidable. In this case, it is determined that the avoidance vector has not been determined, and the target object 31 is not taken out by the robot 1, and another object 3 is selected as the target object 31.

  Alternatively, assuming that the target object 31 is moved according to the avoidance vector, it may be determined whether or not there is an object 32 close to the target object 31 after the movement. When there is an object 32 that is close to the target object 31 after movement, it is considered that interference with the object 32 cannot be avoided, and another object 3 is selected as the target object 31.

  In one embodiment, when the angle formed by the plurality of avoidance vectors Vc, Vd, and Ve is approximately 180 degrees, the interference with the object 32 may be regarded as unavoidable. If the angle formed between the two avoidance vectors is approximately 180 degrees, it means that the two avoidance vectors are nearly parallel to each other. Therefore, the intersection of the straight lines L2 and L3 that determine the end point of the avoidance vector V (see the figure) obtained from the two avoidance vectors cannot be calculated.

  If the two vectors are approximately 180 degrees, it means that the target object exists between the two objects. FIG. 10 shows a state in which there are objects 321 and 322 that overlap the proximity state determination areas 33a and 33d set along two opposing sides 31a and 31d of the target object 31.

  In this case, as indicated by arrows A1 and A2, even if the target object 31 is moved in a direction perpendicular to the avoidance vectors Va and Vd, the same as when the target object 31 is lifted straight up and taken out. Furthermore, there is a possibility of interfering with the objects 321 and 322. Therefore, when the target object 31 and the objects 321 and 322 are in a positional relationship as shown in the figure, it should be determined that interference between the target object 31 and the objects 321 and 322 is unavoidable. .

  Even when another object not shown in the figure is arranged close to the target object 31, the two vectors forming the maximum angle are the same vectors Va and Vd. It is determined that the interference with 322 cannot be avoided. When the target object exists in the entire range of the proximity state determination areas 33a to 33f around the target target object 31, it is naturally determined that the target target object 31 cannot avoid interference with the target object.

  With reference to FIG. 11, another embodiment for determining an avoidance vector from a plurality of vectors will be described. In the example shown in the figure, the three objects 321, 322, and 323 overlap the proximity state determination areas 33 c, 33 d, and 33 e of the target object 31, and the avoidance vector is determined using the three vectors. First, the vectors Vc, Vd, and Ve are moved so that the start points of the three vectors Vc, Vd, and Ve coincide with the center of gravity CG of the target object 31, respectively. Next, straight lines L2 to L4 that pass through the end points of the vectors Vc, Vd, and Ve and extend in a direction perpendicular to the directions of the vectors Vc, Vd, and Ve are obtained.

  Next, a common region R in which regions formed outside the straight lines L2 to L4 overlap with respect to the center of gravity CG is obtained. Further, a point P5 closest to the center of gravity CG in the common region R is obtained. The avoidance vector determination unit 23 determines a vector starting from the center of gravity CG of the target object 31 and having the point P5 in the common region R as an end point as the avoidance vector V. In the present embodiment, when the common region R does not exist, it is determined that interference between the target object 31 and the objects 321, 322, and 323 cannot be avoided. Similar to the embodiment described above, the avoidance vector determination unit 23 may have a function of determining whether or not the obtained avoidance vector V is appropriate.

  FIG. 12 shows a state in which one object 32 overlaps over two proximity state determination regions 33g and 33h. In such a case, the vectors Vg and Vh are obtained in the direction from the centroids CGh and CGg of the proximity state determination areas 33g and 33h toward the centroid CG of the target object 31, respectively. The sizes of the vectors Vg and Vh are equal to the widths of the proximity state determination areas 33g and 33h. The avoidance vector determination unit 23 determines the avoidance vector V from the plurality of vectors thus obtained according to the method described above with reference to FIG.

  As described with reference to FIG. 7, a method for determining the avoidance vector V when the proximity state of the object 32 is determined using the distance between the target object 31 and the object 32 will be described. In this case, the avoidance vector determination unit 23 determines the avoidance vector V using the distance between the target object 31 and the object 32.

  As shown in FIG. 13, three objects 321, 322, and 323 are arranged close to the target object 31. The vectors V1, V2, and V3 are directed to the center C of the target object 31 with the centers C1, C2, and C3 of the objects 321, 322, and 323 as starting points. The size of each vector V1, V2, V3 is equal to the size of the gap to be secured between the target object 31 and the surrounding objects. Of these vectors V1, V2 and V3, two vectors are extracted such that the angle formed between them is maximized. In the illustrated example, vectors V1 and V3 are extracted.

  In one embodiment, as described in connection with FIG. 9, the avoidance vector may be determined based on a straight line that passes through the end points of the vectors V1 and V3 and extends in a direction perpendicular to the vectors V1 and V3.

  In another embodiment, the avoidance vector V may be determined using a combined vector of the vectors V1 and V3 that maximize the angle formed between them.

  Specifically, the target object 31 is moved according to the combined vector of the vectors V1 and V3, and the center of the target object 31 after the movement is calculated. Next, the distance between the center of the newly acquired target object 31 after movement and the centers C1, C2, C3 of the surrounding objects 321, 322, 323 is calculated. Next, it is determined whether or not the objects 321, 322 and 323 are close to the target object 31. When it is determined that the target object 31 and the objects 321, 322, and 323 after moving are not close to each other, the combined vector of the vectors V 1 and V 3 is determined as the avoidance vector V.

  On the other hand, when it is determined that any one of the target object 31 and the objects 321, 322, and 323 after moving is still close to each other, the composite vector is multiplied by a predetermined coefficient to increase the size of the composite vector. change. An upper limit value may be set for the coefficient, and when an appropriate avoidance vector V cannot be determined with a coefficient equal to or less than the upper limit value, interference between the target object 31 and the objects 321, 322, and 323 cannot be avoided. Assuming that there is another target object 31 is selected.

  With reference to FIG. 14, another embodiment using a combined vector VR obtained by combining vectors V1 and V3 will be described. In the present embodiment, the direction of the composite vector VR is determined as the avoidance direction. On the other hand, the size of the avoidance vector is determined according to the following process.

  First, a straight line L5 that is parallel to the composite vector VR and passes through the center C of the target object 31 is calculated. Next, a circle 35 centering on the center C3 of the surrounding object, for example, the object 323 is obtained. The circle 35 has a radius D3 equal to the sum of the diameter of the object 323 and the gap to be secured with the surrounding objects when the target object 31 is removed.

  Next, intersections P6 and P7 between the straight line L5 and the circle 35 are obtained. Among these intersection points P6 and P7, an intersection point (intersection point P6 in the illustrated example) that is closer to the center C of the target object 31 is obtained. Next, the distance from the center C of the target object 31 to the intersection P6 is calculated. The distance thus obtained is the minimum avoidance amount that can reliably avoid contact with the object 323 when the target object 31 is taken out. The minimum amount of avoidance required to avoid the objects 321, 322, and 323 by executing the same process as described above for each of the other objects 321 and 322 that are close to the target object 31. Ask for. The maximum value among the minimum avoidance amounts is set as the size of the avoidance vector V (avoidance amount).

  With reference to FIG. 15, an embodiment in which the avoidance vector V is determined without using the vectors V1 and V3 will be described. The circles 36, 37, and 38 are centered on the centers C1, C2, and C3 of the objects 321, 322, and 323, and the diameters of the objects 321, 322, and 323 and the surrounding objects 321 when the target object 31 is taken out. , 322, 323 and a radius equal to the sum of the gaps to be secured.

  Next, the point P8 located at the smallest distance from the center C of the target object 31 among the areas overlapping each other among the areas outside the circles 36, 37, and 38 formed around the objects 321, 322, and 323. calculate. The avoidance vector determination unit 23 determines a vector having the center C of the target object 31 as a start point and the point P8 as an end point as an avoidance vector V.

  Next, functions of the extraction path correction unit 24 and the extraction operation execution unit 25 will be described in detail. FIG. 16 is a diagram illustrating the extraction path corrected by the extraction path correction unit 24 according to the avoidance vector V. In FIG. 16, the passage points P31 and P32 of the take-out route programmed, and the passage points P41 and P42 of the correction route created by the take-out route correction unit 24 are shown. Each passing point represents a position through which the tool tip point RP of the gripping device 11 of the robot 1 passes. A point P10 represents the position (gripping position) of the tool tip point RP when the gripping device 11 of the robot 1 grips the target object 31.

  When there is no target object arranged close to the target target object 31, the robot 1 grips the target target object 31 with the gripping device 11, and then passes through the passing points P <b> 31 and P <b> 32 to take out the target target object 31. . On the other hand, when the target object 32 is arranged close to the target target object 31, when the target target object 31 is taken out according to the programmed extraction route, the target target object 31 or the robot 1 comes into contact with the target object 32. There is a risk of changing the position of the object 32.

  Therefore, when the target object 32 is arranged close to the target target object 31, the robot 1 takes out the target target object 31 according to the correction path passing through the passing points P 41 and P 42 corrected by the avoidance vector V. Note that the gripping position P10 is not changed. The first passing point P41 of the correction path is equal to the end point of the avoidance vector V starting from the first passing point P31 of the programmed take-out path. Similarly, the second passing point P42 of the correction path is equal to the end point of the avoidance vector V starting from the second passing point P32 of the programmed take-out path.

  FIG. 17 shows a correction path created according to another embodiment. When the target object 31 is taken out, it is most likely that the target object 31 contacts the surrounding object 32 when moving toward the first passing point. Therefore, according to the present embodiment, the first passing point P31 of the take-out route that is the first passing point is changed to P41 according to the avoidance vector V, but the second passing point P32 and the subsequent points are not changed. Thereby, according to the present embodiment, as shown in FIG. 7, the second passing point P42 of the correction path coincides with the programmed extraction path P32.

  FIG. 18 shows a correction path created according to yet another embodiment. In the present embodiment, only the grip position P10 and the target point P11 are specified by the program. That is, unlike the embodiment described with reference to FIGS. 16 and 17, the first passing points P31 and P32 are not specified by the program. In such a case, the extraction path may be corrected so that the passing point P41 is added to the position specified by the end point of the avoidance vector V starting from the gripping position P10. Furthermore, as shown in FIG. 18, the take-out path may be corrected so as to add the second passing point P42.

  FIG. 19 shows the extraction path corrected according to yet another embodiment. In the present embodiment, only the gripping position P10 and the target point P11 are designated by the program, as in the embodiment of FIG. In addition, correction route passing points P41 and P42 are added by the extraction route correction unit 24. However, in the present embodiment, the first passing point P41 added by the correction is calculated according to the vector V ′ obtained by adding the upward vector V ″ to the avoidance vector V from the gripping position P10.

  If the first passing point P41 of the correction path is designated according to the vector V ′ obtained by adding the upward vector V ″, the target object 31 moves so that the target object 31 can be lifted from the floor surface. The object 31 can be taken out without being rubbed against the floor surface.

  FIG. 20 is an overall configuration diagram of a robot system 10 according to an embodiment different from the embodiment described with reference to FIG. The robot system 10 according to the present embodiment is configured to sequentially take out the objects 3 conveyed by the conveyor 51 having a substantially flat placement surface one by one by the gripping device 11 of the robot 1. The robot system 10 according to the present embodiment is configured to be able to execute a process of taking out the object 3 following the movement of the object 3 conveyed by the conveyor 51 using visual tracking technology. On the upstream side of the robot 1 in the transport direction, a visual sensor 4 that captures an image of the object 3 included in the detection area 52 is installed. Image data acquired by the visual sensor 4 is input to the image processing device 42, and information such as the position of the object 3 is calculated.

  An encoder 54 is attached to the conveyor 51, and the movement amount of the conveyor 51, that is, the movement amount of the object 3 is calculated based on the detection information of the encoder 54. The robot controller 2 moves the object 3 together with the conveyor 51 based on the position information of the object 3 obtained from the detection information of the visual sensor 4 and the amount of movement of the object 3 obtained from the detection information of the encoder 54. The position of 3 can be tracked. The robot 1 is configured to sequentially take out the objects 3 when it is detected that the object 3 has reached the work area 53. Since such visual tracking is a well-known technique, detailed description thereof is omitted in this specification. For example, the technique described in Patent Document 5 may be used.

  In the robot system 10 in which the visual sensor 4 is provided upstream of the robot 1, the position information of the target 3 is acquired by the visual sensor 4 after the target 3 has moved outside the detection area 52 of the visual sensor 4. I can't. Therefore, when a certain object 3 is taken out and the position of the object 3 is changed by contacting another object 3, the position information acquired by the visual sensor 4 is no longer accurate. Therefore, it is particularly important to accurately take out the object 3 so as not to change the position of the surrounding object 3.

  In the present embodiment, the conveyor 51 moves at a constant speed in the direction indicated by the arrow in FIG. 20 (from the right side to the left side). Although the target object 3 may be arrange | positioned irregularly on the conveyor 51, the target objects 3 shall not overlap up and down. Further, unless the robot 1 or another object 3 is in contact with the object 3, the object 3 is integrally moved without sliding with respect to the conveyor 51.

  In the present embodiment, the proximity state determination unit 22 and the avoidance vector determination unit 23 have the same functions as in the above-described embodiment. However, it is preferable that the take-out path correction unit 24 and the take-out operation execution unit 25 need to be set in consideration of the fact that the position of the object 3 constantly moves. Specifically, as described in Patent Document 5, it is convenient to set a tracking coordinate system that moves together with the conveyor 51 and to define the take-out path and the correction path of the object 3 in the tracking coordinate system. .

  FIG. 21 shows the passing point P31 of the extraction path expressed in the tracking coordinate system T1. As described above, the tracking coordinate system T1 is a dynamic coordinate system that moves following the operation of the conveyor 51. The position of the conveyor 51 is detected by the encoder 54, and the robot control device 2 is configured to be able to acquire the position of the tracking coordinate system T1 based on the detection information of the encoder 54.

  In FIG. 21, the tracking coordinate system T1 and the passing point P31 of the take-out path at a certain time are shown together with the passing point P31 in the tracking coordinate system T1 at another time. Thus, the position of the passing point P31 in the robot coordinate system R0 or the tool coordinate system T0 constantly changes, while the passing point P31 in the tracking coordinate system T1 is constant.

  Therefore, in the present embodiment, if the robot 1 performs the take-out operation based on the tracking coordinate system T1 that moves together with the conveyor 51 instead of the fixed coordinate system, for example, the robot coordinate system R0, the above-described embodiment. It is possible to easily execute the same process. That is, when the robot 1 moves relative to the dynamic tracking coordinate system T1, the step of taking out the object 3 is executed.

  As described above, according to the robot system according to the present invention using the visual tracking, it is possible to prevent the positions of the surrounding objects from being unexpectedly changed in the target object extraction process. Thereby, the extraction process can be executed stably.

  Although various embodiments of the present invention have been described above, those skilled in the art will recognize that the functions and effects intended by the present invention can be realized by other embodiments. In particular, the components of the above-described embodiments can be deleted or replaced without departing from the scope of the present invention, or known means can be further added. It is obvious to those skilled in the art that the present invention can be implemented by arbitrarily combining features of a plurality of embodiments explicitly or implicitly disclosed in the present specification.

DESCRIPTION OF SYMBOLS 1 Robot 11 Grasp apparatus 12 Arm 13 Motor 14 Claw 2 Robot control apparatus 21 Target object selection part 22 Proximity state determination part 23 Avoidance vector determination part 24 Extraction path | route correction | amendment part 25 Extraction operation execution part 3 Target object 31 Target object 32, 321, 322, 323 Objects 33 a-33 h (proximity state determination area) 4 Visual sensor 42 Image processing device 5 Base (plane)
51 Conveyor R0 Robot coordinate system T0 Tool coordinate system T1 Tracking coordinate system

Claims (12)

A robot system that detects information representing positions of a plurality of objects placed on a plane by a visual sensor, and extracts the plurality of objects one by one according to an extraction path based on the detected information,
A target object selection unit that selects a target object to be taken out by the robot based on information detected by the visual sensor;
Based on information detected by the visual sensor, a proximity state determination unit that determines whether other objects other than the target object are arranged in proximity to the target object;
When the proximity state determination unit determines that at least one object is disposed in proximity to the target object, the target object is detected by the robot based on information detected by the visual sensor. An avoidance vector determination unit that determines an avoidance direction and an avoidance amount parallel to the plane as an avoidance vector so that interference with the at least one object does not occur when taking out an object;
Based on the avoidance vector determined by the avoidance vector determination unit, an extraction path correction unit that creates a correction path that corrects the extraction path so that interference with other objects does not occur;
When the proximity state determination unit determines that there is no object arranged in proximity to the target object, the robot is controlled according to the take-out route to take out the target object, and the proximity state When the determination unit determines that at least one object is arranged close to the target object, execution of an extraction operation to extract the target object according to the correction path created by the extraction path correction unit And
A robot system comprising:   When the avoidance vector determination unit cannot determine an avoidance vector that prevents interference with other objects, the target object selection unit newly sets another object other than the target object as a target object. The robotic system of claim 1, configured to select as an object. The proximity state determination unit sets a plurality of proximity state determination regions around the target object,
When an object other than the target object overlaps at least one of the proximity state determination regions, the object is determined to be disposed in proximity to the target object The robot system according to claim 1 or 2, wherein: The proximity state determination unit sets a plurality of proximity state determination regions around an object other than the target object,
The device is configured to determine that the object is disposed in proximity to the target object when a predetermined outer shape of the target object overlaps a proximity state determination region of the object. Item 3. The robot system according to Item 1 or 2.   When the distance between the target object and an object other than the target object is smaller than a predetermined threshold, the proximity state determination unit is arranged close to the target object The robot system according to claim 1, wherein the robot system is configured to determine that the operation has been performed. The proximity state determination unit generates a polygon corresponding to the outer shape of each object,
The distance between the side of the polygon corresponding to the outer shape of the target object and the vertex of the polygon corresponding to the outer shape of the object other than the target object, or the polygon corresponding to the outer shape of the target object And when the distance between the vertex of the polygon and the side of the polygon corresponding to the outer shape of the object other than the target object is smaller than a predetermined threshold, the object is arranged close to the target object The robot system according to claim 1, wherein the robot system is configured to determine that the operation has been performed.   The avoidance vector determining unit determines a direction determined from the proximity state determination region determined by the proximity state determination unit to be close to the target object toward the target object. A robotic system according to claim 3 or 4, wherein the robotic system is configured to determine as follows.   The avoidance vector determination unit is configured to determine, as the avoidance direction, a direction from the object determined to be close to the target object by the proximity state determination unit toward the target object. The robot system according to claim 5 or 6.   When the proximity state determination unit determines that the target object is close to the target object in a plurality of proximity state determination regions, the avoidance vector determination unit applies each of the plurality of proximity state determination regions. The robot system according to claim 7, wherein the robot system is configured to determine a temporary avoidance vector for the temporary avoidance vector based on the temporary avoidance vector.   When the proximity state determination unit determines that the plurality of objects are arranged in proximity to the target object, the avoidance vector determination unit is provisional for each of the plurality of objects. The robot system of claim 8, configured to determine an avoidance vector and to determine the avoidance vector based on the temporary avoidance vector.   The extraction route correction unit is configured to determine a position of at least the first passing point of the correction route using the avoidance vector determined by the avoidance vector determination unit. Robot system.   The extraction path correction unit moves the position of the start point of the extraction path according to the avoidance vector determined by the avoidance vector determination unit, and further corrects a position moved by a predetermined distance in a direction away from the plane. The robot system according to claim 1, wherein the robot system is configured to determine the position of the first passing point of the path.

Images