JP5332873B2 - Bag-like workpiece recognition device and method - Google Patents

Description

  The present invention relates to a bag-like workpiece recognition apparatus and method.

In the present application, the “bag-shaped workpiece” refers to a rice bag, a cement bag, a fertilizer bag, etc., which is a granular bag packed in a smooth bag with few irregularities. In the case where such a bag-like workpiece is a picking target (work) and the workpiece is gripped and transported, conventionally, the position of the workpiece to be picked is taught without picking up the workpiece and picked by a robot. However, in this case, it is necessary to arrange the workpieces at the teaching position, and there is a problem that it is not possible to cope with the case where a deviation occurs.
Therefore, Patent Documents 1 to 4 have already been proposed as means for recognizing the position and orientation of a workpiece in a stacked state and gripping and conveying the workpiece.

Patent Document 1 searches for a shape similar to a registered shape from three-dimensional data measured using a sensor such as a laser radar or a stereo camera.
Patent Document 2 discloses data of workpieces in which geometric features such as characteristic straight lines and circles and features such as contours are registered from two-dimensional data or three-dimensional data measured using a sensor. Compared with the search.

Patent Document 3 is a recognition method for depalletization, in which a lighting device is attached, a shadow is generated, an edge is extracted, and a contour is recognized. Since a rectangular parallelepiped is assumed, fitting is performed on a straight line using the fact that the edges are orthogonal at the corners. A method of using height information from a stereo camera or the like for selecting an article and a method of obtaining an edge by projecting slit light to obtain an edge are also used.
In Patent Document 4, imaging is performed by irradiating slit light to obtain a shape, and then a shape is linearly fitted in the vicinity between the bags, and a feature space in which a cross section between the bags is expressed in a V shape is used. Recognizes the boundary between bags.

JP 2008-309671 A, “Object recognition method and apparatus” Japanese Patent Application Laid-Open No. 9-212463, “3-D Object Recognition Method and Apparatus” Japanese Patent No. 3894514, “article position recognition device” Japanese Patent No. 3919932, “bag-shaped workpiece posture detection device”

In Patent Document 1 described above, it is necessary to register a workpiece to be recognized in advance. Moreover, it cannot respond when a target object (work) is indefinite shape like a bag.
In Patent Document 2, it cannot be recognized when there is no geometric feature. The bag-like workpiece looks rectangular when viewed from above and can be used as a geometric feature, but it is not a perfect rectangle but an indefinite shape. Therefore, there are cases where it cannot be recognized or calculation time is required for fitting.

In Patent Document 3, since the target shape is assumed to be a rectangular parallelepiped, it cannot be used for a bag-shaped object. In particular, the edge is fitted with a straight line, but since the bag is rounded, the fitting is difficult and erroneous detection may occur.
In Patent Document 4, since the space between the bags is recognized, only the position and rotation on the plane can be known. Therefore, the posture in three dimensions cannot be known, and it cannot be recognized when the stacked bags collapse.

  The present invention has been developed to solve the above-described conventional problems. That is, an object of the present invention is to provide a bag-like workpiece recognition apparatus and method that can detect the position and posture of a bag-like workpiece to be picked with high accuracy for an irregular bag-like workpiece.

According to the present invention, a distance sensor that measures a three-dimensional shape of a plurality of bag-shaped workpieces;
A bag-like workpiece recognition device comprising an object recognition processing device that recognizes the position and orientation of a bag-like workpiece picked from the three-dimensional shape and outputs the result to a robot controller,
The object recognition processing device includes:
A mesh generating device that generates a mesh using a distance between adjacent measurement points of the three-dimensional shape;
A first label allocation device that assigns different label values to measurement points whose distance to an adjacent measurement point is equal to or greater than a predetermined first threshold for each measurement point on the mesh;
A second label assigning device that assigns different label values to measurement points at which an angular difference between normal vectors at adjacent measurement points is equal to or greater than a predetermined second threshold for each measurement point on the mesh;
An information output device that selects a label value of a bag-like workpiece to be picked from the plurality of label values, and outputs information necessary for picking the bag-like workpiece to a robot controller;
From the angle of the normal vector of the three-dimensional shape, the discontinuous portion is regarded as the boundary of the bag-shaped workpiece, and a different label value is assigned to each bag-shaped workpiece, and the position and orientation of the bag-shaped workpiece are recognized for each label value. A bag-like workpiece recognition device is provided.

The selection of the label value of the bag-like workpiece to be picked is selected from the plurality of bag-like workpieces to which labels are assigned, the one with the highest position, the one with the largest area, the one with the most horizontal posture, It is the most inclined.
The information necessary for picking is a shape such as a picking position, a barycentric coordinate, or a width dimension for each selected bag-like workpiece.

According to the present invention, the three-dimensional shape of a plurality of bag-like workpieces having smooth surfaces is measured,
A bag-like workpiece recognition method for recognizing the position and posture of a bag-like workpiece picked from the three-dimensional shape and outputting the result to a robot controller,
Apply the mesh using the distance to the adjacent measurement point of the three-dimensional shape,
For each measurement point on the mesh,
Assign different label values to measurement points whose distance to adjacent measurement points is equal to or greater than a predetermined first threshold,
And assigning different label values to measurement points at which the normal vector angle difference between adjacent measurement points is equal to or greater than a predetermined second threshold,
By selecting a label value of the bag-like workpiece to be picked from the plurality of label values, and outputting information necessary for picking the bag-like workpiece to the robot controller,
From the angle of the normal vector of the three-dimensional shape, the discontinuous portion is regarded as the boundary of the bag-shaped workpiece, and a different label value is assigned to each bag-shaped workpiece, and the position and orientation of the bag-shaped workpiece are recognized for each label value. A bag-like workpiece recognition method is provided.

  According to the apparatus and method of the present invention, an object recognition processing apparatus is provided, and the discontinuous portion is regarded as the boundary of the bag-shaped workpiece from the angle of the normal vector of the three-dimensional shape of the plurality of bag-shaped workpieces. Different label values are assigned for each label, and the position and orientation of the bag-like workpiece are recognized for each label value, so the position and orientation of the picking bag-like workpiece can be detected with high accuracy even for irregular bag-like workpieces. can do.

Therefore, the following effects can be obtained by the apparatus and method of the present invention.
(1) Since teaching is not required, if there is a bag-shaped workpiece within the measurement range of the distance sensor, picking can be performed even if the position of each bag-shaped workpiece is shifted.
(2) Since the position and orientation of the bag-like workpiece to be picked are recognized only by the data processing of the three-dimensional shape acquired by the distance sensor, it is not necessary to register the shape of the bag-like workpiece to be recognized in advance.
(3) Since a different label value is assigned to each bag-like workpiece and each bag-like workpiece is recognized, it can be stably recognized even if there is a change in shape. In addition, since geometric feature fitting is not used, it can be recognized in a certain calculation time.
(4) Since a shape (for example, a rectangular parallelepiped) is not assumed for the bag-like workpiece, there is no fitting process to a straight line or the like, and an indefinite shape can be handled.
(5) The three-dimensional posture can be known from the coordinates of the point group of the output label value. Accordingly, it is possible to select the most horizontal one, or to select the one in which the stacked bags are collapsed and inclined.

It is a whole block diagram of the picking system provided with the bag-like workpiece recognition device of the present invention. It is a schematic diagram of the workpiece | work seen from the sensor. It is a schematic diagram of the apparatus of FIG. 1 seen from the horizontal direction. It is an operation | movement flowchart of the picking system of FIG. It is a flowchart of the recognition algorithm in FIG.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

FIG. 1 is an overall configuration diagram of a picking system including a bag-like workpiece recognition device of the present invention.
In this figure, 1 is a work, 2 is a robot, 3 is a robot hand, 4 is a hand control unit, 5 is a robot control unit, 6 is a general processing unit, and the robot control device 7 is a robot control unit 5 and a general processing unit. It consists of six.

  The workpiece 1 is a bag-like workpiece in which granular materials are packed in a smooth bag having no irregularities, such as a rice bag, a cement bag, a fertilizer bag, and the like. The loading state of the workpiece 1 may be random. Further, the measurement table 8 does not need to be fixed, and may be a conveyor device or the like as long as the measurement of a three-dimensional shape to be described later is not hindered, for example, the measurement time is short.

The robot 2 is, for example, an articulated three-dimensional robot, and can control the position and posture of the robot hand 3 with six degrees of freedom within a predetermined working area. The robot 2 is not limited to an articulated three-dimensional robot, and may be another robot.
The robot hand 3 is, for example, a vacuum suction pad, a gripping device having an opening / closing mechanism, or a composite device thereof, and sucks or grips the upper surface of the picked bag-like workpiece 1 and lifts it one by one to another position. It comes to hold.

The hand control unit 4 outputs a control signal to the robot hand 3 in response to the gripping command 9 a received from the general processing unit 6, receives a position signal from the robot hand 3, and transmits it to the general processing unit 6.
The hand control unit 4 is not essential and may be omitted, and the function may be included in the overall processing unit 6.

  The robot control unit 5 controls the robot 2 based on the position and posture of the picked bag-like workpiece received from the general processing unit 6, and sucks or grips the upper surface of the bag-like workpiece 1 to be picked by the robot hand 3. Lift one by one, hold to another position and transport.

  The overall processing unit 6 transmits the position and posture of the picked bag-shaped workpiece received from the bag-shaped workpiece recognition device 10 of the present invention to the robot control unit 5.

  In FIG. 1, the bag-like workpiece recognition device 10 of the present invention includes a distance sensor 12, a sensor control unit 14, and an object recognition processing device 16.

The distance sensor 12 measures the three-dimensional shape of the plurality of bag-like workpieces 1 having smooth surfaces.
In this example, the distance sensor 12 is a laser radar, is positioned above the measurement table 8, and measures the three-dimensional shape of the plurality of bag-like workpieces 1 from above. Further, a moving / rotating mechanism 13 (for example, a linear motion table or a rotating table) for controlling the height and posture of the distance sensor 12 is provided, and the posture of the distance sensor 12 can be freely changed to adjust the visual field.
For example, when the object is small, the field of view is approached, and when the object is large, the object is measured away to optimize the field of view.

The distance sensor 12 is not limited to the laser radar, and may be a stereo camera with a constant interval so that the distance to the workpiece can be detected.
Moreover, the photometric stereo which measures a three-dimensional shape with a monocular camera may be used. In this case, when recognizing a stack of monochromatic spheres, the periphery of the sphere is dark due to the shadow, and the inside of the sphere becomes bright. Therefore, assuming that the profile of the image taken at this time corresponds to the Z coordinate of the three-dimensional shape, a monochromatic sphere can be measured with a monocular camera.

The sensor control unit 14 receives the posture control signal 9 b from the object recognition processing device 16, outputs this signal to the distance sensor 12, controls the height and posture of the distance sensor 12, and controls a plurality of values measured by the distance sensor 12. The three-dimensional shape (sensor data 9c) of the bag-like workpiece 1 is received and transmitted to the object recognition processing device 16.
The sensor control unit 14 is not essential and may be omitted, and its function may be included in the object recognition processing device 16.

The object recognition processing device 16 has a function of recognizing the position and posture of the bag-like workpiece 1 picked from the three-dimensional shape measured by the distance sensor 12 and outputting the result to the robot control device 7.
The object recognition processing device 16 is a PC (computer) in this example. Note that the robot control device 7 (the robot control unit 5 and the overall processing unit 6) and the sensor control unit 14 described above may be configured by the same PC or may be configured by independent PCs.

FIG. 2 is a schematic diagram of the work viewed from the position of the distance sensor 12.
In this figure, when a plurality of bag-like workpieces 1 having smooth surfaces are viewed from the position of the distance sensor 12, even when the loading state of the workpieces 1 is random, the boundary portion 1a where the workpieces contact each other is accurately detected, It is necessary to accurately recognize the position and posture of each workpiece. In this figure, the X axis and the Y axis are orthogonal axes in the horizontal plane.

FIG. 3 is a schematic view of the apparatus of FIG. 1 viewed from the horizontal direction. FIG. 3A shows a case when viewed from the Y-axis direction of FIG. FIG. 3B is an enlarged view of the boundary portion 1a where the workpieces are in contact with each other.
As can be seen from FIG. 3B, when the object is a bag-shaped workpiece 1, there is always a discontinuous portion in the angle of the normal vector of the boundary portion 1a with the adjacent workpiece. In the present invention, since this discontinuous portion is used as the boundary of the label value and one label value is obtained for each bag by performing labeling, the position and orientation of the bag (work) are determined by the coordinates of the point group having this label value. Recognize

FIG. 4 is an operational flow diagram of the picking system of FIG.
In this figure, this picking system includes steps (processes) S1 to S5.

In step S <b> 1, the height and posture of the distance sensor 12 are adjusted so that the object to be measured (bag-shaped workpiece 1) appropriately enters the visual field (measurement range) of the distance sensor 12. “Appropriate” means that the entire work 1 is within the field of view and is sufficiently large and has a resolution that can be used for the subsequent recognition processing. The posture adjustment method is preferably automatic adjustment by feedback or the like, but may be manual.
In step S2, a three-dimensional shape is acquired by the distance sensor 12. The three-dimensional shape here assumes a point group in which points having three-dimensional coordinates are gathered.

In step S3, from the angle of the normal vector of the three-dimensional shape, the discontinuous portion is regarded as the boundary of the bag-shaped workpiece, and a different label value is assigned to each bag-shaped workpiece, and the position of the bag-shaped workpiece is determined for each label value. Recognize posture.
In step S4, necessary information is sent to the overall processing unit from the recognition result. The necessary information includes the number of recognized objects, the position and orientation information of each recognized object, and the like.
In step S5, the overall processing unit controls the robot and the hand through the robot control unit and the hand control unit, and picks the workpiece to be picked at the designated coordinate point.

FIG. 5 is a flowchart of the recognition algorithm in FIG.
In this figure, the recognition algorithm is composed of steps (processes) S31 to S34.
In step S31, noise removal of the measurement data (three-dimensional shape) acquired by the distance sensor 12 is performed using, for example, a median filter. This step may be omitted.
In step S32, a mesh (for example, a triangular mesh) is applied (generated) using the distance between adjacent measurement points of the measurement data (three-dimensional shape). With this mesh generation, the three-dimensional shape can be grasped in 3D.

In step S32, label values are set so as to satisfy the following conditions 1 and 2 for each measurement point on the mesh.
Condition 1: A different label value is assigned to a measurement point whose distance to an adjacent measurement point is a predetermined first threshold value or more.
Condition 2: Different label values are assigned to measurement points whose normal vector angle difference between adjacent measurement points is equal to or greater than a predetermined second threshold.
Here, the label value is an arbitrary number or symbol.

  In step S33, the label value of the bag-like workpiece 1 to be picked is selected from the plurality of label values, and information necessary for picking the bag-like workpiece 1 is output to the robot controller.

  According to the apparatus and method of the present invention described above, the object recognition processing device 16 is provided, and the discontinuous portion is defined as the boundary of the bag-like workpiece 1 from the angle of the normal vector of the three-dimensional shape of the plurality of bag-like workpieces 1. Since different label values are assigned to each bag-like workpiece 1 and the position and posture of the bag-like workpiece 1 are recognized for each label value, the position of the bag-like workpiece to be picked even if it is an irregular bag-like workpiece And the posture can be detected with high accuracy.

Therefore, the following effects can be obtained by the apparatus and method of the present invention.
(1) Since teaching is not required, if the bag-shaped workpiece 1 exists within the measurement range of the distance sensor 12, picking can be performed even if the position of each bag-shaped workpiece is shifted.
(2) Since the position and orientation of the bag-like workpiece to be picked are recognized only by the data processing of the three-dimensional shape acquired by the distance sensor 12, it is not necessary to register the shape of the bag-like workpiece to be recognized in advance.
(3) Since a different label value is assigned to each bag-like workpiece 1 and each bag-like workpiece is recognized, it can be stably recognized even if there is a shape change. In addition, since geometric feature fitting is not used, it can be recognized in a certain calculation time.
(4) Since the shape (for example, a rectangular parallelepiped) is not assumed with respect to the bag-like workpiece 1, there is no fitting process to a straight line or the like, and it can cope with an indefinite shape.
(5) The three-dimensional posture can be known from the coordinates of the point group of the output label value. Accordingly, it is possible to select the most horizontal one, or to select the one in which the stacked bags are collapsed and inclined.

  In addition, this invention is not limited to embodiment mentioned above, Of course, a various change can be added in the range which does not deviate from the summary of this invention.

1 work (bag-like work), 1a boundary part,
2 robots, 3 robot hands,
4 Hand control unit, 5 Robot control unit,
6 General processing section, 7 Robot controller,
8 Measuring table,
10 Bag-like workpiece recognition device,
12 distance sensor, 13 movement / rotation mechanism,
14 sensor control unit, 16 object recognition processing device

Claims (3)

A distance sensor for measuring a three-dimensional shape of a plurality of bag-like workpieces;
A bag-like workpiece recognition device comprising an object recognition processing device that recognizes the position and orientation of a bag-like workpiece picked from the three-dimensional shape and outputs the result to a robot controller,
The object recognition processing device includes:
A mesh generating device that generates a mesh using a distance between adjacent measurement points of the three-dimensional shape;
A first label allocation device that assigns different label values to measurement points whose distance to an adjacent measurement point is equal to or greater than a predetermined first threshold for each measurement point on the mesh;
A second label assigning device that assigns different label values to measurement points at which an angular difference between normal vectors at adjacent measurement points is equal to or greater than a predetermined second threshold for each measurement point on the mesh;
An information output device that selects a label value of a bag-like workpiece to be picked from the plurality of label values, and outputs information necessary for picking the bag-like workpiece to a robot controller;
From the angle of the normal vector of the three-dimensional shape, the discontinuous portion is regarded as the boundary of the bag-shaped workpiece, and a different label value is assigned to each bag-shaped workpiece, and the position and orientation of the bag-shaped workpiece are recognized for each label value. A bag-like workpiece recognition device characterized by: The selection of the label value of the bag-like workpiece to be picked is selected from the plurality of bag-like workpieces to which labels are assigned, the one with the highest position, the one with the largest area, the one with the most horizontal posture, At least one of the most inclined,
The bag-shaped workpiece recognition apparatus according to claim 1 , wherein the information necessary for picking is a picking position, a barycentric coordinate, or a dimension for each selected bag-shaped workpiece. Measure the three-dimensional shape of multiple bag-like workpieces with smooth surfaces,
A bag-like workpiece recognition method for recognizing the position and posture of a bag-like workpiece picked from the three-dimensional shape and outputting the result to a robot controller,
Apply the mesh using the distance to the adjacent measurement point of the three-dimensional shape,
For each measurement point on the mesh,
Assign different label values to measurement points whose distance to adjacent measurement points is equal to or greater than a predetermined first threshold,
And assigning different label values to measurement points at which the normal vector angle difference between adjacent measurement points is equal to or greater than a predetermined second threshold,
By selecting a label value of the bag-like workpiece to be picked from the plurality of label values, and outputting information necessary for picking the bag-like workpiece to the robot controller,
From the angle of the normal vector of the three-dimensional shape, the discontinuous portion is regarded as the boundary of the bag-shaped workpiece, and a different label value is assigned to each bag-shaped workpiece, and the position and orientation of the bag-shaped workpiece are recognized for each label value. A bag-like workpiece recognition method characterized by:

Images