JP6644846B1 - Work position and orientation recognition device and picking system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position and orientation recognition device capable of recognizing the position and orientation of a randomly stacked work with a small amount of calculation and capable of using a scalar robot with little control information, and a picking system using the same. SOLUTION: Point cloud data P of a work W stacked at random is generated. A radius r of the work W is estimated, and a model M is generated based on the estimated radius r. The generated model M is arranged at the origin O estimated based on the normal vector n, rotated about the z-axis, and the degree of coincidence with the point cloud data P is calculated for each predetermined rotation angle. The rotation angle having the highest degree of coincidence is estimated as the axial direction of the workpiece W. [Selection diagram] Fig. 1

Description

  The present invention relates to a position and orientation recognition device that recognizes the position and orientation of randomly stacked works and a picking system using the same.

  2. Description of the Related Art Conventionally, a picking technique has been known in which a work piled up at random is photographed, a photographed image is analyzed to recognize the position and posture of the work, and a robot hand is controlled based on the recognized position and posture. For example, Patent Literature 1 describes a technique for detecting a position and orientation of an object by aligning a point group generated from a distance image of a detection target with a model point group. Further, Patent Literature 2 discloses a technique for comparing a part of an image of an object with a template image of the object, estimating the position and orientation of the object, and controlling a robot hand based on the estimated information. I have.

JP 2018-088209 A JP-A-2005-079374

  However, according to the techniques of Patent Literatures 1 and 2, there is a problem that a model or a template must be prepared in advance for each work shape to be detected, and the work becomes complicated. Further, since the position and orientation are calculated by precisely collating the workpiece surface and the model surface, there has been a problem that the amount of calculation is enormous.

  Therefore, an object of the present invention is to eliminate the need for preparing a model or the like in advance, suppress the amount of calculation when calculating the position and orientation, and further use a position and orientation recognition device for a work that can be picked by a scalar robot, and to use the device. The purpose of the present invention is to provide a picking system.

In order to solve the above problems, a position and orientation recognition apparatus according to the present invention has the following features.
(1) An apparatus for recognizing the position and orientation of a work based on point cloud data representing a substantially cylindrical work, comprising control means for controlling the apparatus, wherein the control means performs a model based on the size of the work. And an axial direction estimating unit for estimating the axial direction of the workpiece using the model.The axial direction estimating unit rotates the model around a predetermined rotation axis, and The rotation angle having the highest degree of coincidence is estimated as the axial direction.

(2) The axial direction estimating means estimates a normal vector based on the point cloud data, and accumulates the point cloud included in the point cloud data toward a central axis of a substantially cylinder represented by the point cloud data; The rotation axis of the model is selected based on the degree of integration of the point cloud.

(3) The control means includes a size estimating means for estimating the size of the work, and the size estimating means sets a predetermined value as a temporary work diameter, and generates a model corresponding to the temporary work diameter using the model generating means. Then, the axial direction corresponding to the generated model is estimated using the axial direction estimating means, and the temporary work diameter corresponding to the model having the highest degree of coincidence with the point cloud data is estimated as the actual work diameter.

(4) The axial direction estimating means includes means for calculating the average value of the degree of coincidence with the point cloud data for each model, and the size estimating means calculates the temporary work diameter corresponding to the model having the highest average value as the actual work diameter. Is estimated as

(5) means for storing models in descending order of coincidence; control means includes grippability determining means for determining whether or not the workpiece can be gripped based on the model; The model is read out from the means in the order of the degree of coincidence, the possibility of grasping is determined, and a graspable model is output.

The picking system of the present invention has the following features.
(6) The position and orientation recognition device according to any one of (1) to (5), a main hand that grips the workpiece based on the axial direction, and an auxiliary hand that changes the attitude of the workpiece gripped by the main hand. , Is provided.

(7) A camera is provided for detecting the gripping state of the work by the main hand, and the auxiliary hand changes the posture of the work based on the detection result of the camera.

  According to the position and orientation recognition apparatus and the picking system of the present invention, the work to be grasped is limited to a substantially cylindrical shape, and point cloud data is generated from a three-dimensional image of a randomly stacked work, and the point cloud data is generated. Since the size and the position and orientation of the work are calculated based on this, there is an effect that it is not necessary to prepare a model or the like in advance. Further, since a precise collation operation is not required, there is also an excellent effect that the amount of operation can be reduced.

1A is a schematic plan view of a picking system according to an embodiment of the present invention, and FIG. It is a block diagram of a picking system. It is a schematic diagram which shows the process which estimates an axis candidate point from the normal vector of point cloud data. (A) provisional radius r _i: a _{(r min ≦ r i ≦ r} max i = 0~100) models generated from, in schematic view showing a state of rotating the shaft candidate points around z-axis with the origin O is there. (B) is a graph plotting the average value of the degree of coincidence for each tentative radius r _i. It is a flowchart which shows the flow of a process of a picking system. It is an operation explanatory view showing a picking operation. FIG. 3 is a schematic diagram illustrating a process of detecting a grip state of a workpiece from a captured image of a camera. FIG. 4 is an explanatory view (1) illustrating an alignment operation of picked works. FIG. 9 is an operation explanatory view (2) illustrating an alignment operation of picked works.

  Hereinafter, an embodiment in which the present invention is embodied in a picking system will be described with reference to the drawings. In the following description, the coordinate axes of the SCARA robot 3 are defined as an x-axis direction from the back to the front, a y-axis direction from the left to the right in front view, and a z-axis from the vertical bottom to the top. I do.

  As shown in FIGS. 1 and 2, the picking system 1 according to the present embodiment includes a 3D scanner 4 that scans a three-dimensional shape of the surface of a measurement target that is randomly stacked, and points measurement intermediate data received from the 3D scanner 4 to a point. An operation control PC2 that converts the data into group data P and recognizes the position and orientation of the work W based on the point cloud data P; a scalar robot 3 that picks and aligns the work W based on the recognized position and orientation; The camera 3 includes a camera 5 that detects a gripping state of the work W by the robot 3. At this time, it is necessary that the work W included in the measurement target has substantially the same shape and a substantially columnar shape. In this embodiment, the work W is a bolt-shaped work W having a head in a cylindrical body. explain. Note that the camera 5 may be used together with the backlight 6.

  The arithmetic control PC 2 receives a data from an external device such as a 3D scanner 4 and a control unit (CPU) 21 that executes arithmetic control, a storage unit 22 that stores data calculated by the control unit 21, and a control unit 21. And an interface 23 for transmitting control information from the external device to the external device.

  The control unit 21 includes a point cloud data generation unit 24 that converts the measured intermediate data received from the 3D scanner 4 into point cloud data P, a size estimation unit 25 that estimates the size of the work W based on the point cloud data P, A model generating unit 26 that generates a model M using the estimated size as a parameter, an axial direction estimating unit 27 that estimates an axial direction using the generated model M, and a possibility of gripping the workpiece W based on the estimated axial direction. It comprises a gripping possibility determination unit 28 for determining and a gripping state detecting unit 29 for processing a captured image of the camera 5 to detect a gripping state of the work W.

Here, the point group data P refers to an array (x, y, z) of three-dimensional point information. The flow of obtaining the point cloud data P is as follows.
(1) The PC 2 issues a measurement trigger to the 3D scanner 4.
(2) The 3D scanner 4 scans the three-dimensional shape of the surface of the measurement target.
(3) The 3D scanner 4 transmits the measurement intermediate data to the PC 2.
(4) The PC 2 converts the measured intermediate data into the point cloud data P.
In some cases, the 3D scanner 4 performs conversion into the point cloud data P. However, since the generation of the point cloud data P requires a large amount of calculation, it is generally converted into the point cloud data P by the arithmetic control PC 2 using a dedicated library. Examples of the three-dimensional scanning method include a projector method and a stereo camera method.

  The model generation unit 26 receives the radius r as a parameter, and generates a model of the workpiece W composed of a semicircular cylinder having a radius r and a height r.

  The axial direction estimating unit 27 first estimates a normal vector n for each point p included in the point cloud data P, as shown in FIG. Then, a point q obtained by advancing the point p by the radius r in the direction opposite to the normal vector n is set as an axis candidate point of a cylinder including a curved surface of the normal vector n on a side surface, and the axis candidate point q is registered in the storage unit 22. . Note that the point q is obtained by the following equation 1.

  Next, the axial direction estimation unit 27 calculates the degree of accumulation of the axis candidate points q, and specifies a point q with a high degree of accumulation. Then, the origin O of the model M generated in advance is arranged at the point q, and the degree of coincidence with the point group data P is calculated. The degree of coincidence is obtained by dividing “the number of points p arranged near the model M” by “the total number of points p included in a semi-cylinder having a radius r and a height r with the model M as a side surface”. . The axial direction estimating unit 27 rotates the model M from 0 degree to 180 degrees around the z axis, and calculates and holds the degree of coincidence every 5 degrees of the z angle. Then, the z angle having the highest matching degree is registered in the storage unit 22 as the axial direction of the model M.

Size estimator 25, as shown in FIG. 4 (a), assuming a minimum radius r _min and a maximum radius r _max, respectively generate a model M _i to the radius r _i obtained by equation 2 below, implementing the axial direction estimation process for each model M _i. Generating and axial estimation process of the model M _i is omitted because it is same as the processing of the model generating unit 26 and the axial direction estimation unit 27.

Thereafter, the size estimation unit 25, for each radius r _i, the average value of the matching level calculated for each z angle 5 degrees, and outputs a high radius r _i of the most average value as the radius r of the workpiece W. FIG. 4B shows a graph in which the average value of the coincidences is plotted. Here, r _min = 2.0 mm and r _max = 8.0 mm, and the average value of the coincidence is plotted every time the radius r is increased by 0.06 mm. The average value of the degree of coincidence largest r _i, it can be estimated that the actual radius r of the workpiece W.

  The storage unit 22 stores the models M in descending order of the degree of matching. More specifically, since the point cloud data P includes the surface shapes of a plurality of works W, the storage unit 22 calculates a model M oriented in the same axial direction as each of the works W, and stores the models M in descending order of coincidence. M is stored.

The grip possibility determination unit 28 reads the models M from the storage unit 22 in descending order of matching degree, and determines whether the main hand 31 can be gripped. The grip possibility is determined to be impossible when any one of the following conditions (1) to (3) is satisfied.
(1) The gripping position by the main hand 31 is out of the movable range of the SCARA robot 3.
(2) The main hand 31 and the pallet 12 interfere.
(3) The main hand 31 interferes with a workpiece W different from the workpiece W to be gripped.
If gripping is impossible, the model M having the next highest matching degree is read from the storage unit 22. Then, the control unit 21 transmits control information corresponding to the model M determined to be grippable to the SCARA robot 3. As the control information, for example, control parameters such as the coordinates (x _o , y _o , z _o ) of the origin O corresponding to the model M and the radius r can be suitably adopted.

  The SCARA robot 3 includes a main hand 31 for gripping the workpiece W, and an auxiliary hand 34 for changing the posture of the workpiece W based on a detection result of the gripping state detection unit 29. The SCARA robot 3 rotates around the z-axis, and moves the main hand 31 from the initial position to the alignment tray 11 position via the auxiliary hand 34.

  The main hand 31 is attached to the flange surface of the SCARA robot 3 and includes an electric gripper 32 and a gripper 33 for gripping the work W. The electric gripper 32 has a mechanism for moving the gripping claws 33 in parallel to open and close. Since the gripping claw 33 can be held at an arbitrary position, it can correspond to various workpiece W diameters.

  The auxiliary hand 34 is fixed to the housing 10 that houses the picking system 1, and holds the holding claw 36 for holding the work W, the air chuck 35 for opening and closing the holding claw 36, and the holding claw 36 and the air chuck 35. A rotation stage 37 is provided. The rotation stage 37 can rotate in the range of -180 to 180 degrees, and can be positioned at any angle. The air chuck 35 has a mechanism that holds the holding claw 36 in the open position in an initial state and closes the holding claw 36 by applying positive pressure air.

Next, a processing flow of the picking system 1 having the above configuration will be described with reference to FIG. First, when the work W is loaded on the pallet 12 (S1), the 3D scanner 4 scans the work W and transmits the measurement intermediate data to the PC 2. The PC 2 that has received the measurement intermediate data generates the point cloud data P (S2), and estimates the size assuming the radius between r _min and r _max (S3). Then, a model is generated using the real radius r (S4), and the axial direction is estimated by applying the model M to the point cloud data P (S5). Here, the recognition of the position and orientation of the work W mainly includes generation of a model using the real radius r and estimation of the axial direction. The processing of the main part is performed for all the works W included in the point cloud data P, and the storage unit 22 sorts and stores the models M in descending order of the degree of coincidence.

  When the recognition of the position and orientation is completed, the control unit 21 reads the model M having the highest matching degree from the storage unit 22 and sets the model M as a candidate to be grasped (S6: Yes). If the candidate to be grasped can be grasped by the main hand 31 (S7: Yes), a control signal for the picking operation and subsequently the alignment operation is transmitted to the SCARA robot 3 (S8, S9). On the other hand, if gripping is not possible (S7: No), the model M with the next highest degree of coincidence is read from the storage unit 22 and gripping possibility is determined (S10).

Next, the operation of the SCARA robot 3 will be described with reference to FIGS. Here, the initial position of the main hand 31 is coordinates (x ₀ , y ₀ , z ₀ ), the gripping position of the work W received from the PC 2 is the coordinates (x ₁ , y ₁ , z ₁ ), and the predetermined play is α or It will be described as β. Regarding the subscripts added to the symbols (a) and the like in the drawings, “1” indicates that the picking system 1 is viewed from above, and “2” indicates that the picking system 1 is viewed from the side. .

First, the picking operation by the main hand 31 will be described with reference to FIG. The main hand 31 receives the picking control information from the PC 2 and operates the toe (the tip of the gripping claw 33) as shown in FIGS. The details of the operation are described below.
(A) The toe is held at the standby position (x ₀ , y ₀ , z ₀ ).
(B) The toe is moved to coordinates (x ₁ , y ₁ , z ₀ ) just above the workpiece W.
(C) The gripper 33 is rotated so as to be parallel to the central axis of the work W.
(D) The electric gripper 32 is operated to open the toe to “the diameter of the work W + α”, that is, “radius r × 2 + α”.
(E) moving the toe to the coordinates _{(x 1, y 1, z} 1).
(F) Operate the electric gripper 32 to close the toe to “diameter−β”, that is, “radius r × 2−β”.
(G) Move the toe to the coordinates (x ₁ , y ₁ , z ₀ ). That is, it is moved vertically in the z direction, and only the z coordinate is returned to the standby position z ₀ .
Thereafter, the toe is returned to the standby position (x ₀ , y ₀ , z ₀ ). At this time, the work W is not always held horizontally.

Next, the alignment operation by the main hand 31 and the auxiliary hand 34 will be described with reference to FIGS. When the picking operation is completed, as shown in FIG. 7, the main hand 31 holding the workpiece W with the camera 5 is photographed, and the holding state is detected. FIGS. 7A to 7C show the following states.
FIG. 3A is a diagram illustrating a positional relationship among a main hand 31, a camera 5, and a backlight 6;
FIG. 3B is a view of the main hand 31 viewed from the camera 5.
(C) The camera 5 captures an image of the gripping state and outputs a captured image 51. The output captured image 51 is transmitted to the PC 2 and processed by the grip state detection unit 29.
FIGS. 7D to 7H show the following processing in the grip state detection unit 29.
(D) The captured image 51 is binarized.
(E) Mask the image of the gripping nail 33 of the main hand 31 taken in advance and extract only the image of the work W.
(F) Principal component analysis is performed on the extracted image of the work W, and the inclination of the work W is detected.
(G) The left end and the right end of the work W are detected along the detected inclination of the work W.
(H) of the left and right edges, the thicker was the work of the head end W _h, to the opposite side of the tail end W _t.

After detecting the grip state, the work W is transferred from the main hand 31 to the auxiliary hand 34 and from the auxiliary hand 34 to the main hand 31 to change the posture of the work W, as shown in FIGS. 8 (a) to 8 (f) and FIGS. 9 (g) to 9 (l) show the following operations.
(A) In the initial state, the holding claws 36 of the auxiliary hand 34 are open.
(B) Rotate the angle of the rotary stage 37 of the auxiliary hand 34 according to the inclination of the work W.
(C) moving the main hand 31, the tail end W _t of the workpiece W, carry to near the toe of the auxiliary hand 34 (tip of Kyojitsume 36).
(D) further moving the main hand 31, the tail end W _t of the workpiece W, carry between claw unit 36 of the auxiliary hand 34.
(E) Close the holding claws 36 of the auxiliary hand 34 in accordance with the work W. (f) Open the grip claws 33 of the main hand 31 to release the work W.
(G) The main hand 31 is moved in the z direction.
(H) Return the angle of the rotary stage 37 of the auxiliary hand 34 to 0 degrees.
(I) Lower the main hand 31 to the position where the workpiece W is gripped.
(J) The main hand 31 is closed to grip the work W, and the auxiliary hand 34 is opened.
(K) The main hand 31 is moved sideways and the work W is pulled out from the auxiliary hand 34.
(L) The SCARA robot 3 rotates around the z-axis, and the main hand 31 places the work W on the alignment tray.

  According to the picking system 1 having the above-described configuration, the work to be grasped is limited to a substantially cylindrical shape, the point cloud data P is generated from the measured intermediate data of the randomly stacked works W, and based on the point cloud data P Since the position and orientation of the work W are calculated, there is no need to prepare a model or the like in advance, and there is an effect that the work process can be omitted. In addition, there is an excellent effect that the amount of calculation can be reduced because a precise collation calculation is not required when the model M is applied. Further, according to the picking system of the present invention, since the posture of the work held by the main hand is corrected by the auxiliary hand, there is an effect that a SCARA robot with less control information can be adopted.

  In addition, the present invention is not limited to the above-described embodiment, and can be implemented by arbitrarily changing the configuration of each unit without departing from the spirit of the invention.

1 Picking system 2 PC for operation control (Position and orientation recognition device)
Reference Signs List 3 SCARA robot 4 3D scanner 5 Camera 21 Control unit 22 Storage unit 23 Interface 24 Point group data generation unit 25 Size estimation unit 26 Model generation unit 27 Axial direction estimation unit 28 Gripping possibility determination unit 29 Gripping state detection unit 31 Main hand 32 Electric gripper 33 Gripping claw 34 Auxiliary hand 35 Air chuck 36 Nipping claw 37 Rotation stage O Origin P Point group data M Model W Work q-axis candidate point p Each point included in point group data r Radius n Normal vector

Claims (6)

An apparatus for recognizing a position and a posture of a work based on point cloud data representing a substantially cylindrical work,
Comprising control means for controlling the device,
The control unit includes a model generation unit that generates a model based on the size of the work, and an axial direction estimation unit that estimates an axial direction of the work using the model,
The axial direction estimating means estimates a normal vector based on the point cloud data, and accumulates a point cloud included in the point cloud data toward a central axis of a substantially cylinder represented by the point cloud data. Selecting a rotation axis of the model based on the degree of integration of the point cloud,
The position and orientation recognition apparatus, wherein the axis direction estimating means rotates the model around a predetermined rotation axis and estimates a rotation angle having the highest degree of coincidence with point cloud data as an axis direction. The control means includes a size estimating means for estimating the size of the work,
The size estimating means sets a predetermined value as a temporary work diameter, generates a model corresponding to the temporary work diameter using the model generating means, and calculates an axial direction corresponding to the generated model by the axial direction estimating means. The position and orientation recognition apparatus according to claim 1 , wherein the temporary work diameter corresponding to the model having the highest degree of coincidence with the point cloud data is estimated as the actual work diameter. The axial direction estimating means includes means for calculating an average value of the degree of coincidence with the point cloud data for each model,
The position and orientation recognition apparatus according to claim 2 , wherein the size estimating unit estimates a temporary work diameter corresponding to the model having the highest average value as an actual work diameter. Means for storing models in order of the degree of coincidence,
The control unit includes a grippability determining unit that determines whether the workpiece can be gripped based on the model,
The position according to any one of claims 1 to 3, wherein the grippability determining unit reads the models from the storage unit in descending order of the degree of coincidence, determines grippability, and outputs a grippable model. Attitude recognition device. The position and orientation recognition device according to claim 1, a main hand that grips a workpiece based on the axial direction, and an auxiliary hand that changes a posture of the workpiece gripped by the main hand. A picking system comprising: A camera that detects a gripping state of the work by the main hand,
The picking system according to claim 5 , wherein the auxiliary hand changes a posture of the work based on a detection result of the camera.

Images