JPH06241719A - Article position detection device - Google Patents

Abstract

PURPOSE:To obtain an article position detection device which can precisely detect the displacement of an article is even when articles are disposed in series. CONSTITUTION:In an articles position detection device are provided an image taking means 13 for taking an image of an article E and a detection means for detecting the displacement of the article from its setting base state on the basis of the information of a taken image by the image taking means 13. The detection means thus detects the position and orientation of at least one corner of the article E in image taking on the basis of the information about a taken image by the image taking means 13. Accordingly, the displacement of the article E is detected on the basis of the information about the detected position and direction of the corner and about the known size of the article.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is provided with an image pickup means for picking up an image of an article and a detection means for detecting a displacement of the article with respect to a set reference state based on imaged image information of the image pickup means. The present invention relates to a position detection device.

[0002]

2. Description of the Related Art The above-mentioned article position detecting apparatus is used, for example, in a transfer apparatus for transferring stacked articles. In this case, in order to transfer the article accurately, it is necessary to accurately measure the displacement of the article with respect to the set reference state. Here, the set reference state is the reference point (for example, the center point) of the article.
Indicates a state in which the item is aligned with a preset reference position and the inclination of the article is aligned with a preset reference slope. Therefore, the displacement includes the displacement of the reference point and the displacement of the inclination. Conventionally, in measuring the displacement of an article, the articles are arranged with a predetermined gap provided between the articles, and the target article whose displacement is to be measured is imaged by an imaging means such as a camera. That is, by arranging the target article and the adjacent article so as to be separated from each other, the image processing of only the target article can be extracted. For example, Japanese Patent Laid-Open No. 2-2
In Japanese Patent Publication No. 69588, a tape or the like having a color having a contrast with the color of the article is attached to the article, and the reflected light from the tape or the like is imaged to measure the displacement of the article. In order to extract only the reflected light of the tape pasted on the target article, the article adjacent to the target article is arranged separately. In addition, Japanese Patent Application No. 4-
In 213866, the target article is irradiated with slit light and the slit light is imaged, and the displacement of the article is measured. However, even in this case, the target article is detected in order to detect only the slit light. And the article adjacent thereto are separated from each other.

[0003]

However, in the above-mentioned conventional example, the displacement of the article cannot be detected unless the target article and the adjacent article are arranged apart from each other. Alternatively, when the articles are connected due to a collapse of the load, there is a problem that the displacement of the articles cannot be detected. The present invention has been made in view of the above circumstances, and an object thereof is to provide an article position detection device capable of accurately detecting displacement of articles even when articles are arranged in a linked manner. Especially.

[0004]

SUMMARY OF THE INVENTION An article position detecting apparatus of the present invention is an image pickup means for picking up an image of an article, and a detecting means for detecting a displacement of the article with respect to a set reference state based on image pickup image information of the image pickup means. In the first characteristic configuration, the detecting means detects the position and orientation of at least one corner of the article in the captured image based on the captured image information of the imaging means. However, the displacement of the article is detected based on the detected position and orientation of the corner and the known size information of the article. According to a second characteristic configuration, a light projecting unit that projects a slit light having a length protruding from the article is provided on an image pickup surface of the image pickup unit of the article,
The detection means is configured to extract a region corresponding to the slit light on the article based on the imaged image information of the imager and detect the position and orientation of the corner of the article on the captured image. There is a point. According to a third characteristic configuration, image information obtained by performing edge enhancement processing on the captured image information when the light projecting means is in the light projecting state by the detecting means, and when the light projecting means is in the non-light projecting state. The captured image information is subtracted from the image information that has been subjected to the edge enhancement processing, and the subtracted image information is set to a threshold value of 2
The point is that the region corresponding to the slit light on the article is extracted from the binarized image information after the binarization process.

[0005]

According to the first characteristic configuration of the present invention, for example, in a transfer device for transferring articles stacked on a pallet, when the arrangement pattern of the articles and the size of the articles are input, first, the stacking is performed. The article located at the end is selected from the articles in the uppermost row of the selected articles, and an image pickup means such as a camera is moved onto the selected article to perform image pickup. Then, based on the captured image information, for example, at least two sides on the side where there is no connected article due to a change in luminance on the image are detected (edge detection), and the connected article is based on the position of the detected side. Detect the position and orientation of at least one corner on the non-existent side. Then, the displacement of the article can be detected based on the detected position and orientation of the corner and the actual size of the article. Therefore, based on the detected displacement, the transfer device can be operated to transfer the article. According to the second characteristic configuration, when the slit light is projected by the light projecting unit and the image thereof is captured by the image capturing unit, the positions of the light projecting unit and the image capturing unit are in the x-axis direction and the y-axis direction of the captured image. Since there is a deviation, the slit light has a lateral deviation on the outside of the article on the side where there is no connected article. Therefore,
Based on the position of the lateral deviation of the slit light, it is possible to extract the area corresponding to the slit light on the article and detect the position and orientation of at least one corner of the article on the captured image. According to the third characteristic configuration, even if there is a pattern such as characters on the article, the image information obtained by performing the edge enhancement processing on the captured image information in the light projecting state and the image information in the non-light projecting state The pattern on the article and the article outline are erased by subtracting the captured image information from the image information on which the edge enhancement processing has been performed, and binarizing the subtracted image information with a set threshold value. It is possible to extract an image of only slit light. Then, based on the image information of the extracted slit light, the position of the lateral deviation of the slit light is detected, and the area corresponding to the slit light on the article is further extracted, so that at least one of the articles on the captured image is detected. The position and orientation of the corner can be detected.

[0006]

According to the first characteristic configuration, at least one corner on the side where there is no connected article is detected, and the displacement of the article is determined based on the position and orientation of the corner and the actual size of the article. Since it is possible to detect, it is possible to provide an article position detecting device that can detect the displacement of the article even when the articles are arranged in a linked manner. According to the second characteristic configuration, based on the position of the lateral deviation of the slit light,
Since the position and orientation of at least one corner of the article on the captured image can be detected, it is therefore possible to accurately detect the displacement of the article using the slit light even when the articles are connected and present. I was able to do it. According to the third characteristic configuration, it is possible to delete the pattern of characters or the like on the article and detect the displacement of the article. Therefore, even when there is a pattern on the article, the displacement of the article can be accurately detected. I was able to do it.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the article position detecting apparatus of the present invention is applied to a transfer apparatus for transferring stacked articles will be described with reference to the drawings. As shown in FIG. 10, the articles E stacked on the pallet 1 at a predetermined position are transferred to the conveyor 2
A transfer device for transferring on top is provided. A brief description of the transfer device will be given. The turning frame 4 is pivotally supported about the vertical axis α with respect to the base frame 3.
A first arm 5 pivotally supported on an upper portion of the revolving frame 4 about a horizontal axis φ1, and a tip end portion of the first arm 5 pivotally supported about a horizontal axis φ2. The second arm 6 is provided, and the article supporter 7 is rotatably suspended around the longitudinal axis β of the tip end portion of the second arm 6. Reference numeral 8 in the drawing is a controller for controlling the operation of the transfer device. The article support 7 is configured to be maintained horizontally regardless of the swing of the first arm 5 and the second arm 6. That is, it is possible to move up and down and move horizontally while keeping the article support 7 horizontal.

As shown in FIG. 9, the article support 7 includes a plurality of suction pads 10 for suctioning and supporting the upper surface of the article E, and a holding portion 11 for holding the article E sucked and supported by the suction pad 10. A light emitting unit 12 as a light emitting unit that emits light from above the article E, an image capturing unit 13 as an image capturing unit that captures an image of the article E, and a distance sensor 1 that measures the distance between the article E.
4 and are attached.

The arrangement pattern of the articles E placed on the pallet 1 changes depending on the size of the articles E and the like, but a pattern code is defined for each pattern. When the pattern code and the width w and the depth d of the article E are input to the controller 8, the controller 8 transfers the articles E on the conveyor 2 in a predetermined order from the article E at the uppermost end.

As shown in FIG. 1, the light projecting section 12 is provided with an article E.
It is composed of three lasers that irradiate the surface to be imaged of the imaging unit 13 in 3 with three linear slit lights having a length protruding from the article E. Then, each of these three slit lights is configured to be freely switchable between a light projecting state and a non-light projecting state. The surface to be imaged of the imaging unit 13 of the article E is assumed to be rectangular. In addition, the slit lights L1 and L3 are y in the image of the imaging unit 13.
The light projecting unit 12 and the imaging unit 13 are provided so as to match the axial direction and the slit light L2 to match the x-axis direction of the image.

The image information of the image pickup section 13 is stored in the controller 8
Entered in. The controller 8 extracts the region corresponding to the slit light on the captured image based on the captured image information of the image capturing unit 13, and selects the article E (E1, E1, among the regions corresponding to the slit light on the extracted captured image. E2, E3, E
4) Detect the upper slit light area. Then, in the area of the detected slit light, the position coordinates of the end point on the side of the article E1 where the slit light is not connected are found, and based on the position coordinates and the known size of the article E, the article The position coordinates of the four corners of E are detected to detect the displacement of the article E relative to the set reference state. Therefore, the controller 8 functions as a detection unit that detects the displacement of the article E with respect to the set reference state.

First, an extraction method will be described in which the controller 8 extracts only the area corresponding to the slit light on the captured image based on the captured image information of the image capturing section 13. As shown in the flowchart of FIG. 7, the imaging unit 13 first captures an image when the light projecting unit 12 is in the non-light projecting state. When the edge enhancement process is performed on this image, an image Z0 shown in FIG. 2 is obtained. In this image Z0, as shown in the figure,
Images of the articles E (E1, E2, E3, E4) on the uppermost stage are captured. Next, the image capturing unit 13 captures an image when the light projecting unit 12 is in a light projecting state in which the slit lights L1, L2, L3 are projected. When the edge enhancement processing is performed on this image information, an image Z1 shown in FIG. 3 is obtained. And FIG.
Image Z1 of FIG. 2 and the image Z0 of FIG. 2 are subjected to subtraction processing, and the subtracted image information is subjected to binarization processing with a set threshold value, resulting in slit light on the image image as shown in FIG. Only the corresponding areas are extracted. That is, this subtraction processing can erase the outer shape of the article E and the characters (or pattern) on the article E on the image. The image capturing by the image capturing unit 13 and the light projecting by the light projecting unit 12 are performed in a state where the distance to the article E detected by the distance sensor 14 shows a predetermined value.

The above edge enhancement processing will be described with reference to the flowchart of FIG. In the edge enhancement processing, first, the captured image information of the image capturing unit 13 is subjected to Kirsch edge enhancement, noise is removed by a median filter, a histogram is created, and a threshold value is determined based on this histogram. , The binarization processing is performed with the threshold value. After that, noise removal is performed again, and thinning is performed.

Then, the controller 8 extracts the slit light area on the article E (E1, E2, E3, E4) from the area corresponding to the slit light on the picked-up image extracted as described above. Will be described. Image binarized after subtraction processing In FIG.
Since the distances between the imaging means 13 and the light projecting means 12 are always constant, the positions of the slit lights L1, L2, L3 on the article E are known. On the other hand, the slit light outside the article E (E1, E2, E3, E4) causes a lateral shift with respect to the slit light on the article E. This is because the light projecting means 12 is provided with a predetermined displacement in the x-axis direction and the y-axis direction of the image with respect to the image pickup means 13. Therefore, if the slit light at the known position is extracted as described above, T1, T2, T of the slit light in FIG.
The area excluding 3 is the article E (E1, E2, E3, E4)
It is extracted as the upper slit light. In addition, as shown in FIG. 4, the extracted articles E (E1, E2, E3, E4)
A large number of gaps in the upper slit light are generated in the image subtraction processing, and indicate the intersections of the slit light and the characters on the article E or the intersections of the slit light and the outer shape of the article E.

Next, the controller 8 detects the end point of the slit light on the article E (E1, E2, E3, E4) extracted as described above on the side without the article connected to the slit light on the article E1. The detection method to be performed will be described. When the end points at the maximum and minimum positions in each slit light are extracted from the end points of the three slit lights (marked with ● in the figure), the end points P1, P2, P5, R1, R2, R3 (marked with ● in the figure) are extracted. □
A total of 6 points (marked) are extracted. When the end points 3 near the positions of the intersections Q1 and Q2 of the slit lights are extracted from the extracted six points, the end points P1, P2, and P5 (circle marks in the figure)
Is detected. That is, the end points P1, P2, P5 are
It is the end point on the side of the article E1 where the slit light is not connected to the article.

Next, the controller 8 causes the end points P1,
A detection method for detecting slit light on the article E1 based on the position coordinates of P2 and P5 and the size of the article E1 will be described. The coordinates of the end points P1 and P5 are (X1, Y1),
Assuming that (X5, Y5), the side passing through the end points P1 and P5 is represented by the straight line shown in the following expression 1.

## EQU1 ## y = a (x-X1) + Y1 However, a = (Y5-Y1) / (X5-X1).
The side passing through the end point P (X2, Y2) is represented by the straight line shown in the following Expression 2.

[Mathematical formula-see original document] y = a (1 / a) * (X2-x) + Y2 [mathematical formula-see original document] The size of equation 1 and equation 2 and the size of the article E (width w, depth d)
Therefore, the other two sides are represented by the straight lines shown in the following Expression 3.

Y = a (x−X1) + w ′ y = (1 / a) × (X2−x−d ′) + Y2 where w ′ = w / (cos (a)), d ′ = d / (Co
s (a)). When the end points on the two straight lines of the above equation 3 are selected from the end points (marked by ● in the figure) in FIG. 4, the end points P3, P4, P6 (marked by Δ in the figure) are detected. Therefore,
The slit light in the area surrounded by the end points P1 to P6 becomes the slit light on the article E1.

Next, the controller 8 determines the position coordinates of the end points P1 to P6 of the slit light on the article E1 as shown in FIG.
As shown in, the corners A, B, and B of the article E on the captured image
A calculation method for calculating the position coordinates of C and D will be described. The position coordinates of the end points P1 to P6 of the slit light area on the article E are (X1, Y1), (X2, Y2), (X
3, Y3), (X4, Y4), (X5, Y5), (X
6, Y6), the inclination a of the article E is calculated by the following formula 4.

## EQU00004 ## a = (Y5-Y1) / (X5-X1) = (Y6-Y3) / (X6-X3) Then, the equations showing the straight lines AB, DC, BC, and AD are expressed by the following equation 5. Set to.

## EQU5 ## AB: y = (x-X1) + Y1 DC: y = (x-X3) + Y3 BC: y = (X2-x) / a + Y2 AD: y = (X4-x) / a + Y4

Therefore, the position coordinates of the corners A, B, C, D of the article E on the picked-up image are obtained by obtaining the intersections of the straight lines from the equation of the above equation 5, as shown in the following equation 6.

[6] angle A = ((aY4-aY1 + a 2 X1 + X4) / (a 2 +1), (a 2 Y4 + Y1 + aX4-aX1) / (a 2 +1)) angle B = ((aY2-aY1 + a 2 X1 + X2) / (a ^{2 +1), (a 2 Y2} + Y1 + aX2-aX1) / (a 2 +1)) angle C = ((aY2-aY3 + a 2 X3 + X2) / (a 2 +1), (a 2 Y2 + Y3 + aX2-aX3) / (a 2 +1)) angle D = ((aY4-aY3 + a 2 X3 + X4) / (a 2 +1), (a 2 Y4 + Y3 + aX4-aX3) / (a 2 +1))

Further, the controller 8 has the corners A, B, C,
The coordinates of the center point of the article E on the captured image are calculated based on the position coordinates of D, and the positional deviations Δx and Δy of the coordinates of the center point with respect to the center point of the screen in the x-axis direction and the y-axis direction.
To calculate. That is, the displacement of the article E with respect to the set reference state is obtained. The equations for calculating the positional deviations Δx and Δy at this time are omitted.

Next, the operation of the controller 8 will be described with reference to the flowchart of FIG. First, the article E to be transferred is determined based on the input pattern code,
The article support 7 is moved to a predetermined position corresponding to the article E. The position of the article support 7 is corrected so that the distance to the article E detected by the distance sensor 14 becomes constant, the light projecting unit 12 and the image capturing unit 13 are operated, and based on the information from the image capturing unit 13. Then, the displacement of the article E is obtained as described above. next,
The position of the article support 7 is corrected according to the obtained displacement. Then, the article support tool 7 is lowered, and the suction pad 10
The holding section 11 is brought into a holding state to hold the article E. The article support 7 is moved to the conveyor 2 while holding the article E. The article support 7 is lowered to bring the suction pad 10 and the holding portion 11 into the holding release state. Therefore, one transfer operation is completed. The above operation is repeated until the transfer is completed.

[Other Embodiments] In the above embodiment, the Kirsch is used for edge enhancement in the edge enhancement processing, but other methods such as Sobel, Prewitt, and Roberts may be used. In the above-described embodiment, light is projected and imaged from above the article E to detect the displacement of the article E viewed from above, but it is necessary to detect the displacement of the article E viewed from the side. In some cases, the side surface of the article E may be projected and imaged to detect the displacement of the article E viewed from the side surface. In the above embodiment, the end point P of the slit light on the article E1 is
Based on the position coordinates of 1, P2, P5, the end points P3, P4,
When P6 is detected and the position coordinates of the corners A, B, C and D of the article E1 are detected based on the position coordinates of the end points P1 to P6, but the end points P3, P4 and P6 cannot be detected. In addition, the position and direction of the corner B are detected based on the position coordinates of the end points P1, P2 and P5, and the remaining corners A, C and D are detected based on the position and direction of the corner B. good. That is, the position coordinates of the angle B are calculated by obtaining the intersection of the straight line of the formula 1 and the straight line of the formula 2 in the above-described embodiment, and the angle A and the number on the straight line (straight line AB) of the formula 1 are calculated from the position of the angle B. The angle C on the second straight line (straight line BC) is obtained from the width w and the depth d of the article E1. Then, the intersection point of a straight line (straight line AD) that passes through the angle A and is perpendicular to the straight line of the number 1 and a straight line (straight line CD) that passes through the angle C and is perpendicular to the straight line of the number 2 is detected, and the angle D is detected. You may do it. In this way, even if the end points P3, P4, P6 of the slit light on the article E1 cannot be detected, the end points P1, P2, P5 on the side where there is no article connected to the slit light on the article E1 are detected. If it can be detected, the displacement of the article E can be detected. In this method, when the inclination of the article E is small, that is, among the end points P1, P2, P5 of the slit light, P1 and P2 are located on the long side of the article E and P5 is located on the short side. This is what can be applied in some cases. Thus, for example, the article E
Is greatly inclined, and each of the three points P1, P2, and P5 is located on another side, or the article E is further inclined,
When P1 and P2 are located on the short side and P5 is located on the long side, the displacement of the article cannot be detected. However, in the case where the article E is a square, even if the article E is greatly inclined, if P1 and P2 are on the same side,
The displacement of the article can be detected from the positions of the end points P1, P2, P5. In the above embodiment, the displacement of the article E is detected based on the captured image when the light projecting means is in the light projecting state and the captured image when the light projecting means is in the non-light projecting state. The displacement of the article E may be detected based only on the captured image (FIG. 3) when the light projecting means is in the light projecting state. That is, in the captured image of FIG. 3, only the area (known) where the slit light is located on the article E1 is extracted, and P1, P2, and P5 are detected from the end points of the slit light in the area, and the end point P1 is detected. , P2, P5 may be used to detect the displacement of the article E. Further, the displacement of the article E may be detected based only on the captured image when the light projecting means is in the non-light projecting state. For example, edge enhancement processing is performed on the captured image in the non-illuminated state to detect edges corresponding to the sides AB and BC of the article E1 (edge on the side without the connected article E), and this is detected. The position and direction of the corner B may be detected from the position of the edge, and the position coordinates of the remaining corners A, C, and D may be detected based on the position and direction of the corner B and the size of the article E. . In other words, the side without the connected article E is
Since the extraction is relatively easier than the side of the connected article E, the displacement of the article E should be detected if the side of the connected article E without the connection E can be extracted. You can In the above-described embodiment, the image capturing and projecting unit 12 of the image capturing unit 13 is used.
Is performed in a state where the distance to the article E detected by the distance sensor 14 shows a predetermined value, so that the slit light on the article E comes to a predetermined position on the screen. Although the slit light on the article E is extracted, the distance sensor 14 may be omitted and the slit light on the article E may be extracted as described below. Of the area corresponding to the slit light on the captured image as shown in FIG. 4, a histogram in the x-axis direction and a y-axis direction is created in a predetermined range F as shown in FIG. 11, and the histogram in the x-axis direction is created. Of the slit light on the article E is extracted by extracting a region showing the maximum value (f1) in (1) and a region showing the maximum value (f2) and the next largest value (f3) in the histogram in the y-axis direction. You may make it extract. Thereby, the slit light on the article E in the captured image can be extracted without providing the distance sensor 14.

It should be noted that reference numerals are given in the claims for convenience of comparison with the drawings, but the present invention is not limited to the configurations of the accompanying drawings by the entry.

[Brief description of drawings]

FIG. 1 is a perspective view of an article position detection device.

FIG. 2 is an explanatory diagram of image processing.

FIG. 3 is an explanatory diagram of image processing.

FIG. 4 is an explanatory diagram of image processing.

FIG. 5 is an explanatory diagram of the principle of displacement detection.

FIG. 6 is a flowchart of control operation.

FIG. 7 is a flowchart of image processing.

FIG. 8 is a flowchart of edge enhancement processing.

FIG. 9 is a side view of a main part of the article support tool.

FIG. 10 is a side view of the transfer device.

FIG. 11 is an explanatory diagram of image processing according to another embodiment.

[Explanation of symbols]

 8 Detection means 12 Projection means 13 Imaging means C Articles L1, L2, L3 Slit light

Claims (3)

[Claims] 1. An image pickup means (13) for picking up an image of an article (E).
And based on the imaged image information of the imaging means (13),
An article position detection device comprising: a detection means (8) for detecting a displacement of the article (E) with respect to a set reference state, wherein the detection means (8) is a captured image of the imaging means (13). Based on the information, the position and orientation of at least one corner of the article (E) in the captured image is detected, and the article based on the detected position and orientation of the corner and the known size information of the article. (E) An article position detection device configured to detect the displacement. 2. Light projection for projecting slit light (L1, L2, L3) having a length protruding from the article (E) onto the imaged surface of the image pickup means (13) of the article (E). Means (12) is provided, and the detection means (8) extracts a region corresponding to the slit light on the article (E) based on the imaged image information of the imaging means (13),
The article position detecting device according to claim 1, wherein the article position detecting apparatus is configured to detect a position and a direction of a corner of the article (E) on a captured image. 3. The detecting means (8) includes: image information obtained by performing edge enhancement processing on captured image information when the light projecting means (12) is in a light projecting state; and the light projecting means (12). The captured image information in the non-illuminated state is subtracted from the image information subjected to the edge enhancement process, and the subtracted image information is binarized by a set threshold value, and the binarized 3. The article position detecting device according to claim 2, wherein the area corresponding to the slit light on the article (E) is extracted from the obtained image information.