JP2015042965A - Article position recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an article position recognition device capable of appropriately recognizing the position of an article in a vessel in a picked-up image without being affected by a reflected virtual image of an article reflected on the inside surface of the vessel.SOLUTION: The article position recognition device includes: an imaging unit for imaging, from the upside, a vessel C having a flat inside surface Cn and an opening top face; and an article position recognition unit for executing an article position recognition processing of recognizing the position of an article B accommodated in the vessel C on the basis of a picked-up image picked up by the imaging unit. The article position recognition unit extracts, from the picked-up image, an article existence region in which actual article B can exist, and executes the article position recognition processing on the extracted region.

Description

  The present invention recognizes the position of an article accommodated in the container based on an imaging unit that images from above the container whose inner surface is formed flat and the upper surface is opened, and the captured image captured by the imaging unit. The present invention relates to an article position recognition device including an article position recognition unit that executes an article position recognition process.

  Such an article position recognition apparatus is used for the purpose of, for example, recognizing the position of an article to be picked in a container in a picking facility for picking an article contained in the container with a picking robot (for example, a patent) Reference 1).

JP 2010-12567 A

  Although not described in Patent Document 1, for example, when the container is a plastic case or the like having a flat inner surface, in the captured image captured by the imaging unit, the article is reflected on the wall surface and reflected virtual image May occur. When such a reflected virtual image is generated, the conventional article position recognition apparatus misidentifies the reflected virtual image as an actual article image (real image), and recognizes a position shifted from the actual article position as the article position. There was a risk that this would happen. When such a situation occurs, for example, in the picking facility as in Patent Document 1, there is a possibility that the article support portion of the picking robot interferes with the inner side surface of the container and the article cannot be picked properly.

  Therefore, it is desired to realize an article position recognition device that can appropriately recognize the position of the article in the container without being affected by the reflected virtual image of the article reflected on the inner surface of the container in the captured image.

An article position recognition apparatus according to the present invention for solving the above-described problems is based on an imaging unit that images a container having an inner surface formed flat and an open upper surface from above, and a captured image captured by the imaging unit. An article position recognition unit that executes an article position recognition process for recognizing the position of the article contained in the container,
The article position recognition unit is characterized in that an article existence area where an actual article can exist is extracted from the captured image, and the article position recognition process is executed on the extracted area.

According to this configuration, since an article presence area where an actual article can exist is extracted from the captured image captured by the imaging unit, and the article position recognition process is performed on the area, the container in the captured image is displayed. Even if a reflected virtual image is generated on the inner surface of the article, the article position recognition process can be executed in a state where the reflected virtual image is excluded from the captured image.
For this reason, the position of the article in the container can be appropriately recognized without being affected by the reflected virtual image of the article reflected on the inner side surface of the container in the captured image.

  In an embodiment of the article position recognition apparatus according to the present invention, an upper surface height detection unit that detects the height of the upper surface of the article located at the highest position among the upper surfaces of the articles in the container is provided, and the article position The recognition unit sets a virtual frame line that is an intersection line between a plane including the upper surface of the article detected by the upper surface height detection unit and the inner surface of the container in the captured image, and the virtual frame in the captured image It is preferable to extract a region inside the line as the article presence region.

For example, in the above-described picking equipment, it is preferable that picking is performed sequentially from the article located at the highest position among the articles in the container in order to avoid interference between the picking-up action part of the picking robot and the article.
According to the present embodiment, in the captured image, the inner side of the virtual frame line that is the intersection of the plane including the upper surface of the article located at the highest position among the upper surfaces of the articles in the container and the inner side surface of the container. Therefore, the article existing area includes all the articles existing in the container including the article located at the highest position among the articles in the container.
As described above, according to the present embodiment, the region where the article located at the highest position among the articles in the container is present can be appropriately recognized.

Whole perspective view of picking equipment provided with article position recognition device Control block diagram of picking equipment The figure which shows the captured image of the container in which the articles | goods were accommodated The figure which shows the histogram used in order that an upper surface height detection part may detect height. Flow chart of control executed by article position recognition unit The figure which shows the virtual frame line corresponding to the upper surface height in each step | level in case articles | goods are accommodated in multiple steps | paragraphs The figure which shows the extraction result of the article existence area The figure which shows the captured image of the container in which the article | item is not accommodated. The figure which shows the case where the boundary of the bottom upper surface and inner surface of a container is made into a virtual frame line as a comparative example

An embodiment when the article position recognition apparatus of the present invention is applied to a picking facility will be described with reference to the drawings.
FIG. 1 is a perspective view of a picking facility including a picking robot P. FIG. The picking robot P is installed on the side of the transport path of the transport conveyor 22C. The conveyor 22C conveys the container C delivered from an automatic warehouse (not shown) in accordance with a command from the upper management device Hu (see FIG. 2). The container C has an open top surface and is configured so that an article B accommodated in the container C can be taken out from above. The picking robot P is configured to take out the article B from the container C located at a predetermined stop position (picking stop position) in the transport path of the transport conveyor 22C by using a suction support unit 40 described later. In the present embodiment, the container C corresponds to a container. Inside the container C, a plurality of articles of the same type are accommodated.

  The picking robot P includes a pedestal 30 fixed on the floor surface of the picking area, a support cylinder 31 that can move up and down and rotate around a vertical axis, and an upper end of the support cylinder 31 along a horizontal plane. A first arm 32 pivotally supported by the horizontal shaft 36 and swingable with the horizontal shaft 36 as a swing shaft, and a horizontal shaft 37 provided at the end of the first arm 32 opposite to the horizontal shaft 36 are swung. A second arm 33 that can swing as a moving shaft, and a rotary shaft member 34 that can swing using a horizontal shaft 38 provided at the end opposite to the horizontal shaft 37 of the second arm 33 as a swinging shaft. And a suction support portion 40 provided with a plurality of suction portions that perform a suction action on the upper surface portion of the article B.

  An imaging unit 50 is provided above the picking stop position on the conveyor 22C. The imaging unit 50 is fixedly installed at a position where the inside of the container C placed on the transport surface of the transport conveyor 22C and positioned at the picking stop position can be imaged from above. The imaging unit 50 includes a stereo camera including two imaging lenses, and is configured to be able to capture two images from different positions in the horizontal direction at the same time.

As shown in FIG. 2, the control unit H is configured by, for example, a computer including an arithmetic device and a storage device, and executes various controls by executing programs stored in the storage device. It has become.
An imaging unit 50, a picking controller Hp, and a host management device Hu are connected to the control unit H so as to be able to communicate with the control unit H.
The imaging unit 50 is configured to capture an image according to a command from the control unit H and to transmit the captured image to the control unit H.
The picking controller Hp is configured to be able to control the movement mechanism of the picking robot P and the operation of the suction support unit 40.

  The host management device Hu manages the entire picking work schedule. Then, when an article B to be picked is designated by the host management device Hu, a controller for an automatic warehouse (not shown) and a controller for the transfer conveyor 22C, etc., unload the container C containing the corresponding article B from the automatic warehouse. Then, the various conveyors in the automatic warehouse and the operation of the conveyor 22C are controlled so as to convey to the picking stop position on the conveyor 22C.

When the container C is transported to the picking stop position of the transport conveyor 22C based on the article take-out command from the host management device Hu, the control unit H instructs the picking controller Hp to take out the pick-up operation command. The article B can be taken out from the container C by the picking robot P when the picking controller Hp operates the picking robot P in accordance with the take-out operation command. When instructing the take-out operation command, the control unit H tells the picking controller Hp where the picked article B is located in the container C. For this reason, the control unit H needs to recognize in which position in the container C the article B to be picked exists.
The article position recognition device is used for recognizing in which position in the container C the article B to be picked exists, and in this embodiment, the article position is detected from the control unit H and the imaging unit 50. A recognition device is configured.

As shown in FIG. 2, the control unit H includes an article position recognition unit H1 and an upper surface height detection unit H2 as program modules.
The article position recognizing unit H1 recognizes the position of the article B accommodated in the container C based on the captured image (the image obtained by capturing the container C from above; see FIG. 3) captured by the imaging unit 50. Perform recognition processing. In the article position recognition process, when the type of the article B to be taken out is instructed from the upper management apparatus Hu, the template image corresponding to the article B is loaded from the storage device and the pattern matching of the template image with the captured image is executed. As a result, the position of the article B in the captured image (for example, when the article B is a rectangular article as viewed from above, the center coordinates of the article B in the coordinate system having a specific corner of the container C as the origin, the article B Is the process of recognizing the size of the upper surface rectangle and the posture of the article B).

The upper surface height detection unit H2 detects the height of the upper surface of the article located at the highest position among the upper surfaces of the article B in the captured image based on the captured image captured by the imaging unit 50. Execute the process.
Specifically, the distance from the imaging unit 50 for each pixel (pixel) is calculated based on the two captured images captured by the imaging unit 50, and is calculated from the height at which the imaging unit 50 is installed. By subtracting the distance, the height for each pixel is calculated, and the correlation between the height and the number of pixels is graphed (see FIG. 4). In this graph, after calculation for noise removal (for example, smoothing differentiation, etc.), the peak portions (height T1, height T2, and height T3 in FIG. 4) are detected. . In FIG. 4, the pixel group positioned at the height T1 is a set of pixels corresponding to the upper edge of the container C in the captured image, and the pixel group positioned at the height T3 is accommodated in the container C in the captured image. This is a set of pixels corresponding to the upper surface of the article B located at a position lower than the article B having the highest upper surface among the articles B to be displayed. The pixel group located at the height T2 is a set of pixels corresponding to the upper surface of the article B having the highest upper surface among the articles B accommodated in the container C in the captured image. The upper surface height detection unit H2 detects the height T2 of the second peak from the highest as the height of the upper surface of the article B located at the highest position among the upper surfaces of the articles B in the container C.
That is, the upper surface height detection unit H2 detects the height T2 of the upper surface of the article B located at the highest position among the upper surfaces of the articles B in the container C.

In the present embodiment, the container C is made of plastic, and its inner side surface Cn is formed flat. In general, since the inner side surface of the plastic container C has high smoothness, as shown in FIG. 3, reflected light or the like is reflected on the article, and a reflected virtual image Bk of the article is easily generated.
When the article position recognition unit H1 recognizes the position of the article while the reflected virtual image Bk is generated, the reflected virtual image Bk is recognized as an actual image (real image) of the article B, and the position deviated from the real image is changed to the article. There is a risk of recognizing the position of B.

Therefore, in the present embodiment, the article position recognition unit H1 is configured to extract an article existence area where the actual article B can exist from the captured image, and execute article position recognition processing on the extracted area. ing.
Hereinafter, the process performed by the control unit H of the article position recognition apparatus will be described based on the flowchart of FIG.

When the control unit H detects that the container C has arrived at the picking stop position by using a limit switch or the like provided in the transport conveyor 22C, the control unit H instructs the imaging unit 50 to capture the image of the container C, and displays the captured image. Obtain (step # 1).
Subsequently, the upper surface height detection unit H2 in the control unit H detects the height of the upper surface of the article B located at the highest position among the upper surfaces of the articles B in the container C (step # 2), and the detected article. A virtual frame line corresponding to the height of the upper surface of B is set in the captured image (step # 3).
As shown in FIG. 6, the virtual frame line set in step # 3 is an intersection line (container C) between the plane including the upper surface of the article B detected by the upper surface height detection unit H2 and the inner surface Cn of the container C. The contour line corresponding to the height of the upper surface of the article B in the shape of the top view of FIG. FIG. 6 shows the upper surface of the article B corresponding to each of the stages where the article B located at the highest position among the articles B of the container C exists when the article B is accommodated in the container C in a multi-tiered state. Virtual frame lines set for the height of are indicated by broken lines Wk1 to Wk5.

Subsequently, the control unit H extracts an area inside the virtual frame line as an article existence area where an actual article can exist (step # 4). The article presence area extracted by this processing is shown in FIG. 7 (the inside of the area surrounded by the broken line Wk is extracted as the article presence area).
In the control unit H, the article position recognition unit H1 performs article position recognition processing on the article presence area extracted in step # 4 (step # 5).
As a result, as shown in FIG. 7, it is possible to appropriately exclude a portion where the reflected virtual image Bk may be generated in the captured image.

  By the way, it is the boundary Wt between the bottom upper surface Ct of the container C shown in FIG. 8 and the inner side surface Cn of the container C that can extract only the article presence region where the actual article B can exist most reliably from the captured image. Is the virtual frame line, and the inside of the virtual frame line is extracted as the article presence region. However, when the virtual frame line is set in such a manner, as shown in FIG. 9, the article Bi located near the center of the container C in the top view is present in the captured image inside the virtual frame line. Although the article Bo is located in the region but is located adjacent to the inner side surface Cn of the container C, there is a possibility that a portion outside the virtual frame line is generated in the captured image.

  As described above, an article Bo having a part not included in the article presence area is not recognized as the article B as a result of pattern matching when the article position recognition process is executed, and as a result, in the vicinity of the inner side surface Cn of the container C. There is a risk that the article Bo located in the container C will remain in the container C without being taken out.

  In the present embodiment, a virtual frame line that is an intersection line between the plane including the upper surface of the article located at the highest position among the upper surfaces of the article B in the container C and the inner surface of the container C is set in the captured image, Since an area inside the virtual frame line in the captured image is extracted as the article presence area, a portion where the reflected virtual image Bk may be generated in the captured image can be appropriately excluded, but the inner side surface Cn can be appropriately excluded. The situation in which the article B at the adjacent position cannot be recognized as the article B can be avoided, and the inconvenience that the article B remains without being picked in the container C can be appropriately avoided.

[Another embodiment]
(1) In the above embodiment, a plane including the height of the upper surface of the article B whose upper surface is located at the highest position among the articles B in the container C, detected by the upper surface height detection unit H2, and the container C The intersecting line with the inner side surface Cn is set as a virtual frame line, and an area inside the virtual frame line is extracted as an article presence area. For example, the brightness of each pixel of the captured image is detected and adjacent The virtual frame line may be set by other methods, such as setting a virtual frame line by connecting points where the luminance of the pixels to be changed changes by a predetermined value or more. Moreover, the structure which always uses the boundary Wt of the bottom part upper surface Ct and the inner surface Cn of the container C as a virtual frame line irrespective of the height of the upper surface of the article | item B may be sufficient.

(2) In the above embodiment, the article position recognition unit H1 detects the virtual frame line as an intersection line between the plane including the upper surface of the article B detected by the upper surface height detection unit H2 and the inner surface Cn of the container C. However, the present invention is not limited to such a configuration. For example, the upper management device Hu is positioned at the top of the articles B in the container C based on the past picking history. The number of steps from the bottom in the container C within the container C (referred to as step number information) is managed, and the item position recognition unit H1 stores the information in the container C based on the step number information obtained from the host management device Hu. The height of the upper surface of the article B located in the uppermost stage among the articles B of the same is calculated, and the plane parallel to the conveying surface of the conveying conveyor 22C and passing through the height of the upper surface of the article B and the inner surface of the container C are calculated. The intersection line with Cn is configured as a virtual frame line. It may be.

50 Imaging unit B Article C Container Cn Inner side surface H1 Article position recognition unit H2 Upper surface height detection unit

Claims (2)

An imaging unit for imaging from above the container whose inner side surface is formed flat and whose upper surface is opened;
An article position recognizing device comprising: an article position recognizing unit that executes an article position recognizing process for recognizing a position of an article accommodated in the container based on a captured image captured by the imaging unit;
The article position recognition unit is an article position recognition apparatus that extracts an article presence area where an actual article can exist from the captured image and executes the article position recognition process on the extracted area. An upper surface height detector for detecting the height of the upper surface of the article located at the highest position among the upper surfaces of the articles in the container;
The article position recognition unit sets, in the captured image, a virtual frame line that is an intersection line between a plane including the upper surface of the article detected by the upper surface height detection unit and the inner surface of the container. The article position recognition apparatus according to claim 1, wherein an area inside the virtual frame line is extracted as the article existence area.

Images