JP2010032258A - Apparatus and method for recognizing position of work for depalletizing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method for recognizing the positions of works for depalletizing and capable of substantially reducing incorrect detection based on contour data of the upper surfaces of works, heightening recognition accuracy by reducing the effects of the shapes and patterns of the works and the definition of images, and highly accurately detecting the positions and attitudes of the works at all times. <P>SOLUTION: A photographing device 12 photographs a plurality of identical works 1 arranged in the same plane to acquire an image 3. An image processing device 14 acquires contour data 5 on the basis of the image 3, detects the position and attitude of each work on the basis of the contour data 5, registers images of work parts in the image as master image patterns 6, performs pattern matching between the master image patterns and the image of each work part, and outputs the position and attitude of each work in the case that the pattern matching is successful on all the work parts. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a workpiece position recognition apparatus and method for depalletizing.

The unloading of works such as cardboard boxes loaded on a pallet one by one is called “depalletizing”. In the case of depalletizing using a depalletizing industrial robot (depalletizing robot), it is necessary to accurately detect the position and posture of the workpiece located at the uppermost stage from an image obtained by photographing the top surface of the pallet.
For this reason, various means for recognizing the workpiece position have been proposed (for example, Patent Documents 1 to 3).

Patent Document 1 aims to detect the position of an article (work) and enable automatic depalletization.
Therefore, in Patent Document 1, as shown in FIG. 4, an area sensor that detects the height of the workpiece and four sides with respect to the side surface of the uppermost workpiece F based on the height data detected by the area sensor. The upper surface (shadow portion) of the uppermost workpiece F is extracted from the illumination device 52 that illuminates from above, the CCD camera 54 that images the workpiece F from above the pallet P, and the imaging data captured by the CCD camera 54, An image processing device for detecting the contour of the uppermost workpiece F, and by determining the approximate position of the uppermost workpiece F from the contour data of the uppermost workpiece F detected by the image processing device, the position of the workpiece F Is to recognize.

Patent document 2 aims at specifying the box located in the uppermost part easily from among the box bodies of the palletized rectangular parallelepiped.
Therefore, in Patent Document 2, as shown in FIG. 5, the palletized box B is illuminated obliquely from above, photographed by the stereo television camera devices 56 and 56, and a shadowed edge from the photographed image. Are extracted, three-dimensionally measured, and the highest one is selected. Next, the illumination is switched to the illumination from above, the image is taken again and the remaining edges are specified, and then matching is performed to determine the type, posture, and gripping position of the box.

Patent Document 3 aims to accurately and reliably detect the position and orientation of a target article from an image, and to perform high-speed processing applicable to FA and the like.
Therefore, in Patent Document 3, as shown in FIG. 6, an image 63 in which a plurality of identical articles are arranged on the same plane is photographed, the position and posture 69 of the article are detected from the image, and the image is obtained from the shape data 64 of the article. A template 65 indicating the outline of the article on the top is generated, a binary edge image 66 in which the shape of the article is emphasized is created from the image, and article candidate data 67 is created by vector correlation between the template and the edge image. Line segment data 68 constituting an edge is created, and an accurate position and posture 69 of the article are determined by line segment search from the article candidate data and the line segment data.

JP 2001-317911 A, “Article Position Recognition Device” JP-A-7-299978, “Depalletizing image creation method and image creation device” JP 2007-140729 A, “Method and apparatus for detecting the position and orientation of an article”

In the apparatus of Patent Document 1, since a dark shadow is generated on the upper surface and the edge of the uppermost article photographed from above by illuminating the side surface of the uppermost article, a clear contour is detected. Based on the contour data, the approximate position of the uppermost article is determined, and the position of the article is recognized.
However, in Patent Document 1, since the workpiece position is recognized using only contour data, there is a high possibility that erroneous detection occurs in a workpiece such as a cardboard box.

In the method of Patent Document 2, a box illuminated obliquely from above is photographed by a stereo camera, an edge is extracted from the photographed image, and the highest one is selected by three-dimensional measurement. Thereafter, the illumination is switched to the illumination from above, and another image is taken again by another stereo camera to identify the remaining edges. The type, posture, and gripping position of the box are determined from the integrated plurality of edge information.
However, since the means of Patent Document 2 requires a plurality of illuminations and stereo cameras, the scale of the system increases. Moreover, since it is necessary to obtain and integrate a plurality of edges from the stereo camera, the calculation load increases.

With the means of Patent Document 3, since the vector correlation and the line segment search are used using the contour data of the upper surface of the article as a template, the position and orientation of the article can be accurately detected.
However, even with the means of Patent Document 3, the position or posture of an article may be erroneously detected depending on the shape and pattern of the article, the sharpness of an image, and the like. For example, when a plurality of identical articles (for example, cardboard) arranged on the same plane have a pattern or a mating surface that approximates the outline of a single article, erroneous detection of position or orientation may occur.

  The present invention has been developed to solve the above-described conventional problems. That is, the object of the present invention is to greatly reduce the false detection based on the contour data of the workpiece upper surface, to reduce the influence of the workpiece shape, pattern and image sharpness, to improve the recognition accuracy, always high accuracy Another object is to provide a depalletizing work position recognition apparatus and method capable of detecting the position and orientation of a work.

According to the present invention, it is provided with a photographing device that photographs an image in which a plurality of identical workpieces are arranged on the same plane, and an image processing device that performs image processing on the photographed image, and controls the robot by detecting the position and orientation of the workpiece. A workpiece position recognition device that outputs the result to a device,
The workpiece position recognition device acquires contour data from the image, detects the position and orientation of each workpiece from the contour data, registers an image of a workpiece portion in the image as a master image pattern, and the master image pattern A workpiece position recognition device is provided, which performs pattern matching between the image of each workpiece part and outputs the position and orientation of each workpiece when pattern matching is successful for all workpiece parts. .

Further, according to the present invention, the image capturing device captures an image in which a plurality of identical works are arranged on the same plane,
The image processing device acquires contour data from the image, detects the position and orientation of each workpiece from the contour data, registers the image of the workpiece portion in the image as a master image pattern, There is provided a work position recognition method characterized in that pattern matching with an image of a work part is performed, and when the pattern matching is successful for all work parts, the position and orientation of each work are output.

According to a preferred embodiment of the present invention, as a master image pattern, a work image having the highest degree of coincidence between the detected work and the contour data is first registered,
If pattern matching is not successful for all work parts, the images of the work parts having successively higher matching degrees are registered in the master image pattern.

  It is preferable to use a vector correlation value as the degree of coincidence between the detected workpiece and the contour data.

  According to the apparatus and method of the present invention described above, an image of a work part in an image is registered as a master image pattern, pattern matching between the master image pattern and the image of each work part is performed, and images of all work parts are registered. When pattern matching is successful, the position and orientation of each workpiece are output, so that false detection based on the contour data on the workpiece upper surface can be greatly reduced, and the effect of the workpiece shape, pattern, and image clarity. The recognition accuracy can be improved by reducing the above, and the position and posture of the workpiece can always be detected with high accuracy.

The apparatus and method of the present invention are a combination of template matching based on edge detection and pattern matching using an image of a work part, and each is a simple recognition process, so that the calculation load is small.
Further, since the master image pattern is acquired every measurement, it is not necessary to store image data for each work, and there is no need to teach again even if the work is changed.
In addition, since the present invention does not require a plurality of sensors, the measurement unit (imaging device) can be simplified as compared with the prior art.

  Accordingly, recognition accuracy is improved, erroneous detection is less likely to occur, calculation load and memory load can be reduced, and a single measurement unit (imaging device) is required, so that the system scale can be reduced.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

FIG. 1 is an overall configuration diagram of a depalletizing apparatus provided with a workpiece position recognition apparatus of the present invention.
In this figure, a work 1 is, for example, the same cardboard box and is stacked on a pallet 2. The pallet 2 is installed in a predetermined pick station (two places in this figure) adjacent to the industrial robot (depalletizing robot) 10 for depalletizing.

  As shown in this figure, the workpiece position recognition apparatus of the present invention includes an imaging device 12 that captures an image 3 in which a plurality of identical workpieces 1 are arranged on the same plane, and an image processing device 14 that performs image processing on the captured image 3. With.

  In this example, the workpiece position recognition apparatus of the present invention detects only the uppermost workpiece among the workpieces on the pallet, and detects the position and orientation of the uppermost workpiece 1 from the image 3.

The photographing device 12 is, for example, a digital still camera or a digital video camera that can acquire a digital image. Note that these cameras have a resolution capable of photographing at least all the uppermost workpieces and identifying each workpiece.
Note that the present invention does not particularly limit the means for distinguishing the uppermost workpiece from other workpieces. For example, as the imaging device 12, a stereo camera or a three-dimensional laser measuring device spaced apart from each other may be used so that the distance to the workpiece can be detected.

  In this example, the image processing device 14 is an image processing PC (computer), and is connected to the robot control device 15 and the warehouse management PC 16 via a LAN. The warehouse management PC 16 is preliminarily input with workpiece shape data.

The image processing device 14 erases images other than the uppermost workpiece from the image captured by the imaging device 12 on the basis of the distance to the workpiece. Next, the image processing device 14 executes the workpiece position recognition method of the present invention, detects the position and posture of the workpiece, and outputs the result to the robot control device 15.
The robot control device 15 controls the depalletizing robot 10 based on the position and posture of the workpiece by the image processing device 14 and executes a depalletizing operation for unloading the workpieces loaded on the pallet one by one.

  FIG. 2 is a flowchart showing a work position recognition method according to the present invention. FIG. 3 is an explanatory diagram of the work position recognition method of the present invention.

As shown in FIG. 2, the work position recognition method of the present invention includes steps S1 to S9.
In step S1, a single type and a plurality of workpieces 1 palletized by the pick station are photographed from above by the photographing device 12, and an image 3a on the upper portion of the workpiece is obtained.

  FIG. 3A is an example of the image 3a. In this example, ten workpieces 1 are included in the image 3a. Hereinafter, the ten workpieces 1 are referred to as workpieces 1a to 1j, respectively. The outline of each workpiece 1 (1a to 1j) is a rectangle, and four marks or patterns a, b, c, and d exist on the workpiece upper surface.

In step S2, edge extraction is performed on the image 3a on the workpiece.
In step S3, the workpiece outline (template 5) obtained from the workpiece dimension information and the edge information extracted in step S2 are matched (template matching) to detect the position and orientation of each workpiece.
The workpiece dimension information (template) is preferably obtained from the warehouse management PC 16. The template 5 is preferably generated from the workpiece shape data recorded on the warehouse management PC 16 as the workpiece outline (template 5) on the image.

  FIG. 3B shows an image 3b after contour matching (template matching) in step S3. In this figure, reference numeral 5 denotes a template, which shows a state in which the detection of the workpiece 1a has failed, the workpieces 1b and 1c are erroneously detected, and the other workpieces 1d to 1j have been successfully detected.

In step S3, vector correlation is further performed between the detected workpieces 1b to 1j and the template 5, the respective vector correlation values are obtained, and the workpieces 1b to 1j are first candidates in descending order of vector correlation value. , Second candidate, and ranking.
Vector correlation is a method for searching for a position of a geometric shape similar to a template from a binary image (edge image), and is disclosed in, for example, Patent Document 3. Further, the larger the value of the vector correlation value means that the template 5 and the image 3b match.

  In step S4, a work having the highest contour matching degree, that is, a first candidate work (for example, work 1d) having the largest vector correlation value is selected, and an image inside the contour is cut out and registered as the master image pattern 6.

In step S5, an image of the work part (inside the contour) in the image is extracted from another work, and pattern matching between the master image pattern 6 and the image of each work part is sequentially performed. In this pattern matching, pattern matching is also performed for the marks or patterns a, b, c, d on the upper surface of the workpiece.
Then, it is verified whether there is an erroneous candidate in the detected position and posture of each workpiece.

In step S6, it is checked whether the number of workpieces to be transported (in this example, 10) has been successfully verified. If all collations are successful, the process proceeds to step S7, and if there is a failed work, the process proceeds to step S8.
The reason why the specified number (10 in this example) of collation has not succeeded in step S6 is that an illegal master image pattern has been registered.
In the example of FIG. 3B, detection of the work 1a has failed, and the work 1b, 1c has been erroneously detected, so there is a failed work, and the process proceeds to step S8.

  If the collation of the number of workpieces to be transported (10 in this example) is successful (YES), position information (position and posture) is output to the robot controller 15 in step S7, and data is degenerated based on this position information. The palletizing robot 10 grips and transports the workpiece.

  On the other hand, if the collation of the number of workpieces to be transported (10 in this example) is not successful (NO), it is determined in step S8 whether another candidate (second candidate or subsequent candidate) exists. If (YES), the other candidate (the candidate with the next highest pattern matching degree) is registered in the master image pattern 6 in step S9, and steps S5 and S6 are executed again.

On the other hand, if another candidate does not exist in step S8 (NO), it is an error and the process returns to the start again.
When the number of the uppermost workpiece is one, step S3 to step S4 to step S6 are passed from step S3 to step S7.

As the master image pattern described above, the principal component analysis may be performed on all the workpieces detected in step S3, and an average candidate image pattern may be used.
Thereby, the memory of the master image pattern is compressed, and the recognition process can be performed at a higher speed without lowering the recognition accuracy.

  According to the apparatus and method of the present invention described above, the image of the work part in the image 3 (3a) is registered as the master image pattern 6, pattern matching between the master image pattern 6 and the image of each work part is performed, When pattern matching is successful for all workpiece images, the position and orientation of each workpiece are output, so that false detection based on the contour data on the workpiece upper surface can be greatly reduced. It is possible to increase the recognition accuracy by reducing the influence of the sharpness of the image, and always detect the position and posture of the workpiece with high accuracy.

The apparatus and method of the present invention are a combination of template matching (S3) based on edge detection and pattern matching (S5) using an image of a work part, and each is a simple recognition process, so the calculation load is small.
Further, since the master image pattern 6 is obtained every measurement, it is not necessary to store image data for each work.
In addition, since the present invention does not require a plurality of sensors, the measurement unit (imaging device) can be simplified as compared with the prior art.

  Accordingly, recognition accuracy is improved, erroneous detection is less likely to occur, calculation load and memory load can be reduced, and a single measurement unit (imaging device) is required, so that the system scale can be reduced.

  In addition, this invention is not limited to embodiment mentioned above, Of course, a various change can be added in the range which does not deviate from the summary of this invention.

1 is an overall configuration diagram of a depalletizing apparatus including a workpiece position recognition apparatus according to the present invention. It is a flowchart which shows the workpiece | work position recognition method by this invention. It is explanatory drawing of the workpiece | work position recognition method of this invention. It is a schematic diagram of the apparatus of patent document 1. FIG. It is a schematic diagram of the means of patent document 2. It is a schematic diagram of the means of patent document 3.

Explanation of symbols

1, 1a-1j work, 2 palettes,
3,3a Images taken,
3b Image after template matching,
5 Work outline (template),
6 Master image pattern,
10 Depalletizing robot,
12 photographing device, 14 image processing device,
15 Robot controller, 16 Warehouse management PC

Claims (4)

Equipped with an imaging device that captures images of multiple identical workpieces arranged on the same plane, and an image processing device that performs image processing on the captured images, and detects the position and orientation of the workpiece and outputs the result to the robot controller A workpiece position recognition device,
The workpiece position recognition device acquires contour data from the image, detects the position and orientation of each workpiece from the contour data, registers an image of a workpiece portion in the image as a master image pattern, and the master image pattern A workpiece position recognizing apparatus, wherein pattern matching is performed on images of each workpiece portion and pattern matching is successful for all workpiece portions, and the position and orientation of each workpiece are output. Photographing images with multiple identical workpieces lined up on the same plane with a photographing device,
The image processing device acquires contour data from the image, detects the position and orientation of each workpiece from the contour data, registers the image of the workpiece portion in the image as a master image pattern, What is claimed is: 1. A work position recognition method comprising: performing pattern matching with an image of a work part and outputting the position and posture of each work when pattern matching is successful for all work parts. As a master image pattern, the work image with the highest degree of matching between the detected work and the contour data is registered first,
The work position recognition method according to claim 2, wherein, when pattern matching is not successful for all work parts, images of work parts having successively higher matching degrees are registered in a master image pattern.   4. The work position recognition method according to claim 3, wherein a vector correlation value is used as the degree of coincidence between the detected work and the contour data.

Images