JP4784823B2 - Method and apparatus for detecting collapse of goods - Google Patents

Description

  The present invention relates to a method and apparatus for detecting collapse of an article that occurs when an article is transferred from a loading area such as a pallet on which a plurality of articles are stacked.

In automatic warehouses at distribution centers and manufacturing plants, various products packed with box-shaped packing materials such as cardboard boxes are temporarily stored in storage shelves in the warehouse in the state of being loaded on pallets in preparation for shipment.・ It is stored. Usually, the same pallet is stored in a warehouse in a state where a large number of packing materials having the same dimensions are loaded. Hereinafter, a packaging material such as a cardboard in which products are packed, and other objects transferred from one position to another are referred to as “articles”.
When shipping goods from the warehouse, a pallet loaded with necessary types of goods is taken out of the storage shelf by a stacker crane, moved to a position adjacent to the picking robot by a conveyor or the like, and the necessary number of goods is picked up by the picking robot. Picking is automatically performed from the pallet, and is sequentially unloaded onto a carry-out lane installed on the shipping side (see, for example, Patent Document 1 below).

In fully automating such article conveyance / transfer operations, when an article loaded on a pallet collapses, it is important to appropriately detect the collapse.
In view of such circumstances, in Patent Document 2 below, one proposal has been made regarding means for detecting collapse of articles.
The prior art of this patent document 2 is provided with a gate form frame through which a pallet in which articles are stacked can pass on a conveyance path for conveying a pallet on which a plurality of articles are stacked by a conveyor. An article detector that emits signals such as light, radio waves, and sound waves that pass up and down in the vicinity of both sides in the conveying direction of a group of articles stacked on the pallet is provided. During the conveyance, the collapsed state is detected when the signal of the article detector is cut off.

JP-A-8-91579 JP 2001-294303 A

However, even if there is no cargo collapse during the pallet transport, it is possible that other items loaded while the goods are being transferred by the picking robot due to errors in the control of the picking robot or errors in the position information of the goods on the pallet. There is a problem that there is a risk that the article may be pushed to shift its position or to fall or drop.
Also, as a result of the transfer work, the balance of the center of gravity of each article collapses, and there is a possibility that the article collapses, such as the article falling or collapsing after a while after the transfer work is completed. There is.
Therefore, it is necessary to detect such collapse of goods in order to fully automate the conveyance / transfer operation of articles.

  The present invention has been made in view of such circumstances, and can detect a load collapse that occurs during the automatic transfer of an article by a picking robot, and further, a load collapse that occurs after the transfer operation is completed. It is an object of the present invention to provide a method and an apparatus for detecting an article collapse that can be detected.

In order to solve the above problem, the article collapse detection method of the present invention employs the following means.
That is, the load collapse detection method for an article according to the present invention detects the load collapse of the article when the article is transferred from a loading area where a plurality of articles are stacked to a position different from the loading area. (A) For each transfer of an individual article, a first group of images of the articles stacked in the stacking area before and after the respective articles is imaged from above, An image and a second image after transfer, (b) comparing the first image with the second image, and changing the article region other than the region where the transferred article was present It is characterized by determining the presence or absence of occurrence of cargo collapse based on the degree.

Thus, for each transfer of an individual article, the first image before the article is transferred and the second image after the transfer are taken, and the first image and the second image are transferred. Since the images are compared, when an article other than the transferred article is displaced at the time of transferring the article, a change appears in the article area other than the area where the transferred article exists in both images. Therefore, it is possible to determine whether or not the cargo collapse has occurred based on the degree of change.
For this reason, it is possible to detect load collapse that occurs during automatic transfer of articles by the picking robot.
This also improves the reliability of automatic transfer by the picking robot.
In addition, by notifying the supervisor of the occurrence of cargo collapse, it is not necessary to constantly monitor, thereby reducing the burden on the supervisor.

  In the article collapse detection method according to the present invention, in the step (b), the edge images of the first image and the second image are generated and compared, and the edges are compared. A first matching for determining whether or not a cargo collapse has occurred based on a degree of change in a portion corresponding to an article area other than an area where the transferred article is present in the image; the first image; The two images are compared as the grayscale image or the color image, and the collapse of the load is determined based on the degree of change in the portion corresponding to the article area other than the area where the transferred article exists in the grayscale image or the color image. And performing a second matching for determining the presence or absence of occurrence.

  As described above, since geometric matching (first matching) using edge images and pattern matching (second matching) using a grayscale image or a color image as they are are performed, even if one of the matchings cannot detect the collapse of the load, When the collapse of the load is detected by the other matching, it is determined that the collapse of the load has occurred. For this reason, it is possible to reduce detection failure of load collapse.

  Further, in the article collapse detection method according to the present invention, the article group before and after the work in all steps of the transfer work for transferring the desired number of articles from the loading area is three-dimensionally measured. The first distance image and the second distance image after the work are generated, and the articles on the loading area before and after the transfer work based on the first distance image and the second distance image. Estimating the volume of the group, and determining whether or not load collapse has occurred after all the steps of the transfer work, based on the volume difference of the article group before and after the transfer work and the shape data of the article, To do.

In this way, the group of articles before and after the entire transfer process is three-dimensionally measured to generate a distance image, and using this distance image, the number of articles actually transferred is estimated. And the number of articles to be transferred (desired number) are compared to determine whether or not they match, so that even if a load collapse occurs after all the steps of the transfer operation, this can be detected. In other words, if the estimated number and the desired number do not match, it can be determined that the collapse of the article group has occurred due to, for example, the balance of the center of gravity of the article group remaining after the transfer operation has ended. Can be detected.
This also prevents a situation in the inventory database that the product is handled as shipped, although it is not actually shipped. For this reason, inconsistency between the number of articles on the inventory database and the actual number of articles can be eliminated.

  Further, in the article collapse detection method according to the present invention, each time an individual article is transferred or a plurality of articles is transferred, a group of articles before the article is transferred is three-dimensionally measured to be third. Is generated, and when it is determined that load collapse has occurred in the step (b), a fourth distance image is generated by three-dimensionally measuring the group of articles after the articles are transferred. By comparing the third distance image and the fourth distance image, the presence / absence of the occurrence of collapse of the cargo is determined based on the degree of change in the article region other than the region where the transferred article was present ( When the determination is made again separately from the determination in the step b), and it is determined that the collapse of the load has not occurred in the re-determination, the determination in the step (b) is denied and no collapse of the load has occurred. And if it is determined that the collapse of cargo has occurred in the above determination , The collapsing is determined to have occurred an affirmative determination when transferring the article at a stage (b), characterized in that.

As described above, when it is determined by the first load collapse determination that the load collapse has occurred, the presence or absence of the load collapse is determined again by the three-dimensional measurement, so even if there is an error in the determination of the first load collapse. It is possible to negate an erroneous determination by a determination using three-dimensional measurement with high measurement accuracy and perform an accurate load collapse determination.
For this reason, it is possible to greatly improve the accuracy of detecting load collapse and to prevent adverse effects such as interruption of transfer work due to erroneous detection.

  The article collapse detection method according to the present invention also includes an article collapse detection when transferring an article from a loading area where a plurality of articles are stacked to a position different from the loading area. A method for detecting, three-dimensionally measuring the group of articles before and after the operation in all steps of the transfer operation of transferring a desired number of articles from the loading area, and a first distance image before the operation Generating a second distance image after the work, estimating a volume of the group of articles on the loading area before and after the transfer work based on the first distance image and the second distance image; Based on the volume difference of the article group before and after the transfer operation and the shape data of the article, it is determined whether or not a load collapse has occurred after all the steps of the transfer operation.

  As described above, even when a load collapse occurs after all the steps of the transfer operation, this can be detected, so that it is possible to eliminate inconsistency between the number of articles on the inventory database and the actual number of articles.

  In addition, an article collapse detection device according to the present invention detects an article collapse when transferring an article from a loading area in which a plurality of articles are stacked to a position different from the loading area. An apparatus for detecting, comprising: an imaging unit that images the group of articles stacked in the loading area from above; and an image processing unit that performs image processing based on image data captured by the imaging unit, For each transfer of an individual article, the imaging means captures a first image before the article is transferred and a second image after the transfer, and the image processing means A load collapse determination unit is provided that compares the first image with the second image and determines whether or not load collapse has occurred based on the degree of change in the article region other than the region where the transferred article was present. It is characterized by that.

  In the article collapse detection device according to the present invention, the collapse determination unit generates edge images of the first image and the second image, compares the images, and compares the edge images. First matching means for determining the presence or absence of collapse of cargo based on the degree of change in the portion corresponding to the article region other than the region where the transferred article was present, the first image and the first The two images are compared as the grayscale image or the color image, and the collapse of the load is determined based on the degree of change in the portion corresponding to the article area other than the area where the transferred article exists in the grayscale image or the color image. And second matching means for determining the presence or absence of occurrence.

  The article collapse detection device according to the present invention further includes a three-dimensional measuring unit that three-dimensionally measures the group of articles stacked in the loading area, and the image processing unit includes the three-dimensional measuring unit. A distance image generating means for generating a distance image based on the measurement data, wherein the three-dimensional measuring means before and after the operation in all steps of the transfer operation for transferring the desired number of articles from the loading area. The distance image generation means generates a first distance image before the work and a second distance image after the work based on the three-dimensional measurement data, and the image processing means The volume of the article group on the loading area before and after the transfer operation is estimated based on the first distance image and the second distance image, and the difference in the volume of the article group before and after the transfer operation And articles Based on the shape data, comprising the operations after collapsing determining means for determining the presence or absence of the occurrence of collapsing after all the steps of the transfer operation, characterized in that.

  In the article collapse detection device according to the present invention, the three-dimensional measuring means and the distance image generating means are transferred each time an individual article is transferred or a plurality of articles are transferred. The three-dimensional measurement of the article group before the generation of a third distance image, and when the load collapse determination unit determines that the load collapse has occurred, the article group after the article is transferred Three-dimensional measurement is performed to generate a fourth distance image, and when the load collapse determination unit determines that load collapse has occurred, the image processing means determines the third distance image and the fourth distance image. A load collapse re-determination means for re-determining the occurrence of the load collapse based on the image, the load collapse re-determination means comparing the third distance image and the fourth distance image, and transferring the article Whether or not cargo collapse occurred based on the degree of change in the article area other than the area where If it is determined again, and it is determined that no load collapse has occurred in the second determination, the determination by the load collapse determination unit is denied, and no load collapse has occurred. If it is determined that a load collapse has occurred, the determination by the load collapse determination unit is affirmed, and it is determined that a load collapse has occurred when the article is transferred.

  In addition, an article collapse detection device according to the present invention detects an article collapse when transferring an article from a loading area in which a plurality of articles are stacked to a position different from the loading area. An apparatus for detecting, a three-dimensional measuring means for three-dimensionally measuring a group of articles stacked in the loading area, an image processing means for performing image processing based on measurement data from the three-dimensional measuring means, The three-dimensional measuring means three-dimensionally measures a group of articles before the work in all steps of the transfer work for transferring a desired number of the articles from the loading area, and the image processing means Distance image generating means for generating a first distance image before the work and a second distance image after the work based on measurement data from the three-dimensional measuring means, the first distance image and the second distance Before and after the transfer work based on the image The volume of the article group on the loading area is estimated, and based on the difference in the volume of the article group before and after the transfer operation and the shape data of the article, whether or not the load collapse occurs after all the steps of the transfer operation And a post-work load collapse judging means for judging the above.

  The above-described article collapse detection method can be implemented by the article collapse detection apparatus.

  In the present invention, the “stacking area” refers to a predetermined area where articles are stacked for transfer. In the embodiment, the pallet corresponds to this.

  According to the present invention, it is possible to detect a load collapse that occurs during automatic transfer of an article by a picking robot, and it is also possible to detect a load collapse that occurs after the transfer operation is completed. can get.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

[First Embodiment]
The first embodiment of the present invention will be described below.
FIG. 1 is a diagram illustrating a configuration of a picking system 10 to which an article collapse detection device according to a first embodiment is applied. Hereinafter, the article collapse detection device may be simply referred to as “load collapse detection device”.
As shown in FIG. 1, the picking system 10 includes a picking robot 4 and a load collapse detection device 20.

First, the picking robot 4 will be described.
The picking robot 4 is a device that picks the articles 1 one by one from the pallet 2 as a loading area on which a plurality of articles 1 are loaded and transfers them to the unloading position. In the following description, it is assumed that the pallet 2 corresponds to a “loading area”.
The picking robot 4 includes a pedestal portion 5 that can rotate around a vertical axis X, an arm portion 6 that is installed on the pedestal portion 5 and has a plurality of joints that can rotate around a horizontal axis, And a grip portion 7 that is connected to the tip and grips an article.
In the present embodiment, the gripping part 7 is provided with a suction pad that is attached to the tip of the arm part 6 so as to be rotatable about a vertical axis, and sucks the upper surface of the article 1 by negative pressure.

In the picking robot 4 configured as described above, the height and the horizontal position of the gripping unit 7 attached to the tip of the arm unit 6 are changed by bending the joint of the arm unit 6 and rotating the pedestal unit 5. be able to.
Reference numeral 14 denotes a robot / transport control device that controls the picking robot 4 and the pallet conveyor 12.

  As shown in FIG. 1, the load collapse detection device 20 measures a group of articles 1 (hereinafter referred to as an article group) stacked on the pallet 2 from above, and three-dimensionally measures the article group. A three-dimensional measurement sensor 22 and an information processing apparatus 23 that executes various types of information processing including image processing are provided.

  The camera is disposed almost directly above the article group so that the entire article group can be imaged from above. The camera 21 may capture either a grayscale image (monochrome image) or a color image, but in the present embodiment, a camera 21 that captures a grayscale image is used.

The three-dimensional measurement sensor 22 is arranged so that the article group can be measured three-dimensionally. The three-dimensional measurement sensor 22 may be either an active type or a passive type, but in this embodiment, as the active type, a linear or fan-shaped light beam is projected from the light projecting means to the article group, and the article group is By scanning with a light beam and monitoring the position of the bright spot or bright line formed by the light beam using a semiconductor position sensing element (PSD) or camera as the light receiving means, it is possible to reach the site where the bright spot or bright line is formed. Configured to determine the distance.
One or a plurality of the above light receiving means may be used, but in the present embodiment, the number is one. In addition, when the number of light receiving units is one, the entire shape of the article group cannot be captured. Therefore, the upper surface shape of the article group is measured, and the entire shape of the article group is estimated from the upper surface shape and the shape data of the article. Details of this will be described later.
Note that the three-dimensional measurement sensor 22 may be a passive type, for example, using a stereo image method in which a stereo image is captured by two cameras.

The information processing apparatus 23 has a function of receiving image data and measurement data from the camera 21 and the three-dimensional measurement sensor 22 and executing various image processing, and is a dedicated computer equipped with a personal computer or a dedicated microchip. It can consist of computers.
The information processing device 23 is connected to a host computer 24 for instructing a shipping command via a LAN. The host computer 24 is connected to an article information database 26 in which article shape data including the dimension data of the article 1 is stored. Yes.

The storage unit of the information processing apparatus 23 stores a load collapse detection program for causing a computer to execute processing for detecting load collapse described later.
When the information processing apparatus 23 is a personal computer, the load collapse detection program is stored in the storage unit of the personal computer by installing from a recording medium such as a CD-ROM or downloading via a network. When the information processing apparatus 23 is a dedicated computer, it is stored in advance in the microchip.

FIG. 2 is a functional block diagram of the load collapse detection device 20 according to the present embodiment.
As shown in FIG. 2, the load collapse detection device 20 includes an imaging unit 31, a three-dimensional measurement unit 32, and an image processing unit 33.

The image pickup means 31 picks up an image of the article group loaded on the pallet 2 from above, and the camera 21 shown in FIG. 1 functions as the image pickup means 31.
For each transfer of the individual articles 1 loaded on the pallet 2, the imaging means is an image before the article 1 is transferred (hereinafter referred to as “image 1”) and an image after the transfer. (Hereinafter referred to as “image 2”).

The three-dimensional measuring means 32 measures three-dimensionally the group of articles stacked on the pallet 2, and the three-dimensional measuring sensor 22 shown in FIG. 1 functions as the three-dimensional measuring means 32.
The three-dimensional measuring means 32 three-dimensionally measures a group of articles before and after the transfer process in which all desired items 1 are transferred from the pallet 2.

The image processing unit 33 performs image processing based on image data and measurement data from the imaging unit 31 and the three-dimensional measurement unit 32, and the information processing apparatus 23 shown in FIG. Function.
As shown in FIG. 2, the image processing unit 33 includes a load collapse determination unit 34, a distance image generation unit 35, and a post-work load collapse determination unit 36.
The load collapse determination unit 34 compares the image 1 and the image 2 captured by the image capturing unit 31, and the occurrence of load collapse occurs based on the degree of change in the article region other than the region where the transferred article 1 was present. Processing for determining the presence or absence is executed. In the present embodiment, the load collapse determination unit 34 includes a first matching unit 34a and a second matching unit 34b.

FIG. 3 is a diagram for explaining a method for determining an article collapse by the first matching means 34a.
The first matching unit 34a first performs edge extraction processing on each image from the images 1 and 2 to generate an edge image. In FIG. 3, the upper right article on the pallet is transferred in FIG. 3, (A) is an edge image before transfer generated based on image 1, and (B) is based on image 2. It is the edge image after the transfer produced | generated in this way.
Next, the first matching unit 34a compares the article positions detected from both edge images, and obtains the degree of change in the position of the article other than the area where the transferred article was present in the edge image. Then, a threshold is set in advance as a reference for determining that the collapse of the load has occurred, and based on this, the presence or absence of the occurrence of the collapse of the load is determined.

FIG. 4 is a diagram for explaining a method for determining an article collapse by the second matching means 34b.
In FIG. 4, (A) is an image before transfer of an article, and (B) is an image after transfer of the article.
The second matching unit 34b compares the image 1 and the image 2 with the grayscale image or the color image, and compares the grayscale image or the color image with the portion corresponding to the article area other than the area where the transferred article was present. Find the degree of change. This degree of change can be defined, for example, with a degree of similarity based on a normalized correlation method. Then, a threshold is set in advance as a reference for determining that the collapse of the load has occurred, and based on this, the presence or absence of the occurrence of the collapse of the load is determined.

Note that the load collapse determination processing by the first matching means and the second matching means may be performed in parallel or sequentially.
In addition, the load collapse determination by the load collapse determination unit 34 is not limited to that by the first matching unit 34a and the second matching unit 34b, and the transfer of the article based on the image 1 and the image 2 can be performed by another image processing method. The degree of change before and after the loading may be obtained, and the load collapse determination may be performed.
Regardless of which image processing method is used, two or more different image processing methods can be used to determine the degree of change before and after the transfer of the article based on image 1 and image 2, and to perform load collapse determination. preferable.

As described above, the load collapse determination unit 34 determines whether or not load collapse has occurred in each of the first matching unit 34a and the second matching unit 34b, and determines that load collapse has occurred in at least one of them. Determines that the collapse of the cargo has occurred and outputs the result.
The output signal is input to the robot / conveyance control device 14 of FIG. 1, and the transfer operation of the picking robot 4 is interrupted.

Further, the above-described load collapse detection device 20 outputs information for allowing the monitoring person to recognize that the load collapse has occurred when the image processing means 33 (information processing device 23) detects the occurrence of the load collapse. A cargo collapse information output means (not shown) is further provided.
The load collapse information output means can adopt various forms such as displaying on the display unit (display) of the information processing apparatus 23, turning on / flashing a warning lamp, and emitting a warning sound.
For this reason, when the collapse of the load is detected, it becomes possible for the supervisor to quickly respond to the second best response work such as the work for correcting the loading state of the articles.

Next, the distance image generating means 35 and the post-work load collapse determining means 36 shown in FIG. 2 will be described.
As described above, the three-dimensional measuring unit 32 three-dimensionally measures a group of articles before and after the operation in all steps of the transfer operation for transferring a desired number of articles 1 from the pallet 2.
That is, the three-dimensional measuring unit 32 measures the article group in a state where the pallet 2 has been conveyed to the position where the article 1 is transferred. In addition, the three-dimensional measuring unit 32 performs three-dimensional measurement on the article group in a state where the transfer operation of the desired number of articles 1 has been completed. This desired number is the number instructed by the shipping command from the host computer 24 and the number of articles 1 to be transferred by the picking robot 4.

The distance image generation means 35 is based on the measurement data from the three-dimensional measurement means 32, and the distance image before work (hereinafter referred to as “distance image A1”) and the distance after work in all steps of the transfer work. An image (hereinafter referred to as “distance image A2”) is generated.
The post-work load collapse determining means 36 first estimates the volume of the article group on the pallet 2 before and after the transfer work based on the distance image A1 and the distance image A2. As described above, the three-dimensional measuring unit 32 in the present embodiment cannot directly capture the entire shape of the article group, and therefore measures the upper surface shape of the article group, and the upper surface shape and the article information database 26 (see FIG. 1). .) Is used to estimate the overall shape of the article group. That is, since the top surface shape, height, and area of the article group can be calculated by the three-dimensional measuring means 32 (three-dimensional measurement sensor 22), the overall shape of the article group is obtained from these data and the article shape data. And the volume can be estimated.
Then, the post-work load collapse determining means 36 determines whether or not load collapse has occurred after all the steps of the transfer work, based on the volume difference of the product group before and after the transfer work and the product shape data.

Specifically, this determination can be performed as follows.
Since the volume difference between the articles before and after the transfer work should be equal to the total volume of the article 1 actually transferred, the calculated volume difference is calculated from the article shape data. By dividing by the volume per one, the number of articles 1 actually transferred can be estimated.
Then, the post-work load collapse determining means 36 compares the estimated number of articles 1 with the desired number to be transferred, and based on the degree of coincidence, occurrence of load collapse after all the steps of the transfer operation is performed. Determine presence or absence.
That is, when the estimated number and the desired number do not match, it can be determined that the collapse of goods such as the collapse of the article group has occurred due to the collapse of the balance of the center of gravity of the article group remaining after the transfer operation.
When it is determined that the load collapse has occurred, the post-work load collapse determining means 36 outputs the result. The output signal is input to the robot / conveyance control device 14 of FIG. 1, and the transfer operation of the picking robot 4 is interrupted.

FIG. 5 is a flowchart of the load collapse determination process performed by the image processing unit 33 according to the first embodiment.
When the image processing means 33 receives the pallet carry-in signal (S101), it instructs the three-dimensional measurement means to perform three-dimensional measurement, performs three-dimensional measurement of the article group, and based on the measurement data, the distance before the transfer operation An image A1 is generated (S102).
Next, the imaging unit is instructed to take an image, and an image 1 of the article group before the article 1 is transferred is acquired (S103).

Next, when a signal notifying that the automatic transfer operation of one article has been completed is received (S104), the imaging means 31 is instructed to take an image and the image 2 of the article group after the article 1 is transferred is acquired. (S105).
Next, the load collapse determination unit 34 compares the image 1 and the image 2 and determines whether or not the load collapse has occurred by the above-described method (S106, S107). When it is determined that the load collapse has occurred, As a determination result, a signal reporting that the collapse of the cargo has been detected is output (S113).

On the other hand, when it is determined that no cargo collapse has occurred, it is confirmed whether or not the transfer operation corresponding to the predetermined number to be transferred (the above-mentioned desired number) has been completed (S108). If not, the processes of S103 to S107 are repeated.
If it is confirmed in S108 that all transfer operations have been completed, the three-dimensional measurement unit 32 is instructed to perform three-dimensional measurement, and the article group is three-dimensionally measured. Based on the measurement data, the transfer is performed. A distance image A2 after work is generated (S109).

Next, the post-work load collapse determining means 36 compares the distance image A1 and the distance image A2, determines the presence or absence of load collapse by the method described above (S110, S111), and determines that the load collapse has occurred. In this case, as a determination result, a signal reporting that the collapse of the cargo has been detected is output (S113).
On the other hand, if it is determined that the cargo collapse has not occurred, a signal notifying the normal end of the transfer operation is output (S112).

Hereinafter, the operation and effect of the present invention according to the first embodiment will be described.
According to the present invention, each time an individual article is transferred, the image 1 before the article is transferred and the image 2 after the transfer are taken, and the image 1 and the image 2 are compared. When an article other than the transferred article is displaced at the time of transferring the article, a change appears in the article area other than the area where the transferred article exists in both images. Therefore, it is possible to determine whether or not the cargo collapse has occurred based on the degree of change.
For this reason, it is possible to detect load collapse that occurs during automatic transfer of articles by the picking robot 4.
Thereby, the reliability of automatic transfer by the picking robot 4 can be improved.
In addition, by notifying the supervisor of the occurrence of cargo collapse, it is not necessary to constantly monitor, thereby reducing the burden on the supervisor.

  Moreover, since the geometrical matching (first matching) by the edge image and the pattern matching (second matching) using the grayscale image or the color image as they are are performed by the cargo collapse judging means 34, the cargo collapse by one of the matchings. Even if it cannot be detected, if the collapse of the load is detected by the other matching, it is determined that the collapse of the load has occurred. For this reason, it is possible to reduce detection failure of load collapse.

Further, the group of articles before and after all the steps of the transfer operation are three-dimensionally measured to generate a distance image A1 and a distance image A2, and using this distance image, the number of articles actually transferred is estimated, This estimated number is compared with the number of articles to be transferred (desired number), and it is determined whether or not they match, so that even if a load collapse occurs after the entire transfer process, this can be detected. Can do.
This also prevents a situation in the inventory database that the product is handled as shipped, although it is not actually shipped. For this reason, inconsistency between the number of articles on the inventory database and the actual number of articles can be eliminated.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described with reference to FIGS.
FIG. 6 is a functional block diagram of the load collapse detection device 20 according to the present embodiment.
As shown in FIG. 6, the load collapse detection device 20 according to the present embodiment is different from that in the first embodiment in that the image processing means 33 includes a load collapse re-determination means 37. Other configurations of the image processing unit 33, the load collapse determination unit 34, the distance image generation unit 35, and the post-work load collapse determination unit 36 are the same as those in the first embodiment described above except for the points described below.

  In the present embodiment, the three-dimensional measuring unit 32 and the distance image generating unit 35 transfer the articles for each transfer of the individual articles 1 or for each transfer of the plurality of articles 1 (for example, for each group of articles for one stage). When the article group before being placed is three-dimensionally measured to generate a distance image (hereinafter referred to as “distance image B1”), and the load collapse determination unit 34 determines that the load collapse has occurred, the article The article group after 1 is transferred is three-dimensionally measured to generate a distance image (hereinafter referred to as “distance image B2”).

When the load collapse determination unit 34 determines that the load collapse has occurred, the load collapse re-determination unit 37 determines again the occurrence of the load collapse based on the distance image B1 and the distance image B2.
That is, in this embodiment, the determination by the load collapse determination unit 34 is a provisional determination, and the definitive determination of the image processing unit 33 is performed by the load collapse re-determination unit 37.

FIG. 7 is a diagram for explaining a method for re-determination of a load collapse by the load collapse re-determination means 37.
The load collapse re-determination means 37 first estimates the shape and volume of the article group in a state corresponding to each distance image based on the distance image B1 and the distance image B2. The estimation of the shape and volume is performed by the same method as the estimation of the shape and volume by the post-work load collapse determination means 36 described above.
In FIG. 7, it is assumed that the article located at the top, right, and back in the figure is transferred, (A) is the shape of the article group estimated based on the distance image B1, and (B) is the distance. This is the shape of the article group estimated based on the image B2.

Next, the cargo collapse re-determination means 37 obtains the degree of change of the article area other than the area where the transferred article was present.
This degree of change can be defined, for example, with a degree of similarity based on a normalized correlation method. Then, a threshold is set in advance as a reference for determining that the collapse of the load has occurred, and based on this, the presence or absence of the occurrence of the collapse of the load is determined.

When the load collapse re-determination means 37 determines that no load collapse has occurred in the second determination, it is assumed that no load collapse has occurred because the provisional determination by the load collapse determination means 34 is denied.
On the other hand, when it is determined that the load collapse has occurred in the determination again, the provisional determination by the load collapse determination unit 34 is affirmed, it is determined that the load collapse has occurred when the article 1 is transferred, and the result is output.
The output signal is input to the robot / conveyance control device 14 of FIG. 1, and the transfer operation of the picking robot 4 is interrupted.

FIG. 8 is a diagram illustrating a flowchart of the load collapse determination process by the image processing unit 33 according to the second embodiment.
The flowchart shown in FIG. 8 differs from the flowchart shown in FIG. 5 according to the first embodiment in that the processes of S114 to S117 are added.
That is, the articles are transferred by the three-dimensional measuring means 32 and the distance image generating means 35 for each transfer of the individual articles 1 or for each transfer of the plurality of articles 1 (for example, for each one-stage article group). A distance image B1 is generated by three-dimensionally measuring the previous article group (S114).
The generation of the distance image B1 may be substituted by the generation of the distance image A1 when it is the first transfer in all the steps of the transfer operation of the article 1.

If the result of provisional determination by the load collapse determination unit 34 (S106, S107) and it is determined that the load collapse has occurred, the image processing unit 33 instructs the 3D measurement unit 32 to perform 3D measurement. The group image is three-dimensionally measured, and the distance image generation means 35 generates a distance image B2 based on the measurement data (S115).
Next, the load collapse re-determination means 37 compares the distance image B1 and the distance image B2, determines whether or not load collapse has occurred by the above-described method (S116, S117), and determines that the load collapse has occurred. Affirms the provisional determination by the load collapse determination means 34, determines that load collapse has occurred when the article 1 is transferred, and outputs a signal notifying that load collapse has been detected as a determination result (S113).

On the other hand, if it is determined in S117 that no load collapse has occurred, the provisional determination by the load collapse determination means 34 is denied and no load collapse has occurred. It is confirmed whether or not the transfer operation corresponding to the number of times has been completed (S108). If the transfer operation has not been completed, the processes of S103 to S107 and S114 to S117 are repeated.
Other processes are the same as those in the first embodiment.

Hereinafter, the operation and effect of the present invention according to the second embodiment will be described.
According to the present invention, when it is determined by the first load collapse determination (provisional determination) by the load collapse determination means 34 that the load collapse has occurred, the presence / absence of the load collapse is determined again by three-dimensional measurement. Even if there is an error in the determination of the collapse of the load, it is possible to negate the erroneous determination by the determination using the three-dimensional measurement with high measurement accuracy and perform the accurate collapse determination.
For this reason, it is possible to greatly improve the accuracy of detecting load collapse and to prevent adverse effects such as interruption of transfer work due to erroneous detection.

  As described above, as is apparent from the description of the operation and effect in each embodiment, according to the present invention, it is possible to detect a load collapse that occurs during the automatic transfer of an article by a picking robot, and furthermore, the transfer operation is completed. An excellent effect is obtained in that it is possible to detect the subsequent collapse of the load.

  In addition, this invention is not limited to embodiment mentioned above, Of course, it can change variously in the range which does not deviate from the summary of this invention.

It is a figure which shows the structure of the picking system to which the goods collapse detection apparatus concerning 1st Embodiment of this invention is applied. It is a functional block diagram of the goods collapse detection device concerning a 1st embodiment of the present invention. It is a figure explaining the article collapse determination method by the 1st matching means. It is a figure explaining the article collapse determination method by the 2nd matching means. It is a figure which shows the flowchart of the load collapse determination process by the image processing means in 1st Embodiment of this invention. It is a functional block diagram of the goods collapse detection apparatus concerning 2nd Embodiment of this invention. It is a figure explaining the item collapse re-determination method by the load collapse re-determination means. It is a figure which shows the flowchart of the load collapse determination process by the image processing means in 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Goods 2 Pallet 4 Picking robot 5 Pedestal part 6 Arm part 7 Grip part 10 Picking system 12 Pallet conveyor 14 Robot / conveyance control apparatus 20 Load collapse detection apparatus 21 Camera 22 Three-dimensional measurement sensor 23 Information processing apparatus 24 Host computer 26 Article information Database 31 Imaging means 32 Three-dimensional measuring means 33 Image processing means 34 Load collapse determination means 34a First matching means 34b Second matching means 35 Distance image generation means 36 Post-work load collapse determination means 37 Load collapse re-determination means

Claims (10)

A method for detecting collapse of the article when transferring the article from a loading area where a plurality of articles are loaded to a position different from the loading area,
(A) For each transfer of an individual article, an image of the group of articles stacked in the stacking area before and after that is taken from above, a first image before transfer of the article, and after transfer Get a second image,
(B) The first image and the second image are compared, and the presence or absence of occurrence of cargo collapse is determined based on the degree of change in the article region other than the region where the transferred article was present.
An article collapse detection method characterized by the above. In the step (b),
The edge image of each of the first image and the second image is generated, the two images are compared, and the portion corresponding to the article region other than the region where the transferred article exists in the edge image A first matching for determining whether or not a cargo collapse has occurred based on the degree of change in
The first image and the second image are compared as a grayscale image or a color image, and a portion of the grayscale image or the color image corresponding to an article area other than the area where the transferred article was present exists. Performing a second matching for determining whether or not a cargo collapse has occurred based on the degree of change,
The article collapse detection method according to claim 1. Three-dimensional measurement is performed on the group of articles before and after the transfer process for transferring a desired number of articles from the stacking area, and a first distance image before the work and a second after the work. Generate a range image,
Estimating the volume of the article group on the loading area before and after the transfer operation based on the first distance image and the second distance image;
Based on the volume difference of the article group before and after the transfer operation and the shape data of the article, it is determined whether or not load collapse has occurred after all the steps of the transfer operation.
3. A method for detecting a collapse of an article according to claim 1 or 2. For each transfer of an individual article or each transfer of a plurality of articles, a group of articles before the article is transferred is three-dimensionally measured to generate a third distance image,
When it is determined that load collapse has occurred in the step (b), a group of articles after the article is transferred is three-dimensionally measured to generate a fourth distance image,
By comparing the third distance image and the fourth distance image, the presence or absence of occurrence of cargo collapse is determined based on the degree of change in the article region other than the region where the transferred article was present (b) ) Again in addition to the determination in the stage)
If it is determined that no load collapse has occurred in the re-determination, the determination in the stage (b) is denied and no load collapse has occurred,
If it is determined in the second determination that load collapse has occurred, the determination in the step (b) is affirmed and it is determined that load collapse has occurred when the article is transferred.
The article collapse detection method according to claim 3. A method for detecting collapse of the article when transferring the article from a loading area where a plurality of articles are loaded to a position different from the loading area,
Three-dimensional measurement is performed on the group of articles before and after the transfer process for transferring a desired number of articles from the stacking area, and a first distance image before the work and a second after the work. Generate a range image,
Estimating the volume of the article group on the loading area before and after the transfer operation based on the first distance image and the second distance image;
Based on the volume difference of the article group before and after the transfer operation and the shape data of the article, it is determined whether or not load collapse has occurred after all the steps of the transfer operation.
An article collapse detection method characterized by the above. A device for detecting collapse of the article when transferring the article from a loading area where a plurality of articles are loaded to a position different from the loading area,
Imaging means for imaging the group of articles stacked in the stacking area from above;
Image processing means for performing image processing based on image data imaged by the imaging means,
The imaging means captures a first image before the article is transferred and a second image after the article is transferred for each transfer of the article,
The image processing means compares the first image with the second image, and determines whether or not a cargo collapse has occurred based on the degree of change in an article area other than the area where the transferred article was present. Equipped with a load collapse determination means
An article collapse detection device characterized by the above. The load collapse judging means is
The edge image of each of the first image and the second image is generated, the two images are compared, and the portion corresponding to the article region other than the region where the transferred article exists in the edge image First matching means for determining the presence or absence of cargo collapse based on the degree of change of
The first image and the second image are compared as a grayscale image or a color image, and a portion of the grayscale image or the color image corresponding to an article area other than the area where the transferred article was present exists. A second matching means for determining the presence or absence of the occurrence of cargo collapse based on the degree of change,
The article collapse detection device according to claim 6. Further comprising three-dimensional measuring means for three-dimensionally measuring a group of articles stacked in the loading area;
The image processing means further includes distance image generation means for generating a distance image based on measurement data from the three-dimensional measurement means,
The three-dimensional measuring means three-dimensionally measures an article group before and after the work in all steps of the transfer work for transferring a desired number of the articles from the loading area,
The distance image generation means generates a first distance image before the work and a second distance image after the work based on three-dimensional measurement data,
The image processing means estimates the volume of the article group on the loading area before and after the transfer operation based on the first distance image and the second distance image, and the article before and after the transfer operation. Based on the difference in the volume of the group and the shape data of the article, it is provided with a post-work load collapse determining means for determining the presence or absence of load collapse after the entire transfer work process,
The article collapse detection device according to claim 6 or 7, characterized in that The three-dimensional measuring means and the distance image generating means perform a three-dimensional measurement of an article group before the article is transferred for each transfer of each article or a plurality of articles. A distance image is generated, and when the load collapse determination unit determines that the load collapse has occurred, a group of articles after the article is transferred is three-dimensionally measured to generate a fourth distance image;
The image processing means re-determines the occurrence of load collapse based on the third distance image and the fourth distance image when the load collapse determination means determines that the load collapse has occurred. A determination unit;
The load collapse re-determination means compares the third distance image and the fourth distance image, and the occurrence of load collapse occurs based on the degree of change in the article area other than the area where the transferred article was present. In the second determination, if it is determined that no load collapse has occurred, the determination by the load collapse determination means is denied and no load collapse has occurred. If it is determined that the load collapse has occurred, the determination by the load collapse determination means is affirmed and it is determined that the load collapse has occurred when the article is transferred.
The article collapse detection device according to claim 8. A device for detecting collapse of the article when transferring the article from a loading area where a plurality of articles are loaded to a position different from the loading area,
Three-dimensional measuring means for three-dimensionally measuring a group of articles stacked in the loading area;
Image processing means for performing image processing based on measurement data from the three-dimensional measurement means,
The three-dimensional measuring means three-dimensionally measures a group of articles before the work in all steps of the transfer work for transferring a desired number of the articles from the loading area,
The image processing means includes
Distance image generation means for generating a first distance image before the work and a second distance image after the work based on measurement data from the three-dimensional measurement means;
Estimating the volume of the article group on the loading area before and after the transfer operation based on the first distance image and the second distance image, and the difference in volume of the article group before and after the transfer operation; Based on the shape data of the article, it has a post-work load collapse determining means for determining the presence or absence of load collapse after all the steps of the transfer work,
An article collapse detection device characterized by the above.

Images