JP7489678B1 - Pallet detection device and pallet detection method - Google Patents

Abstract

[Problem] To provide a pallet detection device that can quickly and accurately grasp the number of pallets placed on a loading platform in a simpler way. [Solution] The pallet detection device 10 of the present invention comprises an imaging unit 11 that acquires first image information including an image of a pallet loaded on the loading platform, an image processing unit 13 that acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and a pallet detection unit 14 that detects an image of the pallet from the second image information by comparing the second image information with image information of the pallet registered in advance, and the pallet detection unit 14 determines the relative position of the pallet on the loading platform based on at least one of information on the pallet width and pallet height. [Selected Figure] Figure 3

Description

The present invention relates to a pallet detection technology that detects pallets placed on a loading platform.

As a technology for managing luggage in warehouses, a method has been proposed in which the locations of luggage and pallets within the warehouse are managed by reading identification codes attached to the luggage and pallets. Recently, technology has been proposed that reduces the burden of inspecting and inventorying luggage within the warehouse by acquiring image information of luggage and pallets within the warehouse using cameras mounted on transport vehicles or drones that move within the warehouse.

For example, Patent Documents 1 and 2 propose a pallet detection system that uses a robot or drone equipped with a camera that takes images of luggage in a warehouse. By reading barcodes attached to luggage or shelves with a camera mounted on the robot or drone, information on the type and number of luggage can be obtained.

JP 2019-527172 A JP 2019-167177 A

The technology for reading barcodes attached to luggage or pallets as described above requires capturing images of the luggage or pallet from a relatively close distance and from the front, so when inspecting luggage or pallets placed on the bed of a truck or the like, the luggage or pallet may not be detected accurately. When luggage placed on the bed of a truck or the like is being brought in or taken out, there is a demand for technology that can quickly and accurately determine the number of luggage or pallets in a simpler way than luggage placed in a warehouse.

The present invention was made to solve the above problems, and aims to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler way.

In order to solve the problems described above, the pallet detection device of the present invention comprises an imaging unit that acquires first image information including an image of a pallet loaded on a loading platform, an image processing unit that acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and a pallet detection unit that detects the image of the pallet from the second image information by comparing the second image information with pre-registered image information of the pallet, and the pallet detection unit determines the relative position of the pallet on the loading platform based on at least one of information on the width and height of the pallet.

The palette detection unit may also calculate information on at least one of the width and height of the palette using the vertex coordinates of the constituent surfaces of the palette.

In order to solve the problems described above, the pallet detection method of the present invention is a pallet detection method executed in a pallet detection device that includes an imaging unit that acquires an image of a pallet, an image processing unit that processes the image acquired by the imaging unit, and a pallet detection unit that detects an image of the pallet from the image processed by the image processing unit, and includes an image acquisition step in which the imaging unit acquires first image information including an image of a pallet loaded on a loading platform, an image processing step in which the image processing unit acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and a pallet image detection step in which the pallet detection unit detects the image of the pallet from the second image information by comparing the second image information with image information of the pallet that has been registered in advance, and the pallet image detection step determines the relative position of the pallet on the loading platform based on at least one of information on the width and height of the pallet.

The palette image detection step may also calculate information on at least one of the width and height of the palette using vertex coordinates of the constituent surfaces of the palette.

The present invention makes it possible to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner.

FIG. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining an example of palette detection using the palette detection device according to the embodiment of the present invention. FIG. 3 is a functional block diagram of a palette detection device according to an embodiment of the present invention. FIG. 4 is a diagram for explaining the geometric transformation of the loading platform image according to the embodiment of the present invention. FIG. 5 is a diagram for explaining detection of a palette according to an embodiment of the present invention. FIG. 6 shows an example of an operational flow of the palette detection method according to the embodiment of the present invention.

The following describes an embodiment of the present invention with reference to the drawings. Note that the present invention can be implemented in various embodiments and is not limited to the embodiment of the invention described below.

<Configuration of Pallet Detection System>
Fig. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. The pallet detection system in Fig. 1 is composed of a pallet detection device 10 that acquires an image of a pallet 3 on which luggage 2 is placed, which is placed on the loading platform of a transport vehicle such as a truck 1, and detects the number of pallets 3 on the loading platform using the acquired image, and a pallet management server 20 that manages the number of pallets detected by the pallet detection device 10.

The pallet detection system in FIG. 1 is configured to detect a pallet 3 on which cargo 2 is placed, which is loaded on the bed of a truck 1, by comparing image information acquired by an imaging unit mounted on a pallet detection device 10 with image information of a preregistered pallet. The image of the detected pallet 3 and the detection results of the pallet 3 are transmitted to a pallet management server 20 as necessary.

When luggage 2 placed on the bed of a truck 1 or the like is being unloaded or loaded, in order to determine whether luggage to be transported is loaded, it is necessary to detect the number of pallets 3 placed on the bed of the truck 1 or the like. Therefore, compared to checking luggage placed in a warehouse, there are time and location constraints, and therefore a technology is required to quickly and accurately determine the number of pallets 3 in a simpler manner. The pallet detection device 10 of this embodiment is configured to determine the number of pallets 3 on which multiple luggage 2 are placed, using an image of the entire luggage, which includes an image of the pallets 3.

The pallet detection device 10 acquires image information of the entire loading platform, including an image of the pallet 3, and detects the image of the pallet from the image information of the entire loading platform by comparing it with pre-stored image information of the pallet. The pre-stored image information of the pallet includes information representing the characteristics of the pallet, such as the shape of the pallet.

As image information of the pallets 3, images taken for each type of pallet 3 may be stored, or image information learned using image information of a large number of pallets may be used. The type and number of pallets 3 on the loading platform may be ascertained by detecting information representing the characteristics of the pallet, such as the color and shape of the pallet 3, from image information of the entire loading platform, including image information of the pallets 3.

<Geometric transformation of the loading platform image>
When loading or unloading luggage placed on the bed of a truck 1 or the like, the photographer taking the image of the bed is not necessarily in the same position, so the distance and angle between the digital camera or smartphone taking the image of the bed and the bed are not constant depending on the photographer.

As a result, it may not be possible to detect the image information of the pallet that has been stored in advance in the acquired loading platform image, and the detection accuracy of the pallet image may decrease. The pallet detection device 10 of this embodiment is configured to improve the detection accuracy of the pallet image by performing image processing such as geometric transformation on the acquired loading platform image.

Specifically, if the distance or angle between the digital camera or smartphone and the loading platform varies depending on the photographer taking the image of the loading platform, distortion will occur in the shape of the loading platform image. Therefore, by geometrically transforming the shape of the acquired loading platform image (first image information), the image distortion is corrected so that the captured image has a specified shape, and the image information of the pallet is detected using the image (second image information) having the specified shape with the distortion corrected.

Here, if the image having the specified shape with the distortion corrected does not correspond to the specified angle of view, the imaging unit may be zoomed in or out as necessary to acquire information about the loading platform image again, and image processing such as geometric transformation may be performed on the acquired loading platform image to acquire an image having the specified shape with the distortion corrected.

In this way, by correcting image distortion so that the captured image has a specified shape and comparing it with the image information of the pallet using a loading platform image that corresponds to a specified angle of view, it is possible to improve the detection accuracy of the image information of the pallet when detecting the image information of the pallet from the corrected loading platform image.

As a method of geometric transformation of the loading platform image, for example, keystone correction, which corrects a trapezoidal image into a rectangular image, can be considered, but it is not limited to keystone correction. Image transformation can be performed so that a rectangular image of any shape becomes an image corresponding to a specified angle of view, without being limited to keystone correction.

When an image of the required size is obtained by geometric transformation, it is not necessarily necessary to zoom in or out the imaging unit. As an image corresponding to a specified angle of view for detecting image information of the pallet, it is possible to zoom in or out the imaging unit so that the area of the image is approximately the same as the image size at the specified angle of view, or to zoom in or out so that the image size is within a specified range at the specified angle of view. The imaging unit may also be zoomed in or out so that the image size corresponds to the image information of the loading platform that has been learned in advance.

<Detecting the position of pallets on a loading platform>
2 is a diagram for explaining an example of pallet detection using a pallet detection device according to an embodiment of the present invention. In the example of Fig. 2, the bed of a truck 1 has a structure divided into two sections, a left section and a right section, and each of the left and right sections has a depth equivalent to one pallet.

In the case of a truck 1 bed structure divided into two sections, left and right, as shown in Figure 2, it is possible to ascertain the number of pallets in the entire bed by detecting pallets using images acquired from the outside of each of the left and right beds.

When detecting pallets from pallet detection device #1 on the left side of truck 1, if a loading platform image is acquired while a pallet (pallet P in FIG. 2) placed in the right section of the loading platform is visible, pallet detection device #2 on the right side of the loading platform of truck 1 may erroneously detect pallet P that should be detected. Pallet detection devices #1 and #2 may count pallets P in duplicate, making it impossible to accurately determine the number of pallets in the loading platform. Therefore, the pallet detection device of this embodiment is configured to determine the position of the pallet in the loading platform when pallets 3 are placed on a loading platform with depth as shown in FIG. 2, thereby accurately determining the number of pallets.

<Configuration of Pallet Detection Device>
The configuration of the pallet detection device 10 will be described with reference to Fig. 3. Fig. 3 shows an example of a functional block diagram of the pallet detection device according to the embodiment of the present invention.

The pallet detection device 10 comprises an imaging unit 11 for acquiring image information of the loading platform including image information of the pallet, a central processing unit 12 for performing image processing of the image acquired by the imaging unit 11 and pallet detection, a wireless unit 18 for transmitting and receiving information to and from the pallet management server 20, a memory unit 15 for storing various information, etc., an input unit 16 for the photographer to input information, and a display unit 17 for displaying the detection status of the pallet image.

The imaging unit of the pallet detection device 10 is an imaging element for acquiring image information of the entire loading platform, including an image of the pallet. The imaging unit may be a general-purpose RGB camera for detecting the shape and size of the pallet from the image information of the entire loading platform. For example, a digital camera or a smartphone equipped with an RGB camera may be used as the pallet detection device 10 equipped with an imaging unit.

The central processing unit 12 functions as an image processing unit 13 that performs image processing of the image acquired by the imaging unit 11, and as a palette detection unit 14 that detects palettes. The palette detection device 10 can be configured as a computer equipped with a CPU, memory, and an interface for connecting to external devices.

The memory unit 15 can store image information of the pallet and the detection results of the pallet for comparison with the acquired image information of the loading platform. The image information of the pallet includes information for expressing the characteristics of the pallet, such as the color and shape of the pallet. The image information of the pallet may be image data that has been trained with image data of multiple pallets.

<How to detect pallets>
A pallet detection method according to an embodiment of the present invention will be described with reference to Figures 4 and 5. Figure 4 is a diagram for explaining the geometric transformation of the loading platform image according to an embodiment of the present invention, and Figure 5 is a diagram for explaining the detection of a pallet according to an embodiment of the present invention.

In the example of Figure 4, a geometric transformation is performed so that the reference position (a, b, c, d) on the loading platform becomes a rectangular image (A, B, C, D). As a result of the geometric transformation, an image is obtained in which the area of the image for detecting the luggage image is within a specified range at a specified angle of view.

In FIG. 4, the loading platform image obtained by geometric transformation in the imaging unit 11 is within a specified range at a specified angle of view, and is an image with a shape and size that can be compared with the image information of the pallets 3. This image can be used to identify the type and number of pallets 3 by comparing it with the image information of the pallets 3.

Here, if the image acquired by the imaging unit 11 and subjected to the geometric transformation does not have a shape and size that can be compared with the image information of the palette 3, the imaging unit 11 may perform the geometric transformation again using an image acquired by zooming in or out as necessary.

Figure 5 shows an example of the result of detecting the number of pallets using the relative positions of the pallets. In this embodiment, the pallet detection device detects the pallets 3 on the loading platform by comparing the image information acquired by the imaging unit 11 with pre-stored image information of the pallets 3 using an image obtained by geometric transformation, and further calculates the width and height of the pallet 3 using the vertex coordinates of the constituent surfaces of the detected pallet 3. The calculated width and height information of the pallet 3 is used to grasp the relative position of the pallet 3, thereby detecting the number of pallets 3 in the foreground.

In this embodiment, pallets 3 are classified according to the width and height of the pallets 3, and the relative positions of the pallets 3 are ascertained, thereby eliminating duplicate detection of pallets 3. For example, the width and height of the pallets 3 are compared with a predetermined threshold, and the pallets 3 are classified according to whether the width and height of the pallets 3 are greater than or less than the predetermined threshold. Pallets with widths and heights greater than the predetermined threshold are determined to be pallets in the front, and the other pallets are determined to be pallets in the back. In the example of Figure 5, the number of pallets 3 in the front, which have a relatively large width and height, is "8", and the number of pallets 3 in the back, which have a relatively small width and height, is "3".

Note that it is not necessary to use both the width and height information of pallet 3 to determine the relative position of pallet 3. The relative position of pallet 3 can be determined using at least one of the width and height information of pallet 3.

According to this embodiment, even if the distance or angle between the digital camera or smartphone and the loading platform is not constant depending on the photographer taking the image of the loading platform of a truck 1 or the like, the accuracy of detection of pallet images of luggage placed on the loading platform of a truck 1 or the like can be improved by performing image processing such as geometric transformation on the acquired image. Furthermore, by determining the relative position of the pallet 3 on the loading platform based on the information on the width and height of the pallet 3 on the loading platform, detection of duplicate pallets 3 can be eliminated and the number of pallets 3 on the loading platform can be accurately determined.

<Operation flow of the pallet detection device>
The operation of the palette detection method executed in the palette detection device 10 will be described with reference to Fig. 6. Fig. 6 shows an example of an operation flow of the palette detection method according to the embodiment of the present invention.

The imaging unit of the pallet detection device acquires an image of the loading platform including an image of the pallet (S1-1: image acquisition step). The image processing unit of the pallet detection device detects the loading platform area including the image of the pallet from the acquired image (S1-2), and performs a geometric transformation of the loading platform image so that the shape of the image of the loading platform area becomes a predetermined shape (S1-3: image processing step).

The pallet detection unit of the pallet detection device compares the image of the loading platform area after the geometric transformation with image information of the pallet registered in advance (S1-4), detects the image of the pallet from the loading platform image, and detects the vertex coordinates of the constituent surfaces of the pallet (S1-5: pallet image detection step).

The pallet detection unit of the pallet detection device further calculates the width and height of the pallet using the vertex coordinates of the constituent surfaces of the pallet (S1-6), and determines the relative position of the pallet using at least one of the pallet width and height information, and calculates the number of pallets in the front (S1-7).

The image in which the palette is detected and the palette detection results are sent to a palette management server as necessary. The detection results are verified by verifying the image in which the palette is detected and the palette detection results, and if the detection results are incorrect, the palette image may be trained using the corrected detection results to improve detection accuracy.

For example, if a user refers to an image in which a palette has been detected and the palette detection results, and it is determined that an image that is not a palette has been detected as a palette (false detection), or conversely, an image that should have been detected as a palette has not been detected (missed detection), then the system may train the falsely detected images as images other than palettes and the missed detection images as images of palettes, thereby improving the accuracy of detection.

In this way, according to this embodiment, it is possible to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner.

1...truck, 2...baggage, 3...pallet, 10...pallet detection device, 11...imaging unit, 12...central processing unit, 13...image processing unit, 14...pallet detection unit, 15...storage unit, 16...input unit, 17...display unit, 20...pallet management server.

Claims (4)

An imaging unit that acquires first image information including an image of a pallet loaded on a loading platform divided into two sections, a left section and a right section ;
an image processing unit that obtains second image information having a predetermined shape by geometrically transforming a shape of the first image information;
a palette detection unit that detects an image of the palette from the second image information by comparing the second image information with image information of the palette registered in advance,
The pallet detection unit determines a relative position of the pallet on the loading platform based on at least one of information on a width of the pallet and a height of the pallet ,
Using the relative positions, the number of the pallets loaded in the section of the loading platform closer to the imaging unit is calculated.
Pallet detector. The pallet detection device according to claim 1 , wherein the pallet detection unit calculates information on at least one of a width of the pallet and a height of the pallet using vertex coordinates of constituent surfaces of the pallet. A palette detection method executed in a palette detection device including an imaging unit that captures an image of a palette, an image processing unit that processes the image captured by the imaging unit, and a palette detection unit that detects the image of the palette from the image processed by the image processing unit,
an image acquiring step in which the imaging unit acquires first image information including an image of a pallet loaded on a loading platform divided into two sections, a left section and a right section ;
an image processing step in which the image processing unit acquires second image information having a predetermined shape by performing a geometric transformation on the shape of the first image information;
a palette image detection step in which the palette detection unit detects an image of the palette from the second image information by comparing the second image information with image information of the palette registered in advance,
The pallet image detection step includes determining a relative position of the pallet on the loading platform based on at least one of information on a width of the pallet and a height of the pallet,
Using the relative positions, the number of the pallets loaded in the section of the loading platform closer to the imaging unit is calculated.
Palette detection method. 4. The palette detection method according to claim 3, wherein the palette image detection step calculates information on at least one of a width and a height of the palette using vertex coordinates of constituent surfaces of the palette.

Images