JP2024064336A - Cargo handling support device, cargo handling vehicle, and cargo handling support system - Google Patents

Abstract

[Problem] To provide support for loading work in an optimal manner. [Solution] A forklift 1 is equipped with a control unit 27. The control unit 27 judges whether or not a plurality of loads H are arranged in a predetermined arrangement pattern C based on an image of the top surface of the plurality of loads H placed on a pallet P, and provides support for loading work based on the result of this judgment. This makes it possible to appropriately judge whether the loading state is good or bad. In turn, support for loading work can be provided in an optimal manner based on the result of this judgment. [Selected Figure] Figure 4

Description

The present invention relates to a loading assistance device, a loading vehicle, and a loading assistance system.

Patent Document 1 describes a technology in which, when stacking loads including pallets on a cart equipped with forks, a three-dimensional sensor on the tip of the fork scans the top surface of the load to be placed on, and indicates the presence of any recesses.

Patent No. 6880950

However, simply showing the presence of a depression on the top surface makes it difficult for workers, especially those with no experience, to determine whether or not it is OK to stack the loads. This makes it difficult to stack (or unload) multiple loads using a forklift, a so-called "piling operation."
An object of the present invention is to preferably provide assistance for crawling tasks.

The loading and unloading support device according to the present invention comprises:
a determination unit that determines whether or not the plurality of articles are arranged in a predetermined arrangement based on an image of the upper surfaces of the plurality of articles placed on the pallet;
A support unit that supports a loading and unloading operation based on a determination result of the determination unit;
Equipped with.

The present invention can provide optimal support for hand-holding tasks.

FIG. 2 is a side view of the forklift according to the embodiment during a loading operation. 1 is a block diagram showing a schematic control configuration of a forklift according to an embodiment of the present invention; 11 is a diagram showing an example of a palette pattern included in the palette pattern data; FIG. 5 is a flowchart showing a procedure of a loading and unloading support process according to the embodiment. 11 is a diagram for explaining the loading and unloading support process according to the embodiment. FIG. 11 is a diagram for explaining the loading and unloading support process according to the embodiment. FIG. 11 is a diagram for explaining the loading and unloading support process according to the embodiment. FIG. 11 is a diagram for explaining the loading and unloading support process according to the embodiment. FIG. 13 is a diagram for explaining a modified example of the loading and unloading support process according to the embodiment. FIG. 13 is a diagram for explaining a modified example of the loading and unloading support process according to the embodiment. FIG. FIG. 13 is a block diagram showing a schematic control configuration of a forklift and a management device according to a modified example of the embodiment.

Each embodiment of the present invention will be described in detail below with reference to the drawings.

[Forklift configuration]
FIG. 1 is a side view of a forklift 1 according to this embodiment during a loading operation.
The forklift 1 according to this embodiment is an example of a loading vehicle according to the present invention, and by being equipped with the loading support device according to the present invention, the forklift 1 can determine whether the loading condition of the load H on the pallet P is good or bad, thereby enabling the loading operation to be carried out appropriately.
"Pile work" refers to work such as stacking loads H on top of loads H (piling) and unloading loads H that are stacked in multiple layers (unpiling). "Quality of the piled load condition" refers to whether loads H are piled properly on pallets P.
In the following description, a pallet P with multiple loads H attached thereto is referred to as an "attached pallet body (G)," and the task of stacking another attached pallet body G on top of the attached pallet body G is included in "attaching (task)." In this embodiment, the "attached condition" of the attached pallet body G is judged based on whether the loads H on the top layer of the attached pallet body G are arranged in a predetermined array.

Specifically, the forklift 1 comprises a vehicle body 11, forks 12, a lifting body (lift) 13, a mast 14, and wheels 15. The mast 14 is provided at the front of the vehicle body 11, and is driven by a drive source (not shown) to tilt the vehicle body 11 forward and backward. The lifting body 13 is driven by a drive source (not shown) to rise and fall along the mast 14. A pair of left and right forks 12 are attached to the lifting body 13 as holding parts for holding luggage, pallets P, etc.

The pallet P is a loading platform on which multiple loads H can be placed. The pallet P is formed in a rectangular plate shape with two holes (fork pockets) e, and is held by the forklift 1 by inserting a pair of forks 12 into the two holes e.

FIG. 2 is a block diagram showing a schematic control configuration of the forklift 1.
As shown in this figure, in addition to the above-mentioned components, the forklift 1 is equipped with a drive unit 21, an operation unit 22, a display unit 23, an image capturing unit 24, a storage unit 26, and a control unit 27. The loading and unloading support device according to the present invention is configured to include at least the control unit 27.

The drive unit 21 includes a travel motor, a steering motor, and a loading motor (all not shown) which are various drive sources of the forklift 1. The travel motor drives the drive wheels of the wheels 15. The steering motor rotates (performs a steering operation) the steering wheels of the wheels 15. The loading motor is a drive source which performs the operations of raising and lowering the lifting body 13 and tilting the mast 14.
The operation unit 22 is an operation means through which an operator (driver) performs various operations. The operation unit 22 includes, for example, a steering wheel, pedals, levers, various buttons, etc., and outputs operation signals according to the contents of these operations to the control unit 27.

The display unit 23 is, for example, a liquid crystal display, an organic electroluminescence display, or other display, and displays various information based on a display signal input from the control unit 27. The display unit 23 may be a touch panel that also serves as part of the operation unit 22, or may include a speaker capable of audio display (output).

The photographing unit 24 is an example of an image acquisition unit according to the present invention, and photographs the area in front of the forklift 1, acquires image information, and outputs the acquired information to the control unit 27. The photographing unit 24 in this embodiment is disposed under the lifting body 13, and is installed facing diagonally downward and forward so that, when the lifting body 13 is raised, the top surface of a load H on a mounting pallet body G placed on the floor in front of it is included in the photographing range (angle of view) 24R (see FIG. 1).
The type of image that can be obtained by the photographing unit 24 is not particularly limited as long as it can obtain an image including the top surface of the package H, and may be, for example, an image that can obtain an RGB image. Alternatively, the photographing unit 24 may be a scanner device that obtains a two-dimensional or three-dimensional shape by scanning light or sound waves and detecting reflected waves. Alternatively, the photographing unit 24 may be an RGB-D camera (depth camera), LiDAR (Light Detection And Ranging), compound eye camera, or the like that can obtain an image including distance information (depth information).

The storage unit 26 is a memory configured to include, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores various programs and data, and also functions as a work area for the control unit 27 .
In the storage unit 26 of this embodiment, a learning model 261 and pallet pattern data 262 are pre-stored in addition to a cargo handling support program 263 for executing a cargo handling support process (see FIG. 4) described below.

The learning model 261 extracts the top surface area S (see FIG. 6) including the top surface of the load H from the image. The learning model 261 is created in advance by machine learning using, for example, sample images of a plurality of loads H to which circumscribing lines including the top surface are added as training data, and is stored in the memory unit 26. The learning model 261 may be an AI (Artificial Intelligence) having a deep learning neural network.

The pallet pattern data 262 is data that indicates an arrangement pattern (pallet pattern) of loads H (such as cardboard boxes) of approximately the same size when stacking (attaching) multiple loads H on a pallet P. Figures 3(a) to (g) are diagrams showing examples of pallet patterns included in the pallet pattern data 262, specifically, (a) block stacking (flat stacking), (b) brick stacking, (c) pinhole stacking, (d) alternating row stacking, (e) double pinhole stacking, and (f) split stacking. The "first tier" indicates the bottom tier, and the "fourth tier" indicates the top tier.
However, the pallet pattern is not limited to that shown in Fig. 3 and may include, for example, loads H of different sizes. Furthermore, when the method of this embodiment is used to determine whether the mounting condition of the mounting pallet body G is good or bad, it is sufficient that the pallet pattern data 262 includes data on at least the arrangement pattern of the top row.

The control unit 27 is configured with, for example, a CPU (Central Processing Unit) and controls the operation of each part of the forklift 1. Specifically, the control unit 27 operates the drive unit 21 based on the operation content of the operation unit 22, deploys a program pre-stored in the memory unit 26, and executes various processes in cooperation with the deployed program.

[Loading and unloading support processing]
Next, a loading/unloading support process for supporting a loading/unloading operation when the forklift 1 performs a loading operation will be described.
FIG. 4 is a flow chart showing the procedure of the cargo handling support process, and FIGS. 5 to 8 are diagrams for explaining the cargo handling support process.

The loading and unloading support process is a process for determining whether the loading and unloading condition of the loading and unloading pallet body G placed on the floor in front during the loading and unloading work is good or bad. This loading and unloading support process is executed by the control unit 27 reading and developing the loading and unloading support program 263 from the memory unit 26 based on a predetermined execution condition (e.g., an operation performed by the operator, etc.).
In the mounting work, the forklift 1 lifts up and places another mounting pallet body G on top of a mounting pallet body G placed on the floor. Here, the forklift 1, holding the other mounting pallet body G, moves to the front of the mounting pallet body G on the floor, and then raises the forks 12 above the mounting pallet body G on the floor (see FIG. 1).
In the following description, unless otherwise specified, the term "mounted pallet body" refers to the one placed on the floor at the front of the vehicle, not the one held on the forks 12. In addition, the mounted pallet body G in this embodiment is assumed to have multiple loads H stacked in blocks.

As shown in FIG. 4, when the loading and unloading support process is executed, the control unit 27 first acquires an image of the mounting pallet body G viewed from above by the photographing unit 24 (step S1).
In this step, the control unit 27 operates the photographing unit 24 based on the operation of the operator, and photographs the top surface of the mounting pallet body G placed on the floor in front of the forklift 1. More specifically, as the lifting body 13 raises the forks 12 above the mounting pallet body G on the floor, the photographing unit 24 configured integrally with the lifting body 13 also rises above the mounting pallet body G, and the photographing unit 24 captures an image of the mounting pallet body G viewed from above. As a result, an image of the mounting pallet body G in which the top surfaces of all the loads H on the top level can be confirmed is obtained, as shown in FIG. 5, for example. This image is stored in the memory unit 26. This image may also be displayed in real time on the display unit 23.
In addition, the timing of the photographing does not have to be based on the operator's operation. For example, the front of the forklift 1 may be photographed periodically at a predetermined photographing timing (e.g. after the forks 12 are raised), and the mounting pallet body G (cargo H) may be detected from the image.

Next, the control unit 27 inputs the image acquired in step S1 into the learning model 261, and extracts multiple top surface areas S that individually include the top surfaces of multiple loads H from the image (step S2).
Specifically, as shown in Fig. 6, for example, the area (circumscribing line) of the top surface of each load H in the image acquired in step S1 is extracted as the top surface area S by the learning model 261. The circumscribing line means the smallest rectangle (a rectangle surrounded by vertical and horizontal lines in the image) that circumscribing the top surface of the load H in the image. The learning model 261 is a model that is constructed in advance by machine learning using, for example, a neural network, etc., so as to extract the top surface area S of each load H from an image of the mounting pallet body G viewed from above.
The upper surface area S may be an area including the upper surface of the load H, and may be the outline or inscribing line of the upper surface of the load H instead of the circumscribing line. The outline means a shape that follows the outline of the upper surface of the load H, and the inscribing line means the largest rectangle inscribed in the load H in the image (a rectangle surrounded by vertical and horizontal lines in the image). When the load H is photographed from directly above, the outline and the circumscribing line or inscribing line of the load H almost coincide with each other.

Next, the control unit 27 sets a representative point M of each upper surface region S (step S3).
In this embodiment, the position of the center of gravity (i.e., the geometric center) of each upper surface region S is obtained as the representative point M. However, the representative point M is not limited to the position of the center of gravity, and may be a position common to all the upper surface regions S (for example, the lower right corner of the four corners in a plan view).

Next, the control unit 27 determines whether or not all the representative points M are arranged in a predetermined arrangement (arrangement pattern) (step S4).
Here, the control unit 27 reads out the arrangement pattern C, which is arrangement information of the loads H on the top level of the mounting pallet body G, from the pallet pattern data 262 in the storage unit 26, and compares it with the position of each representative point M. As shown in Fig. 7, the arrangement pattern C is arrangement information (including image information) in which planned positions B where the representative points M are to be located are arranged in accordance with the pallet pattern of the mounting pallet body G. In this embodiment, when the operator selects "block stacking" as the pallet pattern of the mounting pallet body G, the arrangement pattern C of the top level of block stacking is read out from the pallet pattern data 262 and compared with the information of each representative point M.
Specifically, the control unit 27 collates the information on the representative points M with the array pattern C to determine whether there are an excess or deficiency of the representative points M and whether the regularity of the array (two-dimensional array in plan view) is good or bad. If the number of the representative points M matches the number of the planned positions B of the array pattern C, the control unit 27 determines that there is no excess or deficiency of the representative points M, and if they do not match, determines that there is an excess or deficiency of the representative points M. Furthermore, if all the representative points M are located within the ranges of the corresponding planned positions B, the control unit 27 determines that the regularity of the array of the representative points M is good, and if even one representative point M is located outside the range of the planned positions B, determines that the regularity of the array of the representative points M is not good.
The operation of the operator to select the pallet pattern may be performed at the time of step S4, or may be performed in advance. Also, instead of the operator selecting the pallet pattern, the pallet pattern of the load H may be estimated from the information on the positions of all the representative points M. Also, it is preferable that the operator can arbitrarily set the range of each planned position B in the arrangement pattern C. Also, the conditions for determining pass/fail may be arbitrarily set, and for example, if the representative point M is within a predetermined number or on the inside, it may be determined to be good even if there is a missing point, etc.
Then, if it is determined that all of the representative points M are not arranged in a specified array (i.e., there is an excess or shortage of the number of representative points M, or the regularity of the array of the representative points M is not good) (Step S4; No), the control unit 27 transitions to the processing of Step S7 described below.

On the other hand, if it is determined in step S4 that all representative points M are arranged in a predetermined array (i.e., there is no excess or deficiency in the number of representative points M, and the array of representative points M has good regularity) (step S4; Yes), the control unit 27 determines whether all representative points M are within the judgment region R (step S5).
In this step, if the two-dimensional arrangement of the representative points M is good in step S4, it is judged whether or not the three-dimensional positions (height positions) are good. Specifically, if distance information (depth information) is obtained together with image information from the photographing unit 24, it is judged whether the height of the uppermost load H of the mounting pallet body G is good or bad. In cases where only two-dimensional image information from the photographing unit 24 is available, this step may be omitted.
Specifically, when distance information of the captured image is available, as shown in Fig. 8, the three-dimensional coordinates of each representative point M are acquired, and it is determined whether or not all representative points M (only three are illustrated in the figure) are within a determination region R in the height direction. The determination region R is for determining whether or not the top surface height of each load H on the topmost level of the mounting pallet body G is appropriate, and its position and range are set in advance. However, the determination region R may be set arbitrarily by the operator, or may be set individually for each load H, and may be set at any time (for example, immediately before step S5).
The configuration for obtaining the distance information is not particularly limited, and for example, a TOF (Time Of Flight) sensor or the like may be provided separately from (or integrated with) the imaging unit 24, or, as described above, an RGB-D camera (depth camera) or a stereo camera capable of obtaining an image including three-dimensional point cloud data may be used as the imaging unit 24. Alternatively, the distance may be obtained based on the scale (and aspect ratio) of the image from the internal parameters of the imaging unit 24.

If it is determined in step S5 that all representative points M are within the judgment region R (step S5; Yes), the control unit 27 determines that the load H on the mounting pallet body G has been properly mounted, and allows the forklift 1 to continue the loading (mounting) operation (step S6).
Here, the control unit 27, based on the operation of the operator, causes the mounting pallet body G lifted by the forks 12 to be placed directly on the mounting pallet body G placed on the floor. In other words, the control unit 27 does not restrict the operation of the forklift 1 against the operation of the operator.
At this time, the control unit 27 may notify the operator that the mounting pallet body G placed on the floor has been properly mounted. The manner in which this notification is made is not particularly limited, and for example, the display unit 23 may display the words "Mounting OK", or may display an image other than words, or may output a sound or turn on a lamp, etc.

On the other hand, if it is determined in step S5 that at least one representative point M is not within the judgment region R (step S5; No), or if it is determined in step S4 that the representative points M are not arranged in a predetermined array (step S4; No), the control unit 27 determines that the load H on the mounting pallet body G is not properly mounted, and notifies the operator of a defective mounting state of the mounting pallet body G (step S7). This makes it possible to warn the operator not to continue mounting when the stability of the mounting pallet body G, which serves as the base, is insufficient.
Specifically, the control unit 27 notifies the operator that the fastening pallet body G placed on the floor has not been properly fastened, for example by displaying the text "Fast fastening" on the display unit 23. However, the manner of this notification is not particularly limited, and an image other than text may be displayed, or a sound may be output, a lamp may be turned on, or the like.
Here, the control unit 27 may also display the current arrangement pattern on the display unit 23. By looking at this display, the operator can recognize which of the loads H is not properly arranged.
Furthermore, the control unit 27 may control the drive unit 21 to temporarily halt or slow down the operation of the forklift 1, thereby prohibiting or restricting the continuation of the loading and unloading work. This makes it possible to avoid a situation in which another mounting pallet body G is placed on a mounting pallet body G that is not stable enough, and thus makes it possible to suppress the occurrence of load collapse.

Next, the control unit 27 determines whether or not to end the loading/unloading support process (step S8), and if it determines not to end the process (step S8; No), the control unit 27 transitions to the process of step S1 described above.
Then, when it is determined that the loading/unloading support process should be ended, for example, due to the completion of the loading work (step S8; Yes), the control unit 27 ends the loading/unloading support process.

In this embodiment, assistance with loading and unloading operations may be provided based on the result of the determination as to whether or not the multiple loads H are arranged in a predetermined arrangement. As described above, "assistance with loading and unloading operations" here includes notification of the determination result (whether or not the multiple loads H are arranged in a predetermined arrangement) and permission or restriction of continuation of loading and unloading operations.

[Technical effect of the present embodiment]
As described above, according to this embodiment, it is determined whether or not the multiple loads H (mounted pallet body G) are arranged in a predetermined arrangement based on an image of the top surface of the multiple loads H placed on the pallet P. Then, support for loading and unloading work is provided based on the result of this determination.
This makes it possible to evaluate the stability of the base pallet body G and detect cargo collapse or signs of such collapse, thereby preventing accidents during loading and unloading operations (loading operations).
Therefore, it is possible to appropriately determine whether the mounting pallet body G is properly mounted, i.e., whether the mounting state is good or bad. Furthermore, it is possible to appropriately support the mounting work based on the determination result.

In addition, according to this embodiment, if it is determined that the multiple loads H on the top level of the mounting pallet body G are not arranged in a specified array, the determination result is notified (for example, displayed on the display unit 23).
This allows the loading and unloading worker (e.g., the operator of the forklift 1) to immediately recognize whether the loading condition is good or bad, i.e., whether the load is collapsing or a sign of such a collapse. Therefore, even an unskilled worker can easily determine whether it is OK to continue stacking the loads H. Furthermore, the loading and unloading work can be appropriately assisted based on the result of the determination.

Furthermore, according to this embodiment, the upper surface region S is extracted from the image using the machine-learned learning model 261.
By using the learning model 261 in this way, it is possible to suppress erroneous recognition of the top surface area S due to, for example, the background of the mounting pallet body G, and to achieve more accurate calculation with less load. In other words, while a method based on extracting the outline (edge) of a load on a camera image or image difference may not be able to accurately detect edges or differences when the background is complex or the environment changes, by using the learning model 261, it is possible to extract the top surface area S more accurately.
Furthermore, if items that can be used as cargo H are incorporated into the learning model 261 in advance, it is possible to appropriately handle cases where there are multiple types of cargo H.
For example, as shown in Fig. 9, even if the article H is not box-shaped (bag-shaped, rod-shaped, etc.), by having the system learn in advance, it is possible to suitably extract its top surface (or a portion corresponding to the top surface) as the top surface area S. Note that Fig. 9 illustrates an example in which the outline of the top surface of the article H is extracted as the top surface area S.
Alternatively, even if a load Ha with a damaged top surface or a load Hb of a different type from other loads H are mixed in, as shown in FIG. 10(a), these can be detected suitably by learning in advance. The detection result can be notified to the operator by displaying representative points Ma and Mb different from the representative point M of the load H, as shown in FIG. 10(b). The damaged load Ha and the load Hb of a different type can be identified based on information that can be obtained from the image, such as the size, shape, color, and pattern of the top surface. Alternatively, a load with a matching rate with the shape of the correct load H or the like of a predetermined value or less may be detected as a damaged item or a different type. Note that FIG. 10(a) illustrates a case in which the outline of the top surface of the loads Ha and Hb is extracted as the top surface areas Sa and Sb, and the center of gravity of the top surface areas Sa and Sb are set as the representative points Ma and Mb.

Furthermore, according to this embodiment, when an image including distance information is acquired, it is determined based on this image whether or not multiple loads H are placed within a predetermined height range.
This makes it possible to evaluate the arrangement of the loads H on the mounting pallet body G not only as a two-dimensional arrangement in a plan view, but also three-dimensionally, so that the quality of the mounting condition can be more appropriately determined.

In addition, according to this embodiment, the photographing unit 24 is arranged so that it can be raised and lowered integrally with the forks 12, and when it is raised together with the forks 12 holding the mounting pallet body G, an image of the top surface of multiple loads H on the mounting pallet body G is acquired.
Therefore, during a series of operations in which another mounting pallet body G is placed on a mounting pallet body G placed on the floor, images of the loads H on the mounting pallet body G on the floor can be acquired and the quality of the mounting condition can be judged. In other words, the quality of the mounting condition can be suitably judged without substantially reducing the work efficiency.

＜Other＞
Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment.
For example, the loading and unloading support process may evaluate the relative position of the load H with respect to the pallet P. This evaluation may be performed, for example, by extracting an area (outline) including the front surface of the pallet P as a pallet area L (see FIG. 6) from the image acquired in step S1 to obtain position information of the pallet P, and determining whether or not each load H on the top shelf is within a predetermined horizontal position range with respect to the pallet P. Extraction of the pallet area L may be performed based on a learning model or other image processing.

In addition, in the above embodiment, in step S2 of the loading and unloading support process, the upper surface area S is extracted from the image using the learning model 261, but the upper surface area S may also be extracted by image processing based on edge extraction, image difference, etc.

In the above embodiment, the case where the attached pallet body G, on which the load H is attached to the pallet P, is stacked up has been described.
However, the loading and unloading support process can also be applied to cases where a plurality of loads H are to be attached (stacked) on a pallet P, and cases where a mounted pallet body G is to be loaded and unloaded. In the former case, the loads H can be stacked on the pallet P while acquiring an image for each layer to check whether the mounted state (loading state) is good or bad. In the latter case, the mounted pallet body G can be loaded and unloaded while checking that there is little risk of the load collapsing on the mounted pallet body G lifted by the forks 12.

In the above embodiment, the cargo handling support device according to the present invention is mounted on the forklift 1, and all of the cargo handling support processing is performed on the forklift 1. However, the above embodiment is not limited to this.
However, at least a part of the loading/unloading support process may be executed in an external facility (for example, a management room in a loading/unloading work site). Specifically, for example, as shown in FIG. 11, a loading/unloading support system 100 is configured including a forklift 1 and a management device 5 configured to be able to communicate with each other through communication units 25, 55. Then, an image taken by the forklift 1 may be transmitted to the management device 5, and a result calculated by a control unit 57 of the management device 5 may be transmitted to the forklift 1. In this case, the management device 5 may manage image information and other information related to the top surface of the load H in a database. The management device 5 may be a control device (computer) in the loading/unloading work site, or may be a mobile terminal or the like. Data required for the process (such as the learning model 261) may be stored in the memory unit 56 of the management device 5. However, the degree of sharing of the process between the forklift 1 and the management device 5 in this case is not particularly limited.

In addition, in the above embodiment, the image of the load H is captured by the image capturing unit 24 mounted on the forklift 1, but the image capturing unit 24 may be installed (fixed) at the loading/unloading work site. In this case, the image captured by the image capturing unit 24 may be transmitted to the forklift 1 or the management device 5 that executes the loading/unloading support process. Also, a notification unit (display unit 23 in the above embodiment) that notifies the user of the quality of the loading/unloading condition may be installed (fixed) at the loading/unloading work site.

In the above embodiment, a forklift is used as an example of a loading vehicle, but the loading vehicle according to the present invention may be various heavy machinery that transports, loads, and unloads loads. The loading vehicle may be manually driven by an onboard (or remote) driver, or may be automatically driven. In the case of automatic driving, the quality of the loading condition is preferably notified to a remote management device. In the case of remote control, it is difficult for an operator to obtain information on the work site, making it difficult to make decisions regarding the work. However, the present invention can be suitably applied in that it is easy to make decisions such as continuing the work if it is possible to determine whether the loads are arranged in a predetermined arrangement.
Furthermore, the pallet on which the load is loaded is not limited to a pallet for a forklift, but may be any type of loading platform for transporting loads and capable of being transported by heavy machinery.
In addition, the details shown in the embodiment can be modified as appropriate without departing from the spirit of the invention.

1. Forklift (cargo handling vehicle)
5 Management device 12 Fork (holding part)
13 Lifting body 23 Display unit 24 Shooting unit (image acquisition unit)
26 Memory unit 27 Control unit (determination unit, support unit, second determination unit)
100 Cargo handling support system 261 Learning model 262 Pallet pattern data 263 Cargo handling support program B Planned position C Array pattern G Mounting pallet body H Cargo L Pallet area M Representative point P Pallet R Judgment area S Top surface area

Claims (9)

a determination unit that determines whether or not the plurality of articles are arranged in a predetermined arrangement based on an image of the upper surfaces of the plurality of articles placed on the pallet;
A support unit that supports a loading and unloading operation based on a determination result of the determination unit;
A loading and unloading support device. The determination unit is
From the images of the upper surfaces of the plurality of objects, representative points of the upper surfaces of the plurality of objects are set;
determining whether the representative points are arranged in a predetermined array;
The loading and unloading support device according to claim 1. The determination unit is
extracting a plurality of top surface regions each including the top surfaces of the plurality of packages from the images of the top surfaces of the plurality of packages using a learning model machine-learned from training data in which the images of the top surfaces of the packages are associated with lines surrounding the areas including the top surfaces;
determining the representative point for each of the plurality of upper surface regions;
The loading and unloading support device according to claim 2. A storage unit that stores in advance an appropriate arrangement pattern of the plurality of loads,
the determination unit compares the plurality of representative points with the arrangement pattern to determine whether the plurality of representative points are arranged in a predetermined arrangement.
The loading and unloading support device according to claim 2. An image acquisition unit that acquires images of the upper surfaces of the plurality of articles,
The predetermined array is a two-dimensional array in a plan view.
The loading and unloading support device according to claim 1. the image acquisition unit acquires an image including distance information,
a second determination unit that determines whether the plurality of objects are placed within a predetermined height range based on the image acquired by the image acquisition unit,
A loading and unloading support device according to claim 5. The loading and unloading support device according to claim 1 ;
A holding part capable of holding the pallet and capable of lifting and lowering;
A loading vehicle equipped with: A control unit for controlling the operation of the loading vehicle is provided,
The control unit permits loading and unloading operations on the plurality of loads when the determination unit determines that the plurality of loads are arranged in a predetermined arrangement.
A loading vehicle according to claim 7. The loading and unloading support device according to claim 1 ;
A management device capable of communicating with the loading and unloading support device;
Equipped with
The management device manages information on the top surface of the load in a database.
Cargo handling assistance system.

Images