JP2023163604A - Cargo handling system - Google Patents

Abstract

To enable a control device to determine a direction of cargo-handling operation.SOLUTION: A cargo handling vehicle is equipped with a cargo handling device, an external sensor and a control device. The external sensor detects a position of an object using coordinates of a three-dimensional coordinate system. The control device determines whether the cargo handling vehicle is positioned at a left side or a right side of a conveying vehicle, from point cloud data that is an assembly of points representing the position of the object. The control device determines a direction of cargo handling operation on the basis of the determined result of whether the cargo handling vehicle is positioned at the left side or the right side of the conveying vehicle. The direction of cargo handling operation is a direction in which the cargo handling device is operated, which differs depending on whether the cargo handling vehicle is positioned at the left side or the right side of the conveying vehicle.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to cargo handling systems.

The cargo handling vehicle disclosed in Patent Document 1 includes a cargo handling device. The cargo handling vehicle performs cargo handling work. Cargo handling operations include loading and unloading. During loading, a load is loaded onto a transport vehicle. In cargo picking, the cargo loaded on the transport vehicle is picked up by a cargo handling vehicle.

Japanese Patent Application Publication No. 2021-160860

The direction in which the cargo handling device is operated may differ depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle.

A cargo handling system that solves the above problems is a cargo handling system that includes an external sensor that detects the position of an object using coordinates in a three-dimensional coordinate system, a cargo handling vehicle equipped with a cargo handling device, and a control device, the control device being , from point cloud data that is a set of points representing the position of the object, it is determined whether the cargo handling vehicle is located on the left or right side of the transport vehicle, and the direction in which the cargo handling device is operated is determined. A cargo handling operation direction that differs depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle is determined depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle.

The control device determines from the point cloud data whether the cargo handling vehicle is located on the left or right side of the transport vehicle. The cargo handling operation direction differs depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle. By determining whether the cargo handling vehicle is located on the left or right side of the transport vehicle, the control device can determine the cargo handling operation direction.

Regarding the cargo handling system, the control device creates a side view of the transport vehicle from the point cloud data, extracts a driver's seat from the side view by image recognition, and extracts the driver's seat from the position of the driver's seat in the side view. It may be determined whether the cargo handling vehicle is located on the left or right side of the transport vehicle.

In the cargo handling system, the control device estimates the self-position of the cargo handling vehicle, and determines whether the cargo handling vehicle is located on the left or right side of the transportation vehicle based on the positional relationship between the self-position and the stop position of the transportation vehicle. It may be determined whether the

According to the present invention, the control device can determine the cargo handling operation direction.

FIG. 2 is a schematic diagram of an area where cargo handling vehicles are operated. FIG. 2 is a perspective view of a transport vehicle and a cargo handling vehicle. It is a perspective view of a side shift device. FIG. 1 is a schematic configuration diagram of a cargo handling vehicle. It is a flowchart which shows cargo handling operation direction determination control. It is a schematic diagram of a point cloud map. FIG. 3 is a schematic diagram of a side view. It is a schematic diagram of a work area. It is a flowchart which shows cargo handling operation direction determination control of a modification. FIG. 3 is a diagram showing the positional relationship between the transport vehicle and the cargo handling vehicle when the area of the stopping position is large.

An embodiment of the cargo handling system will be described below.
As shown in FIG. 1, a parking position PS1 is set in area A1. A plurality of parking positions PS1 are set. The parking positions PS1 are lined up at intervals from each other. The transport vehicle 10 stops at the stop position PS1. The area of the stop position PS1 is wider than the area of the conveyance vehicle 10 when the conveyance vehicle 10 stopped at the stop position PS1 is viewed from above. There are no structures such as pillars at the parking position PS1. The area A1 is, for example, the whole or a part of a place such as a factory, a port, an airport, a commercial facility, a public facility, or the like. In area A1, cargo handling vehicle 20 is operated. The cargo handling vehicle 20 performs loading and unloading. Loading is an operation in which the load C1 placed on the pallet PA1 is loaded onto the transport vehicle 10. Load picking is the work of picking up the load C1 loaded on the transport vehicle 10 from the transport vehicle 10. In the following description, the front, rear, left, right, top, and bottom of the transport vehicle 10 refer to the front, rear, left, right, top, and bottom when the transport vehicle 10 is used as a reference. The left, right, top, and bottom of the cargo handling vehicle 20 are the left, right, top, and bottom when the cargo handling vehicle 20 is used as a reference.

As shown in FIG. 2, the transport vehicle 10 is a wing truck. The transport vehicle 10 may be any type of truck, such as a flat body truck. The transport vehicle 10 includes a driver's seat 11, a front panel 12, a rear door 13, a loading platform 14, a wing side panel 16, and a gate 17.

The driver's seat 11 is a position where the driver of the transport vehicle 10 rides. The front panel 12 is provided behind the driver's seat 11 of the transport vehicle 10. The front panel 12 is provided adjacent to the driver's seat 11. The rear door 13 is provided behind the front panel 12 of the transport vehicle 10. The front panel 12 and the rear door 13 are spaced apart from each other in the longitudinal direction of the transport vehicle 10. The loading platform 14 extends in the longitudinal direction of the transport vehicle 10 between the front panel 12 and the rear door 13. The loading platform 14 includes a loading surface 15. The loading surface 15 is the upper surface of the loading platform 14. A load C1 placed on a pallet PA1 is loaded on the loading surface 15. The wing side panel 16 is provided between the front panel 12 and the rear door 13. The wing side panel 16 is provided so as to be rotatable in the vertical direction of the transport vehicle 10 around the center position of the transport vehicle 10 in the vehicle width direction. One wing side panel 16 is provided on each side of the transport vehicle 10 in the vehicle width direction. The gate 17 is provided so as to extend in the front-rear direction of the transport vehicle 10. The gate 17 is provided along the edge of the loading platform 14 and extending in the front-rear direction of the transport vehicle 10. One gate 17 is provided on each side of the transport vehicle 10 in the vehicle width direction.

<Cargo handling vehicle>
The cargo handling vehicle 20 includes a vehicle body 21 , drive wheels 22 , steering wheels 23 , and a cargo handling device 24 . The cargo handling device 24 is provided at the front of the vehicle body 21. The cargo handling device 24 includes a mast 25, a lift cylinder 28, a lift bracket 29, two forks 30, and a side shift device 40.

The mast 25 includes an outer mast 26 and an inner mast 27. The inner mast 27 is provided so as to be movable up and down relative to the outer mast 26. The lift bracket 29 and the fork 30 move up and down together with the inner mast 27. The lift cylinder 28 moves the inner mast 27 up and down. Lift cylinder 28 is a hydraulic cylinder.

The side shift device 40 moves the fork 30 in the left-right direction of the cargo handling vehicle 20. The side shift device 40 moves the two forks 30 in the left-right direction of the cargo handling vehicle 20 while maintaining the interval between the two forks 30.

As shown in FIG. 3, the side shift device 40 is attached to the lift bracket 29. The lift bracket 29 includes two finger bars 31 and 32. The two finger bars 31 and 32 are provided with an interval in the vertical direction of the cargo handling vehicle 20.

The side shift device 40 includes a shifter 41 and a shift cylinder 46.
The shifter 41 is provided so as to be movable to the left and right of the cargo handling vehicle 20 with respect to the two finger bars 31 and 32. The shifter 41 includes two shifter bars 42 and 43 and two connection members 44 and 45 that connect the two shifter bars 42 and 43. The two shifter bars 42 and 43 are provided with an interval in the vertical direction of the cargo handling vehicle 20. The two connecting members 44 and 45 are provided at intervals in the left-right direction of the cargo handling vehicle 20. Connecting members 44 and 45 connect the ends of shifter bars 42 and 43 to each other. A fork 30 is connected to the shifter bars 42 and 43.

Shift cylinder 46 is a hydraulic cylinder. By supplying and discharging hydraulic oil to the shift cylinder 46, the shifter 41 moves in the left-right direction of the cargo handling vehicle 20. The fork 30 also moves in the left-right direction of the cargo handling vehicle 20 together with the shifter 41.

As shown in FIG. 4, the cargo handling vehicle 20 includes an external sensor 51, a control device 52, an auxiliary storage device 55, a vehicle control device 56, a traveling actuator 59, a cargo handling actuator 60, and a communication device 61. Be prepared. The cargo handling vehicle 20 is a cargo handling system.

The external sensor 51 detects the position of an object using coordinates in a three-dimensional coordinate system. The external sensor 51 is provided at the top of the cargo handling vehicle 20. For example, the external sensor 51 is provided on the head guard of the cargo handling vehicle 20.

Examples of the external sensor 51 include a millimeter wave radar, a stereo camera, a ToF (Time of Flight) camera, and a LIDAR (Laser Imaging Detection and Ranging). In this embodiment, LIDAR is used as the external sensor 51. The external sensor 51 irradiates the surrounding area with a laser and receives the reflected light from the point hit by the laser to derive the distance to the point. The point hit by the laser represents a portion of the object's surface. The position of a point can be represented by coordinates in a polar coordinate system. The coordinates of a point in a polar coordinate system are transformed to coordinates in a rectangular coordinate system. The conversion from the polar coordinate system to the orthogonal coordinate system may be performed by the external sensor 51 or by the control device 52. In this embodiment, it is assumed that the external sensor 51 is converting from a polar coordinate system to an orthogonal coordinate system. The external sensor 51 derives the coordinates of a point in the sensor coordinate system. The sensor coordinate system is a three-axis orthogonal coordinate system with the external world sensor 51 as the origin. The external sensor 51 outputs the coordinates of a plurality of points obtained by laser irradiation to the control device 52 as point group data.

The control device 52 includes a processor 53 and a storage section 54. The storage unit 54 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). The storage unit 54 stores program codes or instructions configured to cause the processor 53 to execute processes. Storage 54, or computer readable media, includes any available media that can be accessed by a general purpose or special purpose computer. The control device 52 may be configured by a hardware circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control device 52, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as ASIC or FPGA, or a combination thereof.

The auxiliary storage device 55 stores information that can be read by the control device 52. Examples of the auxiliary storage device 55 include a hard disk drive and a solid state drive. The auxiliary storage device 55 stores an environmental map D1. The auxiliary storage device 55 stores an image recognition model D2.

The environmental map D1 is information regarding the physical structure of the area A1, such as the shape of objects existing in the area A1 and the size of the area A1. In this embodiment, the environmental map D1 is data representing the structure of the area A1 using coordinates in a map coordinate system. The map coordinate system is a three-axis orthogonal coordinate system. The map coordinate system is a coordinate system whose origin is an arbitrary point in area A1. In the map coordinate system, the horizontal direction is defined by an X-axis and a Y-axis that are perpendicular to each other. It can be said that the XY plane defined by the X axis and the Y axis represents a horizontal plane. In the map coordinate system, the vertical direction is defined by the Z axis orthogonal to the X axis and the Y axis. The coordinates of the map coordinate system are appropriately referred to as map coordinates. The map coordinate system is a three-dimensional coordinate system that represents a three-dimensional position.

The control device 52 estimates the self-position of the cargo handling vehicle 20. The control device 52 outputs the self-position of the cargo handling vehicle 20 to the vehicle control device 56. The self-position is the position of the cargo handling vehicle 20 on the environmental map D1. The self-position is a coordinate indicating one point of the cargo handling vehicle 20 in the map coordinate system. Although one point on the cargo handling vehicle 20 is arbitrary, for example, the central position of the cargo handling vehicle 20 in the horizontal direction can be mentioned.

The self-position is estimated by comparing the detection results of the external sensor 51 and the environmental map D1. The control device 52 extracts landmarks having the same shape as the landmarks obtained from the point cloud data from the environmental map D1. The control device 52 recognizes the positions of landmarks from the environmental map D1. The positional relationship between the position of the landmark and the cargo handling vehicle 20 can be ascertained from the detection result of the external sensor 51. Therefore, the control device 52 can estimate its own position by recognizing the position of the landmark. A landmark is an object that has characteristics that can be identified by the external sensor 51. A landmark is a physical structure whose position is difficult to change. Landmarks include, for example, walls and pillars. Estimating the self-position may be performed by combining estimation of the self-position using the external sensor 51 and dead reckoning using the internal sensor. The self-position estimation may be performed by combining self-position estimation using the external sensor 51 and self-position estimation using a satellite signal transmitted from a GNSS (Global Navigation Satellite System) satellite.

The vehicle control device 56 includes, for example, the same hardware configuration as the control device 52. Vehicle control device 56 includes a processor 57 and a storage section 58.
The travel actuator 59 is an actuator that causes the cargo handling vehicle 20 to travel. Travel actuator 59 includes, for example, a motor that rotates drive wheels 22 and a steering mechanism. The vehicle control device 56 controls the traveling actuator 59 to cause the cargo handling vehicle 20 to travel while grasping its own position.

The cargo handling actuator 60 is an actuator that causes the cargo handling vehicle 20 to perform cargo handling. The cargo handling actuator 60 includes, for example, a motor that drives a pump that supplies hydraulic oil to hydraulic equipment, and a control valve that controls the supply of hydraulic oil. The hydraulic equipment includes a lift cylinder 28 and a shift cylinder 46. The vehicle control device 56 controls the cargo handling actuator 60 to raise and lower the fork 30 and side shift the fork 30.

The cargo handling vehicle 20 automatically travels as the travel actuator 59 is controlled by the vehicle control device 56 . The cargo handling vehicle 20 automatically performs cargo handling by controlling the cargo handling actuator 60 by the vehicle control device 56. The cargo handling vehicle 20 is a self-driving forklift.

The communication device 61 is a communication device that can communicate using any wireless communication method, such as wireless LAN, Zigbee (registered trademark), LPWA (Low Power Wide Area), or a mobile communication system. The communication device 61 is capable of transmitting and receiving wireless signals. The communication device 61 outputs data obtained by demodulating the received radio signal to the vehicle control device 56.

Communication device 61 receives commands from higher-level control device 71 . The upper control device 71 is provided in area A1, for example. The upper control device 71 transmits operation data input by a person to the communication device 61. The operation data includes instructions for the cargo handling vehicle 20. Examples of the command include a command to cause the cargo handling vehicle 20 to proceed, a command to cause the cargo handling vehicle 20 to start loading, and a command to cause the cargo handling vehicle 20 to start picking up cargo. The vehicle control device 56 controls the cargo handling vehicle 20 according to commands from the host control device 71.

The cargo handling operation direction determination control performed by the control device 52 will be explained. The cargo handling operation direction determination control is control for determining the cargo handling operation direction. The cargo handling operation direction is a direction in which the cargo handling device 24 is operated, and is a direction that differs depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. An example of the cargo handling direction is the side shift direction of the fork 30. When the cargo handling vehicle 20 loads cargo, the cargo C1 is sequentially loaded onto the loading platform 14 of the carrier vehicle 10 from the front to the rear of the carrier vehicle 10. The load C1 is loaded from a position close to the driver's seat 11. At this time, in order to reduce the gap between the loads C1 loaded on the loading platform 14, the fork 30 is side-shifted by the side shift device 40. When loading cargo and the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is to the right of the cargo handling vehicle 20. When loading cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is to the left of the cargo handling vehicle 20. When the cargo handling vehicle 20 picks up cargo, it sequentially picks up the cargo C1 loaded on the loading platform 14 of the carrier vehicle 10 from the rear to the front of the carrier vehicle 10. If the gap between the loads C1 is small, there is a risk that adjacent loads C1 will interfere with each other, so the fork 30 is side-shifted by the side shift device 40 in order to increase the gap between the loads C1. When picking up cargo and the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is to the left of the cargo handling vehicle 20. When picking up cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is to the right of the cargo handling vehicle 20.

The cargo handling vehicle 20 approaches the transport vehicle 10 based on a command from the higher-level control device 71, for example. Then, when the separation distance between the cargo handling vehicle 20 and the transport vehicle 10 becomes less than a predetermined distance, cargo handling operation direction determination control is started. The predetermined separation distance is, for example, a distance that can be detected by the transport vehicle 10 by the external sensor 51.

<Cargo handling operation direction determination control>
As shown in FIGS. 5 and 6, in step S1, the control device 52 creates a point cloud map PM1. The point cloud map PM1 is obtained by superimposing point cloud data obtained from the detection results of the external sensor 51. The control device 52 acquires point cloud data from the external sensor 51 multiple times while the cargo handling vehicle 20 is moving. The control device 52 converts the coordinates of each point P1 of the point cloud data from the sensor coordinate system to map coordinates based on the self-position. From the self-position estimated by the control device 52, the origin of the sensor coordinate system in the map coordinate system can be recognized. From the self-position estimated by the control device 52, the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system can be recognized. The control device 52 converts each point P1 of the point cloud data from the coordinates of the sensor coordinate system to the map coordinates based on the origin of the sensor coordinate system in the map coordinate system and the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system. Convert to Each time the control device 52 acquires point cloud data, it creates a point cloud map PM1 by superimposing each point P1 converted into map coordinates. The point cloud map PM1 is a collection of point cloud data. The points P1 of the point cloud map PM1 are denser than the points P1 of the point cloud data.

FIG. 6 shows a point cloud map PM1 obtained by the process of step S1. Each point P1 included in the point cloud map PM1 represents the map coordinates of an object. For convenience of explanation, the explanation will be given by classifying each point P1 included in the point cloud map PM1 into a first point P11, a second point P12, a third point P13, and a fourth point P14. The first point P11 is a point P1 obtained by irradiating the tilt 17 with a laser. The second point P12 is a point P1 obtained by irradiating the driver's seat 11 with a laser. The third point P13 is a point P1 obtained by irradiating the pallet PA1 and the load C1 loaded on the loading surface 15 with a laser. The fourth point P14 is a point P1 that does not correspond to any of the first point P11, the second point P12, and the third point P13.

As shown in FIGS. 5 and 7, in step S2, the control device 52 creates a side view IM1 from the point cloud map PM1. Since the point cloud map PM1 is a collection of point cloud data, it can be said that the side view IM1 is created from the point cloud data. Side view IM1 is a diagram in which point P1 is projected in the vehicle width direction of transport vehicle 10. As a result, the point cloud map PM1 can be made two-dimensional, so that the side view IM1 can be treated as image data. An example of processing performed by the control device 52 when creating the side view IM1 will be described.

The control device 52 calculates the normal vector of each point P1. The normal vector is a vector directed in a direction perpendicular to the plane surrounded by the plurality of points P1. Examples of methods for deriving the normal vector include a method of finding a curved surface from each point P1 and deriving the normal vector of each point P1 from the curved surface, and a method of using the cross product of vectors. For example, when the control device 52 determines the normal vector of one point P1, it determines the cross product of vectors directed from this point P1 to each of two points P1 located within a predetermined range. This cross product is the normal vector.

The control device 52 extracts a point P1 whose normal vector is oriented in the horizontal direction. For example, the control device 52 determines whether the angle of the normal vector to the XY plane of the map coordinate system falls within a predetermined range. When the direction of the normal vector is horizontal, the normal vector is parallel to the XY plane of the map coordinate system. A predetermined range is set so that a point P1 where the direction of the normal vector can be regarded as horizontal can be extracted, taking into consideration the inclination of the transport vehicle 10 and measurement errors.

The control device 52 derives a plane equation from a point P1 whose normal vector is in the horizontal direction. The plane equation can be derived, for example, by using a robust estimation method such as RANSAC (Random Sample Consensus) or the least squares method. A plane represented by a plane equation is a plane that extends in the vertical direction. In the transport vehicle 10 of this embodiment, a flat surface can be obtained by the first point P11 caused by the tilt 17. Further, when picking up cargo, since the pallet PA1 and the cargo C1 are loaded on the transport vehicle 10, a flat surface can be obtained by the third point P13 of the pallet PA1 and the cargo C1. The control device 52 creates a side view IM1 by projecting a point P1 in a direction perpendicular to a plane expressed by a plane equation.

Next, in step S3, the control device 52 determines the position of the driver's seat 11 in the side view IM1 by extracting the driver's seat 11 from the side view IM1. The position of the driver's seat 11 in the side view IM1 can be represented by an image coordinate system. The image coordinate system is a coordinate system in which the horizontal direction of the side view IM1 is the X axis, and the vertical direction is the Y axis. The position of the driver's seat 11 is determined using image recognition. In this embodiment, a case will be described in which image recognition is performed using the image recognition model D2, but image recognition may also be performed by pattern matching.

The image recognition model D2 is a trained model generated by machine learning. The image recognition model D2 uses an algorithm that can determine the class of an object on a region-by-region basis. The class is set to "driver's seat." Examples of machine learning algorithms include SSD (Single Shot Multibox Detector), R-CNN (Regional Convolutional Neural Network), fast R-CNN, faster R-CNN, and YOLO (You Only Look Once). . The image recognition model D2 is generated by supervised learning using teacher data or semi-supervised learning. As the teacher data, data including image data including an object corresponding to a class, the position of the object in the image data, and a label is used. The teacher data can be generated, for example, by surrounding an object in image data with a frame and attaching a label to the image data. In this embodiment, the teacher data may be obtained by enclosing the driver's seat 11 in the image data with a frame and attaching a label of "driver's seat" to the image data. The image data used as the teacher data may be obtained from the point cloud map PM1 similarly to the side view IM1, or may be obtained by imaging with an imaging device.

The image recognition model D2 specifies a region including the driver's seat 11 from the input side view IM1. The area including the driver's seat 11 is represented by a bounding box B1. In the example shown in FIG. 7, the driver's seat 11 represented by the second point P12 is surrounded by a bounding box B1.

As shown in FIGS. 5 and 8, in step S4, the control device 52 determines work areas A11 and A12. Specifically, the control device 52 generates a boundary straight line BL1 that passes through the transport vehicle 10 in the front-rear direction of the transport vehicle 10 in the map coordinate system. The control device 52 determines both sides of the boundary straight line BL1 as work areas A11 and A12. The work areas A11 and A12 are, for example, areas of a predetermined size. The work areas A11 and A12 include a right work area and a left work area. The right work areas are the right work areas A11 and A12 of the transport vehicle 10. The left work areas are work areas A11 and A12 on the left of the transport vehicle 10.

When the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the driver's seat 11 is located on the right side in the side view IM1. If the driver's seat 11 is located to the right of the center position in the X-axis direction of the image coordinate system in the side view IM1, the control device 52 determines that the cargo handling vehicle 20 is located to the right of the transport vehicle 10. . In this case, the control device 52 can determine that the work areas A11 and A12 where the cargo handling vehicle 20 is located are the right work areas. The control device 52 can determine that the work areas A11 and A12 where the cargo handling vehicle 20 is not located are the left work areas.

When the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the driver's seat 11 is located on the left side in the side view IM1. If the driver's seat 11 is located to the left of the center position in the X-axis direction of the image coordinate system in the side view IM1, the control device 52 determines that the cargo handling vehicle 20 is located to the left of the transport vehicle 10. . In this case, the control device 52 can determine that the work areas A11 and A12 where the cargo handling vehicle 20 is located are the left work areas. The control device 52 can determine that the work areas A11 and A12 where the cargo handling vehicle 20 is not located are the right work areas.

Next, in step S5, the control device 52 determines the direction of cargo handling operation. The case where the side shift direction is determined as the cargo handling operation direction will be explained as an example. As described above, the direction of cargo handling operation differs depending on whether the cargo handling vehicle 20 loads or picks up cargo. When the cargo handling vehicle 20 loads cargo in the right work area, the control device 52 determines the cargo handling operation direction to be to the right of the cargo handling vehicle 20. When the cargo handling vehicle 20 loads cargo in the left work area, the control device 52 determines the cargo handling operation direction to be to the left of the cargo handling vehicle 20. When the cargo handling vehicle 20 picks up cargo in the right work area, the control device 52 determines the cargo handling operation direction to be to the left of the cargo handling vehicle 20 . When the cargo handling vehicle 20 picks up cargo in the left work area, the control device 52 determines the cargo handling operation direction to be to the right of the cargo handling vehicle 20 .

After completing the process of step S5, the control device 52 ends the cargo handling operation direction determination control. The control device 52 outputs the cargo handling operation direction determined by the cargo handling operation direction determination control to the vehicle control device 56. The vehicle control device 56 performs control according to the cargo handling direction.

[Effects of this embodiment]
(1) The control device 52 determines whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from the point cloud data. The control device 52 can determine the direction of cargo handling operation depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

(2) The control device 52 extracts the driver's seat 11 from the side view IM1 by image recognition. The position of the driver's seat 11 differs depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. By extracting the driver's seat 11 from the side view IM1, it is possible to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

(3) When the stopping position PS1 of the transport vehicle 10 is determined and the direction in which the transport vehicle 10 stops is determined, the control device 52 determines whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from its own position. You can determine whether However, as shown in FIG. 1, when the cargo handling vehicle 20 is located between two stopping positions PS1, when one of the two transport vehicles 10 is used as a reference, it is on the right, and when the other is used as a reference, it is on the right. A cargo handling vehicle 20 is located on the left. For this reason, when the cargo handling vehicle 20 is located between the two stopping positions PS1, it cannot be determined whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. On the other hand, when extracting the driver's seat 11 from the side view IM1 by image recognition, even if the cargo handling vehicle 20 is located between the two stopping positions PS1, the cargo handling vehicle 20 You can determine whether it is located on the left or right.

(4) The vehicle control device 56 controls the cargo handling vehicle 20 according to commands from the host control device 71. The command is included in the operation data input by a person. If the cargo handling direction is not determined by the control device 52, it is necessary for a person to specify the cargo handling direction when inputting operation data. At this time, there is a risk that a person may input the cargo handling direction incorrectly. On the other hand, since the control device 52 determines the direction of the cargo handling operation, it is possible to prevent a person from inputting the direction of the cargo handling operation erroneously. It is possible to prevent malfunction of the cargo handling vehicle 20 due to a person erroneously inputting the direction of cargo handling operation. Furthermore, the number of man-hours required to create operation data can be reduced.

(5) It can be determined whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 using the external sensor 51 used for estimating its own position. The number of parts can be reduced compared to the case where a dedicated sensor is used to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

[Example of change]
The embodiment can be modified and implemented as follows. The embodiments and the following modification examples can be implemented in combination with each other within a technically consistent range.

The cargo handling operation direction determination control may be control that determines the cargo handling direction from the positional relationship between the vehicle's own position and the stop position PS1.
As shown in FIG. 9, in step S11, the control device 52 estimates its own position. Estimation of self-position is the same process as in the embodiment.

Next, in step S12, the control device 52 determines the cargo handling direction based on the positional relationship between the vehicle's own position and the stop position PS1. When determining the cargo handling direction based on the positional relationship between the vehicle's own position and the stop position PS1, it is necessary to determine the direction in which the transport vehicle 10 should stop. Thereby, when the guided vehicle 10 is stopped at the stop position PS1, it can be determined whether the self position corresponds to the left or right side of the guided vehicle 10. It can be said that the control device 52 can determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 based on its own position. The control device 52 can determine the cargo handling direction depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

When determining the cargo handling direction based on the positional relationship between the vehicle's own position and the parking position PS1, it is preferable that the area of the parking position PS1 is narrow. When the stop position PS1 has a large area, even if the cargo handling vehicle 20 is at the same position, whether the cargo handling vehicle 20 is located on the left or right side of the transportation vehicle 10 may differ depending on the position of the transportation vehicle 10. For example, as shown in FIG. 10, it is assumed that the transport vehicle 10 is stopped at a first position A2 of the stop position PS1, and the cargo handling vehicle 20 is located to the right of the transport vehicle 10. It is assumed that the positions of the cargo handling vehicles 20 are the same, and the transport vehicle 10 has stopped at a second position A3 among the stopping positions PS1. The first position A2 and the second position A3 are positions that sandwich the cargo handling vehicle 20 therebetween. In this case, the cargo handling vehicle 20 is located to the left of the transport vehicle 10 stopped at the second position A3. It is preferable to set the area of the stop position PS1 so that whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 does not differ depending on the position of the transport vehicle 10.

As the image recognition model D2, an algorithm that can determine the class of an object for each side view IM1 may be used. In this case, "rightward" and "leftward" may be set as the classes. An example of a machine learning algorithm is CNN (Convolution Neural Network). As the training data, it is sufficient to use image data in which the conveyance vehicle 10 facing right is labeled as "facing right" and image data in which the conveyance vehicle 10 facing left is labeled as "facing left". .

The control device 52 inputs the side view IM1 into the image recognition model D2 to determine whether the conveyance vehicle 10 shown in the side view IM1 is facing right or left. When the cargo handling vehicle 20 is located to the right of the conveyance vehicle 10, the conveyance vehicle 10 in the side view IM1 appears to the right. When the cargo handling vehicle 20 is located to the left of the transport vehicle 10, the transport vehicle 10 in the side view IM1 appears facing left. Therefore, the control device 52 can determine whether the cargo handling vehicle 20 is located on the right or left of the transport vehicle 10 from the output of the image recognition model D2.

- The control device 52 does not need to create the point cloud map PM1. In this case, the control device 52 converts the point cloud data acquired from the external sensor 51 into map coordinates, and performs the processing from step S2 onwards. That is, the control device 52 determines whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from a single point cloud data without creating a point cloud map PM1 that is a collection of a plurality of point cloud data. You may judge.

The cargo handling device 24 may be equipped with a robot arm, for example. The cargo handling operation direction in this case may be the direction in which the robot arm is operated. For example, the direction in which the cargo C1 is loaded by the robot arm may be the cargo handling operation direction.

The transport vehicle 10 may be any vehicle that can perform transport, and may be, for example, an AGV (Automatic Guided Vehicle) or an AMR (Autonomous Mobile Robot). In this case, the control device 52 uses the image recognition model D2 to determine whether the transport vehicle 10 is facing right or left, thereby determining whether the cargo handling vehicle 20 is located on the right or left of the transport vehicle 10. It is also possible to determine whether the The control device 52 may determine whether the cargo handling vehicle 20 is located on the right or left of the transport vehicle 10 based on its own position.

The vehicle control device 56 may perform part of the cargo handling operation direction determination control. In this case, the vehicle control device 56 can also be said to be a control device.
The higher-level control device 71 may perform part of the cargo handling operation direction determination control. In this case, the higher-level control device 71 can also be said to be a control device. The host control device 71 may perform all the processing of the cargo handling operation direction determination control. In this case, the higher-level control device 71 is the control device. In this way, when the higher-level control device 71 performs at least part of the processing of the cargo-handling operation direction determination control, the cargo-handling vehicle 20 and the higher-level control device 71 constitute the cargo-handling system.

P1... Point, 10... Transport vehicle, 11... Driver's seat, 20... Cargo handling vehicle which is a cargo handling system, 24... Cargo handling device, 51... External world sensor, 52... Control device.

Claims (3)

A cargo handling vehicle equipped with an external sensor that detects the position of an object using coordinates in a three-dimensional coordinate system, and a cargo handling device;
A cargo handling system comprising a control device,
The control device includes:
Determining whether the cargo handling vehicle is located on the left or right side of the transport vehicle from point cloud data that is a set of points representing the position of the object,
The cargo handling operation direction is a direction in which the cargo handling device is operated and is a direction that differs depending on whether the cargo handling vehicle is located on the left or right side of the transport vehicle. Depending on the cargo handling system. The control device includes:
creating a side view of the transport vehicle from the point cloud data;
The driver's seat is extracted from the side view by image recognition,
The cargo handling system according to claim 1, wherein it is determined whether the cargo handling vehicle is located on the left or right side of the transport vehicle based on the position of the driver's seat in the side view. The control device estimates the self-position of the cargo handling vehicle,
The cargo handling system according to claim 1, wherein it is determined whether the cargo handling vehicle is located on the left or right side of the carrier vehicle based on the positional relationship between the own position and the stop position of the carrier vehicle.

Images