JP2021160931A - Cargo handling system - Google Patents

Abstract

To guide a fork lift to a movement target.SOLUTION: A cargo handling system comprises: a fork lift; a camera; and a superior control unit. The camera is arranged so that, a stop position where a truck transporting a pallet stops, is imaged. The stop position is determined in advance. The superior control unit calculates a coordinate of a pallet in a camera coordinate system from the image data acquired by imaging by the camera. The superior control unit calculates a coordinate of the pallet in a map coordinate system, from the coordinate of the pallet in the camera coordinate system. The superior control unit transmits the coordinate of the pallet in the map coordinate system, to a fork lift from a command communication part. The control unit moves the fork lift to the coordinate of the pallet in the map coordinate system.SELECTED DRAWING: Figure 7

Description

The present invention relates to a cargo handling system.

Forklifts that automatically operate without being operated by an operator are described in, for example, Patent Document 1. The forklift described in Patent Document 1 includes an in-vehicle camera and a control device that detects the position and orientation of the pallet from image data obtained by imaging the in-vehicle camera.

Japanese Unexamined Patent Publication No. 2020-1906

When a pallet loaded on a transport vehicle such as a truck is transported to another place or another device by a forklift, the forklift performs a loading operation of loading the pallet loaded on the transport vehicle on the fork. When the pallet is transported to another place by the transport vehicle, the forklift performs a loading operation of loading the pallet in the transport vehicle or a container connected to the transport vehicle. When the forklift that operates automatically as described in Patent Document 1 is to perform the loading work, it is conceivable to set the stop position at which the transport vehicle stops in advance. When the carrier arrives at the stop position, the forklift moves toward the stop position. When the forklift approaches the stop position and the pallet is imaged by the in-vehicle camera, the position of the pallet is detected by the control device. When the control device detects the position of the pallet, the control device controls the forklift so that the forklift moves toward the pallet. As described above, even when the stop position is determined, the position of the pallet loaded on the transport vehicle is indefinite, so that the position of the pallet cannot be detected unless the forklift approaches the stop position. Even when the forklift that operates automatically is to perform the loading work, it is conceivable to set the stop position at which the transport vehicle stops in advance. When the forklift approaches the stop position, the in-vehicle camera captures at least one of the transport vehicle and the container. The control device sets either the transport vehicle or the container as the movement target, and detects the position of the movement target. In this way, even when the forklift performs loading work, the position of the transport vehicle cannot be detected unless the forklift approaches the stop position.

An object of the present invention is to provide a cargo handling system capable of guiding a forklift to a moving target.

The cargo handling system that solves the above problems includes a sensor, a forklift, and the pallet that are provided so that the stop position at which the carrier that transports the pallet is stopped and the predetermined stop position is included in the detectable range. A cargo handling system including a host control device configured to guide the forklift to the movement target with the transport vehicle or container as a movement target, wherein the forklift is an environment in which the forklift is used. A storage device configured to store an environment map represented by coordinates in the map coordinate system, a receiving unit configured to be able to receive information transmitted from the higher-level control device, and the receiving unit received by the receiving unit. The upper control device includes a movement control unit that moves the forklift toward the coordinates of the movement target, which is the coordinates of the map coordinate system obtained from the information, and the upper control device is the map coordinate system based on the detection result of the sensor. It is provided with a derivation unit for deriving information for obtaining the coordinates of the moving target in the above, and a transmitting unit for transmitting information for obtaining the coordinates of the moving target in the map coordinate system to the receiving unit.

When the transport vehicle stops at the stop position, the host control device derives information from the detection result of the sensor that can obtain the coordinates of the movement target in the map coordinate system. The forklift moves toward the coordinates of the movement target in the map coordinate system by transmitting the information that the upper control device can obtain the coordinates of the movement target in the map coordinate system to the forklift. By providing a sensor so that the movement target can be detected and deriving information that can obtain the coordinates of the movement target in the map coordinate system with the upper control device, it is possible to guide the forklift to the movement target using the upper control device. can.

In the cargo handling system that solves the above problems, the upper control device has a map coordinate deriving unit that derives the coordinates of the moving target in the map coordinate system as information for obtaining the coordinates of the moving target in the map coordinate system. You may have.

Regarding the cargo handling system, the sensor is arranged so that two fixed structures are included in the detectable range, and the higher-level control device is said to have the sensor coordinate system of the sensor based on the detection result of the sensor. A sensor coordinate derivation unit for deriving the coordinates of the moving target is provided, and the two fixed structures are provided so as to be located on the same coordinate axis of the map coordinate system, and the two fixed structures are of the two fixed structures. One is provided so as to be located at the origin of the map coordinate system, and the sensor coordinate deriving unit derives the coordinates of the two fixed structures and the moving target in the sensor coordinate system. The map coordinate derivation unit uses the coordinates of the two fixed structures in the sensor coordinate system and the coordinates of the movement target in the sensor coordinate system to perform a three-sided survey to determine the movement target in the map coordinate system. The coordinates of may be derived.

Regarding the cargo handling system, the sensor is a camera, and the camera is arranged so as to image a fixed marker provided on each of the two fixed structures and a marker provided on the moving target. The sensor coordinate derivation unit derives the coordinates of the fixed marker in the sensor coordinate system as the coordinates of the fixed structure in the sensor coordinate system, and sets the coordinates of the marker in the sensor coordinate system as the movement target. May be derived as the coordinates in the sensor coordinate system of.

Regarding the cargo handling system, the sensor is a camera, and the upper control device derives the coordinates of the moving target in the sensor coordinate system of the sensor by the learned model generated by inputting the teacher data into the learner. Then, in the trained model, data in which the coordinates of the moving target in the sensor coordinate system are added as labels to the image data obtained by capturing the moving target may be generated as the teacher data.

According to the present invention, the forklift can be guided to the moving target.

A schematic view of the cargo handling system according to the first embodiment, and a schematic view of a workplace where the cargo handling system is used. A side view showing a truck stopped at a stop position. Side view of the forklift. The perspective view which shows by breaking a part of the forklift. The schematic block diagram of the cargo handling system in 1st Embodiment. The figure which shows the positional relationship between a truck and a forklift when carrying out a unloading work. The interaction diagram which shows the control performed in the cargo handling system in 1st Embodiment. The figure for demonstrating the process performed by the upper control unit when deriving the map coordinates of a pallet. The figure which shows the image data which is input to a learner when generating a trained model. A schematic view of the cargo handling system according to the third embodiment, and a schematic view of a workplace where the cargo handling system is used. The perspective view which shows typically the container which is a movement target. FIG. 11 is a cross-sectional view taken along the line 12-12 of FIG. 11 showing the container. The schematic block diagram of the cargo handling system in 3rd Embodiment. The figure for demonstrating the arrangement position of a laser range finder. The interaction diagram which shows the control performed in the cargo handling system in 3rd Embodiment. The figure which shows an example of the irradiation point obtained when irradiating a container with a laser. The figure for demonstrating the process performed by the upper control device for extracting an edge irradiation point. Sectional drawing which shows the pallet which is a movement target.

(First Embodiment)
Hereinafter, the first embodiment of the cargo handling system will be described. In the following description, up and down means up and down in the vertical direction.

As shown in FIG. 1, the cargo handling system CS includes a forklift 10, a camera 60, and a host control device 70. The forklift 10 is used in a workplace such as a factory, a harbor, an airport, or a commercial facility where a pallet P needs to be transported. The forklift 10 transports the pallet P to another place or another device after performing the loading work of loading the pallet P loaded on the truck T onto the forklift 10. The stop position A1 of the truck T is predetermined. The forklift 10 performs the unloading operation after moving to the unloading position A2. The unloading position A2 is a front position of the pallet P to be unloaded.

Two pillars Pi1 and Pi2 are provided in the work place with the stop position A1 in between. The fact that the two pillars Pi1 and Pi2 sandwich the stop position A1 means that the two pillars Pi1 and Pi2 are stopped when viewed from the horizontal direction orthogonal to the direction in which the two pillars Pi1 and Pi2 are lined up. It means that the position A1 is sandwiched between them. The two pillars Pi1 and Pi2 are existing structures originally provided in the workplace. The two pillars Pi1 and Pi2 are fixed structures. A fixed structure is a structure whose position does not change or does not change easily with the passage of time. The two pillars Pi1 and Pi2 are, for example, pillars that support the eaves. It can be considered that the stop position A1 is set between the two pillars Pi1 and Pi2. It should be noted that between the two pillars Pi1 and Pi2, the stop position A1 is located between the two pillars Pi1 and Pi2 when viewed from the horizontal direction orthogonal to the direction in which the two pillars Pi1 and Pi2 are lined up. Means to be.

Fixed markers MA1 and MA2 are provided on each of the two pillars Pi1 and Pi2. The fixed markers MA1 and MA2 provided on the two pillars Pi1 and Pi2 are provided so as to face the same direction. The fixed markers MA1 and MA2 include an AR marker or QR code (registered trademark) which is an identification image including a plurality of rectangles or a plurality of dots arranged two-dimensionally, and a marker by a illuminant emitting light at a predetermined frequency. , Various markers such as the image of the object itself can be used. The fixed markers MA1 and MA2 are markers whose positions in the workplace do not change. In the following description, one of the two pillars Pi1 and Pi2 is the first pillar Pi1, the other is the second pillar Pi2, and the fixed marker MA1 provided on the first pillar Pi1 is used as the first fixed marker MA1 and the second pillar Pi2. The provided fixed marker MA2 will be described as a second fixed marker MA2.

As shown in FIG. 2, the track T is a flat body track. The truck T includes a loading platform TB, a side tilt SS, a rear tilt RS, and a tire T1. A plurality of pallets P are stacked on the loading platform TB. The truck T is a transport vehicle that conveys the pallets P loaded on the loading platform TB. The side tilt SS is provided on the side of the loading platform TB. The side tilt SS can rotate in the vertical direction. The rear tilt RS is provided at the rear of the loading platform TB. The rear tilt RS can rotate in the vertical direction. While the truck T is running, the loading platform TB is surrounded by the side tilt SS and the rear tilt RS. When the forklift 10 performs the loading work, the side tilt SS and the rear tilt RS are rotated downward, and the side tilt SS and the rear tilt RS do not face the pallet P. That is, when the forklift 10 performs the loading work, the side tilt SS and the rear tilt RS are rotated so as not to interfere with the loading work by the forklift 10.

The pallet P includes a square box-shaped accommodating portion S for accommodating the transported object, and leg portions L provided at the four corners of the accommodating portion S. The pallet P of this embodiment is a mesh pallet. Although not shown, the accommodating portion S accommodates the transported object. Markers MA3 and MA4 are provided on the pallet P. Two markers MA3 and MA4 are provided for each pallet P. For the markers MA3 and MA4 provided on the same palette P, it is preferable that markers having different patterns are used. Further, it is more preferable that all the markers MA3 and MA4 provided on each pallet P mounted on the loading platform TB have different patterns. The markers MA3 and MA4 include an AR marker or QR code which is an identification image including a plurality of rectangles or a plurality of dots arranged two-dimensionally, a marker by a light emitter emitting light at a predetermined frequency, and an image of an object itself. Various markers can be used. The pallets P are loaded on the loading platform TB in such a manner that the markers MA3 and MA4 face the vehicle width direction of the truck T.

The cargo handling system CS is a system that smoothly moves the forklift 10 to the pallet P by guiding the forklift 10 at a position A3 different from the loading position A2 to the position of the pallet P loaded on the truck T. In the first embodiment, the pallet P is the movement target.

As shown in FIGS. 3 and 4, the forklift 10 includes the vehicle body 11, the drive wheels 12 arranged at the lower front part of the vehicle body 11, the steering wheels 13 arranged at the lower rear part of the vehicle body 11, and the front of the vehicle body 11. The cargo handling device 14 provided in the vehicle and the vehicle-mounted camera 45 are provided. The cargo handling device 14 includes a mast 15 provided at the front portion of the vehicle body 11, a backrest 18, a pair of forks 31 fixed to the backrest 18, a lift cylinder 20, and a tilt cylinder 21.

The mast 15 includes an outer mast 16 that is supported so as to be tilted back and forth with respect to the vehicle body 11, and an inner mast 17 that can be raised and lowered with respect to the outer mast 16. The backrest 18 is fixed to the inner mast 17. The backrest 18 and the fork 31 move up and down together with the inner mast 17. The lift cylinder 20 raises and lowers the inner mast 17. The tilt cylinder 21 tilts the mast 15. The lift cylinder 20 and the tilt cylinder 21 are hydraulic cylinders. The pallet P is loaded on the fork 31. The fork 31 is inserted between the legs L and supports the bottom of the accommodating portion S.

The in-vehicle camera 45 is provided between the two forks 31 in the vehicle width direction of the forklift 10. The in-vehicle camera 45 is provided so as to move up and down together with the fork 31. In this embodiment, the vehicle-mounted camera 45 is provided on the backrest 18. The in-vehicle camera 45 is arranged so as to take an image of the front of the forklift 10. The in-vehicle camera 45 is a monocular camera.

As shown in FIG. 5, the forklift 10 includes a drive mechanism 51, a hydraulic mechanism 52, a control device 53, an auxiliary storage device 56, an environment sensor 58, and a communication unit 59.
The drive mechanism 51 is a member for driving the forklift 10 and includes a traveling motor for driving the drive wheels 12 and a steering mechanism for steering the steering wheels 13.

The hydraulic mechanism 52 is a member for controlling the supply and discharge of hydraulic oil to the lift cylinder 20 and the tilt cylinder 21, and includes a cargo handling motor for driving a pump and a control valve. The inner mast 17 moves up and down by supplying and discharging hydraulic oil to the lift cylinder 20. The mast 15 tilts in the front-rear direction of the vehicle body 11 due to the supply and discharge of hydraulic oil to the tilt cylinder 21. The raising and lowering of the inner mast 17 can be said to be the raising and lowering of the fork 31. The tilt of the mast 15 can be said to be the tilt of the fork 31.

The control device 53 includes a processor 54 such as a CPU and a GPU, and a storage unit 55 composed of a RAM, a ROM, and the like. The storage unit 55 stores a program code or a command configured to cause the processor 54 to execute the process. The storage 55, i.e., a computer-readable medium, includes any available medium accessible by a general purpose or dedicated computer. The control device 53 may be configured by a hardware circuit such as an ASIC: Application Specific Integrated Circuit or an FPGA: Field Programmable Gate Array. The control device 53, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or FPGA, or a combination thereof.

The control device 53 operates the forklift 10 by controlling the drive mechanism 51 and the hydraulic mechanism 52 according to the program stored in the storage unit 55. The forklift 10 is a forklift that automatically operates under the control of the control device 53 without being operated by the operator.

An in-vehicle camera 45 is connected to the control device 53. The image data obtained by the imaging of the in-vehicle camera 45 is input to the control device 53.
The auxiliary storage device 56 stores information that can be read by the control device 53. As the auxiliary storage device 56, for example, a hard disk drive or a solid state drive is used. The program code or command stored in the storage unit 55 may be stored in the auxiliary storage device 56 instead of the storage unit 55.

The auxiliary storage device 56 as a storage device stores information indicating an environmental map. The environmental map is information on the physical structure of the surrounding environment of the forklift 10, such as the shape and size of the environment in which the forklift 10 is used. The environment in which the forklift 10 is used includes the stop position A1, and it can be said that the environment map shows the surrounding environment of the stop position A1. It can be said that the environment map is data in which the environment in which the forklift 10 is used, that is, the work place shown in FIG. 1 is represented by the coordinates in the map coordinate system. More specifically, the map coordinate system represents the position of the forklift 10 with respect to the physical structure of the environment in which it is used, with coordinates relative to the origin. In this embodiment, the origin of the map coordinate system is the first fixed marker MA1. Of the coordinate axes of the map coordinate system, the X axis is an axis extending in the direction in which the first fixed marker MA1 and the second fixed marker MA2 are arranged, and the Y axis of the coordinate axes of the map coordinate system is orthogonal to the X axis in the horizontal direction. It is an axis that extends in the direction. Since the first fixed marker MA1 is provided on the first pillar Pi1, the origin of the map coordinate system can be said to be the first pillar Pi1. Since the second fixed marker MA2 is provided on the second pillar Pi2, it can be said that the X-axis of the coordinate axes of the map coordinate system extends in the direction in which the first pillar Pi1 and the second pillar Pi2 are arranged. The above-mentioned stop position A1 can be said to be a position where the X coordinate in the map coordinate system is between the X coordinate of the first pillar Pi1 and the X coordinate of the second pillar Pi2. In FIGS. 1 and 8, the arrow X indicates the X axis of the map coordinate system, and the arrow Y indicates the Y axis of the map coordinate system.

The first fixed marker MA1 and the second fixed marker MA2 are located on the same coordinate axis. Since the two fixed markers MA1 and MA2 are provided on the pillars Pi1 and Pi2, the coordinates of the fixed markers MA1 and MA2 in the map coordinate system are regarded as representing the coordinates of the pillars Pi1 and Pi2 in the map coordinate system. You can also do it. It can be said that the map coordinate system is a coordinate system in which the physical structure in which the fixed markers MA1 and MA2 can be installed is extracted from the environment in which the forklift 10 is used, and the coordinate axes are set according to the physical structure. In the present embodiment, the Y coordinate of the first fixed marker MA1 and the Y coordinate of the second fixed marker MA2 in the map coordinate system are both 0. In the present embodiment, for convenience of explanation, only the X coordinate and the Y coordinate will be described, but the map coordinate system is not limited to the two-axis Cartesian coordinate system and may be a three-axis Cartesian coordinate system. That is, the map coordinate system may include an X-axis and a Z-axis orthogonal to the Y-axis as coordinate axes. In the following description, the coordinates of the map coordinate system are referred to as map coordinates.

The environment map may be stored in the auxiliary storage device 56 in advance as long as the surrounding environment in which the forklift 10 is used can be grasped in advance. When the environmental map is stored in the auxiliary storage device 56 in advance, the coordinates of objects whose positions are difficult to change, such as walls and pillars of the building, are stored as the environmental map. The environmental map may be created by mapping by SLAM: Simultaneous Localization and Mapping. The mapping is performed, for example, by creating a local map from the coordinates obtained by the environment sensor 58 and combining the local maps according to their own positions. Even when the environment map is created by mapping, the coordinates of the first fixed marker MA1 are set to be the origin, and the coordinates of the fixed markers MA1 and MA2 are set to be the coordinates on the X axis.

The control device 53 can move the forklift 10 to the loading position A2 by controlling the drive mechanism 51 while performing self-position estimation for estimating the self-position of the forklift 10 on the environmental map. The self-position estimation may be performed using, for example, an odometry that estimates the self-movement amount using the rotation speed of the traveling motor, or may be performed from the matching result of the landmark and the environmental map. In addition, self-position estimation may be performed by combining these. If the environment in which the forklift 10 is used is outdoors, the control device 53 may estimate its own position using GNSS (Global Navigation Satellite System). GNSS includes GPS (Global Positioning System), IRNSS (Indian Regional Navigational Satellite System), and QZSS (Quasi-Zenith Satellite System). The self-position is a coordinate indicating one point of the vehicle body 11, and is, for example, a coordinate at the center of the vehicle body 11 in the horizontal direction.

The environment sensor 58 is a sensor capable of causing the control device 53 to recognize the relative position between the object located behind the forklift 10 and the forklift 10. As the environment sensor 58, for example, a millimeter wave radar, a stereo camera, LIDAR: Laser Imaging Detection and Ranging, or the like can be used. The environment sensor 58 is provided to detect landmarks when performing self-position estimation.

The communication unit 59 is a communication device capable of communicating by a wireless communication method such as a wireless LAN, Zigbee (registered trademark), LPWA: Low Power Wide Area. The communication unit 59 as a receiving unit can input the demodulated data of the received radio signal to the control device 53, or modulate the data input from the control device 53 and transmit it as a radio signal.

As shown in FIG. 1, the camera 60 as a sensor is arranged at a fixed position in the workplace. The camera 60 is arranged so as to image the stop position A1 and the fixed markers MA1 and MA2. Specifically, when the truck T is stopped at the stop position A1, the camera 60 sets all the markers MA3 and MA4 and the two fixed markers MA1 and MA2 of the pallet P loaded on the loading platform TB of the truck T. It is arranged so that it can be imaged. The camera 60 is a monocular camera. The detectable range of the camera 60 is a range that can be imaged by the camera 60. It can be said that the camera 60 is arranged so that the stop position A1 and the fixed markers MA1 and MA2 are within the detectable range.

As shown in FIG. 5, the host control device 70 includes a control unit 75, a command communication unit 73, an auxiliary storage device 76, and an input device 77. The control unit 75 includes a processor 71 such as a CPU and a GPU, and a storage unit 72 including a RAM and a ROM. The storage unit 72 stores a program code or a command configured to cause the processor 71 to execute the process. The storage 72, i.e., a computer-readable medium, includes any available medium accessible by a general purpose or dedicated computer. The control unit 75 may be configured by a hardware circuit such as an ASIC or FPGA. The control unit 75, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as ASICs and FPGAs, or a combination thereof.

The command communication unit 73 as a transmission unit is a communication device capable of communicating by a wireless communication method such as a wireless LAN, Zigbee, or LPWA. The command communication unit 73 can input the demodulated data of the received radio signal to the control unit 75, or modulate the data input from the control unit 75 and transmit it as a radio signal. The command communication unit 73 generates a radio signal that can be received by the communication unit 59 and transmits it to the communication unit 59.

The image data of the camera 60 is input to the control unit 75 of the host control device 70. The upper control device 70 may be arranged in the workplace, or may be arranged at a position different from the workplace. When the upper control device 70 is arranged at a position different from the workplace, the image data of the camera 60 is transmitted to the upper control device 70 by a communication unit or the like capable of transmitting the image data to the upper control device 70.

As the auxiliary storage device 76, for example, a hard disk drive or a solid state drive is used. The program code or command stored in the storage unit 72 may be stored in the auxiliary storage device 76 instead of the storage unit 72. Further, the auxiliary storage device 76 may store the environmental map in the same manner as the auxiliary storage device 56 mounted on the forklift 10.

The input device 77 is a member that receives an operation of an operator and outputs it as an input signal. Examples of the input device 77 include a keyboard, physical buttons, a microphone, a mouse, and a touch panel.

As shown in FIG. 6, when the loading work by the forklift 10 is performed, the control device 53 is the insertion hole IH and the fork 31 which are holes surrounded by the loading platform TB, the leg portion L, and the accommodating portion S. Raise the lift of the fork 31 so that it faces the tip. Then, by advancing the forklift 10, the fork 31 is inserted into the insertion hole IH, and then by raising the fork 31, the pallet P is loaded on the fork 31. In order to insert the fork 31 into the insertion hole IH, it is preferable to accurately detect the position of the pallet P and position the forklift 10 in front of the pallet P. The control device 53 and the upper control device 70 move the forklift 10 to the pallet P by performing the following control.

As shown in FIG. 7, in step S1, the host control device 70 derives the coordinates of the palette P in the camera coordinate system of the camera 60. The camera coordinate system of the camera 60 is a coordinate system with the camera 60 as the origin. The camera coordinate system is, for example, a coordinate system in which the optical axis is the Y axis and the horizontal direction orthogonal to the Y axis is the X axis. In the present embodiment, only the X coordinate and the Y coordinate will be described, but the camera coordinate system is not limited to the 2-axis Cartesian coordinate system and may be a 3-axis Cartesian coordinate system. That is, the camera coordinate system may include an X-axis and a Z-axis orthogonal to the Y-axis as coordinate axes. Hereinafter, the coordinates of the camera 60 in the camera coordinate system will be referred to as camera coordinates. In the first embodiment, the camera coordinate system is the sensor coordinate system.

The host control device 70 performs image recognition processing for image recognition of markers MA3 and MA4 from image data input from the camera 60. When the markers MA3 and MA4 are extracted from the image data by the image recognition process, the host control device 70 analyzes the images of the markers MA3 and MA4 taken by the camera 60 and determines the markers MA3 from the size of the markers MA3 and MA4. , MA4 camera coordinates are derived. In the present embodiment, since the two markers MA3 and MA4 are provided for each palette P, the camera coordinates of the two markers MA3 and MA4 are derived for each palette P. Since the markers MA3 and MA4 are provided on the palette P, the camera coordinates of the markers MA3 and MA4 can be regarded as the camera coordinates of the palette P. The upper control device 70 derives the camera coordinates of the two fixed markers MA1 and MA2 by the same method. Since the camera coordinates of the fixed markers MA1 and MA2 can be regarded as the camera coordinates of the columns Pi1 and Pi2, the host control device 70 derives the camera coordinates of the fixed markers MA1 and MA2 as the camera coordinates of the columns Pi1 and Pi2. It can be said that there is. Since the camera coordinates of the markers MA3 and MA4 can be regarded as the camera coordinates of the palette P, it can be said that the host control device 70 derives the camera coordinates of the markers MA3 and MA4 as the camera coordinates of the palette P. By performing the process of step S1, the host control device 70 functions as a sensor coordinate derivation unit.

Next, in step S2, the upper control device 70 derives the map coordinates of the pallet P. The upper control device 70 derives the map coordinates of the palette P by the three-sided survey using the camera coordinates of the two fixed markers MA1 and MA2 and the camera coordinates of the markers MA3 and MA4. The map coordinates of the palette P are derived by deriving the map coordinates of the markers MA3 and MA4. Hereinafter, a method of deriving the map coordinates of the marker MA3 will be described by taking one of the markers MA3 and MA4 of the palette P as an example. Hereinafter, one marker MA3 will be described, but the host control device 70 derives map coordinates for all of the markers MA3 and MA4 in the image data by the same method.

As shown in FIG. 8, in the camera coordinate system with the camera 60 as the origin (0,0), the camera coordinates of the first fixed marker MA1 are (xa, ya) and the camera coordinates of the second fixed marker MA2 are (xb, ya). yb), let the camera coordinates of the marker MA3 be (xc, yc). The camera coordinates (xa, ya) of the first fixed marker MA1, the camera coordinates (xb, yb) of the second fixed marker MA2, and the camera coordinates (xc, yc) of the marker MA3 are derived in step S1. Therefore, the upper control device 70 has a distance a between the second fixed marker MA2 and the marker MA3, a distance b between the first fixed marker MA1 and the marker MA3, and the first fixed marker MA1 and the second fixed marker MA2. The distance c between and can be derived. That is, the upper control device 70 can derive the length of each side of the triangle having the first fixed marker MA1, the second fixed marker MA2, and the marker MA3 as vertices. Specifically, the distance a can be derived by the following equation (1), the distance b can be derived by the following equation (2), and the distance c can be derived by the following equation (3).

ここで、第１固定マーカーＭＡ１のカメラ座標（ｘａ，ｙａ）からマーカーＭＡ３のカメラ座標（ｘｃ，ｙｃ）までの距離ｂにおいて、２つの柱Ｐｉ１，Ｐｉ２同士が向かい合う方向に対する距離成分を距離ｘｐとする。距離ｘｐは、第１固定マーカーＭＡ１、第２固定マーカーＭＡ２、及びマーカーＭＡ３を頂点とする三角形において、２つの固定マーカーＭＡ１，ＭＡ２を結ぶ線分にマーカーＭＡ３から垂線を下ろしたときの交点ＣＰと第１固定マーカーＭＡ１との間の距離ともいえる。第１固定マーカーＭＡ１のカメラ座標（ｘａ，ｙａ）からマーカーＭＡ３のカメラ座標（ｘｃ，ｙｃ）までの距離ｂにおいて、距離ｘｐに直交する方向への距離成分を距離ｙｐとする。距離ｙｐは、交点ＣＰとマーカーＭＡ３との間の距離ともいえる。距離ｘｐ，ｙｐは距離ａ，ｂ，ｃを用いて導出することができる。詳細にいえば、距離ｘｐは以下の（４）式、距離ｙｐは以下の（５）式で導出できる。

Here, at the distance b from the camera coordinates (xa, ya) of the first fixed marker MA1 to the camera coordinates (xc, yc) of the marker MA3, the distance component with respect to the direction in which the two pillars Pi1 and Pi2 face each other is defined as the distance xp. do. The distance xp is the intersection CP when a perpendicular line is drawn from the marker MA3 to the line segment connecting the two fixed markers MA1 and MA2 in the triangle having the first fixed marker MA1, the second fixed marker MA2, and the marker MA3 as vertices. It can also be said to be the distance from the first fixed marker MA1. At the distance b from the camera coordinates (xa, ya) of the first fixed marker MA1 to the camera coordinates (xc, yc) of the marker MA3, the distance component in the direction orthogonal to the distance xp is defined as the distance yp. The distance yp can also be said to be the distance between the intersection CP and the marker MA3. The distances xp and yp can be derived using the distances a, b and c. Specifically, the distance xp can be derived by the following equation (4), and the distance yp can be derived by the following equation (5).

距離ｘｐ，ｙｐは、第１固定マーカーＭＡ１からマーカーＭＡ３までの距離ｂを互いに直交する方向に分解した距離成分を表しており、座標（ｘｐ，ｙｐ）は、第１固定マーカーＭＡ１を原点とする座標系でのマーカーＭＡ３の座標と捉えることができる。また、距離ｘｐは地図座標系のＸ軸方向の距離、距離ｙｐは地図座標系のＹ軸方向の距離である。更に、地図座標系の原点を第１固定マーカーＭＡ１としているため、座標（ｘｐ，ｙｐ）は、地図座標系でのマーカーＭＡ３の座標と捉えることができる。

The distances xp and yp represent the distance components obtained by decomposing the distance b from the first fixed marker MA1 to the marker MA3 in the directions orthogonal to each other, and the coordinates (xp, yp) have the first fixed marker MA1 as the origin. It can be regarded as the coordinates of the marker MA3 in the coordinate system. Further, the distance xp is the distance in the X-axis direction of the map coordinate system, and the distance yp is the distance in the Y-axis direction of the map coordinate system. Further, since the origin of the map coordinate system is the first fixed marker MA1, the coordinates (xp, yp) can be regarded as the coordinates of the marker MA3 in the map coordinate system.

Here, since the distance yp derived by Eq. (5) = the Y coordinate yp in the map coordinate system is a value that does not consider whether it is a + coordinate or a-coordinate in the map coordinate system, the following Eq. (6) From, it is determined whether the Y coordinate yp is a + coordinate or a-coordinate. Note that the + coordinate or the-coordinate indicates in which direction the coordinates are located with respect to the coordinate axes in the map coordinate system, and can be arbitrarily set for each coordinate axis. For example, in the map coordinate system showing the environment shown in FIG. 8, when the camera 60 side from the origin is the negative coordinate of the Y axis and the opposite is the + coordinate, the Y axis direction in the map coordinate system depends on whether the coordinate is + or −. It can be said that the marker MA3 indicates whether the marker MA3 is located closer to the camera 60 than the origin or is located farther from the camera 60 than the origin.

（６）式により求められたＬ＞０であれば、Ｙ座標ｙｐは＋座標、Ｌ＜０であればＹ座標ｙｐは−座標である。従って、Ｌ＜０の場合には、（５）式により導出されたＹ座標ｙｐに−１を乗算する。

If L> 0 obtained by the equation (6), the Y coordinate yp is a + coordinate, and if L <0, the Y coordinate yp is a − coordinate. Therefore, when L <0, the Y coordinate yp derived by Eq. (5) is multiplied by -1.

The distance xp derived from Eq. (4) = the X coordinate xp in the map coordinate system is also a value that does not consider whether it is the + coordinate or the-coordinate, but it is the second fixed value in the map coordinate system. By deriving the X coordinate xp that matches the X coordinate of the marker MA2, it is not necessary to determine whether it is a + coordinate or a-coordinate. For example, in the X-axis of the map coordinate system, it is assumed that the second fixed marker MA2 side from the origin is the + coordinate and the opposite is the-coordinate. Due to the stop position A1, the X coordinate of the palette P in the map coordinate system is always + coordinate. Therefore, the xp derived by the equation (4) can be regarded as the X coordinate of the palette P in the map coordinate system. Further, when the second fixed marker MA2 side from the origin is set to-coordinates, the X-coordinate of the palette P is always-coordinates, so multiply the X-coordinate xp derived by Eq. (4) by -1. Just do it. In this way, if the X coordinate of the second fixed marker MA2 in the map coordinate system is + coordinate, the X coordinate xp is also + coordinate, and if the X coordinate of the second fixed marker MA2 in the map coordinate system is-coordinate, X. Coordinates xp may also be such that -coordinates are derived.

The upper control device 70 derives the map coordinates of the pallet P from the map coordinates of the two markers MA3 and MA4 provided on the same pallet P. The upper control device 70 may consider any one of the map coordinates of the two markers MA3 and MA4 as the map coordinates of the palette P, and the center coordinates of the map coordinates of the two markers MA3 and MA4 may be the map coordinates of the palette P. You may think that. In any case, since the map coordinates of the markers MA3 and MA4 are associated with the map coordinates of the palette P, they can be grasped as the map coordinates of the palette P. The upper control device 70 can also derive the posture of the palette P in the map coordinate system from the map coordinates of the two markers MA3 and MA4. For example, the host control device 70 can derive the posture of the palette P in the map coordinate system from the inclination of the line segment connecting the map coordinates of the two markers MA3 and MA4. The host control device 70 functions as a map coordinate derivation unit by performing the process of step S2. In the present embodiment, the upper control device 70 derives the coordinates of the palette P in the map coordinate system as information for obtaining the coordinates of the palette P in the map coordinate system. The upper control device 70 functions as a derivation unit by performing the process of step S2.

As shown in FIG. 7, in step S3, the host control device 70 transmits a command including information indicating the map coordinates of the pallet P from the command communication unit 73. In addition to the information indicating the map coordinates of the pallet P, the command includes various information such as information indicating the attitude of the pallet P and information indicating which of the three pallets P is to be loaded. It may be included. That is, the command includes information about the pallet P to be loaded by the forklift 10. The command is a movement start command for moving the forklift 10 to the map coordinates of the pallet P. The host control device 70 can individually give commands to a plurality of forklifts 10 deployed in the workplace.

In step S11, when the control device 53 receives the command from the host control device 70, the control device 53 moves the forklift 10 toward the map coordinates of the pallet P included in the received command. Since the control device 53 can grasp the map coordinates of the pallet P, the forklift 10 is moved with the map coordinates of the pallet P or the map coordinates slightly before the map coordinates of the pallet P as the target point. As shown in FIG. 1, in the present embodiment, the control device 53 generates a path TP toward the map coordinates of the pallet P, and moves the forklift 10 so as to follow the path TP. As a result, the forklift 10 moves toward the pallet P, which is the target of the unloading work. By performing the process of step S11, the control device 53 functions as a movement control unit.

As shown in FIG. 7, in step S12, when the forklift 10 approaches the pallet P to be loaded, the control device 53 detects the pallet P by the in-vehicle camera 45. The detection of the pallet P by the in-vehicle camera 45 is performed by, for example, the same method as the detection of the markers MA3 and MA4 by the camera 60 performed by the host control device 70. That is, when the markers MA3 and MA4 are extracted from the image data input from the vehicle-mounted camera 45, the control device 53 analyzes the images of the markers MA3 and MA4 to analyze the markers MA3 in the camera coordinate system of the vehicle-mounted camera 45. , MA4 coordinates can be derived. The camera coordinate system of the vehicle-mounted camera 45 is a coordinate system with the vehicle-mounted camera 45 as the origin. The control device 53 derives the coordinates of the pallet P from the image data obtained by the imaging of the vehicle-mounted camera 45, and adjusts the position of the forklift 10 according to the pallet P. As a result, the forklift 10 moves to the unloading position A2 where the unloading work is performed. Since the positional relationship between the position of the vehicle-mounted camera 45 and the self-position is constant, the control device 53 can adjust the position of the forklift 10 by using the coordinates of the pallet P in the camera coordinate system of the vehicle-mounted camera 45. ..

The operation of the first embodiment will be described.
When the truck T loaded with the pallet P stops at the stop position A1, the host control device 70 derives the map coordinates of the pallet P and transmits a command including the map coordinates of the derived pallet P to the forklift 10. When the control device 53 receives the command from the communication unit 59, the forklift 10 is guided to the pallet P which is the target of the unloading work.

It is assumed that the map coordinates of the pallet P are not transmitted from the host control device 70. Since the stop position A1 is predetermined, when the truck T stops at the stop position A1, the forklift 10 moves to the stop position A1. Although the stop position A1 is predetermined, the position of the pallet P is indefinite. Therefore, the control device 53 detects the pallet P from the image data captured by the in-vehicle camera 45. When the in-vehicle camera 45 detects the pallet P, the control device 53 cannot detect the pallet P unless the forklift 10 approaches the pallet P. Further, since a plurality of pallets P can be mounted on the truck T, the angle of view of the in-vehicle camera 45 needs to be widened so that the plurality of pallets P can be imaged. As a result, the work efficiency of the forklift 10 is lowered, the detection accuracy of the pallet P is lowered, and the like.

On the other hand, in the present embodiment, the upper control device 70 guides the forklift 10 to the pallet P. When the forklift 10 approaches the pallet P to be loaded, the control device 53 detects the pallet P from the image data captured by the in-vehicle camera 45. Since the position adjustment between the forklift 10 and the pallet P is performed based on the image data captured by the vehicle-mounted camera 45, the host control device 70 raises the forklift 10 to a rough position where the pallet P can be imaged by the vehicle-mounted camera 45. I just need to be able to guide you. Therefore, high position accuracy is not required for the map coordinates of the palette P detected by the camera 60 and the upper control device 70. Further, since the forklift 10 is guided to the pallet P which is the target of the loading work, it is not necessary to take an image of all the pallets P which can be mounted on the truck T by the in-vehicle camera 45. As the in-vehicle camera 45, a narrow-angle high-precision camera can be used. Therefore, the work efficiency is improved by smoothly moving the forklift 10 to the pallet P, and the position accuracy of the pallet P detected by the control device 53 is improved.

The effect of the first embodiment will be described.
(1-1) By providing a camera 60 capable of capturing the stop position A1 and deriving the coordinates of the palette P in the map coordinate system from the image data of the camera 60 by the upper control device 70, the upper control device 70 is used. The forklift 10 can be guided to the pallet P.

(1-2) The upper control device 70 derives the coordinates of the palette P in the map coordinate system as information for obtaining the coordinates of the palette P in the map coordinate system. The control device 53 does not have to derive the coordinates of the palette P in the map coordinate system from the information obtained from the information obtained by obtaining the coordinates of the palette P in the map coordinate system. It is possible to suppress an increase in the processing load of the control device 53.

(1-3) The two pillars Pi1 and Pi2 are located on the X-axis of the map coordinate system, and the first pillar Pi1 is set as the origin of the map coordinate system. Thereby, the map coordinates of the palette P can be derived from the camera coordinates of the palette P by the three-sided survey.

(1-4) The first fixed marker MA1 and the second fixed marker MA2 are located on the X-axis of the map coordinate system, and the first fixed marker MA1 is set as the origin of the map coordinate system. The coordinates of the markers MA3 and MA4 derived by the three-sided survey can be said to be the coordinates of the coordinate system with the first fixed marker MA1 as the origin. By setting the correspondence between the fixed markers MA1 and MA2 and the map coordinate system as described above, the origin and the coordinate axes coincide with each other in the coordinate system with the first fixed marker MA1 as the origin and the map coordinate system. Therefore, the coordinates of the markers MA3 and MA4 derived by the three-sided survey become the coordinates of the map coordinate system. The processing performed by the control device 53 is less than in the case of deriving the map coordinates of the markers MA3 and MA4 from the coordinates of the markers MA3 and MA4 derived by the three-sided survey.

Further, by locating the first fixed marker MA1 and the second fixed marker MA2 on the X axis of the map coordinate system and using the first fixed marker MA1 as the origin of the map coordinate system, the camera 60 in the map coordinate system The map coordinates of the palette P can be derived even if the position of the pallet P cannot be grasped. Further, the map coordinates of the palette P can be derived without matching the coordinate axes of the camera coordinate system of the camera 60 with the coordinate axes of the map coordinate system. Therefore, when arranging the camera 60, it is not necessary to grasp the position and orientation of the camera 60 in the map coordinate system, and it is not necessary to arrange the camera 60 at a predetermined position. In other words, by setting the correspondence between the fixed markers MA1 and MA2 and the map coordinate system as described above, if the stop position A1 and the two fixed markers MA1 and MA2 can be imaged, the camera 60 can be in an arbitrary position at an arbitrary position. It is not necessary for the host control device 70 or the control device 53 to grasp the position and orientation of the camera 60.

(Second Embodiment)
A second embodiment of the cargo handling system will be described. In the cargo handling system of the second embodiment, the mode of deriving the camera coordinates of the pallet is different from that of the first embodiment. In the following description, a part different from the first embodiment will be described.

The upper control device 70 inputs the teacher data into the machine learning model and derives the camera coordinates of the palette P from the trained model generated. More specifically, the learned model is stored in the readable storage device of the control unit 75 such as the storage unit 72 of the upper control device 70 and the auxiliary storage device 76. When the palette P is imaged by the camera 60 in step S1, the host control device 70 derives the camera coordinates of the palette P from the image data using the trained model.

As shown in FIG. 9, the trained model is generated by inputting teacher data into the learner 74 constructed by the machine learning algorithm. The learning device 74 is a supervised learning model. Examples of machine learning include support vector machines, neural networks, naive bays, deep learning, and decision trees. In this embodiment, deep learning is used as machine learning.

As the teacher data, data obtained by assigning the camera coordinates of the palette P as a label to the image data IM obtained by capturing the palette P is used. More specifically, as shown in FIG. 9, a bounding box surrounding the pallet P is set in the image data IM in which the pallet P loaded on the loading platform TB of the truck T is imaged. Then, the data in which the label is set for each bounding box is used as the teacher data. By inputting this teacher data into the learner 74, a trained model is generated. The trained model outputs the camera coordinates by inputting the image data of the palette P. Although not shown, the image data IM of the pallet P includes a transported object accommodated in the accommodating portion S. That is, the image data IM of the pallet P is the image data of the pallet P in a state where the conveyed object is accommodated in the accommodating portion S.

There is a correlation between the image data of the palette P imaged by the camera 60 and the camera coordinates of the palette P. The closer the coordinates of the camera 60 and the coordinates of the palette P are in the camera coordinate system, the larger the size of the palette P in the image data. Further, the palette P in the image data is distorted according to the posture of the camera 60 and the posture of the palette P. Therefore, the camera coordinates of the palette P can be derived by using the trained model learned by using the learning device 74 described above.

Further, the upper control device 70 derives the camera coordinates of the two pillars Pi1 and Pi2 in the camera coordinate system by using the trained model trained by the same method as the trained model described above. More specifically, a bounding box surrounding the pillars Pi1 and Pi2 is set in the image data obtained by capturing the two pillars Pi1 and Pi2, and the camera coordinates are assigned as a label to each bounding box. The camera coordinates of the two pillars Pi1 and Pi2 can be derived by using the trained model in which machine learning is performed using the teacher data obtained thereby.

The upper control device 70 can derive the camera coordinates of the two pillars Pi1 and Pi2 and the camera coordinates of the palette P by using the trained model. The trained model may be generated by using the teacher data with a label attached to the image data IM in which both the palette P and the two pillars Pi1 and Pi2 are captured.

When deriving the camera coordinates of the pallet P using the trained model, the host controller 70 can derive the camera coordinates of the pallet P without using the markers MA3 and MA4. Therefore, as with the upper control device 70, the control device 53 also detects the pallet P without using the markers MA3 and MA4. Similar to the upper control device 70, the control device 53 may detect the pallet P using the trained model, or may detect the pallet P by another method.

In the second embodiment, in addition to the effects (1-1) to (1-3) of the first embodiment, the following effects can be obtained.
(2-1) The upper control device 70 can derive the camera coordinates of the palette P by using the trained model. Since the host control device 70 can derive the camera coordinates of the pallet P without recognizing the markers MA3 and MA4, it is possible to reduce the trouble of attaching the markers MA3 and MA4 to the pallet P.

(Third Embodiment)
A third embodiment of the cargo handling system will be described. In the following description, a part different from the first embodiment will be described. The same members as those in the first embodiment or the members exhibiting the same functions will be designated by the same reference numerals as those in the first embodiment, and the description thereof will be omitted.

As shown in FIG. 10, the cargo handling system CS1 includes a laser range finder 81 as a sensor and a truck detection camera 82. The truck TC of the third embodiment transports the container C1. The truck TC is a transport vehicle that conveys the container C1 connected to the truck TC. The stop position A11 of the third embodiment is an area including two pillars Pi1 and Pi2. The stop position A11 does not have to be between the two pillars Pi1 and Pi2. The truck TC of the present embodiment transports two containers C1, but the truck TC may transport a single container C1 or may transport three or more containers C1. good.

The forklift 10 performs cargo handling work including loading work of loading the pallet P on the container C1 and loading work of loading the pallet P loaded on the container C1 on the forklift 10. A load W is loaded on the pallet P. In the present embodiment, the case where the forklift 10 performs the loading work will be described as an example. The loading work can be performed by the same method as in the first embodiment. The forklift 10 performs loading work after moving to the cargo handling position A21. The cargo handling position A21 is a front position of the container C1 on which the pallet P is placed.

The forklift 10 stands by at the standby position A31 set in the work place when the loading work is not performed. When the truck TC stops at the stop position A11, the forklift 10 moves to the cargo handling position A21 and performs the loading work.

The truck TC is stopped along the two pillars Pi1 and Pi2. For example, the truck TC is stopped so that one container C1 is located between the two pillars Pi1 and Pi2 when viewed from the horizontal direction orthogonal to the direction in which the two pillars Pi1 and Pi2 are arranged.

As shown in FIGS. 11 and 12, the container C1 has a hollow shape, and the inside is a storage space capable of accommodating the pallet P. The container C1 includes a bottom BW, a top CW, two front walls FW1 and FW2, two rear walls RW1 and RW2, and two side wall SW1 and SW2. The bottom BW, the top CW, the front walls FW1, FW2, the rear walls RW1, RW2, and the side walls SW1 and SW2 are square plate-shaped wall portions, respectively. The bottom BW and the top CW face each other. The front walls FW1, FW2, the rear walls RW1, RW2, and the side wall SW1, SW2 are located between the bottom BW and the top CW. The two side walls SW1 and SW2 face each other. The two front walls FW1 and FW2 and the two rear walls RW1 and RW2 face each other. The front wall FW1 is attached to the side wall SW1 in a state where it can be rotated by a hinge or the like. The front wall FW2 is attached to the side wall SW2 in a state where it can be rotated by a hinge or the like. The rear wall RW1 is attached to the side wall SW1 in a state where it can be rotated by a hinge or the like. The rear wall RW2 is attached to the side wall SW2 in a state where it can be rotated by a hinge or the like. The front walls FW1, FW2 and the rear walls RW1, RW2 can be said to be doors. By rotating the front walls FW1 and FW2, the opening and closing of the opening O11 surrounded by the bottom BW, the top CW and the side walls SW1 and SW2 can be switched. By rotating the rear walls RW1 and RW2, the opening and closing of the opening O12 surrounded by the bottom BW, the top CW and the side walls SW1 and SW2 can be switched.

The container C1 is used in such a manner that the bottom BW and the top CW face each other in the vertical direction and the bottom BW is located below the top CW. It can be said that the front walls FW1 and FW2 and the rear walls RW1 and RW2 are wall portions facing each other in the horizontal direction. It can be said that the side walls SW1 and SW2 are wall portions facing each other in the horizontal direction.

The two front walls FW1 and FW2 are provided with outer surfaces O1 and O2, respectively. The two side walls SW1 and SW2 are provided with outer surfaces O3 and O4, respectively. The two rear walls RW1 and RW2 are provided with outer surfaces O5 and O6, respectively. The outer surfaces O1 and O2 of the front walls FW1 and FW2 and the outer surfaces O5 and O6 of the rear walls RW1 and RW2 are located outside the container C1 in a state where the front walls FW1 and FW2 and the rear walls RW1 and RW2 are closed. It is a side to do. With the front walls FW1 and FW2 blocked, the outer surface O1 of the front wall FW1 and the outer surface O2 of the front wall FW2 can be regarded as the same surface. With the rear walls RW1 and RW2 blocked, the outer surface O5 of the rear wall RW1 and the outer surface O6 of the rear wall RW2 can be regarded as the same surface. The distance L2 between the outer surfaces O1 and O2 of the front walls FW1 and FW2 and the outer surfaces O5 and O6 of the rear walls RW1 and RW2 when the front walls FW1 and FW2 and the rear walls RW1 and RW2 are closed is the side wall SW1. , The distance between the outer surfaces O3 and O4 of SW2 is shorter than the distance L1. In the following description, among the wall portions facing each other in the horizontal direction, the direction in which the wall portions having a long separation distance face each other is the longitudinal direction of the container C1, and the direction in which the wall portions having a short separation distance face each other is the short side of the container C1. The direction. In the present embodiment, the direction in which the side walls SW1 and SW2 face each other is the longitudinal direction, and the direction in which the front walls FW1 and FW2 and the rear walls RW1 and RW2 face each other is the lateral direction. The container C1 may be any container as long as it can accommodate the load W. The container C1 may have any shape as long as it can accommodate the load W. For example, the container C1 may have no top CW, or the rear walls RW1 and RW2 may not open. You may.

As shown in FIG. 13, the forklift 10 includes an in-vehicle sensor 46. The in-vehicle sensor 46 is a sensor for use in the cargo handling work of the forklift 10. In the present embodiment, the vehicle-mounted sensor 46 includes at least a sensor used for loading work of the forklift 10. The in-vehicle sensor 46 is used to detect the loading position, which is the position where the pallet P is placed, when the forklift 10 performs the loading operation. As the in-vehicle sensor 46, for example, a laser range finder, a camera, a millimeter wave radar, or the like, which can make the control device 53 recognize the relative position between the forklift 10 and the container C1 is used.

As shown in FIG. 14, the laser rangefinder 81 measures the distance to an object within the detectable range DA. The laser range finder 81 irradiates the laser while changing the irradiation angle in the horizontal direction at an angle according to the angular resolution. The laser rangefinder 81 of the present embodiment is a two-dimensional laser rangefinder in which the irradiation angle in the vertical direction is not changed. The laser rangefinder 81 is sometimes referred to as a laser range finder or LIDAR.

When the laser hits an object, the laser rangefinder 81 receives the reflected light from the object. Assuming that the portion of the object hit by the laser is the irradiation point P1, the laser rangefinder 81 calculates the distance from the laser rangefinder 81 to the irradiation point P1. The laser range finder 81 outputs information in which the distance to the irradiation point P1 and the irradiation angle are associated with each other as distance information to the host control device 70. By using the irradiation angle and the distance, the coordinates of the irradiation point P1 in the sensor coordinate system with the laser range finder 81 as the origin can be derived. The sensor coordinate system is a Cartesian coordinate system in which one of the axes orthogonal to each other in the horizontal direction is the X axis and the axis different from the X axis is the Y axis. Using the distance derived by the laser rangefinder 81 and the irradiation angle, the distance to the irradiation point P1 can be decomposed into a component in the X-axis direction and a component in the Y-axis direction. As a result, the coordinates of the irradiation point P1 in the sensor coordinate system with the laser range finder 81 as the origin can be derived. Therefore, the distance information can also be regarded as the coordinates of the irradiation point P1 in the sensor coordinate system with the laser range finder 81 as the origin. Let the coordinates of the sensor coordinate system be the sensor coordinates. The XY plane of the sensor coordinate system is a coordinate plane representing the horizontal direction. Therefore, it can be said that the sensor coordinates are the coordinates in the direction representing the horizontal direction. The coordinates of the irradiation point P1 in the sensor coordinate system may be derived by the laser range finder 81 or may be derived by the host control device 70.

The detectable range DA of the laser range finder 81 is determined by the laser irradiation angle θ1 and the laser irradiation distance R. The detectable range DA of the laser range finder 81 is, for example, a fan shape having an irradiable angle θ1 as a central angle and an irradiable distance R as a radius.

The laser range finder 81 is arranged at a fixed position in the workplace. The laser range finder 81 is arranged so that the detectable range DA includes the stop position A11 and the two pillars Pi1 and Pi2. Specifically, the laser range finder 81 is arranged so that at least one container C1 and two pillars Pi1 and Pi2 are irradiated with a laser while the truck TC is stopped at the stop position A11. The laser rangefinder 81 is preferably arranged so that the laser irradiation direction is not tilted in the vertical direction, but the laser irradiation direction may be tilted in the vertical direction if it is slight.

In a state where the truck TC is stopped at the stop position A11 and the front walls FW1, FW2 and the rear walls RW1 and RW2 are closed, the front walls FW1 and FW2 of the container C1 use the laser range finder 81. Turn to. The rear walls RW1 and RW2 of the container C1 are separated from the laser rangefinder 81 by the front walls FW1 and FW2 of the container C1. In the present embodiment, of the wall portions of the container C1, the wall portion facing the laser rangefinder 81 in the state where the truck TC is stopped at the stop position A11 is referred to as a front wall for convenience, but the container C1 and the laser are used. Depending on the positional relationship of the rangefinder 81, any of the wall portions extending between the bottom BW and the top CW can be the front wall.

The truck detection camera 82 is arranged at a fixed position in the work place. The truck detection camera 82 is arranged so as to capture the stop position A11. Specifically, the truck detection camera 82 is arranged so that the truck TC can be imaged when the truck TC is stopped at the stop position A11. The track detection camera 82 is a monocular camera.

The measurement result of the laser range finder 81 is input to the control unit 75 of the host control device 70. The image pickup result of the track detection camera 82 is input to the control unit 75 of the host control device 70.
The host control device 70 detects that the truck TC has stopped at the stop position A11. The detection of the track TC can be performed by the trained model generated by inputting the teacher data into the machine learning model. More specifically, the learned model is stored in the readable storage device of the control unit 75 such as the storage unit 72 of the upper control device 70 and the auxiliary storage device 76. The host control device 70 detects the track TC from the image data input from the track detection camera 82. Machine learning can be performed by the same method as in the second embodiment. As the teacher data, image data obtained by capturing the track TC is used.

When the host control device 70 detects that the truck TC has stopped at the stop position A11, it derives the position of the container C1. The position of the container C1 is determined by deriving the edges of the container C1 that are horizontally separated from each other.

As shown in FIGS. 11 and 12, in the present embodiment, the positions of the two edges E1 and E2 horizontally separated from each other on the two front walls FW1 and FW2 are detected. The two edges E1 and E2 are adjacent edges that are separated from each other in the longitudinal direction of the container C1. One of the two edges E1 and E2 is designated as the first edge E1, and the remaining one is designated as the second edge E2. It can be said that the first edge E1 is the position farthest from the front wall FW2 in the horizontal direction among the front wall FW1. It can be said that the second edge E2 is the position farthest from the front wall FW1 in the horizontal direction among the front wall FW2. If the outer surfaces O1 and O2 of the two front walls FW1 and FW2 are regarded as one plane, it can be said that the two edges E1 and E2 are edges that are horizontally adjacent to each other on the same plane.

The upper control device 70 derives the position of the container C1 from the positions of the two edges E1 and E2, and guides the forklift 10 to the container C1 as the movement target. When the forklift 10 performs the loading work, the pallet P is loaded into the container C1 through the opening O11. Hereinafter, the processing performed by the host control device 70 will be described in detail. The processing performed by the host controller 70 is performed, for example, by the processor 71 executing the program read into the RAM. For convenience of explanation, a case where the edges E1 and E2 of one container C1 among the plurality of containers C1 are detected will be described. Further, the front walls FW1 and FW2 of the container C1 are in a closed state.

In the present embodiment, the positions of the edges E1 and E2 can be detected with high accuracy by using the two edge detection processes of the first edge detection process and the second edge detection process in combination.
As shown in FIG. 15, in step S21, the control unit 75 of the host control device 70 acquires the measurement result of the laser range finder 81. The measurement result of the laser range finder 81 is the sensor coordinates of the irradiation point P1. When the truck TC stops at the stop position A11, the upper control device 70 acquires the irradiation point P1 due to the laser hitting the truck TC and the container C1. In the following description, as an example, a case where the sensor coordinates of the irradiation point P1 are obtained as shown in FIG. 16 will be described.

As shown in FIG. 15, in step S22, the control unit 75 of the host control device 70 acquires the sensor coordinates of the first pillar Pi1 and the second pillar Pi2. From the arrangement position of the laser range finder 81, the rough positions of the first pillar Pi1 and the second pillar Pi2 can be grasped. Therefore, if the range in which the first pillar Pi1 is assumed to exist is set in advance and the coordinates of the irradiation point P1 within this range are acquired by the upper control device 70 as the sensor coordinates of the first pillar Pi1. good. Similarly, a range in which the second pillar Pi2 is assumed to exist is set in advance, and the coordinates of the irradiation point P1 within this range are acquired by the upper control device 70 as the sensor coordinates of the second pillar Pi2. Just do it. If the position of the laser rangefinder 81, the position of the first pillar Pi1, and the positional relationship of the second pillar Pi2 are constant, the sensor coordinates of the first pillar Pi1 and the sensor coordinates of the second pillar Pi2 can be grasped in advance. .. In this case, the sensor coordinates of the first pillar Pi1 and the sensor coordinates of the second pillar Pi2 may be stored in advance in the storage unit 72 or the auxiliary storage device 76 of the upper control device 70.

Next, in step S23, the control unit 75 of the host control device 70 converts the sensor coordinates of the irradiation point P1 into map coordinates. Assuming that the coordinates of the second pillar Pi2 are (x1, y1) when the coordinates of the first pillar Pi1 are (0,0) in the sensor coordinate system, the inclination θ of the map coordinate system with respect to the sensor coordinate system is θ = Atan. It can be derived by (x1, y1). The inclination θ is the inclination of the X-axis of the map coordinate system with respect to the X-axis of the sensor coordinate system. When the inclination θ is known in advance, the inclination θ may be stored in the storage unit 72 or the auxiliary storage device 76 of the host control device 70.

The control unit 75 of the host control device 70 multiplies the sensor coordinates of the irradiation point P1 by a rotation matrix, and then translates the irradiation point P1 so that the coordinates of the first column Pi1 become (0,0). Let the sensor coordinates of the irradiation point P1 be (x', y'), and the coordinates of the irradiation point P1 in the sensor coordinate system in which the inclination with respect to the map coordinate system is 0 be (x, y). By multiplying the sensor coordinates of the irradiation point P1 by the rotation matrix by the following equation (7), the coordinates (x, y) of the irradiation point P1 in the sensor coordinate system in which the inclination with respect to the map coordinate system is 0 are derived. be able to.

以下の（８）式により、回転行列を乗算した後のセンサ座標（ｘ，ｙ）を第１柱Ｐｉ１の座標が（０，０）になるように平行移動させる。

According to the following equation (8), the sensor coordinates (x, y) after multiplying the rotation matrix are translated so that the coordinates of the first column Pi1 become (0,0).

（ｘ_ｍ，ｙ_ｍ）は、地図座標系での照射点Ｐ１の座標である。（ｘ１’，ｙ１’）は、地図座標系との傾きを０にしたセンサ座標系での第１柱Ｐｉ１の座標である。即ち、（ｘ１’，ｙ１’）は、第１柱Ｐｉ１のセンサ座標に回転行列を乗算した座標である。

_(X m, _{y m)} is the coordinate of the irradiation point P1 on map coordinate system. (X1', y1') is the coordinates of the first pillar Pi1 in the sensor coordinate system in which the inclination with respect to the map coordinate system is 0. That is, (x1', y1') is the coordinates obtained by multiplying the sensor coordinates of the first pillar Pi1 by the rotation matrix.

The upper control device 70 performs the first edge detection process.
In step S24, the upper control device 70 extracts a straight line from the map coordinates of the irradiation point P1 obtained in step S23. Since the sensor coordinates of the irradiation point P1 are converted into map coordinates in step S23, in step S24, a straight line in the XY plane of the map coordinate system is extracted as a coordinate plane representing the horizontal direction. In the present embodiment, the host controller 70 extracts a straight line by RANSAC (Random Sample Consensus). The upper control device 70 extracts a plurality of straight lines including the irradiation point P1. The straight line has a predetermined width in the direction orthogonal to the length direction of the straight line in the XY plane of the map coordinate system. A straight line can also be regarded as a rectangular range extending in a direction orthogonal to the extending direction of the straight line in the XY plane of the map coordinate system. The width of the straight line can be appropriately set by the administrator of the cargo handling system CS1 or the like. In the example shown in FIG. 16, two straight lines L11 and L21 are illustrated, but the upper control device 70 draws, for example, several tens to several hundreds of straight lines obtained by combining a plurality of irradiation points P1. Extract.

As shown in FIG. 15, in step S25, the upper control device 70 includes two edges E1 and E2 of the container C1 in the XY plane of the map coordinate system from the plurality of straight lines L11 and L21 extracted in step S24. Extract a straight line. The upper control device 70 extracts the straight line having the most irradiation points P1 located on the straight line as a straight line including the two edges E1 and E2 of the container C1. In the example shown in FIG. 16, of the plurality of straight lines L11 and L21, the straight line L11 is extracted as a straight line including the edges E1 and E2. The irradiation point P1 located on the straight line can also be said to be the irradiation point P1 located within the width of the straight line. When the laser is irradiated on a plane, the irradiation points P1 irradiated on the plane are located on a straight line, so that the irradiation points P1 irradiated on the same plane are likely to be located within the width of the straight line. As described above, when the front walls FW1 and FW2 are closed, the outer surfaces O1 and O2 of the two front walls FW1 and FW2 can be regarded as the same plane. Of the outer surfaces of the container C1 irradiated with the laser, the surfaces having the longest horizontal dimensions can be said to be the outer surfaces O1 and O2 of the front walls FW1 and FW2. Therefore, it can be said that the straight line having the largest number of irradiation points P1 located within the width of the straight line is the straight line obtained by the irradiation points P1 irradiated on the outer surfaces O1 and O2 of the front walls FW1 and FW2.

Depending on the container C1, the front walls FW1 and FW2 may have undulations, and the outer surfaces O1 and O2 may not be smooth. In this case, the irradiation points P1 irradiated on the outer surfaces O1 and O2 of the front walls FW1 and FW2 may not be located in a straight line due to undulations. As described above, since the width of the straight line can be appropriately set, the width of the straight line may be set so as to absorb the deviation of the irradiation point P1 due to the undulations. For example, the width of the straight line may be wider than the amount of deviation of the irradiation point P1 in the thickness direction of the front walls FW1 and FW2 due to the undulations.

As shown in FIG. 15, in step S26, the upper control device 70 extracts the irradiation point P1 obtained by irradiating the edges E1 and E2 with the laser. As shown in FIG. 17, the upper control device 70 has a predetermined distance D1 or more with respect to one side of the irradiation point P1 located within the width of the straight line L11 extracted in step S25 in the extending direction of the straight line L11. Two irradiation points P1 in which adjacent irradiation points P1 do not exist are extracted as irradiation points P1 irradiated on the two edges E1 and E2. In the following description, the irradiation points P1 irradiated to the edges E1 and E2 are referred to as edge irradiation points PE1 and PE2. The edge irradiation point PE1 is an irradiation point P1 obtained by irradiating the first edge E1 with a laser, and the edge irradiation point PE2 is an irradiation point obtained by irradiating the second edge E2 with a laser. It is P1.

Since the straight line L11 is a straight line obtained by irradiating the outer surfaces O1 and O2 of the two front walls FW1 and FW2, the irradiation point P1 is interrupted at the irradiation point P1 irradiated on the edges E1 and E2. become. The irradiation points P1 in which the irradiation points P1 adjacent to one side in the extending direction of the straight line L11 do not exist at a predetermined distance D1 or more can be regarded as the edge irradiation points PE1 and PE2 irradiated on the edges E1 and E2. Further, the straight line L11 obtained by irradiating the outer surfaces O1 and O2 of the two front walls FW1 and FW2 is continuously irradiated until the irradiation point P1 is interrupted by irradiating the edges E1 and E2. P1 is located. Therefore, there is no irradiation point P1 adjacent to one side in the extending direction of the straight line L11 by a predetermined distance D1 or more, and an irradiation point P1 having a continuous irradiation point P1 on the other side. The edge irradiation points PE1 and PE2 irradiated to the edges E1 and E2 may be used. Whether or not the irradiation points P1 continuously exist can be determined by whether or not the number of irradiation points P1 equal to or larger than the threshold value exists within a predetermined distance. The predetermined distance can be appropriately set within the range of the distance between the two edges E1 and E2 or less. The threshold value can be appropriately set in the range below the irradiation point P1 which is assumed to be irradiated within a predetermined distance. For example, irradiation in which there is no irradiation point P1 adjacent to a predetermined distance D1 or more with respect to one side in the extending direction of the straight line L11, and a plurality of irradiation points P1 exist within the distance D1 for the other side. The point P1 may be the edge irradiation points PE1 and PE2 that irradiate the edges E1 and E2. The edge irradiation points PE1 and PE2 may not actually be the irradiation points P1 irradiated on the edges E1 and E2. The edge irradiation points PE1 and PE2 are formed by the laser hitting the irradiation points P1 closest to the edges E1 and E2 or the side walls SW1 and SW2 among the irradiation points P1 obtained by hitting the front walls FW1 and FW2 with the laser. Among the obtained irradiation points P1, there is a case where the irradiation point P1 is closest to the edges E1 and E2. Even in this case, the edge irradiation points PE1 and PE2 are the irradiation points P1 closest to the edges E1 and E2, and can be treated as the irradiation points P1 irradiated on the edges E1 and E2.

Next, in step S27, the upper control device 70 performs the second edge detection process. In the second edge detection process, the edge irradiation points PE1 and PE2 are detected by utilizing the direction change when the irradiation points P1 are connected to each other. Focusing on the edge irradiation point PE1 shown in FIG. 16, one irradiation point P2 of the two irradiation points P2 and P3 adjacent to the edge irradiation point PE1 is the irradiation point P1 irradiated on the outer surface O1 of the front wall FW1. Another irradiation point P3 is an irradiation point P1 irradiated on the outer surface O3 of the side wall SW1. The irradiation points P1 obtained by irradiating the same surface with the laser are likely to be located on a straight line, while the irradiation points P1 obtained by irradiating different surfaces with the laser are difficult to be located on a straight line. Therefore, when the irradiation points P1 are connected by a line segment, the line segment connecting the irradiation point P1 obtained by hitting the first edge E1 with the laser and the irradiation point P1 adjacent to the irradiation point P1 is inclined. The angle will change abruptly. Therefore, when the edge irradiation point PE1 and the irradiation points P2 and P3 adjacent to the edge irradiation point PE1 are connected by a line segment, the inclination of the line segment changes significantly. The upper control device 70 extracts an irradiation point P1 in which the amount of change in the inclination of the line segment when the adjacent irradiation points P1 are connected to each other is larger than a predetermined determination threshold value as the edge irradiation point PE1. The determination threshold is set so that it can detect that the laser has irradiated a different surface. The determination threshold value is set according to, for example, the angular resolution of the laser range finder 81 and the angle at which the outer surfaces O1 and O2 of the front walls FW1 and FW2 intersect with the outer surfaces O3 and O4 of the side walls SW1 and SW2. The upper control device 70 can also extract the edge irradiation point PE2 by the same method.

As shown in FIG. 15, in step S28, the upper control device 70 derives the positions of the two edges E1 and E2. The positions of the two edges E1 and E2 are positions in the map coordinate system. The positions of the two edges E1 and E2 can be said to be the map coordinates of the edges E1 and E2. The upper control device 70 derives the positions of the two edges E1 and E2 from the detection result by the first edge detection process and the detection result by the second edge detection process. When the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process match, the upper control device 70 sets the map coordinates of the edge irradiation point PE1. It is the position of the first edge E1.

In the upper control device 70, when the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process do not match, the separation distance between the two is within the permissible range. In the case of, the position of the first edge E1 is derived from the detection result. The upper control device 70 derives the map coordinates of the intermediate position between the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process as the position of the first edge E1. The intermediate positions between the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process are, for example, the edge irradiation point PE1 extracted by the first edge detection process and the first edge irradiation point PE1. 2 This is the center position of the edge irradiation point PE1 extracted by the edge detection process.

In the upper control device 70, when the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process do not match, the separation distance between the two is out of the allowable range. In the case of, the first edge detection process and the second edge detection process may be performed again. Further, in the upper control device 70, when the edge irradiation point PE1 extracted by the first edge detection process and the edge irradiation point PE1 extracted by the second edge detection process do not match, the separation distance between the two is allowed. If it is out of the range, the predetermined detection result of the first edge detection process and the second edge detection process may be adopted. The permissible range can be appropriately set by the administrator of the cargo handling system CS1 or the like. The permissible range is set so that, for example, the difference between the two edge detection processing methods can be tolerated. Since the methods for deriving the edge irradiation points PE1 and PE2 are different between the first edge detection process and the second edge detection process, different irradiation points P1 are detected as edge irradiation points depending on the measurement result of the laser range finder 81. However, when the difference between the two edge detection processes is the cause, the difference between the edge irradiation points derived by the two edge detection processes is small. For example, the edge irradiation point irradiated by the first edge detection process and the edge irradiation point irradiated by the second edge detection process can be irradiation points P1 adjacent to each other. On the other hand, when the edge irradiation points derived by the two edge detection processes are different due to factors different from the difference due to the edge detection process method, such as the influence of outliers or the case where the number of acquired irradiation points P1 is small. The edge irradiation points can be greatly separated from each other. Therefore, the permissible range may be set so as to allow the deviation of the edge irradiation point due to the difference between the two edge detection processing methods.

In the above description, the mode of deriving the position of the first edge E1 has been described, but the position of the second edge E2 can also be derived by the same method as that of the first edge E1 using the edge irradiation point PE2. ..

As described above, various modes for deriving the positions of the two edges E1 and E2 from the detection results of the first edge detection process and the second edge detection process can be mentioned. The manager of the cargo handling system CS1 can derive the positions of the two edges E1 and E2 in any manner.

Next, in step S29, the upper control device 70 corrects the positions of the edges E1 and E2 derived in step S28 by filtering. The upper control device 70 weights the map coordinates of the first edge E1 extracted this time by the map coordinates of the first edge E1 obtained in the past by performing the processing of the IIR filter. For example, the host control device 70 corrects the position of the first edge E1 from the following equation (9).

Y [n] = aY [n-1] + (1-a) X [n] ... (9)
Y is an output and X is an input. Y [n] is the current output, and Y [n-1] is the previous output. X [n] is the input this time. a is a coefficient. a can be set to any number less than 1. a is, for example, 0.9.

By inputting the map coordinates of the first edge E1 extracted this time as X [n], the output obtained by weighting the map coordinates of the previous first edge E1 and the map coordinates of the first edge E1 this time by a coefficient is obtained. Obtainable.

In the above description, the mode of correcting the position of the first edge E1 has been described, but the position of the second edge E2 can also be corrected by the same method as that of the first edge E1. The upper control device 70 individually corrects both the position of the first edge E1 and the position of the second edge E2.

Next, in step S30, the upper control device 70 derives the position of the container C1 in the map coordinate system, that is, the map coordinates of the container C1. The position of the container C1 in the map coordinate system is an intermediate position between the two edges E1 and E2. As the intermediate position between the two edges E1 and E2, for example, the center position of the two edges E1 and E2 can be mentioned. The position of the container C1 can be said to be the center position in the horizontal direction of the opening O11. Since the forklift 10 loads the pallet P into the container C1 from the opening O11, the position of the container C1 is set so that the in-vehicle sensor 46 can easily detect the loading position when the forklift 10 performs the loading work. ing. In other words, it can be said that the edges E1 and E2 whose positions are detected by the upper control device 70 are selected so that the map coordinates of the container C1 can be derived. The upper control device 70 derives the position of the container C1 in the map coordinate system from the map coordinates of the two edges E1 and E2 derived in step S29. The map coordinates of the container C1 are information that can obtain the coordinates of the container C1 in the map coordinate system. By performing the process of step S30, it can be said that the host control device 70 includes a map coordinate derivation unit and a derivation unit.

In step S31, the host control device 70 transmits a command including information indicating the map coordinates of the container C1 from the command communication unit 73. The command is a movement start command for moving the forklift 10 to the map coordinates of the container C1.

In step S41, when the control device 53 receives the command from the host control device 70, the pallet P is loaded on the fork 31, and the forklift 10 is moved toward the map coordinates of the container C1 included in the received command. Since the control device 53 can grasp the map coordinates of the container C1, the forklift 10 is moved with the map coordinates of the container C1 or the map coordinates slightly before the map coordinates of the container C1 as the target point.

In step S42, when the forklift 10 approaches the container C1, the forklift 10 performs a loading operation. The position where the loading work is started is the cargo handling position A21. It can be said that the cargo handling position A21 is a position that changes depending on the position of the container C1 or the like. Loading by the forklift 10 is performed using the in-vehicle sensor 46. The control device 53 controls the drive mechanism 51 and the hydraulic mechanism 52 so that the upper surface of the bottom BW of the container C1 is set as the loading position and the pallet P is placed at this loading position. The control device 53 detects the loading position by the vehicle-mounted sensor 46, and places the pallet P at the loading position. When the in-vehicle sensor 46 includes a laser range finder that irradiates a laser while changing the irradiation angle with respect to the vertical direction, the height of the loading position can be detected by the detection result of the laser range finder. When the in-vehicle sensor 46 includes a laser range finder that irradiates a laser while changing the irradiation angle with respect to the horizontal direction, the horizontal position of the loading position can be derived from the detection result of the laser range finder. Before the forklift 10 performs the loading work, the front walls FW1 and FW2 are opened so that the forklift 10 can load from the opening O11.

In step S43, the control device 53 moves the forklift 10 to the standby position A31 when the loading operation is completed. As a result, the forklift 10 returns to the standby position A31 and waits at the standby position A31 until the next command is received.

The operation of the third embodiment will be described.
When the truck TC stops at the stop position A11, the upper control device 70 derives the map coordinates of the edges E1 and E2. The upper control device 70 derives the map coordinates of the container C1 from the map coordinates of the edges E1 and E2, and transmits a command including information indicating the derived map coordinates of the container C1 to the forklift 10. When the control device 53 receives the command from the communication unit 59, the forklift 10 is guided to the container C1 to be loaded. Then, the forklift 10 performs the loading work.

The effect of the third embodiment will be described. In the third embodiment, the following effects can be obtained in addition to the effects of the first embodiment.
(3-1) The host control device 70 derives the map coordinates of the container C1. As a result, the container C1 can be set as the movement target, and the forklift 10 can be guided to the movement target.

(3-2) The upper control device 70 is a device different from the control device 53 mounted on the forklift 10. The control device 53 mounted on the forklift 10 performs various controls related to the operation of the forklift 10, such as self-position estimation. Therefore, when the control device 53 detects the positions of the edges E1 and E2 of the container C1 and corrects the edges E1 and E2, the processing load of the control device 53 may become excessively large. In particular, when the edges E1 and E2 are corrected, the processing load tends to increase. On the other hand, since the upper control device 70 does not control the operation of the forklift 10, even if the positions of the edges E1 and E2 are detected and the positions of the edges E1 and E2 are corrected, the processing load is excessively large. It is hard to become. The upper control device 70 can correct the positions of the edges E1 and E2. By correcting the positions of the edges E1 and E2, the detection accuracy of the positions of the edges E1 and E2 can be improved. Thereby, the detection accuracy of the map coordinates of the container C1 can be improved.

Each embodiment can be modified and implemented as follows. Each embodiment and the following modifications can be implemented in combination with each other within a technically consistent range.
○ In each embodiment, if the coordinates of the origin of the map coordinate system in the camera coordinate system can be grasped and the direction of the coordinate axis of the camera coordinate system and the direction of the coordinate axis of the map coordinate system are matched, 2 The camera coordinates of the two pillars Pi1 and Pi2 do not have to be derived. In this case, when the host control device 70 derives the camera coordinates of the pallet P, the map coordinates of the pallet P are derived from the camera coordinates of the pallet P. A storage device that stores information that can be read by the control unit 75, such as the storage unit 72 and the auxiliary storage device 76, stores the coordinates of the origin of the map coordinate system in the camera coordinate system. The coordinates of the origin of the map coordinate system in the camera coordinate system can be said to be the deviation of the coordinates between the origin of the camera coordinate system and the origin of the map coordinate system. When the host control device 70 derives the camera coordinates of the palette P, the host controller 70 shifts the camera coordinates of the palette P by the coordinates of the origin of the map coordinate system in the camera coordinate system. As a result, the camera coordinates of the pallet P become the coordinates with reference to the origin of the map coordinate system, and the map coordinates of the pallet P are derived by the upper control device 70.

In each embodiment, if the coordinates of the camera 60 in the map coordinate system can be grasped and the directions of the coordinate axes of the camera coordinate system and the directions of the coordinate axes of the map coordinate system are matched, the camera coordinates of the palette P The conversion from to the map coordinates of the palette P may be performed by the control device 53. When the host control device 70 derives the camera coordinates of the pallet P, it transmits the camera coordinates of the pallet P to the forklift 10. When the control device 53 receives the camera coordinates of the palette P, the control device 53 derives the map coordinates of the palette P from the camera coordinates of the palette P. A storage device that stores readable information of the control device 53, such as the storage unit 55 and the auxiliary storage device 56, stores the coordinates of the camera 60 in the map coordinate system. When the control device 53 receives the camera coordinates of the palette P, the control device 53 shifts the camera coordinates of the palette P by the amount of deviation between the origin of the map coordinate system and the coordinates of the camera 60 in the map coordinate system. As a result, the control device 53 can derive the map coordinates of the palette P from the camera coordinates of the palette P. If the coordinates of the camera 60 in the map coordinate system can be grasped in this way, the control device 53 can acquire the map coordinates from the camera coordinates of the palette P to the palette P. Therefore, the information for obtaining the coordinates of the palette P (movement target) in the map coordinate system may be the map coordinates of the palette P as in the embodiment, or the camera coordinates of the palette P as described above. You may. In other words, the process of converting the camera coordinates of the pallet P into the map coordinates of the pallet P may be performed by the upper control device 70 or may be performed by the control device 53. The cargo handling systems CS and CS1 may include a coordinate conversion unit that converts the camera coordinates of the palette P into the map coordinates of the palette P.

As described above, at least one of the upper control device 70 and the control device 53 can grasp the deviation of the coordinates between the origin of the camera coordinate system and the origin of the map coordinate system, and the direction of the coordinate axes of the camera coordinate system. As long as the directions of the coordinate axes of the map coordinate system are matched with each other, the camera coordinates of the two pillars Pi1 and Pi2 need not be derived by the host control device 70. Therefore, it is not necessary to set the stop positions A1 and A11 according to the two pillars Pi1 and Pi2, and the degree of freedom of the stop positions A1 and A11 is improved. In addition, the number of processes performed by the host control device 70 can be reduced.

Even if the orientation of the coordinate axes of the camera coordinate system and the orientation of the coordinate axes of the map coordinate system do not match, the deviation angle between the coordinate axes of the camera coordinate system and the coordinate axes of the map coordinate system and the direction of the camera coordinate system If the deviation between the origin and the origin of the map coordinate system can be grasped, the map coordinates of the moving target can be derived from the camera coordinates of the moving target. Even in this case, either the upper control device 70 or the control device 53 can convert the camera coordinates of the moving target to the map coordinates of the moving target. Then, the same effect as the above-mentioned effect can be obtained.

○ In each embodiment, the forklift 10 does not have to include the in-vehicle camera 45. In this case, the movement of the forklift 10 to the pallet P is performed only by the camera coordinates of the pallet P received from the host control device 70.

○ In each embodiment, the forklift 10 may include a millimeter-wave radar, a stereo camera, a LIDAR, or the like, in which the control device 53 can detect the pallet P, instead of the in-vehicle camera 45.

○ In the first embodiment and the third embodiment, the number of markers provided on the pallet P may be one. In this case, the map coordinates of the marker become the map coordinates of the palette P. Further, the host control device 70 can derive the posture of the marker in the camera coordinate system from the distortion of the marker in the image data. The host control device 70 can derive the posture of the palette P in the map coordinate system by deriving the posture of the marker in the environment map from the posture of the marker in the camera coordinate system.

-In each embodiment, the origin of the map coordinate system and the coordinates of the first fixed marker MA1 do not have to match. In this case, the deviation between the coordinates of the first fixed marker MA1 in the map coordinate system and the origin of the map coordinate system is grasped in advance, and the control unit 75 such as the storage unit 72 of the upper control device 70 and the auxiliary storage device 76 The deviation between the coordinates of the first fixed marker MA1 in the map coordinate system and the origin of the map coordinate system is stored in a readable storage device. The upper control device 70 shifts the coordinates of the markers MA3 and MA4 by the three-sided survey by the amount of the deviation between the coordinates of the first fixed marker MA1 in the map coordinate system and the origin of the map coordinate system. As a result, the upper control device 70 can derive the coordinates of the markers MA3 and MA4 in the map coordinate system, and by extension, the coordinates of the palette P in the map coordinate system.

-In the third embodiment, the origin of the map coordinate system and the coordinates of the first pillar Pi1 do not have to match. In this case, the deviation between the coordinates of the first pillar Pi1 in the map coordinate system and the origin of the map coordinate system is grasped in advance, and the control unit 75 such as the storage unit 72 of the upper control device 70 or the auxiliary storage device 76 reads. The deviation between the coordinates of the first pillar Pi1 in the map coordinate system and the origin of the map coordinate system is stored in a possible storage device. When the host control device 70 converts the sensor coordinates of the irradiation point P1 into map coordinates, the upper control device 70 is based on the embodiment by the amount of deviation between the coordinates of the first pillar Pi1 in the map coordinate system and the origin of the map coordinate system. Also extraly moves the irradiation point P1 in parallel. As a result, the host control device 70 can convert the sensor coordinates of the irradiation point P1 into map coordinates.

-In the first embodiment and the third embodiment, the upper control device 70 may omit the equation (5) and derive the Y coordinate yp from the equation (6).
○ In the first embodiment and the third embodiment, if it is possible to grasp in advance whether the Y coordinate yp of the marker MA3 becomes the + coordinate or the − coordinate, it is not necessary to perform the determination by the equation (6). For example, in the Y-axis direction of the map coordinate system, the stop positions A1 and A11 are set only on the + coordinate side, and the stops of the tracks T and TC on the-coordinate side are prohibited, so that the Y coordinate of the marker MA3 becomes the + coordinate. .. In this case, it is not necessary to determine whether the Y coordinate yp becomes the + coordinate or the − coordinate, so that the processing performed by the upper control device 70 can be reduced.

○ In the third embodiment, the upper control device 70 derives the map coordinates of the container C1 from the measurement result of the laser range finder 81 when performing the loading work of loading the pallet P loaded on the container C1 on the forklift 10. You may.

-In the first embodiment, the upper control device 70 does not have to include at least one of the auxiliary storage device 76 and the input device 77.
-In the second embodiment, the trained model may be one in which the coordinates with the pillar Pi1 as the origin are derived by inputting the image data in which the palette P and the two pillars Pi1 and Pi2 are captured. As the teacher data in this case, data obtained by adding the coordinates of the palette P having the pillar Pi1 as the origin to the image data IM obtained by imaging the palette P and the two pillars Pi1 and Pi2 as labels is used. Depending on the positional relationship between the pallet P and the two pillars Pi1 and Pi2, the ratio of the pallet P and the pillar Pi1 in the image data, the positional relationship between the position of the pallet P and the pillar Pi1 in the image data, and the like are different. Therefore, in the trained model in which machine learning is performed using the above-mentioned teacher data, by inputting the image data in which the palette P and the two pillars Pi1 and Pi2 are captured, the coordinates of the palette P with the pillar Pi1 as the origin can be obtained. can get. In this case, as in the first embodiment, by setting the pillar Pi1 as the origin of the map coordinate system, the coordinates of the pallet P derived by the upper control device 70 become the map coordinates of the pallet P.

○ In each embodiment, the forklift 10 may be a forklift capable of switching between automatic operation and manual operation.
○ In each embodiment, the forklift 10 may be a reach type forklift provided with a reach cylinder capable of moving the cargo handling device 14 forward and backward.

○ In each embodiment, the communication unit 59 may have at least a function of receiving a command from the command communication unit 73, and may not have a transmission function.
○ In each embodiment, the command communication unit 73 may have at least a function of transmitting a command to the communication unit 59, and may not have a reception function.

-In each embodiment, the sensor coordinate derivation unit, the map coordinate derivation unit, and the derivation unit may be configured by separate devices.
○ In each embodiment, the storage device for storing the environmental map may be any storage medium that can store the readable information of the control device 53. For example, it may be a non-volatile memory such as EEPROM.

○ In each embodiment, the existing structure provided with the fixed markers MA1 and MA2 may be a fixed structure other than the pillars Pi1 and Pi2. Further, a structure for providing the fixed markers MA1 and MA2 may be installed.

-In each embodiment, the cargo handling systems CS and CS1 may include a plurality of cameras 60.
○ In each embodiment, a plurality of stop positions A1 and A11 may be set. In this case, cameras 60 are individually provided for each of the stop positions A1 and A11. The upper control device 70 may be provided one for each camera 60, or may be only one for each of the plurality of cameras 60. The forklift 10, the camera 60, and the host control device 70 included in the cargo handling systems CS and CS1 may be singular or plural, respectively. Further, in the third embodiment, the laser range finder 81 and the truck detection camera 82 included in the cargo handling system CS1 may be singular or plural, respectively.

-In the third embodiment, the cargo handling system CS1 may be individually provided with an upper control device for each of the camera 60, the laser range finder 81, and the truck detection camera 82. That is, a device that derives the map coordinates of the palette P from the image pickup result of the camera 60, a device that detects the map coordinates of the container C1 from the measurement result of the laser range finder 81, and a track TC that detects the track TC from the image pickup result of the track detection camera 82. It may be provided separately from the device to be used.

-In the third embodiment, the cargo handling system CS1 does not have to include the truck detection camera 82. Even in this case, the host control device 70 can detect that the truck TC is stopped at the stop position A11 from the measurement result of the laser range finder 81. Since the obtained irradiation point P1 changes depending on whether or not the truck TC is stopped at the stop position A11, the host control device 70 can determine the presence or absence of the truck TC from the measurement result of the laser range finder 81.

-In the third embodiment, the upper control device 70 may derive the posture of the container C1 from the map coordinates of the edges E1 and E2. The posture of the container C1 is the inclination of the container C1 with respect to the X-axis or the Y-axis of the map coordinate system. The upper control device 70 can derive the inclination of the straight line including the two edges E1 and E2 with respect to the X-axis or the Y-axis as the attitude of the container C1.

The host control device 70 may transmit the posture of the container C1 in the map coordinate system to the forklift 10. The control device 53 controls the traveling direction of the forklift 10 from the posture of the container C1 in the map coordinate system. Specifically, the control device 53 controls the traveling direction of the forklift 10 so that the forklift 10 travels perpendicularly to the opening O11.

-In the third embodiment, as shown in FIGS. 11 and 12, the container C1 includes two edges E1 and E3 that are adjacent to each other in the lateral direction as adjacent edges that are horizontally separated from each other. The upper control device 70 may detect the positions of these edges E1 and E3 as adjacent edges that are horizontally separated from each other. Since these edges E1 and E3 are located with the side wall SW1 sandwiched between them, if a straight line at the irradiation point P1 obtained by irradiating the side wall SW1 with a laser can be extracted, these edges E1 and E3 are irradiated. The irradiated irradiation point P1 can be extracted. Depending on the positional relationship between the laser range finder 81 and the stop position A11 and the direction of the truck TC, the side wall SW1 is irradiated compared to the number of irradiation points P1 obtained by irradiating the front walls FW1 and FW2 with the laser. As a result, the number of irradiation points P1 obtained is smaller. Therefore, assuming the positional relationship between the laser range finder 81 and the stop position A11 and the direction of the truck TC, the two edges E1 and E3 include a straight line having the second largest number of irradiation points P1 located on the straight line. It may be extracted as a straight line. The host control device 70 can derive the posture of the container C1 in the map coordinate system by detecting the positions of the edges E1 and E3 adjacent to each other in the lateral direction. Further, depending on the type of container, the door is located in the longitudinal direction of the container, and loading may be performed from the opening in the longitudinal direction of the container. When loading this type of container, the intermediate positions of the edges E1 and E3 that are adjacent to each other in the lateral direction may be derived as the map coordinates of the container.

-In the third embodiment, as shown in FIGS. 11 and 12, the container C1 includes edges E4 and E5 on the inner surfaces of the rear walls RW1 and RW2 as adjacent edges that are horizontally separated from each other. More specifically, the container C1 includes an edge E4 at which the inner surface of the rear wall RW1 and the inner surface of the side wall SW1 intersect, and an edge E5 at which the inner surface of the rear wall RW2 and the inner surface of the side wall SW2 intersect. The two edges E4 and E5 are edges adjacent to each other in the longitudinal direction of the container C1. The upper control device 70 may detect the positions of these edges E4 and E5 as edges that are horizontally separated from each other and adjacent to each other. The upper control device 70 can derive the positions of the edges E4 and E5 by the same method as that of the embodiment. It can be said that the upper control device 70 can derive the positions of edges adjacent to each other horizontally separated from each other regardless of whether the front walls FW1 and FW2 of the container C1 are open or closed. .. Then, the upper control device 70 can derive the map coordinates of the container C1 from the positions of the edges E4 and E5.

○ In the third embodiment, in step S24, the upper control device 70 extracts a straight line including two edges E1 and E2 in the XY plane in the map coordinate system by regression analysis such as the least squares method and the Hough transform. May be good.

-In the third embodiment, the upper control device 70 may extract a straight line from the irradiation point P1 existing in a predetermined range in step S24. The positions of the edges E1 and E2 when the truck TC stops at the stop position A11 can be assumed in advance. The upper control device 70 may extract a straight line from the irradiation point P1 existing in the range where the edges E1 and E2 are assumed to be located when the truck TC stops at the stop position A11.

-In the third embodiment, the upper control device 70 may extract all the straight lines obtained by the combination of the plurality of irradiation points P1 in step S24.
-In the third embodiment, the upper control device 70 may remove outliers by using an outlier removal filter before extracting a straight line from the irradiation point P1.

-In the third embodiment, the forklift 10 may be guided to the pallet loaded on the container C1 by using the laser range finder 81. As an example, a case where the flat pallet loaded on the container C1 is set as a movement target and the forklift 10 is guided to the flat pallet will be described.

As shown in FIG. 18, the pallet FP is a flat pallet. The pallet FP includes a demarcation surface HS that defines an insertion hole H1 into which the fork 31 is inserted, and an outer surface OFP into which the insertion hole H1 opens. Two insertion holes H1 are provided. The pallet FP is placed in the container C1 with the insertion hole H1 open in the horizontal direction. The upper control device 70 derives the positions of the two edges E11 and E12 that are most separated from each other in the horizontal direction among the outer surface OFPs, and derives the intermediate positions of the two edges E11 and E12 as the map coordinates of the palette FP. Can be done. As can be seen from FIG. 18, when the outer surface OFP is irradiated with the laser, the irradiation point P1 is interrupted at the position where the insertion hole H1 exists. The irradiation points P1 obtained by irradiating the two edges E11 and E12 with a laser are designated as edge irradiation points PE11 and PE12. The upper control device 70 extracts a straight line including the edges E11 and E12 on the XY plane of the map coordinate system, and then extracts the edge irradiation points PE11 and PE12 from this straight line. The upper control device 70 sets two irradiation points P1 of the irradiation points P1 located on the straight line, which do not have adjacent irradiation points P1 for a predetermined distance or more with respect to one side in the extending direction of the straight line, as the edge irradiation point PE11. , PE12. In this case, as a predetermined distance, a distance longer than the dimension of the insertion hole H1 in the direction orthogonal to the extending direction of the insertion hole H1 in the horizontal direction is set. As a result, the irradiation point P1 irradiated to the edges E13 and E14 due to the intersection of the demarcation surface HS and the outer surface OFP can be prevented from being extracted as the edge irradiation point.

Further, the pallet FP includes two edges E13 and E14 that are closest to each other in the horizontal direction among the edges formed by the intersection of the two defining surfaces HS and the outer surface OFP. The host control device 70 can derive the position and orientation of the pallet P in the map coordinate system by detecting the positions of the two edges E13 and E14. The irradiation points P1 obtained by irradiating the two edges E13 and E14 with a laser are designated as edge irradiation points PE13 and PE14. In this case, as a predetermined distance, a distance shorter than the dimension of the insertion hole H1 in the direction orthogonal to the extending direction of the insertion hole H1 in the horizontal direction is set. This makes it possible to extract the irradiation point P1 by irradiating the two edges E13 and E14 with the laser as the edge irradiation points PE13 and PE14. In this case, the irradiation points P1 irradiated on the two edges E11 and E12 described above can also be extracted as the edge irradiation points PE11 and PE12. When the four edge irradiation points PE11, PE12, PE13, and PE14 are extracted, the irradiation points P1 irradiated to the two edges E11 and E12 are removed, such as removing the two edge irradiation points PE11 and PE12 that are most separated from each other. It may be prevented from being extracted as the edge irradiation points PE11 and PE12.

If the characteristics of the pallet can be detected by using the laser range finder 81, the map coordinates of the pallet can be derived by using the laser range finder 81, not limited to the flat pallet. When the map coordinates of the pallet can be derived using the laser range finder 81, the host controller 70 can use the laser range finder 81 to cause the forklift 10 to perform both the loading operation and the loading operation. Therefore, the cargo handling system CS1 does not have to include the camera 60. When the host control device 70 derives the map coordinates of the moving target without using the camera 60, the fixed markers MA1 and MA2 need not be provided on the pillars Pi1 and Pi2. Similarly, it is not necessary to provide a marker on the movement target.

The upper control device 70 can also derive the sensor coordinates of the pallet FP. In this case, the above processing may be performed using the sensor coordinates instead of the map coordinates. When the upper control device 70 derives the sensor coordinates of the pallet FP, the conversion from the sensor coordinates of the pallet FP to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70. You may. When the control device 53 performs the conversion from the sensor coordinates of the pallet FP to the map coordinates, the orientation of the coordinate axes of the sensor coordinate system and the orientation of the coordinate axes of the map coordinate system are matched. When the control device 53 receives the sensor coordinates of the pallet FP, the control device 53 shifts the sensor coordinates of the pallet FP by the amount of deviation between the origin of the sensor coordinate system and the origin of the map coordinate system. When the upper control device 70 converts the camera coordinates of the pallet FP to the map coordinates, the upper control device 70 uses the sensor coordinates of the pallet FP and the sensor coordinates of the columns Pi1 and Pi2 to perform a three-sided survey of the pallet FP. The map coordinates may be derived, or the map coordinates of the palette FP may be derived using the inclination θ between the sensor coordinate system and the map coordinate system. The information for obtaining the coordinates of the palette FP in the map coordinate system may be the map coordinates of the palette FP or the sensor coordinates of the palette FP.

-In each embodiment, the operation of the input device 77 performed by the operator of the host control device 70 may intervene in the various processes performed by the host control device 70. In each embodiment, when the input device 77 instructs the input device 77 to derive the map coordinates of the pallet P, the upper control device 70 may derive the map coordinates of the pallet P and guide the forklift 10 to the pallet P. good. The host control device 70 may transmit the movement start command to the forklift 10 when the input device 77 instructs the forklift 10 to transmit the movement start command. The host control device 70 may transmit a unloading command to the forklift 10 when the input device 77 instructs the start of the unloading work of the forklift 10. The unloading command is a command for the forklift 10 to start the unloading work. When the host control device 70 transmits a unloading command, the forklift 10 stands by without performing unloading work from the time it moves to the pallet P until it receives the unloading command.

In the third embodiment, when the input device 77 instructs the input device 77 to derive the map coordinates of the container C1, the upper control device 70 derives the map coordinates of the container C1 and guides the forklift 10 to the container C1. May be good. The host control device 70 may transmit a loading command to the forklift 10 when the input device 77 instructs the start of the loading operation of the forklift 10. The loading command is a command for starting the loading work on the forklift 10. When the host control device 70 transmits a loading command, the forklift 10 stands by without performing the loading operation from the time it moves to the container C1 until it receives the loading command.

-In the third embodiment, the upper control device 70 may perform only one of the first edge detection process and the second edge detection process. When the upper control device 70 performs only the first edge detection process, the upper control device 70 sets the map coordinates of the edge irradiation points PE1 and PE2 detected by the first edge detection process as the positions of the edges E1 and E2. When the upper control device 70 performs only the second edge detection process, the upper control device 70 sets the map coordinates of the edge irradiation points PE1 and PE2 detected by the second edge detection process as the positions of the edges E1 and E2. Further, the upper control device 70 does not have to correct the positions of the edges E1 and E2. The positions of the edges E1 and E2 may be the map coordinates of the edge irradiation points PE1 and PE2 detected by the first edge detection process, or may be the map coordinates after correcting the map coordinates. The positions of the edges E1 and E2 may be the map coordinates of the edge irradiation points PE1 and PE2 detected by the second edge detection process, or may be the map coordinates after correcting the map coordinates. The positions of the edges E1 and E2 may be map coordinates derived by using both the first edge detection process and the second edge detection process as in the embodiment, or after the map coordinates are corrected. It may be map coordinates. In either case, the map coordinates of the container C1 can be derived from the positions of the edges E1 and E2, and the forklift 10 can be guided.

-In the third embodiment, the filter processing performed when correcting the positions of the edges E1 and E2 may be a Kalman filter or a moving average filter.
○ In the third embodiment, if the coordinates of the origin of the map coordinate system in the sensor coordinate system can be grasped and the direction of the coordinate axis of the sensor coordinate system and the direction of the coordinate axis of the map coordinate system are matched, It is not necessary to derive the slope θ. A storage device that stores information that can be read by the control unit 75, such as the storage unit 72 and the auxiliary storage device 76, stores the coordinates of the origin of the map coordinate system in the sensor coordinate system. The coordinates of the origin of the map coordinate system in the sensor coordinate system can be said to be the deviation of the coordinates between the origin of the sensor coordinate system and the origin of the map coordinate system. The host control device 70 shifts the sensor coordinates of the irradiation point P1 by the coordinates of the origin of the map coordinate system in the sensor coordinate system. As a result, the sensor coordinates of the irradiation point P1 become coordinates with reference to the origin of the map coordinate system, and the sensor coordinates of the irradiation point P1 can be converted into map coordinates.

Since the upper control device 70 does not have to derive the inclination θ, the upper control device 70 does not have to acquire the sensor coordinates of the two pillars Pi1 and Pi2 in order to derive the inclination θ. Therefore, it is not necessary to set the stop position A1 according to the two pillars Pi1 and Pi2, and the degree of freedom of the stop position A1 is improved.

-In the third embodiment, the host control device 70 may convert the sensor coordinates of the container C1 into the map coordinates of the container C1 after deriving the sensor coordinates of the container C1. The sensor coordinates of the container C1 can be derived by performing the processes performed in steps S24 to S29 using the sensor coordinates. The conversion from the sensor coordinates of the container C1 to the map coordinates can be performed by the same process as in step S23. Further, the sensor coordinates of the container C1 may be converted to the map coordinates by the three-sided survey using the columns Pi1 and Pi2.

If the coordinates of the laser distance meter 81 in the map coordinate system can be grasped and the directions of the coordinate axes of the sensor coordinate system and the directions of the coordinate axes of the map coordinate system are matched, the sensor coordinates of the container C1 can be used. The conversion of the container C1 to the map coordinates may be performed by the control device 53. When the host control device 70 derives the sensor coordinates of the container C1, it transmits the sensor coordinates of the container C1 to the forklift 10. When the control device 53 receives the sensor coordinates of the container C1, the control device 53 derives the map coordinates of the container C1 from the sensor coordinates of the container C1. The coordinates of the laser rangefinder 81 in the map coordinate system are stored in the storage device that stores the readable information of the control device 53 such as the storage unit 55 and the auxiliary storage device 56. When the control device 53 receives the sensor coordinates of the container C1, the control device 53 shifts the sensor coordinates of the container C1 by the deviation between the origin of the map coordinate system and the coordinates of the laser rangefinder 81 in the map coordinate system. As a result, the control device 53 can derive the map coordinates of the container C1 from the sensor coordinates of the container C1. If the coordinates of the laser range finder 81 in the map coordinate system can be grasped in this way, the control device 53 can acquire the map coordinates of the container C1 from the sensor coordinates of the container C1. In this case, the information obtained from the coordinates of the container C1 in the map coordinate system is the sensor coordinates of the container C1.

-In the first embodiment, the movement target may be the track T. When the movement target is the track T, a marker is provided on the track T. The host control device 70 derives the camera coordinates of the marker as the camera coordinates of the track T. The conversion from the camera coordinates of the track T to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70. When the control device 53 performs the conversion from the camera coordinates of the track T to the map coordinates, the orientation of the coordinate axes of the camera coordinate system and the orientation of the coordinate axes of the map coordinate system are matched. When the control device 53 receives the camera coordinates of the track T, the control device 53 shifts the camera coordinates of the track T by the deviation between the origin of the camera coordinate system and the origin of the map coordinate system. When the upper control device 70 converts the camera coordinates of the track T into the map coordinates, the upper control device 70 derives the map coordinates of the track T by the three-sided survey using the columns Pi1 and Pi2 using the markers. be able to. The information for obtaining the coordinates of the track T in the map coordinate system may be the map coordinates of the track T or the camera coordinates of the track T. The upper control device 70 can guide the forklift 10 to the truck T.

Similarly, in the third embodiment, the movement target may be the track TC. When the movement target is the track TC, the host controller 70 may derive the camera coordinates of the track TC from the trained model generated by inputting the teacher data into the machine learning model. The trained model is stored in a readable storage device of the control unit 75 such as the storage unit 72 of the upper control device 70 and the auxiliary storage device 76. The host control device 70 derives the camera coordinates of the track TC from the image data input from the camera 60. Machine learning can be performed by the same method as in the second embodiment. As the teacher data, data obtained by adding the camera coordinates of the track TC as a label to the image data obtained by capturing the track TC is used. When the truck TC stops at the stop position A11, the host control device 70 can derive the camera coordinates of the truck TC. The conversion from the camera coordinates of the track TC to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70. When the control device 53 performs the conversion from the camera coordinates of the track TC to the map coordinates, the orientation of the coordinate axes of the camera coordinate system and the orientation of the coordinate axes of the map coordinate system are matched. When the control device 53 receives the camera coordinates of the track TC, the control device 53 shifts the camera coordinates of the track TC by the deviation between the origin of the camera coordinate system and the origin of the map coordinate system. When the upper control device 70 performs the conversion from the camera coordinates of the track TC to the map coordinates, the upper control device 70 uses the camera coordinates of the track TC and the camera coordinates of the columns Pi1 and Pi2 to perform a three-sided survey of the track TC. The map coordinates may be derived, or the map coordinates of the track TC may be derived using the inclination of the camera coordinate system and the map coordinate system. The upper control device 70 can guide the forklift 10 to the truck TC.

○ In the third embodiment, a marker may be provided in the container C1 so that the forklift 10 can be guided to the container C1 by using the marker. The host control device 70 derives the camera coordinates of the marker as the camera coordinates of the container C1. The conversion from the camera coordinates of the container C1 to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70. When the control device 53 performs the conversion from the camera coordinates of the container C1 to the map coordinates, the orientation of the coordinate axes of the camera coordinate system and the orientation of the coordinate axes of the map coordinate system are matched. When the control device 53 receives the camera coordinates of the container C1, the control device 53 shifts the camera coordinates of the container C1 by the deviation between the origin of the camera coordinate system and the origin of the map coordinate system. When the upper control device 70 performs the conversion from the camera coordinates of the container C1 to the map coordinates, the upper control device 70 performs a three-sided survey using the camera coordinates of the container C1 and the camera coordinates of the columns Pi1 and Pi2 to perform the conversion of the container C1. The map coordinates may be derived, or the map coordinates of the container C1 may be derived using the inclination of the camera coordinate system and the map coordinate system. The information for obtaining the coordinates of the container C1 in the map coordinate system may be the map coordinates of the container C1 or the camera coordinates of the container C1.

○ In the third embodiment, the movement target may be the container C1. When the movement target is the container C1, the host control device 70 inputs the teacher data into the machine learning model and derives the camera coordinates of the container C1 by the trained model generated. The trained model is stored in a readable storage device of the control unit 75 such as the storage unit 72 of the upper control device 70 and the auxiliary storage device 76. The host control device 70 derives the camera coordinates of the container C1 from the image data input from the camera 60. Machine learning can be performed by the same method as in the second embodiment. As the teacher data, data obtained by adding the camera coordinates of the container C1 as a label to the image data obtained by capturing the image of the container C1 is used. When the truck TC stops at the stop position A11, the host control device 70 can derive the camera coordinates of the container C1. The conversion from the camera coordinates of the container C1 to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70. The upper control device 70 can guide the forklift 10 to the container C1.

-In the third embodiment, the map coordinates of the track TC may be derived by using the laser range finder 81. For example, the host control device 70 converts the sensor coordinates of the irradiation point P1 when the track TC is irradiated with the laser into map coordinates. The upper control device 70 extracts the irradiation point P1 that can be a feature from the pattern of the irradiation point P1. The map coordinates of the irradiation point P1 may be used as the map coordinates of the track TC. Similarly, the host controller 70 may derive the sensor coordinates of the track TC from the pattern of the irradiation point P1. In this case, the conversion from the sensor coordinates of the track TC to the map coordinates may be performed by the control device 53 or may be performed by the upper control device 70.

-In the first embodiment, the map coordinates of the palette P may be derived using the inclination of the camera coordinate system and the map coordinate system. The slope between the camera coordinate system and the map coordinate system can be derived in the same manner as in deriving the slope θ described in the third embodiment. The host control device 70 can convert the camera coordinates to the map coordinates in the same manner as the conversion from the sensor coordinates to the map coordinates described in the third embodiment.

-In the third embodiment, the camera 60 may also be used as the track detection camera.
○ In the third embodiment, the laser rangefinder 81 may be capable of irradiating the laser while changing the irradiation angle in the horizontal direction at least, and the laser while changing the irradiation angle in the vertical direction in addition to the horizontal direction. It may be able to irradiate. In this case, the sensor coordinate system may include an X-axis and a Z-axis orthogonal to the Y-axis as coordinate axes.

-In each embodiment, a millimeter-wave radar or a stereo camera may be used as the sensor.
○ In each embodiment, the pallet P may be any pallet such as a flat pallet or a post pallet.

-In each embodiment, the trucks T and TC loaded with the pallets P on which the forklift 10 performs the loading work are not limited to flat body trucks, but may be container trucks and the like. The transport vehicle loaded with the pallet P to which the forklift 10 performs the loading work may be a vehicle other than the trucks T and TC.

○ The container C1 may be capable of accommodating the load W and may be transported by the forklift 10.
○ In each embodiment, the trucks T and TC may stop in any way as long as the map coordinates of the container C1 can be derived using the two pillars Pi1 and Pi2.

A1, A11 ... Stop position, C1 ... Container as a moving target, CS, CS1 ... Cargo handling system, DA ... Detectable range, IM ... Image data, MA1, MA2 ... Fixed marker, MA3, MA4 ... Marker, P, FP ... Pallets as movement targets, Pi1, Pi2 ... Pillars as fixed structures, T, TC ... Movement targets and trucks as transport vehicles, 10 ... Forklifts, 53 ... Control devices that function as movement control units, 56 ... As storage devices Auxiliary storage device, 59 ... Communication unit as a receiver, 60 ... Camera as a sensor, 70 ... Higher control device, 73 ... Command communication unit as a transmitter, 81 ... Laser distance meter as a sensor.

Claims (5)

A sensor provided so that a predetermined stop position is included in the detectable range, which is a stop position at which the transport vehicle that transports the pallet stops.
Forklift and
A cargo handling system including a host control device configured to guide the forklift to the movement target with the pallet, the transport vehicle, or the container as a movement target.
The forklift
A storage device configured to store an environment map in which the environment in which the forklift is used is represented by coordinates in a map coordinate system.
A receiver configured to be able to receive information transmitted from the host controller,
A movement control unit that moves the forklift toward the coordinates of the movement target, which is the coordinates of the map coordinate system obtained from the information received by the receiving unit, is provided.
The upper control device is
A derivation unit that derives information from which the coordinates of the moving target in the map coordinate system can be obtained from the detection result of the sensor, and a derivation unit.
A cargo handling system including a transmission unit that transmits information for obtaining coordinates of the movement target in the map coordinate system to the reception unit. The cargo handling system according to claim 1, wherein the upper control device includes a map coordinate deriving unit that derives the coordinates of the moving target in the map coordinate system as information for obtaining the coordinates of the moving target in the map coordinate system. .. The sensor is arranged such that the two fixed structures are within the detectable range.
The upper control device includes a sensor coordinate deriving unit that derives the coordinates of the moving target in the sensor coordinate system of the sensor from the detection result of the sensor.
The two fixed structures are provided so as to be located on the same coordinate axis of the map coordinate system.
One of the two fixed structures is provided so as to be located at the origin of the map coordinate system.
The sensor coordinate deriving unit derives the coordinates of the two fixed structures and the moving target in the sensor coordinate system, respectively.
The map coordinate derivation unit uses the coordinates of the two fixed structures in the sensor coordinate system and the coordinates of the movement target in the sensor coordinate system to perform a three-sided survey to determine the movement target in the map coordinate system. The cargo handling system according to claim 2, wherein the coordinates of the above are derived. The sensor is a camera
The camera is arranged so as to image a fixed marker provided on each of the two fixed structures and a marker provided on the moving target.
The sensor coordinate deriving unit derives the coordinates of the fixed marker in the sensor coordinate system as the coordinates of the fixed structure in the sensor coordinate system, and the coordinates of the marker in the sensor coordinate system of the moving target. The cargo handling system according to claim 3, which is derived as coordinates in the sensor coordinate system. The sensor is a camera
The upper control device derives the coordinates of the moving target in the sensor coordinate system of the sensor from the trained model generated by inputting the teacher data into the learner.
The cargo handling system according to claim 1, wherein the trained model is generated as the teacher data by adding data obtained by adding the coordinates of the movement target in the sensor coordinate system as a label to the image data obtained by imaging the movement target.

Images