WO2021066138A1

WO2021066138A1 - Operation assistance device for cargo handling vehicle

Info

Publication number: WO2021066138A1
Application number: PCT/JP2020/037532
Authority: WO
Inventors: 浩伸岡本; 順治井上
Original assignee: 株式会社豊田自動織機
Priority date: 2019-10-02
Filing date: 2020-10-02
Publication date: 2021-04-08
Also published as: JP7287231B2; JP2021054635A

Abstract

Provided is an operation assistance device for a cargo handling vehicle equipped with a vertically movable cargo handling unit. The operation assistance device is provided with: a camera configured so as to capture an image of the front of the cargo handling unit; a display unit configured so as to display a camera image captured by the camera; and a guide superimposition unit configured so as to acquire a cargo collection position or a cargo placement position as a target position and cause the display unit to display the camera image with a guide superimposed on a floor surface positioned below the target position.

Description

Operation support device for cargo handling vehicles

This disclosure relates to an operation support device for a cargo handling vehicle.

Patent Document 1 discloses a location indicating device and a location management system. In this device and system, the target position acquired from the map data is displayed in the image that can be seen from the moving body taken by the camera so that the target location of the loading / unloading work can be grasped quickly and accurately. ..

Japanese Unexamined Patent Publication No. 2006-11415

However, when performing unloading work or unloading work on a rack, especially on a high shelf, the unloading position or unloading position deviates from the camera image as the vehicle approaches the rack, and the target unloading position or unloading position The position disappears. Therefore, it becomes difficult to approach the vehicle according to the target unloading position or unloading position.

An object of the present disclosure is to provide an operation support device for a cargo handling vehicle that can guide the target position when the vehicle approaches the target position.

The operation support device for the cargo handling vehicle for achieving the above object is an operation support device for the cargo handling vehicle provided with a vertically movable cargo handling unit, and includes a camera configured to image the front of the cargo handling unit. A display unit configured to display a camera image captured by the camera and a loading position or loading position are acquired as target positions, and the display unit is positioned below the target position with respect to the camera image. A guide superimposing portion configured to superimpose the guide on the floor surface to be displayed is provided.

The block diagram which shows the electrical structure of the remote control system for a forklift. Schematic side view showing a forklift. The schematic perspective view which shows by breaking a part of a forklift. The plan view which shows typically the forklift. Schematic plan view showing shelves and forklifts in the workplace. The figure for demonstrating the display contents on the display part. The figure for demonstrating the display contents on the display part. (A) is a shelf for explaining the guide line and a plan view of the pallet, (b) is a shelf for explaining the guide line and a front view of the pallet, and (c) is a shelf for explaining the guide line. , And the right side view of the pallet. (A) is a plan view of a shelf and a pallet for explaining a guide line in another example, (b) is a front view of a shelf and a pallet for explaining a guide line in another example, and (c) is another example. The right side view of the shelf and the pallet for explaining the guide line in.

Hereinafter, an embodiment embodying the present disclosure will be described with reference to the drawings.
In the present embodiment, the operation support device for the cargo handling vehicle is used for the remote control system for the forklift as the remote control system for the cargo handling vehicle. In the remote control system for forklifts, work is managed while detecting the relative position between the target position and the vehicle using map information (map information) in the controller for the work management system.

As shown in FIG. 1, the forklift remote control system 10 includes a reach-type forklift 20 as a cargo handling vehicle and a remote control device 40 used for remotely controlling the traveling of the forklift 20 and the cargo handling by the cargo handling device. I have. The forklift 20 is arranged in the workplace. Then, the forklift 20 in the workplace can be remotely controlled from the operation room by using the remote control device 40.

The forklift 20 is located in the workplace away from the rack. From this state, the operator remotely controls the forklift 20 to bring the forklift 20 closer to the rack and insert the fork into the hole of the pallet placed on the shelf of the rack. It is possible to carry out loading and / or loading with such a pallet.

As shown in FIGS. 2 and 3, the forklift 20 includes a machine base 21. A pair of left and

right reach legs

22a and 22b are arranged on the front side of the machine base 21, and the

reach legs

22a and 22b extend forward. Specifically, the reach leg 22a is provided on the right side in the traveling direction, and the reach leg 22b is provided on the left side in the traveling direction. A right front wheel 23a and a left front wheel 23b are arranged at the front portions of the

reach legs

22a and 22b. Specifically, the right front wheel 23a is provided on the reach leg 22a on the right side in the traveling direction, and the left front wheel 23b is provided on the reach leg 22b on the left side in the traveling direction. In this way, two paired

front wheels

23a and 23b are provided on the front side of the machine base 21.

A rear wheel 24 and caster wheels (auxiliary wheels) 25 are arranged at the rear of the machine base 21. The rear wheel 24 is provided on the left side of the machine base 21, and the caster wheel 25 is provided on the right side of the machine base 21. The rear wheels 24 are drive wheels and steering wheels.

The forklift 20 travels on two

front wheels

23a and 23b, one rear wheel 24 and one caster wheel 25, for a total of four wheels. The machine base 21 is equipped with a traveling motor 26 as a drive source for the forklift 20 and a battery 27 as a power source for the traveling motor 26. Then, the rear wheels 24 are driven by the traveling motor 26.

The forklift 20 is provided with a cargo handling device 28 in front of the machine base 21. The cargo handling device 28 includes a mast 29 that moves in the front-rear direction along the

reach legs

22a and 22b by driving a reach cylinder (not shown). A pair of left and

side forks

30a and 30b are provided in front of the mast 29 via a lift bracket 31. The

forks

30a and 30b have claws (hereinafter, referred to as fork claws) Nf extending forward. Specifically, the fork claw Nf extends in the horizontal direction when the tilt angle is 0. The

forks

30a and 30b move up and down along the mast 29. That is, the forklift 20 includes a fork claw Nf as a cargo handling portion that can move up and down.

The forklift 20 of the present embodiment is configured so that the driver can sit and operate it. The forklift 20 may be an unmanned forklift without a driver's seat.

As shown in FIG. 3, the forklift 20 is provided with a standing type driver's cab 32 at the rear of the machine base 21. Steering tables 33a and 33b are provided in front of and to the left of the driver's cab 32. The steering table 33a located in front of the driver's cab 32 includes a direction lever 34 for operating the forklift 20 and a plurality of cargo handling levers 35 for operating the cargo handling device 28. The direction lever 34 is operated to drive the rear wheels 24 to drive the vehicle. The steering table 33b located on the left side of the driver's cab 32 is provided with a steering wheel 36 for steering the rear wheels 24. Further, a brake pedal 37 is provided on the floor surface of the driver's cab 32.

The machine base 21 has two pillars 38 and a head guard 39. The driver's cab 32 is surrounded by two pillars 38 erected on the machine base 21 and a head guard 39 fixed to the upper end of the pillars 38. The head guard 39 has a plate shape that extends in the horizontal direction, and has a quadrangular shape in a plan view.

As shown in FIG. 1, the forklift 20 includes a controller 51 as a forklift-mounted device 50, a radio unit 52 as a vehicle communication unit, an image processing unit 53, and a radio 54 as a vehicle communication unit. It has

cameras

71 and 72.

The remote control device 40 includes a controller 61, an operation unit 62, a display unit (monitor) 63, and

radios

64 and 65 as operation device communication units. The remote control device 40 includes a controller 61, an operation unit 62, and a display unit (monitor) 63 as the operation room side device 60.

The radio 64 of the remote control device 40 is arranged in the work place. Further, the radio 65 of the remote control device 40 is arranged in the work place. The controller 61 arranged in the operation room is connected to the radio 64 arranged in the work place by the wired L1. The controller 61 is connected to the radio 65 arranged in the work place by a wired L2. Further, the controller 61 is connected to the work management system controller 90 that controls the entire work management system by a wired L3.

In the workplace, the wireless device 64 of the remote control device 40 and the wireless unit 52 of the forklift-mounted device 50 can perform wireless communication in both directions. Further, in the workplace, the radio 54 of the forklift-mounted device 50 can wirelessly communicate with the radio 65 of the remote control device 40.

In this way, the forklift 20 has a radio unit 52 and a radio 54, and the remote control device 40 has

radios

64 and 65 that perform wireless communication with the radio unit 52 and the radio 54.

The controller 61 of the remote control device 40 is connected to the operation unit 62 and the display unit (monitor) 63. The operation unit 62 is for the operator to remotely control the forklift 20, and the operation content of the forklift 20 by the operator is sent to the controller 61. The operation content includes operation command values for lift, reach, and tilt, and parameters such as speed, acceleration, and steering angle operation command values. The controller 61 transmits vehicle control signals such as lift, reach, and tilt operation command values, speed, acceleration, and steering angle operation command values to the radio unit 52 of the forklift-mounted device 50 via the radio 64. Send wirelessly.

In the forklift-mounted device 50, the controller 51, the wireless unit 52, and the image processing unit 53 are connected to each other so that they can communicate with each other, for example, CAN (Control Area Network) communication. The controller 51 can drive the traveling system actuator and the cargo handling system actuator according to the instruction from the remote control device 40. The traveling system actuator includes an actuator related to traveling such as a traveling motor 26 and a steering motor (not shown). The cargo handling actuator includes actuators related to cargo handling such as a lift cylinder, a reach cylinder, and a tilt cylinder (not shown).

The wireless unit 52 wirelessly transmits vehicle information such as the vehicle speed of the forklift 20 and abnormal information such as obstacle detection information to the controller 61 via the radio 64.
In FIG. 1, the controller 61 is configured to be able to remotely control the traveling of the forklift 20 and the cargo handling by the cargo handling device 28 via the radio 64, the radio unit 52, and the controller 51. That is, the forklift 20 can be remotely controlled by the operation unit 62 of the remote control device 40 instead of the operation unit (direction lever 34, cargo handling lever 35, handle 36, brake pedal 37, etc.) in FIG. There is.

Then, in the remote control device 40, when the operator performs a desired operation using the operation unit 62, the operation content is sent to the forklift 20 by the controller 61 via the radio 64. In the forklift 20, the wireless unit 52 receives the operation content from the remote control device 40, and the controller 51 drives the actuator unit to execute a desired operation.

As shown in FIGS. 2 and 4, a camera 71 is provided at the center of the lift bracket 31 in the forklift 20. The camera 71 is configured to image the front of the fork claw Nf as a cargo handling portion, and is attached so as to face forward and downward. The camera 71 captures an image of a region including the floor surface in front of the forklift 20 in the traveling direction while traveling. Further, a camera 72 is attached to the upper surface of the left reach leg 22b so as to face forward, and the camera 72 images the left front of the forklift 20. That is, the camera 72 is a camera for monitoring an obstacle in front of the reach leg 22b.

As shown in FIG. 1, in the forklift 20, the images captured by the

cameras

71 and 72 are sent by the controller 51 to the remote control device 40 via the image processing unit 53 and the radio 54. In the remote control device 40, the radio 65 receives the camera image from the forklift 20. Then, the controller 61 displays the camera images captured by the

cameras

71 and 72 on the display unit 63 provided in the remote control device 40. The display unit 63 is, for example, a desktop type display. The operator operates while looking at the camera image on the display unit 63.

The controller 61 obtains the position information of the shelf containing the luggage or the position information of the shelf on which the luggage is placed from the map information of the work management system by communicating with the controller 90 for the work management system. That is, the controller 61 acquires the unloading position or the unloading position from the map information as the target position. The unloading position is the position of the section where the load on the shelf is placed, and the unloading position is the position of the section where the load on the shelf is placed. Further, since the camera 71 is provided on the lift bracket 31 which is a movable portion movable with respect to the machine base 21, the controller 61 sets the lift position and the reach position as information for specifying the position of the camera 71. Get from.

8 (a), 8 (b), and 8 (c) are views of the load W1 mounted on the pallet Pa1 as viewed from a plane, a front surface, and a right side surface, respectively. The rack Ra1 is arranged on the floor, and the pallet Pa1 is placed on the shelf Sh4 of the rack Ra1. As shown in FIGS. 8 (a), 8 (b), and 8 (c), the controller 61 as the guide superimposing unit has the pallet Pa1 (load W1) as a cargo handling object with respect to the camera image on the display unit 63. ), That is, the floor surface 80 located below the target position is configured so that the guide line Lg1 as a guide indicating that the target position is above the target position can be superimposed and displayed. That is, a guide indicating the position in a state of being lowered straight on the floor surface 80 corresponding to the target position having a height can be superimposed on the camera image.

Specifically, the work management system holds as map information which shelf in which rack, that is, what luggage is on which shelf. Further, the work management system acquires the position of the forklift 20 by a detection unit (not shown). When the work management system issues an instruction to pick up a specific load, the controller 61 obtains the position (self-position) information of the forklift 20 and the position (target position) information of the load from the work management system. Further, when the camera 71 is located in the movable portion, information (lift position, reach position) for specifying the camera position is obtained from the vehicle. Then, the coordinates for superimposing the guide line Lg1 indicating the target position on the floor surface 80 in the camera image can be calculated and the guide line Lg1 can be superimposed on the camera image.

The controller 61 obtains the superimposed position of the guide line Lg1 in the camera image according to the position of the camera 71. Then, the guide line Lg1 is correctly superimposed on the floor surface 80 projected on the camera image according to the actual position of the movable camera 71 with respect to the machine base 21. That is, the controller 61 detects the coordinates on which the guide line Lg1 indicating that the target position is above the floor surface in the camera image is superimposed from the information such as the lift position and the reach position, and detects the guide line Lg1 on the camera image. Are superimposed.

As shown in FIGS. 8 (a), 8 (b), and 8 (c), the guide line Lg1 is the line of intersection between the vertical plane passing through the center of the loading position and the floor surface. Then, the controller 61 obtains the line of intersection between the vertical plane passing through the center of the section in which the luggage is placed and the floor surface (horizontal plane having a height of 0), and draws it on the screen as the guide line Lg1.

Next, the operation of this embodiment will be described.
Hereinafter, as the display content of the display unit 63, a camera image taken by the camera 71 among the

cameras

71 and 72 will be described.

As shown in FIG. 5, racks Ra1 and Ra2 are arranged on both sides of the aisle in a plan view of the workplace, and the aisle sandwiched between the left and right racks Ra1 and Ra2 extends straight.
As shown in FIG. 6, the rack Ra1 has a multi-stage structure, and has a first-stage shelf Sh1, a second-stage shelf Sh2, a third-stage shelf Sh3, and a fourth-stage shelf Sh4. The load W1 is placed on the pallet Pa1 on the fourth shelf Sh4, which is the higher shelf in the rack Ra1. This pallet Pa1 (load W1) is the cargo handling object at the target position. That is, the load W1 is the load to be transported.

When the vehicle is far from the pick-up position, as shown in FIG. 6, the pallet Pa1 and the load W1 at the pick-up position are displayed on the display screen on the display unit 63. In FIG. 6, a virtual line (dashed-dotted line) shows a line hanging directly below from the center on the front surface of the pallet Pa1, which is the center of the loading position.

The unloading position deviates from the camera image as the vehicle approaches the rack Ra1, and the pallet Pa1 and the load W1 become invisible. Here, when the vehicle approaches the rack Ra1 by a predetermined distance, for example, 5 m, the controller 61 displays the guide line Lg1 on the floor surface 80 located below the target position on the display unit 63 as shown in FIG. It is superimposed and displayed on the camera image.

The guide line Lg1 extends from the front end of the rack Ra1, which is the center of the loading position, in a direction orthogonal to the extending direction of the passage, that is, in the width direction. If the forklift 20 is swiveled on the guide line Lg1 in the direction along the guide line Lg1, the forklift 20 can be made to face the pallet Pa1, and if the heights of the

forks

30a and 30b are adjusted to the height of the pallet Pa1, the load is loaded. It will be possible to take it.

The operator runs the forklift 20 on the guide line Lg1 superimposed on the floor surface 80 while looking at the display screen shown in FIG. 7, and then turns the forklift 20 on the spot to face the pallet Pa1. After that, the operator moves the

forks

30a and 30b upward to adjust the heights of the

forks

30a and 30b, and then moves the

forks

30a and 30b forward to move the holes H1a and H1b of the pallet Pa1 (see FIG. 8B). ). When the insertion is completed, lift the pallet Pa1 and the load W1. Further, the vehicle is moved backward and the vehicle is moved to a desired place together with the pallet Pa1 and the load W1.

In this way, even if the target pallet Pa1 is out of the camera image and cannot be seen, the operator can approach the target pallet Pa1 by relying on the guide line Lg1 on the floor surface.
According to the above embodiment, the following effects can be obtained.

(1) An operation support device for a cargo handling vehicle provided with a fork claw Nf as a vertically movable cargo handling unit is imaged by a camera 71 configured to image the front of the fork claw Nf as a cargo handling unit and a camera 71. It is provided with a display unit 63 configured to display the camera image. Further, the operation support device acquires the loading position as the target position, and displays the guide line Lg1 as a guide superimposed on the floor surface 80 located below the target position with respect to the camera image on the display unit 63. The controller 61 as a guide superimposing unit configured in the above is provided.

According to this, when performing the unloading work on the rack Ra1 (especially the high shelf Sh4), the unloading position deviates from the camera image as the vehicle approaches the rack Ra1, and the target unloading position cannot be seen. The controller 61 as a guide display superimposing unit displays the guide line Lg1 superimposed on the floor surface located below the target position with respect to the camera image on the display unit 63. Therefore, the pallet Pa1 as an object can be approached according to the target. In this way, the target position can be guided when the vehicle approaches the target position.

That is, even if the target position (position of the pallet Pa1) is high and cannot be seen in the camera image, the operator approaches the object by relying on the target position shown on the floor surface by indicating the position with the guide line on the floor surface. be able to.

(2) The guide line Lg1 as a guide is an intersection line between the vertical plane passing through the center of the loading position and the floor surface, and is easy for the operator to understand.
(3) The position of the pallet Pa1 (load W1) as the target position is obtained from the map information, and is therefore practical.

(4) The camera 71 is movable with respect to the machine base 21, and the controller 61 as a guide superimposing unit obtains the superimposition position of the guide line Lg1 as a guide in the camera image according to the position of the camera 71. Therefore, even if the camera 71 moves, the guide display can be performed.

(5) The operation support device for the cargo handling vehicle is used in the forklift remote control system 10 as the remote control system for the cargo handling vehicle. The remote control system 10 for a forklift includes a forklift 20 as a cargo handling vehicle and a remote control device 40. The forklift 20 includes a cargo handling device 28 on the machine base 21, and also has a radio unit 52 and a radio 54 as vehicle communication units. Further, the remote control device 40 has

radios

64 and 65 as operation device communication units that perform wireless communication with the radio unit 52 and the radio 54, and remotely controls the traveling of the forklift 20 and the cargo handling by the cargo handling device 28. Used for. Therefore, when the vehicle is remotely controlled only according to the image of the camera, the target position can be guided when the vehicle approaches the target position.

The embodiment is not limited to the above, and may be embodied as follows, for example.
○ The target position may be acquired from the camera image by image recognition. That is, the target position may be information obtained from recognition of the target position (pallet recognition, etc.) and measurement. More specifically, when a plurality of palettes are recognized, a target is specified from among them. Image recognition may be performed by the controller 61 or the image processing unit 53 in FIG.

When the target position is obtained from the image recognition of the palette, the line of intersection between the vertical plane passing through the center and the floor surface may be obtained from the coordinates of the recognized palette and drawn as a guide line.
Further, the guide line Lg1 as a guide was the center of the loading position, but is not limited to this, and for example, the guides shown in FIGS. 9 (a), 9 (b), and 9 (c). Lines Lg10, Lg11, Lg12, Lg13 may be used.

In FIGS. 9 (a), 9 (b), and 9 (c), the guide lines Lg10, Lg11, Lg12, and Lg13 as guides correspond to the left and right ends of the holes (pallet holes H1a, H1b) of the pallet Pa1. It is something to do. Specifically, the guide line Lg10 corresponds to the right end of the pallet hole H1a, the guide line Lg11 corresponds to the left end of the pallet hole H1a, the guide line Lg12 corresponds to the right end of the pallet hole H1b, and the guide line Lg13 corresponds to the pallet hole H1b. Corresponds to the left end. Then, the guide lines Lg10, Lg11, Lg12, and Lg13 are superimposed and displayed on the camera image displayed on the display unit 63 on the floor surface 80 located below the target position.

The centers of the pallet holes H1a and H1b may be used as guides in FIGS. 9 (a), 9 (b), and 9 (c).
The unloading position was the position of the section on which the load was placed on the shelf, but when using image recognition, the unloading position is the position of the pallet or the load.

○ The target position may be acquired according to the instruction of the operator (worker). That is, at the stage where the display of the floor surface 80 is visible on the display unit 63, the floor surface on the camera image may be marked by the operator using the input unit 100 (indicated by a virtual line in FIG. 1).

The pick-up position was the position of the section where the load on the shelf was placed, but if specified by the operator, the pick-up position is the position of the pallet or the load.
○ The installation location of cameras may be changed, and the number of cameras may also be changed. ○ The case of unloading work has been explained, but the same applies to the case of unloading work. The target position is the position where the load is placed (loading position).

The controller 61 as the guide superimposing unit may acquire the loading position or the loading position as the target position.
When performing the loading operation, it is easy to understand that the guide is the line of intersection between the vertical plane passing through the center of the loading position and the floor surface.

In FIG. 4, the camera 71 is provided at the center of the lift bracket 31 in consideration of loading, but when the loading work is performed, if it becomes difficult to image the front of the vehicle, the lift bracket 31 is located in the left-right direction. A camera may be provided at the end of the.

○ The superposition of guide lines was performed when the distance between the vehicle and the target reached a predetermined value, but it is not limited to this. For example, the superposition of the guide lines may always be performed regardless of the distance between the vehicle and the target.
○ The guide does not have to be a straight line. The guide may be, for example, a figure (circle, triangle, etc.).

○ The operation support device for cargo handling vehicles was used for remote control systems for forklifts, but it is not limited to this. For example, it may be used for a manned forklift. That is, instead of having an unmanned forklift equipped with a camera and a remote control device having a display unit, the application may be applied to, for example, a manned forklift equipped with a camera and a display unit.

○ The forklift may be a counter type forklift.
○ The cargo handling vehicle was a forklift, but it may be applied to a cargo handling vehicle other than a forklift that has a vertically movable cargo handling unit.

○

Controllers

51, 61, 90 are not limited to those equipped with a central processing unit and a memory to execute software processing. For example, a dedicated hardware circuit (for example, ASIC or the like) that performs hardware processing on at least a part of what has been software-processed in each of the above embodiments may be provided. That is, the

controllers

51, 61, and 90 may have any of the following configurations (a) to (c). (A) A processing device that executes all of the above processing according to a program and a program storage device such as a ROM that stores the program are provided. (B) A processing device and a program storage device that execute a part of the above processing according to a program, and a dedicated hardware circuit that executes the remaining processing are provided. (C) A dedicated hardware circuit for executing all of the above processes is provided. Here, there may be a plurality of software processing circuits including a processing device and a program storage device, and a plurality of dedicated hardware circuits. That is, the processing may be executed by a processing circuit including at least one of one or more software processing circuits and one or more dedicated hardware circuits.

Claims

It is an operation support device for cargo handling vehicles equipped with a cargo handling unit that can move up and down.
A camera configured to image the front of the cargo handling section,
A display unit configured to display a camera image captured by the camera, and a display unit.
It is configured to acquire the loading position or the loading position as the target position, and the display unit is configured to display the guide superimposed on the floor surface located below the target position with respect to the camera image. Guide superimposition part and
An operation support device for cargo handling vehicles equipped with.
The operation support device for a cargo handling vehicle according to claim 1, wherein the guide is a line of intersection between a vertical plane passing through the center of the loading position or the center of the loading position and a floor surface.
The target position is the operation support device for the cargo handling vehicle according to claim 1 or 2, which is obtained from the map information.
The operation support device for a cargo handling vehicle according to claim 1 or 2, wherein the target position is acquired from the camera image by image recognition.
The target position is the operation support device for the cargo handling vehicle according to claim 1 or 2, which is acquired by the instruction of the operator.
The camera is movable with respect to the chassis of the cargo handling vehicle.
The operation support device for a cargo handling vehicle according to any one of claims 1 to 5, wherein the guide superimposing unit is configured to obtain a superimposing position of the guide in the camera image according to the position of the camera.
The operation support device for a cargo handling vehicle is used in a remote control system for a cargo handling vehicle.
The remote control system for a cargo handling vehicle includes the cargo handling vehicle and a remote control device.
The cargo handling vehicle has a machine base, a cargo handling device provided on the machine base, and a vehicle communication unit.
The remote control device has an operation device communication unit that wirelessly communicates with the vehicle communication unit, and is used for remotely controlling the traveling of the cargo handling vehicle and the cargo handling by the cargo handling device. The operation support device for a cargo handling vehicle according to item 1.