JP6217472B2 - Forklift work support device - Google Patents

Description

  The present invention relates to a forklift work support device that supports a load handling work of a forklift.

As a conventional technique related to a forklift work support device, for example, an industrial vehicle visibility improving system disclosed in Patent Document 1 is known.
An industrial vehicle visibility improving system disclosed in Patent Document 1 is a head-mounted display that is mounted on an industrial vehicle operator and transmits an external scene and projects an image, and a camera that is fixed in front of a shield and captures a forward scene. And a camera that is fixed to the head-mounted display and captures a front view, and an image processing device.
The image processing apparatus includes: an image deforming unit that deforms a captured image of the camera based on the captured image of the camera; an image cutting unit that cuts a range corresponding to a shield from the deformed image; and a head-mounted display that displays the cut partial image And an image output unit for outputting to the output.

According to the industrial vehicle visual field improvement system disclosed in Patent Document 1, the image behind the shielding object is deformed by image processing, and a range corresponding to the shielding object is cut out from the deformed image to form a transmission type head mounted display. Is displayed.
As a result, the image behind the shielding object can be projected over the external scenery, and the operator can obtain a clear view regardless of the shielding object.

JP 2008-143701 A

However, the industrial vehicle visual field improvement system disclosed in Patent Document 1 requires a camera that is fixed in front of the shield and images the front scene and a camera that is fixed to the head mounted display and images the front scene. is there.
Further, the industrial vehicle visibility improving system disclosed in Patent Document 1 is not a technique for displaying a fork.
Furthermore, in the industrial vehicle visual field improvement system disclosed in Patent Document 1, the load is placed on the fork, and the visual field is not improved in a state where the fork is raised and the load shields the front of the camera.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide a forklift work capable of minimizing the number of cameras and always displaying the position of the fork and providing good work support. To provide support equipment.

In order to solve the above-mentioned problems, the present invention is a forklift work support device for supporting a load handling work of a forklift, wherein the body of the forklift includes a head guard at an upper part of a driver's seat, and is provided at a front part of the head guard. A camera provided in a vertically swingable manner for imaging the front of the vehicle body including the tip of the fork, a lift detection means for detecting the lift of the fork, and the camera in conjunction with the raising and lowering of the fork A camera swinging means for swinging the camera up and down, an image display means for displaying a camera image captured by the camera, and a position of the fork on the image display means based on a detection value of the elevation detection means. An image control unit that is set as a displayable virtual fork and displays the image on the image display unit by superimposing the virtual fork on a camera image captured by the camera , Characterized by comprising a.

In the present invention, the camera swings according to the lift of the fork to capture an image in front of the vehicle body.
The image control means sets a virtual fork that can be displayed on the image display means based on the detection value of the elevation detection means, and displays the virtual fork superimposed on the camera image and displayed on the image display means.
Therefore, the virtual fork can always be displayed on the image display means regardless of whether or not there is a load on the fork, and the fork lift operator can recognize the position of the tip of the fork.

Further, since no obstacle is present in front of the camera, the camera can be captured forward without being blocked by the shield.

Further, in the forklift work support device, the lift detection means is connected to the wire coupled to the fork, the wire reel that winds the wire in accordance with the lift of the fork, and the image control means. A rotation detector that detects the rotation of the wire reel may be provided.
In this case, the lift of the fork can be detected by the wire, the wire reel, and the rotation detector.

In the forklift work support device described above, a tilt angle sensor that detects a tilt angle of the mast device including the fork is connected to the image control unit, and the image control unit is responsive to a detection angle of the tilt angle sensor. The virtual fork may be corrected and displayed.
In this case, since the image control unit corrects and displays the virtual fork according to the detection angle of the tilt angle sensor, the virtual fork can be displayed more accurately in response to the forward or backward tilt of the mast device.

In the forklift work support device, the image display unit may be a head-mounted display that is attached to the forklift operator and transmits an external scenery and can project an image.
In this case, since the virtual fork is displayed on the head mounted display, the operator can recognize the position of the fork more intuitively.

  ADVANTAGE OF THE INVENTION According to this invention, while being able to make the minimum number of cameras, the forklift work assistance apparatus which always displays the position of a fork and enables favorable work assistance can be provided.

It is a side view which shows the forklift which concerns on this invention. 1 is a schematic configuration diagram of a forklift work support device according to the present invention. It is explanatory drawing explaining superimposition of the virtual fork with respect to a camera image. It is a figure which shows the virtual fork image correct | amended by the tilt angle. It is a figure explaining the relationship between the lift of a fork and the camera image on which the virtual fork was superimposed.

Hereinafter, a forklift work support device according to the present invention will be described with reference to the drawings.
Note that “front / rear”, “left / right”, and “up / down” that specify the direction are based on a state in which the forklift operator is seated on the driving seat of the driver's seat and faces the forward side of the forklift.

As shown in FIG. 1, the forklift 10 includes a cargo handling device 12 at the front of a vehicle body 11.
A driver's seat 13 is provided near the center of the vehicle body 11.
A driving wheel 14 as a front wheel is provided at the front portion of the vehicle body 11, and a steering wheel 15 as a rear wheel is provided at the rear portion of the vehicle body 11.
The forklift 10 according to this embodiment is an engine forklift that travels by the driving force of an internal combustion engine (not shown) mounted on the vehicle body 11.
The vehicle body 11 is provided with a head guard 16 that covers the upper portion of the driver's seat 13.

The cargo handling device 12 includes a mast 19 having an outer mast 17 and an inner mast 18.
The pair of left and right outer masts 17 is provided with an inner mast 18 that can slide inside the outer mast 17.
Between the vehicle body 11 and the outer mast 17, a tilt cylinder 20 that is operated by hydraulic pressure is installed.
The mast 19 tilts in the front-rear direction with the lower end portion as a fulcrum by the operation of the tilt cylinder 20.
The mast 19 is provided with a lift cylinder 21 that is operated by hydraulic pressure, and the operation of the lift cylinder 21 causes the inner mast 18 to slide up and down within the outer mast 17.
The mast 19 is provided with a pair of left and right forks 22 via a lift bracket 23, and the lift bracket 23 is provided so as to move up and down with respect to the inner mast 18.
The vehicle body 11 is provided with a cargo handling pump (not shown) for supplying hydraulic oil to the lift cylinder 21 and the tilt cylinder 20 and a cargo handling pump, and the cargo handling pump is driven by an internal combustion engine.

The cargo handling device 12 is provided with a lift sensor 24 that continuously detects the lift of the fork 22.
The lift sensor 24 corresponds to a lift detection means and is used for continuously detecting the lift of the fork 22. In this embodiment, a reel type lift sensor is used.
The reel-type lift sensor 24 includes a wire (not shown) having one end connected to the fork 22, a reel (not shown) around which the wire is wound, and rotation detection for detecting the amount of rotation of the reel. (Rotary encoder).
In addition, the cargo handling device 12 is provided with a tilt angle sensor 25 that detects the tilt angle of the mast 19.
The tilt angle sensor 25 detects a tilt angle when the mast 19 is tilted forward or backward.

A driver seat 13 in the vehicle body 11 is provided with a driver seat 26 on which an operator can sit.
The driving seat 26 is disposed on an engine hood 27 provided on the vehicle body 11.
An accelerator pedal (not shown) is provided on the floor of the driver seat 13, and the accelerator pedal is for adjusting the vehicle speed of the forklift 10.
The forklift 10 controls the drive of the internal combustion engine so that the vehicle speed is in accordance with the amount of depression of the accelerator pedal by the operator.

A steering wheel 28 for steering is provided in front of the driving seat 26.
A forward / reverse lever (not shown) is provided on the left side of the steering wheel 28, and “forward” or “reverse” of the forklift 10 is selected by operating the forward / backward lever.
On the right side of the steering wheel 28, a lift lever (not shown) for operating the lift cylinder and a tilt lever (not shown) for operating the tilt cylinder are installed.

By the way, the forklift 10 of the present embodiment is provided with a forklift work support device (hereinafter referred to as “work support device”) for supporting the cargo handling work.
The work support device is a device that always displays the position of the fork 22 as a virtual fork regardless of whether or not there is a load placed on the fork 22.
As shown in FIG. 2, the work support apparatus according to the present embodiment includes a camera 30 provided on the head guard 16, an electric motor 31 that swings the camera 30, a controller 32 that performs image processing, a head mounted display 33, and a memory 34. Have
In addition, the work support apparatus includes a lift height sensor 24 and a tilt angle sensor 25.

The camera 30 is for capturing images of the front of the vehicle body 11 including the tip of the fork 22 continuously, and is an electronic camera composed of a CCD image sensor or a CMOS image sensor.
The camera 30 of this embodiment is provided in the head guard 16 so as to be swingable in the vertical direction.
Specifically, as shown in FIG. 1, a support member 35 is provided in the vicinity of the left and right centers in the front portion of the head guard 16, and the camera 30 swings via a fulcrum shaft 36 provided on the support member 35. It is supported freely.

The support member 35 provided on the head guard 16 is provided with an electric motor 31 that controls the swing of the camera 30.
Accordingly, the electric motor 31 is swung in the vertical direction by the rotation of the fulcrum shaft 36 by the electric motor 31.
The swing range of the camera 30 of the present embodiment is set so as to be able to capture an image including the tip of the fork 22 corresponding to the lift height of the fork 22.
That is, in the swing range of the camera 30, the imaging direction of the camera 30 is set in a range from a sufficient depression angle to an elevation angle so that the tip of the fork 22 can be imaged.
The installation position of the camera 30 in the head guard 16 is a position where there is no obstruction ahead in the vehicle body 11.

The vehicle body 11 is provided with a controller 32 as image control means. The controller 32 executes arithmetic processing and various programs, and also controls image processing of camera images and devices provided in the vehicle body 11.
The arithmetic processing and program by the controller 32 include arithmetic processing and programs related to image processing such as camera images.
A memory 34 is electrically connected to the controller 32.
A large number of virtual fork images corresponding to the height of the fork 22 are stored in the memory 34 in advance.
The image processing of the camera image by the controller 32 and the virtual fork image will be described later.

The controller 32 is electrically connected to the elevation sensor 24 and the tilt angle sensor 25.
The lift sensor 24 continuously detects the lift of the fork 22 during operation of the forklift 10 and transmits a detection signal of the lift of the fork 22 to the controller 32.
The tilt angle sensor 25 detects the tilt angle of the mast 19 and transmits a detection signal to the controller 32.
The camera 30 is electrically connected to the controller 32, and the controller 32 receives data of a camera image captured by the camera 30.
The electric motor 31 is electrically connected to the controller 32, and the electric motor 31 is controlled based on a command from the controller 32.
In the present embodiment, the controller 32 controls the electric motor 31 so as to interlock with the raising and lowering of the fork 22.
The controller 32 receives the detection signal of the lift of the fork 22 from the lift sensor 24 and controls the electric motor 31 according to the lift of the fork 22.

Next, the head mounted display 33 will be described.
The head mounted display 33 corresponds to an image display means for displaying an image, and is mounted on the head of the operator of the forklift 10 in the present embodiment.
The controller 32 is communicably connected to the head mounted display 33.
The head mounted display 33 according to the present embodiment includes a mounting unit 37 that is mounted on the operator's head and a display unit 38 that can transmit an external scenery and project an image.
Therefore, the operator wearing the head-mounted display 33 secures a visual field that can be viewed from the driver's seat 13 through the display unit 38 by transmitting the external scenery, and also receives various information by projecting an image or the like transmitted from the controller 32. Be recognized.
Incidentally, the head mounted display 33 of the present embodiment has a function of displaying an image C for work support described later in a partial area of the display unit 38.
The head mounted display 33 is provided with a switch (not shown) for switching display / non-display of information from the controller 32.
For this reason, the information from the controller 32 can always be displayed on the head mounted display 33, but the information from the controller 32 can also be hidden by switching the switch.

Next, the image processing of the camera image captured by the camera 30 by the controller 32 and the virtual fork image will be described.
As shown in FIG. 3, the controller 32 superimposes a virtual fork image B indicating the position of the fork 22 corresponding to the camera image A as a virtual fork F on the camera image A, and displays the virtual fork image B on the camera image A. Image processing for obtaining a superimposed composite image C is performed.
The virtual fork image B of the present embodiment is an image in which the position of the fork 22 seen from the installation position of the camera 30 when the fork 22 has no load W is drawn with a dotted line as the virtual fork F.
A large number of virtual fork images B drawn corresponding to the height of the fork 22 are prepared.
The composite image C is an image provided to the operator of the forklift 10 for work support. In the composite image C, a color that makes the dotted line indicating the virtual fork F stand out or a sufficient color so that the position of the virtual fork F can be easily grasped. The line thickness is set.

In the present embodiment, as shown in FIG. 4, a large number of virtual fork images B drawn by correcting the position of the virtual fork F according to the tilt angle are prepared.
A virtual fork F indicated by a dotted line in FIG. 4 indicates the position of the fork 22 when the tilt angle is 0 °.
In FIG. 4, the virtual fork Fa indicated by a one-dot chain line indicates the position of the fork 22 when the tilt angle is 5 ° (forward tilt), and the virtual fork Fb indicated by another one-dot chain line has a tilt angle of 10 ° (backward tilt). ) Shows the position of the fork 22.
That is, the virtual forks Fa and Fb are virtual forks drawn with correction according to the tilt angle.

A large number of virtual fork images B are stored in the memory 34 in advance.
The controller 32 transmits image data so that the composite image C can always be displayed on the head mounted display 33.
When there is no load W on the fork 22 in the composite image C, the virtual fork F drawn by the dotted line is displayed in agreement with the actual fork 22 shown in the camera image.
As shown in FIG. 3, when there is a load W on the fork 22 in the composite image C, the controller 32 displays the actual fork 22 not displayed in the camera image A as a virtual fork F drawn by a dotted line.
The load W may include not only the load itself but also the load itself and a pallet on which the load is placed.

Next, the operation of the work support apparatus of this embodiment will be described.
The operator attaches the head mounted display 33 to the head when the forklift 10 is operated.
The lift height sensor 24 continuously detects the lift height of the fork 22 during operation of the forklift 10 and transmits a signal of a detected value of the lift height of the fork 22 to the controller 32.
The controller 32 receives the detection value signal from the elevation sensor 24 and controls the electric motor 31 to swing the camera 30 so that the tip of the fork 22 falls within the imaging range of the camera 30.
The camera 30 that is swung as the fork 22 moves up and down continuously captures a camera image (still image) A according to the height of the fork 22, and the captured camera image A is transmitted to the controller 32.
The controller 32 superimposes the virtual fork image B in which the virtual fork F is set based on the detection value from the elevation sensor 24 so as to correspond to each continuously captured camera image A.

As shown in FIG. 5, when the lift of the fork 22 supporting the load W is low, for example, a composite image C (L) is obtained, and a virtual fork F is drawn in the composite image C (L). The fork 22 is hidden by the load W, the outer mast 17 and the like and is not shown.
On the other hand, when the lift height of the fork 22 is high, for example, a composite image C (H) is obtained. In the composite image C (H), in addition to the virtual fork F, the actual fork 22 includes one end portion. The virtual fork F is displayed so as to overlap the actual fork 22.
Further, when the lifting height of the fork 22 supporting the load W indicates an intermediate height, for example, a composite image C (M) is obtained, and a virtual fork F is drawn in the composite image C (M), The actual fork 22 is hidden by the load W, the outer mast 17 and the like and is not shown.

Then, the composite image C obtained by superimposing the virtual fork image B corresponding to the camera image A is displayed on the head mounted display 33.
In the present embodiment, the composite image C is displayed in a partial area of the display unit 38 of the head mounted display 33 (see FIG. 3).
When the tilt angle sensor 25 detects a tilt angle indicating the forward or backward tilt of the mast 19, a virtual fork image B on which a virtual fork Fa (Fb) corrected in accordance with the tilt angle is drawn. Is superimposed on the camera image A (see FIG. 4).

When carrying out the cargo handling work with the forklift 10, it is important for the operator to recognize where the tip of the fork 22 is located.
When the load is placed on the fork 22, the tip of the fork 22 may not be recognized at all regardless of the height of the fork 22.
In the work support device according to the present embodiment, the controller 32 displays the composite image C on the head mounted display 33 even when a load is placed on the fork 22, so that the position of the fork 22 is recognized by the display of the virtual fork F. Is done.
When the load is not placed on the fork 22, the position of the tip of the fork 22 can be recognized by the operator's visual observation, and can also be recognized by the composite image C displayed on the head mounted display 33.
In this case, the composite image C displayed on the head mounted display 33 is displayed in a state where the virtual fork F overlaps the actual fork 22.

The work support apparatus of the present embodiment has the following operational effects.
(1) The camera 30 swings according to the height of the fork 22 and images the front. The controller 32 sets a virtual fork F that can be displayed on the head mounted display 33 based on the detection by the height sensor 24, and displays a composite image C in which the virtual fork image B is superimposed on the camera image A on the head mounted display 33. . Therefore, the virtual fork F can always be displayed on the head mounted display 33 regardless of the presence or absence of the load W on the fork 22, and the operator of the forklift 10 can recognize the position of the tip of the fork 22. In particular, even when the fork 22 is moving up and down, the virtual fork F is always displayed in the forward image taken corresponding to the lifting and lowering, so that the position of the tip of the fork 22 can always be correctly recognized.

(2) Since the camera 30 is provided in the front part of the head guard 16, there is no shielding object in front of the camera 30, and the camera 30 can image the front of the vehicle body 11 without being blocked by the shielding object. Is possible.

(3) The lift sensor 24 is connected to the wire coupled to the fork 22, a wire reel that winds the wire in accordance with the lift of the fork 22, and a controller 32, and a rotation detector that detects the rotation of the wire reel. Therefore, the height of the fork 22 can be detected by the wire, the wire reel, and the rotation detector.

(4) The controller 32 displays a virtual fork Fa (Fb) obtained by correcting the virtual fork F according to the detection angle of the tilt angle sensor 25. For this reason, the virtual fork Fa (Fb) can be displayed more accurately in response to the forward or backward inclination of the mast 19.

(5) The head mounted display 33 is attached to the operator of the forklift 10 and can project information transmitted from the controller 32 while transmitting the external scenery. For this reason, a virtual fork can be displayed on the head mounted display 33, and the operator can recognize the position of the fork 22 more intuitively.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the invention. For example, the following modifications may be made.

In the above embodiment, the head mounted display is used as the image display means, but the image display means is not limited to the head mounted display. The image display means is not particularly limited as long as it can display an image like a liquid crystal monitor provided in a driver's seat and can recognize an image by an operator.
In the above embodiment, the reel type lift detection sensor is used as the lift detection unit. However, the lift detection unit is not limited to the reel type lift detection sensor, and has a configuration for detecting the lift of the fork. For example, a lift detection means for optically detecting the lift of the fork using a reflector and an optical sensor may be used.
○ In the above embodiment, using an electric motor as a camera swinging means, the camera moving means may be any device capable of swinging the camera in the vertical direction is not limited to the electric motor.
In the above embodiment, the virtual fork image corresponding to the camera image is prepared in advance. However, for example, the position of the virtual fork may be calculated based on the detection signal of the fork height detection means. In this case, a virtual fork image in which a virtual fork corresponding to the obtained virtual fork position is drawn may be created by the image control means, and the virtual fork image may be superimposed on the created camera image by the image control means. By generating the virtual fork image by obtaining the position of the virtual fork each time a camera image is obtained, it is not necessary to prepare a large number of virtual fork images in advance.

DESCRIPTION OF SYMBOLS 10 Forklift 11 Car body 12 Handling apparatus 13 Driver's seat 16 Head guard 17 Outer mast 18 Inner mast 19 Mast 20 Tilt cylinder 21 Lift cylinder 22 Fork 23 Lift bracket 24 Lift sensor 25 Tilt angle sensor 30 Camera 31 Electric motor 32 Controller 33 Head mount display 34 Memory 35 Support member 36 Support point shaft 37 Mounting portion 38 Display portion A Camera image B Virtual fork image C, C (L), C (M), C (H) Composite image F, Fa, Fb Virtual fork

Claims (4)

A forklift work support device for supporting forklift handling work,
The body of the forklift has a head guard at the top of the driver's seat,
A camera that is provided at the front portion of the head guard so as to be swingable in the vertical direction, and that images the front of the vehicle body including the tip of the fork;
A lift detection means for detecting the lift of the fork;
Camera swinging means for swinging the camera up and down in conjunction with raising and lowering of the fork;
Image display means for displaying a camera image captured by the camera;
The position of the fork is set as a virtual fork that can be displayed on the image display means based on the detection value of the height detection means, and the virtual fork is superimposed on the camera image captured by the camera on the image display means. A forklift work support device comprising: an image control means for displaying. The elevation detection means includes
A wire connected to the fork;
A wire reel that winds the wire according to the height of the fork;
The forklift work support device according to claim 1 , further comprising a rotation detector connected to the image control unit and detecting rotation of the wire reel . A tilt angle sensor for detecting a tilt angle of a mast device including the fork is connected to the image control means;
The forklift work support device according to claim 1 , wherein the image control unit corrects and displays a virtual fork according to a detection angle of the tilt angle sensor . The forklift work support device according to any one of claims 1 to 3, wherein the image display means is a head-mounted display that is attached to an operator of the forklift and transmits an external scene and is capable of projecting an image .

Images