JP2017019649A - Display system of industrial vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display system of an industrial vehicle which can complement a front dead angle of an imaging part when a cargo handling device is operated without increasing the number of imaging parts.SOLUTION: A display system 13 of an industrial vehicle includes: a camera 14 for imaging a front of a traveling device; a memory 17 which stores an image imaged and acquired by the camera 14 at a start of operation of a cargo handling device as a recorded image; an image processing part 21 which synthesizes the latest image imaged and acquired by the camera 14 and the recorded image stored in the memory 17 and generates a synthesized image; and a display 16 which displays the synthesized image generated by the image processing part 21. The image processing part 21 synthesizes by transmitting the one of the recorded image and the latest image to the other of the recorded image and the latest image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an industrial vehicle display system.

  As a conventional industrial vehicle display system, for example, a technique described in Patent Document 1 is known. The display system for an industrial vehicle described in Patent Document 1 is provided in front of the mast, is fixed to a first camera that captures a front scene from the mast, and a head-mounted display worn by an operator, and is substantially the same as the visual field of the operator. A second camera that captures the field of view; and an image processing device that performs image processing of an image captured by the first camera and the second camera. The image processing apparatus deforms the captured image obtained by the first camera based on the captured image obtained by the second camera, cuts the deformed image only in a range corresponding to the mast, and cuts the cut partial image. Output to the head mounted display.

JP 2008-143701 A

  However, the following problems exist in the prior art. In other words, when a load is loaded on a claw (fork) that is a part of the cargo handling device, depending on the height of the load, the camera (imaging unit) may be hidden behind the load and the front scene cannot be imaged from the mast. There is. In addition, since the camera is provided in front of the mast, when the fork is moved up and down, a front blind spot of the camera that cannot be complemented occurs. Therefore, even when the fork is moved up and down, it is necessary to increase the number of cameras in order to supplement the front blind spot of the camera.

  An object of the present invention is to provide an industrial vehicle display system capable of complementing the front blind spot of an imaging unit when operating a cargo handling device without increasing the number of imaging units.

  According to one aspect of the present invention, in an industrial vehicle display system having a traveling device and a cargo handling device disposed on the front side of the traveling device, an imaging unit that images the front of the traveling device and an imaging unit at the start of operation of the cargo handling device. Image processing for generating a composite video by synthesizing a storage unit that stores an image captured and acquired as a recorded image, and a latest video captured and acquired by the imaging unit and a recorded image stored in the storage unit And a display unit for displaying the synthesized video generated by the image processing unit, and the image processing unit synthesizes the recorded image and the latest video through the other of the recorded image and the latest video. It is characterized by.

  In such an industrial vehicle display system, when the operation of the cargo handling device is started, an image acquired by imaging the front of the traveling device by the imaging unit is stored as a recorded image in the storage unit. Then, a synthesized video is generated by synthesizing the latest video acquired by imaging the front of the traveling device by the imaging unit and the recorded image stored in the storage unit, and the synthesized video is displayed on the display unit. Here, when the synthesized video is generated, the other of the recorded image and the latest video is transmitted through one of the recorded image and the latest video and synthesized. For this reason, even if the cargo handling apparatus is operated, both the recorded image and the latest video are clearly displayed as a whole on the display unit. Thereby, even if it does not increase the number of image pick-up parts, when a cargo handling apparatus is operated, the front blind spot of an image pick-up part can be supplemented.

  The image processing unit may combine the latest image with the recorded image. In this case, since the recorded image close to the background is clearer than the latest video, the synthesized video displayed on the display unit is easy to see.

  The storage unit may store, as a recorded image, an image captured and acquired by the imaging unit when an operation of the operation unit for operating the cargo handling device is started. In this case, when the operation of the operation unit for operating the cargo handling device is started, the imaging unit automatically images the front of the traveling device. Therefore, the operator does not have to instruct the imaging timing by the imaging unit with a manual operation switch or the like, and the burden on the operator is reduced.

  A travel detection unit that detects whether or not the travel device has traveled more than a predetermined distance; and an image eraser that erases a recorded image from the storage unit when the travel detection unit detects that the travel device has traveled more than a predetermined distance; May be further provided. When the traveling device travels, the recorded image stored in the storage unit is changed from the actual image of the imaging unit, so that the recorded image becomes unnecessary. Therefore, when it is detected that the traveling device has traveled more than a predetermined distance, the process of combining the latest video and the recorded image becomes unnecessary by deleting the recorded image from the storage unit, thus simplifying the process. Can be achieved.

  A state detection unit that detects a traveling state of the traveling device, and an image correction unit that corrects the recorded image stored in the storage unit based on the traveling state of the traveling device detected by the state detection unit may be further provided. . When the traveling state of the traveling device changes, the recorded image stored in the storage unit is changed from the actual image of the imaging unit, and thus the recorded image becomes unnecessary. Therefore, by correcting the recorded image stored in the storage unit based on the traveling state of the traveling device, an appropriate recorded image corresponding to the latest traveling state of the traveling device can be obtained.

  ADVANTAGE OF THE INVENTION According to this invention, even if it does not increase the number of image pick-up parts, when operating a cargo handling apparatus, the display system of the industrial vehicle which can complement the front blind spot of an image pick-up part is provided.

It is a side view showing a forklift as an industrial vehicle to which a display system concerning one embodiment of the present invention is applied. 1 is a schematic configuration diagram illustrating a display system according to an embodiment of the present invention. It is a flowchart which shows the detail of the display process procedure performed by the controller shown by FIG. FIG. 3 is a diagram showing an example of a recorded image stored in the memory shown in FIG. 2 and an example of a synthesized image generated by synthesizing the recorded image and the latest video of the camera. It is a schematic block diagram which shows the display system which concerns on other embodiment of this invention. It is a flowchart which shows the detail of the display processing procedure performed by the controller shown by FIG. It is a figure which shows the modification of the synthesized image shown by FIG.4 (b).

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a side view showing a forklift as an industrial vehicle to which a display system according to an embodiment of the present invention is applied. In the figure, the forklift 1 includes a traveling device 2 having four wheels 2 a and a cargo handling device 3 disposed on the front side of the traveling device 2.

  The cargo handling device 3 has a mast 4 attached to the front portion of the traveling device 2. A lift bracket 5 is supported on the mast 4 so as to be movable up and down. A fork 6 for loading a load is attached to the lift bracket 5. In addition, the cargo handling device 3 includes a lift cylinder 7 that raises and lowers the fork 6 and a tilt cylinder 8 that tilts the mast 4.

  The traveling device 2 has a cab 9. In the cab 9, a lift operation lever 10 that is an operation unit for operating the lift cylinder 7, a tilt operation lever 11 that is an operation unit for operating the tilt cylinder 8, and a steering 12 for steering. And are arranged.

  FIG. 2 is a schematic configuration diagram showing a display system according to an embodiment of the present invention. The display system 13 of this embodiment is provided in the traveling device 2. The display system 13 includes a camera (imaging unit) 14, a rotation sensor 15, a display (display unit) 16, a memory (storage unit) 17, and a controller 18.

  The camera 14 is attached to the upper surface of the head guard 9a that defines a part of the cab 9 (see FIG. 1), and images the front of the traveling device 2. The number of cameras 14 is one. Here, the imaging range of the camera 14 is above the height position of the head guard 9 a in front of the traveling device 2.

  The rotation sensor 15 is a sensor that measures the number of rotations of the wheel 2a. The indicator 16 is disposed at the front upper part in the cab 9 (see FIG. 1) and displays the situation in front of the traveling device 2. The memory 17 stores, as a recorded image, an image captured and acquired by the camera 14 at the start of the operation of raising the fork 6 of the cargo handling device 3.

  The controller 18 includes an imaging control unit 19, an image storage unit 20, an image processing unit 21, a display control unit 22, a travel detection unit 23, and an image erasing unit 24.

  When an operation for raising the fork 6 (lift raising operation) is started by the lift operation lever 10, the imaging control unit 19 uses the camera 14 to drive the traveling device 2 immediately before driving the lift cylinder 7 to raise the fork 6. The camera 14 is controlled so as to image the front of the camera.

  The image storage unit 20 stores, in the memory 17, an image acquired by being captured by the camera 14 when the lift raising operation is started as a recorded image. The recorded image is a still image.

  The image processing unit 21 synthesizes the latest video captured and acquired by the camera 14 and the recorded image stored in the memory 17 to generate a synthesized video. At this time, the image processing unit 21 synthesizes the latest image of the camera 14 with the recorded image stored in the memory 17. The latest video of the camera 14 is a moving image.

  The display control unit 22 causes the display device 16 to display only the latest video of the camera 14 or the composite video generated by the image processing unit 21.

  Based on the measurement value of the rotation sensor 15, the travel detection unit 23 detects whether the travel device 2 has traveled more than a predetermined distance, that is, whether the forklift 1 has traveled more than a predetermined distance. The image erasing unit 24 erases the recorded image from the memory 17 when the traveling detection unit 23 detects that the traveling device 2 has traveled more than a predetermined distance.

  FIG. 3 is a flowchart showing details of the display processing procedure executed by the controller 18. The camera 14 always captures a moving image in front of the traveling device 2, and the controller 18 always acquires the latest video of the camera 14.

  In FIG. 3, the controller 18 first determines whether or not a lift raising operation has been started by the lift operation lever 10 (step S101). When the lift raising operation is started, the controller 18 controls the camera 14 so as to capture a still image in front of the traveling device 2 by the camera 14, and acquires an image captured by the camera 14 (step S102). Subsequently, the controller 18 stores the captured image of the camera 14 as a recorded image in the memory 17 (step S103).

  FIG. 4A is a diagram illustrating an example of a recorded image. The recorded image shown in FIG. 4A is a still image in which the shelf 25 in which the luggage A loaded on the fork 6 (see FIG. 4B) is stored is shown together with the upper part of the mast 4.

  Subsequently, the controller 18 generates a composite video by transmitting and synthesizing the latest video of the camera 14 to the recorded image stored in the memory 17 (step S104). At this time, the latest video of the camera 14 is combined with the recorded image in a translucent state (for example, a transmittance of 50%).

  FIG. 4B is a diagram illustrating an example of a composite image. The composite image shown in FIG. 4B shows a state in which the load A loaded on the fork 6 is lifted by the lift cylinder 7 in order to store the load A at a predetermined position on the shelf 25. At this time, the latest mast 4, fork 6 and luggage A are shown in a more transparent state than the shelf 25.

  Subsequently, the controller 18 causes the display device 16 to display the composite video generated in step S104 (step S105). For this reason, on the display device 16, the latest video is superimposed on the recorded image.

  At this time, in the composite image shown in FIG. 4B, the shelf 25 is not shown in a transparent state, but the latest mast 4, fork 6 and luggage A are shown in a transparent state. For this reason, in the display device 16, a part of the shelf 25 is not hidden by the mast 4, the fork 6 and the luggage A. Therefore, the operator can confirm the storage position of the luggage A in the shelf 25 by the display 16.

  Subsequently, the controller 18 determines whether or not the traveling device 2 has traveled more than a predetermined distance based on the measurement value of the rotation sensor 15 (step S106). The travel distance of the travel device 2 is calculated from the rotation speed of the wheel 2a measured by the rotation sensor 15 and the diameter of the wheel 2a. Here, the predetermined distance is, for example, about several tens of centimeters.

  When the traveling device 2 is not traveling more than a predetermined distance, the controller 18 executes the above steps S104 and S105 again. The controller 18 deletes the recorded image from the memory 17 when the traveling device 2 travels a predetermined distance or more (step S107). Thereafter, the controller 18 displays only the latest video of the camera 14 on the display 16 (step S108).

  The controller 18 also displays only the latest video of the camera 14 on the display 16 when it is determined in step S101 that the lift raising operation has not been started by the lift operation lever 10 (step S108).

  In the above, steps S101 and S102 are executed by the imaging control unit 19. Step S103 is executed by the image storage unit 20. Step S104 is executed by the image processing unit 21. Step S <b> 105 is executed by the display control unit 22. Step S106 is executed by the travel detection unit 23. Step S107 is executed by the image erasing unit 24. Step S <b> 108 is executed by the display control unit 22.

  As described above, in the present embodiment, when the lifting operation of the fork 6 of the cargo handling device 3 is started, an image acquired by imaging the front of the traveling device 2 by the camera 14 is stored in the memory 17 as a recorded image. Then, a synthesized video is generated by synthesizing the latest video acquired by photographing the front of the traveling device 2 with the camera 14 and the recorded image stored in the memory 17, and the synthesized video is displayed on the display 16. The Here, when the synthesized video is generated, the latest video is transmitted through the recorded image and synthesized. For this reason, even if the fork 6 is moved up, both the recorded image and the latest video are clearly displayed on the display 16 as a whole. Thus, the front blind spot of the camera 14 can be supplemented when the fork 6 is moved up without increasing the number of cameras 14. As a result, the operability of the cargo handling device 3 by the operator can be improved.

  Further, since the latest video is transmitted through the recorded image stored in the memory 17 and synthesized, the recorded image close to the background becomes clearer than the latest video. Therefore, the composite image displayed on the display 16 is easy to see.

  Further, when the lift raising operation is started by the lift operation lever 10, the camera 14 automatically images the front of the traveling device 2. Therefore, the operator does not have to instruct the imaging timing by the camera 14 with a manual operation switch or the like, and the burden on the operator is reduced.

  For example, when the traveling device 2 travels so as to move forward toward the shelf 25, the distance between the shelf 25 and the camera 14 is reduced, and the recorded image stored in the memory 17 is an actual captured image of the camera 14. Therefore, the recorded image becomes unnecessary. In the present embodiment, when it is detected that the traveling device 2 has traveled a predetermined distance, the recorded image is erased from the memory 17, so that the process of combining the latest video and the recorded image becomes unnecessary, and the process is simplified. Can be planned.

  In the present embodiment, when the traveling device 2 travels a predetermined distance or more, the recorded image is erased from the memory 17. However, when the operation for lowering the fork 6 is performed by the lift operation lever 10, the recorded image is also erased from the memory 17. The recorded image may be erased.

  FIG. 5 is a schematic configuration diagram showing a display system according to another embodiment of the present invention. The display system 13 of the present embodiment includes a camera 14, a rotation sensor 15, a wheel angle sensor 30, an erasing switch 31, a display device 16, a memory 17, and a controller 32. The wheel angle sensor 30 is a sensor that measures the turning angle of the wheel 2a. The erase switch 31 is a manual operation switch for the operator to instruct to erase the recorded image from the memory 17.

  The controller 32 includes an imaging control unit 19, an image storage unit 20, an image processing unit 21, a display control unit 22, an image correction unit 33, and an image erasing unit 34.

  The image correction unit 33 corrects the recorded image stored in the memory 17 based on the traveling state of the traveling device 2 detected from the measured values of the rotation sensor 15 and the wheel angle sensor 30. The rotation sensor 15 and the wheel angle sensor 30 constitute at least a part of a state detection unit that detects the traveling state of the traveling device 2.

  The image erasure unit 34 erases the recorded image from the memory 17 when the erasure switch 31 is turned on.

  FIG. 6 is a flowchart showing details of the display processing procedure executed by the controller 32. The description of the same procedure as that in the flowchart shown in FIG. 3 is omitted.

  In FIG. 6, after executing steps S <b> 101 to S <b> 105, the controller 32 acquires the captured image of the camera 14 in step S <b> 102 based on the measured values of the rotation sensor 15 and the wheel angle sensor 30. It is determined whether or not the time has changed (step S111). At this time, the controller 32 calculates the travel distance of the traveling device 2 calculated from the rotational speed of the wheel 2 a measured by the rotation sensor 15 and the diameter of the wheel 2 a and the turning angle of the wheel 2 a measured by the wheel angle sensor 30. Based on the above, it is determined whether or not the traveling state of the traveling device 2 has changed.

  When the traveling state of the traveling device 2 changes, the controller 32 corrects the recorded image stored in the memory 17 based on the latest traveling state of the traveling device 2 (step S112). Then, the controller 32 overwrites and saves the corrected recorded image in the memory 17 (step S113). And the controller 32 performs said procedure S104, S105 again.

  On the other hand, when the traveling state of the traveling device 2 has not changed, the controller 32 determines whether or not the erasure switch 31 is ON (procedure S114). When the erasure switch 31 is not ON, the controller 32 executes the above steps S104 and S105 again. When the erase switch 31 is ON, the controller 32 erases the recorded image from the memory 17 (step S115).

  In the above, steps S111 and S112 are executed by the image correction unit 33. Step S113 is executed by the image storage unit 20. Steps S114 and S115 are executed by the image erasing unit 34.

  By the way, as described above, when the traveling state of the traveling device 2 changes, the recorded image stored in the memory 17 changes from the actual captured image of the camera 14, and thus the recorded image becomes unnecessary. In the present embodiment, since the recorded image stored in the memory 17 is corrected based on the traveling state of the traveling device 2, an appropriate recorded image corresponding to the latest traveling state of the traveling device 2 can be obtained.

  The present invention is not limited to the above embodiment. For example, in the above embodiment, as shown in FIG. 4B, the image processing unit 21 transmits the latest video of the camera 14 to the recorded image stored in the memory 17 and synthesizes the synthesized video. However, the form is not particularly limited. As illustrated in FIG. 7, the image processing unit 21 may generate a composite image by transmitting and synthesizing the recorded image stored in the memory 17 with the latest video of the camera 14. That is, in the recorded image shown in FIG. 7, the shelf 25 is shown in a state that is more transparent than the latest mast 4, fork 6 and luggage A. In this case, since the process of transmitting the latest video from the camera 14 every time is not necessary, the image processing can be simplified.

  Moreover, in the said embodiment, when lift raising operation is started by the lift operation lever 10, the front of the traveling apparatus 2 is imaged with the camera 14, and the picked-up image is preserve | saved in the memory 17 as a recorded image. The form is not limited. For example, an operator has an imaging switch for manually instructing imaging by the camera 14, and when the imaging switch is turned on, the camera 14 images the front of the traveling device 2 and uses the captured image as a recorded image. It may be stored in the memory 17.

  Furthermore, in the above-described embodiment, the on-vehicle camera 14 and the display device 16 are used. However, a smart device or a feature phone may be used as the imaging unit, and the smart device or head may be used as the display unit. A mount display or the like may be used.

  Further, in the above embodiment, the controllers 18 and 32 and the memory 17 are provided in the traveling device 2 of the forklift 1, but the images and videos of the camera 14, the output signal to the display 16, the sensor measurement signal, and the like are wirelessly transmitted. If the transmission / reception can be performed by the controller 18, the controllers 18 and 32 and the memory 17 may not be provided in the forklift 1.

  Moreover, although the industrial vehicle of the said embodiment is the forklift 1 provided with the cargo handling apparatus 3 which has the fork 6 which raises / lowers, this invention is equipped with the cargo handling apparatus which has attachments, such as a side shift fork which shifts to right and left, for example. It can also be applied to other industrial vehicles.

  DESCRIPTION OF SYMBOLS 1 ... Forklift (industrial vehicle), 2 ... Traveling apparatus, 3 ... Cargo handling device, 10 ... Lift operation lever (operation part), 13 ... Display system, 14 ... Camera (imaging part), 15 ... Rotation sensor (state detection part) , 16 ... Display (display unit), 17 ... Memory (storage unit), 18 ... Controller, 21 ... Image processing unit, 23 ... Travel detection unit, 24 ... Image deletion unit, 30 ... Wheel angle sensor (state detection unit) 32, controller, 33, image correcting unit, 34, image erasing unit.

Claims (5)

In an industrial vehicle display system having a traveling device and a cargo handling device disposed on the front side of the traveling device,
An imaging unit for imaging the front of the traveling device;
A storage unit that stores an image captured and acquired by the imaging unit at the start of operation of the cargo handling device as a recorded image;
An image processing unit that generates a synthesized video by synthesizing the latest video imaged and acquired by the imaging unit and the recorded image stored in the storage unit;
A display unit for displaying the composite video generated by the image processing unit,
The display system for an industrial vehicle, wherein the image processing unit synthesizes one of the recorded image and the latest video through the other of the recorded image and the latest video.   The industrial image display system according to claim 1, wherein the image processing unit synthesizes the latest image through the recorded image.   The said storage part memorize | stores the image imaged and acquired by the said imaging part as said recording image, when operation of the operation part for operating the said cargo handling apparatus is started. 2. The industrial vehicle display system according to 2. A travel detector that detects whether the travel device has traveled more than a predetermined distance;
The image erasing unit for erasing the recorded image from the storage unit when the travel detection unit detects that the travel device has traveled more than a predetermined distance. The industrial vehicle display system according to any one of the preceding claims. A state detection unit for detecting a traveling state of the traveling device;
The image correcting unit for correcting the recorded image stored in the storage unit based on a traveling state of the traveling device detected by the state detecting unit. An industrial vehicle display system according to claim 1.

Images