JP2022144723A - Cargo handling vehicle - Google Patents

Abstract

To provide a cargo handling vehicle which can properly secure a front visual field without a driver wearing anything special.SOLUTION: A cargo handling vehicle comprises: a front camera 4 which images the front side of a cargo handling gear 3; an image extraction unit which extracts an image to be projected from an image captured by the front camera 4; a projector 6 which projects the extracted image from the rear side of the cargo handling gear 3 toward the cargo handling gear 3 to display a transmission image on the cargo handling gear 3; a projection image camera 7 which images the transmission image and generates the projection image; a correction coefficient output unit which outputs a correction coefficient of each luminance of each color component of each pixel of the image to be projected on the basis of each luminance of each color component of each pixel of the image to be projected and each luminance of each color component of each pixel of the projection image; and a correction processing unit which corrects each luminance of each color component of each pixel of the image to be projected on the basis of the correction coefficient. The projector 6 projects the image to be projected corrected by the correction processing unit toward the cargo handling gear 3 again.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to a cargo handling vehicle provided with cargo handling equipment.

The cargo handling vehicle is equipped with a cargo handling device, and performs cargo handling work using the cargo handling device. Among these types of cargo handling vehicles, for example, a forklift has a cargo handling device in front of the driver's seat. This cargo handling device has a vertically extending mast, a carriage connected to the mast, a hydraulic cylinder for moving the carriage up and down along the mast, and a fork and a backrest connected to the carriage. A driver scoops up a pallet on which a load is placed using a fork and transports the load.

By the way, since the cargo handling equipment and the cargo carried by the cargo handling equipment are in front of the driver, they block part of the front view of the driver. As a result, the driver may mistakenly check what is ahead when moving, resulting in an accident. Accordingly, visibility improvement systems have been developed, for example, as disclosed in US Pat. This field-of-view improvement system includes a head-mounted display, a camera that captures an image of the front of the cargo handling apparatus, a camera attached to the head-mounted display, and an image processing device. The image processing device synthesizes the images captured by the two cameras to generate an image in front of the driver as seen through the cargo handling device, and displays the generated image on the head-mounted display. The driver can check the front of the cargo handling equipment by the image of the head-mounted display. However, in the case of this invention, a head-mounted display must be prepared for each driver, leading to an increase in cost.

JP-A-2008-143701

Therefore, the problem to be solved by the present invention is to provide a cargo handling vehicle capable of appropriately ensuring a forward field of view without requiring the driver to wear anything.

In order to solve the above problems, the cargo handling vehicle according to the present invention includes:
a driving unit provided in the vehicle body;
a cargo handling device provided in front of the driving unit;
a front camera that captures an image in front of the cargo handling device and generates a first front image;
an image extraction unit that extracts a second front image to be projected toward the material handling device from the first front image;
a projector that projects a second front image from behind the cargo handling device toward the cargo handling device and displays a transmitted image on the cargo handling device;
a projection image camera that captures a transmission image and generates a projection image;
Based on each luminance of each color component of each pixel of the second front image and each luminance of each color component of each pixel of the projection image, a correction coefficient of each luminance of each color component of each pixel of the second front image is output. a correction coefficient output unit;
a correction processing unit that corrects each luminance of each color component of each pixel of the second front image based on the correction coefficient,
The projector is characterized by projecting the second front image corrected by the correction processing section.

For example, the cargo handling vehicle
When the correction coefficient output unit receives the luminance of each color component of each pixel of the second front image and the luminance of each color component of each pixel of the projection image, the correction coefficient output unit outputs the luminance of each color component of each pixel of the second front image. It has a correction factor trained model trained to output correction factors.

The cargo truck preferably comprises:
further comprising a head position detection unit that detects the height of the driver's head,
An image extraction unit extracts a second front image from the first front image based on the height of the driver's head.

For example, the cargo handling vehicle
The cargo handling vehicle is a forklift,
A forward camera is provided at the tip of the fork.

The cargo truck preferably comprises:
a fork position detector that detects the position of the fork;
a head position detection unit that detects the height of the driver's head,
An image extractor extracts a second front image from the first front image based on the position of the fork and the height of the driver's head.

The cargo truck preferably comprises:
An image extraction unit has an image extraction trained model that outputs an image to be projected by the projector when the first forward image, the height of the driver's head and the position of the fork are input.

The cargo truck preferably comprises:
The projector also projects a second front image toward the cargo being carried, causing the cargo to display a transmission image.

According to the cargo handling vehicle of the present invention, the driver can appropriately ensure a forward field of view without wearing anything in particular.

A is a side view showing a cargo handling vehicle according to one embodiment of the present invention, and B is an enlarged view of the B portion of A. FIG. Figure 2 shows a forward view of the cargo truck shown in Figure 1; 2 is a functional block diagram of a control unit shown in FIG. 1; FIG. It is a figure which shows the 2nd front image which the 1st front image which the front camera produced|generated, and the image extraction part extracted. FIG. 11 shows the cargo handling equipment and the cargo on which the second forward image is projected; It is a figure which shows the cargo handling apparatus and cargo on which the 2nd front image correct|amended by the correction|amendment process part was projected.

A cargo handling vehicle C according to an embodiment of the present invention will be described below with reference to the accompanying drawings. In the drawing, the X-axis indicates the front-rear direction, the Y-axis indicates the left-right direction, the Z-axis indicates the vertical direction, and the axes are orthogonal to each other.

As shown in FIG. 1, the cargo handling vehicle C according to the present embodiment is a forklift, and includes a vehicle body 1, an operating section 2 provided on the vehicle body 1, and a cargo handling device 3 provided in front of the operating section 2. , a front camera 4 , a head position detection unit 5 , a projector 6 , a projection image camera 7 , and a control unit 8 .

The vehicle body 1 includes front and rear wheels 10, a vehicle body 11, four pillars 12 provided on the vehicle body 11 and extending vertically, and a head guard 13 provided on the upper end of the pillar 12 and extending horizontally. have.

The driving unit 2 has a seat 20 and an operation unit 21 including a handle, a lever, and the like. A driver D sits on the seat 20 and operates the operation unit 21 to drive the cargo handling vehicle C and operate the cargo handling device 3 .

The cargo handling device 3 has a pair of left and right masts 30 , an upper beam 31 , a carriage 32 , a backrest 33 , a pair of left and right forks 34 , and a pair of left and right lift cylinders 35 . The mast 30 is provided in front of the vehicle body 1 and extends vertically, and the carriage 32 is connected to the mast 30 so as to be able to move up and down. Backrest 33 and forks 34 are connected to carriage 32 , which is raised and lowered along mast 30 by lift cylinders 35 . As shown in FIG. 2 , the upper beam 31 spans between the left and right masts 30 at the upper ends of the masts 30 .

The cargo handling device 3 scoops up the pallet P with the forks 34, and the cargo handling vehicle C transports the pallet P together with the load W placed on the pallet P, as shown in FIGS.

The front camera 4 is provided on either or both of the left and right forks 34, as shown in FIG. 1B. The front camera 4 captures an image in front of the material handling apparatus 3 and generates a first front image I1. FIG. 4 shows the first front image I1 generated by the front camera 4. As shown in FIG. Since the imaging range of the front camera 4 changes according to the height and lateral position of the fork 34 , the first front image I<b>1 generated by the front camera 4 also changes according to the position of the fork 34 . Note that the front camera 4 may be horizontally fixed by a gimbal so that the imaging direction does not swing up and down due to the inclination of the fork 34 . The first front image I1 generated by the front camera 4 is output to the image extractor 81 (see FIG. 3).

The head position detection unit 5 is provided on the lower surface of the head guard 13 and detects the height of the driver's D head. The detected height of the head is output to the image extractor 81 . The head position detection unit 5 may include, for example, a camera and an image analysis unit, and detect the height of the head of the driver D by analyzing an image captured by the camera with the image analysis unit. .

The control unit 8 is provided in the vehicle body 11, and includes a cargo handling device control unit 80, an image extraction unit 81, a correction coefficient output unit 82, and a correction processing unit 83, as shown in FIG.

The cargo handling device control section 80 controls the lift cylinder 35 to raise and lower the fork 34 based on the electric signal from the operation section 21 . The cargo handling device control unit 80 also has a fork position detection unit that detects the position of the fork 34 . The cargo handling device 3 may have, for example, a known fork bar and side hydraulic cylinders, and the cargo handling device control unit 80 may be configured to move the forks 34 in the left-right direction by the side hydraulic cylinders. The position of the fork 34 detected by the fork position detection section is output to the image extraction section 81 .

The image extraction unit 81 has an image extraction trained model. When the first front image I1, the height of the driver D's head, and the position of the fork 34 are input, the image extraction trained model generates an image to be projected by the projector 6 (hereinafter referred to as "second front image I2”) is learned in advance. The height of the head of the driver D varies depending on the sitting height and sitting manner of the driver D, and the first forward image I1 generated varies depending on the position of the forks 34 .

The image extraction trained model may be trained, for example, by supervised learning using neural networks and deep learning. Specifically, the image extraction trained model is a combination of the first front image I1, the height of the driver D's head, and the position of the fork 34 as input data and the second front image I2 as output data. The data may be passed as teacher data, and the correlation between the input data and the output data may be learned in advance. The second forward image I2 is optimized for the forward field of view of the driver D and generated. This image extraction trained model learning method is merely an example and is not limited to this. FIG. 4 shows the second front image I2 extracted by the image extractor 81. As shown in FIG.

As shown in FIG. 1, the projector 6 is provided on the lower surface of the head guard 13, and projects the second front image I2 from behind the cargo handling device 3 toward the cargo handling device 3 and the cargo W as shown in FIG. Then, the transmitted image I3 is displayed on the cargo handling device 3 and the cargo W. The position of the projector 6 is merely an example, and the second front image I2 can be projected toward the cargo handling device 3 and the cargo W, and the projection is obstructed by the shadow of the driver D. It is not particularly limited as long as it is not a position. As shown in FIG. 5, the transmissive image I3 displayed on the cargo handling device 3 and the cargo W is visually different in color from the second forward image I2. This is because the reflection characteristics of the projection surface are not uniform since the projection surface is different from a monochromatic screen, and the projection surface is affected by the color of the projection surface itself. In addition, the cargo handling device 3 and the cargo W may be stained or peeled off. is displayed differently than the face of

The projection image camera 7 is provided on the lower surface of the head guard 13, captures the transmission image I3 displayed on the cargo handling device 3 and the cargo W, and generates a projection image I4. The position of the projection image camera 7 is merely an example and is not limiting. The projected image I4 is output to the correction coefficient output unit 82. FIG.

The correction coefficient output unit 82 calculates each color of each pixel of the second front image I2 based on each luminance of each color component of each pixel of the second front image I2 and each luminance of each color component of each pixel of the projection image I4. Outputs the correction coefficient for each luminance of the component. The correction coefficient output unit 82 according to this embodiment has a correction coefficient learned model. The correction coefficient of each luminance of each color component of each pixel of the output second front image I<b>2 is output to the correction processing section 83 .

The correction coefficient learned model is input with each luminance of each color component of each pixel of the second front image I2 and each luminance of each color component of each pixel of the projection image I4, and then each pixel of the second front image I2. is learned in advance to output the correction coefficient of each pixel of each color component. The correction coefficient trained model may be trained, for example, by supervised learning using neural networks and deep learning. In this case, for example, the correction coefficient learned model uses a combination of each luminance of each color component of each pixel of the projection image I4 and each luminance of each color component of each pixel of the second front image I2 as teacher data, and the correlation may be learned.

The correction processing unit 83 corrects the brightness of each color component of each pixel of the second front image I2 based on the correction coefficients input from the correction coefficient output unit 82 . The second front image I2 in which each luminance of each color component of each pixel is corrected is output to the projector 6. FIG.

The projector 6 projects the second front image I2 corrected by the correction processing unit 83 onto the cargo handling device 3 and the cargo W again. The displayed transmission image I3 is captured again by the projection image camera 7 to generate a projection image I4. Further, the correction coefficient output unit 82 outputs a correction coefficient corresponding to the projected image I4, and the correction processing unit 83 corrects each brightness of the second front image I2 and outputs the corrected image to the projector 6. FIG. By repeating these steps, a more appropriate transmissive image I3 is displayed on the cargo handling device 3 and cargo W, as shown in FIG. As a result, the driver D can appropriately ensure a forward field of view without wearing anything in particular.

Although the cargo handling vehicle C according to one embodiment of the present invention has been described above, the cargo handling vehicle C according to the present invention is not limited to this. For example, the cargo handling vehicle C according to the present invention may be implemented by the following modifications.

- The correction coefficient output unit 82 may acquire, for example, the light distribution characteristics of the projector 6 with respect to the cargo handling device 3 and the cargo W, and output the correction coefficient using the light distribution characteristics. Specifically, the control unit 8 further has a light distribution characteristic acquisition unit that acquires the light distribution characteristic. The projector 6 projects a single-color reference image onto the cargo handling device 3 and the load W before projecting the second front image I2, and the projection image camera 7 captures the projected reference image to produce a projection image I4. is generated and output to the light distribution characteristic acquisition unit. The light distribution characteristic acquisition unit analyzes and normalizes the luminance of each color component of each pixel of the projection image I4, and creates an orientation distribution indicating the light distribution characteristic of each color component of the projector 6 with respect to the cargo handling device 3 and the cargo W. do. A correction coefficient calculator 82 outputs a correction coefficient using the light distribution. Thereby, a correction coefficient adapted to the manner of the cargo handling device 3 and the cargo W is output.

- The cargo handling vehicle C is not limited to the above-mentioned forklift. For example, the cargo handling vehicle C may be a reach forklift or a cargo handling vehicle C equipped with attachments such as rams and spreaders. In this case, the front camera 4 may be provided at the tip of the ram or the like.

- The number of cameras that the front camera 4 has, the number of projectors that the projector 6 has, and the number of cameras that the projection image camera 7 has are not limited to one. Each of the front camera 4, the projector 6, and the projection image camera 7 may be imaged by a plurality of cameras or may be projected by a plurality of projectors.

C cargo handling vehicle D driver W cargo P pallet R road surface I1 first front image I2 second front image I3 transmission image 1 vehicle body 10 wheel 11 vehicle body body 12 pillar 13 head guard 2 driving unit 20 seat 21 operating unit 3 cargo handling device 30 mast 31 upper beam 32 carriage 33 backrest 34 fork 35 lift cylinder 4 front camera 5 head position detector 6 projector 7 projected image camera 8 controller 80 cargo handling device controller 81 image extractor 82 correction coefficient output unit 83 correction processor

Claims (7)

a driving unit provided in the vehicle body;
a cargo handling device provided in front of the driving unit;
a front camera that captures an image in front of the cargo handling device and generates a first front image;
an image extraction unit that extracts a second front image to be projected toward the material handling device from the first front image;
a projector that projects the second front image from behind the cargo handling device toward the cargo handling device and displays a transmitted image on the cargo handling device;
a projection image camera that captures the transmission image and generates a projection image;
A correction coefficient for each luminance of each color component of each pixel of the second front image based on each luminance of each color component of each pixel of the second front image and each luminance of each color component of each pixel of the projection image. a correction coefficient output unit that outputs
a correction processing unit that corrects each luminance of each color component of each pixel of the second front image based on the correction coefficient,
The cargo handling vehicle, wherein the projector projects the second front image corrected by the correction processing unit. When each luminance of each color component of each pixel of the second front image and each luminance of each color component of each pixel of the projection image are input, the correction coefficient output unit receives each luminance of each color component of each pixel of the second front image, and 2. The cargo handling vehicle according to claim 1, further comprising a correction coefficient trained model trained to output a correction coefficient for each color component of . further comprising a head position detection unit that detects the height of the driver's head,
The cargo handling vehicle according to claim 1 or 2, wherein the image extraction unit extracts the second front image from the first front image based on the height of the driver's head. The cargo handling vehicle is a forklift,
The cargo handling vehicle according to claim 1 or 2, wherein the front camera is provided at the tip of the fork. a fork position detection unit that detects the position of the fork;
a head position detection unit that detects the height of the driver's head,
The image extraction unit extracts the second front image from the first front image based on the position of the fork and the height of the driver's head.
The cargo handling vehicle according to claim 4, characterized in that: The image extraction unit has an image extraction trained model that outputs the second front image when the first front image, the height of the driver's head, and the position of the fork are input. 6. A cargo handling vehicle according to claim 5, characterized in that: The cargo handling vehicle according to any one of claims 1 to 6, wherein the projector further projects the second front image toward a load to be carried and causes the load to display the transmission image. .

Images