JP7057053B1 - Cargo handling system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a cargo handling system in which a driver of a cargo handling vehicle can visually confirm traveling safety even in a facility in which a storage shelf is arranged. A cargo handling system S includes an installed camera 1 and a cargo handling vehicle C. The installed camera 1 takes an image of the first passage on the first road surface R1 and generates the first image. The cargo handling vehicle C is composed of an image extraction unit that extracts a second image from the first image, a projector that projects the second image onto the storage shelf Q to display a transparent image I3, and a transparent image I3 displayed on the storage shelf Q. Each color of each pixel of the second image is based on the projection image camera that generates the projected image I4, each brightness of each color component of each pixel of the second image, and each brightness of each color component of each pixel of the projected image I4. It includes a correction coefficient output unit that outputs a correction coefficient for each brightness of the component, and a correction processing unit that corrects each brightness of each color component of each pixel of the second image based on the correction coefficient. The projector again projects the second image corrected by the correction processing unit toward the storage shelf Q. [Selection diagram] Fig. 1

Description

The present invention relates to a cargo handling system including a cargo handling vehicle.

Cargo handling vehicles such as forklifts may be used where multiple storage shelves are arranged. When moving from the aisle between the storage shelves, the driver of this cargo handling vehicle takes great care not to collide with people or other cargo handling vehicles passing through other aisles that intersect the aisle. There is a need. However, the storage shelves and the loads placed on the storage shelves obstruct the view to other aisles, and it is not possible to see people or cargo handling vehicles passing through the other aisles.

Therefore, for example, in the forklift safety device disclosed in Patent Document 1, a safety lamp is provided on the forklift, and the spot light emitted by the safety lamp notifies the surrounding workers of the approach of the forklift to enhance the safety of cargo handling work. Further, in the forklift safety device disclosed in Patent Document 2, the surrounding workers are alerted by sound and light.

However, in the invention described in the above document, although it is possible to call attention, it is not possible for the driver to visually confirm the safety of driving.

Japanese Unexamined Patent Publication No. 2018-149820 Japanese Patent No. 4623631

Therefore, an object to be solved by the present invention is to provide a cargo handling system that allows the driver of a cargo handling vehicle to visually confirm the safety of driving even in a facility where storage shelves are arranged.

In order to solve the above problems, the cargo handling system according to the present invention is
An installed camera installed on the storage shelf, capturing the first passage adjacent to the storage shelf, and generating the first image.
Image extraction to extract from the first image the second image to be projected from the second aisle adjacent to the storage shelf and intersecting the first aisle toward the storage shelf, the load and pallet stored in the storage shelf, or either of them. Department and
Cargo handling vehicle and
A projector provided on a cargo handling vehicle that projects a second image from a second aisle onto a storage shelf, load and pallet, or any of them, and displays a transmitted image on the storage shelf, load and pallet, or any of them.
A projection image camera installed on the cargo handling vehicle that captures the displayed transmission image and generates a projection image.
Correction coefficient that outputs the correction coefficient of each brightness of each color component of each pixel of the second image based on each brightness of each color component of each pixel of the second image and each brightness of each color component of each pixel of the projected image. Output section and
A correction processing unit that corrects each luminance of each color component of each pixel of the second image based on the correction coefficient is provided.
The projector is characterized in that it projects a second image corrected by the correction processing unit.

The cargo handling system is, for example,
When the output unit inputs the brightness of each color component of each pixel of the second image and the brightness of each color component of each pixel of the projected image, the correction coefficient of each color component of each pixel of the second image is input. Has a correction factor trained model trained to output.

The cargo handling system is preferably
Further equipped with a position detection unit that detects the position of the cargo handling vehicle,
The image extraction unit extracts the second image from the first image based on the position of the cargo handling vehicle.

The cargo handling system is, for example,
The position detection unit has an image extraction trained model trained to output a second image when the first image and the position of the cargo handling vehicle are input.

The cargo handling system is, for example,
Further equipped with an installed camera and a server capable of communicating with the cargo handling vehicle,
The server has any or all of an image extraction unit, a correction coefficient output unit, and a correction processing unit.

The cargo handling system is, for example,
The cargo handling vehicle has any or all of an image extraction unit, a correction coefficient output unit, and a correction processing unit.

According to the cargo handling vehicle of the present invention, the driver of the cargo handling vehicle can visually confirm the safety of driving even in the facility where the storage shelves are arranged.

It is a schematic whole view of the cargo handling system which concerns on one Embodiment of this invention. It is a figure which shows the 1st image generated by an installed camera and the 2nd image extracted by an image extraction unit. It is a side view of the cargo handling vehicle shown in FIG. It is a functional block diagram of the control part shown in FIG. It is a figure which shows the storage shelf which projected the 2nd image and displayed the transparent image. It is a figure which shows the storage shelf which projected the corrected 2nd image and displayed the transparent image. It is a functional block diagram of the cargo handling system which concerns on another embodiment of this invention.

Hereinafter, the cargo handling system S according to the embodiment of the present invention will be described with reference to the attached drawings. In the figure, the X-axis indicates the front-rear direction, the Y-axis indicates the left-right direction, and the Z-axis indicates the up-down direction. Also, the axes are orthogonal to each other.

FIG. 1 is a schematic overall view of the cargo handling system S. The cargo handling system S includes a storage shelf Q, first and second aisles, an installed camera 1, and a cargo handling vehicle C.

The storage shelf Q has a shelf board Qa and a frame Qb that supports the shelf board Qa. The pallet P1 is placed on the shelf board Qa, and the load W1 is placed on the pallet P1. The storage shelf Q may be arranged in the facility, for example.

The space on the first road surface R1 is the first passage, and the space on the second road surface R2 is the second passage. The first and second aisles are separated by storage shelves Q and intersect each other. As shown in FIG. 1, the first passage may be a passage through which a passerby M or another cargo handling vehicle passes.

In this embodiment, the installed camera 1 is composed of two cameras 1a and 1b, and is arranged in a frame Qb adjacent to the first passage. The cameras 1a and 1b image the first passage and generate the first images I1a and I1b. The first image I1 is an image including the first images I1a and I1b. FIG. 2 shows the first images I1a and I1b generated by the cameras 1a and 1b. The installed camera 1 according to the present embodiment is configured to image the entire first passage, but is merely an example, and is configured not to capture a place far away from an intersection intersecting with the second passage, for example. You may be. The first image I1 generated by the installed camera 1 is output to the image extraction unit 81 (see FIG. 4).

As shown in FIGS. 1 and 3, the cargo handling vehicle C includes a vehicle body 2, a driving unit 3, a cargo handling device 4, a projector 6, a projection image camera 7, and a control unit 8. In the present embodiment, the cargo handling vehicle C is a manned forklift, but is merely an example, and may be, for example, a manned and unmanned forklift.

The vehicle body 2 includes front and rear wheels 20, a vehicle body body 21, four pillars 22 provided on the vehicle body body 21 and extending vertically, and a head guard 23 provided on the upper end of the pillars 22 and extending horizontally. Have.

The driver unit 3 has a seat 30 and an operation unit 31 including a steering wheel, a lever, and the like. The driver D sits on the seat 30 and operates the operation unit 31 to drive the cargo handling vehicle C and operate the cargo handling device 4.

The cargo handling device 4 has a pair of left and right masts 40, a carriage 42, a backrest 43, a pair of left and right forks 44, and a pair of left and right lift cylinders 45. The mast 40 is provided in front of the vehicle body 2 and extends vertically, and the carriage 42 is connected to the mast 40 so as to be able to move up and down. The backrest 43 and the fork 44 are connected to the carriage 42, and the carriage 42 is moved up and down along the mast 40 by the lift cylinder 45.

The cargo handling device 4 scoops up the pallet P2 by the fork 44. As shown in FIGS. 1 and 3, the cargo handling vehicle C carries the load W2 mounted on the pallet P2 together with the pallet P2. The cargo handling vehicle C shown in FIG. 1 is heading backward (in the direction of the arrow).

The control unit 8 is provided in the vehicle body main body 21, and has a vehicle body position detection unit 80, an image extraction unit 81, a correction coefficient output unit 82, and a correction processing unit 83, as shown in FIG. ..

The vehicle body position detecting unit 80 detects the position of the cargo handling vehicle C. The vehicle body position detecting unit 80 may be configured by a known technique. For example, the vehicle body position detection unit 80 receives a GPS signal outdoors to detect the position of the cargo handling vehicle C, and indoors, it performs relative positioning and corrects the reading position by reading a marker provided at a predetermined position indoors. The position of the cargo handling vehicle C may be detected by using the position correction method. The position of the cargo handling vehicle C detected by the vehicle body position detection unit 80 is output to the image extraction unit 81.

The image extraction unit 81 has an image extraction trained model. The image extraction trained model is trained in advance to output an image (hereinafter referred to as "second image") I2 to be projected by the projector 6 when the first image I1 and the position of the cargo handling vehicle C are input. Has been done.

The image extraction trained model may be trained, for example, by supervised learning by neural network and deep learning. Specifically, in the image extraction trained model, the combination data in which the first image I1 and the position of the cargo handling vehicle C are used as input data and the second image I2 is used as output data is passed as teacher data, and the input data is used as the input data. The correlation with the output data may be learned in advance. The second image I2 as the teacher data looks like a storage shelf when the driver D of the cargo handling vehicle C traveling in the second aisle sees the second image I2 projected on the storage shelf Q, the pallet P1 and the load W1. It is optimized and generated so as to see through the Q, the pallet P1 and the load W1 and give the illusion of looking at the first passage. The learning method of this image extraction trained model is merely an example, and is not limited thereto. FIG. 2 shows the second image I2 extracted by the image extraction unit 81.

As shown in FIG. 3, the projector 6 is provided on the lower surface of the head guard, and as shown in FIG. 1, the second image I2 is projected toward the storage shelf Q, the pallet P1 and the load W1, and the transmitted image is projected. I3 is displayed on the storage shelf Q, the pallet P1, and the load W1. The position of the projector 6 is merely an example, and the second image I2 can be appropriately projected toward the storage shelf Q, the pallet P1 and the load W1, and the driver D and the vehicle body 2 are shadowed. It is not particularly limited as long as it is not a position where the projection is obstructed.

As shown in FIG. 5, the transparent image I3 displayed on the storage shelf Q, the pallet P1 and the load W1 has a different apparent color from the second image I2. This is because the projection surface is different from the white screen, so that the reflection characteristics of the projection surface are not constant and the color of the projection surface itself is affected. Further, the storage shelf Q, the pallet P1 and the load W1 may be dirty or peeled off, and even in this case, a part of the transparent image I3 is displayed in the portion where the dirt or peeling is generated. The color is displayed differently from the other faces.

The projection image camera 7 is provided adjacent to the projector 6 on the lower surface of the head guard 23, captures the transmission image I3 displayed on the storage shelf Q, the pallet P1 and the load W1, and generates the projection image I4. The position of the projection image camera 7 is merely an example and is not limited thereto. The projected image I4 is output to the correction coefficient output unit 82.

The correction coefficient output unit 82 is based on the brightness of each color component of each pixel of the second image I2 and the brightness of each color component of each pixel of the projected image I4, and the correction coefficient output unit 82 of each color component of each pixel of the second image I2. The correction coefficient for each luminance is output. The correction coefficient output unit 82 according to the present embodiment has a correction coefficient trained model. The correction coefficient of each luminance of each color component of each pixel of the output second image I2 is output to the correction processing unit 83.

In the correction coefficient trained model, when the brightness of each color component of each pixel of the second image I2 and the brightness of each color component of each pixel of the projected image I4 are input, each color of each pixel of the second image I2 is input. It is trained in advance to output the correction coefficient of each pixel of the component. The correction factor trained model may be trained, for example, by supervised learning by neural network and deep learning. In this case, for example, in the correction coefficient trained model, the correlation between the brightness of each color component of each pixel of the projected image I4 and the brightness of each color component of each pixel of the second image I2 is used as training data. You may be trained.

The correction processing unit 83 corrects each luminance of each color component of each pixel of the second image I2 based on the correction coefficient input from the correction coefficient output unit 82. The second image I2 in which the brightness of each color component of each pixel is corrected is output to the projector 6.

The projector 6 again projects the second image I2 corrected by the correction processing unit 83 onto the storage shelf Q, the pallet P1, and the load W1. The transmission image I3 displayed on the storage shelf Q, the pallet P1 and the load W1 is captured again by the projection image camera 7, and the projection image I4 is generated. Further, the correction coefficient output unit 82 outputs the correction coefficient corresponding to the projected image I4, and the correction processing unit 83 corrects each luminance of the second image I2 and outputs the correction coefficient to the projector 6. By repeating these steps, as shown in FIG. 6, a more suitable transmission image I3 is displayed on the storage shelf Q, the pallet P1, and the load W1. As a result, the driver D can visually confirm the first passage from the second passage, so that the safety of traveling can be visually confirmed even in the facility where the storage shelf Q is arranged.

Although the cargo handling system S according to the embodiment of the present invention has been described above, the cargo handling system S according to the present invention is not limited thereto. For example, the cargo handling system according to the present invention may be implemented by the following modifications.

The correction coefficient output unit 82 may acquire, for example, the light distribution characteristics of the projector 6 with respect to the storage shelf Q, the pallet P1 and the load W1, and output the correction coefficient using the light distribution characteristics. Specifically, the control unit 8 further includes a light distribution characteristic acquisition unit that acquires light distribution characteristics. The projector 6 projects a reference image consisting of one color onto the storage shelf Q, the pallet P1 and the load W1 before projecting the second image I2, and the projection image camera 7 captures the reference image and displays the projected image I4. Generate and output to the light distribution characteristic acquisition unit. The light distribution characteristic acquisition unit analyzes and normalizes each luminance of each color component of each pixel of the projected image I4, and indicates an orientation distribution showing the light distribution characteristic of each color component of the projector 6 with respect to the storage shelf Q, the palette P1 and the load W1. To create. The correction coefficient output unit 82 outputs the correction coefficient using the light distribution. As a result, the correction coefficient adapted to the mode of the storage shelf Q, the pallet P1 and the load W1 is output.

-The cargo handling vehicle C is not limited to the above forklift. For example, the cargo handling vehicle C may be a reach type forklift or a cargo handling vehicle equipped with an attachment such as a ram or a spreader.

The number of cameras included in the installed camera 1, the number of projectors included in the projector 6, and the number of cameras included in the projection image camera 7 are not particularly limited. The installed camera 1, the projector 6, and the projection image camera 7 may be imaged by a plurality of cameras or projected by a plurality of projectors, respectively.

The cargo handling system S further includes a server 100 capable of wired or wireless communication with the installed camera 1 and the cargo handling vehicle C, and the server 100 is either an image extraction unit 81, a correction coefficient output unit 82, or a correction processing unit 83. You may have all. As shown in FIG. 7, when the server 100 has all of the image extraction unit 81, the correction coefficient output unit 82, and the correction processing unit 83, the first image I1 generated by the installed camera 1 is the image extraction unit of the server 100. The second image I2 output to the 81 and extracted from the image extraction unit 81 of the server 100 is output to the projector 6 of the cargo handling vehicle C. Then, the projected image I4 generated by the projected image camera 7 is output to the correction coefficient output unit 82 of the server 100, and the second image I2 corrected by the correction processing unit 83 of the server 100 is transferred to the projector 6 of the cargo handling vehicle C. It is output. In this case, the cargo handling vehicle C does not have to include the image extraction unit 81, the correction coefficient output unit 82, and the correction processing unit 83.

C Cargo handling vehicle D Driver W1, W2 Load P1, P2 Pallet R1 1st road surface R2 2nd road surface I1 1st image I2 2nd image I3 Transmission image 1 Installation camera 2 Car body 20 Wheel 21 Car body 22 Pillar 23 Head guard 3 Driving Section 30 Seat 31 Operation section 4 Cargo handling device 40 Mast 42 Carriage 43 Backrest 44 Fork 45 Lift cylinder 6 Projector 7 Projection image camera 8 Control section 80 Vehicle position detection section 81 Image extraction section 82 Correction coefficient output section 83 Correction processing section

Claims (6)

An installation camera installed on a storage shelf, which images the first passage adjacent to the storage shelf and generates a first image.
The first image is a second image to be projected from the second aisle adjacent to the storage shelf and intersecting the first passage toward the storage shelf, the load and pallet stored in the storage shelf, or any of them. Image extraction unit to extract from
Cargo handling vehicle and
Provided on the cargo handling vehicle, the second image is projected from the second aisle toward the storage shelf, the load and / or the pallet, and a transmitted image is projected onto the storage shelf, the load and the pallet or the pallet thereof. A projector to display on either side,
A projection image camera provided on the cargo handling vehicle, which captures the displayed transmitted image and generates a projection image,
The correction coefficient of each luminance of each color component of each pixel of the second image is output based on each luminance of each color component of each pixel of the second image and each luminance of each color component of each pixel of the projection image. Correction coefficient output unit and
A correction processing unit that corrects each luminance of each color component of each pixel of the second image based on the correction coefficient is provided.
The projector is a cargo handling system characterized by projecting the second image corrected by the correction processing unit. When the brightness of each color component of each pixel of the second image and the brightness of each color component of each pixel of the projected image are input to the correction coefficient output unit, each color of each pixel of the second image is input. The cargo handling system according to claim 1, wherein the cargo handling system has a correction coefficient trained model trained to output a correction coefficient of a component. Further provided with a position detection unit for detecting the position of the cargo handling vehicle,
The cargo handling system according to claim 1 or 2, wherein the image extraction unit extracts the second image from the first image based on the position of the cargo handling vehicle. The position detection unit has an image extraction-learned model trained to output the second image when the first image and the position of the cargo handling vehicle are input.
The cargo handling system according to claim 3, wherein the cargo handling system is characterized by the above. Further equipped with a server capable of communicating with the installed camera and the cargo handling vehicle,
The cargo handling system according to any one of claims 1 to 4, wherein the server has any or all of the image extraction unit, the correction coefficient output unit, and the correction processing unit. The cargo handling system according to any one of claims 1 to 4, wherein the cargo handling vehicle has any or all of the image extraction unit, the correction coefficient output unit, and the correction processing unit.

Images