JP6917167B2 - Bird's-eye view image display device for construction machinery - Google Patents

Description

The present invention relates to a bird's-eye view image display device mounted on a construction machine.

In emergency disaster recovery work, when operating a remote-controlled construction machine remotely, the operator needs to operate it from a safe place sufficiently far from the construction place.
Although it is possible to operate visually in places where the construction area is narrow, usually, 1) a work camera mounted on a construction machine to check the progress of work, and 2) a fixed installation, construction machine. Images from two types of cameras, a confirmation camera for taking a picture of itself and confirming its movement status and surrounding conditions, are presented to the operator.
Here, when the construction machine moves, it is necessary to perform work such as switching the image of the confirmation camera in 2) following the movement, which increases the work process of the operator and may reduce the work efficiency. ..
Therefore, a bird's-eye view image display device has been proposed in which a plurality of cameras are provided in a construction machine, images supplied from each camera are processed, and a bird's-eye view image as seen from above the construction machine is created (Patent Document 1 below). reference). According to this method, since the confirmation camera is provided in the construction machine, it is not necessary to switch the confirmation camera when the construction machine moves. Further, since the images of a plurality of cameras are combined as one bird's-eye view image, the entire circumference of the construction machine can be displayed on one screen, and the efficiency of the confirmation work can be improved.
Further, Patent Document 2 below describes a technique for detecting an object that has moved from outside a specific area to within a specific area by using an image of a fixedly installed camera.

JP-A-2015-154240 JP-A-2015-96293

However, even when a bird's-eye view image is used, in order to proceed with the work while checking the surrounding conditions, it is necessary to check the image of the work camera and the image of the confirmation camera alternately or at the same time, which is a burden on the operator. Is big.
For example, while working on a construction machine, it is required to keep objects such as people, animals, and vehicles out of the work area, but while the construction machine is moving (including when turning), it is in the bird's-eye view image. Since all objects other than construction machinery appear to be moving, even if there are objects that are relatively close to the construction machinery, they may not be noticed. The above-mentioned Patent Document 2 is a technique using an image of a fixed camera, and cannot be applied when the image itself moves like a bird's-eye view image of a construction machine.
The present invention has been made in view of such circumstances, and an object of the present invention is to provide a bird's-eye view image display device for a construction machine, which is advantageous in reducing the burden on an operator during the work of the construction machine.

In order to achieve the above object, the bird's-eye view image display device for a construction machine according to the invention of claim 1 is a plurality of cameras mounted on the construction machine and a photograph of the surroundings of the construction machine taken by the plurality of cameras. A construction machine including a bird's-eye view image generation unit that generates a bird's-eye view image of the entire circumference of the construction machine centered on the construction machine, and a display unit that displays a bird's-eye view image generated by the bird's-eye view image generation unit. In the bird's-eye view image display device, the intersections of the grids set on the bird's-eye view image are set as representative points, and the movement of each of the representative points is based on the change in the position of the representative points between the frames of the bird's-eye view image. A movement vector detection unit that detects a vector and a caution object detection unit that detects an object for which the representative point is set, which has a movement vector different from the movement vector of the other representative points accompanying the movement of the construction machine, as a caution object. It is characterized by having and.
In the bird's-eye view image display device for a construction machine according to the invention of claim 2, the caution object detection unit has a movement vector whose movement direction is different from that of other representative points due to the movement of the construction machine. It is characterized in that an object for which a point is set is detected as the caution object.
In the bird's-eye view image display device for a construction machine according to the invention of claim 3, the caution object detection unit has a movement vector whose movement amount is different from that of other representative points due to the movement of the construction machine. It is characterized in that an object for which a point is set is detected as the caution object.
In the bird's-eye view image display device for construction machinery according to the invention of claim 4, the length of the movement vector is the amount of movement of the movement vector per unit time, and the caution object detection unit is the movement of the construction machine. When there is an object that moves in the same direction as the direction, the length of the movement vector of the other representative point due to the movement of the construction machine is different from the length of the movement vector of the representative point set for the object. It is characterized in that an object is detected as the caution object.
The bird's-eye view image display device for construction machinery according to the invention of claim 5 further includes a display control unit that makes the display mode of the caution object different from the display mode of another area when the caution object is detected. It is characterized by being prepared.
The bird's-eye view image display device for a construction machine according to the invention of claim 6 is further provided with a drive control unit that limits the drive of the construction machine when the caution object is detected.
The bird's-eye view image display device for a construction machine according to the invention of claim 7 is characterized in that the construction machine is operated unmanned by remote control.

According to the invention of claim 1, a moving object around the construction machine can be automatically detected, and the work efficiency is improved as compared with the case where the operator of the construction machine confirms the image in parallel with the work. It is advantageous to make it. In addition, since an object showing a movement vector different from the movement vector due to the movement of the construction machine is detected as a caution object, the relative position change of the object due to the movement of the construction machine and the actual movement of the object are distinguished from each other and the caution object. Can be detected, which is advantageous in accurately detecting an object that should be truly noted.
According to the second aspect of the present invention, since an object showing a movement vector whose movement direction is different from the movement vector accompanying the movement of the construction machine is detected as a caution object, the caution object can be detected by a simple process.
According to the invention of claim 3, since an object showing a movement vector whose movement amount is different from the movement vector accompanying the movement of the construction machine is detected as a caution object, the movement direction of the caution object and the movement direction of the construction machine are the same. Attention objects can be detected even when the direction is correct.
According to the invention of claim 4, the existence and the position of the caution object can be visually shown to the operator of the construction machine, which is advantageous in improving the convenience when operating the construction machine.
According to the invention of claim 5, when a caution object is detected, the drive of the construction machine is automatically restricted, so that the moving object can be quickly dealt with without waiting for the judgment of the operator of the construction machine. It becomes advantageous in.
According to the invention of claim 6, the caution object around the construction machine can be grasped even by the operator who is away from the construction machine, and the convenience in unmanned operation of the construction machine by remote control is improved. It becomes advantageous in.

It is a side view of the backhoe to which the bird's-eye view image display device for construction machinery of the embodiment is applied. It is a top view of a backhoe. It is a block diagram which shows the structure of the remote control system which makes an unmanned operation of a backhoe by remote control. It is a block diagram which shows the structure of a plurality of cameras and a control part of a construction machine side unit. It is a functional block diagram of the control part of the construction machine side unit. It is a figure which shows an example of the bird's-eye view image. It is a figure which shows typically the detection method of a caution object. It is a figure which shows typically the detection method of a caution object. It is a figure which shows typically the detection method of a caution object. It is a figure which shows the display example of the bird's-eye view image. It is an operation flowchart of the bird's-eye view image display device for construction machinery.

(Embodiment)
Hereinafter, the present embodiment will be described with reference to the drawings.
First, a construction machine equipped with a bird's-eye view image display device for a construction machine according to the present invention will be described.
In this embodiment, a case where a construction machine used in a dangerous construction site such as an emergency disaster recovery work is a backhoe that is operated unmanned by remote control will be described.
In this case, the operator remotely controls the backhoe from a safe place sufficiently distant from the construction place.

First, the configuration of the backhoe will be described.
As shown in FIGS. 1 and 2, the backhoe 10 includes a lower traveling body 12, an upper swinging body 14, a boom 16, an arm 18, and a bucket 20.

The lower traveling body 12 travels on the ground G by the rotation of the crawler 1202.
The upper swivel body 14 is provided on the upper part of the lower traveling body 12 so as to be horizontally swivel around a swivel shaft.
The operation chamber 1402 is provided in the upper swing body 14, and the lower traveling body 12 travels, the upper swing body 14 swings, the boom 16 swings, the arm 18 swings, and the bucket 20 swings in the operation chamber 1402. A plurality of operating devices such as operating levers and operating pedals (not shown) are installed for operating the above.

The boom 16 is swingably supported by the upper swing body 14 via a support shaft whose base end extends in the horizontal direction.
The arm 18 is swingably supported by the tip of the boom 16 via a support shaft whose base end extends in the horizontal direction.
The bucket 20 is swingably supported by the tip of the arm 18 via a support shaft whose base end extends in the horizontal direction.
A boom cylinder 1602 for swinging the boom 16 is provided between the upper swing body 14 and the boom 16.
An arm cylinder 1802 for swinging the arm 18 is provided between the boom 16 and the arm 18.
A bucket cylinder 2002 that swings the bucket 20 is provided between the arm 18 and the bucket 20.
The boom cylinder 1602, arm cylinder 1802, and bucket cylinder 2002 are hydraulic cylinders.
Therefore, as the boom cylinder 1602 expands and contracts, the boom 16 swings with respect to the upper swing body 14.
Further, the arm 18 is swung with respect to the boom 16 by expanding and contracting the arm cylinder 1802.
Further, as the bucket cylinder 2002 expands and contracts, the bucket 20 swings with respect to the arm 18.

Next, a remote control system for remotely controlling the backhoe 10 will be described.
FIG. 3 is a block diagram showing a configuration of a remote control system.
The remote control system 30 includes a remote control unit 32 and a construction machine side unit 34.
The remote control unit 32 is provided at a location away from the work site where the backhoe 10 works.
The remote control unit 32 includes an operation unit 36, a display unit 38, a control unit 40, and a communication unit 42.
The operation unit 36 is operated by an operator to remotely control a construction machine, and is composed of, for example, a joystick or an operation switch.
The display unit 38 displays a bird's-eye view image as shown in FIG. 6 transmitted from the construction machine side unit 34, and is composed of a display such as a liquid crystal display device.
In the present embodiment, the display unit 38 constitutes the display means.
The control unit 40 generates an operation command corresponding to the operation performed by the operation unit 36 and supplies the operation command to the communication unit 42, and receives the bird's-eye view image transmitted from the construction machine side unit 34 via the communication unit 42 and displays the display unit 38. Is to be supplied to.
The communication unit 42 communicates an operation command and a bird's-eye view image with the communication unit 50 of the construction machine side unit 34.
These two communication units 42 and 50 are connected by a wireless line.
As the wireless line, various conventionally known wireless lines such as SS wireless can be used, and the type, configuration, and method of the wireless line are arbitrary. Further, a plurality of types of wireless lines may be combined, or one or more relay stations may be provided.

The construction machine side unit 34 is provided on the backhoe 10, and includes a drive unit 44, a plurality of cameras 46, a control unit 48, and a communication unit 50.
The drive unit 44 drives each of the operating devices of the backhoe 10, and various conventionally known actuators such as an air cylinder and a motor can be used.
The plurality of cameras 46 are composed of a front camera 4602, a rear camera 4604, a left camera 4606, and a right camera 4608.

As shown in FIGS. 1 and 2, the front camera 4602 is mounted on the front portion of the upper swing body 14 and images the front of the backhoe 10.
The rear camera 4604 is mounted on the rear portion of the upper swing body 14 and images the rear of the backhoe 10.
The left camera 4606 is mounted on the left side of the upper swing body 14 and images the left side of the backhoe 10.
The right camera 4608 is mounted on the right side of the upper swing body 14 and images the right side of the backhoe 10.
In this embodiment, the angle of view of each camera 46 is 180 degrees.
The angle of view of each camera 46 may be less than 180 degrees as long as a bird's-eye view image described later can be generated. However, by setting the angle of view of each camera 46 to 180 degrees and installing the cameras 46 on the front, back, left and right sides of the upper swivel body 14 as described above, an image covering the entire circumference of the upper swivel body 14 (backhoe 10) can be obtained. It can be obtained efficiently and without generating a blind spot, which is advantageous in accurately generating a bird's-eye view image as shown in FIG. In the bird's-eye view image of FIG. 6, in addition to the backhoe 10 viewed from above, a pole S1 and a color cone (registered trademark) S2 around the backhoe 10 are shown.
Further, each camera 46 is attached to the upper swivel body 14 via a pan head (not shown) so that the optical axis of each camera 46 can be adjusted around the vertical axis and the horizontal axis.

As shown in FIG. 4, the control unit 48 is configured by a computer.
The computer has a CPU 4802, a ROM 4804, a RAM 4806, a hard disk device 4808, a keyboard 4810, a mouse 4812, a display 4814, an input / output interface 4816, etc., which are connected via an interface circuit (not shown) and a bus line 4801.
The ROM 4804 stores a predetermined control program and the like, and the RAM 4806 provides a working area.
The hard disk device 4808 stores a program for realizing a bird's-eye view image generation unit 52, a movement vector detection unit 54, and a caution object detection unit 56, which will be described later.
The keyboard 4810 and the mouse 4812 receive operation input by the operator.
The display 4814 displays an image, and is composed of, for example, a liquid crystal display device.
The input / output interface 4816 exchanges data with and from peripheral devices, and in the present embodiment, receives captured images from each camera 46.
The control unit 48 receives the operation command transmitted from the remote control unit 32 via the communication unit 50, and operates the operation device of the backhoe 10 by controlling the drive unit 44 based on the operation command. It is a thing.
Further, as shown in FIG. 5, the control unit 48 has a bird's-eye view image generation unit 52, a movement vector detection unit 54, a caution object detection unit 56, a display control unit 58, and a drive control unit when the CPU 4802 executes the program. It realizes 60.

The bird's-eye view image generation unit 52 generates a bird's-eye view image (FIG. 6) of the entire circumference of the backhoe 10 centered on the backhoe 10 by using the captured images around the backhoe 10 taken by each camera 46. The bird's-eye view image generated by the bird's-eye view image generation unit 52 is transmitted to the communication unit 42 of the remote control unit 32 via the communication unit 50. When the remote control unit 32 can switch between the bird's-eye view image and the image of each camera 46 for display, the bird's-eye view image and the captured image of each camera 46 may be transmitted to the remote control unit 32.
The bird's-eye view image generation unit 52 generates the bird's-eye view image by combining the viewpoint conversion images obtained by performing the viewpoint conversion processing on the captured images obtained from each camera 46. Such viewpoint conversion processing and viewpoint conversion images As the processing of the combination of, various conventionally known methods can be adopted.

The movement vector detection unit 54 detects the movement vector in the bird's-eye view image of the object reflected in the bird's-eye view image.
In the present embodiment, the movement vector detection unit 54 detects the movement of an object in the bird's-eye view image by using Optical flow, which is a function of OpenCV provided as an image processing library, and moves the movement. The direction and movement speed (movement amount per unit time) are expressed as a movement vector.
In Optical flow, a representative point is set in the image (video) and the movement of the representative point is calculated. In the present embodiment, as shown in FIG. 7, a grid is set on the bird's-eye view image, and the intersection of the grids is set as a representative point. That is, a "dense representative point" shall be set. In FIG. 7, the intersections of the vertical lines L1 and the horizontal lines L2 constituting the grid are the representative points P, respectively. In FIG. 7, the interval between the representative points P is increased to ensure visibility, but in reality, for example, the representative points P have a density that can be immediately detected when a human being is reflected on the bird's-eye view screen. Is set.
It is also possible to set a "sparse representative point" instead of a dense representative point. A sparse representative point is one that extracts feature points such as corners and inflection points of an object in an image and uses those points as representative points.

The movement vector detection unit 54 compares the position of the representative point P on a certain frame (image at a predetermined time t0) with the position of the representative point P on the next frame (image at a predetermined time t0 + Δt) and compares the position of the representative point P with the movement vector V1. Is detected.
For example, when the backhoe 10 moves forward, the camera 46 mounted on the backhoe 10 also moves forward, so that the object (representative point) in the image appears to move relatively backward. The movement of such an object is detected as a movement vector V1 in the direction opposite to the movement direction V0 of the backhoe 10 as shown in FIG.
It should be noted that, for example, a representative point set at a location overlapping the backhoe 10 itself, such as the representative point PX in FIG. 8, is considered to have the same movement vector as a representative point set at a location not overlapping the backhoe 10.

The bird's-eye view image transmitted to the remote control unit 32 may or may not include the plot of the representative point P, but in either case, the movement vector (steady state) detected with the movement of the backhoe 10. Movement vector) V1 is not displayed. This is to ensure visibility on the bird's-eye view screen and to clarify the difference from the display mode when the attention object is detected by the attention object detection unit 56 described later.

The attention object detection unit 56 detects as a caution object an object showing a movement vector different from the movement vector accompanying the movement of the construction machine.
FIG. 8 shows only the steady movement vector accompanying the movement of the backhoe 10, but for example, when there is an object approaching the moving backhoe 10, a movement vector different from other representative points is detected. .. For example, when there is an object T approaching the backhoe 10 as shown in FIG. 9, the movement vector V2 of the representative point PY set on the object T is different from the movement vector (steady movement vector) V1 of the other representative points P. It becomes.
In this case, the caution object detection unit 56 detects that there is a caution object in the vicinity of the representative point PY.

For example, the following two methods can be considered as a method of discriminating between the steady-state movement vector (V1 in FIG. 9) and the movement vector (V2 in FIG. 9) accompanying the movement of the construction machine.
1) Discrimination by vector direction The movement vector accompanying the movement of the construction machine faces a certain direction unless the construction machine suddenly changes direction. On the other hand, when a moving object enters the bird's-eye view image, the movement vector of the representative point corresponding to the object faces a direction different from the movement vector of the other representative points in the bird's-eye view image.
Therefore, the caution object detection unit 56 detects as a caution object an object showing a movement vector whose movement direction is different from the movement vector accompanying the movement of the construction machine.

2) Discrimination by vector length For example, if there is an object that moves in the same direction as the moving direction of the construction machine, the caution object cannot be detected by the discrimination in the vector direction. Therefore, for example, the length of the vector (movement amount per unit time) corresponding to the movement speed (predetermined speed band) of the construction machine may be set, and the movement vector outside the length may be detected. ..
That is, the caution object detection unit 56 detects as a caution object an object showing a movement vector whose movement amount (vector length) is different from that of the movement vector accompanying the movement of the construction machine.
Note that this method cannot detect an object that moves in the same direction as the construction machine at approximately the same speed, but an object that moves in the same direction as the construction machine at approximately the same speed maintains a certain distance from the construction machine. Since it will move, it is unlikely that any inconvenience will occur due to approaching construction machinery.
Further, in the above explanation, the case where the moving direction of the construction machine and the moving direction of the object are the same has been described, but it is of course possible to detect the caution object by the length (moving speed) of the vector regardless of the direction of the vector. ..

When the caution object is detected by the caution object detection unit 56, an alert is generated to notify the operator to that effect. Various conventionally known methods can be used for the alert to be generated. For example, the display control unit 58 makes the display mode of the attention object in the display unit 38 of the remote control unit 32 different from the display mode of other areas. , The position of the caution object can be notified to the operator.
For example, as shown in FIG. 9 described above, when a movement vector V2 different from the other movement vectors V1 is detected at the representative point PY, the display control unit 58 detects the contour of the object T near the representative point PY by image processing. .. Then, image processing is performed so that the inside of the contour is highlighted (for example, displayed in red or yellow, blinking, etc.) on the display unit 38 of the remote control unit 32. Further, if the movement vector V2 of the object T is displayed on the display unit 38 at this time, the operator can grasp the moving direction of the object T, and the convenience is improved.
In addition to changing the display on the display unit 38, voice notification (for example, output of an alert sound or output of a message such as "an object has entered the work area") may be performed.

Further, when the caution object is detected by the caution object detection unit 56, the drive control unit 60 may limit the drive of the construction machine.
The restrictions on the drive of the construction machine include, for example, stopping the drive of the construction machine and reducing the moving speed. In this way, by automatically limiting the drive of the construction machine when a caution object is present in the bird's-eye view image, it is possible to quickly respond to the moving object without waiting for the operator's judgment.

FIG. 11 is a flowchart showing the processing of the bird's-eye view image display device for construction machinery.
In FIG. 11, when a caution object is detected in the bird's-eye view image, the caution object in the bird's-eye view image is highlighted on the display unit 38 of the remote control unit 32.
First, an image around the backhoe 10 is taken by a plurality of cameras 46 (step S1100). The bird's-eye view image generation unit 52 generates a bird's-eye view image centered on the backhoe 10 by using a plurality of images taken by the plurality of cameras 46 (step S1102).
The movement vector detection unit 54 detects the movement vector V of each representative point based on the change in the position of the representative point P between the frames of the bird's-eye view image (step S1104).
The attention object detection unit 56 determines whether or not there is a movement vector different from the movement vector (steady movement vector) accompanying the movement of the backhoe 10 in the bird's-eye view image, that is, whether or not there is a caution object (step S1106). ..
If there is no movement vector different from the steady movement vector (step S1106: No), the bird's-eye view image is transmitted to the remote control unit 32 side as it is (step S1110). The remote control unit 32 displays the received bird's-eye view image on the display unit 38 (step S1112). The operator operates the backhoe 10 while referring to the bird's-eye view image displayed on the display unit 38.
On the other hand, if there is a movement vector different from the steady movement vector (step S1106: Yes), the display control unit 58 extracts the contour of the attention object from the bird's-eye view image and performs image processing so as to highlight the inside of the contour (step S1118). ). Then, the image-processed bird's-eye view image is transmitted to the remote control unit 32 side (step S1110). The remote control unit 32 displays the image-processed bird's-eye view image on the display unit 38 (step S1112). In this case as well, the operator operates the backhoe 10 while referring to the bird's-eye view image displayed on the display unit 38. However, since a caution object is displayed in the bird's-eye view image, for example, the backhoe 10 is temporarily stopped. It will be possible to take measures such as confirming safety.

As described above, according to the bird's-eye view image display device for construction machinery according to the embodiment, moving objects around the backhoe 10 can be automatically detected, and the operator of the backhoe 10 can perform an image in parallel with the work. It is advantageous in improving work efficiency as compared with the case of confirming.
In addition, since an object showing a movement vector different from the steady movement vector due to the movement of the backhoe 10 is detected as a caution object, attention should be paid to distinguish between the relative position change of the object due to the movement of the backhoe 10 and the actual movement of the object. It is possible to detect an object, which is advantageous in accurately detecting an object that should be truly noted.
Further, in the bird's-eye view image display device for construction machinery according to the embodiment, if an object showing a movement vector whose movement direction is different from the steady movement vector accompanying the movement of the backhoe 10 is detected as a caution object, the caution object can be easily processed. Can be detected.
Further, in the bird's-eye view image display device for construction machinery according to the embodiment, if an object showing a movement vector whose movement amount is different from the steady movement vector accompanying the movement of the backhoe 10 is detected as a caution object, the movement direction of the caution object can be obtained. Attention objects can be detected even when the backhoe 10 is moving in the same direction.
Further, in the bird's-eye view image display device for construction machinery according to the embodiment, if the operator of the backhoe 10 and the like are made to visually indicate the existence and the position of the object of interest, the convenience at the time of operating the backhoe 10 is improved. It will be advantageous on.
Further, in the bird's-eye view image display device for construction machinery according to the embodiment, if the drive of the backhoe 10 is automatically restricted when a caution object is detected, the backhoe 10 can be moved quickly without waiting for the operator's judgment. It is advantageous in dealing with objects.
Further, according to the bird's-eye view image display device for construction machinery according to the embodiment, the attention object around the backhoe 10 can be grasped even by an operator who is away from the backhoe 10, and the construction machine is unmanned by remote control. It is advantageous in improving convenience during driving.

In the present embodiment, the control unit 48 of the construction machine side unit 34 realizes the bird's-eye view image generation unit 52, the movement vector detection unit 54, the caution object detection unit 56, the display control unit 58, and the drive control unit 60. explained.
However, the images captured by each camera 46 are transmitted to the remote control unit 32, and the control unit 40 of the remote control unit 32 transmits the bird's-eye view image generation unit 52, the movement vector detection unit 54, the caution object detection unit 56, and the display control unit 58. , The drive control unit 60 may be realized.

Further, in the present embodiment, the case where four cameras 46 are provided in the construction machine side unit 34 has been described, but the number of cameras 46 is not limited as long as a bird's-eye view image can be generated.

Further, in the present embodiment, the case where the backhoe 10 can be remotely controlled has been described, but the backhoe 10 may be directly operated by a worker. In this case, the display means may be provided so that the operator who operates the backhoe 10 can see it. For example, the display 4814 of the control unit 48 may be used as the display means.
Further, the construction machine is not limited to the backhoe 10, and may be various conventionally known construction machines such as a mobile crane and a bulldozer.

10 Backhoe (construction machinery)
30 Remote control system 32 Remote control unit 34 Construction machine side unit 36 Operation unit 38 Display unit 40 Control unit 42 Communication unit 44 Drive unit 46 Camera 4602 Front camera 4604 Rear camera 4606 Left camera 4608 Right camera 48 Control unit 50 Communication unit 52 Bird's-eye view image generation unit 54 Movement vector detection unit 56 Attention object detection unit 58 Display control unit 60 Drive control unit P, PX, PY Representative point T Object

Claims (7)

With multiple cameras mounted on construction machinery,
A bird's-eye view image generation unit that generates a bird's-eye view image of the entire circumference of the construction machine centered on the construction machine by using images taken around the construction machine taken by the plurality of cameras.
A bird's-eye view image display device for construction machinery including a display unit for displaying a bird's-eye view image generated by the bird's-eye view image generation unit.
A movement vector detection unit that sets the intersections of the grids set on the bird's-eye view image as representative points and detects the movement vector of each of the representative points based on the change in the position of the representative points between the frames of the bird's-eye view image. ,
A caution object detection unit that detects an object for which the representative point is set, which has a movement vector different from the movement vector of the other representative points accompanying the movement of the construction machine, as a caution object.
A bird's-eye view image display device for construction machinery, which is characterized by being equipped with. The caution object detection unit detects an object for which the representative point is set, which has a movement vector whose movement direction is different from that of the movement vector of the other representative points accompanying the movement of the construction machine, as the caution object.
The bird's-eye view image display device for construction machinery according to claim 1. The caution object detection unit detects an object for which the representative point is set, which has a movement vector whose movement amount is different from the movement vector of the other representative points accompanying the movement of the construction machine, as the caution object.
The bird's-eye view image display device for construction machinery according to claim 1 or 2. The length of the movement vector is the amount of movement of the movement vector per unit time.
When there is an object that moves in the same direction as the moving direction of the construction machine, the caution object detecting unit has the length of the movement vector of the other representative point accompanying the movement of the construction machine and the representative set for the object. When the length of the movement vector of the point is different, the object is detected as the caution object.
The bird's-eye view image display device for construction machinery according to claim 3. When the caution object is detected, the display control unit further includes a display control unit that makes the display mode of the caution object in the display unit different from the display mode of other regions.
The bird's-eye view image display device for construction machinery according to any one of claims 1 to 4, characterized in that. When the caution object is detected, a drive control unit that limits the drive of the construction machine is further provided.
The bird's-eye view image display device for construction machinery according to any one of claims 1 to 5, characterized in that. The construction machine is operated unmanned by remote control.
The bird's-eye view image display device for construction machinery according to any one of claims 1 to 6, characterized in that.

Images