JP6401141B2 - Vehicle obstacle detection device - Google Patents

Description

  The present invention relates to a vehicle surrounding obstacle detection device including a monitor that displays the surroundings of a transport vehicle, a work vehicle, and the like.

  As a transport vehicle, a large dump truck is known that transports a transported object such as crushed stone at a work site such as a mine. In such a large dump truck, it is difficult for an operator to recognize the periphery of the vehicle body from the size of the vehicle body. In view of this, there has been known one in which a plurality of cameras are mounted on a vehicle body, and images taken by the cameras are combined to display an image around the vehicle body (a bird's-eye view image) on a cab monitor (for example, Patent Documents). 1).

International Publication No. 2012/169352

  Incidentally, the dump truck described in Patent Document 1 detects a plurality of obstacles located around the vehicle body and displays them on a monitor. However, in general, as the obstacle becomes farther from the vehicle body, it becomes difficult to accurately detect the obstacle, and the frequency of false detection increases. The range in which an obstacle can be detected varies depending on the size, color, and the like of the obstacle. That is, for example, a large obstacle such as an automobile can detect a relatively far distance from the vehicle body (own vehicle). However, a small obstacle such as a human can be detected from the vehicle body (own vehicle) compared to a large obstacle. It can only detect up to a short distance. Therefore, the synthesized video displayed on the monitor has a problem that the range in which the obstacle detection accuracy is high and the range in which the detection accuracy is low are mixed, resulting in confusion for the operator.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a vehicle surrounding obstacle detection device that can recognize an area where an operator can detect an obstacle on a monitor. It is to provide.

In order to solve the above-described problems, the present invention provides a plurality of cameras that are provided on a vehicle body and that captures the periphery of the vehicle body, a controller that processes images captured by the plurality of cameras, and an image processed by the controller. In the vehicle surrounding obstacle detection device comprising a monitor for display, the controller combines the images captured by the plurality of cameras into one image, and the vehicle body is located at the center of the combined image. An image composition processing unit that creates a composite image that can recognize the periphery of the vehicle body by arranging a symbol, an obstacle detection unit that detects an obstacle around the vehicle body, and the vehicle image displayed on the composite image displayed on the monitor An inner area that is set in a range close to the vehicle body around the vehicle body and that can detect an obstacle of a predetermined size, and that is located on the outer peripheral side of the inner area and in a range away from the vehicle body Is provided with an area display unit that displays to identify an outer area, the obstacle detecting unit, when the obstacle is located in the inner area, displays the obstacle marker to the obstacle And when the obstacle is located in the outer area, a marker display processing unit that hides the obstacle marker of the obstacle is provided, and the area display processing unit defines the inner area as the vehicle body. An area section processing section that partitions the first area close to the second area and the second area far from the vehicle body, and the marker display processing section changes the display mode of the obstacle marker in the first area and the second area. A marker display control unit to be changed is provided .

  According to the present invention, it is possible to recognize an area where an operator can detect an obstacle from an image displayed on a monitor.

1 is a front view showing a dump truck on which a vehicle surrounding obstacle detection device according to a first embodiment of the present invention is mounted. It is a perspective view which shows the inside of the cab of the dump truck in FIG. It is a control block diagram of a vehicle surrounding obstacle detection device. FIG. 4 is a control block diagram of the controller in FIG. 3. It is a front view which shows the bird's-eye view image displayed on the monitor in FIG. It is a front view which shows the direct image | video of the right camera displayed on the monitor. It is a front view which shows the bird's-eye view image when an obstacle cannot be detected. It is a control block diagram of the controller by the 2nd Embodiment of this invention. It is a front view which shows the bird's-eye view image displayed on the monitor in FIG. It is a control block diagram of a controller according to a third embodiment of the present invention. It is a front view which shows the bird's-eye view image displayed on the monitor in FIG. It is a front view which shows the bird's-eye view image by the 1st modification of this invention. It is a front view which shows the bird's-eye view image by the 2nd modification of this invention. It is a control block diagram of the controller by the 3rd modification of this invention.

  Hereinafter, a dump truck equipped with a vehicle surrounding obstacle detection device according to an embodiment of the present invention will be described as an example with reference to the accompanying drawings.

  1 to 7 show a first embodiment of the present invention. A dump truck 1 shown in FIG. 1 is a large dump truck used for transporting, for example, minerals mined in a mine. The dump truck 1 includes a vehicle body 2 having left and right front wheels 3 and left and right rear wheels 4, and a loading platform 16 provided at the upper rear portion of the vehicle body 2 so as to be rotatable (tiltable) in the vertical direction. Has been. The vehicle body 2 includes a frame 5, a building 7, a deck 8, a cab 12, and the like.

  The frame 5 serves as a base of the vehicle body 2 and is formed as a strong support structure. The frame 5 includes a base frame 6 that extends in the front-rear direction and an upper horizontal beam (not shown) that is disposed in the middle part of the base frame 6 in the front-rear direction. An engine (not shown) is arranged at the center of the base frame 6. On the front end side of the base frame 6, there is provided a building 7 in which a heat exchange device (not shown) such as a radiator is accommodated. On the other hand, a pair of left and right cargo bed mounting portions 6A (only the left side is shown) are provided on the rear end side of the base frame 6 so as to project rearward. In addition, a rear camera 22B described later is provided between the left and right cargo bed mounting portions 6A on the rear end side of the base frame 6.

  The deck 8 is supported from below by the frame 5 and the building 7, and thus is disposed at a position above the left and right front wheels 3. The deck 8 forms a flat passage surface on the building 7. On the deck 8, a control cabinet 9, a grid box (not shown), a cab 12, and the like are installed.

  In the control cabinet 9, an inverter for controlling the motor and a control panel for controlling the current generated by a generator (not shown) provided in the engine and the engine speed are accommodated. The grid box also contains a device comprising a resistor that consumes DC power when the vehicle is decelerated and a blower that cools the resistor.

  A handrail 10 is provided on the outer periphery of the deck 8 to ensure safety when performing maintenance work or the like. In addition, on the front side of the building 7, a step 11 for getting on and off formed in a staircase shape is provided. This step 11 for getting on and off constitutes a scaffold when an operator or the like gets on and off the deck 8. Further, a front camera 22F described later is provided on the front end side of the deck 8, and a right camera 22R and a left camera 22L are provided on the left and right ends of the deck 8.

  The cab 12 defines a cab in which an operator gets inside, and is disposed on the front side of the loading platform 16. As shown in FIG. 2, a steering wheel 13, a panel 14 for displaying instruments and the like, a driver seat (not shown), an accelerator pedal, a brake pedal, an operation lever, and the like are provided in the cab 12. A monitor 48 described later is attached to the left front column 15 of the cab 12.

  The loading platform 16 is mounted on the vehicle body 2 so as to be tiltable. The loading platform 16 receives a large amount of minerals mined by, for example, a hydraulic excavator (not shown). The loading platform 16 is formed as a large container having a total length of 10 to 15 meters in order to load a large amount of minerals and the like. The rear bottom portion of the loading platform 16 is rotatably connected to left and right loading platform mounting portions 6A provided on the rear end side of the base frame 6 via a connection pin 17 or the like. Also, on the front portion side of the loading platform 16, a flange portion 16 </ b> A that extends horizontally from the upper end portion thereof to the front is integrally provided, and this flange portion 16 </ b> A covers the cab 12 from the upper side so as to cover the front end of the deck 8. It extends to the vicinity of the part.

  The hoist cylinder 18 is for tilting the loading platform 16 up and down with respect to the vehicle body 2 (only one is shown in FIG. 1). The hoist cylinder 18 is composed of, for example, a multistage hydraulic cylinder, and is provided between the base frame 6 and the loading platform 16 so as to be extendable and contractible. Therefore, when the hoist cylinder 18 is extended or reduced, the front side of the loading platform 16 is rotated (lifted / lowered) in the vertical direction with the connecting pin 17 as a fulcrum. Thereby, the mineral etc. which were loaded on the loading platform 16 can be discharged.

  Next, the vehicle surrounding obstacle detection device 21 mounted on the dump truck 1 will be described.

  The vehicle surrounding obstacle detection device 21 displays the situation around the dump truck 1 on the monitor 48 by a plurality of cameras 22R, 22L, 22F, and 22B provided on the dump truck 1, and the operator This makes it easier to understand the surrounding situation. The vehicle surrounding obstacle detection device 21 includes a plurality of cameras 22R, 22L, 22F, and 22B, a controller 23, and a monitor 48.

  A plurality (four) of cameras 22R, 22L, 22F, and 22B are provided on the outer periphery of the dump truck 1, respectively. These four cameras 22R, 22L, 22F, and 22B are cameras capable of capturing a color image, for example, and image the four sides (surroundings) of the dump truck 1. Specifically, the right camera 22 </ b> R is provided on the right end side of the deck 8 and images the right side of the dump truck 1. On the other hand, the left camera 22L is provided on the left end side of the deck 8 and images the left side of the dump truck 1.

  The front camera 22F is provided on the front end side of the deck 8 and images the front of the dump truck 1. The rear camera 22B is provided on the rear end side of the base frame 6 and images the rear of the dump truck 1. That is, each camera 22R, 22L, 22F, 22B is disposed at a height of 2 to 5 meters from the ground. The camera through images 22R1, 22L1, 22F1, and 22B1 captured by the cameras 22R, 22L, 22F, and 22B are transmitted to the controller 23.

  The controller 23 processes the images captured by the cameras 22R, 22L, 22F, and 22B. The controller 23 is disposed, for example, in the cab 12, and each camera 22R, 22L, 22F, 22B is connected to the input side, and a monitor 48 described later is connected to the output side. The storage unit 24 of the controller 23 stores a plan view of the dump truck 1 as viewed from above as a symbol 25 of the vehicle body. Note that the right side of the vehicle body symbol 25 shown in FIG. The controller 23 includes a video processing unit 26, an area display processing unit 31, a direct video area display processing unit 37, and an obstacle detection unit 41.

  The video processing unit 26 inputs camera through videos 22R1, 22L1, 22F1, and 22B1 captured by the cameras 22R, 22L, 22F, and 22B. The video processing unit 26 includes an overhead video synthesis processing unit 27 and a direct video processing unit 29.

  The overhead view video composition processing unit 27 synthesizes the images captured by the four cameras 22R, 22L, 22F, and 22B and processes them into one image (overhead view video). The overhead video composition processing unit 27 constitutes a video composition processing unit of the present invention. The overhead view video composition processing unit 27 extracts, for example, a cutout video 22R2 (dotted line portion in FIG. 3) for processing from the right camera through video 22R1 into an overhead view video.

  Then, the overhead video composition processing unit 27 creates the right overhead video 28R of the dump truck 1 by rotating the cutout video 22R2 by 180 ° or expanding and contracting. Similarly, the other cameras 22L, 22F, and 22B process the cut-out video to create the left overhead video 28L, the front overhead video 28F, and the rear overhead video 28B. Note that the cut-out video may be cut out from the video that has been corrected by performing noise removal, distortion correction, color correction, and the like from the input camera-through video 22R1, 22L1, 22F1, and 22B1.

  Then, the bird's-eye view video composition processing unit 27 pastes the vehicle body symbol 25 on the center, pastes the right bird's-eye view image 28R on the right side of the vehicle body symbol 25, and pastes the left bird's-eye view image 28L on the left side of the vehicle body symbol 25. . Further, the bird's-eye view video composition processing unit 27 pastes the front bird's-eye view video 28F on the front side of the vehicle body symbol 25 and pastes the rear bird's-eye view video 28B on the rear side of the vehicle body symbol 25. Thereby, the overhead view video composition processing unit 27 can create the overhead view video 28. The created bird's-eye view image 28 is transmitted to the obstacle detection unit 41 periodically (with a predetermined period).

  The direct video processing unit 29 processes the direct video 30 (only a modified version of the right camera through video 22R1 is shown) captured by each camera 22R, 22L, 22F, 22B. The direct video processing unit 29 periodically (with a predetermined period) the obstacle detection unit 41 for the direct video 30 corrected by performing noise removal, distortion correction, and the like from the camera through videos 22R1, 22L1, 22F1, and 22B1. Send to. In the following description, the direct video 30 of the right camera 22R is described as a representative example for each direct video, but the same applies to the direct video of the other cameras 22L, 22F, and 22B.

  The area display processing unit 31 identifies and displays a preset inner area 32 for detecting an obstacle and an outer area 33 other than the inner area 32 in the overhead view video 28 displayed on the monitor 48. The area display processing unit 31 includes an area line display unit 34 that displays an area line 35 that distinguishes between the inner area 32 and the outer area 33. That is, the area line 35 indicates the position (boundary) farthest from the dump truck 1 in the area where the obstacle is detected.

  The inner area 32 is an area where the obstacle detection accuracy (detection accuracy) is high. On the other hand, the outer area 33 is an area where obstacles are not detected or erroneously detected and the obstacle detection accuracy is low. The inner area 32 and the outer area 33 are set in advance based on, for example, the probability that an obstacle can be detected correctly, and are set as appropriate according to actual detection results and specifications.

  As shown in FIG. 3 and FIG. 5, the area line 35 corresponds to the left and right and front and rear overhead view images 28R, 28L, 28F, and 28B included in the overhead view image 28, respectively. The front area line 35F and the rear area line 35B are included. The area line 35 is stored in the vehicle area 31A of the area display processing unit 31, for example.

  Accordingly, the inner area 32 is a right inner area 32R that is an area between the right area line 35R and the dump truck 1 (vehicle symbol 25), and a left area that is an area between the left area line 35L and the dump truck 1. It is constituted by an inner area 32L, a front inner area 32F that is an area between the front area line 35F and the dump truck 1, and a rear inner area 32B that is an area between the rear area line 35B and the dump truck 1. Yes. The outer area 33 is configured as a right outer area 33R, a left outer area 33L, a front outer area 33F, and a rear outer area 33B, corresponding to the left and right and front and rear bird's-eye views 28R, 28L, 28F, and 28B, respectively. .

  Here, the setting position of the area line 35, that is, the setting of the inner area 32 for detecting an obstacle will be described.

  In the present embodiment, obstacle detection is performed based on the images captured by the cameras 22R, 22L, 22F, and 22B. That is, the obstacle detection unit 41 to be described later includes an overhead video 28 (previous bird's-eye video 28) captured one cycle before from the video processing unit 26 and a currently captured overhead video 28 (current overhead video 28). , The video (object) moving for a specified time is determined as a moving object. Similarly, for the direct video, the previous direct video captured from the video processing unit 26 and the current direct video are compared, and the video (object) moving for a specified time is determined as a moving object. For example, in the case of detecting a moving car, the car image is moving, but the background of the car does not move, so this car can be detected as a moving object.

  For the obstacle that is stationary, for example, the obstacle detection unit 41 stores in advance the color, shape, size, and the like of an object that can be an obstacle at the work site and the like, and the overhead image 28 captured from the video processing unit 26. It can be detected by determining whether or not an object (color, shape, size, etc.) stored in is present.

  An obstacle such as an automobile has a smaller number of pixels (number of pixels) constituting an image as the distance from the dump truck 1 (cameras 22R, 22L, 22F, and 22B) increases. When the number of pixels decreases, the difference in color and brightness from the background decreases, and an automobile or the like cannot be determined with respect to the background, and the undetected rate increases. In addition, there are few pixels in the image that are easily misidentified, such as rocks, puddles, and uneven color of the ground. Since the color and brightness change depending on the degree of sunlight, etc., if the number of pixels of the detection target (moving object) is reduced, it will be easier to falsely detect rocks, puddles, and uneven color on the ground as obstacles. There's a problem. Therefore, the detection of an obstacle is judged by an image having a number of pixels equal to or greater than a predetermined value, and false detection is reduced.

  Also, the obstacle detection range from such an image varies depending on the performance of the cameras 22R, 22L, 22F, and 22B and the surrounding environment (illuminance, fog, rain, etc.). Therefore, the obstacle detection range is a value (distance from the dump truck 1) obtained experimentally in consideration of the performance of the cameras 22R, 22L, 22F, and 22B, the surrounding environment, and the like. As an example, when the weather is rainy, the detection distance is shorter than that on a clear day. Therefore, the position of the area line 35 is set to a shorter distance than the position where an obstacle can be detected on a clear day.

  That is, the position of the area line 35 is a boundary where an image having a number of pixels set in consideration of the size of the obstacle, the surrounding environment, and the like can be detected. For example, in the case of moving object detection, the image changes with the number of pixels equal to or greater than a predetermined value in the inner area 32, whereas the image changes with the number of pixels less than the predetermined value in the outer area 33. It will be. Similarly, in the case of stationary object detection, an object that becomes an obstacle in the inner area 32 is imaged with a pixel number equal to or greater than a predetermined value, whereas an object that becomes an obstacle in the outer area 33 It will be imaged with the number of pixels less than the set value. Then, the area display processing unit 31 displays the area line 35 stored in the vehicle area 31 </ b> A on the monitor 48 described later in accordance with the scale of the overhead view video 28.

  The area line display control unit 36 changes the display mode of the area line 35 when the obstacle detection unit 41 described later cannot detect the obstacle. As described above, when an obstacle that is a moving object is detected, the obstacle is determined by comparing the previous bird's-eye video 28 captured from the video processing unit 26 with the current bird's-eye video 28. In this case, even when the dump truck 1 is traveling, for example, the previous overhead image 28 captured from the video processing unit 26 and the current overhead image 28 are compared to extract an obstacle, thereby detecting an obstacle. Can be detected. However, for example, when the dump truck 1 is traveling at a predetermined speed or more, there are many errors between the previous bird's-eye view image 28 and the current bird's-eye view image 28 captured from the image processing unit 26, so that an erroneous detection of an obstacle is detected. The frequency may increase and it may be difficult to detect an obstacle.

  Therefore, as shown in FIG. 7, the area line display control unit 36 sets the area line 35 as a non-detectable area line 35 </ b> A by shortening the area line 35 to the vehicle body symbol 25 when the obstacle detection unit 41 cannot detect the obstacle. indicate. Thereby, the operator can recognize that the obstacle detection unit 41 is in a state where the obstacle cannot be detected. Note that when the vehicle speed sensor (not shown) provided in the vehicle body 2 detects a predetermined speed, the obstacle detection unit 41 may determine that the obstacle cannot be detected.

  The direct video area display processing unit 37 serves as a preset inner area that can detect an obstacle in the direct video 30 (only the image captured by the right camera 22R is illustrated) captured by each camera 22R, 22L, 22F, 22B. The direct video inner area 38 and the direct video outer area 39 as an outer area other than the direct video inner area 38 are identified and displayed. The direct video area display processing unit 37 identifies the direct video inner area 38 and the direct video outer area 39 by the direct video area line 40 (see FIG. 6).

  In this case, since the direct video 30 is captured by the right camera 22R, the direct video area line 40 corresponds to the right area line 35R displayed on the overhead video 28. Further, the direct video area display processing unit 37 is also used for direct video corresponding to the area lines 35L, 35F, and 35B displayed on the overhead view video 28, respectively, on the direct video captured by the other cameras 22L, 22F, and 22B. Display area lines.

  The obstacle detection unit 41 detects an obstacle around the vehicle body. The obstacle detection unit 41 detects an obstacle based on images periodically input from the overhead view video composition processing unit 27 and the direct video processing unit 29. Specifically, the obstacle detection unit 41 periodically captures videos from the overhead video synthesis processing unit 27 and the direct video processing unit 29 and stores them in the storage unit 42.

  The obstacle detection unit 41 compares the current video stored in the storage unit 42 with the previous video to determine a moving video and a non-moving video in the video, and determines the moving video as a moving object. It can be detected as an obstacle. In addition, the obstacle detection unit 41 extracts a video having a predetermined color, shape, size, and the like from the bird's-eye view video 28 and the direct video 30 stored in the storage unit 42, and detects a stationary obstacle. Can be detected.

  The obstacle detection unit 41 includes a marker display processing unit 43. When the obstacle is located in the inner area 32 set by the area display processing unit 31, the marker display processing unit 43 displays the obstacle marker 45 on the obstacle, and the obstacle is displayed in the area display processing unit. When located in the outer area 33 set by 31, the obstacle marker 45 is erased from the obstacle.

  As shown in FIGS. 3 and 5, the obstacle detection unit 41 indicates that the automobile 44 is running (moving object detection) from the previous bird's-eye view video and the current bird's-eye view video stored in the storage unit 42, for example. recognize. Then, the marker display processing unit 43 of the obstacle detection unit 41 displays the obstacle marker 45 in which the periphery of the automobile 44 is highlighted in a frame shape when the automobile 44 is located in the right inner area 32R. . On the other hand, the marker display processing unit 43 of the obstacle detection unit 41 has an obstacle because the automobile 46 is located in the right outside area 33R even if the automobile 46 located in the right outside area 33R is detected as a moving object. The object marker 45 is not displayed.

  Whether or not the obstacle marker 45 is displayed has been described as an example of detecting a moving obstacle (cars 44 and 46), but the same applies to the case of detecting a stationary obstacle. For this reason, the marker display processing unit 43 displays the obstacle marker 45 when the stationary obstacle is located in the inner area 32. On the other hand, the marker display processing unit 43 does not display the obstacle marker 45 when the stationary obstacle is located in the outer area 33. The obstacle marker 45 is not limited to a frame surrounding the obstacle, but may be any object that highlights the obstacle, for example, displaying a conspicuous circle at the center of the obstacle.

  By displaying the preset inner area 32 for detecting an obstacle by the area line 35 and the outer area 33 other than the inner area 32, the operator can recognize the range in which the obstacle can be detected. Thereby, the confusion of the operator can be reduced. Further, the operator can look for the obstacle marker 45 by paying attention to the inner area 32. Thereby, the recognition with respect to the obstruction which should be careful can be improved.

  Also, as shown in FIGS. 3 and 6, the obstacle detection unit 41 indicates that the automobile 44 is running from the previous direct image and the current direct image stored in the storage unit 42 (moving object detection). ) Then, the marker display processing unit 43 of the obstacle detection unit 41 has a direct image obstacle in which the periphery of the vehicle 44 is highlighted in a frame shape when the vehicle 44 is directly located in the inner area 38 for the image. The marker 47 is displayed. On the other hand, the marker display processing unit 43 of the obstacle detection unit 41 is located directly in the outer area 39 for video even if the moving object is detected in the vehicle 46 directly in the outer area 39 for video. Therefore, the obstacle marker 47 for video is not directly displayed. Whether or not to directly display the obstacle marker 47 for video has been described as an example of detecting a moving obstacle (cars 44 and 46), but the same applies to the case of detecting a stationary obstacle.

  Thereby, even in the direct image 30, the operator can recognize the range in which the obstacle can be detected, so that the confusion of the operator can be reduced. In addition, the operator can look directly at the inside area 38 for video and search for the obstacle marker 47 for direct video. Thereby, the recognition with respect to the obstruction which should be careful can be improved.

  The position of the area line 35 and the direct video area line 40 (distance from the dump truck 1) is set corresponding to the smallest obstacle to be detected in the work site, so that the operator can You can recognize all the obstacles you want to recognize.

  The monitor 48 displays the video processed by the controller 23. As shown in FIG. 2, the monitor 48 is attached to, for example, the left front support 15 in the cab 12 so that the operator can operate and check the monitor 48 while sitting on the driver's seat. The monitor 48 includes a power switch 49, a changeover switch 50, and a display unit 51.

  When the operator turns on the power switch 49, the monitor 48 is activated and the cameras 22R, 22L, 22F, 22B and the controller 23 are activated. Then, the overhead image 28 and / or the direct image 30 processed by the controller 23 is displayed on the display unit 51 of the monitor 48.

  The changeover switch 50 switches the video displayed on the display unit 51, and includes an up switch 50A that sequentially forwards each video and a down switch 50B that reversely forwards each video. Thereby, the operator can display a desired image on the display unit 51. Specifically, when the operator operates the changeover switch 50, the overhead image 28 shown in FIG. 5 can be switched to the direct image 30 of the right camera 22R shown in FIG. Further, when the operator operates the changeover switch 50, the direct video of the left camera 22L, the direct video of the front camera 22F, the direct video of the rear camera 22B, and the two-screen display of the overhead video 28 and each direct video can be sequentially switched. It has become.

  The vehicle surrounding obstacle detection device 21 mounted on the dump truck 1 according to the first embodiment has the above-described configuration. Next, obstacle detection by the vehicle surrounding obstacle detection device 21 will be described. The obstacle detection process is repeatedly executed at predetermined time intervals (in a predetermined control cycle) while the power switch 49 of the monitor 48 is turned on, for example.

  First, the obstacle detection process of the overhead image 28 will be described.

  When the power switch 49 of the monitor 48 is turned on, each camera 22R, 22L, 22F, 22B starts imaging. The captured images are sequentially sent to the image processing unit 26 of the controller 23. The overhead video composition processing unit 27 of the video processing unit 26 synthesizes the images captured by the four cameras 22R, 22L, 22F, and 22B and processes them into one overhead image 28. Then, the bird's-eye view video composition processing unit 27 transmits the created bird's-eye view image 28 to the obstacle detection unit 41 periodically (with a predetermined period).

  The area line display unit 31 of the area display processing unit 31 identifies the preset inner area 32 for detecting an obstacle in the overhead view video 28 and the outer area 33 other than the inner area 32. The area line 35 stored in the vehicle area 31A is provided (drawn).

  Then, the obstacle detection unit 41 detects an obstacle based on the bird's-eye view video 28 periodically sent from the video processing unit 26. When the obstacle is located in the inner area 32, the marker display processing unit 43 of the obstacle detection unit 41 displays the obstacle marker 45 on the automobile 44 that is an obstacle, for example, as shown in FIGS. To do. On the other hand, when the obstacle is located in the outer area 33, the marker display processing unit 43 does not display the obstacle marker 45 on the automobile 46, for example, as shown in FIGS.

  Next, the obstacle detection process of the direct image 30 will be described.

  The direct video processing unit 29 of the video processing unit 26 periodically (in a predetermined cycle) obstructs the direct video 30 corrected by performing noise removal, distortion correction, etc. from the camera-through video 22R1, 22L1, 22F1, 22B1. It transmits to the detection part 41.

  The direct video area display processing unit 37 is used to identify the direct video inner area 38 that detects an obstacle in the direct video 30 and the direct video outer area 39 that does not detect an obstacle. The direct video area line 40 stored in 37 is given (drawn).

  The obstacle detection unit 41 detects an obstacle based on the direct video 30 periodically transmitted from the video processing unit 26. The marker display processing unit 43 of the obstacle detection unit 41 directly displays an image on an automobile 44 that is an obstacle as shown in FIGS. 3 and 6, for example, when the obstacle is located directly in the inner area 38 for images. The obstacle marker 47 is displayed. On the other hand, the marker display processing unit 43 does not directly display the obstacle marker 47 for the image on the automobile 46 as shown in FIGS. 3 and 6 when the obstacle is located in the outer area 39 for the image directly. .

  The overhead video 28 and direct video 30 processed by the obstacle detection unit 41 are transmitted to the monitor 48. In this case, by operating the changeover switch 50 of the monitor 48, the overhead view video 28, the direct video 30 and the like are selected and transmitted, and the selected video is displayed on the display unit 51 of the monitor 48.

  Next, the case where the obstacle detection unit 41 cannot detect an obstacle will be described.

  The obstacle detection unit 41 compares, for example, the previous bird's-eye view image 28 captured from the image processing unit 26 with the current bird's-eye view image 28, and determines the image (object) moving for a specified time as a moving object. . Specifically, when detecting a moving automobile (moving object detection), the image of the automobile is moving, but since the background of the automobile does not move, the automobile can be detected as a moving object. However, for example, when the dump truck 1 travels at a predetermined speed or higher, the background of the vehicle moves together with the image of the car, so the frequency of obstacle detection errors increases and it is impossible to detect the obstacles properly. It may become. Even when a stationary obstacle is detected, if the dump truck 1 travels at a predetermined speed or more, the image is distorted, noise in the image increases, and the frequency of false detection of the obstacle tends to increase. is there.

  In this case, when the area line 35 is displayed on the bird's-eye view image 28, the operator may misunderstand that the vehicle surrounding obstacle detection device 21 is in a state where the obstacle can be detected. Therefore, the area line display control unit 36 of the area display processing unit 31 changes the display mode of the area line 35 when the obstacle detection unit 41 cannot detect the obstacle. Specifically, as shown in FIG. 7, the area line display control unit 36 displays the area line 35 as an undetectable area line 35 </ b> A that is contracted to the vehicle body symbol 25.

  Thus, in the first embodiment, the area line 35 recognizes the inner area 32 that detects an obstacle and the outer area 33 that does not detect an obstacle in the overhead view image 28 displayed on the display unit 51 of the monitor 48. It is possible. Accordingly, the operator can determine at a glance how far the obstacle detection device 21 around the vehicle is from the dump truck 1 as the obstacle detection range, thereby reducing operator confusion. it can.

  The obstacle detection unit 41 displays the obstacle marker 45 on the obstacle when the obstacle is located in the inner area 32, and displays the obstacle marker 45 when the obstacle is located in the outer area 33. The obstacle marker 45 is erased from the obstacle. Thereby, the operator can look for the obstacle marker 45 while paying attention to the inside area 32 and can improve recognition of the obstacle to be noted.

  Further, the area line display control unit 36 shortens the area line 35 to the vehicle body symbol 25 when the obstacle detection unit 41 cannot detect the obstacle. As a result, the operator can recognize at a glance that the obstacle cannot be detected, thereby reducing the confusion of the operator.

  Further, the display unit 51 of the monitor 48 can display the direct video 30 and the like captured by the cameras 22R, 22L, 22F, and 22B. The direct video 30 displays a direct video inner area 38 that detects an obstacle and a direct video outer area 39 that does not detect an obstacle. Thereby, the operator can determine at a glance how far the obstacle detection device 21 around the vehicle is within the obstacle detection range even in the direct image 30 or the like. Confusion can be reduced.

  Next, FIG. 8 and FIG. 9 show a second embodiment of the present invention. The feature of the second embodiment is that the detectable area can be recognized for each type of obstacle. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  In the second embodiment, the area display processing unit 31 further includes a type area display unit 61 in addition to the area line display unit 34. The type area display unit 61 displays, for each obstacle type, inner areas 62 and 64 that detect the obstacle and outer areas 63 and 65 that do not detect the obstacle. In other words, an automobile that is a large obstacle has a greater obstacle detection range (distance from the dump truck 1) than a human (such as a worker) that is a small obstacle. Therefore, as shown in FIG. 9, the type area display unit 61 displays, for example, a vehicle inner area 62 that detects a car and a vehicle outer area 63 that does not detect a car in the overhead view video 28. Further, the type area display unit 61 identifies and displays a human inner area 64 that detects a human and a human outer area 65 that does not detect a human in the overhead view video 28.

  In this case, the area line display unit 34 of the area display processing unit 31 identifies a vehicle area line 66 for identifying the vehicle inner area 62 and the vehicle outer area 63 in the overhead view image 28 (see the dotted line in FIG. 9). Is displayed, and a human area line 67 for identifying the human inner area 64 and the human outer area 65 (see the one-dot chain line in FIG. 9) is displayed. The vehicle area line 66 is stored in the vehicle area 68 of the area display processing unit 31, and the human area line 67 is stored in the human area 69 of the area display processing unit 31.

  Then, the obstacle detection unit 41 detects an obstacle based on the bird's-eye view video 28 periodically sent from the video processing unit 26. When the obstacle is located in the vehicle inner area 62, the marker display processing unit 43 of the obstacle detection unit 41, for example, as shown in FIG. Is displayed. On the other hand, the marker display processing unit 43 does not display the vehicle obstacle marker 71 on the automobile 72 as shown in FIG. 9, for example, when the obstacle is located in the vehicle outer area 63.

  Similarly, the marker display processing unit 43 of the obstacle detection unit 41 displays the human obstacle marker 74 on the human 73 that is an obstacle in the human inner area 64, and the human 75 is in the outer human area 65. When positioned, the human obstacle marker 74 is not displayed on the human 75.

  That is, the obstacle located in the human outer area 65 and the vehicle inner area 62 displays the vehicle obstacle marker 71 when the obstacle is the automobile 70, for example. In this case, the human obstacle marker 74 is not displayed. In this case, the detection of the obstacle of the automobile and the detection of the obstacle of the human can be determined by, for example, a difference in the image, color, shape, etc. of the number of pixels to be monitored.

  The symbol display unit 76 displays an obstacle symbol corresponding to the type of obstacle. As shown in FIG. 9, the symbol display unit 76 displays a car symbol 77 on the vehicle area line 66 of the overhead view video 28 and a human symbol 78 on the human area line 67. Thereby, the operator can recognize at a glance what kind of obstacle each area line 66, 67 corresponds to. The automobile symbol 77 is stored in the vehicle area 68 together with the vehicle area line 66. The human symbol 78 is stored in the human area 69 together with the human area line 67.

  The car symbol 77 may be displayed in characters such as “Car” instead of the picture (or photo) of the car. Similarly, the human symbol 78 may be displayed in characters such as “Worker”.

  Thus, also in the second embodiment configured as described above, it is possible to obtain substantially the same operational effects as those of the first embodiment described above. In particular, in the second embodiment, area lines 66 and 67 indicating the boundary portion of the area to be detected for each type of obstacle are displayed. As a result, the operator can determine at a glance how far each obstacle is from the dump truck 1 so that it can be detected, thereby reducing the confusion of the operator.

  The vehicle area line 66 for identifying the vehicle inner area 62 and the vehicle outer area 63 is displayed with a car symbol 77 to identify the human inner area 64 and the human outer area 65. In the area line 67, a human symbol 78 is displayed. Thus, the operator can easily recognize what kind of obstacle each area line 66, 67 corresponds to, so that confusion of the operator can be reduced.

  Next, FIG. 10 and FIG. 11 show a third embodiment of the present invention. The feature of the third embodiment is that the display mode of the obstacle marker displayed on the obstacle close to the outer area among the obstacles located in the inner area is changed. Note that in the third embodiment, the same components as those in the first embodiment described above are denoted by the same reference numerals, and description thereof is omitted.

  The area display processing unit 31 according to the third embodiment further includes an area partition processing unit 81 in addition to the area line display unit 34. The area section processing unit 81 partitions the inner area 32 into a first area 82 close to the vehicle body 2 (dump truck 1) and a second area 83 far from the vehicle body 2. As shown in FIG. 11, the area line 35 and the vehicle body symbol 25 are partitioned into a first area 82 and a second area 83 by a partition line 84 (see a two-dot chain line in FIG. 11). In this case, the partition line 84 shown in FIG. 11 is displayed with a virtual line, and is not actually displayed in order to avoid confusion with the area line 35. The partition line 84 may be displayed separately from the area line 35, for example, displayed thinner than the area line 35.

  The marker display processing unit 43 according to the third embodiment includes a marker display control unit 85. The marker display control unit 85 displays a first area obstacle marker 87 when the obstacle is located in the first area 82, and displays a second area obstacle when the obstacle is located in the second area 83. The obstacle marker 88 is displayed. In this case, the obstacle marker 87 for 1st area | regions is a frame-shaped marker emphasized with the continuous line, for example. On the other hand, the second area obstacle marker 88 is a blinking frame-shaped marker that is displayed in a different manner from the first area obstacle marker 87.

  The second area obstacle marker 88 may be a marker that is lighter in color than the first area obstacle marker 87 (for example, translucent or the like) instead of the blinking frame marker, It may be a dotted line. That is, it is preferable that the second area obstacle marker 88 has a display mode that is less conspicuous than the first area obstacle marker 87.

  As shown in FIG. 11, when the automobile 86 that is an obstacle is traveling from the first area 82 of the inner area 32 toward the outer area 33, the automobile 86 is located in the first area 82. When there is, the first area obstacle marker 87 is displayed. When the automobile 86 is located in the second area 83, the second area obstacle marker 88 is displayed. When the automobile 86 is located in the outer area 33, the second area obstacle marker 88 is deleted.

  Thus, also in the third embodiment configured as described above, it is possible to obtain substantially the same operational effects as those of the first embodiment described above. In particular, in the third embodiment, the inner area 32 is divided into a first region 82 close to the vehicle body 2 and a second region 83 far from the vehicle body 2. When the obstacle is located in the second area 83, the obstacle marker 88 for the second area is displayed on the obstacle so that the obstacle marker is not noticeable. Thereby, for example, when the obstacle (the automobile 86) moves from the inside area 32 to the outside area 33 over the area line 35, the obstacle marker 87 for the first area suddenly disappears and the operator is confused. Can be reduced.

  In the first embodiment described above, the area line 35 is a right area line 35R corresponding to the right overhead image 28R, a left area line 35L corresponding to the left overhead image 28L, and a previous area line corresponding to the front overhead image 28F. Since 35F and the rear area line 35B corresponding to the rear bird's-eye view video 28B are displayed, the area line 35 has an uneven shape as a whole. However, the present invention is not limited to this, and a rectangular area line 91 may be set inside the area line 35 indicated by a two-dot chain line, for example, as in the first modification shown in FIG. In this case, the area between the area line 91 and the vehicle body symbol 25 is the inner area 92, and the outer area across the area line 91 from the vehicle body symbol 25 is the outer area 93. The same applies to the second and third embodiments.

  In the first embodiment described above, when the obstacle detection unit 41 cannot detect an obstacle, the area line display control unit 36 cannot detect the area line 35 contracted to the vehicle body symbol 25. The case of the area line 35A has been described as an example. However, the present invention is not limited to this. For example, as in the second modification shown in FIG. 13, when the obstacle detection unit 41 cannot detect an obstacle, the area line display control unit 36 35 may be an undetectable area line 101 that is contracted to the inside of the vehicle body symbol 25. Further, the area line 35 may be erased or the color may be lightened. Further, the area line display control unit 36 may be omitted. The point that such a configuration can be changed is the same in the second and third embodiments and the first modification.

  In the first embodiment described above, the obstacle detection unit 41 has been described as an example in which the obstacle detection unit 41 detects an obstacle based on the overhead image 28 transmitted from the video processing unit 26. However, the present invention is not limited to this, and the obstacle detection unit 111 detects an obstacle from the camera-through images 22R1, 22L1, 22F1, and 22B1, as in a third modification shown in FIG. Based on the above, the video processing unit 112 may create the overhead view video 113. The same applies to the second and third embodiments and the first and second modifications.

  In the first embodiment described above, the obstacle detection unit 41 has been described by taking as an example a case where an obstacle is detected based on a bird's-eye view video or a camera-through video. However, the present invention is not limited to this. For example, the dump truck 1 may be provided with a radar transmitter and a radar receiver or an ultrasonic transmitter and an ultrasonic receiver to detect an obstacle. The same applies to the second and third embodiments and the first, second, and third modifications.

  In the first embodiment described above, the case where the overhead view video 28 is created by the four cameras 22R, 22L, 22F, and 22B has been described as an example. However, the present invention is not limited to this. For example, the overhead view video may be created using two or three cameras, or the overhead view video may be created using four or more cameras. The same applies to the second and third embodiments and the first, second, and third modifications.

  Further, in each of the above-described embodiments, a case has been described in which the overhead view video composition processing unit 27 creates the overhead view video 28 as seen from above the dump truck 1. However, the present invention is not limited to this. For example, the video composition processing unit creates a spherical overhead image by synthesizing images captured by a plurality of cameras, and presets for detecting obstacles in the spherical overhead image. The inner area and the outer area other than the inner area may be identified and displayed.

  Moreover, in each embodiment mentioned above, the case where the vehicle surrounding obstacle detection apparatus 21 was mounted in the ultra-large dump truck 1 was mentioned as an example, and was demonstrated. However, the present invention is not limited to this. For example, the vehicle surrounding obstacle detection device 21 may be used in a work vehicle such as a hydraulic excavator, a hydraulic crane, or a wheel loader.

DESCRIPTION OF SYMBOLS 1 Dump truck 2 Car body 21 Vehicle surrounding obstacle detection device 22R Right camera 22L Left camera 22F Front camera 22B Rear camera 23 Controller 25 Car body symbol 27 Overhead video composition processing part
28, 113 bird's-eye view image 31 area display processing unit 32, 62, 64, 92 inner area 33, 63, 65, 93 outer area 34 area line display unit 35, 66, 67, 91 area line 36 area line display control unit 37 directly Video area display processing unit 38 Direct video inner area 39 Direct video outer area 41, 111 Obstacle detection unit 43 Marker display processing unit 44, 46, 70, 72, 86 Automobile (obstacle)
45, 71, 74, 87, 88 Obstacle marker 48 Monitor 61 Type area display 73, 75 Human (obstacle)
76 Symbol display section 77 Automobile symbol 78 Human symbol 81 Area section processing section 82 First area 83 Second area 85 Marker display control section

Claims (6)

A vehicle surrounding obstacle comprising: a plurality of cameras provided on the vehicle body for imaging the surroundings of the vehicle body; a controller for processing images captured by the plurality of cameras; and a monitor for displaying images processed by the controllers In the detection device,
The controller is
A video composition processing unit that creates a composite image that can recognize the periphery of the vehicle body by arranging a vehicle body symbol in the center of the composited image by combining the images captured by the plurality of cameras into a single image When,
An obstacle detection unit for detecting obstacles around the vehicle body;
The composite image displayed on the monitor has an inner area set in a range close to the vehicle body around the vehicle body and capable of detecting an obstacle of a predetermined size, and located on the outer peripheral side of the inner area. An area display processing unit for identifying and displaying an outside area set in a range away from the vehicle body ,
The obstacle detection unit displays an obstacle marker on the obstacle when the obstacle is located in the inner area, and displays the obstacle when the obstacle is located in the outer area. A marker display processing unit for hiding the obstacle marker of an object,
The area display processing unit includes an area division processing unit that divides the inner area into a first region close to the vehicle body and a second region far from the vehicle body,
The vehicle surrounding obstacle detection device , wherein the marker display processing unit includes a marker display control unit that changes a display mode of the obstacle marker in the first area and the second area .   The vehicle surrounding obstacle detection device according to claim 1, wherein the area display processing unit includes a type area display unit that displays the inner area and the outer area for each type of the obstacle.   The vehicle surrounding obstacle detection device according to claim 2, wherein the type area display unit includes a symbol display unit that displays an obstacle symbol corresponding to the type of the obstacle. The area display processing unit includes an area line display unit that displays an area line that distinguishes between the inner area and the outer area,
The Area line display unit, said when the obstacle detecting unit is impossible detecting the obstacle, according to claim 1 comprising with an area-line display control unit for changing the display mode of the area line, 2 or 3 Vehicle surrounding obstacle detection device according to claim 1. The controller further includes a direct video area display processing unit for identifying and displaying a preset inner area for detecting the obstacle and an outer area other than the inner area on the direct video captured by each camera. The vehicle surrounding obstacle detection device according to claim 1, 2, 3 or 4 . The vehicle surrounding obstacle detection device according to claim 4 , wherein the area line display control unit changes a display mode of the area line when the vehicle body is traveling at a predetermined speed or more.

Images