JP2015210661A - Road surface estimation apparatus for mine work vehicle and road surface estimation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road surface estimation system for mine work vehicle capable of properly detecting road surface under off load environment.SOLUTION: The road surface estimation system includes: a wide road position estimation section 14 that estimates a plane area D1 in a piece of three-dimensional image information based on a piece of distance information in the three-dimensional image information; a road shoulder detection section 15 that detects the position of a road shoulder S in the three-dimensional image based on the plane area D1 estimated by the wide road position estimation section 14; and a road surface area estimation section 16 that estimates a road surface area D2 including the road shoulder S in the three-dimensional image as the boundary based on the position of the road shoulder S detected by the road shoulder detection section 15.

Description

  The present invention relates to a road surface estimation apparatus for a mining work vehicle such as an off-road dump truck and a road surface estimation system including the same.

  Generally, in mines, construction machines such as hydraulic excavators and dump trucks are used for mining and transporting earth and sand. Construction machinery used in mines is required to be unmanned from the viewpoint of safety and cost reduction. In dump trucks, since the amount of earth and sand transported per unit time is directly linked to the progress of mining, efficient operation is required. Therefore, in order to efficiently transport a large amount of earth and sand outside the mining site, a mining system using an autonomously traveling dump truck capable of continuous operation is required.

  However, since the mine traveling road where the dump truck travels is off-road and there are many bad roads, there is a concern about collision with obstacles such as other vehicles by allowing the dump truck to travel autonomously and drive it unattended. If an obstacle occurs on the traveling road and an autonomous traveling unmanned dump truck comes into contact with the obstacle and stops, the operation of the mine is stopped for a long time. Therefore, in order to improve the reliability of the autonomously traveling dump truck, it is possible to detect obstacles on the front vehicle and the traveling road at an early stage and to follow the front vehicle and avoid obstacles. A highly reliable obstacle detection device is required.

  Conventionally, as this kind of obstacle detection device, a configuration using two stereo cameras is disclosed in Patent Document 1, for example. In Patent Document 1, a white line on a road is extracted from an image captured by two stereo cameras by a feature extraction unit, and whether or not the extracted white line is an obstacle region is detected by a difference detection unit. To do. Then, the height of the obstacle is calculated from the detection result of the difference detection unit, and the true obstacle region is detected by the height calculation unit.

JP 2001-283204 A

  In the obstacle detection device according to Patent Document 1, a white line on a road is detected by two stereo cameras, an obstacle area (road surface area) is estimated from the detected white line, and an obstacle in the road area is detected. Although it is the structure which detects an object, the white line is not drawn on the traveling road in the off-road environment where the dump truck etc. which are used for a mine drive, and the obstacle detection device concerning the above-mentioned patent documents 1 is off-road It cannot be used to detect the road surface of the traveling road under the environment.

  The present invention has been made from the above-described prior art, and an object of the present invention is to provide a road surface estimation device and a road surface estimation system for a mining work vehicle that can appropriately detect a road surface even in an off-road environment. It is to provide.

  In order to achieve this object, a road surface estimation device for a mining work vehicle according to the present invention receives three-dimensional image information, and based on distance information in the image information, calculates a plane area in the image information. Based on the plane area estimated by the plane area estimation unit, the plane area estimated by the plane area estimation unit, a road shoulder detection unit that detects a road shoulder position having a predetermined height or more in the image information, and the road shoulder detection unit And a road surface area estimation unit for estimating a road surface area having the road shoulder as a boundary in the image information based on the road shoulder position detected in this manner.

  The present invention configured as described above estimates a plane area in the image information from the three-dimensional image information, and then, based on the plane area, calculates a shoulder position having a predetermined height or more in the image information. I try to detect it. Therefore, it is sufficient to detect the shoulder position only for the planar region, the detection region for detecting the shoulder position can be narrowed, and the shoulder detection process can be simplified. In addition, since the road surface area with the road shoulder as a boundary is estimated based on the detected road shoulder position, there is no need to detect the road surface area from the image information again, and the road surface in the image information. The area can be easily estimated. Therefore, even on the road surface in an off-road environment where a road shoulder is provided, the road surface area can be detected appropriately, and the road surface in the image information can be detected appropriately.

  Further, the present invention is the above invention, wherein the road shoulder detection unit detects a road shoulder located on both sides of the road surface from the image information, and the road surface region estimation unit detects a distance between the road shoulders detected by the road shoulder detection unit. The road surface area is estimated.

  The present invention configured as described above detects road shoulder positions located on both sides of the road surface from the three-dimensional image information and estimates the space between the road shoulders as a road surface area. The road surface area can be estimated more appropriately and easily than in the case of estimating. Therefore, the road surface detection process under an off-road environment can be performed with higher accuracy.

  In the invention described above, the three-dimensional image information includes luminance information, and the road surface area estimation unit estimates a similar area similar to the road surface area from the luminance information in the image information. A similar area estimation unit is included, and the road surface area is estimated by adding the similar area.

  In general, a road surface in an off-road environment is compacted flat, and therefore tends to have higher brightness than other areas. Therefore, according to the present invention, a similar area similar to the road surface area is estimated from the luminance information in the image information, and the road surface area is estimated by adding the similar area. For this reason, the road surface area estimated by the road surface area estimation unit can be supplemented, and the road surface area can be estimated more accurately by the road surface area estimation unit.

  In the present invention, the three-dimensional image information includes color information, and the road surface area estimation unit estimates a similar area similar to the road surface area from the color information in the image information. A similar area estimation unit is included, and the road area is estimated by adding the similar area.

  In general, since the road surface under an off-road environment is flattened, the color information in the image information tends to be different from that in other areas. Therefore, the present invention estimates a similar area similar to the road surface area from the color information in the image information, and adds the similar area to estimate the road surface area. As a result, the road surface area estimated by the road surface area estimation unit can be supplemented, and the road surface area can be estimated more accurately by the road surface area estimation unit.

  Further, in the present invention according to the above invention, the road shoulder detection unit detects a road shoulder closer to the mining work vehicle from the image information, and the road surface area estimation unit is detected by the road shoulder detection unit. A region within a predetermined range from the side road shoulder is estimated as a road surface region.

  The present invention configured as described above detects a road shoulder closer to the mining work vehicle from three-dimensional image information, and estimates a region within a predetermined range from the road shoulder as a road surface region. Accordingly, it is not necessary to estimate all road surface areas in the image information, and it is highly possible that the mining work vehicle will travel, and it is possible to detect the area on the side where the possibility of a collision or the like is high. In addition, the processing by the road surface area estimation unit can be further simplified.

  In the invention described above, the invention further includes an obstacle detection unit that detects an imaging target in the image information that matches a predetermined criterion from the road surface area in the image information as an obstacle. It is said.

  Since the present invention configured as described above can appropriately detect the road surface by the road surface region estimation unit even in an off-road environment where a road shoulder is provided, the road surface region estimated from the road surface region estimation unit can be used. By detecting an imaging target that matches a predetermined criterion as an obstacle, the detection area for detecting the obstacle becomes more accurate, and the obstacle can be detected appropriately and accurately.

  A road surface estimation system for a mine work vehicle according to the present invention comprises: an image acquisition unit that acquires three-dimensional image information; and the road surface estimation device for a mine work vehicle according to claim 1, The road surface estimation apparatus is characterized in that the image information acquired by the image acquisition unit is input.

  In the present invention configured as described above, after estimating a plane area in the image information by the road surface estimation device from the three-dimensional image information acquired by the image acquisition unit, based on the plane area, Further, a road shoulder having a height higher than a predetermined height is detected. Therefore, it is sufficient to detect the shoulder position only for the estimated plane region, the detection region for detecting the shoulder position can be narrowed, and the shoulder detection process can be simplified. In addition, since the road surface area with the road shoulder as a boundary is estimated based on the detected road shoulder position, there is no need to detect the road surface area again from the detected image information. The road surface area can be easily estimated. Therefore, even on the road surface in an off-road environment where a road shoulder is provided, the road surface area can be detected appropriately, and the road surface in the image information can be detected appropriately.

  The present invention estimates a plane area in the image information based on the distance information in the three-dimensional image information, and then detects a road shoulder position having a predetermined height or more in the image information based on the plane area. And based on this detected road shoulder position, it is set as the structure which estimates the road surface area | region which makes the road shoulder in image information a boundary. With this configuration, the present invention can narrow the detection area for detecting the shoulder position, and can simplify the shoulder detection process. Also, since the road surface area in the image information with the road shoulder as the boundary is estimated, there is no need to detect the road surface area from the image information again, and the road surface area in the image information can be easily estimated. Can do. Therefore, it is possible to appropriately detect the road surface even on the road surface in the off-road environment where the road shoulder is provided. Problems, configurations, and effects other than those described above will be made clear from the following description of embodiments.

It is the schematic which shows the working vehicle for mines where the road surface estimation apparatus which concerns on 1st Embodiment of this invention is used. It is the schematic which shows the mine system where the said working vehicle for mines is used. It is the schematic which shows the drive structure of the said working vehicle for mines. It is the schematic which shows the road surface condition which the said working vehicle for mines drive | works. It is a figure which shows the parallax image imaged with each camera of the image acquisition part of the said working vehicle for mines, (a) is the captured image of the camera 11a, (b) is the captured image of the camera 11b. It is a figure which shows the appearance image imaged in the image acquisition part of the said working vehicle for mines. It is a figure which shows the process of the image acquired in the image acquisition part of the said working vehicle for mines, (a) is a captured image, (b) is a plane area extraction image (c) is a road surface area estimated image. It is a flowchart which shows the process of the road surface estimation apparatus mounted in the said working vehicle for mines. It is the schematic which shows the road surface detection condition by the road surface estimation apparatus which concerns on 2nd Embodiment of this invention.

  EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing the obstacle detection system provided with the road surface estimation apparatus which concerns on this invention is demonstrated based on figures.

  FIG. 1 is a schematic diagram illustrating an off-road dump truck that is a mine vehicle 1 as a mine work vehicle in which an obstacle detection system including a road surface estimation device according to a first embodiment of the present invention is used. FIG. 2 is a schematic diagram showing a mine system in which the vehicle 1 is used. FIG. 3 is a schematic diagram showing a drive configuration of the vehicle 1. FIG. 4 is a schematic diagram showing a road surface condition on which the vehicle 1 travels. 5A and 5B are diagrams illustrating parallax images captured by the cameras 11a and 11b of the external environment measuring unit 12 of the vehicle 1, in which FIG. 5A is a captured image of the camera 11a and FIG. 5B is a captured image of the camera 11b. . FIG. 6 is a diagram showing image information calculated by the external environment measuring unit 12 from the two-dimensional parallax image information shown in FIGS. 5A and 5B detected by the two cameras 11a and 11b. It is. FIG. 7 is a diagram illustrating processing of an image acquired by the external environment measuring unit 12 of the vehicle 1, where (a) is a captured image, (b) is a planar area extraction image (c), and a road surface area estimation image. The image having the color information of the object in the parallax images shown in FIG. 6 and FIG. 7A as seen with the naked eye is called an appearance image. The appearance image includes color information corresponding to each pixel in the parallax image.

  As shown in FIG. 2, the vehicle 1 is an unmanned traveling type that can travel autonomously on a road surface R that is a traveling path (conveyance path) under an off-road environment provided in a mine. The mine is provided with a control center 3 for transmitting / receiving predetermined information to / from the vehicle 1 and for controlling the traveling of the vehicle 1 and for loading loads such as earth and sand on the vehicle 1. A hydraulic excavator 4 or the like is used. The road surface R is an obstacle detection area by the obstacle detection system, and road shoulders S are provided on both sides in the width direction. As shown in FIG. 4, each road shoulder S is provided with embankment B, a cutting surface C that is a cliff formed by cutting a mine with a hydraulic excavator 4 or the like. The embankment B and the cutting surface C are formed to have a predetermined height or higher, for example, a height of 1.5 m or higher.

  As shown in FIG. 1, the vehicle 1 includes a vehicle main body 1a, a driver seat 1b provided on the upper front side of the vehicle main body 1a, and a vessel 1c as a working portion provided on the vehicle main body 1a so as to be undulated, Left and right front wheels 1d and rear wheels 1e that support the vehicle main body 1a so as to travel are provided. On the front deck 1f of the vehicle main body 1a, a stereo camera device 11 for detecting the environment around the vehicle main body 1a, particularly in front of the traveling direction, is attached. The vehicle 1 travels while detecting and avoiding obstacles on the road surface R recognized by the stereo camera device 11. A GPS device 1g is attached to the front corner on the deck 1f. The GPS device 1g is a position detection unit for detecting the traveling position of the vehicle main body 1a.

  The stereo camera device 11 includes a pair of cameras 11a and 11b, and acquires three-dimensional three-dimensional image information having color information of the outside world using the two cameras 11a and 11b. The three-dimensional image information includes distance information to the imaging target calculated from the difference between the two-dimensional parallax image information shown in FIGS. 5A and 5B detected by the two cameras 11a and 11b. And the appearance image shown in FIG. 6 and FIG. 7A are included. The appearance image has color information corresponding to each pixel in the parallax image. The stereo camera device 11 is attached so that the center between the left and right cameras 11a and 11b is located at the center position that is the center portion in the left-right direction on the front side of the vehicle body 1a. Internal parameters such as focal lengths and lens distortions of the cameras 11a and 11b, and external parameters indicating the positional relationship and the installation position on the vehicle body are synchronized with each other. The cameras 11a and 11b are directed forward of the vehicle 1 so that the optical axes are parallel to each other. As shown in FIG. 1, the imaging regions 11c and 11d of the cameras 11a and 11b are partially overlapped. Has been.

  As shown in FIG. 3, the vehicle 1 includes a travel drive device 5. The travel drive device 5 drives an engine 5a as a power source, a generator 5b driven by the engine 5a, a power control device 5c to which power generated by the generator 5b is supplied, and a rear wheel 1e. And a travel motor 5d. The power supplied to the travel motor 5d is controlled from the power control device 5c. The power control device 5 c is controlled by a controller 6 mounted on the vehicle 1.

  The controller 6 is provided with a steering motor 5e that is a steering device for steering the vehicle main body 1a via the power control device 5c, and a brake device 5f that is a braking device such as a retarder brake for braking the vehicle main body 1a. It is a vehicle-mounted controller that controls the driving of the vehicle. The vehicle main body 1a is equipped with an in-vehicle wireless device 5g that is a communicable communication unit for wirelessly transmitting a predetermined signal to the control center 11. The controller 6 includes a storage unit 6a in which map information related to at least a travel area in which the vehicle 1 travels is stored in advance. The map information stored in the storage unit 6a includes road surface R information on which the vehicle 1 travels, road shoulder S position information, and the like.

<Obstacle detection system>
The obstacle detection device 10 is a road surface estimation device that estimates a road surface R from three-dimensional image information, and detects an obstacle O on the estimated road surface R. The obstacle detection device 10 is a stereo camera system as an image analysis device for detecting obstacles on the road surface R such as other vehicles 1, work vehicles, rocks, minerals, and earth and sand. Specifically, the obstacle detection system 10 includes an external environment measurement unit 12 including a stereo camera device 11, a distance calculation unit 13, a global road surface position estimation unit 14, a road shoulder detection unit 15, and a road surface region estimation unit 16. And an obstacle detection unit 17. Each process by the external environment measurement unit 12, the distance calculation unit 13, the global road surface position estimation unit 14, the road shoulder detection unit 15, the road surface region estimation unit 16, and the obstacle detection unit 17 is performed by the controller 6.

  The external environment measurement unit 12 uses the camera acquired images shown in FIGS. 5A and 5B as image information detected by the cameras 11a and 11b, and FIGS. 5A and 5B. The parallax images calculated based on the differences in the appearance of the camera acquired images shown in FIG. The parallax image is created based on one of the camera-acquired images. For example, when the parallax image is created based on the image of FIG. 5A, the position of the object captured in FIG. It searches for where it is located in the camera acquired image of 5 (b). At this time, the position difference between these camera acquired images is larger as the object is closer and smaller as the object is farther. Based on this difference, the three-dimensional position of the object photographed by each camera 11a, 11b is acquired. This technique is generally called stereo matching. At this time, each point (pixel) in the appearance image shown in FIGS. 6 and 7A includes the color information of the object in the parallax image. The appearance image has color information corresponding to each pixel in the parallax image. The external environment measurement unit 12 is an image acquisition unit that acquires three-dimensional image information (three-dimensional point group) that is three-dimensional distance information in each of the cameras 11 a and 11 b of the stereo camera device 11. The three-dimensional point group is point sequence information in which each point (pixel) in the detected image has information of (x, y, z, r, g, b). Further, the external environment measurement unit 12 performs an appearance image on each of the cameras 11a and 11b as an initialization process when a switch (not shown) for turning on and off the power supply of the obstacle detection system 10 is turned on. The calibration (calibration) data of each camera 11a, 11b, etc. are read, making it possible to acquire.

  The distance calculation unit 13 determines the distance to the imaging target at each point in the appearance image and the three-dimensional shape in the image based on the three-dimensional image information detected by the external environment measurement unit 12, particularly the parallax image information. Get information. The global road surface position estimation unit 14 receives the three-dimensional image information detected by the external environment measurement unit 12 and estimates the plane region D1 in the three-dimensional image information shown in FIG. 7B, that is, a rough road surface R position. It is a plane area estimation part to perform. The global road surface position estimation unit 14 compresses each point in the three-dimensional image information detected by the external environment measurement unit 12 into two-dimensional image information excluding distance information from each point to the imaging target. From the color information or luminance information of each point in the two-dimensional image information, the rough position of the road surface R (planar region D1) in the two-dimensional image information is detected.

  The road shoulder detection unit 15 detects the position of the road shoulder S such as the embankment B or the cutting surface C provided on both sides of the road surface R from the three-dimensional image information detected by the external environment measurement unit 12. The road shoulder detection unit 15 superimposes the rough road surface R position information in the two-dimensional image information estimated by the global road surface position estimation unit 14 on the three-dimensional image information detected by the external environment measurement unit 12, and the like. A rough road surface R position in the three-dimensional image information is estimated, and a predetermined height, for example, 1.0 m, is determined from an area within a predetermined range centering on both sides of the estimated rough road surface R position. A substantially continuous object having the above height is defined as a shoulder S, and the position of the shoulder S is detected.

  The road surface area estimation unit 16 estimates the road surface area D2 in the 3D image information from the 3D image information detected by the external environment measurement unit 12. The road surface region estimation unit 16 includes a road surface base region estimation unit 16a, a surrounding similar region detection unit 16b, and a road surface region complement unit 16c. The road surface base area estimation unit 16a detects a road surface base area that is an imaging target in which the obstacle detection unit 17 detects an obstacle. The road surface base region estimation unit 16a superimposes the road shoulder position information detected by the road shoulder detection unit 15 on the 3D image information detected by the external environment measurement unit 12, and the road shoulder S in the 3D image information. A position is specified, and a region having the specified shoulder S position as a boundary, that is, a region between the shoulders S is set as a road surface base region in the three-dimensional image information.

  The surrounding similar region detection unit 16b is similar to the road surface base region estimated by the road surface base region estimation unit 16a based on the color information and luminance information of each point in the three-dimensional image information detected by the external environment measurement unit 12. A similar region is calculated. The surrounding similar region detection unit 16b includes a color calculation and luminance information in the road surface base region estimated by the road surface base region estimation unit 16a and a distance calculation unit 13 in the three-dimensional image information detected by the external environment measurement unit 12. Compare the color information or luminance information of each point for which the distance to the imaging target could not be calculated, and if the difference between these color information or luminance information is less than a predetermined value, the point in the image information is Calculate as a similar region. Note that the peripheral similar region detection unit 16b may calculate a similar region using only one of color information and luminance information of each point in the three-dimensional image information.

  The road surface region complementing unit 16c is not included in the road surface base region, for example, by overlapping the road surface base region estimated by the road surface base region estimation unit 16a and the similar region detected by the surrounding similar region detection unit 16b. A complementary area that is a road surface area is calculated. As shown in FIG. 7C, the road surface region complementing unit 16c adds the calculated supplemental region to the road surface base region, as shown in FIG. 7C, and the road surface region D2 is a target region for detecting obstacles. Determine as. That is, the road surface area complementing unit 16c estimates a road surface area D2 by adding a similar area.

  The obstacle detection unit 17 detects irregularities in the road surface region D2 estimated by the road surface region estimation unit 16, that is, the obstacle O. Examples of the obstacle O include other vehicles 1, work vehicles, rocks, minerals, and earth and sand. Examples of the work vehicle include a hydraulic excavator 4, a grader, a bulldozer, a load loader, a patrol car, and the like. The obstacle detection unit 17 also detects the road surface gradient, road width, curve curvature, undulation rate (unevenness degree), and the like of the road surface region D2 estimated by the road surface region estimation unit 16. The obstacle detection unit 17 includes the three-dimensional image information detected by the external environment measurement unit 12 in the road surface region D2 supplemented by the road surface region supplementation unit 16c, up to the imaging target of each point of the road surface region D2. When the variation in the distance information (z component) is equal to or greater than a predetermined range, it is determined that the predetermined determination criterion is met, and the imaging target in the three-dimensional image information is detected as an obstacle O. The obstacle detection unit 17 detects the size and height of the obstacle O based on the variation range of the distance information of each point in the road surface area D2 in the three-dimensional image information or the degree of the variation.

  The controller 6 applies the traveling position information acquired by the GPS device 1g to the map information stored in the storage unit 6a, for example, so that the road surface area in front of the traveling position in the map information is the road surface area estimation unit 16. When the road surface area is different from the road surface area D2 estimated in step 5, the obstacle information on the position, height, and size of the obstacle O detected by the obstacle detection unit 17 is displayed on the in-vehicle wireless device 5g. To the control center 3.

  The controller 6 is configured such that the height of the obstacle O detected by the obstacle detection unit 17 is lower than the vehicle height (minimum ground height) of the vehicle 1 and the width of the obstacle O is the left and right front wheels 1d and rear wheels 1e. When it is determined that the distance between the wheels is smaller, the power control device 5c, the steering motor 5e, the brake device 5f, and the like are appropriately controlled so that the obstacle O passes between the left and right front wheels 1d and the rear wheels 1e. To avoid obstacles. Further, the controller 6 determines that the height of the obstacle O detected by the obstacle detection unit 17 is higher than the vehicle height (minimum ground height) of the vehicle 1 or the width of the obstacle O is left and right. When it is determined that the distance between the front wheel 1d and the rear wheel 1e is larger than that, the power control device 5c, the steering motor 5e, the brake device 5f, and the like are appropriately controlled to appropriately steer and adjust the speed of the vehicle 1 to avoid obstacles. Let

<Operation>
Next, processing of the obstacle detection system according to the first embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing a process of the road surface estimation system mounted on the vehicle 1.

  When the power supply of the obstacle detection device 10 is turned on by the switch, the initialization process of the external environment measuring unit 12 is performed (step S1, hereinafter simply referred to as “S1” or the like). In the initialization process of S1, the appearance images shown in FIGS. 6 and 7A can be acquired by the cameras 11a and 11b of the stereo camera 11, and the calibration data and the like of these cameras 11a and 11b are adjusted. The parallax images shown in FIGS. 5A and 5B can be calculated from the captured images detected by the cameras 11a and 11b.

  Thereafter, a three-dimensional image having color information of each point is calculated by the external environment measurement 12 based on each appearance image and parallax image detected by each camera 11a, 11b (S2). Next, the distance to the imaging target at each point in the three-dimensional image calculated in S2 and the three-dimensional shape information are calculated by the distance calculation unit 13 (S3). Further, the three-dimensional image information calculated in S2 is input to the global road surface position estimation unit 14, and for each point in the three-dimensional image, the two-dimensional image excluding distance information to the imaging target of each point. Is calculated by the global road surface position estimation unit 14. At the same time, when the color information of each point in the two-dimensional image report is within a predetermined range, as shown in FIG. 7 (b), an area where there is a possibility of the road surface R, that is, the rough road surface R It is detected as a position (planar region D1) (S4).

  Next, the rough road surface R position in the three-dimensional image is estimated by, for example, superimposing the planar area D1 detected in S4 on the three-dimensional image calculated in S2, and the rough road surface R position is estimated. A substantially continuous object having a height of, for example, 1.0 m or more from an area within a predetermined range centered on both side areas is defined as a road shoulder S, and the position of the road shoulder S is detected by the road shoulder detection unit 15 (S5). .

  Thereafter, the shoulder S position in the three-dimensional image is specified by, for example, superimposing the shoulder S position information detected in S5 on the three-dimensional image calculated in S2, and the distance between the identified shoulders S is determined. Is estimated as a road surface base region by the road surface base region estimation unit 16a (S6). Color information and luminance information in the road surface base area estimated in S6, and color information and luminance information of each point in the three-dimensional image detected in S2 where the distance to the imaging target could not be calculated in S3 And each point where the difference in color information or luminance information is equal to or smaller than a predetermined value is calculated as a region that is likely to be the road surface R, that is, a similar region, by the surrounding similar region detection unit 16b. (S7).

  Next, a road surface area that is not included in the road surface base area, that is, a complementary area is added to the road surface area complementing unit 16c by overlapping the road surface base area estimated in S6 and the similar area calculated in S7. A region obtained by adding the calculated complementary region to the road surface base region is determined as a road surface region D2 shown in FIG. 7C (S8). Thereafter, for the road surface area D2 determined in S8 in the three-dimensional image detected in S2, the variation in distance information to the imaging target of each point of the road surface area D2 is a predetermined range. Whether or not this is the case is determined by the obstacle detection unit 17, and a portion where the variation is equal to or larger than a predetermined range is detected as an obstacle O (S9). In S9, the size and height of the obstacle O are also detected according to the variation range of the distance information of each point in the road surface area D2 in the three-dimensional image detected in S2 or the degree of variation. The

  Thereafter, the road surface area in front of the travel position in the map information stored in the storage unit 6a is calculated by the controller 9 from the travel position information acquired by the GPS device 1g, and the travel position in the calculated map information. The road surface area is compared with the road surface area D2 determined in S8, and it is determined whether these road surface areas coincide with each other (S10). In S10, when it is determined that the road surface area D2 determined in S8 is different from the road surface area in front of the travel position in the map information (No), the position of the obstacle O, the high Obstacle information regarding the height and size is transmitted to the control center 3 via the in-vehicle wireless device 5g (S11).

  On the other hand, if it is determined in S10 that the road surface area D2 determined in S8 matches the road surface area in front of the travel position in the map information (Yes), and the obstacle information is transmitted to the control center in S11. 3, the height of the obstacle O detected in S9 is lower than the vehicle height of the vehicle 1, and the width of the obstacle O is greater than the wheel spacing between the left and right front wheels 1d and the rear wheel 1e. It is judged whether it is small (S12).

  In S12, it is determined that the height of the obstacle O detected in S9 is lower than the vehicle height of the vehicle 1 and that the width of the obstacle O is smaller than the wheel interval between the left and right front wheels 1d and the rear wheel 1e. In the case of (Yes), the vehicle 1 is travel-controlled by the controller 6 so that the obstacle O detected in S9 straddles between the left and right front wheels 1d and the rear wheels 1e, and the obstacle O is avoided (S13). ). On the other hand, if it is determined in S12 that the height of the obstacle O detected in S9 is higher than the vehicle height of the vehicle 1, or the width of the obstacle O is the wheels of the left and right front wheels 1d and rear wheels 1e. When it is determined that the interval is larger (No), the controller 1 steers and adjusts the speed of the vehicle 1 so as not to collide with the obstacle O detected in S9, and the obstacle O is avoided ( S14).

  Note that the processing of S1 to S14 is repeatedly performed at a predetermined frame, for example, about 30 frames per second, and the obstacle detection at S9 is performed at intervals of, for example, about 0.1 second. It can also be configured to detect as an obstacle O that obstructs the travel of the vehicle 1 only when it is continuously detected at.

<Effect>
As described above, in the obstacle detection system according to the first embodiment, the planar region D1 in the three-dimensional image captured by the cameras 11a and 11b of the stereo camera device 11 and calculated by the external environment measuring unit 12 is global. After detection by the road surface position estimation unit 14, the road shoulder detection unit 15 detects the road shoulder S position in the three-dimensional image from the distance information in the three-dimensional image. As a result, the plane area D1 detected by the global road surface position estimation unit 14 can be estimated as a rough road surface R position. For example, a road shoulder is detected for a region within a predetermined range centering on both sides of the rough road surface R position. By detecting the shoulder S in the part 15, the shoulder S provided on both sides of the road surface R can be detected appropriately.

  That is, the road shoulder detection unit 15 needs to detect the road shoulder S by detecting the road shoulder S in the three-dimensional image based on the plane region D1 estimated by the global road surface position estimation unit 14. A certain detection area can be narrowed. Therefore, it is not necessary to detect the road shoulder S in the road shoulder detection unit 15 for all regions in the three-dimensional image calculated by the external environment measurement unit 12, and the processing amount by the road shoulder detection unit 15 can be reduced, and the road shoulder detection unit. 15 can reduce false detection. Therefore, the road shoulder detection process by the road shoulder detection unit 15 can be simplified, and the processing time of the road shoulder detection by the road shoulder detection unit 15 can be further increased.

  Further, in order to estimate the region between the road shoulders S detected by the road shoulder detection unit 15 as the road surface region by the road surface region estimation unit 16, the road shoulder detection unit 15 detects each road shoulder S provided on both sides of the road surface R. If it is possible, the road surface area estimation unit 16 can automatically estimate the road surface area. Therefore, it is not necessary to detect the road surface area again from the 3D image information, and the road surface area in the 3D image can be estimated easily and appropriately compared with the case where the road surface area is estimated from one of the road shoulders S. . Therefore, even if it is the road surface R in the off-road environment without the white line etc. which were provided in the mine etc., it is set as the road surface R by which the embankment B and the cutting surface C which have the height more than predetermined | prescribed on the road shoulder S of both sides were provided. Thus, the road surface area where the vehicle 1 travels in the off-road environment can be detected with high accuracy, and the detection process of the road surface R from the three-dimensional image in the off-road environment can be performed more simply and accurately. Can do.

  Further, the road surface R provided in the mine is flattened by, for example, a load loader so that the vehicle 1 can travel, so that the reflectance of the irradiated light is large compared to other areas, and the stereo camera When images are taken by the cameras 11a and 11b of the apparatus 11, the brightness tends to be higher than that in other areas. Therefore, a region between the road shoulders S in the three-dimensional image measured by the external environment measuring unit 12 is set as a road surface base region, and the color information and luminance information in the road surface base region and the three-dimensional detected by the external environment measuring unit 12 are used. In the image, the distance calculation unit 13 compares the color information or luminance information of each point for which the distance to the imaging target could not be calculated, and the similar region is calculated as the similar region by the peripheral similar region detection unit 16b. A region obtained by superimposing the road surface base region and the similar region is defined as a road surface region D2 by the road surface region complementing unit 16c.

  As a result, the road surface region complementing unit 16c also supplements the road surface region D2 that cannot be calculated by the road surface base region estimation unit 16a based on the distance information in the three-dimensional images captured by the cameras 11a and 11b of the stereo camera device 11. be able to. Therefore, the road surface area estimation unit 16 can estimate the road surface area D2 more accurately and reliably, so that erroneous detection of the road surface area D2 by the road surface area estimation unit 16 can be reduced.

  Furthermore, only the road surface area D1 estimated by the road surface area estimation unit 16 in the three-dimensional image captured by the cameras 11a and 11b of the stereo camera device 11 and detected by the external environment measurement unit 12 is detected as an obstacle detection unit. Obstacle detection by 17 may be performed. Therefore, compared with the case where the obstacle detection unit 17 detects the obstacle O for all regions in the three-dimensional image calculated by the external environment measurement unit 12, the processing amount by the obstacle detection unit 17 can be reduced. Similarly to the road shoulder detection unit 15, the obstacle detection process by the obstacle detection unit 17 can be simplified, and the obstacle detection processing time by the obstacle detection unit 17 can be further increased.

  Therefore, the obstacle O and the road surface shape on the road surface R can be detected appropriately and accurately even in an off-road environment such as a mine where the embankment B or the cutting surface C having a predetermined height or more is provided on the shoulder S. Can do. Therefore, for example, small obstacles O and irregularities on the road surface R such as rocks and loads falling can be detected with high accuracy, so that the vehicle 1 can be adjusted according to the detected obstacles O and the height and size of the irregularities. By controlling the traveling to detect the accident factor of the vehicle 1 and avoiding a rear-end collision or the like, the work efficiency of the vehicle 1 in an off-road environment such as a mine can be increased. Therefore, the autonomous traveling of the vehicle 1 can be performed more appropriately.

[Second Embodiment]
FIG. 9 is a schematic diagram illustrating a road surface detection state by the road surface estimation apparatus 10 according to the second embodiment of the present invention. The second embodiment differs from the first embodiment described above in that the first embodiment estimates the area between the road shoulders S detected by the road shoulder detection unit 15 as the road surface area, whereas the second embodiment. Detects a road shoulder S closer to the vehicle body 1a by the road shoulder detection unit 15, and estimates a region within a predetermined range from the detected road shoulder S1 as a road surface region. In the second embodiment, the same or corresponding parts as those in the first embodiment are denoted by the same reference numerals.

<Configuration>
In the second embodiment, the road shoulder detection unit 15 detects the positions of the road shoulders S and S1 provided on both sides of the road surface R from the three-dimensional image information detected by the external environment measurement unit 12, and the three-dimensional From the distance information in the image information, the road shoulder S1 on the side close to the vehicle body 1a among the detected road shoulders S is specified. In other words, the road shoulder detection unit 15 superimposes the planar region D1 estimated by the global road surface position estimation unit 14 on the three-dimensional image detected by the external environment measurement unit 12 or the like, thereby obtaining a planar region in the three-dimensional image. D1 is estimated, and the position of the road shoulder S1 is detected from within a predetermined range centering on the edge of the estimated plane area D1 on the side closer to the vehicle body 1a.

  The road surface base region estimation unit 16a superimposes the position of the road shoulder S1 detected by the road shoulder detection unit 15 on the 3D image detected by the external environment measurement unit 12, and specifies the position of the road shoulder S1 in the 3D image, The specified road shoulder S1 position is defined as a boundary on the traveling side, that is, a right boundary, and an area in a predetermined range, for example, a half from the road shoulder S1 position toward the center of the road surface is defined as a road surface base region. Based on the color information and luminance information of each point in the three-dimensional image detected by the external environment measuring unit 12, the surrounding similar region detection unit 16b causes the vehicle body 1a to travel from the lateral center position of the three-dimensional image. A similar region similar to the road surface base region is calculated on the side, that is, the right half region R1. The road surface area complementing unit 16c calculates a complementary area based on the road surface base area and the similar area, and determines the area obtained by adding the complementary area to the road surface base area as the road surface area D2. The obstacle detection unit 17 detects an obstacle O in the road surface area D2 in the three-dimensional image detected by the external environment measurement unit 12.

<Effect>
As described above, in the obstacle detection system according to the second embodiment, in addition to the operational effects that can be achieved in the first embodiment, among the detected road shoulders S and S1, the road shoulder S1 on the side closer to the vehicle body 1a. , And only the road surface R1 on the traffic side having the road shoulder S1 as a boundary (a region that is approximately a half of the road surface R closer to the traveling side) is an object for detecting the obstacle O. That is, in the case where the traffic of the vehicle 1 is determined in advance, the situation of the road surface R1 on the side of the road on which the vehicle 1 travels is highly likely to cause a collision or the like. It is sufficient to grasp the road surface R1 situation near the road with reference to the shoulder S.

  Therefore, it is possible to efficiently estimate the road surface area on the traffic side required when the vehicle 1 travels, and it is possible to efficiently detect obstacles in the road surface area on the traffic side. Therefore, compared with the obstacle detection system according to the first embodiment in which the area between the road shoulders S is estimated as a road surface area, weighting on the road surface R can be performed, and all road surface areas in the three-dimensional image need to be estimated. Therefore, the road surface area estimated from the three-dimensional image can be narrowed. Therefore, the amount of processing when the shoulder detection unit 15 detects the road shoulder S can be reduced, and the obstacle detection area by the obstacle detection unit 17 can be narrowed. Therefore, the processing by the obstacle detection system is further simplified and speeded up. be able to.

[Others]
In addition, this invention is not limited to embodiment mentioned above, Various deformation | transformation aspects are included. For example, the above-described embodiments have been described in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to those having all the configurations described.

  Further, in each of the above embodiments, the vehicle 1 has been described by taking a mine dump truck as an example. For example, the vehicle 1 is provided in a mine in addition to a work vehicle such as a hydraulic excavator 4, a grader, a bulldozer, a load loader, a patrol car, and the like. The obstacle detection system according to the present invention can also be mounted on a mining work vehicle that travels on the road surface R. Moreover, not only the autonomously traveling vehicle 1 but also a manned traveling vehicle 1 that travels by an operator's operation may be used. The obstacle detection system according to each of the above embodiments may be, for example, a vehicle detection system that detects a mine work vehicle (another vehicle) traveling on the road surface R of the mine.

  Furthermore, in each of the above embodiments, all the processing by the obstacle detection system is performed by the obstacle detection device 10 mounted on the vehicle 1, but predetermined information is transmitted and received using the in-vehicle wireless device 5g or the like. It is also possible to adopt a configuration in which at least one of the obstacle detection devices 10 is performed at a place other than the vehicle 1 such as the control center 3.

  In each of the above embodiments, the configuration using the stereo camera device 11 as the external environment measuring unit 12 has been described. However, a 3D laser range finder may be used as long as a three-dimensional stereoscopic image (stereoscopic image) can be acquired. In addition to a combination of a monocular camera and a 2D laser range finder, a monocular camera can be used to acquire a stereoscopic image using a time difference image of the monocular camera, or a sensor such as a laser scanner or millimeter wave sensor that can acquire distance information. The aspect which acquires a stereo image in combination may be sufficient.

  Furthermore, in the second embodiment, for example, information related to traffic classification (travel lane) on the road surface R such as right-hand traffic is stored in the storage unit 6a, and from the three-dimensional image information detected by the external environment measurement unit 12, Based on the information related to the traffic classification stored in the storage unit 6a, only the position of the road shoulder S1 on the road side that is closer to the traveling side of the road surface R is detected, and half of the detected road surface area D2 near the road shoulder S1. It may be configured to determine the obstacle O in the region R1. In this case, even in a mine or the like where the travel lane of the vehicle 1 changes according to the condition of the road surface R, the information related to the travel lane is stored in the storage unit 6a, and the travel lane information stored in the storage unit 6a is stored in the travel lane information. Accordingly, it is possible to detect the road shoulder S1 on the traffic side of the vehicle 1 and detect the obstacle O in the road surface area near the detected road shoulder S1.

1 Vehicle (Mine work vehicle)
10 Obstacle detection device (road surface estimation device)
11 Stereo camera device 11a, 11b Camera 12 External environment measurement unit (image acquisition unit)
13 Distance calculator 14 Global road surface position estimator (planar area estimator)
DESCRIPTION OF SYMBOLS 15 Road shoulder detection part 16 Road surface area estimation part 16a Road surface base area estimation part 16b Surrounding similar area detection part (similar area estimation part)
16c Road surface area complement part 17 Obstacle detection part B Embankment C Cutting surface D1 Plane area D2 Road surface area R Road surface S Road shoulder O Obstacle

Claims (7)

Three-dimensional image information is input, and based on the distance information in the image information, a plane area estimation unit that estimates the plane area in the image information,
Based on the planar region estimated by the planar region estimation unit, a road shoulder detection unit that detects a shoulder position having a height of a predetermined value or more in the image information;
Based on the road shoulder position detected by the road shoulder detection unit, a road surface area estimation unit that estimates a road surface area with the road shoulder as a boundary in the image information;
A road surface estimation device for a mining work vehicle. The road surface estimation device for a mine work vehicle according to claim 1,
The shoulder detection unit detects each shoulder located on both sides of the road surface from the image information,
The road surface area estimation unit estimates a road surface area detected by the road shoulder detection unit as the road surface area. A road surface estimation device for a mining work vehicle. The road surface estimation device for a mining work vehicle according to claim 2,
The three-dimensional image information has luminance information,
The road surface area estimation unit includes a similar area estimation unit that estimates a similar area similar to the road surface area from luminance information in the image information, and adds the similar area to estimate the road surface area. A road surface estimation device for mining work vehicles. The road surface estimation device for a mining work vehicle according to claim 2,
The three-dimensional image information has color information,
The road surface region estimation unit includes a similar region estimation unit that estimates a similar region similar to the road surface region from color information in the image information, and adds the similar region to estimate the road surface region. A road surface estimation device for mining work vehicles. The road surface estimation device for a mine work vehicle according to claim 1,
The road shoulder detection unit detects a road shoulder on the side close to the mining work vehicle from the image information,
The road surface area estimation unit estimates a region within a predetermined range from a road shoulder on the near side detected by the road shoulder detection unit as a road surface area. A road surface estimation apparatus for a mining work vehicle. The road surface estimation device for a mine work vehicle according to claim 1,
A road surface for a mine work vehicle, comprising: an obstacle detection unit that detects an imaging target in the image information that matches a predetermined criterion from the road surface area in the image information as an obstacle. Estimating device. An image acquisition unit for acquiring three-dimensional image information;
A road surface estimating device for a mining work vehicle according to claim 1,
The road surface estimation system for a mining work vehicle, wherein the image information acquired by the image acquisition unit is input to the road surface estimation device.