JP5827508B2 - Obstacle detection device for vehicle and vehicle using the same - Google Patents

Description

  The present invention relates to an obstacle detection device for a vehicle that detects an obstacle on a runway with a camera installed in the vehicle, and a vehicle using the same.

  2. Description of the Related Art Conventionally, some golf carts that run on a course in a golf course have an automatic running function. In the automatic traveling, traveling is mainly performed along an electromagnetic induction wire or the like, and there is also an automatic traveling that has a function of detecting an obstacle in front of the cart by providing an ultrasonic sensor on the front surface of the golf cart. When an obstacle is detected by the ultrasonic sensor, the golf cart is controlled to decelerate or stop.

(1) Technology of Patent Document 1 The travel control device described in Patent Document 1 detects a road reference line such as a white line provided on the road, and automatically travels based on the road reference line. A traveling control device that performs quick speed control according to road alignment is described.

JP-A-9-292919

  In the vehicle described in Patent Document 1, it is difficult to specify a road for a road without a white line. For example, in the case of a golf course, there is often no white line at the boundary between a golf course lawn and a golf cart runway. Because the difference in brightness between the lawn and the runway is small, it is difficult to distinguish the runway from the lawn even if the binarization process is performed based on the images taken by the visible light camera. I can't. In addition, there is a problem that it is not possible to determine whether an obstacle is on the runway with an ultrasonic sensor. The ultrasonic sensor may recognize all objects in front of the vehicle as obstacles, and may recognize obstacles up to trees and uphills outside the road.

  The present invention has been made in view of such circumstances, and can identify a road even on a road without a white line, and can detect only an obstacle on the road. An object of the present invention is to provide a detection device and a vehicle using the same.

In order to achieve the above object, the present invention has the following configuration.
That is, the obstacle detection device for a vehicle according to the present invention specifies an infrared camera that captures the front of the vehicle, a stereo camera that captures the front of the vehicle, and a road ahead of the vehicle based on the infrared image captured by the infrared camera. Identification and runway identification unit, and the stereo image creating section that creates a stereo image based on the images taken by the stereo camera, on the basis of the track information identified by the runway identification unit, the track on the stereo image And an obstacle detection device for a vehicle comprising an obstacle detection unit that detects an obstacle on the running road.

  According to the present invention, a runway is specified based on an infrared image, and a region in each image taken by a stereo camera corresponding to the runway of the specified infrared image, that is, a stereo image of a runway in each image taken by the stereo camera. Create An obstacle on the road is detected from the created stereo image. Since the runway is specified by the infrared image, the runway can be detected even if there is no white line on the runway. In addition, since only obstacles on the runway are detected, no obstacles outside the runway are mistaken.

  The runway specifying unit binarizes each pixel into a low-intensity pixel and a high-intensity pixel with reference to a first threshold predetermined for each pixel of the infrared image. And a road estimation unit that estimates a region where a predetermined region and the high luminance pixel overlap as a partial road region, and specifies the temporary road by sequentially labeling the high luminance pixels adjacent to the partial road region It is preferable to have a labeling unit, and a runway correction unit that identifies low-luminance pixels included in the temporary runway and the temporary runway as a runway.

By binarizing each pixel into a low-brightness pixel and a high-brightness pixel with reference to a first threshold value that is predetermined for each pixel in the infrared image, the infrared amount of the infrared image is higher than that of the first threshold value. A large area and a small area can be distinguished. Since the runway is included in the region where the amount of infrared rays is larger than the first threshold and a part of the low- luminance pixel region is the runway, the infrared image may be divided into two parts: the region including the runway and the region not including the runway. it can. An area where the low- luminance pixel area and a predetermined area overlap is estimated as a partial road area. Since the runways exist continuously, low- luminance pixels that are continuous with the partial runway region are also recognized as runways, and are thus sequentially labeled and specified as temporary runways. In addition, since the high- intensity pixels included in the temporary runway are also in the runway region, by specifying the low-intensity pixels and the temporary runway included in the temporary runway as the runway, even the runway without a white line is accurate You can identify the runway well.

  The obstacle detection unit may detect an obstacle by calculating a three-dimensional coordinate only in a region corresponding to the travel path on the stereo image. The accuracy can be improved by using the three-dimensional coordinates for the detection of the obstacle. In addition, it is possible to limit the calculation area of the three-dimensional coordinates to the area corresponding to the road on the stereo image by detecting the obstacle after specifying the road on the stereo image, and reducing the calculation load. Can do.

In addition, an infrared camera that captures the front of the vehicle, a stereo camera that captures the front of the vehicle, a travel path identifying unit that identifies a travel path ahead of the vehicle based on the infrared image captured by the infrared camera, and the stereo camera. In each image, a stereo image creation unit that creates a stereo image of a runway based on an area corresponding to the runway identified on the infrared image, and detects an obstacle on the runway based on the stereo image The obstacle detection device for vehicles provided with the obstacle detection part to perform may be sufficient. By limiting the creation of the stereo image to the road part, the calculation load for creating the stereo image can be reduced, and the obstacle can be detected at a higher speed.

  The stereo image creation unit is photographed by the stereo camera and a runway matching unit that identifies a runway in each image taken by the stereo camera corresponding to the coordinate position of the runway identified based on the infrared image. It is preferable to have a stereo runway image creation unit that creates a stereo image of the runway based on the runway specified in each image.

  Since the runway in each image photographed by the stereo camera is specified using the coordinate position of the runway identified based on the infrared image, the runway on each image photographed by the stereo camera can be easily identified. . Furthermore, since the stereo image is created only in the runway region, the calculation load can be reduced compared to creating a stereo image of the entire image.

  The obstacle detection unit determines a three-dimensional coordinate measurement unit that measures the three-dimensional coordinates of the road based on a stereo image of the road, and determines an obstacle on the road based on the three-dimensional coordinates of the road. It is preferable to have an obstacle discrimination unit. According to this configuration, since the three-dimensional coordinates are measured for each pixel in the traveling area on the stereo image and the obstacle is detected based on the three-dimensional coordinates, the obstacle can be detected with high accuracy.

  The obstacle determination unit calculates a change rate in the vehicle height direction with respect to the depth direction based on the three-dimensional coordinates of the runway, and is set as a change rate in the vehicle height direction with respect to the depth direction. Two threshold values may be compared, and it may be determined that there is an obstacle on the three-dimensional coordinates of the runway, where the rate of change in the vehicle height direction with respect to the depth direction is greater than the second threshold value. According to this configuration, since the obstacle is determined based on the second threshold value that is predetermined with respect to the rate of change in the vehicle height direction with respect to the depth direction, the obstacle can be detected appropriately.

  The stereo image creation unit includes an edge enhancement unit that enhances an edge of a region corresponding to the runway identified based on the infrared image in an image taken by at least one of the stereo cameras. The stereo runway image creation unit creates a stereo image of a point or line in which the identified edge in the runway is emphasized, and the obstacle detection unit creates a three-dimensional image on the point or line in which the edge is emphasized. Based on a three-dimensional coordinate measuring unit for measuring coordinates and a three-dimensional coordinate on the point or line where the edge is emphasized, the rate of change in the vehicle height direction relative to the depth direction is calculated, and the vehicle height direction relative to the depth direction And a predetermined second threshold value are compared, and the change rate in the vehicle height direction relative to the depth direction is larger than the second threshold value on the three-dimensional coordinates of the runway May have an obstacle discriminating unit for discriminating a harmless substance is present.

According to this configuration, in the image taken by at least one camera of the stereo camera, emphasizing the edges of the region corresponding to the run channel identified on the basis of the infrared image, to create a stereo image of the edge enhancement part . Furthermore, the rate of change in the vehicle height direction with respect to the depth direction is calculated based on a point or a three-dimensional coordinate on the line where the edge is emphasized. The rate of change in the vehicle height direction with respect to the depth direction is compared with a predetermined second threshold, and an obstacle is present on the three-dimensional coordinates of the runway where the rate of change in the vehicle height direction with respect to the depth direction is greater than the second threshold. It is determined that it exists. Since the obstacle is identified by creating the stereo image and calculating the rate of change in the vehicle height direction with respect to the depth direction only for the edge emphasis portion, the obstacle detection load can be reduced.

  Moreover, it is preferable that the said infrared camera and the said stereo camera are arrange | positioned on the straight line in the vehicle width direction. According to this configuration, since the infrared camera and the stereo camera are arranged in a straight line in the vehicle width direction, the correspondence between the road on the infrared image and the road on each image taken by the stereo camera is facilitated. be able to.

  The stereo camera preferably includes a plurality of visible light cameras. Since the stereo camera has a plurality of visible light cameras, the resolution of the image by the stereo camera can be increased, and a highly accurate stereo image can be created. As a result, it is possible to obtain the three-dimensional coordinates of the road with high positional accuracy.

  Moreover, it is preferable that the infrared camera is disposed between the cameras of the stereo camera having two visible light cameras. According to this configuration, since the infrared camera is arranged between the cameras of the stereo camera, it is possible to reduce the positional deviation between the runway on the infrared image and the runway on each image taken by the stereo camera.

  Moreover, it is preferable that the imaging region of the infrared camera and the imaging region of at least one camera of the stereo camera are aligned. According to this configuration, since the shooting area of the infrared camera and the shooting area of at least one camera of the stereo camera are aligned, the position of the runway on the infrared image and the runway on each image taken by the stereo camera Matching can be facilitated.

  A vehicle according to the present invention is a vehicle including the vehicle obstacle detection device. By providing the vehicle obstacle detection device, an obstacle on the runway can be detected.

  According to the present invention, a runway is specified based on an infrared image, and a region in each image taken by a stereo camera corresponding to the runway of the specified infrared image, that is, a stereo image of a runway in each image taken by the stereo camera. Create An obstacle on the road is detected from the created stereo image. Since the runway is specified by the infrared image, the runway can be detected even if there is no white line on the runway. In addition, since only obstacles on the runway are detected, no obstacles outside the runway are mistaken.

1 is a front view of a vehicle according to an embodiment. It is a functional block diagram which shows the structure of the vehicle which concerns on an Example. It is a block diagram which shows the structure of the obstruction detection apparatus which concerns on an Example. It is a block diagram which shows the structure of the runway specific | specification part which concerns on Example 1. FIG. It is explanatory drawing which shows the infrared image by which the binarization process which concerns on Example 1 was carried out. It is explanatory drawing which shows the infrared image by which the binarization process which concerns on Example 1 was carried out. It is explanatory drawing which shows estimation of a runway in the binarized image which concerns on Example 1. FIG. It is explanatory drawing which shows the labeled track area which concerns on Example 1. FIG. It is explanatory drawing which shows the corrected track area which concerns on Example 1. FIG. 3 is a block diagram illustrating a configuration of a stereo image creation unit according to Embodiment 1. FIG. It is a block diagram which shows the structure of the obstruction detection part which concerns on an Example. 6 is a flowchart illustrating a flow of obstacle detection according to the first embodiment. 10 is a block diagram illustrating a configuration of a stereo image creation unit according to Embodiment 2. FIG. It is a block diagram which shows the structure of the obstruction detection apparatus which concerns on Example 3 and 4. FIG. It is a block diagram which shows the structure of the obstruction detection part which concerns on Example 3. FIG. It is a block diagram which shows the structure of the obstruction detection part which concerns on Example 4. FIG.

  Embodiment 1 of the present invention will be described below with reference to the drawings. As an embodiment of a vehicle in the present invention, a golf cart that automatically runs is given. In the following description, front and rear and left and right are based on the direction in which the vehicle 1 moves forward.

1. Schematic configuration of vehicle Referring to FIGS. 1 and 2. FIG. 1 is a front view illustrating a schematic configuration of the vehicle 1 according to the embodiment, and FIG. 2 is a functional block diagram illustrating the configuration of the vehicle 1. The vehicle 1 is a golf cart that travels automatically or manually in a golf course. The vehicle 1 can be automatically driven by being guided by electromagnetic waves emitted from a guide wire embedded in the runway. An infrared camera 2 and a stereo camera 3 composed of two visible light cameras 3a and 3b are provided on a straight line in the center of the front surface of the vehicle 1 with the infrared camera 2 in between. The stereo camera 3 may be composed of three or more visible light cameras. The infrared camera 2 and at least the visible light camera 3a, which is a reference camera of the stereo camera 3, are associated with each coordinate position of each image in advance by calibration or the like. Since the infrared camera 2 and the stereo camera 3 are arranged on a straight line, it is possible to easily associate each coordinate position of each image. In addition, since the two visible light cameras 3a and 3b are arranged with the infrared camera 2 in between, it is easy to align the runway on the infrared image and the runway on each image taken by the stereo camera. it can. In addition, since the stereo camera 3 includes the visible light cameras 3a and 3b, it is possible to obtain an image having a higher resolution than that of the infrared camera.

  Further, the vehicle 1 includes an obstacle detection device 4 that detects a road on which the vehicle 1 travels and an obstacle on the road, and a warning output unit that issues a warning to the driver when the obstacle detection device 4 detects an obstacle. 8, a traveling speed control unit 9 that controls deceleration or stop by detecting an obstacle, and a drive motor 10 that drives wheels and whose rotational speed is controlled by the traveling speed control unit 9 are provided. In the present embodiment, the vehicle 1 is driven by a motor, but is not limited thereto, and may be driven by an engine.

2. Configuration of Obstacle Detection Device Next, the configuration of the obstacle detection device provided in the vehicle 1 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a configuration of the obstacle detection apparatus.

  The obstacle detection device 4 specifies the travel path of the vehicle 1 based on the infrared camera 2 that captures an infrared image in front of the vehicle 1, the stereo camera 3 that captures a visible light image in front of the vehicle 1, and the infrared image. A runway identifying unit 5, a stereo image creating unit 6 that creates a stereo image based on a visible light image captured by the stereo camera 3, and an obstacle detecting unit 7 that identifies an obstacle on the runway are provided. The runway identification unit 5, the stereo image creation unit 6, and the obstacle detection unit 7 are composed of a microprocessor and a memory. Next, each component will be described in order.

  Please refer to FIG. FIG. 4 is a block diagram showing the configuration of the runway identification unit 5. The runway specifying unit 5 includes a binarization processing unit 11 that performs binarization processing on the infrared image, a runway estimation unit 12 that estimates a part of the runway in the binarized infrared image, and the estimated runway A labeling unit 13 for labeling images of the same luminance value adjacent to and a runway correction unit 14 for correcting the labeled runway.

  The binarization processing unit 11 binarizes the infrared image captured by the infrared camera 2 with a predetermined threshold value. FIG. 5 is an example of a binarized infrared image. In the infrared image taken by the infrared camera 2, the region that reflects infrared light is white and the region that absorbs infrared light is black. Due to this property, the lawn is white with high brightness, and the runway 25 is black with low brightness. The reason why the lawn appears white is that the chlorophyll contained in the leaves of the plant reflects infrared rays. In addition to the runway 25, the tree trunk 26, the sky 27, the building 28, and the like are shown in black. By binarizing the infrared image with a predetermined threshold, it is possible to divide the infrared image into a high luminance region and a low luminance region (shaded portion in FIG. 5). The high luminance area is an area where lawn and leaves are captured, and the low luminance area includes a runway 25, a tree trunk 26, a sky 27, a building 28, and the like. Note that, for example, if a leaf or lawn falls in the runway 25, it is detected as a high brightness area 29.

  The runway estimation unit 12 estimates a low luminance region near the lower center of the image as a part of the runway. This is because the infrared camera 2 is provided on the front surface of the vehicle 1, and therefore a running path is always included near the lower center of the infrared image captured by the infrared camera 2. Therefore, it is possible to estimate a low-luminance region near the lower center in the binarized infrared image (hereinafter referred to as a binarized image) as a part of the runway.

  6 and 7 are explanatory diagrams showing the estimation of the runway in the binarized image. In the binarized image, when the lower left corner of the image is the origin O of the image, and a pixel in the low luminance area exists in the range Ar of x1 ≦ x ≦ x2 and y1 ≦ y ≦ y2 in the binarized image ( 6), the pixel is estimated as a part 30 of the runway region (FIG. 7).

  The labeling unit 13 sequentially connects (labels) pixels in a low-luminance area that is continuous with the road area estimated by the road estimation unit 12. Since the running road is continuous, all the pixels in the low luminance area that are continuous with the estimated running road area can be used as the running road area. FIG. 8 is an explanatory view showing the labeled runway 31.

  The runway correction unit 14 also corrects the high luminance area included in the labeled runway as the runway. In addition, a low-luminance region that is not continuous with the labeled track is not recognized as a track. That is, the labeled runway 31 in FIG. 8 and the high luminance region 29 surrounded by the runway 31 are newly corrected as the runway 32. Further, it is determined that low luminance regions such as the tree trunk 26, the sky 27, and the building 28 that are not continuous with the labeled runway 31 are not runways (see FIG. 9). From the above, it is possible to accurately identify a runway even on a runway without a white line. The detected coordinate information of the running path 32 is sent to the stereo image creation unit 6.

  Next, the stereo image creation unit will be described with reference to FIG. FIG. 10 is a block diagram showing the configuration of the stereo image creation unit. The stereo image creation unit 6 matches the runway identified on the infrared image with each visible light image to identify the runway on the visible light image, and the runway based on the runway in each visible light image. And a stereo runway image creation unit 17 for creating a stereo image. Stereo images are created by using the image captured by the visible light camera 3a as the left eye image and the image captured by the visible light camera 3b as the right eye image, respectively.

  The runway matching unit 16 specifies the runway on the visible light image by specifying the coordinates of the runway identified on the infrared image on each visible light image captured by the visible light cameras 3a and 3b. That is, the area | region of each visible light image corresponding to the coordinate of the runway 32 on the binarized image sent from the runway correction | amendment part 14 of the runway specific | specification part 5 is pinpointed as a runway. Note that the coordinates of the area that is slightly larger including the runway 32 corrected by the runway correction unit 14 may be used as the coordinates of the runway. The visual field shift between the infrared camera 2 and the visible light cameras 3a and 3b can be absorbed. Image information of the specified road area on the visible light image is sent to the stereo road image creation unit 17.

  The stereo track image creation unit 17 creates a stereo image of the track based on the track on each visible light image. That is, a stereo image having a parallax d (x, y) with each pixel of the runway photographed by the visible light camera 3b corresponding to each pixel of the runway photographed by the visible light camera 3a is created by the stereo matching method. . As a stereo matching method, matching may be performed based on corresponding points between the left-eye image and the right-eye image, or matching may be performed so as to obtain the smallest correspondence relationship of the difference values based on the distribution of luminance values. The created stereo image of the road is sent to the obstacle detection unit 7.

  Next, the obstacle detection unit will be described with reference to FIG. FIG. 11 is a block diagram illustrating a configuration of the obstacle detection unit. The obstacle detection unit 7 includes a three-dimensional coordinate measurement unit 21 that measures a three-dimensional coordinate of a road from a stereo image of the road, and an obstacle determination unit that determines an obstacle on the road based on the measured three-dimensional coordinates. 22.

  The three-dimensional coordinate measuring unit 21 measures the three-dimensional coordinates of each point in the road area based on the stereo image of the specified road area. That is, the three-dimensional coordinates of the runway with the focal point of the visible light camera 3a as the reference camera as the origin, the coordinates (x, y) on the image taken by the visible light camera 3a and the parallax d (x, y) in the stereo image Can be calculated using the following formula.

Z = T · f / d (x, y) (1)
X = (x−x ₀ ) · Z / f (2)
Y = (y−y ₀ ) · Z / f (3)
Here, f is the focal length of the visible light camera 3a, and T is the camera interval of the stereo camera 3 (visible light cameras 3a and 3b). (X _0, y ₀ ) is the center coordinates of the image captured by the visible light camera 3a. Z represents the distance direction (depth direction), X represents the horizontal direction, and Y represents the vertical direction (height direction). As described above, the three-dimensional world coordinates can be measured from the two-dimensional coordinates on the screen.

  The obstacle determination unit 22 determines the presence or absence of an obstacle on the road based on the three-dimensional coordinates in the specified road area. In general, the road on which the vehicle 1 travels has an upper limit on inclination and a gentle change in inclination. Accordingly, it is determined that there is an obstacle when the slope θ exceeds a preset threshold value or when the slope change Δθ exceeds a preset threshold value in the road area. If this condition is not satisfied, it is determined that there is no obstacle.

For example, when an obstacle is determined using the inclination in the Z-Y direction as a determination criterion, it can be determined by the following equation. In the following formula:
u = x−x ₀ , v = y−y ₀ , C = T · f
And β and γ are preset values.

θ = ΔY / ΔZ
= (Y (u, v + 1) -Y (u, v)) / (Z (u, v + 1) -Z (u, v))
= ((V + 1) .C / d (u, v + 1) -v.C / d (u, v)) / (C / d (u, v + 1) -C / d (u, v))
= ((V + 1) / d (u, v + 1) -v / d (u, v)) / (1 / d (u, v + 1) -1 / d (u, v))
> Β (4)

| Δθ |
= (Y (u, v + 2) -Y (u, v + 1)) / (Z (u, v + 2) -Z (u, v + 1))-(Y (u, v + 1) -Y (u, v)) / ( Z (u, v + 1) -Z (u, v))
= ((V + 2) / d (u, v + 2) -v / d (u, v + 1)) / (1 / d (u, v + 2) -1 / d (u, v + 1))-((v + 1) / d ( u, v + 1) -v / d (u, v)) / (1 / d (u, v + 1) -1 / d (u, v))
> Γ (5)

  In the equation (4), the magnitude of the inclination in the Z-Y direction is used as a reference, and in the formula (5), the obstacle is discriminated based on the magnitude of the change in the inclination in the Z-Y direction. Can be detected. When the expression (4) or (5) is satisfied, it is determined that an obstacle exists. When it is determined that an obstacle exists, an obstacle detection signal is sent to the warning output unit 8 and the traveling speed control unit 9. When an obstacle detection signal is sent, the warning output unit 8 issues a warning to the driver that there is an obstacle on the road by voice or video. When the obstacle detection signal is sent, the traveling speed control unit 9 performs deceleration or stop control of the vehicle 1 according to the distance Z to the obstacle.

  Next, the operation for identifying the road and detecting the obstacle in the first embodiment will be described with reference to FIG. FIG. 12 is a flowchart showing a processing procedure for obstacle detection.

  An infrared image is taken by the infrared camera 2 provided on the front surface of the vehicle 1 (step S01). In synchronism with this shooting, a stereo image is shot by the stereo camera 3 (step S02). The captured infrared image is binarized with a predetermined threshold as a reference (step S03). By the binarization processing, an overlapping portion between a preset lower center portion of the image and the low luminance region of the infrared image divided into the high luminance region and the low luminance region is specified as a part of the running region (step S04). A low luminance area continuous with a part of the identified road area is labeled as a road (step S05). The high brightness area included in the labeled runway area is also specified as a runway, and the low brightness area not adjacent to the labeled runway area is not specified as the runway, thereby specifying the runway on the binarized image (step S06). ).

  Next, a runway region on the left eye image and the right eye image captured by the stereo camera 3 corresponding to the runway region specified in the binarized image is specified (step S07). A stereo image having parallax information is created only in the identified road area on the left eye image and the right eye image (step S08). Based on the created stereo image and the focal length of the stereo camera 3, the three-dimensional coordinates of the runway are measured (step S9). From the three-dimensional coordinates of the road, the magnitude and change of the slope of the road are calculated, and by comparing these values with preset threshold values, the presence or absence of an obstacle is determined, and the position of the obstacle is specified. (Step S10).

  According to Example 1, since a runway area is specified by an infrared image, a runway can be detected even if there is no white line on the runway. In addition, since the stereo image is created only for the road area on the infrared image and the three-dimensional coordinate measurement and obstacle detection on the road are performed, the processing speed is improved compared to the process of creating the stereo image for the entire image, It can prevent judging that it is an obstacle with respect to what is outside a runway. Moreover, since the change in inclination is taken into consideration, the uphill is not erroneously detected as an obstacle. Moreover, an inexpensive obstacle detection apparatus can be created with a simple configuration.

The obstacle detection apparatus according to the second embodiment will be described with reference to FIG.
FIG. 13 is a block diagram illustrating a configuration of the stereo image creation unit 6 ′ according to the second embodiment. In the second embodiment, the runway region in which the stereo image creation and the three-dimensional coordinate measurement in the first embodiment are performed is changed. Therefore, the obstacle detection device and the vehicle structure other than those described here are the same as those in the first embodiment. In FIG. 13, the parts denoted by the same reference numerals as those in the first embodiment have the same configuration as that in the first embodiment, and thus the description thereof is omitted here.

  The feature of the second embodiment is that, instead of creating a stereo image in all the runway areas in the first embodiment, the edge emphasis processing of the runway is performed on the left eye image that is the reference image, and the emphasized point or line is This is the point where obstacles are detected by measuring three-dimensional coordinates. This utilizes the fact that the brightness value of the road without an obstacle has a small change, and that the brightness value of the road area with the obstacle has a large change.

  The edge emphasizing unit 41 edge-enhances the corresponding runway region in the left eye image photographed by the visible light camera 3a. For this edge enhancement, for example, a high-pass filter may be implemented. The stereo runway image creation unit 17 ′ uses the edge-enhanced point area and the line partial area from the edge-enhanced runway area image in the left-eye image and the corresponding edge-enhanced area image in the right-eye image. Create a stereo image. The created stereo image is sent to the obstacle detection unit 7 and the three-dimensional coordinate measurement unit 21 measures the three-dimensional coordinates on the point or line where the edge is emphasized. The obstacle discriminating unit 22 calculates the rate of change in the vehicle height direction with respect to the depth direction based on the point or the three-dimensional coordinates on the line in which the edge is emphasized. The obtained second threshold value is compared, and it is determined that there is an obstacle on the three-dimensional coordinates of the road where the rate of change in the vehicle height direction with respect to the depth direction is larger than the second threshold value. According to the second embodiment, since it is not necessary to create a stereo image and measure the three-dimensional coordinates of the entire road area, it is possible to reduce the calculation load for detecting an obstacle.

An obstacle detection apparatus according to the third embodiment will be described with reference to FIGS. 14 and 15.
FIG. 14 is a block diagram illustrating the configuration of the obstacle detection device 54 according to the third embodiment, and FIG. 15 is a block diagram illustrating the configuration of the obstacle detection unit 57 according to the third embodiment. The third embodiment is a modification of the stereo image creation and obstacle detection method in the first embodiment. Therefore, the obstacle detection device and the vehicle structure other than those described here are the same as those in the first embodiment. In FIG. 14 and FIG. 15, since the part shown with the code | symbol same as the code | symbol shown in Example 1 is the structure similar to Example 1, description here is abbreviate | omitted.

  The feature of the third embodiment is that a stereo image of the entire captured image is created from the stereo image created only in the road area in the first embodiment. In connection with this, the output of a runway specific | specification part is changed into an obstacle detection part, and a runway is specified with respect to the produced stereo image.

  The stereo image creation unit 56 creates a stereo image of the entire captured image based on the left eye image and the right eye image input from the visible light cameras 3a and 3b. The created stereo image is sent to the obstacle detection unit 57. Further, the coordinate information of the runway 32 specified by the runway specifying unit 5 is sent to the obstacle detection unit 57.

  The obstacle detection unit 57 matches the runway identified on the infrared image with the created stereo image to identify the runway on the stereo image, and the three-dimensional coordinates of the runway from the stereo image of the runway. It has a three-dimensional coordinate measurement unit 21 for measuring, and an obstacle determination unit 22 for determining an obstacle on the road based on the measured three-dimensional coordinates.

  The runway matching unit 58 specifies the runway on the stereo image by designating the coordinates of the runway specified on the infrared image on the created stereo image. That is, the region of the stereo image corresponding to the coordinates of the runway 32 on the binarized image sent from the runway correction unit 14 of the runway specifying unit 5 is specified as the runway. Note that the coordinates of the area that is slightly larger including the runway 32 corrected by the runway correction unit 14 may be used as the coordinates of the runway.

  Even when the obstacle detection device of the third embodiment is used, since the runway is specified by the infrared image, the runway can be detected even if there is no white line on the runway. In addition, since only obstacles on the runway are detected, no obstacles outside the runway are mistaken.

An obstacle detection apparatus according to the fourth embodiment will be described with reference to FIGS. 14 and 16.
FIG. 16 is a block diagram illustrating the configuration of the obstacle detection unit 67 according to the fourth embodiment. The fourth embodiment is a further modification of the obstacle detection method in the third embodiment. Therefore, the obstacle detection device and the vehicle structure other than those described here are the same as those in the third embodiment. In FIG. 16, the portions denoted by the same reference numerals as those in the third embodiment or the first embodiment have the same configuration as the third embodiment or the first embodiment, and thus the description thereof is omitted here.

  The feature of the fourth embodiment is that the travel path is specified for the stereo image of the entire captured image in the third embodiment, but in the fourth embodiment, the three-dimensional coordinates of the entire captured image are obtained based on the stereo image of the entire captured image. It is a point to measure and to identify the runway with respect to the three-dimensional coordinates of the entire photographed image.

  The obstacle detection unit 67 matches the three-dimensional coordinate measuring unit 61 that measures the three-dimensional coordinates of the entire image from the stereo image of the entire image, and the three-dimensional coordinates created by matching the travel path specified on the infrared image. A runway matching unit 68 that identifies a runway on a dimensional coordinate and an obstacle determination unit 22 that determines an obstacle on the runway based on the measured three-dimensional coordinates.

  The three-dimensional coordinate measuring unit 61 measures the three-dimensional coordinates of each point on the image based on the created stereo image of the entire image. The method for measuring the three-dimensional coordinates is the same as in the first embodiment. The runway matching unit 68 specifies the runway on the three-dimensional coordinates by designating the coordinates of the runway specified on the infrared image on the created three-dimensional coordinates. That is, the area on the three-dimensional coordinate corresponding to the coordinates of the runway 32 on the binarized image sent from the runway correction unit 14 of the runway specifying unit 5 is specified as the runway. Note that the coordinates of the area that is slightly larger including the runway 32 corrected by the runway correction unit 14 may be used as the coordinates of the runway.

  Even when the obstacle detection device of the fourth embodiment is used, the runway is specified by the infrared image, so that the runway can be detected even if there is no white line on the runway. In addition, since only obstacles on the runway are detected, no obstacles outside the runway are mistaken.

  The present invention is not limited to the above embodiment, and can be modified as follows.

  (1) In the above embodiment, a small noise may be removed by performing a noise removal filter on the infrared image after the binarization process. An expansion-contraction algorithm can be employed as the noise removal filter. By removing the small noise, the distinction between the low luminance region and the high luminance region is clarified.

  (2) In the above embodiment, the inclination is calculated from the coordinates of two points, but may be calculated from the coordinates of three or more points. Similarly, the inclination change may be calculated from four or more coordinates. In addition, the inclination used for discrimination of the obstacle and the calculation of the inclination change do not necessarily have to be performed on adjacent points in the image.

  (3) In the above embodiment, the obstacle is determined from one inclination or one inclination change. However, the obstacle is determined in consideration of a plurality of inclinations or inclination changes. Also good.

  (4) In the above embodiment, the obstacle is determined from the inclination in the ZY direction and the change in inclination, but the XY direction may be used. Moreover, you may discriminate | determine an obstruction from the inclination of several directions, and inclination change.

  (5) In the above embodiment, the obstacle is determined from the inclination in the Z-Y direction and the change in inclination. However, the obstacle may be determined only from the change in inclination.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 2 ... Infrared camera 3 ... Stereo camera 3a, 3b ... Visible light camera 4 ... Obstacle detection apparatus 5 ... Runway identification part 6 ... Stereo image creation part 11 ... Binarization process part 12 ... Runway estimation part 13 ... Labeling Unit 14 ... Runway correction unit 16 ... Runway matching unit 17 ... Stereo runway image creation unit 21 ... Three-dimensional coordinate measurement unit 22 ... Obstacle discrimination unit

Claims (13)

An infrared camera for photographing the front of the vehicle;
A stereo camera for photographing the front of the vehicle;
A road identification unit that identifies the road ahead of the vehicle based on the infrared image captured by the infrared camera;
A stereo image creation unit for creating a stereo image based on each image captured by the stereo camera;
An obstacle detection device for a vehicle, comprising: an obstacle detection unit that identifies a road on the stereo image and detects an obstacle on the road based on the road information specified by the road identification unit. An infrared camera for photographing the front of the vehicle;
A stereo camera for photographing the front of the vehicle;
A road identification unit that identifies the road ahead of the vehicle based on the infrared image captured by the infrared camera;
In each image taken by the stereo camera, a stereo image creation unit that creates a stereo image of a runway based on an area corresponding to the runway specified on the infrared image;
An obstacle detection unit for detecting an obstacle on the road based on the stereo image;
An obstacle detection device for a vehicle comprising: In the obstacle detection device for vehicles according to claim 1 or 2 ,
The runway identification unit is
A binarization processing unit that binarizes each of the pixels into a low-intensity pixel and a high-intensity pixel with reference to a predetermined first threshold for each pixel of the infrared image;
A runway estimation unit that estimates a region where a predetermined region and the low luminance pixel overlap as a partial runway region;
A labeling unit that sequentially identifies the low- luminance pixels adjacent to the partial runway region and identifies a temporary runway;
An obstacle detection device for a vehicle, comprising: a high- intensity pixel included in the temporary runway and a runway correction unit that identifies the temporary runway as a runway. The vehicle obstacle detection device according to claim 1 ,
The obstacle detection unit is
An obstacle detection device for a vehicle that detects an obstacle by calculating a three-dimensional coordinate only in an area corresponding to the road on the stereo image. The vehicle obstacle detection device according to claim 2 ,
The stereo image creation unit
A runway matching unit that identifies a runway in each image captured by the stereo camera corresponding to the coordinate position of the runway identified based on the infrared image;
A vehicle obstacle detection device comprising: a stereo running road image creating unit that creates a stereo image of the running road based on the running road specified in each image photographed by the stereo camera. In the obstacle detection device for vehicles according to claim 5,
The obstacle detection unit is
A three-dimensional coordinate measuring unit that measures the three-dimensional coordinates of the road based on a stereo image of the road;
An obstacle detection device for a vehicle, comprising: an obstacle determination unit that determines an obstacle on the road based on the three-dimensional coordinates of the road. The vehicle obstacle detection device according to claim 6,
The obstacle determination unit
Based on the three-dimensional coordinates of the runway, the rate of change in the vehicle height direction with respect to the depth direction is calculated,
The change rate in the vehicle height direction with respect to the depth direction is compared with a predetermined second threshold, and the change rate in the vehicle height direction with respect to the depth direction is larger than the second threshold on the three-dimensional coordinates of the runway An obstacle detection device for a vehicle that determines that there is an obstacle. In the obstacle detection device for vehicles according to claim 5,
The stereo image creation unit
An edge enhancement unit that emphasizes an edge of an area corresponding to the runway identified based on the infrared image in an image captured by at least one camera of the stereo camera;
The stereo runway image creation unit creates a stereo image of a point or line with emphasized edges in the identified runway,
The obstacle detection unit includes a three-dimensional coordinate measurement unit that measures a three-dimensional coordinate on a point or line in which the edge is emphasized,
A rate of change in the vehicle height direction with respect to the depth direction is calculated based on a point or a three-dimensional coordinate on the line where the edge is emphasized, and a rate of change in the vehicle height direction with respect to the depth direction and a predetermined second threshold value And an obstacle determination unit that determines that there is an obstacle on the three-dimensional coordinates of the runway having a change rate in the vehicle height direction with respect to the depth direction larger than the second threshold. Detection device. In the obstacle detection device for vehicles according to any one of claims 1 to 8,
The vehicle obstacle detection device, wherein the infrared camera and the stereo camera are arranged in a straight line in a vehicle width direction. The vehicle obstacle detection device according to any one of claims 1 to 9, wherein the stereo camera includes a plurality of visible light cameras. In the vehicle obstacle detection device according to any one of claims 1 to 10,
An obstacle detection device for a vehicle, wherein the infrared camera is disposed between the cameras of the stereo camera having two visible light cameras. In the obstacle detection device for vehicles according to any one of claims 1 to 11,
An obstacle detection device for a vehicle, wherein a shooting area of the infrared camera and a shooting area of at least one camera of the stereo camera are aligned.   A vehicle comprising the vehicle obstacle detection device according to any one of claims 1 to 12.

Images