JP2008040965A - Road surface estimation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a road surface estimation device 3 capable of quickly estimating a road surface with high accuracy. <P>SOLUTION: The road surface estimation device is provided with a three-dimensional object detecting means 12 for detecting a three-dimensional object in front of one's own vehicle 1, a bottom edge position detecting means 14 for detecting the bottom edge position of the detected three-dimensional object, a virtual road surface calculating means 16 for calculating a virtual road surface according to the bottom edge position of the three-dimensional object and the reference position of the one's own vehicle 1, and a real road surface estimating means 18 for estimating a real road surface on the basis of the virtual road surface. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a road surface estimation device.

A technique for capturing an image around a vehicle with a camera and recognizing an object in the image has been developed. In particular, as a premise for recognizing pedestrians and the like existing on the road surface, development of a method for recognizing the road surface position is desired.
Patent Document 1 discloses a method of extracting a road vanishing point where the ends of two road parallel lines intersect and calculating posture parameters such as a yaw angle and a pitch angle of a camera with respect to the road based on the coordinates of the road extraction point. Has been proposed. Further, Non-Patent Document 1 proposes a method of obtaining a projective transformation between images using a stereo moving image and simultaneously extracting a region corresponding to a road plane portion in space.
Japanese Patent No. 2544898 Seki, Okutomi, “Obstacle Detection by Extracting Planar Regions Using Stereo Video”, Transactions of Information Processing Society of Japan: Computer Vision and Image Media, December 2004, Vol. 45, no. SIG13 (CVIM10)

However, although the method of Patent Document 1 has a high processing speed, there is a problem that two road parallel lines are required in the image in order to extract the road vanishing point, and the adaptable scene is limited. Further, although the method of Non-Patent Document 1 can accurately detect a road plane portion, there is a problem that many parameters are optimized and the processing is heavy.
Therefore, an object of the present invention is to provide a road surface estimation apparatus that can quickly and accurately estimate a road surface.

  In order to solve the above-mentioned problem, the invention according to claim 1 is a means (for example, implementation) for detecting a three-dimensional object (for example, the support 50 in the embodiment) in front of the own vehicle (for example, the own vehicle 1 in the embodiment). Three-dimensional object detection means 12) in the form, means for detecting the lower end position of the detected three-dimensional object (for example, the lower end position 54 in the embodiment) (for example, the lower end position detection means 14 in the embodiment), Means (for example, temporary road surface calculation means in the embodiment) for obtaining a temporary road surface (for example, the surface of the temporary pitch angle θn in the embodiment) based on the lower end position and the reference position of the host vehicle (for example, the camera position in the embodiment). 16) and means for estimating the actual road surface (for example, the surface of the actual pitch angle θ in the embodiment) based on the temporary road surface (for example, the actual road surface estimation means 1 in the embodiment) And 8).

  According to the first aspect of the present invention, the temporary road surface can be quickly obtained from the lower end position of the three-dimensional object and the reference position of the host vehicle, and the actual road surface can be accurately estimated even when the host vehicle is inclined. Can do.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic configuration diagram of a road surface estimation apparatus according to the present embodiment. The road surface estimation device 3 includes stereo cameras 5L and 5R for capturing an image ahead of the vehicle. The stereo cameras 5L and 5R are composed of a CCD camera or the like, and are attached to the ceiling portion inside the windshield. The left camera 5L is disposed in the vicinity of the rear-view mirror at the center in the vehicle width direction, and the right camera 5R is disposed at the right end in the vehicle width direction. The right camera 5R may be disposed near the rear-view mirror in the center in the vehicle width direction, and the left camera 5L may be disposed at the left end in the vehicle width direction. The distance between the left camera 5L and the right camera 5R is set to about 30 to 40 cm.

  Images taken by the stereo cameras 5L and 5R are input to the computer 10 mounted on the vehicle. The computer 10 includes a three-dimensional object detection unit 12 that detects a three-dimensional object in front of the host vehicle 1, a lower end position detection unit 14 that detects a lower end position of the detected three-dimensional object, a lower end position of the three-dimensional object, and a position of the camera 5. The temporary road surface calculating means 16 for obtaining the temporary road surface and the actual road surface estimating means 18 for estimating the actual road surface based on the temporary road surface are constructed.

The operation of the road surface estimation device 3 according to this embodiment described above will be described with reference to FIGS.
FIG. 2 is a flowchart of the road surface estimation method. First, images in front of the vehicle are taken by the stereo cameras 5L and 5R (S20).
FIG. 3 is an example of images taken by the stereo cameras 5L and 5R. The captured image is output to the three-dimensional object detection means 12 of the computer 10.

  Next, the vertically long edge in the image is extracted by the three-dimensional object detection means 12 (S22). Specifically, first, the entire image is scanned, the luminance of pixels adjacent in the horizontal direction is compared, and a pair of pixels having a large luminance difference is extracted. Next, the luminance of the high-brightness pixel in the pair of pixels and the pixel adjacent in the lower direction are compared. If these luminance differences are small, it is determined that the high luminance pixel row is continuous in the downward direction, and the luminance is compared with the pixels adjacent in the downward direction. If the luminance difference is large, it is determined that the lower end portion of the high luminance pixel column. If the length of the detected high-luminance pixel row exceeds a predetermined value, the high-luminance pixel row is extracted as a vertically long edge.

  FIG. 4 shows a vertically long edge extracted from the image of FIG. The contour of the guardrail support 50 in the image of FIG. 3 is extracted as a vertically long edge 52 in FIG. The outline of the building 60 in the image of FIG. 3 is also extracted as the vertically long edge 62 in FIG. Further, the outline of the building window 70 in the image of FIG. 3 is also extracted as a vertically long edge 72 in FIG. In FIG. 4, black circles 54 and 64 indicate the lower end of the vertically long edge on the road surface, and white circles 74 indicate those not positioned on the road surface.

Next, the three-dimensional object detection means 12 detects a vertically long edge of the three-dimensional object (S23).
FIG. 5 is an explanatory diagram of a method for detecting a vertically long edge of a three-dimensional object. As shown in FIG. 5A, a case where a planar pattern 90 having a predetermined shape is present in addition to the three-dimensional object 80 will be described as an example. FIG. 5B is an image obtained by photographing the three-dimensional object 80 and the planar pattern 90 with the left camera 5L, and FIG. 5C is an image obtained with the right camera 5R.

The contours 82 and 84 of the three-dimensional object 80 are extracted as vertically elongated edges extending in the vertical direction in both the image of the left camera 5L and the image of the right camera 5R. However, even if the contours 92 and 94 of the planar pattern 90 are extracted as vertically long edges extending in the vertical direction in the image of the left camera 5L, they are extracted as edges extending in the oblique direction in the image of the right camera 5R. Therefore, in both the image of the left camera 5L and the image of the right camera 5R, what is extracted as a vertically long edge having substantially the same shape is detected as a vertically long edge of the three-dimensional object 80.
Specifically, when a vertically long edge is extracted from the image of the left camera 5L, it is determined whether a vertically long edge having substantially the same shape exists at a position in the image of the right camera 5R corresponding to the position of the vertically long edge. Since the positions of the left camera 5L and the right camera 5R are fixed, it is easy to specify corresponding positions in both images. If a vertically long edge having substantially the same shape exists, it is detected as a vertically long edge of the three-dimensional object 80.

  Next, the lower end position detecting means 14 detects the lower end position of the vertically long edge of the three-dimensional object (S24). Here, the position of the lower end pixel of the vertically long edge is detected as the lower end position. Note that the position of the lower end pixel of the high luminance pixel row detected in S22 may be recorded and read as the lower end position of the vertically long edge.

  FIG. 6 is an explanatory diagram of a method for calculating the temporary road surface (temporary pitch angle θn). When the vehicle 1 is stopped, the optical axis of the camera 5 is parallel to the actual road surface. The camera 5 is fixed upward by a distance h from the actual road surface. In this case, a plane that is parallel to the optical axis of the camera 5 and that is positioned below the camera 5 by a distance h is an actual road surface.

However, the vehicle may be inclined at a pitch angle θ due to acceleration / deceleration of the vehicle 1 or unevenness 2 on the actual road surface. Along with this, the optical axis of the camera 5 is also inclined at an angle θ with respect to the actual road surface. In this case, the surface that forms an angle θ with the optical axis of the camera 5 and that is positioned below the camera 5 by a distance h is the actual road surface. That is, the work for obtaining the pitch angle θ of the vehicle coincides with the work for obtaining the position of the actual road surface. Therefore, an operation for obtaining the vehicle pitch angle θ is performed for each image captured by the camera 5.
In the present embodiment, the provisional pitch angle θn of the vehicle 1 is calculated for each vertically long edge in the image, assuming that the lower end of the three-dimensional object corresponding to the vertically long edge is located on the actual road surface. Next, the actual pitch angle θ of the vehicle 1 is estimated based on a plurality of provisional pitch angles θn on the assumption that the probability that the lower end of the three-dimensional object is located most on the actual road surface is large.

  First, a temporary road surface (temporary pitch angle θn) is calculated by the temporary road surface calculation means 16 (S26). The specific procedure is as follows. First, a distance L from the camera 5 to the three-dimensional object 53 is calculated using an image taken with a stereo camera. Next, a distance H from the optical axis of the camera 5 to the lower end position 55 of the three-dimensional object 53 is calculated. Here, if the distance from the optical axis of the camera 5 to the lower end position 54 of the vertically long edge 52 is y, and the distance from the camera 5 to the focal position Fo is f, the distance H can be calculated by Equation 1. .

  Next, a difference ΔH between the distance H from the optical axis of the camera 5 to the lower end position 55 of the three-dimensional object 53 and the height h of the camera is calculated by Equation 2.

  The temporary pitch angle θn is calculated by Equation 3 using ΔH and L.

  Next, an actual road surface (actual pitch angle θ) is estimated by the actual road surface estimation means 18 (S28). As shown in FIGS. 3 and 4, there is a high probability that the lower end position of the three-dimensional object is the most on the actual road surface. Therefore, as a method for estimating the actual pitch angle θ from a plurality of provisional pitch angles θn, a method of adopting the maximum value, average value, or median value of the provisional pitch angles θn can be considered. In the present embodiment, the actual pitch angle θ of the vehicle is estimated by a method (boating) that employs the most frequent value of the temporary pitch angle θn.

FIG. 7 is an explanatory diagram of boarding. First, the range (rank) of the temporary pitch angle θn is set. In FIG. 7, a plurality of ranks having a width of 0.03 rad are set around θn = 0. Next, the temporary pitch angle θn calculated in S26 is distributed (voted) to the set rank. In FIG. 7, the rank of θn = −0.06 to −0.03 rad is the highest-ranked vote rank. Therefore, −0.045 rad, which is the median value of the most frequently obtained vote ranks, is adopted as the actual pitch angle θ as the most frequent value of the temporary pitch angle θn.
A surface that forms an actual pitch angle θ with the optical axis of the camera 5 and that is positioned below the camera 5 by a distance h is estimated to be an actual road surface.

  As described in detail above, the road surface estimation device 3 according to the present embodiment shown in FIG. 1 includes the three-dimensional object detection means 12 that detects a three-dimensional object in front of the host vehicle 1 and the detected lower end position of the three-dimensional object. A lower end position detecting unit 14 for detecting, a temporary road surface calculating unit 16 for obtaining a temporary road surface from the lower end position of the three-dimensional object and the reference position of the host vehicle 1, and an actual road surface estimating unit 18 for estimating the actual road surface based on the temporary road surface. It is characterized by providing. According to this configuration, the lower end position of the three-dimensional object (for example, the lower end position 54 of the vertically long edge 52) and the camera position of the host vehicle 1 (for example, the distance and height h of the stereo camera, the optical axis direction, the focal position Fo, etc.) Thus, the temporary road surface can be easily obtained, and the actual road surface can be accurately estimated even when the host vehicle 1 is inclined.

  In addition, it is desirable to combine a recognition device for pedestrians and obstacles with the road surface estimation device according to the present embodiment. When it is recognized that a pedestrian or the like is present on the actual road surface estimated in the present embodiment, the pedestrian or the like can be cautioned that it may collide with the own vehicle.

The present invention is not limited to the embodiment described above.
For example, in the embodiment, an image in front of the vehicle is captured with a stereo camera, but an actual road surface may be estimated by capturing images on the side of the vehicle or behind the vehicle. In the embodiment, the optical axis of the camera is set to be horizontal with the road surface when the vehicle is stopped. However, it may be set to form a predetermined angle with the road surface.

It is a schematic block diagram of the road surface estimation apparatus which concerns on embodiment. It is a flowchart of a road surface estimation method. It is an example of the image | photographed image. FIG. 4 is a vertical edge extracted from the image of FIG. 3. It is explanatory drawing of the detection method of the longitudinally long edge of a solid object. It is explanatory drawing of the calculation method of a temporary road surface. It is explanatory drawing of boarding.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Own vehicle 3 ... Road surface estimation apparatus 5 ... Camera 10 ... Computer 12 ... Solid object detection means 14 ... Lower end position detection means 16 ... Temporary road surface calculation means 18 ... Actual road surface estimation means

Claims (1)

Means for detecting a three-dimensional object in front of the host vehicle;
Means for detecting a lower end position of the detected three-dimensional object;
Means for obtaining a temporary road surface by a lower end position of the three-dimensional object and a reference position of the host vehicle;
Means for estimating an actual road surface based on the temporary road surface;
A road surface estimation device comprising:

Images