JP2017156965A - Road region detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road region detection device capable of correcting a range of a road region into an original region while preventing an obstacle region at a distant side of a road from being detected erroneously as a road region in a road where a road grade is decreased at the distant side.SOLUTION: A road region detection method includes: calculating a parallax differential value Δdisp obtained by subtracting a parallax value projected on a V-disparity plane from a parallax value of a created road model; and extracting a locus (waveform part) of the parallax differential value Δdisp. Further, a V coordinate position of a disappearance point is calculated from an image. If it is discriminated that an area of a graphic enclosed by a waveform part closest to the V coordinate position of the disappearance point among multiple waveform parts and a V coordinate axis is greater than an area of a graphic in any other interval and a maximum parallax differential value in the locus is equal to or greater than a predetermined threshold, a far end of a road region estimated based on the road model is changed into a maximum end side of a V coordinate position of the waveform part closest to the V coordinate position of the disappearance point.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a road area detection device that detects a road area from an image acquired by an image acquisition means.

  A road area detection device applied to a driver assistance system such as collision avoidance to an obstacle and ACC (Adaptive Cruise Control) needs to identify the road area ahead of the vehicle and the obstacle as accurately as possible.

  Therefore, one of the conventional road area detection devices (hereinafter referred to as “conventional device”) is a parallax diagram in which the disparity for each pixel calculated from the captured image is expressed on the U-V plane. Use to create a U-disparity map (U-disparity plane). Then, the conventional apparatus searches the left and right road boundaries (edges such as white lines and guardrails) for each row from the lower end to the upper end of the U-disparity plane. Furthermore, the conventional apparatus detects a road area between the left and right road boundaries searched on the U-disparity plane (see, for example, Patent Document 1).

  More specifically, the U-disparity plane is an image in which the horizontal axis corresponds to the U coordinate axis, which is the horizontal axis of the parallax diagram, and the vertical axis corresponds to the disparity. Each pixel value on the U-disparity plane expresses the frequency of parallax existing in a plurality of minute sections formed by dividing a parallax diagram by a plurality of virtual lines parallel to the vertical axis by luminance (gradation). According to the example shown in FIG. 15A, the point corresponding to the road boundary (edge) 81 has a relatively high brightness on the U-disparity plane when the vehicle is traveling on a flat road, The points corresponding to the possible area (ie road area) 82 have a relatively low brightness. Therefore, the conventional device detects the left-right edge search 83 on the U-disparity plane as a road region.

JP 2014-67407 A

  However, for example, when a vehicle is traveling on a flat road with a distant slope on the far side, the distant road area is not visible on the image, and obstacles far away on the road (for example, guardrails and fences) May exist in the “region that is a road region” (see region 84 in FIG. 15B). In this case, the disparity of the road far away is different from the disparity when the road is flat far away. This difference also appears on the U-disparity plane, and in the case of a flat road with a distant slope on the far side, as shown in FIG. 15C, an area 85 showing relatively high luminance is shown on the far road area. appear. However, the conventional apparatus cannot distinguish between these parallax differences, and as a result, an area where an obstacle exists (hereinafter also referred to as an “obstacle area”) is referred to as a road area (that is, a downhill area). There was a problem that the road was mistakenly detected as a flat road. This problem is not limited to the case where the road changes from a flat road to a downhill, but applies to all cases where the road slope decreases if the slope of the uphill is positive and the slope of the downhill is negative.

  The present invention has been made to address the above problems. That is, one of the objects of the present invention is to erroneously detect an obstacle area far from the road as a road area on a road where the slope of the road far away from the road slope of the road on which the vehicle is currently traveling decreases. Therefore, an object of the present invention is to provide a road area detection device that can correct the range of the road area to the original range and thus can correctly recognize an obstacle.

Therefore, the road area detection device of the present invention (hereinafter also referred to as “the present device”)
Image acquisition means (20), parallax image creation means, V-parallax plane creation means, road model creation means, and far end position determination means are provided.

  The image acquisition means acquires a stereo image by the left camera (20L) and the right camera (20R), both of which capture the road surface ahead of the vehicle (step 310).

  The parallax image creation means calculates a parallax value of each point on the stereo image from the stereo image, and arranges the parallax value on an image having the same V coordinate axis and U coordinate axis as the stereo image (40 ) Is created (step 320).

  The V-parallax plane creating means subdivides the parallax image by a plurality of virtual lines (43) parallel to a V coordinate axis that is a vertical axis of the parallax image to obtain a plurality of minute sections (dU), By projecting each parallax value of the plurality of minute sections onto a V-disparity plane having the V coordinate axis of the parallax image as a vertical axis and the parallax values of the minute sections as a horizontal axis, a plurality of V Create a parallax plane (50) (step 330).

  The road model creation means extracts a main straight line as a road model from the parallax values projected onto each of the plurality of V-parallax planes, thereby obtaining a road model (53 ) Is created (step 340).

  The far end position determining means estimates the far end of each road area of the minute section based on the road model (step 350, routine of FIG. 11), and each V of the plurality of V-parallax planes. At the coordinate position, a parallax difference value (Δdisp) which is a value obtained by subtracting the projected parallax value from the parallax value of the created road model is calculated (step 1220).

  Further, the far end position determining means projects the parallax difference value onto a V-parallax difference value plane (60) having the V coordinate axis as a vertical axis and the parallax difference value as a horizontal axis. A trajectory of a parallax difference value existing between two adjacent points having a value of 0 and showing a positive value is recognized as one waveform portion (61) (step 360, step 1230, step 1280).

  In addition, the far end position determining means calculates the V coordinate position of the vanishing point (96) from one of the acquired stereo images (step 1310), and sets the V coordinate position of the vanishing point in the waveform portion. The area of the figure surrounded by the nearest waveform part (61a) and the V coordinate axis is larger than the area of the figure surrounded by any other waveform part and the V coordinate axis, and the vanishing point V It is determined whether or not a specific condition that the maximum parallax difference value (ΔdispMAX) in the waveform portion closest to the coordinate position is equal to or larger than a predetermined threshold (Δdispth) is satisfied (steps 370, 1320, and 1330).

  Further, the far end position determining means sets the far end of the road region to the estimated far end (step 1410: No) when the specific condition is not satisfied (step 1330: No), and the specific condition Is established (step 1330: Yes), the far end of the road region is changed to the maximum value (63) of the V coordinate position of the waveform portion closest to the V coordinate position of the vanishing point (in step 1410, “ Yes ", step 1420).

  On a road where the gradient of the road surface far away from the gradient of the road surface on which the vehicle is currently traveling is decreasing, see the road model created above (see the broken line 91 in FIG. 16A). ) And the parallax value (see the solid line 92 in the figure), the “parallax difference value” is relatively small on the road surface on which the vehicle is currently traveling (see the region 93 in the figure). ), There is a tendency to be relatively large (see the region 94 in the figure) on a distant road surface.

  On the other hand, for example, when edges on the image (white road lines, guard rails on the road side, etc.) are extracted and there are a plurality of extension lines connecting these edges, the intersection of these extension lines is defined as the vanishing point 96. In the case of the road, as shown in FIG. 16B, the road surface height 95 far from the road exists at a position lower than the height of the vanishing point 96.

  Therefore, the inventor found that the boundary between the “range where the parallax difference value is relatively small” and the “range where the parallax difference value is relatively large” substantially coincides with the far end of the original road area (the upper end of the road area on the image plane). Focusing on this, the following method for specifying a “range where the parallax difference value is relatively large” was conceived.

  In the method, first, the obtained parallax difference value is projected onto the V-parallax difference value plane, and the locus of the parallax difference value existing between two adjacent points where the parallax difference value becomes 0 is a positive value. Is recognized as one waveform portion. When a plurality of waveform portions are obtained, it is estimated that the waveform portion closest to the V coordinate position of the vanishing point represents “the degree of deviation between the actual road region and the road region based on the road model”. Next, the figure surrounded by the waveform portion and the V coordinate axis is a predetermined size (larger than the area of the figure surrounded by any other waveform portion and the V coordinate axis, and the vanishing point V coordinate. If the maximum parallax difference value in the waveform portion closest to the position has a predetermined threshold value or more, it is determined that the above estimation is likely.

  Thereby, according to the device of the present invention, a range in which the parallax difference value is relatively large is specified, and the maximum end (image plane) of the locus (waveform portion) of the parallax difference value corresponding to the range of the parallax difference value (image plane) Can be estimated to be the far end of the original road area (the upper end in the image plane).

  As described above, according to the device of the present invention, the obstacle area far from the road is erroneously detected as the road area on the road where the slope of the road surface far away from the slope of the road surface on which the vehicle is currently traveling is reduced. The range of the road area can be corrected to the original range. As a result, according to the method of the present invention, an obstacle can be correctly recognized.

  In the above description, in order to help understanding of the present invention, names and / or symbols used in the embodiment are attached to the configuration of the invention corresponding to the embodiment described later in parentheses. However, each component of the present invention is not limited to the embodiment defined by the reference numerals. Other objects, other features and attendant advantages of the present invention will be readily understood from the description of the embodiments of the present invention described with reference to the following drawings.

FIG. 1 is a schematic configuration diagram of a road area detection device according to an embodiment of the present invention. FIG. 2 is an example of a stereo camera image taken (acquired) by the camera shown in FIG. FIG. 3 is a flowchart showing a “road area detection / correction routine” executed by the CPU of the road area detection apparatus shown in FIG. FIG. 4 is an example of a parallax image created by the road area detection apparatus shown in FIG. 1, where FIG. 4A shows a flat road, and FIG. 4B shows a road whose slope decreases at a distance. It is an example of the parallax image in a case. FIG. 5 is an example of a V-parallax plane obtained by calculation by the road area detection apparatus shown in FIG. 1, FIG. 5 (A) is an example of a parallax image, and FIG. 5 (B) is the vicinity of the center of the parallax image. FIG. 5C shows an example of the V-parallax plane corresponding to the minute section near the middle between the center and the right end of the parallax image. FIG. 6 is an example of a V-parallax plane obtained by calculation by the road region detection apparatus shown in FIG. 1, and is a diagram showing an example of a road where the road gradient decreases far away. FIG. 7 is a diagram for explaining a road area estimation method performed by the road area detection apparatus shown in FIG. 1. FIG. 8A is a diagram for explaining a trajectory (waveform portion) formed by projecting the parallax difference value calculated by the road area detection apparatus shown in FIG. 1 on the V-parallax difference value plane. (B) is a figure for demonstrating the relationship between a waveform part and the V coordinate position of a vanishing point. FIG. 9 is a diagram for explaining a vanishing point calculation method performed by the road area detection apparatus shown in FIG. 1. FIG. 10 is a diagram for explaining a road region far end position changing method performed by the road region detection apparatus shown in FIG. FIG. 11 is a flowchart showing a “road area estimation routine” executed by the CPU of the road area detection apparatus shown in FIG. FIG. 12 is a flowchart showing a “parallax difference value trajectory (waveform portion) extraction routine” executed by the CPU of the road area detection apparatus shown in FIG. 1. FIG. 13 is a flowchart showing a “gradient reduction presence / absence determination routine” executed by the CPU of the road area detection apparatus shown in FIG. FIG. 14 is a flowchart showing a “road region far end position change routine” executed by the CPU of the road region detection apparatus shown in FIG. 15A is an example of a U-disparity diagram, FIG. 15B is an example of an image in which a distant road cannot be seen, and FIG. 15C is an U-disparity diagram of the image of FIG. It is the figure which each showed the example converted into. FIG. 16A is a diagram illustrating an example of a V-parallax diagram, and FIG. 16B is a diagram illustrating an example of how to obtain a vanishing point.

  Hereinafter, a road area detection device (hereinafter also referred to as “the detection device”) according to an embodiment of the present invention will be described with reference to the drawings.

(Constitution)
As shown in FIG. 1, this detection apparatus is applied to the vehicle 10 shown in FIG. The present detection device includes a camera device 20 and an electronic control device 30.

  The camera device 20 includes a pair of left and right stereo cameras (a left camera 20L and a right camera 20R). The camera device 20 is fixed to the vehicle body 18 so that the left camera 20L and the right camera 20R are positioned at both ends of an inner rear view mirror (so-called room mirror) 22, respectively. The optical axis 24L of the left camera 20L has a predetermined depression angle φ in the state where the camera device 20 is fixed to the vehicle body 18 and coincides with the vehicle body longitudinal direction. Similarly, the optical axis 24R of the right camera 20R has a predetermined depression angle φ when the camera device 20 is fixed to the vehicle body 18 and coincides with the vehicle body longitudinal direction. Furthermore, a straight line connecting the center of the image sensor of the left camera 20L and the center of the image sensor of the right camera 20R is parallel to the lateral width direction of the vehicle body 18. Accordingly, each of the left camera 20L and the right camera 20R can capture a road surface in the vehicle traveling direction (vehicle front). Note that the imaging devices of the left camera 20L and the right camera 20R are solid-state imaging devices such as a CCD and a CMOS.

  More specifically, each of the left camera 20L and the right camera 20R is electrically connected to the electronic control unit 30 and is separated from the vehicle 10 by a predetermined distance forward according to an instruction from the electronic control unit 30. The road surface to a distant position can be imaged. The images captured by the left camera 20L and the right camera 20R constitute a pair of left and right images as shown in FIG. Hereinafter, this set of images is also referred to as a “stereo image”.

  The electronic control unit (ECU) 30 is an electronic circuit including a known microcomputer, and includes a CPU, a ROM, a RAM, an interface I / F, and the like. ECU is an abbreviation for electric control unit. The CPU implements various functions to be described later by executing instructions (routines) stored in a memory (ROM).

(Operation)
Next, the operation of this detection apparatus will be described.

<Overall flow>
The CPU of the electronic control unit 30 executes a “road area detection / correction routine” shown by a flowchart in FIG. 3 every time a predetermined time (for example, 100 ms) elapses. Accordingly, the CPU starts the process from step 300 at a predetermined timing, sequentially executes the processes of steps 310 to 380 described below, proceeds to step 395, and once ends this routine.

  Step 310: The CPU causes the camera device 20 to take an image and receives a set of images (that is, a stereo image) obtained thereby.

  Step 320: The CPU creates a parallax image from the received set of images.

  More specifically, the CPU first calculates a parallax based on a received set of images by a block matching method which is one of well-known stereo matching methods. That is, the CPU divides one image (for example, an image (right image) taken by the right camera 20R) of the received set of images into small square or rectangular areas, and each divided area. The position where the target being photographed is photographed in the corresponding region of the other image (for example, the image photographed by the left camera 20L (left image)) is determined, and the right image and the left are determined based on the position. The amount of positional deviation between images is calculated as parallax. For example, as shown in FIG. 2, the x coordinate position of the sign M1 installed on the road side is x1 in the right image and (x1 + d) in the left image. Accordingly, the parallax is a difference (positional deviation) d in the horizontal direction (x-axis direction).

  Next, the CPU creates a parallax image using the calculated parallax. The parallax image is an image on the U-V plane in which the horizontal axis is the U coordinate axis and the vertical axis is the V coordinate axis, and the parallax image at each (U, V) coordinate is expressed by luminance (gradation). .

  FIG. 4A shows an example of a parallax image obtained when the road is flat far away. In the parallax image 40, the area 41a in the vicinity of the vehicle 10 in the road area 41 is a bright image because the parallax is relatively large, and the distant area 41b is a dark image because the parallax is relatively small. Furthermore, an area 42 outside the both ends of the road area 41 is an obstacle area 42 such as a roadside guardrail and a fence, and is expressed brighter than the road area 41. This is because the obstacle region 42 tends to be closer to the vehicle 10 when the distance relationship between the road region 41 and the obstacle region 42 and the vehicle 10 at the same V coordinate position is compared. That is, at the same V coordinate position, the parallax of the obstacle area 42 tends to be larger than the parallax of the road area 41.

  On the other hand, FIG. 4B is an example of the parallax image 40 obtained when the road gradient decreases far away. When the road gradient decreases far away, the parallax is very small in the area far from the point where the road disappears from the image. Therefore, in the parallax image 40 of FIG. 4B, the region 41c far from the change point of the road gradient is darker than the example (41b) shown in FIG.

  Step 330: The CPU creates a V-disparity plane 50 from the parallax image 40 obtained in step 330, as shown in FIG.

  More specifically, as shown in FIG. 5A, the CPU subdivides the parallax image 40 created in step 320 by “a plurality of virtual lines 43 parallel to the V coordinate axis (vertical axis)”. Thus, a minute interval dU is obtained. The interval width of this minute interval dU is suitably 3 to 5 pixels. The CPU further projects the parallax values existing in each of the plurality of minute sections dU divided by the virtual line 43 onto the V-parallax plane (hereinafter also referred to as “V-disparity plane”). . The V-disparity plane is a plane for each of the minute sections dU, the vertical axis is the same as the V coordinate axis of the parallax image (that is, the vertical axis is the same as the vertical axis of the parallax image), and the horizontal axis is the minute section. This is a disparity value in dU. The parallax value 51 projected on the V-disparity plane 50 is also referred to as a parallax point group 51 hereinafter.

  In the minute section dU corresponding to the V-disparity plane 50 shown in FIG. 5B, there is no obstacle area 42 (guardrail, fence, etc.) far to the road, so that the parallax points projected on the V-disparity plane 50 The group 51 extends substantially linearly from the parallax d0 to the region where the parallax is very small, and then follows the V axis in the region where the parallax is substantially zero. On the other hand, the obstacle region 42 exists in the minute section dU corresponding to the V-disparity plane 50 shown in FIG. Since the obstacle region 42 has a substantially constant parallax value, the parallax point group is a straight line 52 extending in a direction parallel to the V coordinate axis.

  Further, in the case of a road whose gradient decreases far away, the parallax point group 51 is a change point of the road gradient in the V-disparity plane 50 for the minute section dU near the center of the parallax image 40 shown in FIG. The inclination is suddenly changed at (see point P1 in FIG. 6B). This is because the change in the parallax with respect to the change in the V coordinate becomes large in an area far from the change point of the road gradient (an area where the parallax is small).

  Step 340: The CPU creates a road model based on the parallax values projected on the plurality of V-disparity planes.

  More specifically, the CPU first extracts “main straight lines” existing on the V-disparity plane 50 corresponding to each minute section dU. The “main straight line” is the longest straight line in each V-disparity plane 50 except for a straight line parallel to the V coordinate axis. As described above, the parallax point group 51 of the road area projected onto the V-disparity plane 50 is substantially linear. The projected parallax point group (straight line) 51 of the road area becomes the “longest straight line” of the V-disparity plane 50, that is, the “main straight line”.

  The extraction of the main line is performed using, for example, the Hough transform, which is a classic line extraction method. In the Hough transform, the length of the perpendicular connecting the straight line obtained by approximating the parallax point group 51 on the V-disparity plane 50 and the origin of the V-disparity plane is ρ, and the angle between the perpendicular and the V coordinate axis is θ. Then, the combination of ρ and θ corresponding to the approximate straight line is voted on the ρ-θ plane (orthogonal coordinates). The CPU extracts ρ and θ (one point on the ρ-θ plane) having the largest vote value by the Hough transform for each minute section dU as “main straight lines”. Then, the CPU creates each extracted “main straight line” as a “road model” on the V-disparity plane 50 of each minute section dU.

  Step 350: The CPU compares the road model created in Step 340 with the parallax point group 51 (projected on the V-disparity plane 50) calculated in Step 330 for each minute section dU. Estimate the road area.

  More specifically, as shown in FIG. 7, the CPU writes the road model 53 and the parallax point group 51 created on the V-disparity plane 50 together. The CPU further adds a “height threshold ΔVth” to the road model 53 that increases as the parallax increases (subtracts the “height threshold ΔVth” from the road model 53 on the V coordinate axis). The threshold value Vth is obtained, and this threshold value Vth is also written. In this example, the height threshold ΔVth is a value that increases linearly with respect to the parallax.

  When the upper end 52a of the obstacle 52 exceeds the road area determination threshold value Vth, the CPU estimates the area 54 below the lower end 52b of the obstacle 52 as the road area. The road area determination threshold value Vth is set to a value that allows recognition of normal guardrails, fences, and the like installed on the roadside as obstacles.

  Step 360: The CPU obtains a “parallax difference value” that is a value obtained by subtracting the parallax value of the parallax point group 51 from the parallax value of the road model at each V coordinate of the V-disparity plane 50 corresponding to each minute section dU. calculate.

More specifically, for example, when the coordinates of the upper end of the V coordinate axis are 0 and the coordinates of the lower end are (j−1) (j is a positive integer), the CPU follows each equation (1) below. The “parallax difference value Δdisp (V)” of the V-disparity plane corresponding to is calculated from 0 to (j−1) and stored in the RAM.

Δdisp (V) = dispmodel (V) −dispact (V) (1)

In equation (1), dispmodel (V) is the parallax value of the road model at the coordinate V, and dispense (V) is the parallax value of the parallax point group 51 at the coordinate V.

  As shown in FIG. 8A, the CPU further calculates a parallax difference among the trajectories drawn by projecting the parallax difference value Δdisp (V) for each coordinate V stored in the RAM onto the V-parallax difference value plane 60. A trajectory 61 between “two adjacent points” having a value Δdisp (V) of “0” and having a positive value is recognized as one “waveform portion”.

  Step 370: The CPU calculates the height (V coordinate position) of the vanishing point from the image input at step 310. The vanishing point is a point in the straight road image captured by the camera device 20 that becomes smaller as the road width goes farther and eventually converges to one point.

  For example, the position of the vanishing point is calculated using a known edge extraction method. More specifically, as shown in FIG. 9, the CPU extracts a plurality of linearly continuous edge groups such as white lines, guard rails, and fences in the image captured by the left camera 20L. The CPU calculates an intersection of a plurality of straight lines formed by connecting these edge groups as a vanishing point 96 and obtains the vanishing point height (V coordinate position). Note that an image captured by the right camera 20R may be used to calculate the vanishing point 96. This is because the coordinate U in the U coordinate axis direction of the vanishing point 96 does not match between the captured image by the left camera 20L and the captured image by the right camera 20R, but the coordinate V in the V coordinate axis direction matches.

  Next, as shown in FIG. 8B, the CPU uses the “parallax difference value trajectory (waveform portion) 61” in each minute section dU calculated in step 360 as shown in FIG. It is determined whether or not it is decreasing with respect to a nearby road gradient (whether or not the road gradient is decreased).

  More specifically, the CPU sets the waveform portion 61 until the parallax difference value Δdisp (V) changes from “0” to a positive value and then becomes “0” again (61a, 61b, 61c,...). And the waveform portion closest to the V coordinate position 62 of the vanishing point 96 is extracted from the waveform portion. In this example, the waveform portion closest to the V coordinate position 62 of the vanishing point 96 is 61a. When a plurality of waveform portions 61 are obtained as in this example, the waveform portion closest to the V coordinate position 62 of the vanishing point 96 indicates the “degree of divergence between the actual road region and the road region based on the road model”. Presumed to represent.

Next, the CPU determines whether or not the waveform unit 61a satisfies the following two conditions simultaneously. The CPU determines that the above estimation is likely if the waveform section 61a satisfies the following two conditions simultaneously.
(Condition 1) The area of the figure surrounded by the waveform portion 61 and the V coordinate axis is larger than the area of the figure surrounded by any other waveform portion and the V coordinate axis.
(Condition 2) The maximum parallax difference value ΔdispMAX in the parallax difference value locus 61 is equal to or greater than a predetermined threshold value Δdispth.

  The parallax difference value waveform portion 61a of this example satisfies both of the above conditions 1 and 2. Therefore, the CPU determines that the road gradient is decreasing far away. These conditions are for distinguishing between the case where the trajectory 61 of the parallax difference value is generated due to noise at the time of parallax calculation and the case where the distant road gradient is obtained with respect to the road gradient near the vehicle. Set to

  Step 380: When the CPU determines in step 370 that the road gradient is decreasing far away, as shown in FIG. 10, the upper end of the road area estimated from the road model 53 in step 350 (ie, as shown in FIG. 10) , The far end of the road) 62 is changed to the maximum end 63 of the V coordinate position of the locus 61a of the parallax difference value. The maximum end 63 corresponds to a point P2 at which the road model 53 and the parallax point group 51 start to separate on the V-disparity plane.

  Hereinafter, specific operations of Step 350, Step 360, Step 370, and Step 380 will be described.

<Road area estimation routine>
When the CPU proceeds to step 350, the CPU executes a “road area estimation routine” shown by the flowchart in FIG. Accordingly, the CPU proceeds to step 1110 via step 1100, and applies the extracted road model 53 to the V-disparity plane (see FIG. 7).

  Next, the CPU proceeds to step 1120, adds the height threshold value ΔVth to the road model 53, calculates the road area determination threshold value Vth, as shown in FIG. 7, and proceeds to step 1130, where the road area determination threshold value Vth. It is determined whether there is a parallax value (parallax point group 51) exceeding. If there is a parallax value exceeding the road area determination threshold Vth, the CPU makes a “Yes” determination at step 1130 to proceed to step 1140, and the maximum value of the V coordinate of the portion having the parallax value (that is, the obstacle 52 The region 54 (region 54 below the lower end 52b in FIG. 7) having a larger V coordinate than the lower end 52b) is estimated as a road region, and the process proceeds to step 1195 to proceed to step 360 in FIG.

  On the other hand, if there is no parallax point group 51 exceeding the road area determination threshold Vth, the CPU makes a “No” determination at step 1130 to proceed to step 1150, where the CPU 11 determines that the intersection P3 between the road model 53 and the V coordinate axis The lower area (V coordinate is large) is estimated as a road area, and the process proceeds to Step 1195 to proceed to Step 360 in FIG.

<Parallax difference value locus (waveform portion) extraction routine>
When the CPU proceeds to step 360, the CPU executes a “parallax difference value locus calculation routine” shown by the flowchart in FIG. Accordingly, the CPU proceeds to step 1210 via step 1200, sets the value of the coordinate V to “0”, proceeds to step 1220, and calculates the parallax difference value Δdisp (V) according to the above-described equation (1).

  Next, the CPU proceeds to step 1230 to determine whether or not the parallax difference value Δdisp (V) is larger than “0”. When the parallax difference value Δdisp (V) is larger than “0”, the CPU makes a “Yes” determination at step 1230 to proceed to step 1240 to store the parallax difference value Δdisp (V) in the RAM. On the other hand, when the parallax difference value Δdisp (V) is “0” or less (including a negative value), the CPU makes a “No” determination at step 1230 to proceed to step 1270 to set the value of the parallax difference value. Set to “0” and proceed to step 1240 to store the calculated parallax difference value Δdisp (V) in the RAM in the ECU 30.

  Next, the CPU increments the value of the coordinate V by 1 in step 1250 and proceeds to step 1260 to determine whether or not the value of the coordinate V is equal to or greater than (j−1). If the value of the coordinate V is equal to or greater than (j−1), the CPU makes a “Yes” determination at step 1260 to proceed to step 1280. If the value of the coordinate V is less than (j−1), the CPU makes a “No” determination at step 1260 to proceed to step 1220, execute the processing of step 1220 to step 1250 and step 1270, and then proceed to step 1260. . That is, the CPU starts this routine from the value 0 of the V coordinate and repeats the processing of this routine until the value of the coordinate V becomes (j−1).

  Next, the CPU proceeds to step 1280 to project the parallax difference value Δdisp (V) onto a V-parallax difference value plane (V-Δdisp plane) 60 having the V coordinate axis as the vertical axis and the parallax difference value Δdisp as the horizontal axis. After extracting the locus (waveform portion) 61 of the parallax difference value, the process proceeds to step 370 in FIG.

<Slope decrease presence / absence determination routine>
When the CPU proceeds to step 370, the CPU executes a “gradient reduction presence / absence determination routine” shown by the flowchart in FIG. 13. Therefore, the CPU proceeds to step 1310 via step 1300 and calculates the vanishing point height (V coordinate position) from the image acquired by the camera device 20.

  Next, the CPU proceeds to step 1320 to extract “the waveform portion (trajectory) 61a of the parallax difference value closest to the vanishing point height 62” as shown in FIG. 8B. In other words, when a plurality of parallax difference value trajectories 61 are obtained in step 1280, the CPU selects the “parallax difference value waveform portion 61a closest to the vanishing point height 62” from these.

  Next, the CPU proceeds to step 1330 to determine whether or not a specific condition is satisfied, that is, whether or not the “waveform portion 61a” extracted in step 1320 satisfies the above two conditions (condition 1 and condition 2) simultaneously. Determine whether or not.

  When the two conditions described above are satisfied at the same time, the CPU makes a “Yes” determination at step 1330 to proceed to step 1340 to set the value of the road gradient decrease flag Xdkg to “1” (the road gradient decreases). The process proceeds to step 1395. On the other hand, if either of the two conditions described above is not satisfied, the CPU makes a “No” determination at step 1330 to proceed to step 1350 to set the value of the road gradient decrease flag Xdkg to “0” ( Proceed to step 1395 (deciding that the road gradient has not decreased). Thereafter, the CPU proceeds to step 380 in FIG.

<Road area far end position change routine>
When the CPU proceeds to step 380, the CPU executes a "road area far end position changing routine" shown by a flowchart in FIG. Therefore, the CPU proceeds to step 1410 via step 1200 and determines whether or not the value of the road gradient decrease flag Xdkg is “1”.

  When the CPU has passed through step 1340 in FIG. 13 (determined that there is a gradient decrease), the value of the road gradient decrease flag Xdkg is “1”. Accordingly, the CPU makes a “Yes” determination at step 1410 to proceed to step 1420, where “the waveform portion (trajectory of the parallax difference value closest to the vanishing point is located at the upper end (far end) 62 of the road region estimated at step 350. ) “Corrected to the height 63 of the lower end of 61a (the maximum end of the V coordinate position)” and set the value of the road gradient decrease flag Xdkg to “0” and proceed to step 1495, and then proceed to step 395 in FIG. . On the other hand, when the CPU has passed through step 1350 in FIG. 13 (determined that there is no gradient decrease), the value of the road gradient decrease flag Xdkg is “0”. Accordingly, the CPU makes a “No” determination at step 1410 to directly proceed to step 1495, and then proceeds to step 395 in FIG. That is, the road area estimated in step 350 is not changed.

  As described above, the detection apparatus creates the road models 53 based on the parallax values projected on the plurality of V-disparity planes (V-disparity planes) 50 corresponding to the minute sections of the parallax image, Based on this road model, the far end of the road area in the minute section is estimated.

Further, the detection apparatus calculates “parallax difference value Δdisp” that is a value obtained by subtracting the projected parallax value 51 from the parallax value of the created road model 53 at each V coordinate position of the V-disparity plane 50. In addition, the parallax difference value Δdisp is projected onto the V-parallax difference value plane 60 with the V coordinate axis as the vertical axis and the parallax difference value Δdisp as the horizontal axis,
A trajectory of a parallax difference value Δdisp (V) existing between two adjacent points where the parallax difference value Δdisp (V) is “0” and showing a positive value is recognized as one waveform unit 61.

  In addition, the present detection apparatus calculates the V coordinate position of the vanishing point 96 from the acquired image, and is surrounded by the waveform portion 61a closest to the V coordinate position of the vanishing point 96 in the waveform portion 61 and the V coordinate axis. The area of the figure is larger than the area of the figure surrounded by any other waveform portion and the V coordinate (having the largest area among the areas of the figures in all the sections), and the vanishing point 96 When the maximum parallax difference value ΔdispMAX in the waveform portion 61a closest to the V coordinate position is equal to or greater than a predetermined threshold, the far end 62 of the road region is the V coordinate position of the waveform portion 61a closest to the V coordinate position of the vanishing point. Change to the maximum end side 63.

  Therefore, according to the present detection apparatus, in the road where the gradient of the road surface far away from the gradient of the road surface on which the vehicle is currently traveling is reduced, an obstacle region far from the road is not erroneously detected as a road region, The range of the road area can be corrected to the original range, and as a result, the obstacle can be recognized correctly.

  The present invention is not limited to the above embodiment, and various modifications can be employed within the scope of the present invention. For example, the parallax calculation is performed by the block matching method, but may be performed by a semi-global matching (SGM) method or other methods.

DESCRIPTION OF SYMBOLS 10 ... Vehicle, 20 ... Camera apparatus, 30 ... Electronic control unit (ECU), 40 ... Parallax image, 41 ... Road area, 42 ... Obstacle area, 50 ... V-parallax plane, 51 ... Parallax value (parallax point group) 53 ... Road model, 60 ... V-parallax difference value plane, 61 ... Trajectory of parallax difference value, 96 ... Vanishing point.

Claims (1)

Image acquisition means for acquiring a stereo image by the left camera and the right camera, both of which capture the road surface in front of the vehicle,
Parallax image creating means for calculating a parallax value of each point on the stereo image from the stereo image and creating a parallax image in which the parallax value is arranged on an image having the same V coordinate axis and U coordinate axis as the stereo image;
The parallax image is subdivided by a plurality of virtual lines parallel to the V coordinate axis that is the vertical axis of the parallax image to obtain a plurality of minute sections, and the parallax values of the plurality of minute sections are obtained as the parallax images. V-parallax plane creating means for creating a plurality of V-parallax planes by projecting onto a V-parallax plane having the V-coordinate axis of
Road model creating means for creating a road model for each of the plurality of V-parallax planes by extracting a main straight line as a road model from the parallax values projected on each of the plurality of V-parallax planes;
Estimating the far end of each road area of the minute section based on the road model,
A parallax difference value that is a value obtained by subtracting the projected parallax value from the parallax value of the created road model at each V coordinate position of each of the plurality of V-parallax planes;
The parallax difference value is projected on a V-parallax difference value plane having the V coordinate axis as the vertical axis and the parallax difference value as the horizontal axis, and exists between two adjacent points where the parallax difference value is 0. Recognizing a trajectory of a parallax difference value and showing a positive value as one waveform part,
Calculate the V coordinate position of the vanishing point from one of the acquired stereo images,
Of the waveform portion, the area of the figure surrounded by the waveform portion closest to the V coordinate position of the vanishing point and the V coordinate axis is the figure surrounded by any other waveform portion and the V coordinate axis. Determining whether or not a specific condition is satisfied that the maximum parallax difference value in the waveform portion that is larger than the area and closest to the V coordinate position of the vanishing point is equal to or greater than a predetermined threshold;
If the specific condition is not satisfied, set the far end of the road area to the estimated far end,
A far end position determining means for changing the far end of the road region to the maximum value of the V coordinate position of the waveform portion closest to the V coordinate position of the vanishing point when the specific condition is satisfied;
A road area detection device comprising: