JP2014232439A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus configured to properly detect a vanishing point.SOLUTION: An image processing apparatus includes: imaging means 10 which captures an image in a traveling direction of a vehicle and outputs a pickup image corresponding to the image captured; direction component detection means 20 which sets a plurality of detection areas in the pickup image, detects direction components in the detection areas, and outputs predetermined count values for the direction components; generation means 20 which generates a histogram of the direction components for each of the detection areas by counting the count values of the direction components in each detection direction of the direction components; evaluation area setting means 20 which sets an evaluation area for evaluating possibility of the presence of vanishing point along a peak direction of the histogram of the direction components, to set a plurality of evaluation areas in the pickup image; and vanishing point detection means 20 which detects a coordinate position in the pickup image where the evaluation areas overlaps most, as a vanishing point.

Description

  The present invention relates to an image processing apparatus and an image processing method.

  2. Description of the Related Art Conventionally, a technique for detecting a vanishing point in a captured image by detecting a linear component such as a white line from the captured image is known (see, for example, Patent Document 1).

JP 2008-123036 A

  However, since the conventional technique detects vanishing points based on a straight line component such as a white line, when the straight line component cannot be detected from the captured image, such as when the road is curved or unpaved, There was a problem that the vanishing point could not be detected properly.

  The subject of this invention is providing the image processing apparatus which can detect a vanishing point appropriately.

  In the present invention, a plurality of detection areas are set in a captured image, and a direction component in each detection area is counted for each detection direction, thereby generating a histogram of direction components for each detection area. Then, it is determined that there may be a vanishing point in the peak direction of the directional component histogram, and an evaluation region for evaluating the possibility that the vanishing point exists along the peak direction of the directional component histogram. By setting, a plurality of evaluation areas are set in the captured image, and among the coordinate positions in the captured image, the coordinate position where the evaluation areas most overlap is detected as a vanishing point, thereby solving the above problem. .

  According to the present invention, even when a linear component such as a white line does not exist in the captured image, the direction component scattered in the captured image is detected, and the coordinate position present in the direction indicated by the most directional component is used as the vanishing point. By detecting, even when the road is curved or unpaved, the vanishing point can be detected appropriately.

It is a block diagram which shows the image processing system which concerns on this embodiment. It is a figure which shows an example of a detection area. It is a figure which shows an example of a direction component histogram. It is a figure for demonstrating the setting method of a voting area. It is a figure for demonstrating the method to weight the vote value of a voting area. It is a figure which shows an example of the captured image when another vehicle exists ahead of the own vehicle. It is a figure for demonstrating the detection method of a vanishing point. It is a figure for demonstrating the determination method of the reliability of a vanishing point. It is a flowchart which shows the image processing which concerns on 1st Embodiment. It is a figure which shows an example of the captured image which imaged the unpaved curved road. It is a figure for demonstrating the other example of the detection method of a vanishing point. It is a figure for demonstrating the detection method of a road shape. It is a figure for demonstrating the detection method of a road shape. It is a figure for demonstrating the other example of the detection method of a road shape. It is a flowchart which shows the image processing which concerns on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, the present invention will be described by exemplifying an image processing system that is mounted on a vehicle and detects a vanishing point from a captured image obtained by capturing the front of the host vehicle.

  FIG. 1 is a configuration diagram illustrating an image processing system according to the present embodiment. As shown in FIG. 1, the image processing system according to the present embodiment includes a camera 10 and an image processing device 20.

  The camera 10 includes an image sensor in which a plurality of photoelectric conversion elements such as a CCD image sensor, a MOS sensor, or a CID are two-dimensionally arranged. The image is output to the image processing apparatus 20. Moreover, as shown in FIG. 1, the camera 10 is installed in the vehicle so as to image the front in the traveling direction of the host vehicle, and images the front in the traveling direction of the host vehicle. Further, in the present embodiment, the depression angle of the camera 10 is set so that the road surface of the road is imaged in an image region that is about 1/2 to 2/3 of the captured image.

  The image processing apparatus 20 includes a ROM (Read Only Memory) in which a program for detecting a vanishing point is stored, a CPU (Central Processing Unit) as an operation circuit for executing the program stored in the ROM, and a captured image. And a RAM (Random Access Memory) for temporarily storing the data. As an operation circuit, instead of or in addition to a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. Can be used.

  The image processing apparatus 20 according to the present embodiment executes a program stored in the ROM by the CPU, thereby enabling a detection area setting function, a histogram generation function, a voting area setting function, a voting function, and a vanishing point detection function. And each function of the reliability judgment function is realized. Hereinafter, each function realized by the image processing apparatus 20 will be described.

  The detection area setting function of the image processing apparatus 20 sets an area for detecting a direction component from a captured image as a detection area. Here, FIG. 2 is a diagram illustrating an example of the detection area set by the detection area setting function. Specifically, as shown in FIG. 2, the detection area setting function first sets, as a processing range, an image area below a captured image in which a road is captured in a captured image acquired from the camera 10. For example, in the present embodiment, since the camera 10 is installed so that a road on which the host vehicle travels is imaged in an image area of about 1/2 to 2/3 from the lower end of the captured image, the detection area setting is performed. As shown in FIG. 2, the function sets an image area of about 1/2 to 2/3 from the lower end of the captured image as a processing range.

  Then, the detection area setting function sets a plurality of detection areas within the processing range, as shown in FIG. For example, in the present embodiment, the detection area setting function can set the detection area with a size of 5 × 5 pixels or a size of 9 × 9 pixels. Note that the size of the detection area is not particularly limited, but should be such a size that the direction component can be calculated, and that a plurality of detection areas can be set within the processing range. In the example illustrated in FIG. 2, three detection areas are illustrated, but the installation positions and the number of detection areas are not particularly limited. For example, as a configuration in which the detection areas are set at arbitrary positions within the processing range. Alternatively, the processing range may be divided into a plurality of areas, and the divided areas may be set as detection areas. Alternatively, in order to detect all the direction components within the processing range, the detection area may be shifted by one pixel and a plurality of detection areas may be set.

  Next, the histogram generation function of the image processing apparatus 20 will be described. The histogram generation function detects a direction component in the detection area set by the detection area setting function, and generates a direction component histogram based on the detected direction component. 3 is a diagram showing an example of the direction component histogram. The direction component histograms shown in FIGS. 3A to 3C are based on the direction components detected in the detection areas A to C shown in FIG. It is.

  The histogram generation function first detects the direction component of each detection region in order to generate a direction component histogram. For example, the histogram generation function prepares a plurality of Gabor filters corresponding to a plurality of directions, and uses each Gabor filter corresponding to each direction to perform a convolution calculation of the luminance value of the image in the detection region, The edge component detected in the detection direction is detected as a direction component corresponding to each direction in the detection region. For example, in the present embodiment, the histogram generation function, when the direction in the detection area is expressed from 0 ° to 360 ° with the center position of the detection region as the center point, the direction component in the detection region is expressed in units of 1 °, for example. Alternatively, direction components in a plurality of directions can be detected in the detection region by detecting in units of 5 °. Further, the histogram generation function outputs a count value for each detected direction component in order to generate a direction component histogram described later. For example, the histogram generation function can output a count value “1” for each detected direction component. The count value is not limited to “1” and can be set as appropriate.

  The histogram generation function generates a direction component histogram based on the direction components in a plurality of directions detected in the detection area. Specifically, the histogram generation function counts the direction component count value for each detection direction in which the direction component is detected, so that the direction of the direction component is detected as shown in FIG. A direction component histogram is generated with the sum of the count values of the vertical axis as the vertical axis.

  For example, in the example shown in FIG. 2, the detection region A includes direction components in a plurality of directions almost uniformly. Therefore, as shown in FIG. 3A, the direction component histogram of the detection region A is compared. Flat. On the other hand, in the example shown in FIG. 2, since the detection region B includes a road edge extending in the upper left direction of the captured image, many direction components in the upper left direction are detected. As a result, the direction of the detection region B In the component histogram, as shown in FIG. 3B, the direction component has a peak in the upper left direction where the road edge extends. Further, in the example shown in FIG. 2, the detection region C includes a road edge extending in the upper left direction of the captured image and a road sign pole extending in the upward direction of the captured image. As a result, in the direction component histogram of the detection region C, as shown in FIG. 3 (C), the upper left direction where the road edge extends and the upper direction where the pole of the road sign extends. It has a peak of the direction component.

  Next, the voting area setting function of the image processing apparatus 20 will be described. The voting area setting function sets a voting area based on the direction component histogram generated by the histogram generation function. Here, FIG. 4 is a diagram for explaining a voting area setting method. In the example shown in FIG. 4, a method for setting the voting area B based on the direction component histogram generated in the detection area B shown in FIG.

  Specifically, as shown in FIG. 4, the voting area setting function sets a radial area starting from the center O of the detection area as the voting area. The voting area setting function sets a voting area along a direction in which the direction component histogram reaches a peak. For example, since the detection region B shown in FIG. 4 includes a road edge extending in the upper left direction of the captured image, many direction components in the upper left direction are detected. As a result, in the direction component histogram of the detection region B, FIG. As shown in FIG. 3 (B), the directional component histogram has a peak in the direction component histogram of the detection region B as shown in FIG. A radial voting area B is set along the direction in which the image is taken, that is, the upper left direction of the captured image.

  Further, when setting the voting area, the voting area setting function determines the size of the voting area based on the variance in the vicinity of the peak of the direction component histogram. Specifically, the voting area setting function reduces the angle θ with the starting point O of the voting area as the apex as the variance in the vicinity of the peak of the direction component histogram is smaller (the sharpness of the peak of the direction component histogram is larger). The larger the variance in the vicinity of the peak of the direction component histogram (the smaller the sharpness of the peak of the direction component histogram), the larger the angle θ with the start point O of the voting region as the apex, thereby determining the size of the voting region. To do. For example, the voting area setting function can determine the size of the voting area by determining the angle θ of the voting area according to the standard deviation σ or 2σ of the direction component histogram.

  When the direction component histogram has two or more peaks, the voting area setting function sets a voting area for each peak of the direction component histogram. That is, as shown in FIG. 3C, when there are two peaks in the direction component histogram, the voting area setting function can set two voting areas.

  The voting area setting function sets a plurality of voting areas for the plurality of detection areas set within the processing range. For example, when five detection areas are set in the processing range, the voting area setting function sets five or more voting areas starting from the center positions of these five detection areas.

  Next, the voting function of the image processing apparatus 20 will be described. The voting function votes a voting value to each coordinate position (each unit area that can be specified by XY coordinates) in the voting area set by the voting area setting function. The voting method by the voting function is not particularly limited. For example, the voting function may be configured to uniformly vote the voting value “1” to all the coordinate positions in the voting area, or the direction component histogram. A configuration may be used in which the voting value is weighted for each coordinate position in the voting area based on the variance in the vicinity of the peak. Hereinafter, a method for weighting the vote value for each coordinate position in the vote area will be described in detail.

When weighting voting values for each coordinate position in the voting area, the voting function first calculates the standard deviation σ of the direction component histogram as a value indicating the variance in the vicinity of the peak of the direction component histogram. When the voting function determines that the standard deviation σ is equal to or greater than the predetermined first determination value and the peak of the direction component histogram is sharp, as shown in FIG. The area is divided into _three areas B _{1 to} B _3, and the weights of the voting values at the respective coordinate positions in the voting area B are set to “0.1”, “0.8”, “0” in the order of the areas B _{1 to} B _3. Set to 1 '. The voting function, each of the regions _B 1 .about.B ₃ performing the weighted vote value obtained by multiplying 1 to the weight of the region _B 1 .about.B ₃ as voting value. For example, in the example shown in FIG. 5A, a vote value “0.1” is voted for each coordinate position in the region B ₁ , and a vote value “0.8” is voted for each coordinate position in the region B _2. Thus, the voting value “0.1” is voted at each coordinate position in the region B ₃ .

In addition, the voting function is shown when the variance in the vicinity of the peak of the direction component histogram (that is, the standard deviation σ) is less than the first determination value and not less than the second determination value smaller than the first determination value. As shown in FIG. 5B, the voting area B is divided into _five radial areas B _{1 to} B _5, and the weights of the voting values at the respective coordinate positions in the voting area B are set in the order of the areas B _{1 to} B ₅ . Set to '0.05', '0.2', '0.5', '0.2', '0.05'. The voting function, the respective weighting Been voting region B ₁ .about.B _5, vote value obtained by multiplying 1 to the weight of voting areas B ₁ .about.B ₅ as voting value. For example, in the example shown in FIG. 5B, a voting value “0.05” is voted at each coordinate position in the voting area B ₁ , and a voting value “0.2” is obtained at each coordinate position in the voting area B _2. There found, the coordinate position of the voting region B ₃ found voting value "0.5", voting value "0.2" is found in each coordinate position of the voting region B _4, voting region B ₅ The vote value “0.05” is voted at each coordinate position.

Further, although the voting function is not shown, when the variance in the vicinity of the peak of the direction component histogram (that is, the standard deviation σ) is less than the second determination value, it can be determined that the peak of the direction component histogram is gentle. The voting area B is divided into _seven radial areas B _{1 to} B _7, and the weights of the voting values at the respective coordinate positions in the voting area B are set to “0.05” and “0” in the order of the areas B _{1 to} B _7. .1 ”,“ 0.2 ”,“ 0.3 ”,“ 0.2 ”,“ 0.1 ”,“ 0.05 ”. The voting function is in each coordinate position of each region B ₁ .about.B ₇ performing the weighted vote value obtained by multiplying 1 to the weight of the region B ₁ .about.B ₇ as voting value.

  Note that the method of weighting the voting values in the voting area is not limited to the above-described method. For example, as shown in FIG. 5A, regardless of the variance in the vicinity of the peak of the direction component histogram. The voting area may be divided into three radial areas and the voting values in the voting area may be weighted.

  In the present embodiment, the voting function is configured so that the direction in which the direction component histogram reaches a peak is the vertical direction (Y-axis direction) or the horizontal direction (X-axis direction). The voting value of the voting area set along the Y-axis direction) and the voting value of the voting area set along the horizontal direction (X-axis direction) are set to zero in the vertical direction (Y-axis direction). The voting value is not voted at the coordinate position of the voting area set along the horizontal direction (X-axis direction) and the coordinate position of the voting area set along the horizontal direction (X-axis direction). The direction component in the vertical direction (Y-axis direction) and the direction component in the horizontal direction (X-axis direction) are as shown in detection areas D and E in FIG. This is because the direction component detected from the pole of the road sign is highly likely to be excluded from the vanishing point detection target. FIG. 6 is a diagram illustrating an example of a captured image when another vehicle is present in front of the host vehicle.

  In addition, because car bumpers, buildings, road sign poles, etc. have high contrast, the intensity of the direction component of the detection area including car bumpers, buildings, road sign poles tends to increase. The variance in the vicinity of the peak of the directional component histogram tends to increase. Therefore, the voting function calculates the variance (that is, standard deviation σ) in the vicinity of the peak of the directional component in the vertical direction (Y-axis direction) or the horizontal direction (X-axis direction), and the standard deviation σ is greater than or equal to a predetermined value. However, the weight of the voting value in the voting area set along the vertical direction (Y-axis direction) or the horizontal direction (X-axis direction) may be set to zero.

  Furthermore, as shown in FIG. 3C, the voting function has a plurality of directional component histogram peaks, and a plurality of voting regions are set for one directional component histogram. Vote the voting value for the coordinate position. Also, the voting function, when there are three or more directional component peaks in one directional component histogram, is the voting value only at the coordinate position of the voting area corresponding to the peak having the peak size from the top to the second. The vote value may be voted only at the coordinate position of the voting area corresponding to the peak having the magnitude of 80% or more with respect to the largest peak and the largest peak, for example. .

  Furthermore, the voting function determines whether or not the magnitude of the peak of the direction component histogram is less than a predetermined reference value. If the magnitude of the peak is less than the reference value, the voting area corresponding to this peak It is good also as a structure which does not vote for a voting value with respect to these coordinate positions. As a result, for example, as shown in FIG. 3A, a substantially flat direction component histogram can be excluded from the vanishing point detection target, and the vanishing point detection accuracy can be improved. The size of the reference value is not particularly limited. For example, the maximum peak is detected in each direction component histogram, the average value of the maximum peaks of all the direction component histograms is calculated, and the average value of the maximum peaks is calculated. Can be set as the reference value. Similarly, the voting function determines whether or not the variance in the vicinity of the peak of the direction component histogram is greater than or equal to a predetermined reference value, and if the variance in the vicinity of the peak of the direction component histogram is less than the reference value, The vote value may not be voted with respect to the coordinate position of the voting area corresponding to.

  Next, the vanishing point detection function of the image processing apparatus 20 will be described. The vanishing point detection function detects the vanishing point in the captured image based on the vote result of the vote value by the voting function. Specifically, the vanishing point detection function calculates a voting value at each coordinate position in the captured image, and detects a coordinate position having the largest voting value as a vanishing point.

  Here, FIG. 7 is a diagram for explaining a vanishing point detection method. For example, when five detection areas are set within the processing range as shown in FIG. 7A, five or more voting areas are set as shown in FIG. 7B. Thus, when a plurality of voting areas are set, different voting areas may overlap at some coordinate positions as shown in FIG. 7B. When the vanishing point detection function calculates the voting value of each coordinate position, for the coordinate position where a plurality of voting areas overlap, the voting values voted in the respective voting areas are totaled, and the pixel position The voting value of is calculated. For example, the voting area Y and the voting area Z overlap at the coordinate position x, the voting value at the coordinate position x in the voting area Y is “0.8”, and the voting value at the coordinate position x in the voting area Z Is “0.2”, the voting value at the coordinate position x is calculated as “1.0”.

  And a vanishing point detection function compares the vote value of all the coordinate positions, and detects the coordinate position with the largest vote value as a vanishing point.

  In this embodiment, the vanishing point is detected for each captured image captured in a predetermined shooting frame, and the vanishing point detected this time is more than a certain distance from the position of the vanishing point detected last time. The configuration may be such that it is determined that the vanishing point has not been detected. Also, using a time series filter such as Kalman filter or particle filter, the average value of the coordinate positions of multiple vanishing points detected up to a certain time ago is calculated, and the vanishing point detected this time is the average of past vanishing points. It is good also as a structure which judges that a vanishing point was not able to be detected, when away from the coordinate position of a value more than a fixed distance. Thereby, the detection accuracy of a vanishing point can be improved.

  Next, the reliability determination function of the image processing apparatus 20 will be described. The reliability determination function determines the reliability of the vanishing point detected by the vanishing point detection function. In order to determine the vanishing point reliability, the reliability determining function first sets a radial region for determining the vanishing point reliability. Specifically, as shown in FIG. 8A, the reliability determination function uses a vanishing point detected by the vanishing point detection function as a reference, and an area obtained by radially dividing the image area below the vanishing point. Set as radial area. Further, the reliability determination function sets a plurality of radial regions so that the angle θ ′ of each radial region having the vanishing point as a vertex is, for example, 10 ° or 20 ° when setting the radial region. Can do.

  Then, for example, when the voting region set based on the direction component histogram includes a vanishing point, the reliability determination function identifies the detection region from which such a direction component histogram is obtained as a detection target, Among the plurality of detection areas existing in the area, the number of detection areas specified as detection targets is counted for each radial area. Then, as shown in FIG. 8B, the reliability determination function generates a histogram with the count value as the vertical axis and each radial region as the horizontal axis as the reliability histogram.

  Here, in FIG. 8A, the direction of the peak of the direction component histogram of each detection region is indicated by an arrow. For example, in the example shown in FIG. 8 (A), the peak direction of the direction component histogram is the vanishing point in the detection areas in the radial areas 3 and 8 including the road edge and in the radial area 5 including the traces of the tires and the saddles. These detection areas are identified as detection targets. Therefore, as shown in FIG. 8B, a reliability histogram having a high count value is generated at positions corresponding to the radial region 3, the radial region 5, and the radial region 8.

  For example, vanishing points detected based on directional components such as road edges, white lines, tire tracks, and ridges are highly reliable, and such directional components have a specific radial shape as shown in FIG. There is a tendency to be detected only in the region, and the variance of the reliability histogram increases accordingly. On the other hand, when the direction component (that is, noise) other than the road surface influences the detection of the vanishing point, the variance of the reliability histogram is reduced accordingly. Therefore, the reliability determination function calculates the variance of the reliability histogram. For example, as shown in FIG. 8B, when the variance of the reliability histogram is equal to or higher than a predetermined reliability determination value, the vanishing point is calculated. It can be determined that the reliability is high. On the other hand, the reliability determination function can determine that the reliability of the vanishing point is low when the variance of the reliability histogram is less than a predetermined reliability determination value. Note that the value of the reliability determination value described above is not particularly limited, and can be set as appropriate through experiments or the like. Alternatively, only the variance in the vicinity of the peak of the reliability histogram may be calculated, and when the variance in the vicinity of the peak of the reliability histogram is equal to or higher than a predetermined reliability determination value, it may be determined that the reliability of the vanishing point is high. .

  The reliability determination function may be configured to count “1” evenly for each detection area to be detected when generating the reliability histogram, or to calculate the vote value in the voting area. When weighting is performed, a detection area corresponding to such a voting area is detected as a detection target only when the weight of the voting value at the vanishing point is a predetermined threshold value (for example, 0.5) or more. It is good also as a structure which specifies and counts as an area | region.

  Next, the image processing apparatus according to the first embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating image processing according to the first embodiment. Note that the image processing shown in FIG. 9 is executed by the image processing device 20 when, for example, the ignition is turned on.

  First, in step S101, the processing range is set by the detection area setting function. Specifically, as shown in FIG. 2, the detection area setting function sets, as a processing range, an image area that is about 1/2 to 2/3 from the lower end of the captured image of the captured image captured by the camera 10. . In step S102, the detection area setting function sets a plurality of detection areas within the processing range set in step S101 as shown in FIG.

  In step S103, the direction component is detected in each detection region set in step S102 by the histogram generation function. In step S104, the direction component histogram is generated by the histogram generation function based on the direction component detected in step S103, as shown in FIG.

  In step S105, the voting area is set based on the direction component histogram generated in step S104 by the voting area setting function. Specifically, as shown in FIG. 4, the voting area setting function sets a radial voting area along a direction in which the direction component histogram generated in step S104 peaks.

  In step S106, a vote value is voted for each coordinate position of the voting area set in step S105 by the voting function. In step S107, the vanishing point is detected by the vanishing point detection function based on the vote value voted in step S106. Specifically, the vanishing point detection function calculates a voting value at each coordinate position of the captured image, and detects a coordinate position having the largest voting value as a vanishing point.

  In step S108, in order to determine the reliability of the vanishing point detected in step S107, first, a radial area is set by the reliability determination function. Specifically, as shown in FIG. 8A, the reliability determination function sets a plurality of radial regions by dividing an image region below the vanishing point radially with reference to the vanishing point. .

  In step S109, a reliability histogram is generated by the reliability determination function. For example, when the voting area set based on the direction component histogram includes a vanishing point, the reliability determination function identifies the detection area from which such a direction component histogram is obtained as a detection target and exists in the radial area. Among the plurality of detection areas to be detected, the number of detection areas specified as detection targets is counted for each radial area, thereby generating a reliability histogram.

  In step S110, the reliability of the vanishing point is calculated based on the reliability histogram generated in step S109 by the reliability determination function. Specifically, the reliability determination function calculates the variance of the reliability histogram, and determines that the reliability of the vanishing point is high when the variance of the reliability histogram is equal to or greater than a predetermined reliability determination value. The process proceeds to step S111. On the other hand, if the variance of the reliability histogram is less than the predetermined reliability determination value, it is determined that the reliability of the vanishing point is low, and the process proceeds to step S112.

  In step S111, since it is determined that the vanishing point has high reliability, it is determined that the vanishing point has been detected, and the vanishing point detected in step S107 is detected by the image processing illustrated in FIG. Output as.

  On the other hand, if it is determined in step S110 that the vanishing point reliability is low, the process proceeds to step S112. In step S112, vanishing point non-detection processing is performed by the vanishing point detection function. For example, the vanishing point detection function repeats the process of changing the installation position of the detection area within the processing range and detecting the vanishing point based on the direction component of the changed detection area. Then, the vanishing point detection function determines that the vanishing point has been detected when a vanishing point having a reliability equal to or higher than a predetermined reliability determination value is detected, and outputs the detected vanishing point. On the other hand, the vanishing point detection function detects a vanishing point if it cannot detect a vanishing point that is equal to or higher than the reliability judgment value even if the process of detecting the vanishing point by changing the installation position of the detection area is repeated a predetermined number of times. It is determined that it cannot be performed, and a detection result indicating that the vanishing point cannot be detected is output. Alternatively, the vanishing point detection function may be configured to output the vanishing point detected in step S107 by adding information that the reliability is low.

  As described above, in the first embodiment, a plurality of detection regions are set in a captured image obtained by capturing a road ahead of the host vehicle, and the direction component is detected for each detection region based on the direction component in each detection region. Generate a histogram. Then, for each direction component histogram, a voting area is set along the direction in which the direction component histogram reaches a peak, and a voting value is voted at each coordinate position of the voting area, and a coordinate position having the largest voting value is detected as a vanishing point. To do. Thereby, in this embodiment, since the coordinate position where the direction component scattered most in a captured image goes to the most can be detected as a vanishing point, for example, when the road is a curve or an S-shaped road, or is unpaved. Even in this case, the vanishing point can be appropriately detected based on the direction components such as the tire trace and the wrinkles.

  Here, FIG. 10A is a diagram illustrating an example of a captured image obtained by capturing an unpaved bent road. As shown in FIG. 10A, in an unpaved curved road, there may be a directional component such as a tire mark or a wrinkle even when a linear component such as a white line does not exist. In such a case, a direction component histogram is generated, and as shown in FIG. 10B, a voting area is set based on the direction component histogram, and a vote value is voted at each coordinate position of the voting area. The coordinate position where the direction component in the captured image is most frequently detected can be detected as a vanishing point.

  In this embodiment, a plurality of voting areas are set along the detection direction of the direction component scattered in the captured image, and the voting voted in each voting area at the coordinate position where the plurality of voting areas overlap. The voting value at each coordinate position is calculated by summing the values. Thus, in this embodiment, by detecting a vanishing point from a plurality of directional components scattered in the captured image, even if there is no white line or the like on the road, each directional component is unclear. Even if it exists, a vanishing point can be detected appropriately.

  Furthermore, in this embodiment, the smaller the variance in the vicinity of the peak of the direction component histogram, the smaller the size of the voting area (the angle θ of the voting area with the center position of the detection area as the vertex). Here, since a linear component such as a white line has a large contrast and a direction component in a predetermined direction can be easily obtained, the variance in the vicinity of the peak of the direction component histogram is small. In addition, a straight line component such as a white line is highly likely to have a vanishing point in that direction. Therefore, by reducing the size of the voting area as the variance near the peak of the direction component histogram is smaller, the voting area is limited to an area where there is a high possibility that a vanishing point exists, thereby detecting the vanishing point. Accuracy can be improved. Further, in this embodiment, the smaller the variance in the vicinity of the peak in the direction component histogram, the greater the weighting of the vote value in the area corresponding to the peak direction in the voting area, so that the direction in accordance with the linear component such as the white line is increased. The weight of the vote value can be increased, and as a result, the vanishing point detection accuracy can be further increased.

  In addition, in this embodiment, voting is performed for each coordinate position of the voting area set along the vertical direction (Y-axis direction) and each coordinate position of the voting area set along the horizontal direction (X-axis direction). By not voting the value, the direction component in the vertical direction (Y-axis direction) and the direction component in the horizontal direction (X-axis direction) are likely to be caused by bumpers, buildings, road sign poles, etc. Can be excluded from the vanishing point detection target, thereby improving the vanishing point detection accuracy.

  Further, in the present embodiment, it is determined whether or not the magnitude of the peak of the direction component histogram is equal to or greater than a predetermined reference value, and the vote value is applied only to the voting area corresponding to the peak whose peak magnitude is equal to or greater than the reference value. By voting, as shown in FIG. 3A, the direction component histogram in which the peak of the direction component cannot be obtained can be excluded from the vanishing point detection target, thereby increasing the vanishing point detection accuracy. And the processing load at the time of vanishing point detection can be reduced.

  Furthermore, in this embodiment, even when there are a plurality of peaks in the direction component histogram, the voting area is set along the direction of each peak. Here, when there are a plurality of peaks in the direction component histogram, the vanishing point exists in the direction of one peak, and the vanishing point does not exist in the other direction. It is also possible to adopt a configuration in which no is set. However, in the direction where no vanishing point exists, the possibility that a voting area is set from another detection area is low, and the possibility that a vanishing point is erroneously detected is low. On the other hand, when there is a peak in the direction component histogram, the configuration of setting the voting area along the direction of the peak can simplify the vanishing point detection process. The detection process can be reduced and speeded up. Even in such a configuration, since there is a road marking on the road, the vanishing point can be detected stably even when peaks are detected in a plurality of directions in the direction component histogram.

  Furthermore, in the present embodiment, by setting a radial region with the detected vanishing point as a reference, among the detection regions existing in the radial region, by counting the number of detection regions corresponding to the voting region including the vanishing point, A reliability histogram is generated, and the reliability of the vanishing point is determined based on the generated reliability histogram. Here, vanishing points based on linear components such as white lines are highly reliable, and such linear components tend to exist only in a specific radial region among a plurality of radial regions that spread radially around the vanishing point. Yes, the variance of the reliability histogram tends to increase accordingly. Therefore, in the present embodiment, when the variance of the reliability histogram is equal to or higher than a predetermined reliability determination value, it is determined that the reliability of the vanishing point is high, and only the vanishing point that is determined to have high reliability is detected. Thus, the accuracy of vanishing point detection can be increased.

  In the above-described embodiment, as illustrated in FIG. 2, the configuration in which an image region of about 1/2 to 2/3 from the lower end of the captured image is set as the processing range is illustrated, but the present embodiment is not limited to this configuration. For example, as shown in FIG. 11A, the processing range may be a horizontally long line. Also in this case, as shown in FIG. 11B, the vanishing point can be detected based on the direction component within the processing range in the form of a line. In particular, as shown in FIG. 6, even when the vanishing point cannot be detected properly because another vehicle exists in front of the host vehicle, the other vehicle in front of the host vehicle is not included as shown in FIG. By setting the processing range, the vanishing point can be detected appropriately. When there is another vehicle in front of the host vehicle, many directional components in the horizontal direction such as bumpers of the other vehicle are detected. Therefore, when a horizontal direction component is detected and the horizontal direction component is detected to be equal to or greater than a predetermined value, it is determined that there is another vehicle in front of the host vehicle, and the other vehicle in front of the host vehicle is The processing range can be set in a line shape so as not to be included.

  In the present embodiment, direction components other than the direction in which the vanishing point exists may be detected, such as a scratch on the road surface or a road marking drawn on the road surface, but the ratio of the presence of such a direction component is low. Therefore, the vanishing point detection accuracy is not greatly affected.

<< Second Embodiment >>
Subsequently, a second embodiment of the present invention will be described. The image processing system according to the second embodiment has the same configuration as that of the image processing system according to the first embodiment shown in FIG. 1, and the image processing according to the first embodiment except that it operates as described below. Works like the system.

  In addition to the function of the image processing device 20 according to the first embodiment, the image processing device 20 according to the second embodiment is based on the vanishing point detected by the vanishing point detection function, and the road ahead of the traveling direction of the host vehicle. A road shape detection function for detecting the shape is further provided. The road shape detection function according to the second embodiment will be described below with reference to FIG. FIG. 12 is a diagram for explaining a road shape detection method.

  In order to detect the road shape ahead of the host vehicle, the road shape detection function first sets a horizontal line area within the processing range as shown in FIG. In this embodiment, since a plurality of horizontal line areas are arranged in order from the bottom within the processing range, the road shape detection function may first set the first horizontal line area near the lower end of the processing range. preferable. Further, the size of the horizontal line area (size in the Y-axis direction) is not particularly limited, but is, for example, a size that can arrange 5 to 7 horizontal line areas within the processing range, or 10 within the processing range. The size can be set such that up to 20 horizontal line regions can be arranged.

  Next, the road shape detection function sets a plurality of detection areas in the horizontal line area, and detects a vanishing point based on the direction component of the horizontal line area based on the direction component of the detection area of the horizontal line area. For example, in the example shown in FIG. 12A, the road shape detection function detects the vanishing point A based on the direction components of a plurality of detection areas (not shown) installed in the horizontal line area A.

Then, as shown in FIG. 12A, the road shape detection function sets lines L _AR and L _AL connecting the detected vanishing point A and the left and right ends of the horizontal line region A, respectively, as shown in FIG. As shown in B), a new horizontal line region B is set in a triangular range (a range indicated by a broken line in FIG. 12B) formed by the vanishing point A and the lines L _AR and L _AL .

The road shape detection function detects the vanishing point B based on the direction component of the horizontal line region B, and sets the lines L _BR and L _BL connecting the detected vanishing point B and the left and right ends of the horizontal line region B, respectively. Then, a new horizontal line region C is set within a triangular range formed by the vanishing point B and the lines L _BR and L _BL . Similarly, the road shape detection function repeats the setting of the horizontal line area and the detection of the vanishing point, thereby converting a plurality of vanishing points corresponding to the plurality of horizontal line areas into a series of vanishing points as shown in FIG. Detect as data.

  The road shape detection function detects the road shape ahead of the host vehicle based on the detected vanishing point sequence data. Here, the vanishing point detected for each horizontal line area is based on the direction component of each horizontal line area. Therefore, when the road in the horizontal line area goes straight, the vanishing point of this horizontal line area is detected in the straight direction of the host vehicle, and when the horizontal line area curves leftward, The vanishing point of the line area is detected on the left side of the straight line direction of the host vehicle. If the horizontal line area is curved to the right side, the vanishing point of the horizontal line area is on the right side of the straight line direction of the own vehicle. Detected.

  Therefore, the road shape detection function detects that the road shape is a straight line when the vanishing point column data extends straight in the straight direction, and the vanishing point column data curves to the right. If it is, the road shape is detected as a right curve, and if the vanishing point column data is curved to the left, the road shape is detected as a left curve. Further, as shown in FIG. 13, when the vanishing point column data meanders to the left and right, the road shape is detected to meander to the left and right.

  Further, as shown in FIG. 14, the road shape detection function detects the midpoints of the left and right ends of the horizontal line area for each horizontal line area, and converts it into a plurality of midpoint column data calculated in the plurality of horizontal line areas. Based on this, the shape of the road ahead of the host vehicle can also be detected. For example, in the example shown in FIG. 14, the middle point column data meanders as in the example shown in FIG. 13. Therefore, the road shape detection function meanders the shape of the road ahead of the host vehicle to the left and right. Detect as a thing.

  Furthermore, in the second embodiment, the road shape detection function detects the road gradient based on the vanishing point column data. For example, when the horizontal area region is a downhill, the vanishing point of the horizontal area region is detected below (the Y coordinate positive direction side) below the vanishing point when the horizontal area is flat. On the other hand, when the horizontal area region is uphill, the vanishing point of the horizontal area region is detected above (the Y coordinate negative direction side) the vanishing point when the horizontal area A1 is flat. Therefore, the road shape detection function stores, for example, the Y coordinate position of the vanishing point when each horizontal area region is flat, and the Y coordinate position of the vanishing point when each horizontal area region is flat, The road gradient ahead of the host vehicle can be detected by comparing the Y coordinate position of the vanishing point of each horizontal area that is actually detected.

  Next, an image processing apparatus according to the second embodiment will be described with reference to FIG. FIG. 15 is a flowchart illustrating image processing according to the second embodiment. Note that the image processing shown in FIG. 15 is executed by the image processing device 20 when, for example, the ignition is turned on.

  First, in step S201, the processing range is set as in step S101. In step S202, as shown in FIG. 12A, the horizontal line area is set within the processing range set in step S201 by the road shape detection function. In step S203, a plurality of detection areas are set in the horizontal line area set in step S202.

  In steps S204 to S210, vanishing points are detected based on the direction component of each detection region, as in steps S103 to S111 of the first embodiment. That is, in the second embodiment, the vanishing point of the horizontal line area is detected based on the direction component of the horizontal line area set in step S202.

  In step S211, it is determined whether or not all the horizontal line areas have been set by the road shape detection function. As shown in FIG. 13, when all the horizontal line areas are set within the processing range and a plurality of vanishing points corresponding to the plurality of horizontal line areas are detected as the column data of vanishing points, step S212 is performed. Proceed to On the other hand, if all the horizontal line areas have not been set, the process returns to step S203, and as shown in FIG. 12B, a new area is added within the triangular range connecting the vanishing point and the left and right ends of the horizontal line area. A horizontal line area is set.

  In step S212, since it is determined that the vanishing point sequence data has been detected, the road shape detection function detects the road shape based on the vanishing point sequence data. Specifically, the road shape detection function detects that the road shape is a straight line when the vanishing point column data extends straight in the straight direction, and the vanishing point column data is rightward. If it is curved, the shape of the road is detected as a right curve, and if the vanishing point column data is curved to the left, the shape of the road is detected as a left curve.

  Furthermore, in step S213, the road shape detection function detects a road gradient ahead of the host vehicle based on the vanishing point column data. For example, the road shape detection function compares the Y coordinate position of the vanishing point when each horizontal area region is flat with the Y coordinate position of the vanishing point of each horizontal area that is actually detected. The road gradient can be detected.

  As described above, in the second embodiment, a plurality of horizontal line areas are set in the processing range, and vanishing points are detected based on the direction component of each horizontal line area, thereby corresponding to the plurality of horizontal line areas. A plurality of vanishing points are detected as vanishing point column data, and a road shape ahead of the host vehicle is detected based on the detected vanishing point column data. Since each vanishing point detected for each horizontal line area is based on the direction component of each horizontal line area, the road shape can be detected appropriately based on the column data of vanishing points in the horizontal line area. .

  In the second embodiment, when the horizontal line area is set, it is within a triangular range that connects the vanishing point based on the direction component of the already set horizontal line area and the left and right end portions of the already set horizontal line area. Therefore, by setting a new horizontal line area, the newly set horizontal line area can be limited to an image area obtained by imaging the road surface, and direction components other than the road surface can be excluded from the vanishing point detection targets. Therefore, the vanishing point detection accuracy can be increased.

  Furthermore, in 2nd Embodiment, the road gradient ahead of the own vehicle is detected based on the row | line | column data of a vanishing point. When the horizontal area region is a downhill, the vanishing point of the horizontal area region is detected below (the Y coordinate positive direction side) the vanishing point when the horizontal area is flat. In the case of the uphill, the vanishing point of the horizontal area region is detected above the vanishing point (Y coordinate negative direction side) when the horizontal area is flat. Therefore, in the second embodiment, for each horizontal area area, the Y coordinate position of the vanishing point when the horizontal area area is flat is compared with the Y coordinate position of the vanishing point of each actually detected horizontal area. Thus, the road gradient ahead of the host vehicle can be detected appropriately.

  The embodiment described above is described for facilitating understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  For example, in the above-described embodiment, the configuration in which an image area of about 1/2 to 2/3 from the lower end of the captured image is set as the processing range is illustrated. However, the present invention is not limited to this configuration. An image area from a position below the number of pixels (for example, about 10 to 20 pixels) to the lowest end of the captured image may be set as a processing range.

  In the second embodiment described above, the configuration in which a plurality of horizontal line regions are arranged adjacent to each other when detecting column data of vanishing points is illustrated. However, the present invention is not limited to this configuration. A plurality of horizontal line regions may be shifted by several pixels in the Y-axis direction so that part of the regions overlap each other, or the plurality of horizontal line regions may be separated by several pixels in the Y-axis direction. It is good also as a structure to arrange.

  Furthermore, in the above-described embodiment, the voting area is set along the peak of the direction component histogram, the voting value is voted on the voting area, and the coordinate position having the largest voting value is detected as the vanishing point. For example, a voting area for detecting a vanishing point is set along the peak of the direction component histogram, and the coordinate value where the voting area most overlaps among the coordinate positions in the captured image is set as the vanishing point. It is good also as a structure detected as. In this case as well, the vanishing point can be detected properly even if there is no straight line component such as a white line because the coordinate position in the direction in which the directional component scattered in the captured image is most frequently detected can be detected. can do.

  Note that the camera 10 of the above-described embodiment is the imaging unit of the present invention, the detection region setting function of the image processing device 20 is the detection region setting unit of the present invention, and the histogram generation function of the image processing device 20 is the direction component of the present invention. The voting area setting function of the image processing apparatus 20 is used as the evaluation area setting means of the present invention, the voting function of the image processing apparatus 20 is used as the voting means of the present invention, and the vanishing point detecting function of the image processing apparatus 20 is used. Is the vanishing point detection means of the present invention, the reliability determination function of the image processing apparatus 20 is the radial area setting means, the reliability histogram generation means, and the reliability determination means of the present invention is the road shape detection function of the image processing apparatus 20. Corresponds to the band-shaped area setting means, vanishing point string detection means, and road shape detection means of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Camera 20 ... Image processing apparatus

Claims (16)

Imaging means for imaging the traveling direction of the host vehicle and outputting a captured image corresponding to the captured image;
Detection area setting means for setting a plurality of detection areas in the captured image;
Direction component detection means for detecting edge components in a plurality of directions as direction components for each direction in the detection region and outputting a predetermined count value for each of the detected direction components;
Generating means for generating a histogram of the direction component for each detection region by counting the count value of the direction component for each detection direction of the direction component;
An evaluation region for determining that there is a possibility that a vanishing point exists in the peak direction of the histogram of the direction component, and for evaluating the possibility that the vanishing point exists along the peak direction of the histogram of the direction component An evaluation area setting means for setting a plurality of the evaluation areas in the captured image,
An image processing apparatus comprising: vanishing point detecting means for detecting, as the vanishing point, a coordinate position where the evaluation areas most overlap among the coordinate positions in the captured image. The image processing apparatus according to claim 1.
Voting means for voting a vote value according to the possibility that the vanishing point exists at each coordinate position in the evaluation area;
The vanishing point detecting means calculates a total value of the voting values for each coordinate position, and detects a coordinate position having the largest total voting value among the coordinate positions in the captured image as the vanishing point. An image processing apparatus. The image processing apparatus according to claim 2,
The voting means, when voting the voting value to each coordinate position in the evaluation area, increases the weighting of the voting value as the coordinate position is closer to the peak direction of the histogram of the direction component. Image processing device. The image processing apparatus according to claim 2 or 3,
The voting means, when voting the voting value to each coordinate position in the evaluation area, the larger the variance in the vicinity of the peak of the histogram of the direction component, the voting value at the coordinate position near the peak of the histogram An image processing apparatus characterized by increasing the weighting of. The image processing apparatus according to any one of claims 2 to 4,
When the peak direction of the histogram of the directional component is a vehicle width direction or a direction perpendicular to the vehicle width direction, the voting means performs the evaluation on the evaluation area set along the peak of the histogram. An image processing apparatus characterized by not voting a vote value. The image processing apparatus according to any one of claims 2 to 4,
The voting means, when the peak direction of the histogram of the direction component is a vehicle width direction or a direction perpendicular to the vehicle width direction, and the peak size of the histogram is a predetermined value or more, An image processing apparatus, wherein the vote value is not voted for the evaluation area set along the peak of the image. The image processing apparatus according to any one of claims 1 to 6,
The evaluation region setting means, when there are a plurality of peaks in the histogram of the direction component, selects one or more peaks based on the size of the peak or the variance in the vicinity of the peak, An image processing apparatus, wherein one or more evaluation regions are set along a selected peak direction. In the image processing device according to any one of claims 2 to 7,
The voting means does not vote the voting value for the evaluation area set along the peak of the histogram when the magnitude of the peak of the histogram of the direction component is less than a predetermined reference value. An image processing apparatus. The image processing apparatus according to any one of claims 1 to 8,
After the vanishing point is detected by the vanishing point detection means, a plurality of radial regions are set by dividing an image area below the vanishing point in the captured image radially with the vanishing point as a center. Radial area setting means;
A reliability histogram for determining the reliability of the vanishing point by counting the number of the detection areas corresponding to the evaluation area including the vanishing point among the detection areas set in the radial area. A reliability histogram generation means for generating
An image processing apparatus comprising: a reliability determination unit that determines the reliability of the vanishing point based on the variance of the reliability histogram. The image processing apparatus according to claim 9.
The vanishing point detection means causes the detection area setting means to change the installation position of the detection area when it is determined that the reliability of the vanishing point is not more than a predetermined reliability determination value. Image processing device. The image processing apparatus according to claim 1,
Within the processing range, a belt-shaped region setting means for setting a region extending in the vehicle width direction as a belt-shaped region;
Vanishing point sequence detecting means for detecting a plurality of vanishing points detected in the plurality of band-like regions as vanishing point column data by detecting the vanishing points based on the direction component of the band-like region;
An image processing apparatus comprising: road shape detection means for detecting a shape of a road ahead of the host vehicle based on the vanishing point sequence data. The image processing apparatus according to claim 11.
The image processing apparatus according to claim 1, wherein the belt-shaped region setting unit sequentially sets a plurality of the belt-shaped regions from the lower part to the upper part of the captured image. The image processing apparatus according to claim 12.
The band-shaped area setting means is configured such that when the band-shaped area set first is a first band-shaped area and the band-shaped area set next to the first band-shaped area is a second band-shaped area, the first band-shaped area is set. An image processing apparatus characterized in that the second belt-like region is set in a triangular region formed by left and right ends of the region and vanishing points detected based on a direction component of the first belt-like region. . The image processing apparatus according to claim 11,
The road shape detection means detects a plurality of midpoints detected in the plurality of band-like regions as midpoint row data by detecting a midpoint in the vehicle width direction of the band-like region for each band-like region. An image processing device that detects the shape of a road ahead of the host vehicle based on the detected middle point column data. The image processing apparatus according to any one of claims 11 to 14,
The road shape detecting means detects a road gradient ahead of the host vehicle based on a coordinate position in a direction perpendicular to the vehicle width direction of the vanishing point, which is detected based on a direction component of the belt-like region. Road shape detection device.   A plurality of detection regions are set in the captured image, and edge components in a plurality of directions in the detection region are detected as direction components of the detection region, and predetermined count values are output for the detected direction components, respectively. Then, by counting the count value of the direction component for each detection direction of the direction component, a histogram of the direction component is generated for each detection region, along the direction in which the histogram of the direction component peaks, An image in which an evaluation area for evaluating the possibility that the vanishing point exists is set, and among the coordinate positions in the captured image, a coordinate position where the evaluation area overlaps most is detected as a vanishing point. Processing method.

Images