JP5805619B2 - Boundary line recognition device - Google Patents

Description

  The present invention relates to a boundary line recognition device for recognizing a boundary line that divides a running road based on image information taken by a camera mounted on a vehicle.

  A scene including the road surface in front of the vehicle is photographed with a photographing device mounted on the vehicle, and two straight lines representing the left and right white lines of the lane in which the vehicle is traveling are detected from the photographed image data, and from the intersection of the detected two straight lines An object on a road surface is detected in a triangular area between two straight lines up to the host vehicle (for example, Patent Document 1).

  In Patent Document 1, when the vehicle lane is curved in front of the vehicle, the region on the image where the road surface in front of the vehicle is present It is divided into multiple pieces in the direction of travel. Then, sequentially from the divided area close to the vehicle, the left and right end white lines of the running lane in the divided area are detected, and the object is detected in the traveling lane area between the detected left and right end white lines.

JP-A-6-215290

  When object detection is performed in a predetermined area based on a straight line representing the left and right end white lines, it is necessary to suppress erroneous recognition of the straight line representing the left and right end white lines in order to improve the object detection accuracy. However, in the case of a white line far from the vehicle, the apparent width of the boundary line on the image is very narrow. For this reason, when the contrast (edge strength) between the boundary line and the road surface decreases due to the blur of the camera lens, roadside objects and the like arranged along the boundary line may be erroneously recognized as the boundary line at a distance from the vehicle. In particular, when a boundary line such as a white line is inclined in the horizontal direction on the image, such a tendency becomes remarkable.

  In view of the above circumstances, the present invention has as its main object to provide a boundary line recognition device capable of suppressing erroneous recognition of a boundary line at a distance from a vehicle.

  In order to solve the above-described problem, a boundary line recognition apparatus according to claim 1 is provided with a camera mounted on a vehicle for photographing a road and a boundary line that divides the road based on luminance of an image photographed by the camera. In the boundary line candidate extraction unit that extracts edge candidate points and extracts the boundary line candidate lines from a series of the extracted edge candidate points, and the boundary line candidate lines extracted by the boundary line candidate extraction unit, A width calculation unit that calculates an apparent width of the candidate line on the image from the width of the candidate line in the horizontal direction of the image and the angle of the candidate line with respect to the vertical direction of the image; and the candidate line is a boundary The higher the degree of having a feature as a line, the higher the probability that the candidate line is the boundary line, and a boundary line feature calculation unit that calculates a plurality of the features by the boundary line feature calculation unit A boundary line recognizing unit that recognizes the boundary line by integrating the probabilities obtained, and a plurality of features as the boundary line are calculated by the width calculating unit with respect to the unclearness of the image Including a ratio of the widths of the two being larger than a predetermined value.

  According to the boundary line recognition apparatus of the first aspect, the edge candidate points of the boundary line and the candidate lines of the boundary line are extracted from the luminance of the image photographed by the camera mounted on the vehicle. The apparent width of the extracted candidate line on the image is calculated. Further, the probability that the candidate line is a boundary line is calculated higher as the degree of the candidate line having the boundary line feature is larger. Then, the probabilities calculated for the features of the plurality of boundary lines are integrated to recognize the boundary lines.

  If the apparent width of the boundary line becomes narrower with respect to the unclearness of the image at a distance from the vehicle, the inventor of the present application decreases the extraction accuracy of the edge candidate points, and the roadside objects installed along the boundary line We focused on the increased risk of misrecognizing a line. Where the apparent width of the boundary line is narrow, the apparent width of the candidate line from which an edge of a roadside object or the like installed along the boundary line is often narrowed. Therefore, the fact that the ratio of the apparent width of the candidate line to the unclearness of the image is larger than a predetermined value is included in one of the features as a plurality of boundary lines. For this reason, in the integration of the probabilities that are boundary lines, by reflecting the probabilities calculated from these characteristics, it is possible to suppress erroneous recognition of the boundary lines in the distance of the vehicle.

The block diagram which shows the structure of the boundary line recognition apparatus which concerns on one Embodiment. An arithmetic expression that integrates probabilities. The flowchart which shows the process sequence which calculates the probability which is a boundary line based on the apparent width of a candidate line. The figure which shows the apparent width and horizontal width of a candidate line. The figure which shows the primary differential distribution which differentiated the brightness | luminance of the candidate line in the horizontal direction. The figure which shows the luminance distribution of the candidate line in a horizontal direction. The figure which shows a response | compatibility with the ratio of the apparent width with respect to unclearness, and the probability which is a boundary line. The figure which shows the boundary line recognized before and after reflecting the probability which is a boundary line computed based on the apparent width of a candidate line.

  With reference to FIG. 1, the structure of the boundary line recognition apparatus which concerns on one Embodiment is demonstrated. The boundary line recognition apparatus according to this embodiment includes a camera 10 and an image processing apparatus 70.

  The camera 10 is a CCD camera, for example, and is fixed, for example, on the back side of a rearview mirror so as to face the front of the vehicle so as to capture the road ahead of the vehicle. The camera 10 captures a road ahead of the vehicle and outputs the captured image information to the image processing device 70.

  The image processing apparatus 70 is configured as a microcomputer including a CPU, a ROM, a RAM, an I / O, a bus line connecting these, and the like. In the present embodiment, the functions of the boundary line candidate extraction unit 20, the boundary line feature calculation unit 30, and the boundary line recognition unit 60 are realized by the CPU executing a program stored in the ROM.

  The boundary line candidate extraction unit 20 extracts edge candidate points of boundary lines such as white lines that divide the runway based on the luminance of the image photographed by the camera 10, and the boundary line candidate lines are obtained from the series of the extracted edge candidate points. To extract. In the horizontal direction intersecting the vehicle traveling direction on the image, the luminance of the image changes greatly at the left and right contour (edge) portions of one boundary line. The boundary line candidate extraction unit 20 continuously processes the image information acquired from the camera 10 with a predetermined sampling period, and extracts a plurality of points whose luminance changes rapidly in the horizontal direction of the image as edge candidate points. To do. Then, Hough transform is performed on the extracted plurality of edge candidate points to acquire a series of edge candidate points, and a plurality of candidate lines having the left and right contours as the acquired series of edge candidate points are extracted.

  The boundary line feature calculation unit 30 includes a width calculation unit 31, a vote count calculation unit 32, an edge strength calculation unit 33, and a branch path determination unit 34. The boundary line feature calculation unit 30 calculates, for each of the plurality of candidate lines extracted by the boundary line candidate extraction unit 20, the degree of having the feature as the boundary line of the main road at each edge candidate point. The greater the degree of having, the higher the probability that the candidate line is the boundary line of the main road.

  The inventor of the present application extracts the edge of a roadside object or the like installed along the boundary line as an edge candidate point when the apparent width of the boundary line becomes narrow with respect to the unclearness of the image at a distance from the vehicle. We noticed that there is a high risk of misrecognizing etc. as boundaries. In other words, if the width of the boundary line is not wider than the predetermined value for the unclearness of the image, a non-boundary line may be misrecognized as a boundary line, or the boundary line may be misrecognized as not being a boundary line. Becomes higher. Therefore, the ratio of the apparent width of the candidate line to the unclearness of the image is larger than a predetermined value as a feature as the boundary line of the main road.

  In addition, the number of votes by the Hough transform is greater than the predetermined number, the edge intensity represented by the luminance differential value in the horizontal direction is greater than the predetermined value, and the runway delimited by the boundary line is not a branch road As the boundary of the main track.

  The width calculation unit 31 calculates the apparent width of the candidate line at each edge candidate point. Then, when the ratio of the calculated apparent width to the image blur is smaller than a predetermined value, the probability that the candidate line is the boundary of the main track is minimized, and the candidate line is recognized as the boundary of the main track. It suppresses that.

  When the vote count by the Hough transform is smaller than a predetermined number, the vote count calculation unit 32 minimizes the probability that the candidate line is the boundary line of the main track and recognizes the candidate line as the boundary line of the main track Suppress. When the edge strength of the candidate line is smaller than a predetermined value, the edge strength calculation unit 33 minimizes the probability that the candidate line is the boundary line of the main road and recognizes the candidate line as the boundary line of the main road Suppress. The branch road determination unit 34 determines whether or not the road delimited by the candidate line is a branch road. If the branch road is determined to be a branch road, the probability that the candidate line is the boundary line of the main road is minimized and the candidate line is determined. Is recognized as the boundary of the main track.

  The boundary line recognition unit 60 includes a boundary line feature integration unit 40 and a boundary line selection unit 50, and integrates the probabilities calculated by the boundary line feature calculation unit 30 with respect to features as a plurality of boundary lines. Recognize

  The boundary line feature integration unit 40 integrates the probabilities calculated by the width calculation unit 31, the vote count calculation unit 32, the edge strength calculation unit 33, and the branch path determination unit 34 using the arithmetic expression shown in FIG. Calculate the integration probability. First, the probability X calculated by the width calculating unit 31 is A, the probability calculated by the vote calculating unit 32 is B, and the probability X obtained by integrating the probabilities calculated by the width calculating unit 31 and the vote calculating unit 32 is calculated. To do. Further, the probability X obtained by integrating the probabilities calculated by the width calculation unit 31 and the vote number calculation unit 32 is A, and the probability calculated by the edge strength calculation unit 33 is B, the width calculation unit 31, the vote number calculation unit 32, And the probability X which integrated the probability calculated by the edge strength calculation part 33 is calculated. In this way, the probabilities are sequentially integrated, and the probabilities calculated by the boundary line feature calculating unit 30 are integrated for the features as a plurality of boundary lines. Note that the characteristics of the boundary line are not limited to the above four characteristics. For other features as the boundary line, the probability that the candidate line is the boundary line of the main road may be calculated, and the calculated probabilities may be integrated.

  The boundary line selection unit 50 determines whether the candidate line is a boundary line based on the probabilities integrated by the boundary line feature integration unit 40, and recognizes the candidate line recognized as the boundary line of the main road and the candidate line. Select included edge candidate points. Specifically, among candidate lines whose integrated probability is higher than a determination value that can be determined as a boundary line, the candidate lines that are paired on the left side and the right side of the vehicle and that are closest to the vehicle on the vehicle side Edge candidate points representing contours are selected.

  When the vehicle is automatically controlled, the vehicle steering amount is set and the object on the road surface is detected based on the shape of the candidate line selected by the boundary line selection unit 50. Further, when the vehicle is outside the selected candidate line, an alarm sound is output. That is, the candidate line selected as the boundary line of the main road is set as the boundary line of the control target, and various automatic controls are executed on the vehicle.

  Next, a processing procedure for calculating the probability of being a boundary line will be described in detail with reference to FIG. This processing procedure is executed by the width calculation unit 31.

  First, in S11, the width w of the candidate line in the horizontal direction of the image is calculated. Subsequently, in S12, the angle θ of the candidate line with respect to the vertical direction (vehicle traveling direction) is calculated. Subsequently, in S13, the apparent width of the candidate line on the image is calculated. The apparent width of the candidate line on the image is the width of the candidate line in the direction perpendicular to the extending direction of the candidate line, as shown in FIG. The apparent width of the candidate line is calculated as wcos θ from the width w of the candidate line in the horizontal direction calculated in S11 and the angle θ of the candidate line with respect to the vertical direction calculated in S12. The unit of the apparent width of the candidate line on the image is a pixel.

  Subsequently, in S14, the unclearness of the image is calculated. Specifically, as shown in FIG. 5, the first-order differential distribution is calculated by differentiating the peripheral luminance including the edge candidate points in the horizontal direction of the image, and the predetermined width of the calculated first-order differential distribution is set as the unclearness of the image.

  In an ideal situation where the image is not blurred, the first-order differential distribution of luminance in the horizontal direction of the image has sharp peaks at the left and right edge portions of the boundary line, and has no spread. On the other hand, in a situation where the image is blurred due to lens specifications, rain, or the like, the first-order differential distribution is a distribution having a spread at the left and right edge portions. Therefore, the predetermined width representing the extent of the first-order differential distribution is defined as the image blur.

  Alternatively, as shown in FIG. 6, in the peripheral luminance distribution including the edge candidate points distributed in the horizontal direction of the image, the width in which the luminance is distributed from the first predetermined value to the second predetermined value larger than the first predetermined value. Is the image blur.

  In an ideal situation where the image is not blurred, the luminance distribution of the boundary line in the horizontal direction of the image is a distribution having sharp peaks that suddenly rise from the minimum luminance to the maximum luminance at the left and right edge portions of the boundary line. On the other hand, in a situation where the image is blurred due to lens specifications, rainfall, etc., the brightness distribution spreads dullly at the left and right edge portions of the boundary line, which is less than the ideal situation from the minimum brightness to the maximum brightness. The distribution rises gently. In this luminance distribution, the width in which the luminance from the first predetermined value to the second predetermined value is distributed well represents the extent of the edge portion of the boundary line. Therefore, the range in which the luminance from the first predetermined value to the second predetermined value is distributed with respect to the maximum luminance is defined as the unclearness of the image.

  Although the apparent width direction of the candidate line on the image is different from the differential direction of the primary differential distribution and the distribution direction of the luminance distribution, the approximate blur level can be calculated by either of the above two blur level calculation methods. . Moreover, even if it calculates with either of the said 2 unsharpness calculation methods, the unsharpness of an image will be a comparable value, and a unit will be a pixel.

  Subsequently, in S15, the ratio of the apparent width of the candidate line to the unclearness of the image is calculated from the unclearness of the image calculated in S14 and the apparent width of the candidate line calculated in S13. Specifically, the ratio is calculated by dividing the apparent width of the candidate line by the unclearness of the image.

  Subsequently, in S16, using the probability map shown in FIG. 7, the probability that the candidate line is a boundary line is calculated from the ratio of the apparent width of the candidate line to the image blur calculated in S15.

  When the boundary line is far from the vehicle, the apparent width of the boundary line on the image is narrower than when it is near the vehicle. In particular, as shown in FIG. 7, when the boundary line is curved in the horizontal direction on the image, the apparent width of the boundary line on the image becomes very narrow and the edge of the boundary line is far away from the vehicle. The part is easily blurred. For this reason, the apparent width of the boundary line is narrower on the image relative to the unclearness of the image at a distance from the vehicle than at the vicinity of the vehicle, and it is difficult to distinguish the edge portion from the other portions. The extraction accuracy of candidate points and candidate lines is lowered.

  If the ratio of the apparent width of the candidate line on the image to the unclearness of the image is smaller than the predetermined value, the apparent width of the candidate line on the image becomes narrower than the predetermined value, and the boundary It becomes difficult to distinguish the edge portion of the line from the other portion. Therefore, when the ratio is smaller than a predetermined value, the probability of being a boundary line is set to a predetermined minimum probability, and erroneous recognition of the boundary line is suppressed. On the other hand, if the above ratio is larger than a predetermined maximum value, that is, if the apparent width of the candidate line on the image is wider than the image blur, whether or not the candidate line is a boundary line. The reliability of the determination becomes sufficiently high. Therefore, when the ratio is larger than a predetermined maximum value, the probability of being a boundary line is set to a predetermined maximum probability. When the ratio is between a predetermined value and a predetermined maximum value, the probability that the candidate line is a boundary line is reduced as the ratio of the apparent width of the candidate line to the image blur is smaller.

  Here, the predetermined minimum probability is a value that makes the integrated probability very low when the probabilities calculated for the features as the plurality of boundary lines are integrated by the boundary line feature integration unit 40, and the candidate line is a boundary. It is a value that suppresses recognition as a line.

  This processing is thus completed, and the width calculation unit 31 outputs the probability of the boundary line calculated in S16 to the boundary line feature integration unit 40.

  FIGS. 8A and 8B show an example of the result of recognizing the boundary line by the image processing apparatus 70. FIG. White squares indicate edge candidate points selected by the boundary line selection unit 50, and black squares indicate excluded edge candidate points. FIG. 8A is a diagram in the case where the probability calculated by the width calculation unit 31 is not reflected in the probability obtained by integrating the probabilities of the boundary lines calculated for the features as a plurality of boundary lines. On the other hand, FIG. 8B is a diagram in the case where the probability calculated by the width calculating unit 31 is reflected on the probability obtained by integrating the probabilities of the boundary lines calculated for the features as a plurality of boundary lines. is there.

  8 (a) and 8 (b), the road is curved far away from the vehicle, and poles are installed at regular intervals along the left side of the road near the curve. Near the curve, pole edges are extracted as edge candidate points.

  Edge candidate points representing poles should be excluded from edge candidate points to be selected as boundary lines. In FIG. 8A, some edge candidate points representing poles that have high continuity with the boundary lines are included. , Selected as the boundary of the main track. That is, in the vicinity of the curve, a series of edge candidate points representing poles are erroneously recognized as boundary lines. On the other hand, in FIG. 8B, a series of edge candidate points erroneously recognized as the boundary line in FIG. 8A is not selected as the boundary line of the main road. The apparent width of a candidate line made up of a series of edge candidate points representing a pole is narrow with respect to the unclearness of the image. Therefore, the probability calculated by executing the processes of S11 to S16 is reduced, and the probability of integration by the boundary line feature integration unit 40 is reduced. As a result, in FIG. 8B, the edge candidate point representing the pole is not selected as the boundary line of the main road.

  According to this embodiment described above, the following effects are obtained.

  -In the integration of the probability that the candidate line is a boundary line, by reflecting the probability calculated by the feature that the ratio of the apparent width of the candidate line to the unclearness of the image is larger than a predetermined value, the boundary in the distance of the vehicle Misrecognition of lines can be suppressed.

  -As the ratio of the apparent width of the candidate line to the unclearness of the image is smaller, the probability that the extracted candidate line is a boundary line is reduced, so that it is possible to suppress erroneous recognition of the boundary line.

  Since the image blur is calculated based on a predetermined width in the differential value distribution of luminance or a distribution width of the predetermined luminance with respect to the maximum luminance, it is possible to obtain a blur reflecting the specification and situation of the lens.

(Other embodiments)
The present invention is not limited to the description of the above embodiment, and may be modified as follows.

  ・ Calculate the primary differential distribution by differentiating the surrounding luminance including the edge candidate points in the direction of the angle θ with respect to the horizontal direction of the image, that is, the apparent width direction of the candidate line on the image, and calculate the primary differential distribution The predetermined width may be used as the unclearness of the image. In this way, it is possible to improve the accuracy of the unclearness of the image.

  In the peripheral luminance distribution including the edge candidate points distributed in the direction of the angle θ with respect to the horizontal direction of the image, the width in which the luminance from the first predetermined value to the second predetermined value is distributed may be defined as the unclearness of the image. . In this way, it is possible to improve the accuracy of the unclearness of the image.

  The specification value of the amount of lens blur included in the camera 10 may be converted into a length on the image, and the converted value may be used as the unclearness of the image. In this way, the ratio of the apparent width of the candidate line on the image to the unclearness of the image can be easily calculated. In addition, it is possible to obtain high-precision blurringness when it is not raining or fogging.

  -When it is raining or fogging, the image blur is higher than the amount of lens blur due to rain or fog, so that it may be corrected to increase the lens blur amount during raining or fogging. Specifically, as shown by a broken line in FIG. 1, a rain sensor 11 (rainfall detection means) for detecting rainfall and a fog sensor 12 (fog detection means) for detecting fog are mounted on the vehicle. Then, when rain or fog is detected by the rain sensor 11 and the fog sensor 12, correction is performed so that the conversion value of the specification value of the lens blur amount is increased as compared with the case where rain and fog are not detected. The corrected value is used as the image blur.

  In this way, it is possible to improve the accuracy of the image blur during rainfall or fogging, and to further suppress erroneous detection of the boundary line. Instead of the rain sensor 11, the rain may be detected by turning on the wiper switch, and the fog may be detected by turning on the fog lamp switch instead of the fog sensor 12.

  Furthermore, the rain sensor 11 or fog sensor 12 detects the amount of rainfall or the amount of fog, and corrects the conversion value of the lens blur amount specification value to increase as the amount of rainfall or the amount of fog increases. Also good. In this way, it is possible to further improve the accuracy of the unclearness of the image during rain or fog.

  A reference value is set between a predetermined value set in the above embodiment and a predetermined maximum value for the ratio of the apparent width of the candidate line on the image to the unclearness of the image. If the ratio is larger than the reference value, the probability of being a boundary line may be calculated as, for example, a predetermined high probability, and if it is smaller, the probability of being a boundary line may be calculated as, for example, a predetermined low probability. That is, the probability of being a boundary line may be set to a discontinuous binary value.

  DESCRIPTION OF SYMBOLS 10 ... Camera, 20 ... Boundary line candidate extraction part, 30 ... Boundary line feature calculation part, 31 ... Width calculation part, 40 ... Boundary line feature integration part, 50 ... Boundary line selection part, 60 ... Boundary line recognition part, 70 ... Image processing device.

Claims (8)

A camera (10) mounted on a vehicle and shooting a runway;
A boundary candidate extraction unit that extracts edge candidate points of the boundary line that divides the runway based on the luminance of the image photographed by the camera, and extracts the candidate line of the boundary line from a series of the extracted edge candidate points ( 20)
In the candidate line of the boundary line extracted by the boundary line candidate extraction unit, the image of the candidate line is calculated from the width of the candidate line in the horizontal direction of the image and the angle of the candidate line with respect to the vertical direction of the image. A width calculation unit (31) for calculating the apparent width above;
A boundary line feature calculation unit (30) that calculates a higher probability that the candidate line is the boundary line as the degree of the candidate line having a characteristic as a boundary line increases;
A boundary line recognition unit (60) for recognizing the boundary line by integrating the probabilities calculated by the boundary line feature calculation unit for a plurality of the features,
The boundary line recognition apparatus includes a plurality of characteristics as the boundary lines, wherein a ratio of the apparent width calculated by the width calculation unit to the unclearness of the image is larger than a predetermined value.   The boundary line recognition apparatus according to claim 1, wherein the boundary line feature calculation unit calculates the probability lower as the ratio of the apparent width to the unclearness of the image is smaller.   The boundary line according to claim 1 or 2, wherein the boundary line feature calculation unit calculates the probability to a predetermined minimum probability when a ratio of the apparent width to the unclearness of the image is smaller than the predetermined value. Recognition device.   The boundary line feature calculation unit calculates the probability to a predetermined maximum probability when a ratio of the apparent width to the unclearness of the image is larger than a predetermined maximum value set larger than the predetermined value. The boundary line recognition apparatus according to claim 1.   The boundary line recognition according to any one of claims 1 to 4, wherein the unsharpness of the image is a predetermined width of a primary differential distribution calculated by differentiating luminance around the candidate point in the horizontal direction of the image. apparatus.   As for the unclearness of the image, the luminance from the first predetermined value to the second predetermined value larger than the first predetermined value is distributed in the peripheral luminance distribution including the edge candidate points distributed in the horizontal direction of the image. It is a width | variety, The boundary line recognition apparatus in any one of Claims 1-4.   The boundary line recognition apparatus according to any one of claims 1 to 4, wherein the unclearness of the image is a converted value obtained by converting a specification value of a blur amount of a lens included in the camera into a length on the image. The vehicle includes a rain detection means (11) and a fog detection means (12),
When rain or fog is detected by the rain detection means and the fog detection means, the conversion value of the specification value of the blur amount is increased as compared to the case where the rain and the fog are not detected. The boundary line recognition apparatus according to claim 7, wherein the value is corrected as described above.