JP5209656B2 - In-vehicle white line recognition device - Google Patents

Description

  The present invention relates to an in-vehicle white line recognition device that recognizes a white line on a road on which a vehicle is traveling for driving assistance or the like.

Conventionally, an in-vehicle white line recognition device that is used in a lane departure warning device as a driving assistance device and recognizes a white line on a road is known (see Patent Document 1).
The lane departure warning device includes an in-vehicle camera that acquires an image around the vehicle, an image processing unit that processes the image, and an alarm generation unit. The image processing unit detects and estimates a white line from an image obtained from the in-vehicle camera. The warning generator calculates the distance of the white line from the vehicle on the left and right, and if either is below the threshold, that is, if the vehicle approaches the white line beyond the distance set by the threshold, the driver is alerted. give.

JP 2005-18148 A

  Depending on the road conditions, the distance between the left and right white lines (lane width) on the road may be very narrow. For example, in a German construction section as shown in FIG. 8, provisional lines are affixed with tape, and there are multiple lanes, and the width of each lane becomes very narrow. In this state, if a white line is recognized and a lane departure warning is given as usual, false alarms frequently occur.

  Therefore, when the distance between the pair of white lines is narrow, it may be determined that the lane width is narrow and the white lines are not recognized. However, the composite lines are not recognized at all.

  The present invention has been made in view of the above points, and an object thereof is to provide an in-vehicle white line recognition device capable of recognizing a composite line while excluding recognition of a white line constituting a narrow lane.

  The in-vehicle white line recognition device according to the present invention includes an in-vehicle camera that acquires an image around the vehicle, a white line candidate extraction unit that extracts a white line candidate from the image acquired by the in-vehicle camera, and a certainty of white line-likeness of the white line candidate. A certainty factor calculating means for calculating and a white line selecting means for selecting a white line to be recognized based on the certainty factor calculated by the certainty factor calculating means; the certainty factor calculating means for the white line candidate for which the certainty factor is to be calculated; When the following first condition and second condition are satisfied, the certainty factor is lowered as compared with the case where one of them is not satisfied.

  First condition: There are other white line candidates closer to the center of the vehicle than the white line candidate for which the certainty factor is to be calculated, and the distance between them (the white line candidate for which the certainty factor is to be calculated and the other The distance to the white line candidate) is larger than a predetermined lower limit value A.

Second condition: The distance is smaller than a predetermined reference value B (B> A).
The in-vehicle white line recognition device of the present invention has, for example, the white line candidates L4 and L5 in FIG. 7B, the white line candidates L1 to L4 in FIG. 7C, and the white line candidate L1 in FIG. As described above, it is possible to reduce the certainty of white line candidates constituting only narrow lanes and not recognize them. As a result, false alarms do not frequently occur.

Moreover, even if the distance between the white line candidates L1 and L2 constituting the composite line is very narrow as shown in FIG. 7B, it can be correctly recognized without lowering the certainty.
In the present invention, whether the first condition and the second condition is satisfied, it is determined as follows. That is, the certainty factor calculation means searches for other white line candidates that satisfy the first condition, starting from the candidate white line for which the certainty factor is to be calculated, toward the center of the vehicle. Then, the establishment of the second condition can be determined based on the distance between the first other white line candidate that satisfies the first condition and the white line candidate whose reliability is to be calculated. If no other white line candidate satisfying the first condition is found even after the search, it can be determined that the first condition is not satisfied at that time.

Direction of the search, from the white line candidate retrieved immediately near a direction toward the center side of the vehicle. Make Sind calculation means, the direction of the search, when a direction toward the white line candidate for which to compute confidence factor, and end the search, as the distance, sets a predetermined fixed value. As a result, the search does not become an infinite loop.

  The in-vehicle white line recognition device of the present invention includes a plurality of certainty factor calculation means, and the white line candidate for determining whether the first condition and the second condition are satisfied and the white line candidate to be searched are at least one certainty factor calculation means. It can be assumed that the certainty factor calculated by is greater than or equal to a predetermined value. As a result, white line candidates (not actually white lines) caused by noise or the like do not need to be processed, thereby reducing the processing load.

  The in-vehicle white line recognition device according to the present invention is any one of the lower limit value A, the reference value B, or both of road information based on map data, vehicle speed, white line candidate type, limit vehicle speed, and country or region setting. It is possible to provide changing means for changing according to the above. In that case, appropriate control according to the actual road condition can be performed.

2 is a functional block diagram illustrating a schematic configuration of an in-vehicle white line recognition device 1. FIG. It is explanatory drawing showing the position of the vehicle-mounted camera 10 in a vehicle. 3 is a functional block diagram illustrating a schematic functional configuration of an image processing unit 20. FIG. It is a graph showing the setting conditions of reliability. It is a graph showing the setting conditions of reliability. It is explanatory drawing showing the search method of specific white line candidate (beta). (A)-(d) is explanatory drawing showing the calculation result of the certainty degree of a specific white line candidate in the example of a specific road, respectively. It is a photograph showing a road with a very narrow individual lane width.

Embodiments of the present invention will be described with reference to the drawings.
[Embodiment]
1. Configuration and operation of the in-vehicle white line recognition device 1 The configuration and operation of the in-vehicle white line recognition device 1 will be described with reference to FIGS. FIG. 1 is a block diagram showing a schematic configuration of an in-vehicle white line recognition device 1 to which the present invention is applied, FIG. 2 is an explanatory diagram showing the position of an in-vehicle camera 10 in a vehicle, and FIG. FIG. 4 is a functional block diagram showing a schematic functional configuration of the unit 20, FIG. 4 and FIG. 5 are graphs showing confidence setting conditions described later, and FIG. 6 is a description showing a method for searching for a specific white line candidate β described later. FIG. 7 is an explanatory diagram showing an example of calculating the certainty factor in a specific road example.

  As illustrated in FIG. 1, the in-vehicle white line recognition device 1 includes an in-vehicle camera 10 that acquires an image around the vehicle and an image processing device 20 that performs image processing on the image around the vehicle acquired by the in-vehicle camera 10. Using the white line recognized by the in-vehicle white line recognition device 1, an alarm device (not shown) calculates the distance of the white line from the vehicle to the left and right, and if either is below the threshold, that is, the vehicle is set to the white line with the threshold. Alert the driver when approaching distance.

  As shown in FIG. 3, the image processing apparatus 20 includes a plurality of DSPs 30 to 110, a BUS 120 for data transmission between the DSPs 30 to 110, ROM (not shown), RAM, and I / O.

  The DSP 30 is a DSP that performs white line candidate extraction for extracting white line candidates on a road by image processing from an image acquired by the in-vehicle camera 10. Hereinafter, the DSP 30 is referred to as a white line candidate extraction unit 30.

  Specifically, from an image acquired by the in-vehicle camera 10, a white line-like thing on the road (white line candidate) based on the number of votes (solid line / broken line determination result) in the pattern matching and straight line extraction Hough transform processing that are known image processing ). A plurality of white line candidates may be extracted from one frame image.

The DSP 40 is a DSP that calculates the certainty of white line likelihood for the white line candidates extracted by the white line candidate extraction unit 30. Hereinafter, the DSP 40 is referred to as a white line feature calculation unit 40. The certainty factor is a numerical value (likelihood) set within a range of 0.01 to 1, and is set by the white line feature calculation unit 40 for each of the processes (1) to (9) described later. . Therefore, a plurality of certainty factors are set for one white line candidate. The white line feature calculation unit 40 is divided into a first calculation unit 40a that executes the processes (1) to (7) and a second calculation unit 40b that executes the processes (8) to (9). The first calculation unit 40a is configured to calculate the certainty factor based on a single white line, and the second calculation unit 40b is configured to calculate the certainty factor based on the positional relationship between the plurality of white lines. Each processing of (1) to (9) executed by the white line feature calculation unit 40 is as follows.
(1) The certainty factor is calculated based on the ratio of the contrast of the white line candidate to the road surface around the white line candidate (excluding the white line candidate part) (hereinafter, the contrast ratio of the white line candidate itself).

Specifically, as shown in FIG. 4, if the contrast ratio of the white line candidate itself is lower than 150%, the certainty factor is lowered. In FIG. 4, the horizontal axis is the contrast ratio of the white line candidate itself, and the vertical axis is the certainty factor to be set.
(2) As a feature indicating the likelihood of white line, either the internal luminance distribution or the internal edge amount of the white line candidate extracted by the white line candidate extraction unit 30 is used.

The white line is usually white and diffusely reflects light emitted from a lighting device such as a vehicle headlight or street light, and therefore has a larger luminance dispersion value than the road surface.
Further, the size of the inner edge amount indicates the flatness of the region. In other words, if the flatness is large, the region has few edges. Therefore, if the internal edge amount is small, the flatness is large, and conversely, if the internal edge amount is large, the flatness is small. Since the white line has a greater flatness than the road surface, the amount of internal edges is small.

Therefore, the internal luminance variance or the internal edge amount of the white line candidate is calculated, and the certainty factor is set according to the calculated internal luminance variance or the internal edge amount. If it is small, the confidence level is increased.)
(3) As a characteristic indicating the likelihood of white lines, either the edge direction dispersion of white line candidates or the peak ratio or size of the histogram in the edge direction is used.

  The white line is usually drawn in one direction on the road (even if the road is curved, the curvature is relatively small, so it is considered to be drawn in one direction in the range of the image acquired by the in-vehicle camera 10. Therefore, the dispersion of the edge direction of the white line is larger than the road surface. Further, since the white line is a line segment, the ratio and size of the histogram in the edge direction are larger than the road surface.

Therefore, as information indicating the likelihood of white lines, the distribution of the edge direction of white line candidates and the ratio or size of the histogram of the edge direction can be used.
Therefore, the variance of the edge direction of the white line candidate, the ratio or size of the histogram of the edge direction are calculated, and the certainty factor is set according to the variance of the edge direction of the candidate white line, the ratio or size of the histogram of the edge direction. (If the variance in the edge direction of the white line candidate and the ratio or size of the histogram in the edge direction are large, the certainty factor is increased, and if it is small, the certainty factor is decreased.)
(4) If the pattern of the white line candidate itself extracted by the road surface feature extraction unit 70 or the difference in average brightness is equal to or smaller than a predetermined value, the certainty of the white line candidate is lowered.

  Specifically, a shadow may be reflected on the road surface by a guard rail or the like on the side of the road. For example, light that is inserted through a gap between guardrails due to direct sunlight from the lateral direction may be reflected in a straight line on the road surface and look like a white line. There are other cases where the area surrounded by black coal tar repairs appears in a straight line.

In order to discriminate between the white line and the area surrounded by the shadow or repair trace on the road, the pattern and average luminance of the area and the white line candidate may be compared. Accordingly, if the pattern difference between the area extracted by the road surface feature extraction unit 70 and the white line candidate is large, the confidence level is increased as the two are different, and if the average luminance difference is small, the two lines are matched. Decrease confidence.
(5) When the difference in the total amount of edges extracted by the edge feature extraction unit 80 is equal to or less than a predetermined value, the certainty of white line candidates is lowered.

  Even if the brightness difference with the road surface is low due to fading of the white line, it looks like a white line to the human eye. This is due to the human sensitivity that spots appear to be emphasized when there is no pattern on the road.

Therefore, if the difference between the total amount of edges of the area surrounded by the white line candidate and the white line extracted by the edge feature extraction unit 80 is a predetermined value or less, the certainty of the white line candidate is lowered.
(6) When a streak pattern is detected by the streak pattern detection unit 90, the certainty factor is lowered.

In some cases, vertical stripes are carved on the road to prevent the vehicle from slipping. In that case, the vertical stripes carved on the road have lower brightness (darker) than the road surface, but the brightness of the vertical stripe ends may not be different from the white line ends. Therefore, when a streak pattern is detected on a white line candidate by the streak pattern detection unit 90, the certainty factor of the white line candidate is lowered.
(7) When a traffic sign is detected by the traffic sign detection unit 100, the certainty factor is lowered.

On the road, in addition to white lines, traffic signs such as arrows, speed limits, stop letters, and pedestrian crossings are drawn. Therefore, when the traffic sign detection unit 100 detects a traffic sign or the like at the position of the white line candidate, the certainty of the white line candidate is lowered.
(8) In this process, among the white line candidates extracted by the white line candidate extraction unit 30, the first likelihood calculated by the first white line feature integration unit 50 described later is a white line candidate (hereinafter referred to as a predetermined value). , A specific white line candidate) only. Among the specific white line candidates, when there is one detected by the composite line detection unit 110 as a combination (composite line) of a plurality of white line candidates, among the combinations, the innermost (most different from the center line of the vehicle) Only the certainty of the white line candidate on the side) is not changed (maintained at 0.50), and the certainty of other white line candidates is uniformly set to 0.01. When there is a composite line, the processing becomes complicated if there are a plurality of white line candidates to be recognized on one side of the road. Therefore, the certainty is increased so that only the innermost white line candidate is recognized.
(9) In this process, only specific white line candidates are processed. Then, any specific white line candidate is set as a specific white line candidate (hereinafter referred to as a specific white line candidate α) whose reliability is to be calculated. Starting from the specific white line candidate α, the other specific white line candidates satisfying the following first condition are sequentially searched toward the center of the host vehicle.

First condition: Other white line candidates exist closer to the center of the vehicle than the specific white line candidate α, and the distance between them is greater than the lower limit A (45 cm).
If there is a specific white line candidate that satisfies the first condition, it is set as a specific white line candidate β. When there are a plurality of specific white line candidates satisfying the first condition, the specific white line candidate first found in the above search is set as the specific white line candidate β. Then, it is determined whether or not the following second condition is satisfied for the distance D between the specific white line candidate α and the specific white line candidate β.

Second condition: The distance D is smaller than the reference value B (2.75 m).
When the first condition and the second condition are satisfied, the certainty factor of the specific white line candidate α is lowered. Specifically, as shown in FIG. 5, the certainty factor is set according to the distance D. The horizontal axis of FIG. 5 is the distance D, and the vertical axis is the certainty factor to be set. When the distance D is 2.75 m or more, the certainty factor is constant at 0.50. However, when the distance D is 2.25 to 2.75 m, the certainty factor decreases as the distance D decreases, and the distance D decreases. In the case of 2.25 m or less, the certainty factor is uniformly 0.01.

  On the other hand, when the first condition is not satisfied (when other specific white line candidates exist only in a range closer to the lower limit A, or when other specific white line candidates are not found even after searching) When the second condition is not satisfied (when the distance D is greater than or equal to the reference value B), the certainty factor of the specific white line candidate α is left at the initial value (0.50). Each of the specific white line candidates is set as the specific white line candidate α, and the above processing is performed.

  The above search is performed as follows. As shown in FIG. 6A, the search is performed from the specific white line candidate α on the road 201 toward the center ML of the vehicle. It is determined whether or not the first condition is satisfied with the other specific white line candidate L1 found first, and if the first condition is satisfied, the specific white line candidate L1 is set as the specific white line candidate β. On the other hand, if the first condition is not satisfied, the search is further performed from the specific white line candidate L1 toward the center ML of the vehicle. Similarly, for the other specific white line candidate L2 discovered next, it is determined whether the first condition is satisfied, and if the first condition is satisfied, the specific white line candidate L2 is set as the specific white line candidate β and is satisfied. If not, a search is further performed from the specific white line candidate L2 toward the center ML of the vehicle. This procedure is continued until, in principle, another specific white line candidate that satisfies the first condition is found or no other white line candidates are found.

  However, in the case as shown in FIG. 6B, the search is terminated halfway. That is, in the case shown in FIG. 6B, even if the specific white line candidates L1, L2, L3... Ln, Ln + 1 are sequentially searched starting from the specific white line candidate α on the road 201, the first condition Assume that no specific white line candidate satisfying the condition is found. Here, it is assumed that the specific white line candidate Ln is on the same side as the specific white line candidate α with respect to the vehicle center ML, but the specific white line candidate Ln + 1 is on the opposite side of the specific white line candidate α with respect to the vehicle center ML. .

  In this case, if the above-described principle is followed, the search is further performed from the specific white line candidate Ln + 1 to the center ML side of the vehicle (in the direction of the specific white line candidate α). When the direction is toward the specific white line candidate α, the search is terminated. Then, a predetermined fixed value (a value larger than the reference value B) is set as the distance D, and the second condition is not satisfied. This is because if the search is continued as it is, the search falls into an infinite loop (specific white line candidates Ln and Ln + 1 continue to be searched alternately).

  This process will be described in more detail based on the example shown in FIG. In the case of FIG. 7A, specific white line candidates L1, L2, and L3 exist on the road 201. 203 is the own vehicle. The interval between the specific white line candidates L1 and L2 and the interval between L2 and L3 are larger than the lower limit value A and the reference value B, respectively. In the following, the interval between the specific white line candidates Li and Lj is expressed as dij (i and j are natural numbers). FIG. 7A shows an example in which there is a wide lane (a lane between specific white line candidates L1 and L2 and a lane between specific white line candidates L2 and L3).

  When the specific white line candidate L1 is the specific white line candidate α, the search is performed in the direction of the specific white line candidate L2, and another specific white line candidate L2 is found. Since d12 is larger than the lower limit A as described above, the specific white line candidates L1 and L2 satisfy the first condition, and the specific white line candidate L2 becomes the specific white line candidate β. Since d12 is larger than the reference value B as described above, the second condition is not satisfied, and the certainty of the specific white line candidate L1 remains 0.50 (FIGS. 7A to 7D). Then, it is expressed by ○ that the certainty does not decrease).

  Similarly, when the specific white line candidate L2 is the specific white line candidate α, the specific white line candidate L1 becomes the specific white line candidate β and the second condition is not satisfied (because d12> reference value B), so the specific white line candidate L2 The certainty remains at 0.50. Further, when the specific white line candidate L3 is the specific white line candidate α, the specific white line candidate L2 becomes the specific white line candidate β and the second condition is not satisfied (d23> reference value B), so the specific white line candidate L3 The certainty level remains at 0.50. Therefore, when there is a wide lane, specific white line candidates on both sides of the lane can be correctly recognized.

  In the case shown in FIG. 7B, there are specific white line candidates L1, L2, L3, L4, and L5 on the road 201. 203 is the own vehicle. Here, d12 <lower limit value A, d13> reference value B, d23> reference value B, lower limit value A <d34 <reference value B, and lower limit value A <d45 <reference value B. . FIG. 7B shows a wide lane (a lane sandwiched between specific white line candidates L2 and L3), a narrow lane (a lane sandwiched between specific white line candidates L3 and L4, and a lane sandwiched between specific white line candidates L4 and L5). This is an example that exists. Moreover, the specific white line candidates L1 and L2 constitute a composite line.

  When the specific white line candidate L1 is the specific white line candidate α, the search is performed in the direction of the specific white line candidate L2. However, since the specific white line candidates L1 and L2 do not satisfy the first condition (d12 <lower limit value A, ), The search further proceeds, and a specific white line candidate L3 is found. Since the specific white line candidates L1 and L3 satisfy the first condition (d13> lower limit A), the specific white line candidate L3 becomes the specific white line candidate β. Since d13 is larger than the reference value B as described above, the second condition is not satisfied, and the certainty of the specific white line candidate L1 remains 0.50.

  Similarly, when the specific white line candidate L2 is the specific white line candidate α, the specific white line candidate L3 becomes the specific white line candidate β, and the second condition is not satisfied (d23> reference value B), so the specific white line candidate L2 The certainty remains at 0.50. Further, when the specific white line candidate L3 is the specific white line candidate α, the specific white line candidate L2 becomes the specific white line candidate β and the second condition is not satisfied (d23> reference value B), so the specific white line candidate L3 The certainty level remains at 0.50. Further, when the specific white line candidate L4 is set as the specific white line candidate α, the specific white line candidate L3 becomes the specific white line candidate β, and the second condition is satisfied (d34 <reference value B), so the specific white line candidate L4 The certainty factor decreases from 0.50 according to FIG. 5 (in FIG. 7B to 7D, the certainty factor is represented by x). When the specific white line candidate L5 is the specific white line candidate α, the specific white line candidate L4 becomes the specific white line candidate β and the second condition is satisfied (d45 <reference value B), so the specific white line candidate L5 The certainty factor decreases from 0.50 according to FIG.

  Therefore, the specific white line candidates L2 and L3 on both sides of the wide lane can be recognized, and the specific white line candidate L1 constituting the composite line can also be recognized. On the other hand, specific white line candidates L4 and L5 sandwiching a narrow lane can be prevented from being recognized.

  In the case shown in FIG. 7C, specific white line candidates L1, L2, L3, and L4 exist on the road 201. 203 is the own vehicle. Here, d12, d23, and d34 are all larger than the lower limit value A and smaller than the reference value B. FIG. 7C shows an example in which only narrow lanes (lanes sandwiched between specific white line candidates L1 and L2, lanes sandwiched between specific white line candidates L2 and L3, and lanes sandwiched between specific white line candidates L3 and L4) exist. is there.

  When the specific white line candidate L1 is the specific white line candidate α, the search is performed in the direction of the specific white line candidate L2, and the specific white line candidate L2 is found. Since the specific white line candidates L1 and L2 satisfy the first condition (d12> lower limit A), the specific white line candidate L2 becomes the specific white line candidate β. Since d12 is smaller than the reference value B as described above, the second condition is satisfied, and the certainty factor of the specific white line candidate L1 decreases from 0.50 according to FIG.

  Similarly, when the specific white line candidate L2 is the specific white line candidate α, the specific white line candidate L3 becomes the specific white line candidate β and the second condition is satisfied (because d23 <reference value B), so the specific white line candidate L2 The certainty is reduced from 0.50 according to FIG. When the specific white line candidate L3 is the specific white line candidate α, the specific white line candidate L2 becomes the specific white line candidate β and the second condition is satisfied (because d23 <reference value B), so that the specific white line candidate L3 The certainty factor decreases from 0.50 according to FIG. When the specific white line candidate L4 is the specific white line candidate α, the specific white line candidate L3 becomes the specific white line candidate β and the second condition is satisfied (because d34 <reference value B), so that the specific white line candidate L4 The certainty factor decreases from 0.50 according to FIG.

Therefore, the specific white line candidates L1, L2, L3, and L4 sandwiching the narrow lane can be prevented from being recognized.
In the case illustrated in FIG. 7D, specific white line candidates L <b> 1 and L <b> 2 exist on the road 201. 203 is the own vehicle. Here, d12 is larger than the lower limit value A and smaller than the reference value B. FIG. 7D shows a narrow lane (a lane between specific white line candidates L1 and L2) and a wide lane (a specific white line candidate L2 exists only on the left side and a specific white line candidate cannot be recognized on the right side). This is an example.

  When the specific white line candidate L1 is the specific white line candidate α, the search is performed in the direction of the specific white line candidate L2, and the specific white line candidate L2 is found. Since the specific white line candidates L1 and L2 satisfy the first condition (d12> lower limit A), the specific white line candidate L2 becomes the specific white line candidate β. Since d12 is smaller than the reference value B as described above, the second condition is satisfied, and the certainty factor of the specific white line candidate L1 decreases from 0.50 according to FIG.

  If the specific white line candidate L2 is the specific white line candidate α, the search is performed in the direction of the host vehicle 203, but there is no other specific white line candidate, so the specific white line candidate β does not exist, and the first condition Does not hold. As a result, the certainty factor of the specific white line candidate L2 remains 0.50.

Therefore, the specific white line candidate L1 that sandwiches only the narrow lane can be prevented from being recognized. On the other hand, the specific white line candidate L2 across the wide lane can be recognized.
The DSP 50 is a DSP that outputs the first likelihood indicating the likelihood of a white line by multiplying and integrating the certainty calculated and output in the above (1) to (7). This is called a first white line feature integration unit 50.

  The integration processing in the first white line feature integration unit 50 is determined from integration by multiplication of certainty factors based on a Bayesian estimation framework. For example, if the white line-likeness certainty factors A and B calculated by two of the processes shown in (1) to (7) of the white line feature calculation unit 40 are 0.6 and 0.7, respectively, integration is performed. The first likelihood is (0.6 * 0.7) / (0.6 * 0.7 + 0.4 * 0.3) = 0.78.

Here, 0.4 and 0.3 are not likely white lines due to A and B (1-0.6, 1-0.7), respectively.
Also, when the processing in the white line feature calculation unit 40 increases, the contents of the integration process are essentially unchanged just by increasing the number of input certainty factors of the likelihood integration process, and are highly extensible. Accordingly, it is possible to realize a structure in which new information can be easily added or deleted while realizing high accuracy and high robustness.

  The DSP 55 further multiplies and integrates the certainty calculated and output in (8) to (9) above and the first likelihood integrated in the first white line feature integration unit 50. The DSP 55 outputs a second likelihood indicating the likelihood of a white line. Hereinafter, the DSP 55 is referred to as a second white line feature integration unit 55. The integration process in the second white line feature integration unit 55 is performed in the same manner as the integration process in the first white line feature integration unit 50.

  For white line candidates for which the processes (8) to (9) are not performed, the first likelihood calculated by the first white line feature integration unit 50 is used as it is, and the second white line feature integration unit 55 is used as it is. It is set as the 2nd likelihood calculated by (2).

  The DSP 60 is a DSP that selects a white line candidate having the maximum second likelihood among the second likelihoods output from the second white line feature integration unit 55 as a white line on the road. Hereinafter, the DSP 60 is referred to as a white line. This is called a selection unit 60.

  Since the second white line feature integration unit 55 outputs the second likelihood that the “white line likelihood” is integrated with respect to the white line candidate, the white line selection unit 60 determines the integrated second likelihood. The largest white line is selected as the white line on the road.

  The DSP 70 is a DSP that extracts features including an area on the road sandwiched between white line candidates extracted by the white line candidate extraction unit 30 and a pattern or average luminance of the white line candidates themselves from an image acquired by the in-vehicle camera 10. This DSP 70 is referred to as a road surface feature extraction unit 70.

  For example, light that is inserted through the gap of the guardrail due to direct sunlight from the lateral direction may be reflected on the road surface in a straight line, and it may appear as a white line or an area surrounded by repair marks from coal tar may appear in a straight line. Then, the pattern of the area and the white line is extracted by pattern matching or a luminance difference.

  The DSP 80 is a DSP that extracts from the image acquired by the in-vehicle camera 10 a region on the road sandwiched between white line candidates extracted by the white line candidate extraction unit 30 and the total amount of edges of the white line candidates themselves. This is called an extraction unit 80.

The total amount of edges is obtained by extracting edges from a binarized image obtained by binarizing the in-vehicle camera 10 and summing the number of extracted edges.
In the DSP 90, the edge amount in the vehicle traveling direction on the road in a region near the vehicle traveling direction of the white line candidate extracted by the white line candidate extracting unit 30 from the image acquired by the in-vehicle camera 10 is a predetermined value or more. This DSP 90 is hereinafter referred to as a streak pattern detection unit 90.

  In order to prevent the vehicle from slipping, the vertical stripes carved on the road have lower luminance (darker) than the road surface, but the luminance at the edge of the vertical stripe may not be different from the edge of the white line. Therefore, the edge amount is calculated from the binarized image of the image acquired by the in-vehicle camera 10 and the edge amount in the vehicle traveling direction on the road in the region near the vehicle traveling direction of the white line candidate is not less than a predetermined value. It is detected as a pattern.

The DSP 100 is a DSP that detects a traffic sign drawn on a road from an image acquired by the in-vehicle camera 10. Hereinafter, the DSP 100 is referred to as a traffic sign detection unit 100.
The traffic sign detection unit 100 detects a traffic sign drawn on the road by pattern matching. As traffic signs to be detected, there are pedestrian crossings and the like in addition to traffic signs drawn on the road surface such as arrows, speed limits, and stop characters.

  The DSP 110 is a DSP that detects a combination (composite line) of white line candidates whose distance is within 45 cm among the white line candidates extracted by the white line candidate extraction unit 30. Hereinafter, the DSP 110 is combined with the composite line detection unit 110. Call. The value of 45 cm is a value set so that a composite line can be detected but white line candidates other than the composite line cannot be detected. This value can be set as appropriate according to road conditions and the like.

2. Effects produced by the in-vehicle white line recognition device 1
(i) The in-vehicle white line recognition device 1 can prevent white lines constituting a narrow lane from being recognized. As a result, false alarms do not occur frequently. Moreover, the white line which comprises a compound line can be recognized.
(ii) The in-vehicle white line recognition device 1 ends the search when searching for the specific white line candidate β in the process of (9) above, when the direction of the search is a direction toward the specific white line candidate α. Therefore, the search does not become an infinite loop.
(iii) The in-vehicle white line recognition device 1 uses only specific white line candidates (white line candidates whose first likelihood is equal to or greater than a predetermined value) in the process (9). That is, the specific white line candidate α and the white line candidates to be searched are only specific white line candidates. This eliminates the need for white line candidates due to noise or the like as processing targets, thereby reducing the processing burden.

 (iv) The in-vehicle white line recognition device 1 calculates the likelihood by integrating the certainty factors regarding a plurality of types of white line features. Therefore, whatever the type of white line feature calculation unit 40, in other words, even if the certainty factor is calculated by any processing content, they are easily combined to obtain the likelihood, A white line candidate can be selected based on the likelihood.

Therefore, even when the white line feature calculation unit 40 is improved or a process of a new function is added, the plurality of processes can be easily integrated.
(v) The white line feature calculation unit 40 uses at least one of the luminance distribution of the white line candidate extracted by the white line candidate extraction unit 30 or the internal edge amount of the white line as a characteristic indicating the white line characteristic. A difference in luminance can be obtained, and a difference in pattern flatness can be obtained from the amount of internal edges.

That is, since it is possible to obtain a difference between a guard rail having different brightness and pattern flatness, a scratch on the road, a repair trace, and a white line, it is possible to obtain a more accurate certainty of the white line.
Also, the white line feature calculation unit 40 uses, as a feature indicating white line likeness, either the edge direction variance of the white line candidate extracted by the white line candidate extraction unit 30 or the ratio or size of the peak of the histogram in the edge direction. .

  Therefore, even when the white line on the road is fading and disappearing, or when the white line is faint and thin, the white line edge direction variance or edge peak histogram ratio or size The difference between the white line, the grazing line, and the road surface repair trace can be obtained by either of them. Accordingly, it is possible to obtain a more accurate certainty of white line-likeness.

  In addition, a road surface feature extraction unit 70 that extracts a feature on the road between the white line candidates extracted by the white line candidate extraction unit 30 and features including the pattern or average luminance of the white line candidates themselves from the image acquired by the in-vehicle camera 10. The white line feature calculation unit 40 reduces the certainty of the white line likelihood of the white line candidate when the difference in the pattern or average luminance of the white line candidate itself extracted by the road surface feature extraction unit 70 is a predetermined value or less. .

  Therefore, if there is something with a brightness or pattern different from the white line, such as a shadow by a roadside guardrail or a road surface repair mark, the certainty can be lowered, and the accuracy of white line recognition can be increased. .

  The image processing apparatus further includes an edge feature extraction unit 80 that extracts a region on the road between the white line candidates extracted by the white line candidate extraction unit 30 and the total amount of edges of the white line candidates themselves from the image acquired by the in-vehicle camera 10, and includes a white line feature calculation unit 40, when the difference in the total amount of edges extracted by the edge feature extraction unit 80 is less than or equal to a predetermined value, the certainty of white line likelihood of a white line candidate is lowered.

Therefore, even when the white line is faint, the certainty as a white line candidate can be prevented from decreasing.
In addition, from the image acquired by the in-vehicle camera 10, a white line candidate extracted by the white line candidate extraction unit 30 in a region in the vicinity of the direction substantially perpendicular to the vehicle traveling direction has a luminance in the vehicle traveling direction on the road that is a predetermined value or less. A streak pattern detection unit 90 that detects a pattern of shapes is provided.

  In addition, when a streak pattern (for example, a slip prevention groove) is detected by the streak pattern detection unit 90, the white line feature calculation unit 40 reduces the certainty of white line likelihood of a white line candidate. Therefore, when vertical stripes are carved on the concrete road surface to prevent slipping, it is possible to reduce the certainty of the white line-likeness of the vertical stripes.

  In addition, a traffic sign detection unit 100 that detects a traffic sign drawn on a road from an image acquired by the in-vehicle camera 10 is provided, and when a traffic sign is detected at a position of a white line candidate by the traffic sign detection unit 100, a white line The feature calculation unit 40 reduces the certainty of the white line likelihood of the white line candidate. Therefore, the certainty of the white line-likeness of the traffic sign drawn on the road can be lowered.

In addition, this invention is not limited to the said embodiment at all, and it cannot be overemphasized that it can implement with a various aspect in the range which does not deviate from this invention.
For example, the in-vehicle white line recognition device 1 may include one or more of the following (A) to (E).

  (A) Means for obtaining road information based on map data: Map data can be obtained from a navigation system, a communication device connected to an external server, or the like. The road information includes the width of the road, the number of lanes, the type of road (general road, highway, etc.), and the like.

(B) Means for detecting the speed of the own vehicle: The type of road (such as a general road or a highway) can be determined based on the speed of the own vehicle.
(C) Means for detecting the type of white line candidate: Examples of the type of white line candidate include a solid line, a broken line, a botsdot, a single line, a composite line, and the like.

  (D) Means for detecting road speed limit: The speed limit can be obtained from map data, for example. The type of road (general road or highway, etc.) can be determined based on the limit vehicle speed.

  (E) Means capable of setting the country or region (the destination region, for example, North America): This setting may be obtained from the navigation system or may be provided with a device for recognition separately. Moreover, you may provide in the production line of the vehicle-mounted white line recognition apparatus 1. FIG.

  And the vehicle-mounted white line recognition apparatus 1 estimates the road condition (for example, the width of the lane) based on the information obtained in any of the above (a) to (e), and the lower limit value A, the reference value B, or A means for changing both of them to an optimum value according to the road condition may be provided.

In the above-described embodiment, various processes are assigned to a plurality of DSPs and executed, but may be executed as a program in one CPU without being assigned to a DSP.
In the embodiment, the white line feature calculation unit 40 is configured by one DSP. However, the internal processing of the white line feature calculation unit 40 as illustrated in FIG. 3 may be allocated to a plurality of DSPs. In this way, higher speed processing is possible.

The white line feature calculation unit 40 may perform the following processing in addition to or in place of the processing (1) to (9).
Total extension distance: The longer the length of the white line candidate in the longitudinal direction, the greater the certainty of the white line candidate.

  Further, the in-vehicle white line recognition device 1 uses the recognized white line to calculate the distance of the white line on the left and right from the own vehicle. An alarm unit may be provided that gives an alarm to the driver when approaching the distance set by the threshold.

  Further, the method of setting the certainty factor according to the distance D may be other than that shown in FIG. For example, if the reference value B (for example, a value set in the range of 2.25 to 2.75 m) is exceeded, the certainty factor is uniformly 0.50, and if it is less than the reference value B, the certainty factor is uniformly 0. .01 may be used.

DESCRIPTION OF SYMBOLS 1 ... Car-mounted white line recognition apparatus, 10 ... Car-mounted camera, 20 ... Image processing apparatus,
30 ... White line candidate extraction unit, 40 ... White line feature calculation unit,
50 ... 1st white line feature integration part, 55 ... 2nd white line feature integration part,
60 ... White line selection unit, 70 ... Road surface feature extraction unit, 80 ... Edge feature extraction unit,
90 ... streak pattern detection unit, 100 ... traffic sign detection unit,
110 ... Composite line detection unit, 201 ... Road, 203 ... Own vehicle,
L1, L2, L3, L4, L5 ... Specific white line candidates

Claims (3)

An in-vehicle camera that acquires images around the vehicle,
White line candidate extraction means for extracting white line candidates from the image acquired by the in-vehicle camera by image processing;
A certainty factor calculating means for calculating a certainty factor of white line likelihood of the white line candidate;
A white line selection unit that selects a white line to be recognized based on the certainty factor calculated by the certainty factor calculation unit;
With
The certainty factor calculating means starts from a white line candidate that is a target for calculating the certainty factor, and sequentially searches for another white line candidate that satisfies the following first condition toward the center of the vehicle, A target for calculating the certainty factor by determining whether the following second condition is satisfied based on a distance between the first other white line candidate satisfying the first condition and a white line candidate for which the certainty factor is calculated. for white line candidate having the, when the first condition and the second condition is satisfied, compared with the case of not satisfied even one of them, lower the confidence,
The direction of the search is a direction from the most recently searched white line candidate toward the center of the vehicle,
The certainty factor calculating means terminates the search and sets a predetermined fixed value as the distance when the direction of the search is a direction toward a white line candidate for which the certainty factor is to be calculated. An in-vehicle white line recognition device characterized by the above.
First condition: the center side of the vehicle than the white line candidate for which to calculate the certainty factor, the other is present white line candidate, the distance between them is greater than a predetermined lower limit value A.
Second condition: The distance is smaller than a predetermined reference value B (B> A). A plurality of the certainty factor calculating means;
White line candidate to determine the establishment of the first condition and the second condition, according to claim 1, wherein the confidence at least one of the confidence factor computing means is calculated is not less than a predetermined value In-vehicle white line recognition device. Changing means for changing the lower limit value A and / or the reference value B according to any of road information based on map data, speed of the vehicle, type of candidate white line, speed limit vehicle, and country or region setting vehicle white line recognition apparatus according to claim 1 or 2, characterized in that it comprises a.