JP5677900B2 - In-vehicle white line recognition device - Google Patents

Description

  The present invention relates to an in-vehicle white line recognition device that recognizes a white line on a road on which a vehicle is traveling for driving assistance or the like.

Conventionally, an in-vehicle white line recognition device that is used in a lane departure warning device as a driving support device and recognizes a white line on a road is known (see Patent Document 1).
The lane departure warning device includes an in-vehicle camera that acquires an image around the vehicle, an image processing unit that processes the image, and an alarm generation unit. The image processing unit detects a white line from an image obtained from the in-vehicle camera. The warning generator calculates the distance between the vehicle and the white line on the left and right, and if either is below the threshold, that is, if the vehicle approaches the white line beyond the distance set by the threshold, the vehicle will deviate from the lane. Alert the driver as the possibility increases.

  One of the situations that cause problems with such an apparatus is a situation where the white line on the road is a composite line composed of a plurality of white lines. The composite line is not only composed of two white lines, but may be composed of three white lines or a thick block-like broken line, and the width and shape of the entire composite line are not constant. Therefore, in order to determine the departure of the lane as described above, it is necessary to determine any part of the composite line as a white line to be controlled (hereinafter also simply referred to as a control target line).

  As a device having a function of determining a control target line from a composite line, a lane boundary detection device described in Patent Document 2 has been proposed. This apparatus discriminates a plurality of composite line patterns, and sets one of the innermost, middle, and innermost plus alpha offsets within the width of the composite line as a control target line.

JP 2005-18148 A JP 2003-178399 A

  The determination of the control target line in the composite line is preferably changed according to the type of the line constituting the composite line. Specifically, when a short dashed line is included in the composite line, this broken line may be an auxiliary line. Selecting this auxiliary line as a control target line will compare it with selecting an appropriate white line. As a result, the lane departure warning device determines that the vehicle will depart from the lane early or late, and the warning cannot be performed at an appropriate timing. On the other hand, if the length of the broken line is long, it may be handled in the same manner as the solid line. For example, in the case of a combined line of a solid line and a long broken line, the long broken line may be interpreted in the same way as the solid line and interpreted as an alarm target timing. As described above, there is a case where it is not appropriate to select the control target line based simply on the broken line or the solid line.

The present invention has been made in view of the above-described problems, and an object thereof is to provide an in-vehicle white line recognition device that can appropriately select a white line to be controlled.
Another object of the present invention is to provide an in-vehicle white line recognition device that can determine the type of white line with high accuracy.

  The invention described in claim 1 made to solve the above-described problem is an in-vehicle white line recognition device including an in-vehicle camera that acquires an image around the vehicle, and further has the following characteristics.

  First, a white line is extracted from an image acquired by an in-vehicle camera by image processing. As a method for extracting a white line, a known method such as a method of performing edge conversion from an image by detecting an edge point or pattern matching can be employed.

  Then, it is determined whether or not the extracted white line is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance. Although the determination method is not particularly limited, for example, the distance of a white line can be estimated from an image around the vehicle, and a line having a predetermined distance or less can be determined as a composite line. Further, it may be determined by pattern matching or the like.

  Further, the extracted white line is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more, and a short broken line whose length is less than the predetermined threshold. The length of the broken line here refers to the length of the paint part among the paint part where the line in the broken line exists, that is, the colored part on the road, and the gap part corresponding to the interval between the lines.

And this vehicle-mounted white line recognition apparatus selects the white line used as the object of predetermined | prescribed control from the extracted white line. At this time, for the white line determined to constitute the composite line, the white line determined as the long broken line is selected as the white line to be controlled with priority over the white line determined as the short broken line. Examples of the predetermined control include lane departure determination and automatic steering.

  In the on-vehicle white line recognition device configured as described above, a white line excluding a short broken line constituting a composite line is preferentially selected as a white line to be controlled. That is, it is possible to appropriately select a white line to be a control target by suppressing the short broken line drawn as the auxiliary line from being selected as the control target. As a result, for example, it becomes difficult to cause a problem that determination of lane departure is made at an incorrect timing. On the other hand, the long broken line is not excluded, and the alarm can be correctly set.

As described above, the method for determining the white line as one of a solid line, a long broken line, and a short broken line is not particularly limited. For example, it is possible to estimate the length of the broken line by a known method such as pattern matching and determine whether the length exceeds a predetermined threshold. Further, to detect the edge points from the acquired image with a car mounting camera, the number of votes Hough transform the edge point extracting white lines on the basis of the obtained straight line by Hough transform, corresponding to the extracted white lines The white line may be determined to be a long broken line when the time change of is larger than a predetermined threshold.
The invention according to claim 2 is based on an in-vehicle camera that acquires an image around the vehicle, and a straight line obtained by detecting an edge point from the image acquired by the in-vehicle camera and performing Hough transform on the edge point. A white line extracting means for extracting a white line, and a composite line determining means for determining whether the white line extracted by the white line extracting means is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance. And the white line extracted by the white line extraction means is at least one of a solid line, a long broken line whose length is not less than a predetermined threshold, and a short broken line whose length is a broken line less than the predetermined threshold. Line type determination means, and control line selection means for selecting a white line to be subjected to predetermined control from the white lines extracted by the white line extraction means, the control line selection means, composite For the white line determined to constitute the composite line by the determining means, the white line determined as the solid line or the long broken line by the line type determining means is given priority over the white line determined as the short broken line. The white line to be controlled is selected, and the line type determination unit selects the white line when the time change in the number of votes of the Hough transform corresponding to the white line extracted by the white line extraction unit is larger than a predetermined threshold. An in-vehicle white line recognition device characterized in that the long dashed line is determined.

  The number of votes for the Hough transform is determined by the length of the paint part existing in the area to be image-processed in the captured image acquired by the in-vehicle camera. In the long broken line, the length of the paint part and the gap part is longer than that of the short broken line. Therefore, the difference between the case where the paint part occupies a large and small part in the region is large, and the time change of the number of votes increases accordingly.

  On the other hand, since the short broken line has a short paint portion and gap portion, a plurality of paint portions are likely to exist in the above-described region, the proportion of the paint portion is stable, and the change in the number of votes with time is small. By using such a feature, the length of the broken line can be accurately determined.

The invention described in claim 3 is an in-vehicle white line recognition device including an in-vehicle camera that acquires an image around the vehicle, and further has the following characteristics.
First, a white line is extracted from an image acquired by an in-vehicle camera by image processing. Then, it is determined whether or not the extracted white line is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance. Further, the extracted white line is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more, and a short broken line whose length is less than the predetermined threshold. The above processing is the same as the on-vehicle white line recognition device according to claim 1.

And this vehicle-mounted white line recognition apparatus selects the white line used as the object of predetermined | prescribed control from the extracted white line. At this time, if it is determined that each of the white lines extracted most on the vehicle center side on the left and right sides of the vehicle is a white line constituting the composite line and is determined to be a short broken line, it is determined as the short broken line. The long broken line constituting the composite line is selected as a white line to be controlled preferentially over the white line determined as the short broken line from the composite lines formed by the white line.

  In the in-vehicle white line recognition device configured as described above, when a short broken line of a composite line exists on the left and right center sides of the vehicle, the white line excluding the short broken line is preferentially selected as a control target. When a short broken line is detected on the left and right of the vehicle, the short broken line is highly likely to be an auxiliary line. The white line to be selected can be appropriately selected.

According to a fourth aspect of the present invention, there is provided an in-vehicle camera that acquires an image around a vehicle, a white line extracting unit that extracts a white line from an image acquired by the in-vehicle camera, and a white line extracted by the white line extracting unit. Composite line determination means for determining whether or not a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance, and the white line extracted by the white line extraction means is at least a solid line, a long line A line type determining unit that determines whether the long broken line is a broken line having a length equal to or greater than a predetermined threshold, and a short broken line that is a broken line having a length less than the predetermined threshold; and the white line extracted by the white line extracting unit The control line selection means for selecting a white line to be controlled from the inside, and the time change of the thickness of the white line extracted by the white line extraction means is measured. Thick And a dividing unit that divides the extracted white line into two white lines consisting of a broken line and a solid line when the state and the thin state vary, and the line type determination unit has a thickness of the white line. When the period measured as the thick state is larger than a predetermined threshold value, the divided broken line is determined to be the long broken line, and the control line selecting unit configures a combined line by the combined line determining unit. For the determined white line, the white line determined as the solid line or the long broken line by the line type determining unit is selected as the white line to be controlled in preference to the white line determined as the short broken line. This is an in-vehicle white line recognition device.

  The in-vehicle white line recognition device configured as described above can determine that when the thickness of the extracted white line changes, the white line is erroneously detected as a single white line including a plurality of white lines including a broken line. Whether it is a long broken line or a short broken line can be determined.

Block diagram showing schematic configuration of in-vehicle white line recognition device Explanatory drawing showing the position of the in-vehicle camera in the vehicle Functional block diagram showing the schematic functional configuration of the image processing unit Diagram showing an example template The figure explaining the method of the pitch judgment of a broken line by change of the number of votes Figure showing an example of confidence distribution Diagram showing an example of the integrated confidence distribution The flowchart explaining the process sequence of the process which selects the white line of a control object Schematic diagram showing the road The figure explaining the method of the pitch judgment of the broken line by the change of the thickness of a white line The figure explaining the method of the pitch judgment of a broken line by change of a pattern matching result

Embodiments of the present invention will be described below with reference to the drawings.
[Example]
1. Configuration of In-vehicle White Line Recognition Device 1 As shown in FIG. 1, the in-vehicle white line recognition device 1 includes an in-vehicle camera 10 that acquires an image around the vehicle and an image around the vehicle acquired by the in-vehicle camera 10 (hereinafter, this image is simply taken). And an image processing device 20 that performs image processing. FIG. 2 is an explanatory diagram showing the position of the in-vehicle camera in the vehicle.

  The white line recognized by the in-vehicle white line recognition device 1 is used for warning control of the vehicle by a lane departure warning device (not shown). This alarm control calculates the distance of the white line from the vehicle on the left and right, and if either is below the threshold, that is, if the vehicle is closer to the white line than the distance set by the threshold, the driver is alerted. It is control to give.

  The in-vehicle white line recognition device 1 extracts a plurality of white line-like objects (white line candidates) from the captured image, selects a white line to be controlled from among them, and outputs information on the selected white line to the alarm device. .

  As shown in FIG. 3, the image processing apparatus 20 includes a plurality of DSPs (Digital Signal Processors) 30 to 80, a BUS 120 for data transmission between the DSPs 30 to 80, and ROM, RAM, and I / O (not shown). Yes.

  The DSP 30 is a DSP that performs white line candidate extraction for extracting a white line candidate on a road from a captured image acquired by the in-vehicle camera 10 by image processing. Hereinafter, the DSP 30 is referred to as a white line candidate extraction unit 30. Specifically, white line candidates are extracted from the image acquired by the in-vehicle camera 10 by the number of votes (solid line / broken line determination result) in the edge detection process and the straight line extraction Hough conversion process, which are known image processes.

  The DSP 40 is a DSP that calculates the certainty of white line likelihood for the white line candidates extracted by the white line candidate extraction unit 30. Hereinafter, the DSP 40 is referred to as a white line feature calculation unit 40. The certainty factor is a numerical value set within a range of 0.01 to 1, and is set by the white line feature calculation unit 40 for each result of each processing (a) to (e) described later by the DSP 50. . Hereinafter, the DSP 50 is referred to as a feature extraction unit 50.

  (A) The certainty factor is calculated based on the ratio of the contrast of the white line candidate to the road surface around the white line candidate (excluding the white line candidate part). Specifically, when the contrast ratio of the white line candidate is lower than a predetermined value (for example, 150%), the certainty factor is lowered.

  (B) From the photographed image acquired by the in-vehicle camera 10, a feature including the area on the road between the white line candidates extracted by the white line candidate extraction unit 30 and the pattern or average luminance of the white line candidates themselves is extracted. For example, the light that is inserted through the gap of the guardrail due to direct sunlight from the lateral direction may appear straight on the road surface and look like a white line, or the area surrounded by the repair mark by coal tar may appear linear. Then, the pattern of the area and the white line is extracted by pattern matching or a luminance difference.

  In order to distinguish a white line from an area surrounded by light or repair traces inserted on the road, the pattern and average luminance of the area and the white line candidate may be compared. Therefore, if the pattern difference between the extracted area and the white line candidate is large, the certainty level is increased as they are different, and if the average luminance difference is small, the two line pattern candidates are determined to match.

  (C) Extract a region on the road between white line candidates extracted by the white line candidate extraction unit 30 and the total amount of edges of the white line candidates themselves from the photographed image. The total amount of edges is obtained by extracting edges from a binarized image obtained by binarizing a captured image and summing the number of extracted edges.

  Even if the brightness difference with the road surface is low due to fading of the white line, it looks like a white line to the human eye. This is due to the human sensitivity that spots appear to be emphasized when there is no pattern on the road.

Therefore, when the difference between the total amount of edges of the area surrounded by the white line candidate and the white line is equal to or smaller than a predetermined value, the certainty of the white line candidate is lowered.
(D) A streak pattern in which the edge amount in the vehicle traveling direction on the road is greater than or equal to a predetermined value in a region in the vicinity of the white line candidate extracted by the white line candidate extracting unit 30 from the captured image in a direction substantially perpendicular to the vehicle traveling direction. Is detected.

  In order to prevent the vehicle from slipping, the vertical stripes carved on the road have lower luminance (darker) than the road surface, but the luminance at the edge of the vertical stripe may not be different from the edge of the white line. Therefore, the edge amount is calculated from the binarized image of the image acquired by the in-vehicle camera 10, and the white line candidate in the region in the vicinity of the vehicle traveling direction has an edge amount in the vehicle traveling direction on the road that exceeds the predetermined value. Detect as a pattern. When a streak pattern is detected on the white line candidate, the certainty factor of the white line candidate is lowered.

  (E) A traffic sign drawn on the road is detected from the image acquired by the in-vehicle camera 10 by pattern matching. As traffic signs to be detected, there are pedestrian crossings and the like in addition to traffic signs drawn on the road surface such as arrows, speed limits, and stop characters. When a traffic sign or the like is detected at the position of the white line candidate, the certainty of the white line candidate is lowered.

The feature extraction unit 50 performs the processes (a) to (e), and the white line feature calculation unit 40 calculates a plurality of certainty factors for one white line candidate based on the result.
The DSP 60 is a DSP that outputs a likelihood indicating white line likelihood by multiplying and integrating the certainty calculated and output in each of the above (a) to (e), and hereinafter, this DSP 60 is integrated with the white line feature integration. This is referred to as part 60.

  The integration processing in the white line feature integration unit 60 is determined from integration by multiplication of certainty factors based on a Bayesian estimation framework. For example, when the certainty factors A and B of the white line likelihood calculated by two of the processes shown in (a) to (e) of the feature extraction unit 50 are 0.6 and 0.7, respectively, they are integrated. The first likelihood is (0.6 * 0.7) / (0.6 * 0.7 + 0.4 * 0.3) = 0.78.

Here, 0.4 and 0.3 are not likely white lines due to A and B (1-0.6, 1-0.7), respectively.
A white line candidate whose likelihood output by the white line feature integration unit 60 is equal to or greater than a predetermined threshold is used as a white line existing on the road surface in the following processing. The white line candidate extraction unit 30, the white line feature calculation unit 40, the feature extraction unit 50, and the white line feature integration unit 60 correspond to white line extraction means in the present invention.

  The DSP 70 is a DSP that selects the type of white line. Hereinafter, the DSP 70 is referred to as a composite line determination unit 70. The composite line determination unit 70 corresponds to the composite line determination unit and the line type determination unit in the present invention.

  The composite line determination unit 70 performs pattern matching using a plurality of templates on each of white line candidates (hereinafter sometimes simply referred to as white lines) whose likelihood output by the white line feature integration unit 60 is equal to or greater than a predetermined threshold. Further, the pitch of the broken line is determined according to the number of votes of the Hough transform by the white line candidate extracting unit 30, and the certainty distribution is output based on the result, and the line type of the white line is determined.

  Note that the line type referred to here is whether the white line is a single line or a composite line, the type of the composite line (triple line, double line), the position in the composite line, whether the white line is a broken line, And types of broken lines (long broken lines, short broken lines) and the like. The compound line is composed of a plurality of white lines existing within a predetermined distance. Details will be described below.

<Determination of pattern matching and broken line pitch>
(1) Pattern matching In pattern matching, matching is performed using four templates prepared in advance: triple lines, double lines, two broken triple lines, and 2.5 broken triple lines.

  A triple line template is a template having a composite line pattern composed of three white lines (see FIG. 4A), and a double line template is a composite line composed of two white lines. A template having a pattern (see FIG. 4B).

  In addition, a template with two broken triple lines is originally a composite line composed of three white lines, but the contrast between the white lines and the road surface is between the first and second lines or between the second and third lines. This is a template when it becomes smaller and the portion is not recognized as a road surface, and appears as two thick white lines and one normal white line.

  A broken triple line 2.5 template is originally a composite line consisting of three white lines, but one part of the boundary between the white line and the road surface collapses, and there are five points with high contrast. A template for the case. These templates are only examples, and templates other than those described above may be used.

Each time pattern matching is performed using a single template for a certain white line, a parameter for determining what kind of line the white line is is output.
A specific pattern matching example will be described with reference to FIGS. In addition, in order to make description easy, it demonstrates using the figure which looked at the road from the upper direction.

  FIG. 4A shows a triple line template which is an example of a template used for pattern matching. This template consists of three white lines, line A, line B, and line C. The numerical value in the figure is the length (cm), and the lines A and C are wider than the line B (in the direction crossing the vehicle traveling direction, the left-right direction in FIG. 4A).

As a specific process, first, one white line is selected from white lines extracted from the photographed image (white line candidates whose likelihood is equal to or greater than a predetermined threshold).
The selected white line is matched with each of the lines A, B, and C in FIG. For example, when a line A is matched to a certain white line (A), the white line (A) matches A, and there are white lines that match two lines B and C in addition to the white line (A). In some cases, it is determined that A of the triple lines of the template matches the white line (A). Even if the white line (A) matches A, if there is no white line matching the lines B and C, it is determined that A does not match the white line (A).

  In this way, matching with the white line (A) is performed for each of the three A, B, and C to determine what matches, but if none of the three matches, the template is It is determined that the white line (a) did not match.

FIG. 4B shows an example of a double line template. The matching method is the same as described above.
Each pattern matching result is prepared in advance as a combination of parameters indicating the probability of whether the white line is a triple line, double line, or single line, and if it is a composite line, the position Is selected from For example, when the triple line matching in FIG. 4 (A) is performed, there are four results that match A, match B, match C, and do not match any of them. The four combinations of parameters are prepared, and are selected and output according to the result of pattern matching.

(2) Determination of Pitch of Broken Line The pitch of the broken line indicates the lengths of the paint portion where the line exists and the gap portion which is the interval between the lines. The pitch of the broken line is determined based on the elapsed time of the number of votes obtained by the Hough transform. The principle will be described with reference to FIGS.

  FIGS. 5A and 5B are the same road, and the timings for acquiring images are different. An area 130 surrounded by a broken line in the above figure is an area for counting the number of votes obtained by Hough conversion. The white line A is a long broken line, the white line B is a solid line, and the white line C is a short broken line.

  In the area 130 of FIG. 5A, the number of votes obtained by Hough conversion is the largest for the white line B, which is a solid line, followed by the white line A being the largest, and the white line C being the smallest. This is because the number of votes increases according to the length of the white line existing in the region 130 (the portion indicated by the thick line in the figure). In FIG. 5B after a certain time has elapsed, the number of votes for the white line A is greatly reduced. On the other hand, the white line B, which is a solid line, does not change, and the change of the white line C is also small.

  These time changes are shown in FIG. A, B, and C in the figure correspond to the white lines A, B, and C in FIGS. 5A and 5B, respectively. The number of votes for the white line B does not change greatly over time. On the other hand, although the number of votes for the white line C does not vary greatly with time, the number of votes for the white line C is smaller than that for the white line B. This is because, since the white line C is a broken line, the number of votes is reduced as compared with the white line B, but the amount of the paint part of the white line C existing in the region 130 is small because the broken line is short.

  The number of votes for the white line A varies greatly with time. This is because, as can be seen from a comparison between FIG. 5A and FIG. 5B, the paint portion of the white line A existing in the region 130 changes greatly with time. Specifically, the number of votes obtained is large in FIG. 5A, and the number of votes obtained is small in FIG. Thus, when the broken line is long, the time change of the number of votes obtained becomes large.

As can be seen from FIG. 5C, the white lines A and C both have fewer votes than the white line B per predetermined time. From the number of votes obtained per predetermined time, it can be seen that the white line is a broken line.
Thus, what can be estimated that the length of a broken line is more than a predetermined threshold value is determined as a long broken line (long broken line in the present invention), and those shorter than the predetermined threshold are short broken lines (short broken line in the present invention). Is determined.

  When the number of votes obtained per predetermined time is small, it can be determined that the ratio of the gap portion in the broken line is large. Therefore, the length and interval of the broken line can be estimated from the number of votes obtained per predetermined time and the state of time change. The method of changing the number of votes is determined by the relationship between the size of the area 130 and the length of the broken line. Basically, the shorter the broken line, and the shorter the interval between the broken lines, the smaller the change in time, the longer the broken line, and the broken line. The longer the interval, the greater the time change.

<Output of confidence distribution>
Depending on the pattern matching result and the determination result of the broken line pitch, the certainty distribution as shown in FIGS. 6A and 6B is output. This certainty distribution is a collection of parameters indicating the certainty that a certain white line is the line type for various line types.

  FIG. 6A shows a case where pattern matching succeeds with the center line in a template in which a certain white line is a triple line, and the white line is a broken line and the length of the broken line is long. FIG. 6B shows the certainty distribution in the case where pattern matching is not successful with respect to the triple line pattern.

  In the figure, “triple, double, single” indicates how many composite lines or a single line. “Break, actual” indicates a solid line or a broken line. Further, “long, short” indicates the length of the broken line when it is a broken line.

  In addition, “inside and outside” indicate an inner side that is closer to the vehicle position and an outer side that is farther from the vehicle position. That is, using FIG. 4, when performing pattern matching, if the white line A matches the white line on the left side from the vehicle, the white line is outside, and if the white line C matches, the white line is inside. It becomes. The white line on the right side of the vehicle is on the inside if it matches the white line A, and on the outside if it matches the white line C.

  In the certainty distribution shown in FIG. 6A, the parameter of “triple break / center / long” is the highest. And those having similar characteristics also have higher parameters. For example, even if it is determined that the white line is a long broken line, it may be influenced by faint or dirty paint, so it is “triple break / center / short” or “triple / center”. The possibility is high and the parameters are high. Also, the parameter is lower as it is considered that the line types are not similar.

  An example of criteria for determining whether or not similar features are present will be given. When it is determined that the line is a triple line, the double line is more similar to the single line, and the double line parameter becomes higher. If it is determined that the line is outside the triple line, the parameter outside the double line also increases.

On the other hand, the certainty distribution when the pattern matching shown in FIG. 6B is not successful is low in all parameters.
The certainty factor distribution has parameters determined in advance according to the pattern matching result and the determination result of the broken line pitch. That is, one certainty distribution created in advance for the combination of the pattern matching result and the determination result of the broken line pitch is output.

<Create integrated confidence distribution>
Four types of pattern matching and broken line pitch determination are performed for one white line, and four confidence distributions corresponding to each result are output. Then, the output parameters of each certainty factor distribution are integrated to create an integrated certainty factor distribution as shown in FIG.

Similarly, for the other white lines extracted from the captured image, four confidence distributions are output, and their parameters are integrated to create an integrated confidence distribution.
<Correction of integrated certainty distribution>
The integrated certainty distribution created as described above is corrected by the following two points.

(1) Correction by past information When there is a blur on the white line of the road, or when the gap is a broken line and the gap is large, the white line may not be extracted at the position where the white line exists. If the white line that was not extracted is a white line that should be controlled, or if the white line that should be controlled changes depending on the presence or absence of the white line, the white line set as the control target is changed at short intervals. Therefore, it becomes difficult to perform stable control, which is not preferable.

  Therefore, white lines that have been extracted in the past (white line candidates with a likelihood greater than or equal to a predetermined threshold) are of the same type as those previously extracted until a predetermined time or until the vehicle has traveled a predetermined distance. Processing is performed assuming that white lines continue to exist.

  In the present embodiment, it is assumed that there is a white line until one second has passed after the white line is not extracted. Alternatively, it may be handled that the white line continues to exist until a predetermined distance (for example, 5 m) elapses after the white line is not extracted.

  In addition, since the pattern matching result may change, the following may be performed. Based on the statistical certainty distribution created last time (or the finally determined line type), the parameter is raised or lowered only for a predetermined line type in the newly created statistical certainty distribution. For example, for line types that had a parameter greater than or equal to a predetermined threshold in the previously created statistical certainty distribution, increase the parameters in the newly created statistical certainty distribution, and decrease the parameters for other line types Can be considered.

Thereby, it is suppressed that the white line set as a control object changes at a short interval.
(2) Correction based on map information by navigation system When map information stored in a navigation system (not shown) provided in a vehicle includes information on a lane (white line) in which the vehicle is traveling, integration is performed using that information. Adjust parameters of confidence distribution.

Specifically, for example, when there is information that there is a double solid line and a single solid line in the traveling lane, parameters other than the double solid line and the single solid line are greatly reduced.
The parameters of the integrated certainty distribution can be corrected by (1) and (2) described above.

Returning to FIG.
The DSP 80 is a DSP that selects a white line to be controlled from white lines extracted based on the photographed image. Hereinafter, the DSP 80 is referred to as a control target white line selection unit 80. This control target white line selection unit 80 corresponds to the control line selection means in the present invention.

  For the white line whose likelihood output by the white line feature integration unit 60 is equal to or greater than a predetermined threshold, the composite line determination unit 70 outputs an integrated certainty distribution that is a parameter group for determining the line type. The control target white line selection unit 80 selects a control target white line using the integrated certainty distribution.

The type of the white line is a line type having a maximum probability parameter of the line type (for example, a triple line center solid line in the case of the integrated confidence distribution in FIG. 7).
In the present embodiment, the white line closest to the vehicle is selected as the white line to be controlled except for the short broken line constituting the composite line. A specific processing procedure will be described with reference to the flowchart shown in FIG. Moreover, the schematic diagram which shows a road is shown in FIG. This process is performed separately in the left-right direction around the vehicle.

  First, a white line closest to the vehicle center is extracted from white lines whose likelihood output by the white line feature integration unit 60 is equal to or greater than a predetermined threshold (S1). In FIG. 9, when the vehicle is traveling on the arrow, if a process is performed on the left side, a C white line is extracted.

  Next, it is determined whether the white line extracted in S1 is a single line or a solid line (S2). If it is a single line or a solid line (S2: YES), the process proceeds to S6. On the other hand, if it is not a single line or a solid line (S2: NO), that is, if it is a composite line and a broken line, it is determined whether or not the broken line is a long broken line (S3).

  If it is a long broken line (S3: YES), the process proceeds to S6. If it is not a long broken line (S3: NO), it is determined whether there is a white line further outside the white line extracted in S1, that is, at a position far from the vehicle center (S4). If there is no white line on the outside (S4: NO), the process proceeds to S6. In the case of FIG. 9, since the white lines A and B exist outside the white line C, S4 is YES.

  On the other hand, if there is a white line further outside in S4 (S4: YES), a white line closest to the center of the vehicle next to the white line extracted in S1 is extracted (S5). In the case of FIG. 9, the white line B is extracted next to the white line C. Thereafter, the process returns to S2. When the process returns from S5 to S2, the process such as S2 is executed using the white line extracted in S5.

In S6, the white line extracted in S1 or S5 is set as a control target. After this S6, this process ends.
Thereafter, similar processing is performed on the left and right sides of the vehicle in the captured image. In the case of FIG. 9, the white line D, which is a single line, is set as the control target line.

Information on the white line as the control target set in this way is output to the lane departure warning device described above, and a lane departure warning is issued as necessary.
2. Effect In the in-vehicle white line recognition device 1 of the present embodiment, in the composite line in which the solid line and the short dashed auxiliary line are drawn, the short broken line is prevented from being selected as the control target, and the solid line and the long dashed line are combined. The line can be selected as a control object without ignoring the long dashed line. As a result, for example, it becomes difficult to cause a problem that determination of lane departure is made at an incorrect timing.

  Further, in the on-vehicle white line recognition device 1 of the present embodiment, the type of composite line is determined by integrating the certainty distribution obtained by combining the result of pattern matching using a plurality of templates and the determination result of the pitch of the broken line. Therefore, the white line to be controlled can be selected with high accuracy.

[Modification]
As mentioned above, although the Example of this invention was described, it cannot be overemphasized that this invention can take a various form, as long as it belongs to the technical scope of this invention, without being limited to the said Example at all.

  For example, in the above-described embodiment, the configuration in which a white line other than a line that is a composite line and a short broken line among white lines extracted from a captured image is selected as a control target has been exemplified. May be.

  For example, a plurality of judgment conditions for determining a white line to be controlled are set, a parameter indicating the priority as a control target is output for each judgment condition depending on whether the condition is satisfied, and these parameters are integrated. You may decide.

  Specifically, as described above, it is conceivable to set a high parameter to be output as the priority of the control target for white lines other than a composite line and a short broken line. In addition, the higher the white line candidate likelihood is, the higher the parameter is output. In the case of a triple composite line, the central white line outputs a higher parameter, the solid line outputs a higher parameter than the broken line, etc. Can be mentioned. Then, it is conceivable to set a parameter having the largest parameter as a control target as a result of integrating these parameters.

  Further, for example, when each of the white lines extracted on the most vehicle center side on the left and right sides of the vehicle is determined to be a white line that constitutes a composite line and is determined to be a short broken line, You may comprise so that the white line except the said white line may be selected as a white line used as the object of the said control preferentially rather than the white line determined as the said short broken line from the compound line which the determined white line comprises.

  In the in-vehicle white line recognition device configured as described above, when a short broken line of a composite line exists on the left and right center sides of the vehicle, the white line excluding the short broken line is preferentially selected as a control target. When a short broken line is detected on the left and right of the vehicle, the short broken line is likely to be an auxiliary line. The white line to be selected can be appropriately selected.

Moreover, in the said Example, although the structure which judges the pitch of a broken line based on the number of votes of Hough conversion was illustrated, you may determine the length of a broken line by pattern matching.
Specifically, it is conceivable to determine the pitch of the broken line by pattern matching using a template that defines the width and pitch of the white line.

  In addition, since a white line with a large width may be a short broken line, the width of the white line can be reduced by using pattern matching using a template with a thick white line or the interval between edge points (straight lines after Hough transform). If the width is within a predetermined range, it may be determined as a short broken line.

  In addition, as shown in FIG. 10A, when the solid line 160 and the broken line 162 are close and parallel to each other, it may be detected as one thick white line 164. At this time, the thickness of the detected white line changes periodically according to the position where the broken line exists (see FIG. 10B). Therefore, when the thickness change of the white line is measured and the thickness of the white line fluctuates between a relatively thick state and a thin state at a predetermined cycle, the white line is composed of a broken line and a solid line. It is possible to divide as two white lines.

  The period during which the white line is continuously measured as being thick is longer as the broken line is longer. Therefore, when the above-described period is greater than a predetermined threshold, the divided broken line may be determined to be a long broken line.

  Further, as shown in FIG. 5B, when the white line A hardly or completely does not exist in the region 130 (the gap portion exists), the adjacent white line B and white line C are determined to be double lines as a result of pattern matching. There is a case. In that case, as shown in FIG. 5A, the white line A is determined to be a triple line in the paint portion.

  Therefore, when the number of composite lines determined for the white line B and the white line C (single line, double line, triple line) periodically varies, the length of the broken line of the white line A based on the variation period May be determined. Specifically, as shown in FIG. 11, a period during which pattern matching for the white line B is successful with a double-line template (flag N in the figure) and a period when pattern matching with a triple-line template is successful (flag Y in the figure). ), And the length of the paint part and the gap part can be determined by the period and the ratio of the double line and the triple line, and the white line A is a long broken line Whether it is a short broken line or not can be determined.

  Similarly, for the double line composed of the white line A and the white line B, the white line B is determined to be a single line (matching is not successful for any template of the double line and the triple line), and the double line It can be determined in the same manner from the period and ratio of the period of success in the template.

  The determination of the composite line is not limited to the one described in the embodiment, and can be determined by various methods. For example, among the white lines extracted from the photographed image, a combination of white lines whose distances are within 45 cm may be detected, and these may be determined as composite lines. If it is determined that a composite line exists, it is determined whether or not it is a broken line by a method using the number of votes obtained by the Hough transform, pattern matching, or the like, and a control target is determined by the same method as in FIG. Good.

DESCRIPTION OF SYMBOLS 1 ... In-vehicle white line recognition apparatus, 10 ... In-vehicle camera, 20 ... Image processing apparatus, 30 ... White line candidate extraction part, 40 ... White line feature calculation part, 50 ... Feature extraction part, 60 ... White line feature integration part, 70 ... Composite line determination , 80 ... white line selection unit to be controlled, 130 ... area, 160 ... solid line, 162 ... broken line, 164 ... white line

Claims (4)

An in-vehicle camera that acquires images around the vehicle,
White line extraction means for extracting a white line by image processing from an image acquired by the in-vehicle camera;
Composite line determination means for determining whether the white line extracted by the white line extraction means is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance; and
The white line extracted by the white line extraction means is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more and a short broken line whose length is less than the predetermined threshold. A line type determination means to perform,
Control line selection means for selecting a white line to be subjected to predetermined control from the white lines extracted by the white line extraction means,
The control line selection unit determines the white line determined as the long broken line by the line type determining unit as the short broken line for the white line determined by the composite line determining unit to constitute the composite line. The on-vehicle white line recognition device, wherein the white line to be controlled is selected with priority over the white line. An in-vehicle camera that acquires images around the vehicle,
A white line extracting means for detecting an edge point from an image acquired by the in-vehicle camera and extracting a white line based on a straight line obtained by Hough transforming the edge point;
Composite line determination means for determining whether the white line extracted by the white line extraction means is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance; and
The white line extracted by the white line extraction means is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more and a short broken line whose length is less than the predetermined threshold. A line type determination means to perform,
Control line selection means for selecting a white line to be subjected to predetermined control from the white lines extracted by the white line extraction means,
The control line selection means, for the white line determined to constitute the composite line by the composite line determination means, the white line determined by the line type determination means as the solid line or the long broken line as the short broken line Select as the white line to be controlled in preference to the determined white line,
The line type determining means determines that the white line is the long broken line when the time change in the number of votes of the Hough transform corresponding to the white line extracted by the white line extracting means is larger than a predetermined threshold value. Car white line recognition device. An in-vehicle camera that acquires images around the vehicle,
White line extraction means for extracting a white line by image processing from an image acquired by the in-vehicle camera;
Composite line determination means for determining whether the white line extracted by the white line extraction means is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance; and
The white line extracted by the white line extraction means is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more and a short broken line whose length is less than the predetermined threshold. A line type determination means to perform,
Control line selection means for selecting a white line to be subjected to predetermined control from the white lines extracted by the white line extraction means,
The control line selecting means determines that each of the white lines extracted most on the vehicle center side on the left and right sides of the vehicle is a white line constituting a composite line by the composite line determining means, and the line type determining means If it is determined to be a short broken line, the long broken line constituting the composite line is prioritized over the white line determined as the short broken line among the composite lines formed by the white line determined as the short broken line. The on-vehicle white line recognition device is characterized in that it is selected as a white line to be controlled. An in-vehicle camera that acquires images around the vehicle,
White line extraction means for extracting a white line by image processing from an image acquired by the in-vehicle camera;
Composite line determination means for determining whether the white line extracted by the white line extraction means is a white line constituting a composite line composed of a plurality of white lines existing within a predetermined distance; and
The white line extracted by the white line extraction means is determined as at least one of a solid line, a long broken line whose length is a predetermined threshold or more and a short broken line whose length is less than the predetermined threshold. A line type determination means to perform,
Control line selection means for selecting a white line to be subjected to predetermined control from the white lines extracted by the white line extraction means;
When the time change of the thickness of the white line extracted by the white line extraction means is measured, and the thickness of the white line varies between a relatively thick state and a thin state at a predetermined cycle, the extracted white line Dividing means for dividing the white line into two white lines consisting of a broken line and a solid line,
The line type determination means determines that the divided broken line is the long broken line when a period of time when the thickness of the white line is measured as the thick state is greater than a predetermined threshold value.
The control line selection means, for the white line determined to constitute the composite line by the composite line determination means, the white line determined by the line type determination means as the solid line or the long broken line as the short broken line The on-vehicle white line recognition device, wherein the white line to be controlled is selected with priority over the determined white line.