JP2014021525A - Travel division line detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a travel division line appropriately and precisely.SOLUTION: A travel division line detection device 10 comprises: a first edge detecting unit 22 detecting an edge of a travel division line from a road surface image captured by an in-vehicle camera 11; a detection area setting unit 26 setting a detection area on the basis of a last detection result of the travel division line; a second edge detecting unit 23 scanning plural directions including at least slant directions in an image plane of the road surface image in the detection area set by the detection area setting unit 26, and detecting an inside edge of a travel area on the basis of the edge detected by the first edge detecting unit 22; and a travel division line estimating unit 25 estimating the travel division line adapting to the travel area from the edge detected by the second edge detecting unit 23.

Description

  The present invention relates to a travel lane marking detection device.

Conventionally, for example, for each of a plurality of search lines extending in the horizontal direction in a search area set on a captured image, a change in luminance is detected from the inner side to the outer side in the vehicle width direction, and the luminance changes more than a predetermined value from dark to bright. A lane position detection device that detects an initial edge point is known (see, for example, Patent Document 1). This lane position detection apparatus improves the detection accuracy of the lane start point by performing noise removal using a Hough transform or the like on the detected point point group.
Conventionally, for example, for each of a plurality of search lines extending in the horizontal direction in each of the left and right white line detection areas set on the image, a change in luminance is detected from the inner side to the outer side in the vehicle width direction, and the luminance changes from dark to bright. 2. Description of the Related Art A vehicle white line recognition device that detects edge points that change more than a predetermined distance as white line candidate points is known (see, for example, Patent Document 2). The vehicular white line recognition device calculates a white line approximate line using a quadratic least square method based on the detected point groups of left and right white line candidate points.
Conventionally, for example, feature point data is extracted from at least two areas including a traveling lane detected in the previous process using images obtained by continuously capturing images of a road surface, for example. A road surface traveling lane detection device that detects a traveling lane in the current process based on the most suitable straight line is known (see, for example, Patent Document 3).

Japanese Patent No. 4638369 JP 2012-22574 A Japanese Patent No. 4128837

By the way, according to the lane position detecting device and the vehicle white line recognizing device according to the above-described prior art, since the scanning is performed only in the horizontal direction on the image, for example, an edge can be accurately detected in a curve having a large curvature. The problem of being unable to occur In addition, when noise removal using the Hough transform is performed, there is a problem that it is difficult to properly detect an edge, for example, a curve with a large curvature due to the straight line detection by the Hough transform. Arise.
Further, according to the road surface traveling lane detection device according to the above-described conventional technology, the region for extracting the feature point data is only set based on the traveling lane detected on the image plane in the previous processing, Since the three-dimensional information of the lane (for example, the relative inclination of the road surface and the vehicle pitch fluctuation, etc.) is not taken into account, the feature point data cannot be properly extracted and the detection accuracy of the traveling lane is improved. May decrease.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a travel lane marking detection device capable of detecting a travel lane marking appropriately and accurately.

  In order to solve the above-described problems and achieve the object, the traveling lane marking detection apparatus according to the first invention of the present invention is imaged by imaging means for imaging the road surface (for example, the in-vehicle camera 11 in the embodiment). A travel lane marking detection device that detects a travel lane marking of a moving object from a road image obtained by the first edge detection means that detects an edge of the travel lane marking from the road image (for example, a first edge detection device according to an embodiment). 1 edge detection unit 22), detection region setting means (for example, detection region setting unit 26 in the embodiment) for setting a detection region based on the previous detection result of the travel lane marking, and the detection region setting unit Based on the edges detected by the first edge detecting means, scanning in a plurality of directions including at least an oblique direction in the image plane of the road surface image in the detection region set by The second edge detection means (for example, the second edge detection unit 23 in the embodiment) for detecting the edge inside the travel area, and the edge detected by the second edge detection means is adapted to the travel area. Traveling lane marking estimating means for estimating the traveling lane marking (for example, the traveling lane marking estimating unit 25 in the embodiment).

  Furthermore, in the travel lane marking detection device according to the second aspect of the present invention, the second edge detection means is an edge detection means in the area for detecting the edge in the detection area (for example, the area in the embodiment). Edge positive / negative identification means (for example, edge positive / negative identification in the embodiment) for identifying the rising edge as a positive edge and the falling edge as a negative edge among the edges detected by the inner edge detection section 31) and the inner edge detection means Part 32) and the two traveling division lines on the front side and the rear side in the scanning direction, based on the identification result of the edge positive / negative identification means, the positive edge in the traveling division line on the front side Traveling area inner edge detection means for detecting and detecting the negative edge in the rearward travel line (for example, inner edge detection unit 33 in the embodiment) And, equipped with a.

  Furthermore, in the travel lane marking detection device according to the third aspect of the present invention, the edge positive / negative identification means comprises at least a vertical direction and a horizontal direction on the image plane and two diagonal directions opposite to each other in the detection region. A plurality of directions including four directions are scanned to identify the positive edge and the negative edge in the detection area.

  Furthermore, in the travel lane marking detection apparatus according to the fourth aspect of the present invention, the imaging means can acquire three-dimensional information, and the travel lane marking estimation means is detected by the second edge detection means. A robust estimation is performed on the edge in a two-dimensional plane to estimate the travel lane marking.

  Furthermore, in the travel lane marking detection device according to the fifth aspect of the present invention, the travel lane marking estimation means is nonlinear in the first two-dimensional plane with respect to the edge detected by the second edge detection means. The running lane marking is estimated by executing robust estimation including a least square method using a function and executing a least square method using a linear function in the second two-dimensional plane.

  Furthermore, in the travel lane marking detection apparatus according to the sixth aspect of the present invention, the imaging means can acquire three-dimensional information, and the travel lane marking estimation means is detected by the second edge detection means. The running lane marking is estimated by performing robust estimation on the edge in a three-dimensional space.

  Further, in the travel lane marking detection device according to the seventh aspect of the present invention, the detection area setting means displays the travel lane line in the three-dimensional space estimated by the travel lane marking estimation means last time in a two-dimensional plane. The detection area is set using a conversion result obtained by reverse projection conversion.

  Furthermore, in the travel lane marking detection device according to the eighth aspect of the present invention, the travel lane marking estimation means includes all of the travel lane line estimation candidates and the second edge detection means detected in the robust estimation. When the distance to the edge is used, the normal distance between the travel lane marking estimation candidate and the edge is defined as the distance.

  Furthermore, in the travel lane marking detection device according to the ninth aspect of the present invention, the detection area setting means executes a first area setting mode when the previous detection result of the travel lane marking does not exist, and the travel The second region setting mode is executed when the previous detection result of the lane marking exists, and the third region setting is performed when the previous detection result of only one of the two traveling lane markings suitable for the traveling region exists. The mode is executed, and the fourth region setting mode is executed when there is a previous detection result of the two travel lane markings that match the travel region.

  Furthermore, in the travel lane marking detection device according to the tenth aspect of the present invention, the detection area setting means assumes that the previous detection result of the travel lane line is a straight line in the first area setting mode. The detection area is set, and in the second area setting mode, the detection area is set based on the previous detection result of the travel line, and in the third area setting mode, the two travel line lines are set. The first region set based on the one and the second region obtained on the assumption that the other of the two travel division lines is a straight line, the first region is given priority over the second region. The detection area is set based on the area obtained by combining, and in the fourth area setting mode, the detection area is set based on the previous detection result of the two travel lane markings that match the travel area. .

  Furthermore, in the travel lane marking detection device according to the eleventh aspect of the present invention, the detection area setting means in the fourth area setting mode, the previous detection result of the two travel lane lines suitable for the travel area. When two regions set based on the above are combined, an overlapping region is excluded from the detection region.

  Further, in the traveling lane marking detection device according to the twelfth aspect of the present invention, the first edge detection means smoothes the road surface image and performs noise removal (for example, steps in the embodiment). S11), edge detection means for detecting the edge of the road surface image smoothed by the smoothing processing means (for example, step S12 in the embodiment), and the edge output by the edge detection means Thinning processing means for thinning (for example, step S13 in the embodiment) and hysteresis threshold processing means for removing noise from the edges thinned by the thinning processing means using two thresholds (for example, Step S14) in the embodiment.

  Further, in the traveling lane marking detection device according to the thirteenth aspect of the present invention, the imaging means uses the road surface image as a grayscale image.

According to the travel lane marking detection apparatus according to the first aspect of the present invention, by scanning a plurality of directions including at least an oblique direction on the image plane of the road surface image, the edge can be appropriately and accurately detected even in a curve having a large curvature, for example. It can be detected well.
Further, by detecting the edge inside the travel area among the edges of the travel lane marking, the detection result of the travel lane marking can be appropriately used for various controls (for example, control for preventing lane departure, etc.).
Furthermore, detection accuracy can be improved by detecting edges in two stages by the first edge detection means and the second edge detection means.

  According to the travel lane marking detection device according to the second aspect of the present invention, of the two travel lane lines in the detection region, the travel lane line scanned first (that is, the travel lane line on the rear side in the scanning direction). ) To detect a negative edge, and a positive edge is detected from a travel division line to be scanned later (that is, a travel division line on the front side in the scanning direction), thereby easily detecting an inner edge of the travel region. it can.

  According to the travel lane marking detection apparatus according to the third aspect of the present invention, the curvature is obtained by scanning at least four directions including the vertical direction and the horizontal direction and two diagonal directions opposite to each other (such as diagonal directions). Edges can be detected appropriately and accurately even in large curves.

  According to the travel lane marking detection apparatus according to the fourth aspect of the present invention, for example, by performing robust estimation by RANSAC (RANdom SAmple Consensus) or the like on a two-dimensional plane, an increase in computation load is prevented, and noise is increased. It is possible to appropriately and easily estimate the most probable traveling lane marking while removing.

  According to the travel lane marking detection apparatus according to the fifth aspect of the present invention, the least square method using a nonlinear function such as a quadratic curve is performed by using the robust estimation based on, for example, RANSAC (RANdom SAmple Consensus). By executing the least square method using a linear function in a second two-dimensional plane that is different from the first two-dimensional plane, it is possible to prevent the calculation load from increasing and It can be estimated well.

  According to the travel lane marking detection apparatus according to the sixth aspect of the present invention, the estimation accuracy is improved by estimating the travel lane marking in the three-dimensional space using, for example, robust estimation by RANSAC (RANdom SAmple Consensus) or the like. be able to.

  According to the travel lane marking detection apparatus according to the seventh aspect of the present invention, the detection area can be set appropriately, and the estimation accuracy of the travel lane marking can be improved.

  According to the travel lane marking detection apparatus according to the eighth aspect of the present invention, it is possible to improve the estimation accuracy of the travel lane marking.

  According to the travel lane marking detection device of the ninth aspect of the present invention, the detection area can be appropriately set according to the presence or absence of the previous detection result of the travel lane marking, and the estimation accuracy of the travel lane marking is improved. Can be made.

According to the traveling lane marking detection device of the tenth aspect of the present invention, in the first region setting mode, even if there is no previous detection result of the traveling lane marking, a larger detection region is obtained by linear approximation. By setting, the edge can be detected properly.
Further, in the second area setting mode, the detection area can be appropriately set based on the previous detection result of the travel lane marking.
Furthermore, in the third area setting mode, the edge can be appropriately detected by setting the detection area by using the previous detection result of the traveling lane marking and the linear approximation together.
Further, in the fourth area setting mode, the detection area can be appropriately set based on the previous detection result of the two travel lane markings that match the travel area.

  According to the traveling lane marking detection device of the eleventh aspect of the present invention, for example, an abnormal event such as erroneous detection of a traveling lane marking or an overlapping region related to a detection error is excluded from the detection region, and the edge is appropriately set. Can be detected.

  According to the travel lane marking detection device of the twelfth aspect of the present invention, the edge detection accuracy by the second edge detection means can be improved.

  According to the travel lane marking detection device of the thirteenth aspect of the present invention, an increase in calculation load can be suppressed.

1 is a configuration diagram of a travel lane marking detection device according to an embodiment of the present invention. It is a figure which shows the Sobel filter used by the 2nd edge detection part of the travel lane marking detection apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the positive edge point detected by the travel lane marking detection apparatus which concerns on embodiment of this invention, and a negative edge point. The figure which shows the quadratic curve which concerns on the least squares method performed by XY plane, and the linear line which concerns on the least squares method performed by XZ plane by the travel lane marking detection apparatus which concerns on embodiment of this invention. It is. It is a figure which shows the example of the distance of the edge point calculated by the traveling lane marking detection apparatus which concerns on embodiment of this invention, and the quadratic curve of the estimation candidate of a traveling lane line. It is a figure which shows the example of the detection area set by the traveling lane marking detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the travel lane marking detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the 1st edge detection process of the travel lane marking detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the detection area setting process of the traveling lane marking detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the 2nd edge detection process of the travel lane marking detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the travel lane marking estimation process of the travel lane marking detection apparatus which concerns on embodiment of this invention.

  Hereinafter, an embodiment of a travel lane marking detection device of the present invention will be described with reference to the accompanying drawings.

The travel lane marking detection device 10 according to the present embodiment is configured to include an in-vehicle camera 11 and a processing device 12, for example, as shown in FIG.
Furthermore, the processing device 12 includes, for example, an image acquisition unit 21, a first edge detection unit 22, a second edge detection unit 23, a three-dimensional information extraction unit 24, a travel lane marking estimation unit 25, and a detection area setting. Part 26.
Furthermore, the second edge detection unit 23 includes, for example, an in-region edge detection unit 31, an edge positive / negative identification unit 32, and an inner edge detection unit 33.

  The in-vehicle camera 11 is a stereo camera with a pair of cameras mounted on the vehicle at a predetermined interval in the horizontal direction, for example, and includes a road surface ahead of the traveling direction of the host vehicle obtained by imaging with the pair of cameras. A pair of images of the outside world (for example, a right image and a left image made up of pixels in a two-dimensional array) are output. Furthermore, based on a pair of images, it has so-called three-dimensional information composed of pixels of a two-dimensional array to which information on the distance to be imaged is given by a triangulation method based on the distance between a pair of cameras and parallax. Output image data.

  For example, based on the image data output from the in-vehicle camera 11, the processing device 12 detects a traveling lane marking on the road surface ahead of the traveling direction of the host vehicle.

  The image acquisition unit 21 acquires, for example, the image data output from the in-vehicle camera 11 and a pair of images, and generates a grayscale image based on one of the pair of images (for example, the right image).

  For example, the first edge detection unit 22 sequentially performs a smoothing process, an edge strength and gradient estimation process, a thinning process, a hysteresis threshold process on the grayscale image generated by the image acquisition unit 21. Execute.

  For example, in the smoothing process, the first edge detection unit 22 reduces noise by performing a smoothing process on the gray scale image generated by the image acquisition unit 21 using a Gaussian filter or the like.

For example, in the edge intensity and gradient estimation process, the first edge detection unit 22 estimates the edge intensity and gradient using an Sobel filter or the like on the image obtained by the smoothing process.
For example, in the thinning process, the first edge detection unit 22 reduces the thickness of the edge according to the estimation result of the edge strength and the gradient.

For example, in the hysteresis threshold process, the first edge detection unit 22 uses two different high-side threshold values and low-side threshold values, and determines that the edge point is an edge point when the edge intensity of the pixel to be determined is larger than the high-side threshold value. If the edge intensity of the pixel to be determined is smaller than the low threshold, it is determined that the pixel is not an edge point.
In addition, when the edge strength of the determination target pixel is not less than the low side threshold and not more than the high side threshold, if the edge point exists in the vicinity of the determination target pixel, the pixel is determined to be an edge point. If there is no edge point in the vicinity, it is determined that it is not an edge point.

The second edge detection unit 23 is, for example, at least oblique in the image plane of the grayscale image in the detection region (for example, the right traveling lane line region and the left traveling lane line region) set by the detection region setting unit 26 described later. A plurality of directions including directions are scanned.
Then, based on the edge points detected by the first edge detection unit 22, an inner edge of the traveling area (for example, a lane defined by the left and right traveling division lines) is detected.

  For example, among the edge points detected by the first edge detection unit 22, the in-region edge detection unit 31 is an edge existing in the right travel lane line region and the left travel lane line region set by the detection region setting unit 26. Estimate the edge strength and slope of a point.

  More specifically, the in-region edge detection unit 31 uses, for example, a differential filter such as a Sobel filter, and at least the vertical direction (90 ° direction) and the horizontal direction (0 ° direction) in the image plane of the grayscale image, and A plurality of directions including four directions including two diagonal directions opposite to each other (for example, a diagonal direction by a 45 ° direction and a 135 ° direction) are scanned.

  The in-region edge detection unit 31 is based on, for example, the four Sobel filters fV, fH, f45, and f135 shown in FIG. 2, and the edge strength ∇f and the gradient th (that is, the travel lane marking at the edge point) by the following formula (1) Slope).

  The edge positive / negative identifying unit 32 identifies, for example, the rising edge of luminance as a positive edge point and the falling edge of luminance as a negative edge point based on the gradient of the edge point at the edge point detected by the in-region edge detection unit 31.

For example, the inner edge detection unit 33 has two detection areas corresponding to the two traveling division lines on the front side and the rear side in the scanning direction (that is, the right traveling division line area and the left traveling line set by the detection area setting unit 26). The inner edge of the traveling area is detected with respect to the lane marking area.
For example, the inner edge detection unit 33 detects a positive edge point in the traveling division line on the front side in the scanning direction based on the identification result of the edge positive / negative identification unit 32, and in the traveling division line on the rear side in the scanning direction. Detect negative edge points.

  More specifically, for example, as shown in FIG. 3, the inner edge detection unit 33 is arranged on the left side which is the rear side of the scanning direction D with respect to the scanning direction D from the left side to the right side in the image plane of the grayscale image P. In the detection area corresponding to the travel lane marking LL (left travel lane marking area LA), the negative edge point NE is detected. On the other hand, the positive edge point PE is detected in the detection region (the right traveling segment line region RA) corresponding to the right traveling segment line RL on the front side in the scanning direction.

  For example, the three-dimensional information extraction unit 24 performs three-dimensional information on the edge point detected by the second edge detection unit 23 based on the image data having the three-dimensional information acquired by the image acquisition unit 21, that is, Extraction of horizontal (Y direction) and vertical (Z direction) position information corresponding to a two-dimensional array on the image plane of the gray scale image and distance information in the forward direction (X direction) of the host vehicle To do.

  The travel lane marking estimation unit 25 is, for example, for each of the two detection areas detected by the second edge detection unit 23 (that is, the right travel lane line area and the left travel lane line area set by the detection area setting unit 26). Based on the three-dimensional information extracted by the three-dimensional information extraction unit 24 for the edge point, the travel lane marking is estimated.

  For example, the travel lane marking estimation unit 25 executes robust estimation including a least square method using a nonlinear function in a first two-dimensional plane, and executes a least square method using a linear function in a second two-dimensional plane. Accordingly, based on the shape of the travel division line on the XY plane, which is the image plane of the gray scale image, and the relative inclination of the road surface with respect to the own vehicle on the ZX plane and the pitch variation of the own vehicle, etc. Estimate the line.

  More specifically, for example, the lane marking estimation unit 25 performs a least square method using a quadratic curve on the XY plane using robust estimation based on RANSAC (RANdom SAmple Consensus), and uses a linear function on the ZX plane. Perform multiplication.

For example, the travel lane marking estimation unit 25 first extracts a predetermined number of edge points randomly from among the edge points detected by the second edge detection unit 23 as a predetermined series of edge point calculation processing.
Next, based on the extracted three-dimensional information of the edge points, for example, as shown in FIG. 4 (A), the least square method using a quadratic curve is executed on the XY plane, and as shown in FIG. 4 (B), for example. Then, the least square method using a linear function is executed in the ZX plane.
Next, on the XY plane, the distance between the quadratic curve of the running segment line estimation candidate obtained by the least square method and all edge points detected by the second edge detector 23 (for example, the normal distance, etc.) ) And the number of edge points at which this distance is less than or equal to a predetermined threshold.

Then, this series of edge point calculation processing is repeatedly executed a predetermined number of times, and the quadratic curve of the running lane line estimation candidate that maximizes the calculated number of edge points is selected as the most probable estimation candidate.
Then, from all the edge points detected by the second edge detection unit 23, an edge point whose distance to the quadratic curve of the most probable estimation candidate (for example, a normal distance) is equal to or less than a predetermined threshold is extracted. Then, a least square method using a quadratic curve is executed on the extracted edge point on the XY plane.
Then, the quadratic curve based on the calculation result is used as the estimation result of the travel line.

Note that the travel lane marking estimation unit 25 may perform robust estimation using RANSAC or the like on the ZX plane as necessary in the predetermined series of edge point calculation processing described above.
For example, the travel lane marking estimation unit 25 selects an appropriate edge from the edge point detected by the second edge detection unit 23 based on the linear function of the travel lane line estimation candidate obtained by the least square method on the ZX plane. A point (for example, an edge point where the distance to the linear function of the running lane marking estimation candidate is a predetermined threshold value or less) may be extracted.

  Note that the travel lane marking estimation unit 25 uses, for example, a normal distance as the distance between the edge point detected by the second edge detection unit 23 and the quadratic curve of the travel lane line estimation candidate as shown in FIG. As shown in (A) and the following mathematical formulas (2) and (3), a radius centered on the position (α, β) of the edge point with the quadratic curve f (x, y) on the XY plane as a constraint condition A point (x, y) giving an extreme value of the function g (x, y) indicating the circle of r is calculated.

  In this case, the lane marking estimation unit 25 uses, for example, the Lagrange's undetermined multiplier method, the variable λ, and the function G (x, y) as shown in the following formula (4), the following formula (5) The extreme value conditions as shown in FIG. 5 are solved as simultaneous equations relating to (x, y), and the extreme values are calculated. Then, the radius r corresponding to the calculated extreme value is set as the normal distance.

The travel lane marking estimation unit 25 uses another distance instead of the normal distance as the distance between the edge point detected by the second edge detection unit 23 and the quadratic curve of the travel lane line estimation candidate. Also good.
For example, as illustrated in FIG. 5B, the travel lane marking estimation unit 25 uses a point on the quadratic curve f (x, y) of the travel lane line estimation candidate on the XY plane (instead of the normal distance). The minimum value of the distance L1 between x, y) and the position (α, β) of the edge point may be adopted.
In this case, the search area LA having the minimum value of the distance L1 is, for example, a point (α, y) on the quadratic curve f (x, y) having the same x coordinate or y coordinate as the position (α, β) of the edge point. y) and (x, β).

  For example, as shown in FIG. 5C, the travel lane marking estimation unit 25 is on the quadratic curve f (x, y) having the same x coordinate or y coordinate as the position (α, β) of the edge point. The quadratic curve f (x, y) at the intersection (Γ, Δ) of the normal line of the straight line La including the points (α, y) and (x, β) and the quadratic curve f (x, y) The distance L2 between the tangent line Lb and the straight line Lc parallel to the tangent line Lb and including the position (α, β) of the edge point may be used instead of the normal line distance.

  The detection area setting unit 26 detects, for example, an edge point detected by the second edge detection unit 23 based on the previous detection result of the travel lane marking (that is, the previous estimation result by the travel lane marking estimation unit 25). A region (for example, a right travel segment line region and a left travel segment line region corresponding to left and right travel segment lines that define a travel region such as a lane of a travel route) is set.

For example, the detection region setting unit 26 is configured such that the detection region where the edge point is detected by the second edge detection unit 23 is two-dimensional while the previous estimation result by the traveling lane marking estimation unit 25 is three-dimensional information. Therefore, reverse projection transformation is performed on the previous estimation result.
The detection area setting unit 26, for example, based on the calculation result of the least square method by the quadratic curve on the XY plane and the least square method by the linear function on the ZX plane, which is executed by the traveling segment line estimation unit 25, A detection region is set using a conversion result obtained by back-projecting a three-dimensional traveling division line, which is a previous estimation result by the line estimation unit 25, into a two-dimensional plane.

  Further, the detection area setting unit 26 executes, for example, the first area setting mode when there is no previous detection result of the travel lane line as an independent process for each of the left and right travel lane lines, When the previous detection result of the line exists, the second area setting mode is executed.

  Furthermore, the detection area setting unit 26, for example, as a process for a combination of two left and right traveling lane markings suitable for an appropriate traveling area, is performed when there is a previous detection result for only one of the lane markings. The third area setting mode is executed, and the fourth area setting mode is executed when the previous detection results of both travel lane markings exist.

In the first area setting mode, the detection area setting unit 26 sets, for example, a detection area having a predetermined first search width for this straight line on the assumption that the previous detection result of the travel lane marking is a straight line. To do.
In this case, for example, as shown in FIG. 6A, for example, the boundary is set for the right traveling lane line area RA and the left traveling lane line area LA corresponding to the left and right traveling lane markings by the center position CP of the host vehicle. The areas RA and LA are set so as not to exceed the boundary.

In addition, in the second area setting mode, the detection area setting unit 26 determines a predetermined value for the travel lane line of the detection result based on the previous detection result of the travel lane line, as shown in FIG. 6B, for example. Detection areas having the second search width (for example, the right traveling lane line area RA and the left traveling lane line area LA corresponding to the left and right traveling lane markings) are set.
Note that the second search width is set to be smaller than the first search width, for example.

  Further, in the third area setting mode, the detection area setting unit 26 selects the first area (for example, the travel of the previous detection result) based on one of the two travel lane lines where the previous detection result exists. A detection area having a predetermined second search width is set for the lane marking. Further, assuming that the other traveling segment line of the two traveling segment lines that does not have the previous detection result is a straight line, the second region (for example, detection having a predetermined first search width with respect to the assumed straight line). Area). Then, the detection region is set by combining the first region with priority over the second region.

  For example, as shown in FIG. 6C, the detection area setting unit 26 includes a first area corresponding to the right travel division line where the previous detection result exists and the left travel division line where the previous detection result does not exist. Are combined so as to overlap with the second region corresponding to. And when the overlap area of the 1st field and the 2nd field exists, the overlap area with the 1st field is excluded from the 2nd field, and the left run division line area LA is set, and it is the same as the 1st field The right travel lane marking area RA is set.

  In the fourth area setting mode, the detection area setting unit 26 determines two areas (for example, predetermined predetermined values for the travel classification lines of the previous detection results based on the previous detection results of the two travel classification lines. A region having a second search width is set, and a detection region is set by combining the two regions. In addition, when an overlapping area exists when two areas are combined so as to overlap, the overlapping area is excluded from the detection area.

  For example, as shown in FIG. 6D, the detection area setting unit 26 overlaps the right traveling lane line area RA and the left traveling lane line area LA corresponding to the two left and right traveling lane lines where the previous detection result exists. The detection areas are set by combining them to match and excluding the overlapping area AA.

  The travel lane marking detection device 10 according to the present embodiment has the above-described configuration. Next, the operation of the travel lane marking detection device 10, particularly, the operation of the processing device 12 will be described.

  First, in step S01 shown in FIG. 7, for example, image data output from the in-vehicle camera 11 and a pair of images are acquired, and a grayscale image is obtained based on one of the pair of images (for example, the right image). Generate.

Next, in step S02, a first edge detection process is executed.
Next, in step S03, detection area setting processing is executed.
Next, in step S04, a second edge detection process is executed.
Next, in step S05, a three-dimensional information extraction process is executed.

In this three-dimensional information extraction process, the three-dimensional information on the edge plane detected by the second edge detection process based on the three-dimensional image data acquired from the in-vehicle camera 11, that is, in the image plane of the grayscale image. Information on the position in the horizontal direction (Y direction) and the vertical direction (Z direction) corresponding to the two-dimensional array and information on the distance in the forward direction (X direction) of the host vehicle are extracted.
Next, in step S06, a travel lane marking estimation process is executed, and the process proceeds to the end.

  Details of the first edge detection process in step S02 described above will be described below.

First, for example, in step S11 shown in FIG. 8, smoothing processing is performed on a grayscale image using a Gaussian filter or the like.
Next, in step S12, the edge strength and the gradient are estimated using a Sobel filter or the like on the image obtained by the smoothing process.
Next, in step S13, the thickness of the detected edge is reduced according to the estimation result of the edge strength and the gradient.

Next, in step S14, using two different high-side threshold values and low-side threshold values, if the edge strength of the pixel to be determined is larger than the high-side threshold value, it is determined as an edge point, and the determination target When the edge strength of the pixel is smaller than the low threshold, it is determined that the pixel is not an edge point. In addition, when the edge strength of the determination target pixel is not less than the low side threshold and not more than the high side threshold, if the edge point exists in the vicinity of the determination target pixel, the pixel is determined to be an edge point. If there is no edge point in the vicinity, it is determined that it is not an edge point.
Then proceed to return.

  Details of the detection area setting process in step S03 described above will be described below.

First, for example, in step S21 shown in FIG. 9, it is determined whether or not there is a previous detection result of the left traveling segment line among the two left and right traveling segment lines suitable for an appropriate traveling region.
If this determination is “NO”, the flow proceeds to step S 28 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S22.

  In step S22, based on the previous detection result of the left travel lane line, the left travel lane line area LA having a predetermined second search width is set for the detected travel lane line.

Next, in step S23, it is determined whether or not there is a previous detection result of the right travel segment line among the two left and right travel segment lines that fit the appropriate travel region.
If this determination is “NO”, the flow proceeds to step S 26 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S24.

In step S24, based on the previous detection result of the right traveling lane line, a right traveling lane line area RA having a predetermined second search width is set for the detected traveling lane line.
In step S25, the combination of the two left traveling lane line areas LA and the right traveling lane line area RA is excluded from the combination of the left traveling lane line area LA and the right traveling lane line area RA. To set the detection area and proceed to return.

In step S26, assuming that the previous detection result of the right travel lane marking is a straight line, a right lane marking area RA having a predetermined first search width is set for this straight line.
In step S27, the left traveling lane line area LA is combined with the left traveling lane line area RA in preference to the right traveling lane line area RA for the combination of the left traveling lane line area LA and the detection area. Set and proceed to return.

  Further, in step S28, assuming that the previous detection result of the left travel segment line is a straight line, a left travel segment line area LA having a predetermined first search width is set for this straight line.

Next, in step S29, it is determined whether or not there is a previous detection result of the right travel segment line among the two left and right travel segment lines that fit the appropriate travel region.
If this determination is “NO”, the flow proceeds to step S 32 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S30.

In step S30, based on the previous detection result of the right traveling lane line, a right traveling lane line area RA having a predetermined second search width is set for the detected traveling lane line.
In step S31, the combination of the left traveling lane line area LA and the right traveling lane line area RA is combined with the right traveling lane line area RA prior to the left traveling lane line area LA. Set and proceed to return.

Further, in step S32, assuming that the previous detection result of the right travel segment line is a straight line, the right travel segment line area RA having a predetermined first search width is set for this straight line.
In step S33, a boundary is set according to the center position of the host vehicle with respect to the combination of the left traveling lane line area LA and the right traveling lane line area RA, and each left traveling lane line area LA and right traveling lane line area. Set RA so that it does not exceed the boundary and proceed to return.

  Details of the second edge detection process in step S04 described above will be described below.

First, for example, in step S41 shown in FIG. 10, among the edge points detected by the first edge detection process, within each right travel lane line area RA and left travel lane line area LA set by the detection area setting process. Estimate edge strength and gradient of existing edge points.
In this process, using a differential filter such as a Sobel filter, at least the vertical direction (90 ° direction) and the horizontal direction (0 ° direction) in the image plane of the grayscale image and two diagonal directions opposite to each other (for example, A plurality of directions including four directions composed of 45 ° direction and diagonal direction by 135 ° direction are scanned.

  Next, in step S42, at the edge points detected in each of the right traveling lane line area RA and the left traveling lane line area LA, the rising edge of the luminance is a positive edge point and the falling edge of the luminance based on the gradient of the edge point. Is identified as a negative edge point.

Next, in step S43, for example, is the edge point identified as positive or negative in the scanning direction from the left side to the right side in the image plane of the grayscale image a negative edge point in the left travel lane marking area LA? Determine whether or not.
If this determination is “NO”, the flow proceeds to step S 45 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S44.

  Next, in step S44, the process proceeds to return with the negative edge point in the left travel lane marking area LA as the edge point used for detecting the left travel lane marking.

Further, in step S45, for example, whether the edge point identified as positive or negative with respect to the scanning direction from the left side to the right side in the image plane of the grayscale image is a positive edge point in the right travel lane marking area RA. Determine whether.
If this determination is “NO”, the flow proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S46.

  Next, in step S46, the process proceeds to return with the positive edge point in the right travel lane marking area RA as the edge point used for detecting the right travel lane marking.

  Details of the travel lane marking estimation process in step S06 described above will be described below.

  First, for example, in step S51 shown in FIG. 11, a predetermined number of edges are randomly selected from the edge points detected by the second edge detection process for each of the left traveling lane line area LA and the right traveling lane line area RA. Extract points.

  Next, in step S52, using the extracted edge points, a least square method using a quadratic curve is executed on the XY plane, and a least square method using a linear function is executed on the ZX plane.

  Next, in step S53, on the XY plane, the distance (for example, the distance between the quadratic curve of the running lane marking estimation candidate obtained by the method of least squares and all the edge points detected by the second edge detection process) Normal distance) is calculated, and the number of edge points at which this distance is less than or equal to a predetermined threshold is calculated.

Next, in step S54, it is determined whether or not the number of execution repetitions of the series of edge point calculation processes in steps S51 to S53 described above has reached a predetermined number of repetitions.
If this determination is “NO”, the flow returns to step S 51 described above.
On the other hand, if this determination is “YES”, the flow proceeds to step S55.

  Next, in step S55, the number of edge points at which the distance to the travel lane line estimation candidate is equal to or less than a predetermined threshold is calculated from the calculation results of a series of edge point calculation processes executed over a predetermined number of repetitions. The quadratic curve of the running lane marking estimation candidate is selected as the most probable estimation candidate.

  Next, in step S56, the distance (for example, the normal distance) of the most probable estimation candidate to the quadratic curve out of all edge points detected by the second edge detection process is equal to or less than a predetermined threshold. Edge points are extracted, and a least square method using a quadratic curve is performed on the extracted edge points on the XY plane. The quadratic curve based on the calculation result is used as the estimation result of the travel line, and the process proceeds to return.

As described above, according to the travel lane marking detection device 10 according to the present embodiment, the second edge detection unit 23 causes at least the vertical direction (90 ° direction) and the horizontal direction (0 ° direction) on the image plane of the grayscale image. ) And two diagonal directions (for example, 45 ° and 135 ° diagonal directions) are scanned.
As a result, in addition to the straight road, the edge point can be detected appropriately and accurately, for example, in a curve having a large curvature.

In addition, the detection processing by the first edge detection unit 22 is executed prior to the detection processing by the second edge detection unit 23, so that detection accuracy can be improved by detecting edge points in two stages. it can.
In addition, by detecting the edge point inside the travel area among the edge points of the travel lane marking, for example, when the detection result of the travel lane marking is used for control such as lane departure prevention, appropriate control can be executed. it can.

Further, the travel lane marking estimation unit 25 executes the least square method using a quadratic curve on the XY plane using the robust estimation by RANSAC, for example, when executing the least square method using a three-dimensional function in a three-dimensional space. In comparison, it is possible to ensure desired estimation accuracy while preventing an increase in calculation load.
In addition, by executing the least square method using a linear function on the ZX plane, the relative inclination of the road surface to the host vehicle on the ZX plane and the pitch variation of the host vehicle are reflected in the estimation of the driving lane marking and the detection area setting. It is possible to improve the estimation accuracy of travel lane markings.

Furthermore, according to the first region setting mode executed by the detection region setting unit 26, even if there is no previous detection result of the travel lane marking, by setting a larger detection region by linear approximation, Edge points can be detected properly.
Furthermore, according to the second region setting mode, the detection region can be appropriately set based on the previous detection result of the travel lane marking.
Furthermore, according to the third region setting mode, the edge point can be properly detected by setting the detection region by using the previous detection result of the traveling lane marking and the linear approximation together.
Furthermore, according to the fourth area setting mode, it is possible to appropriately set the detection area based on the previous detection result of the two travel lane markings that match the travel area. In addition, the edge points can be set appropriately by excluding, from the detection area, areas that overlap in the right and left lane marking areas due to abnormal events such as erroneous detection of lane markings and detection errors. Can be detected.

  In the above-described embodiment, the travel lane marking estimation unit 25 performs the robust estimation including the least square method using a nonlinear function in the two-dimensional plane. However, the present invention is not limited to this. The running lane marking may be estimated by performing robust estimation including a least square method using a dimension function.

  It should be noted that the technical scope of the present invention is not limited to the above-described embodiment, and includes various modifications made to the above-described embodiment without departing from the spirit of the present invention. That is, the configuration of the above-described embodiment is merely an example, and can be changed as appropriate.

10 traveling lane marking device 11 vehicle-mounted camera (imaging means)
22 1st edge detection part (1st edge detection means)
23 Second edge detection unit (second edge detection means)
24 three-dimensional information extraction unit 25 travel lane marking estimation unit (travel lane marking estimation means)
26 detection area setting section (detection area setting means)
31 Intra-region edge detection unit (intra-region edge detection means)
32 Edge positive / negative identification part (edge positive / negative identification means)
33 Inner edge detection unit (running region inner edge detection means)

Claims (13)

A traveling lane marking detection device that detects a traveling lane marking of a moving object from a road surface image captured by an imaging unit that images a road surface,
First edge detection means for detecting an edge of the travel lane marking from the road surface image;
Detection area setting means for setting a detection area based on the previous detection result of the travel lane marking;
In the detection area set by the detection area setting means, a plurality of directions including at least an oblique direction in the image plane of the road surface image are scanned, and based on the edges detected by the first edge detection means, Second edge detecting means for detecting the inner edge;
Traveling lane marking estimating means for estimating the traveling lane marking adapted to the traveling area from the edge detected by the second edge detecting means;
A travel lane marking detection device comprising: The second edge detecting means includes
In-area edge detection means for detecting the edge in the detection area;
Edge positive / negative identifying means for identifying rising edges as positive edges and falling edges as negative edges in the edges detected by the in-region edge detecting means;
Based on the identification result of the edge positive / negative identification means for the two traveling division lines on the front side and the rear side in the scanning direction, the positive edge is detected in the traveling division line on the front side, and the rear In the traveling division line on the side, traveling region inner edge detection means for detecting the negative edge,
The travel lane marking detection device according to claim 1, comprising:   The edge positive / negative identifying means scans in the detection area a plurality of directions including at least four directions including at least a vertical direction and a horizontal direction on the image plane and two diagonal directions opposite to each other in the detection area. The travel lane marking detection device according to claim 2, wherein an edge and the negative edge are identified. The imaging means can acquire three-dimensional information,
2. The travel lane marking estimation unit estimates the travel lane marking by performing robust estimation on a two-dimensional plane with respect to the edge detected by the second edge detection unit. The travel lane marking detection device according to claim 3.   The travel lane marking estimation unit performs robust estimation including a least square method using a nonlinear function on a first two-dimensional plane with respect to the edge detected by the second edge detection unit; 5. The travel lane marking detection device according to claim 4, wherein the travel lane marking is estimated by executing a least square method using a linear function in a two-dimensional plane. The imaging means can acquire three-dimensional information,
The travel lane marking estimation unit estimates the travel lane marking by performing robust estimation in a three-dimensional space on the edge detected by the second edge detection unit. The travel lane marking detection device according to any one of claims 1 to 3.   The detection area setting means sets the detection area using a conversion result obtained by back projecting the traveling lane marking in the three-dimensional space estimated by the traveling lane marking estimation means to a two-dimensional plane. The travel lane marking detection device according to any one of claims 4 to 6, wherein   The travel lane marking estimation unit uses the travel lane line estimation candidates when using the distances between the travel lane line estimation candidates and all the edges detected by the second edge detection unit in the robust estimation. The travel lane marking detection device according to any one of claims 4 to 7, wherein a normal distance to the edge is the distance. The detection area setting means includes:
When there is no previous detection result of the travel lane marking, execute the first region setting mode,
When there is a previous detection result of the travel lane marking, execute the second region setting mode,
When there is a previous detection result of only one of the two travel division lines that match the travel region, execute the third region setting mode,
9. The fourth region setting mode is executed when there is a previous detection result of the two travel lane markings that match the travel region. 9. Traveling line detection device. The detection area setting means includes:
In the first region setting mode, the detection region is set on the assumption that the previous detection result of the travel lane marking is a straight line,
In the second region setting mode, the detection region is set based on the previous detection result of the travel lane marking,
In the third region setting mode, a first region that is set based on the one of the two traveling lane markings, and a second region that is obtained assuming that the other of the two traveling lane markings is a straight line; And setting the detection region based on a region obtained by combining the first region with priority over the second region,
10. The travel lane marking detection device according to claim 9, wherein in the fourth region setting mode, the detection region is set based on a previous detection result of two travel lane lines that match the travel region. . The detection area setting means includes:
In the fourth region setting mode, when two regions set based on the previous detection results of the two travel lane markings that match the travel region are combined, an overlapping region is excluded from the detection region. The travel lane marking detection device according to claim 10, wherein The first edge detecting means includes
Smoothing processing means for smoothing the road surface image to remove noise;
Edge detecting means for detecting the edge of the road surface image smoothed by the smoothing processing means;
Thinning processing means for thinning the edge output by the edge detection means;
Hysteresis threshold processing means for removing noise from the edge thinned by the thinning processing means using two threshold values;
The travel lane marking detection device according to any one of claims 1 to 11, further comprising: The travel lane marking detection device according to any one of claims 1 to 12, wherein the imaging unit uses the road surface image as a grayscale image.