KR101372425B1 - System for detecting lane markings - Google Patents

Abstract

The present invention relates to a lane marking detection system.
For example, a horizontal LoG filter unit for applying a plurality of horizontal scale values and filtering the image based on a scale-space, and performing a horizontal LoG filter, and a predetermined threshold value for the image data filtered through the horizontal LoG filter unit. A first data processor including a first binary image data unit configured to generate first binary image data, and a first data combiner configured to logically combine the first binary image data; A vertical LoG filter unit applying a plurality of vertical scale values and filtering the image by vertical LaG (Laplacian of Gaussian) based on a scale-space, and applying a predetermined threshold to the image data filtered by the vertical LoG filter unit A second data processor including a second binary image data unit for generating second binary image data, and a second data combiner for logically combining the second binary image data; And a third data combiner for logically combining binary image data coupled through the first data combiner and the second data combiner, respectively.

Description

Lane marking detection system {SYSTEM FOR DETECTING LANE MARKINGS}

The present invention relates to a lane marking detection system.

Recently, various techniques have been developed to provide safety and convenience by replacing or assisting a driver. For example, front vehicle detection and tracking for collision avoidance and leading vehicle tracking, side vehicle detection for safe lane changes, pedestrian detection, driver status recognition to prevent drowsy driving, and lane marking to prevent lane departure. Detection and the like.

Lane marking detection typically uses front cameras and computer vision installed in a dashboard or rear view mirror location. Research on lane marking detection technology has been carried out for many decades in a wide variety of approaches. Because lane marking has bright features on a relatively dark background, it can be used to approach edge features, techniques based on region features such as color and texture, and segmented linear segmentation, clathoids, parabolas, hyperbolas Modeling approaches, such as scorpions, phonetic curves, and snakes, are also being attempted. These modeling techniques generally use the Hough transform, which is particularly robust to occlusion. However, it is not easy to achieve robust performance due to lighting, preservation and irregularity of lane markings, similar objects causing confusion, heavy occlusion and the like.

The present invention provides a lane marking detection system that uses scale-space to provide a robust robustness to environmental changes without the assumption of global geometry, while at the same time reducing noise candidate region detection.

In the lane marking detection system according to an exemplary embodiment of the present invention, a horizontal LoG filter unit for applying a plurality of horizontal scale values and filtering the horizontal LaG (Laplacian of Gaussian) filter based on a scale-space, the horizontal LoG filter unit First data including a first binary image data unit for generating first binary image data by applying a predetermined threshold to the filtered image data, and a first data combiner for logically combining the first binary image data. Processing unit; A vertical LoG filter unit applying a plurality of vertical scale values and filtering the image by vertical LaG (Laplacian of Gaussian) based on a scale-space, and applying a predetermined threshold to the image data filtered by the vertical LoG filter unit A second data processor including a second binary image data unit for generating second binary image data, and a second data combiner for logically combining the second binary image data; And a third data combiner for logically combining binary image data respectively coupled through the first data combiner and the second data combiner.

의 수식을 만족하고, 상기 ω는 상기 영상에서 나타나는 차선의 폭이며, 상기 영상의 화소를 단위로 할 수 있다.In addition, the range of the horizontal scale value (σ) is

Ω is a width of a lane appearing in the image and may be a pixel of the image.

의 수식을 만족하고, 상기 ω는 상기 영상에서 나타나는 차선의 폭이며, 상기 영상의 화소를 단위로 할 수 있다.In addition, the range of the vertical scale value (σ) is

Ω is a width of a lane appearing in the image and may be a pixel of the image.

delete

According to the present invention, the use of scale-space for lane marking detection can provide a significantly more stable performance and more accurate detection results against environmental changes.

1 is a graph showing pulses having widths of 21, 15, 7, and 3 pixels, respectively.
FIG. 2 is a graph showing a scale space when LoG filters having scale values of 10, 7, 3, and 1 are applied to the pulses of FIG. 1.
3 is a block diagram schematically illustrating an overall configuration of a lane marking detection system according to an exemplary embodiment of the present invention.
4 is a photograph showing a first experimental image.
5A and 5B are diagrams illustrating the results of horizontal LoG filtering of the image of FIG. 4 by applying two different scale values.
6A and 6B are diagrams illustrating the binary images of FIGS. 5A and 5B, respectively.
7A and 7B are graphs showing the results of applying the thresholds to FIGS. 5A and 5B, respectively.
8 is a view showing a result of combining Figure 7a and 7b.
9A to 9C are diagrams illustrating the results of vertical LoG filtering the image of FIG. 4 by applying three different scale values.
10A to 10C are diagrams illustrating the binary images of FIGS. 9A to 9C, respectively.
FIG. 11 illustrates a result of combining image data applying a threshold to FIGS. 9A to 9C.
FIG. 12 is a diagram illustrating a result of logical AND combining the image data of FIGS. 8 and 11.
13 to 17 are diagrams for comparing the results according to the conventional variable thresholding technique, the results of the Canny edge image and the half transform, and the results according to the present embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

First, in order to facilitate understanding of an embodiment of the present invention, the scale space and Laplacian of Gaussian (LoG) filtering applied to an embodiment of the present invention will be briefly described.

1. Scale Space

The scale space for a signal can be expressed by convoluting it with a Gaussian kernel of various widths, as shown in Equation 1 below.

In Equation 1, σ represents a standard deviation of the Gaussian kernel, and for the scale parameter α, the original image f and the scaled image f ′ have a relationship as in Equation 2 below.

Such scale space representations of f and f 'may be defined as in Equation 3 below.

In Equation 3, the spatial variables and the scale parameter may be converted according to Equation 4 below.

Therefore, the relationship between L and L 'can be summarized as in Equation 5 below.

In addition, the m-th order space derivative of Equation 4 satisfies Equation 6 below.

In order to maintain the same filtering result for L and L ', the normalized Gaussian Laplacian of the image may be defined as in Equation 7 below.

In general, the local maximum of these feature responses can be detected and selected as the optimal scale. Thus, the optimal scale of the pixel is accompanied by spatial position information and reflects the pixel characteristic magnitude. If the input image is rescaled by the constant parameter α in equation (4), the scale that obtains the maximum response for all scales is multiplied by the same constant. Thus, if the normalized scale space maximum of any pixel of the image f is obtained at scale sigma ₀ of the scale space representation, it can be seen that the scale space maximum is obtained at scale ασ ₀ of the scale space representation of f '.

2. LoG Filtering

1 is a graph showing pulses having widths of 21, 15, 7, and 3 pixels, respectively. FIG. 2 is a graph showing a scale space when LoG filters having scale values of 10, 7, 3, and 1 are applied to the pulses of FIG. 1.

It can be seen from FIG. 2 that the maximum amplitude occurs coinciding with the position of a pulse of a particular magnitude. When the scale is relatively small compared to the pulse width, two poles are generated inside both boundary lines, but as the scale grows, the poles are gathered to the center. It can also be seen that for a pulse of a particular width, a LoG filter using that standard deviation value provides the maximum response. Therefore, the LoG operator can be used to find the scale and position of an object simultaneously.

Hereinafter, the configuration of the lane marking detection system according to the present embodiment will be described.

3 is a block diagram schematically illustrating an overall configuration of a lane marking detection system 1000 according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the lane marking detection system 1000 according to an exemplary embodiment of the present invention includes a first data processor 100, a second data processor 200, and a third data combiner 300.

The first data processor 100 includes a horizontal LoG filter unit 110, a first binary image data unit 120, and a first data combiner 130. The horizontal LoG filter unit 110 may apply horizontal plural horizontal scale values to horizontal LoG filter the photographed road image. The first binary image data unit 120 may generate first binary image data, respectively, by applying a predetermined gate value to the image data filtered through the horizontal LoG filter unit 110. The first data combiner 130 may combine the first binary image data generated through the first binary image data unit 120.

The second data processor 200 includes a vertical LoG filter 210, a second binary image data 220, and a second data combiner 230. The vertical LoG filter 210 may apply vertical plural horizontal scale values to vertical LoG filter the image. The second binary image data unit 220 may generate second image data by applying a predetermined gate value to the image data filtered by the vertical LoG filter unit 210. The second data combiner 230 may combine the second image data generated through the second binary image data unit 220.

Hereinafter, an experimental example of lane marking detection will be described with reference to the accompanying drawings.

4 is a photograph showing a first experimental image. 5A and 5B are diagrams illustrating the results of horizontal LoG filtering of the image of FIG. 4 by applying two different scale values. 6A and 6B are diagrams illustrating the binary images of FIGS. 5A and 5B, respectively. 7A and 7B are graphs showing the results of applying the thresholds to FIGS. 5A and 5B, respectively. 8 is a view showing a result of combining Figure 7a and 7b.

In general, the width and position of lane markings are not determined in actual images, and there is no prior knowledge about them. The width of the lane markings on the image varies depending on the distance from the bottom and the center vertical line of the image, and there are some lane markings that do not meet the regulations or are damaged, so that the image can be detected to detect the lane marking width with a range rather than a specific value. Expressed in discrete one-dimensional LoG filtering scale space of various scales. To do this, the image is convolved with horizontal and vertical LoG filters, respectively, and the results are combined. In this regard, the overall algorithm for the lane marking detection method may be represented as in Table 1 below.

A detailed description of the algorithm of Table 1 for the experimental image of FIG. 4 is as follows. First, an experimental example of the first data processing unit 100 will be described.

The size of the image of FIG. 4 is 165 x 220, and the width? Of the lower lane is about 10 pixels. The range of the scale value sigma does not exceed the maximum ω / 2 based on the result of the "LoG filtering" described above, and is preferably larger than the minimum one pixel. More specifically, it is preferable to satisfy the following Equation 8 of the scale value σ.

In this experimental example, scale values of σ ₁ and σ ₂ were applied to the image of FIG. 4, and σ ₁ = 5 and σ ₂ = 3 according to the condition of Equation 8. Two scale values σ ₁ = 5 and σ ₂ = 3 were applied, and horizontal LoG filtering of the image of FIG. 4 resulted in the same results as in FIGS. 5A and 5B. 5A is a response f1 of a horizontal LoG filter corresponding to a scale value σ ₁ = 5, and FIG. 5B is a response f2 of a horizontal LoG filter corresponding to a scale value σ ₂ = 3.

6A and 6B are binary images, and FIGS. 5A and 5B are compared for each pixel and displayed only when the maximum value is obtained. If these binary images are represented by b1 and b2, respectively, they can be defined as in Equation 9 below.

Here, when n represents the number of scales, i, j = 1, 2,... becomes n. Therefore, the following equation can be established in any coordinate pair (x, y).

Next, binary image data as shown in FIGS. 7A and 7B may be obtained by applying a predetermined threshold to horizontal LoG filtered image data as shown in FIGS. 5A and 5B. It can be seen from Fig. 2 that it is appropriate to select about -0.7 (absolute value 0.7) so that only one scale can be selected without ambiguity for each pulse.

By logically combining such binary image data, data as shown in FIG. 8 may be obtained. 8 combines two binary image data using different brightness values. In this case, as shown in FIG. 8, two brightness values exist, and the darker it corresponds to a larger scale space. That is, when T is a threshold value and n is the number of scales, this process is expressed as a pseudo code as shown in Table 2 below.

Next, an experimental example of the second data processor 200 will be described.

9A to 9C are diagrams illustrating the results of vertical LoG filtering the image of FIG. 4 by applying three different scale values. 10A to 10C are diagrams illustrating the binary images of FIGS. 9A to 9C, respectively. FIG. 11 illustrates a result of combining image data applying a threshold to FIGS. 9A to 9C.

Since the experimental example by the second data processing unit 200 is similar to the experimental example of the first data processing unit 100 described with reference to FIGS. 5A to 8, detailed description thereof will be omitted. However, in the case of vertical LoG filtering, the vertical scale value σ ₁ = ω is further added because the width in the driving direction is wider than the horizontal width. That is, the vertical LoG filtering applied three scale values σ ₁ = 10, σ ₂ = 5, and σ ₃ = 3 in this case. This vertical scale value range can be summarized as in Equation 11 below.

FIG. 12 illustrates a final result of combining the image data of FIGS. 8 and 11. Such a result may be obtained by logically combining the results of the first data combiner 130 and the second data combiner 230 through the third data combiner 300.

Hereinafter, a comparative example between the conventional lane marking detection technique and the technique of the present embodiment will be described.

13 to 17 are diagrams for comparing the results according to the conventional variable threshold processing technique, the results according to the Canny edge image and the half transform, and the results according to the present embodiment. In more detail, the original image, the variable thresholding method, the Hough transform, and the results according to the present embodiment are illustrated.

For the fairness of the experiment, typical parameter values were used for each technique, and these values were fixed for the entire experiment.

(A) of FIG. 13 is an original image including a curved section, (b) shows a variable threshold processing result, and (c) shows a Canny edge image and a half transform result thereof. It can be seen that the conventional variable threshold processing result shown in FIG. 13 (b) failed to detect the lane marking area, and the result shown in (c) shows that the Hough transform is not effective in the curve section. However, almost all lane markings are detected in FIG. 13D as a result of the present embodiment. Other detection areas fall into the category that can be easily removed by simple mid-level image processing techniques.

FIG. 14A illustrates a relatively clear image of a road sign and a roadside tree. The variable thresholding result shown in FIG. 14 (b) failed to detect even the lane markings which appeared relatively clearly, and irrelevant regions were detected. In the analysis of the half transform result of FIG. 14C, the lane marking does not generally appear as a straight line in the actual image, and thus the important lane marking is excluded in the candidate selection step because the straight line is broken. On the other hand, FIG. 14 (c) detects and provides all the main lane markings, and other detection areas belong to a category that can be easily removed by a simple mid-level image processing technique.

FIG. 15A illustrates a toll gate image, in which a brightness difference between a road surface and a lane marking is small and noise is high. The variable threshold processing result shown in FIG. 15B shows that the image is generally bright and the contrast is low, so that the variable threshold processing is useless for detection. The canny edge detection result shown in FIG. 15C shows that even when the lane marking is relatively clear to the human eye, even though the Canny edge detector is used, the detected edges may hardly continue. The edge image is almost noise level, so the straight line detected by the Hough transform is also outside of the actual lane marking. Of course, this problem can be reduced by filtering, but it can be seen that this embodiment also has the advantage of eliminating the need for pretreatment when compared with the experimental results according to the present embodiment shown in FIG. . As shown in FIG. 15D, all lane markings were accurately and clearly detected.

In the image illustrated in FIG. 16A, a false contour exists due to low camera resolution and image quality. In addition, sunlight is shining between the blocks in the upper middle. The variable threshold result shown in FIG. 16B for this image completely failed lane detection. Referring to (c) of FIG. 16, it can be seen that the Canny edge detector detects an unrelated curved edge due to the influence of a false contour line, and a horizontal straight line due to the Hough transform due to the horizontal edge due to the effect of sunlight shining between blocks. This was detected a lot. On the other hand, referring to Figure 16 (d), the results according to the present embodiment was hardly affected by the false contour and the illumination (sunlight). Other detection areas fall into the category that can be easily removed by simple mid-level image processing techniques.

In contrast to the existing techniques for the image of FIG. 17 (a), which is a small image having a size of 220 x 146, blurring phenomenon is proposed. It can be seen that the markings of the driving lanes are detected, resulting in a remarkably advantageous result for detecting lane markings.

According to the present embodiment, the proposed technique can provide much more stable performance than the existing typical lane marking detection techniques. In addition, the lane marking detection results generally contain fewer unnecessary components, and important components always include the most. In addition, the fact that it generally has both an important component detection feature and a noise or irrelevant component rejection feature, which are features that are generally in a trade-off or trade-off relationship for a particular technique, is particularly stressed. In addition, in general, experiments with various real environments show that the proposed method is significantly more robust than the existing methods. In this embodiment, a one-dimensional filter is used, and no preprocessing is required. It also maintains good performance, which reduces the amount of computation.

It will be apparent to those skilled in the art that the present invention may be practiced in various ways without departing from the spirit and scope of the present invention without departing from the spirit and scope of the present invention. It is.

1000: lane marking detection system
100: first data processing unit
110: horizontal LoG filter unit
120: first binary image data portion
130: the first data combiner
200: second data processing unit
210: vertical LoG filter unit
220: second binary image data portion
230: second data combiner
300: the third data combiner

Claims (4)

The system detects lane markings from the image of the road.
A horizontal LoG filter unit applying a plurality of horizontal scale values and horizontal LoG filtering the image based on a scale-space, and applying a predetermined threshold to the image data filtered through the horizontal LoG filter unit to apply the first binary image data. A first data processor including a first binary image data unit for generating each and a first data combiner for logically combining the first binary image data;
The second binary image data is applied by applying a plurality of vertical scale values and applying a predetermined threshold to the image data filtered through the vertical LoG filter unit for vertical LoG filtering the image based on the scale-space. A second data processor including a second binary image data unit for generating each and a second data combiner for logically combining the second binary image data; And
And a third data combiner for logically combining binary image data coupled through the first data combiner and the second data combiner, respectively. The method of claim 1,
The range of the horizontal scale value (σ) satisfies the following formula,

The ω is the width of the lane appearing in the image, the lane marking detection system in units of pixels of the image. 3. The method of claim 2,
The range of the vertical scale value (σ) satisfies the following formula,

The ω is the width of the lane appearing in the image, the lane marking detection system in units of pixels of the image. delete

Images