KR101121777B1 - Lane detection method - Google Patents

Abstract

The lane detection method is provided. The lane detection method according to the present invention analyzes each frame of an image photographed by a front camera to generate a lane candidate edge link, and generates a lane parameter set for the lane candidate edge link, wherein the lane parameter set is a lane position. And storing the lane parameter set, and comparing the lane parameter sets included in each frame to remove the noise lane parameter set or replace it with the effective lane parameter set of an adjacent frame. Inter-frame correction step.

Description

Lane detection method

The present invention relates to a lane detection method. More particularly, the present invention relates to a method of detecting a lane on which a vehicle is driven by analyzing an image photographed by a camera installed at a front portion of the vehicle.

Lane detection technology refers to a technology for detecting a lane of a road on which a vehicle is driven by analyzing an image captured by a camera installed at a front side of a vehicle. Lane detection methods may be classified into a feature based method and a model based method by analyzing the captured image.

Feature-based methods detect lanes from features that can be observed at the edge of the lane symbol. The feature-based method is' A. Broggi, S. Berte, “Vision-based road detection in automotive systems: a real-time expectation-driven approach”, Journal of Articial Intelligence Research 3 (1995) 325-348, et al.

The model-based method, on the other hand, represents a lane with a type of curve model that can be determined with some geometrical parameters. The model-based method is also described in Yue Wang, Eam Khwang Teoh, Dinggang Shen, “Lane detection and tracking using B-snake,” Image and Vision Computing, vol. 22, 2004, pp. 269-280 'and many others. In particular, Jung, C. R. and C. R. Kelber, “A lane departure warning system using lateral offset with uncalibrated camera,” Proc. IEEE Conf. on Intelligent Transportation Systems, Vienna, Austria, Sep. 13-16, 2005. pp. 102-107. 'Describes techniques for using a linear-parabolic model to detect lanes. In model-based methods, how to evaluate the parameters is important. That is, whether or not the lane is detected is determined according to how the parameter obtained from the image photographed by the camera is evaluated.

The property-based method generally detects lanes only on straight roads or freeways where the lanes are clearly painted. In contrast, the model-based method can detect a lane to some extent even if the road conditions are poor compared to the feature-based method. In other words, it is determined that the model-based method is more useful than the feature-based method. However, since the model-based method needs to process a lot of information related to modeling, there is a problem in that a large amount of computation is required, and processing takes time because a continuous error reduction algorithm is required to properly evaluate a parameter.

The technical problem to be solved by the present invention is to provide a lane detection method that can accurately detect the lane without being affected by the road conditions with the least amount of calculation.

Another technical problem to be solved by the present invention is to provide a lane detection method for detecting a lane by analyzing only the inside of the ROI by setting a lane presence possibility region of the image photographed by the front camera.

Another technical problem to be solved by the present invention is to detect an edge representing a lane, and then connect a gap that may exist between the edges, and an edge link pair to mean a left and right lane. It is to provide a lane detection method for detecting the noise edge to remove.

Another technical problem to be solved by the present invention is to provide a lane detection method capable of detecting not only the information on the position of the lane, but also the type of the lane and color information of the lane.

The technical problems of the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

The lane detection method according to an aspect of the present invention for achieving the technical problem is to analyze each frame of the image taken by the front camera to generate a lane candidate edge link, and to set the lane parameter set for the lane candidate edge link The lane parameter set generates a single frame analysis step including lane position information, lane color information, and lane type information, and stores the lane parameter set, compares the lane parameter set included in each frame, and compares the noise lane parameter. Interframe correction step of removing the set or replacing it with a valid lane parameter set of adjacent frames.

According to the present invention as described above, there is an effect that can accurately detect the lane without being affected by the road conditions with the least amount of calculation.

In addition, the information on the location of the lane as well as the information on the color of the lane and the type of the lane can be detected.

1 is a flowchart illustrating a lane detection method according to an embodiment of the present invention.
2 is a conceptual diagram of a conventional method for setting a region of interest.
3 is a conceptual diagram of a method for setting a region of interest according to an embodiment of the present invention.
4 is a detailed flowchart of an edge processing method according to an embodiment of the present invention.
5 is an example of applying an edge detection filter according to an embodiment of the present invention.
6 is a diagram illustrating an example of spaced connection considering a direction according to an embodiment of the present invention.
7 is an illustration of a lane candidate edge link generation operation according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, the present invention will be described with reference to the drawings for a block diagram or a process flow chart for explaining a lane detection method. At this point, it will be understood that each block of the flowchart illustrations and combinations of flowchart illustrations may be performed by computer program instructions. Since these computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, those instructions executed through the processor of the computer or other programmable data processing equipment may be described in flow chart block (s). It creates a means to perform the functions. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in the flowchart block (s). Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps are performed on the computer or other programmable data processing equipment to create a computer-implemented process to generate a computer or other programmable data. Instructions for performing the processing equipment may also provide steps for performing the functions described in the flowchart block (s).

In addition, each block may represent a portion of a module, segment, or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, the two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the corresponding function.

Hereinafter, an edge detection method is referred to to detect lane information. The edge detection method is described in reference 'http://en.wikipedia.org/wiki/Edge_detection', 'T. Lindeberg (2001) "Edge detection", in M. Hazewinkel (editor), Encyclopedia of Mathematics, Kluwer / Springer ',' D. Ziou and S. Tabbone (1998) "Edge detection techniques: An overview", International Journal of Pattern Recognition and Image Analysis ',' J. M. Park and Y. Lu (2008) "Edge detection in grayscale, color, and range Images", in B. W. Wah (editor) Encyclopedia of Computer Science and Engineering ', may be referred to throughout this specification.

First, a lane detection method according to the present invention will be described with reference to FIG. 1.

First, frames included in the video photographed by the front camera are sequentially input (S100). The frame refers to a still image for each unit time of the video captured by the front camera. The frame may be down sampled for a calculation speed (S102). Down sampling (S102) may be to reduce the resolution of the still image.

The frame may be analyzed through operation of the single frame level S110 of this embodiment, and as a result, a lane candidate edge link may be generated, and a lane parameter set for the lane candidate edge link may be generated.

Hereinafter, the operation of the single frame level S110 will be described in more detail.

First, a region of interest (ROI) in a down-sampled frame (S102) may be set (S112). When the region of interest is set, the lane parameter set is generated only for data in the region of interest. 2 illustrates an example of a region of interest setting that is generally performed in the lane detection method. As shown in FIG. 2, a method of setting a region of interest, which is generally implemented, is to set 202 a region of the center portion excluding a predetermined height of the top and bottom of the frame 200 as the region of interest. However, the ROI set by the ROI setting method, which is generally performed, is too large to sufficiently reduce the amount of computation. The region of interest setting method according to the present exemplary embodiment has an effect of further reducing the amount of computation by setting only a narrower region as the region of interest as compared with the method of setting a region of interest as shown in FIG. 2.

3 shows an example of setting a region of interest according to the present embodiment. FIG. 3 illustrates a process of setting a region of interest for the same still image as the frame illustrated in FIG. 2. First, as shown in FIG. 2, a case in which the ROI is set for an area of the center except for a predetermined height of the top and bottom of the frame 200 will be described. However, it is only an embodiment of the present invention to set the region of interest targeting only the region of the center portion, and the region of interest setting target image may be an original frame image.

First, a grid of a predetermined interval is generated on a frame (300). As shown in FIG. 3, the grid may be formed in the horizontal and vertical directions, but the grid is not necessarily formed in the horizontal and vertical directions. That is, there is no restriction in the formation direction of a grid. The grid is preferably formed with at least one pixel width.

Next, the color of the area enclosed by the grid is unified with a predetermined background color to generate a first modified frame (302). The background color may be black, for example. The background color may be similar to the color of asphalt applied to the roadway. Alternatively, the background color may be similar to the color of the cement applied to the roadway. The background color is preferably different from the color of the lane.

Next, an edge detection filter is applied to the first deformed frame 302 to generate a second deformed frame 304. The edge detection filter may be a Sobel edge detection filter. For a description related to the Sobel edge detection filter, reference is made to 'http://en.wikipedia.org/wiki/Sobel_operator', 'K. Engel (2006). Real-time volume graphics ,. pp. 112-114. ',' Pattern Classification and Scene Analysis, Duda, R. and Hart, P., John Wiley and Sons, '73, pp271-2 '.

The edge detection filter may be individually applied to data of each of the R, G, and B channels of the first modified frame 302. The lanes generally have a specific color, such as yellow, blue, and white. Therefore, if the edge detection filter is applied to each channel independently, there is an effect of increasing the probability that the lane shape is detected as an edge. The second modified frame 304 may be generated by combining the results of applying the edge detection filter to the data of each of the R, G, and B channels by a convolution operation. In addition, the second deformation frame 304 may be binarized 306 to the background color and the region of interest color so that the shape corresponding to the lane is clearly distinguished from the background color.

Next, the third modified frame 308 may be generated by setting colors of an edge pixel on the second modified frame 304 and adjacent pixels in a predetermined range of the edge pixel as the ROI color.

Next, the set of pixels of the ROI color on the third modified frame 308 may be set as the ROI 310. Comparing the ROI 202 shown in FIG. 2 with the ROI 310 set according to the present embodiment, it may be understood that the ROI 310 set according to the present embodiment is narrower. The narrower region of interest means that the area to be scanned for the detection of the lane is so narrow that the amount of computation for the detection of the lane is reduced.

After the ROI is set as shown in FIG. 3 (S112). An edge detection filter is applied to an image in the ROI to generate a lane candidate edge link corresponding to the shape of the lane (S113). 4 is a flowchart illustrating an operation of generating a lane candidate edge link corresponding to the shape of a lane. Hereinafter, 'edge' refers to a pixel determined as an edge pixel by applying an edge detection filter, and 'edge linke' refers to a set of edges adjacent to each other. In addition, hereinafter, 'adjacent' is when (x, y) coordinates on a frame representing a specific pixel is (x <0 <frame width-1, y <0 <frame height-1), (x- 1, y-1), (x-1, y), (x-1, y + 1), (x, y-1), (x, y + 1), (x + 1, y-1) , (x + 1, y), (x + 1, y + 1) is referred to to mean the position of eight pixels.

In operation S113, the lane candidate edge link is generated by applying an edge detection filter in the region of interest 310 (S1132), and a first edge link that is a set of adjacent edge pixels among the detected edge pixels. Generating a second edge link by connecting a gap below a gap limit pixel existing between the first edge link (S1134) and the left and right central axes of the frame among the second edge links. And generating the lane candidate edge link in which the noise edge link is removed based on the distance from the target signal.

In operation S1132, a smoothing operation using a Gaussian filter may be performed as a preprocess before applying the edge detection filter, and a Sobel edge detection filter may be applied. In addition, a hysteresis threshold may be used so that no separated edge pixel occurs, and non-local maximum suppression is used to solve a localization issue present in the Sobel edge detection filter. The method may be further used before using the Sobel edge detection filter. References related to this can be found in 'Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 1986. ' FIG. 5 is a diagram illustrating an edge pixel generation result when only the Sobel edge detection filter is used in the region of interest 310 (500) and when the non-maximal suppression method is additionally used (502) before the Sobel edge detection filter is used. As shown in FIG. 5, and before using the Sobel edge detection filter, if the non-maximal suppression method is further used (502), it can be seen that the edge pixels are sharper.

In operation S1134, the first edge link, which is a set of adjacent edges, is generated by scanning one direction of the frame in which the edge pixel is detected (S1132).

In operation S1136, a gap may exist between the first edge links to generate a second edge link. Even if an edge link corresponding to one lane may be broken in the middle of the image conversion process such as applying a filter, an interval of less than a predetermined gap limit pixel is connected in step S1136, and as a result, in the original one lane One that is correspondingly segmented and classified as two or more first edge links may be connected to one second edge link.

In step S1136, the average slope of each first edge link is measured, and the first edge link whose measured average slope is not within a predefined slope range is considered noise and excluded from the second edge link. Can be. The predefined slope range may be, for example, between 20 ° and 160 °. In this case, the lane has an effect of removing noise by using the fact that the lane must have an angle within a predetermined range with the traveling direction of the vehicle.

In operation S1136, the gaps existing between the first edge links may be connected with reference to an average slope of the first edge links. That is, the pixels included in the first edge link are included in the second edge link by scanning from the start pixel of the first edge link in the direction of the end pixel, and the average slope of the second edge link is measured. When there is an edge pixel belonging to the other first edge link when there is a maximum edge gap pixel in the average slope direction from the endpoint pixel of one edge link, and if there is an edge pixel belonging to the other first edge link The second edge link may be generated by connecting gaps between the first edge links. 6 is a view showing an example of the interval connection in consideration of the direction according to this embodiment. An example 600 for measuring the average slope and an example 602 for spacing connections are shown. 6, the edge pixels are colored in a dark color for convenience of description.

As shown in the example of the spacing of the connections, the first edge link is traversed in the average slope direction 606 of the first edge link, and when the end point 604 of the first edge link is reached, Three adjacent pixels in the average slope direction 606 are candidates for the new edge pixel. Among the candidate edge pixels, a new edge pixel may be determined based on the luminance of the pixel color. Note that the number of pixels added as a new edge pixel should be less than or equal to the maximum predetermined spacing limit pixel. The number of interval limit pixels may be, for example. The spacing using the average slope of the edge link according to the present embodiment has the effect of reducing noise generated by connecting each edge link as much as possible even when the edge is not actually connected.

In operation S1136, when the length of the average slope direction of the second edge link is less than a predetermined minimum lane length, the second edge link may be removed from the list of the second edge link. The minimum lane length may be 15 pixels, for example. In this case, there is an effect that can further filter out edge links that cannot be lanes with noise.

In operation S1138, noise may be further removed through left and right edge pairing during the second edge link. That is, for each of the second edge links, the second edge links positioned at opposite sides of the left and right central axes of the frame and including at least one pixel located at the same height are scanned and designated as paired links. Calculating a ratio at which the distance on the same height of the pixels included in the two edge links with the pixels included in the paired links is equal to or less than a predetermined limit lane area, and wherein the ratio equal to or less than the predetermined limit lane area is less than a predetermined edge link pair formation rate. A second edge link may be designated as the noise edge link to remove it from the second edge link. In this case, there is an effect that the lane can be further removed by using that the lane should be less than a certain width. The link pair formation rate may be, for example, 0.8.

The predetermined limit lane width may be defined differently for each height on the frame, and may decrease as the height on the frame increases. Since the frame image has a narrower left and right width when the distance is far from the projection method, and a longer distance lane is displayed as the height on the frame increases, the limit lane width for a specific height becomes higher as the height increases. Decreases. Increasing the height on the frame is the same as rising from the bottom to the top of the frame. The predetermined limit lane width may be configured in the form of a reference table in which a predetermined limit lane width corresponds to each height on the frame.

7 is an illustration of a lane candidate edge link generation operation S113 according to the present embodiment.

In the result 702 of detecting the edge pixel in the region of interest 700 (S1132), the result 704 of the edge link generation (S1134) is shown. In the result 704, it can be seen that the right lane is not visible, because there is a gap in the middle of the right lane, so that the length of the average slope direction of each first edge link is less than the minimum lane length. When the second edge link is generated by connecting the edge intervals to correct this (S1136), the right lane is visible again (706). Finally, edge pairing eliminates the noise of the second edge link 707 where the opposite second edge link within the lane limit area does not exist or the rate at which the ratio exists is less than the edge link pairing ratio. The candidate edge link is generated (S1139).

Returning to Fig. 1 again, the following operation will be described.

During the lane candidate edge link, noise may be further removed based on the color of the pixel included in the lane candidate edge link, and color information of the lane may be set (S114). That is, when the ratio of pixels having a predetermined lane color among pixels included in the lane candidate edge link is greater than or equal to a predetermined lane color threshold ratio, the lane color information for the lane candidate edge link is designated as the predetermined lane color. The lane candidate edge link, to which the lane color information is not specified, may be determined as noise and removed from the lane candidate edge link. In this case, there is an effect that it is possible to prevent the pixel of the color which the lane cannot have is recognized as the lane. In this case, when the color value of the frame is in the RGB format, after converting to the YUV format, it is preferable to perform an operation of further removing noise based on the color of the pixel (S114).

Next, a straight line model is generated that reflects pixels included in the lane candidate edge link, and the lane position information is configured using a parameter representing the straight line model (S115). In more detail, the lane position information is obtained by applying a Hough transform to the lane candidate edge link to express a lane candidate edge link in the equation xcosθ + ysinθ = ρ (θ, ρ) form. A description of the Hough transform is described in Duda, R. O. and P. E. Hart, "Use of the Hough Transformation to Detect Lines and Curves in Pictures," Comm. ACM, Vol. 15, pp. 11-15 (January, 1972), '' P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959 ',' http://en.wikipedia.org/wiki/Hough_transform#cite_note-1 '.

Next, the continuity of the lane may be checked and the type information of the lane may be set (S116). The type of lane may be one of a straight lane and a dotted lane. The lane type information may be determined by the formula shown in Table 1.

Li is the lane candidate edge link (i ≦ 0 <number of lane candidate edge links), P (solid | Li), and P (dash | Li) are posterior probability functions on the Bayesian probability model probability function), which is the probability that Li is a solid lane and a dashed lane, respectively, where P (solid | Li) = P (solid) P (Li | solid) and P (dash | Li) = P (dash) P (Li | dash), P (Li | solid) and P (Li | dash) are likelihood probabilities of the straight and dashed lanes, respectively. P (dash) may be a prior probability.

Where P (solid) = 1? It may be P (dash). Also, P (solid) may be predefined differently from each other according to the lane color information. For example, if the lane color is yellow, P (solid) = 0.8, if the lane color is blue, P (solid) = 0.7, and if the lane color is white, P (solid) = 0.5. In this case, it is possible to calculate the type of each lane by reflecting a certain correlation existing between the color of the lane and the type of the lane.

P (Li | solid) may be computed using the ratio of the length of the straight line and the length of the lane candidate edge link measured to represent the lane candidate edge link. P (Li | dash) = 1-P (Li | solid).

For a description of the Bayesian Probabilistic Model, see ET. Jaynes. Probability Theory: The Logic of Science Cambridge University Press, (2003). ISBN 0-521-59271-2 '.

When all operations of the single frame level S110 are performed, a lane parameter that is information about lanes included in each frame is generated, and the lane parameter is stored (S117). When the predetermined frame is analyzed, the interframe correction level S120 is started. In the inter-frame correction level S120, the lane parameter set is stored and the noise lane parameter set is removed by comparing the lane parameter set included in each frame, or replaced with the effective lane parameter set of an adjacent frame.

In the inter-frame correction level (S120), the lane position information of the lane parameters included in each frame is analyzed, and when the lane position information of the previous frame exceeds a predetermined change value threshold, it is determined as noise data and then removed. It may be replaced with the lane location information (S122).

In addition, as a result of analyzing the lane position information among lane parameters in one frame, when a lane exists on the right side from the left and right center axis of the frame but no lane exists on the left side of the frame, The lane parameter set may be added to the current frame (S124).

In addition, the lane color information and the lane type information may be summed for each predetermined update period frame and may be unified to a value having the highest frequency (S126).

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

Claims (8)

A lane candidate edge link is generated by analyzing each frame of an image captured by the front camera, and a lane parameter set for the lane candidate edge link is generated, wherein the lane parameter set includes lane position information, lane color information, and lane type. A single frame analysis step comprising information; And
Inter-frame correction step of storing the lane parameter set, comparing the lane parameter set included in each frame to remove the noise lane parameter set or replace it with a valid lane parameter set of an adjacent frame;
The single frame analysis step,
Setting a region of interest;
Applying an edge detection filter within the region of interest to detect edge pixels;
Creating a first edge link that is a set of adjacent edge pixels of the detected edge pixels;
Generating a second edge link by concatenating a gap below an interval limit pixel existing between the first edge links; And
And generating the lane candidate candidate edge link having the noise edge link removed from the second edge link based on a distance from a left and right central axis of the frame. The method according to claim 1,
The setting of the ROI may include:
Generating a grid at intervals on the frame;
Generating a first modified frame by unifying the color of the area enclosed by the grid with a predetermined background color;
Generating a second modified frame by applying an edge detection filter to the first modified frame;
Generating a third modified frame by setting a color of an edge pixel on the second modified frame and an adjacent pixel of a predetermined range of the edge pixel as a region of interest color; And
And setting the set of pixels of the region of interest color on the third deformed frame as the region of interest. delete The method according to claim 1,
Generating the second edge link,
Measuring an average slope of the first edge link; And
Generating the second edge link except for the first edge link where the measured average slope is not within a predefined slope range. The method according to claim 1,
Generating the second edge link,
Scanning in the direction from the start pixel to the end pixel of the first edge link to include pixels included in the first edge link in the second edge link;
Measuring an average slope of the second edge link; And
Determining whether there exists an edge pixel belonging to the other first edge link when the pixel progresses from the endpoint pixel of the first edge link by the gap limit pixel in the average slope direction;
If there are edge pixels belonging to the other first edge link, connecting the gaps existing between the first edge links to generate a second edge link. The method according to claim 1,
Generating the lane candidate edge link,
Scanning the second edge link for each of the second edge links, the second edge link including at least one pixel located opposite to the left and right central axes of the frame and located at the same height, and designating a pair of links;
Calculating a ratio of a pixel included in the second edge link having a same height as a pixel included in a paired link equal to or less than a predetermined limit lane width; And
Designating the second edge link as the noise edge link, wherein the ratio that is less than the predetermined limit lane width is less than a predetermined edge link pair formation rate. The method according to claim 1,
The lane type information,
expression

Determined by
Li is the lane candidate edge link (i ≦ 0 <number of lane candidate edge links), P (solid | Li), and P (dash | Li) are posterior probability functions on the Bayesian probability model probability function), meaning the probability that Li is a solid lane and a dashed lane, respectively.
P (solid | Li) = P (solid) P (Li | solid), P (dash | Li) = P (dash) P (Li | dash), P (Li | solid) and P (Li | dash) ) Are the likelihood probabilities of the straight lane and the dotted lane, respectively, and P (solid) and P (dash) are prior probability. The method of claim 7, wherein
Wherein P (solid) is predefined lane different from each other according to the lane color information.

Images