KR101773822B1 - Vision based lane recognition apparatus - Google Patents

Abstract

We propose an image-based lane recognition system that is robust to noise and capable of detecting straight lines and curves. The proposed apparatus includes an interest area setting unit for setting an area of interest including a driving lane in a road image from a vision sensor, an IPM unit for performing inverse perspective mapping (IPM) on an image of a region of interest provided from the ROI setting unit, A filtering and threshold processing unit for filtering the IPM image output from the IPM unit and converting a pixel having a value lower than a predetermined threshold value into a specific value in pixels of the filtered IPM image, A lane-tracking unit for fitting lanes detected by the lane detecting unit, an inverse IPM unit for performing inverse IPM conversion on the lane fitted by the lane-following unit, and an inverse-IPM unit for converting the in- And a lane-output unit for generating and outputting a final lane by deriving a road equation by applying a road model to the lane.

Description

[0001] VIDEO BASED LINE RECEPTION APPARATUS [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image-based lane recognition apparatus that can be used in a vehicle safety system and the like. More particularly, the present invention relates to an apparatus capable of recognizing a driving lane from an input image using a vision sensor.

Conventional lane detection methods include edge-based, color-based, and road-model-based methods.

In the case of a straight lane detection method using a simple edge extraction and a Hough transform, there is a limit to curve detection.

Also, in case of the probabilistic lane detection algorithm based on the road model, if the road contains noise similar to the lane, the error range of the detected lane is large and the opposite curve may be detected by probability.

Therefore, a robust lane recognition algorithm capable of detecting straight lines and curves is required.

Related prior arts include a method of robustly performing lane tracking, lane type, lane color, lane departure warning, etc. in a very simple manner with a small amount of computation even in the case of vehicle vibration, lighting change, Patent Literature No. 2012-0009590 (Lane Recognition System and Method).

Other prior related arts include a method of preprocessing a road image input from a camera, performing a log-polar conversion on a straight line or curve line in the ROI, then performing a curve fitting with the RANSAC algorithm, The contents which can improve the efficiency and increase the detection accuracy are disclosed in Korean Patent Laid-Open Publication No. 2013-0095967 (lane recognition method and apparatus).

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image-based lane recognizing apparatus which is robust against noise and can detect straight lines and curved lines.

According to another aspect of the present invention, there is provided an image-based lane recognition apparatus comprising: a region of interest setting unit that sets a region of interest including a driving lane in a road image from a vision sensor; An IPM unit for performing Inverse Perspective Mapping (IPM) to remove the perspective effect of the image set in the ROI provided from the ROI setting unit; A filtering and threshold processing unit performing filtering for noise reduction on the IPM image output from the IPM unit and converting a pixel having a value less than a predetermined threshold value into a specific value in pixels of the filtered IPM image; A lane detecting unit detecting lanes in the image processed by the filtering and threshold value processor; A lane-tracking unit for fitting a lane detected by the lane detecting unit; A reverse IPM unit for performing inverse IPM conversion on the lane fitted by the lane-tracking unit; And a lane output unit for generating and outputting a final lane by deriving a road equation by applying a road model to the lane converted into the inverse IPM by the inverse IPM unit.

At this time, the ROI setting unit may set a ROI corresponding to 86% of the disappearance lines to the top and 90% of the bottom of the ROI based on a heuristic method.

At this time, the lane detecting unit selects the candidate group of the lane using the cumulative histogram in the image processed by the filtering and threshold value processor 16, localizes the selected lane candidate group, and outputs the localized lane candidate The lane can be detected by applying the B-Spline and the RANSAC algorithm to each.

At this time, the lane-tracking unit may use the GVF SNAKE algorithm.

At this time, the lane-outputting unit may generate a final lane by deriving a road equation by applying a road model to a starting point and an intermediate point of the lane converted into the reverse IPM and a lowest point of the ROI.

According to the present invention having such a configuration, a ROI or a curve is detected and fitted in an ROI from an image input through a vision sensor, and converted into an IPM, and then a start point, a midpoint, (Lane Departure Warning System) and LKAS (Lane Keeping Assist System) by deriving the road equation by applying the road model to the lane departure warning system and the lane lateral offset of the lane in the application system such as the Lane Departure Warning System (LDWS) It is possible to improve the efficiency and increase the detection accuracy.

1 is a block diagram of an image-based lane recognition apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating an image-based lane recognition operation according to an embodiment of the present invention.
FIGS. 3 to 6 show examples of screens employed in the explanation of the image-based lane recognition operation according to the embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a block diagram of an image-based lane recognition apparatus according to an embodiment of the present invention.

1, the image-based lane recognizing apparatus includes a vision sensor 10, a ROI setting unit 12, an IPM unit 14, a filtering and threshold value processing unit 16, A lane-tracking unit 20, an inverse IPM unit 22, and a lane-output unit 24. The lane-

The vision sensor 10 is installed in front of the vehicle to acquire a road image in front of the vehicle. For example, the vision sensor 10 includes a camera.

The ROI setting unit 12 sets a region of interest (ROI) including the driving lane on the road image from the vision sensor 10. [

The IPM unit 14 performs an inverse perspective mapping (Inverse Perspective Mapping) process to remove the perspective effect of the input image by inputting the image in which the ROI is set through the ROI setting unit 12. A typical road image is composed of a solid line or a dotted line in 2D, but an actual road in which a lane exists is present in the 3D plane, so perspective effects must be removed for lane detection or lane recognition. That is, it is difficult to accurately evaluate the lane and road parameters due to the perspective effect. Therefore, in order to remove the perspective effect, it is necessary to acquire, for example, the position, coordinate, optics, (IPM) which maps a three-dimensional plane to a two-dimensional plane using a condition.

The filtering and threshold value processing unit 16 performs filtering for noise reduction on the IPM image output from the IPM unit 14 and performs filtering on the filtered IPM image with a value lower than a threshold value based on a threshold value by histogram analysis Are all set to zero. In other words, in order to find the portion corresponding to the lane in the filtered IPM image, it is necessary to make the pixels other than the portion of the lane to be a value of, for example, "0x00". And performs such an operation in the filtering and threshold value processor 16. On the other hand, in order to distinguish the lane from the background, there is a certain reference value, and this reference value is called a threshold value.

The lane detecting section 18 detects lanes (i.e., straight lines and curved lines) in the processed image (i.e., the region of interest) in the filtering and threshold value processing section 16.

The lane-finding section 20 fits the lane detected by the lane detecting section 18.

The reverse IPM unit 22 performs inverse IPM conversion of the lane fitted by the lane-following unit 20.

The lane output section 24 generates a final lane by deriving a road equation by applying a road model to a start point and an intermediate point of the lane converted into the reverse IPM and the lowermost point of the ROI.

FIG. 2 is a flow chart for explaining an image-based lane recognition operation according to an embodiment of the present invention, and FIGS. 3 to 6 illustrate examples of screens employed in an image-based lane recognition operation according to an embodiment of the present invention .

First, the ROI setting unit 12 receives a road image in front of the vehicle from the vision sensor 10 (S10).

Then, the ROI setting unit 12 sets the ROI based on the lost line derived by the external parameters of the camera (for example, installation height, pitch, yaw, etc. of the camera) (S12). That is, the region-of-interest setting unit 12 may output an image as illustrated in FIG. For example, based on the heuristic method, the ROI setting unit 12 may set the ROI to an area corresponding to 86% of the disappearance line at the top and 90% of the bottom edge , The pixel information is inaccurate).

Thereafter, the IPM unit 14 performs inverse perspective mapping (IPM) to remove the perspective effect of the image (see FIG. 3) in which the ROI is set through the ROI setting unit 12 (S14) . The IPM unit 14 can output an image as illustrated in FIG.

Then, the filtering and threshold value processor 16 performs filtering for noise reduction on the inverse-perspective-mapped (IPM) image (see FIG. 4), and the values below the threshold value through the histogram analysis are all 0 " (S16).

Thereafter, the lane detecting unit 18 selects a lane candidate group using the accumulated histogram in the image processed by the filtering and threshold value processor 16 (S18).

Then, the lane detecting unit 18 localizes the selected lane candidate group. At this time, the lane detecting unit 18 performs positioning using the RANSAC algorithm (S20).

In addition, the lane detecting unit 18 detects a lane (i.e., a straight line or a curve) by applying the B-Spline and RANSAC algorithms to the localized lane candidates (S22). Accordingly, the lane detecting section 18 can output an image as illustrated in Fig.

Thereafter, the lane-finding section 20 fits the lane detected by the lane detecting section 18 (S24). In the previous research, the position of the next lane was estimated by using a Kalman filter or the like for tracking the lane. In the embodiment of the present invention, the lane was tracked by fitting the detected lane and the GVF SNAKE algorithm was used. Here, the GVF SNAKE algorithm is conditioned on the balance of internal energy (internal E) and external energy (external E), for example "internal E + external E = 0". Those skilled in the related art can fully understand the GVF SNAKE algorithm through well-known techniques, so that a detailed description thereof will be omitted.

Then, the reverse IPM unit 22 performs reverse IPM conversion on the lane fitted by the GVF SNAKE algorithm (S26).

Thereafter, the lane output unit 24 generates a final lane by deriving the road equation by applying the road model to the starting point and the midpoint of the lane converted into the reverse IPM and the lowest point of the ROI (S28). For example, the lane-output section 24 can display the final lane L on the reverse-IPM-converted image (i.e., the original image) as shown in Fig.

The embodiment of the present invention as described above can be applied to a lateral offset of a lane in an application system such as an LDWS (Lane Departure Warning System) or an LKAS (Lane Keeping Assist System), and the linear and curved lane detection efficiency And the detection accuracy can be improved.

As described above, an optimal embodiment has been disclosed in the drawings and specification. Although specific terms have been employed herein, they are used for purposes of illustration only and are not intended to limit the scope of the invention as defined in the claims or the claims. Therefore, those skilled in the art will appreciate that various modifications and equivalent embodiments are possible without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

10: Vision sensor 12: ROI setting unit
14: IPM module 16: Filtering and threshold value processor
18: lane detection unit 20: lane tracking unit
22: reverse IPM section 24: lane output section

Claims (10)

A region of interest setting section for setting a region of interest including a driving lane in a road image from the vision sensor;
An IPM unit for performing Inverse Perspective Mapping (IPM) on an image in which a ROI provided from the ROI setting unit is set;
A filtering and threshold processing unit that performs filtering on the IPM image output from the IPM unit and converts a pixel having a value less than a predetermined threshold value into a specific value in pixels of the filtered IPM image;
A lane detecting unit detecting lanes in the image processed by the filtering and threshold value processor;
A lane-tracking unit for fitting a lane detected by the lane detecting unit;
A reverse IPM unit for performing inverse IPM conversion on the lane fitted by the lane-tracking unit; And
And a lane output unit for generating and outputting a final lane by deriving a road equation by applying a road model to the lane converted into the reverse IPM by the reverse IPM unit,
Wherein the lane-
In the image processed by the filtering and threshold processing unit, a lane candidate group is selected using the cumulative histogram, the selected lane candidate group is localized, and the localized lane candidate is subjected to B-Spline and RANSAC algorithm Wherein the lane detecting unit detects the lane by applying the lane detecting unit to the lane detecting unit. The method according to claim 1,
Wherein the ROI setting unit comprises:
Based on the heuristic method, a portion corresponding to the first set value of the disappearance line is set to the top and the attention area is set to the area corresponding to the second set value at the bottom. delete The method according to claim 1,
The lane-
Image based lane recognition device using GVF SNAKE algorithm. The method according to claim 1,
The lane-
And the road model is applied to the starting point and the midpoint of the lane converted into the reverse IPM and the lowermost point of the ROI, and the road equation is derived to generate and output the final lane. An image-based lane recognition method performed by an image-based lane recognition device,
Setting an area of interest including a driving lane in a road image from a vision sensor;
Performing inverse perspective mapping (IPM) on the image in which the ROI is set;
Performing filtering on the inverse perspective mapped IPM image;
Setting a pixel having a value lower than a preset threshold value in a pixel of the filtered IPM image to a specific value and processing the pixel;
Detecting a lane in the processed IPM image;
Fitting the detected lane;
Performing inverse IPM conversion of the fitted lane; And
And outputting a final lane generated by deriving a road equation by applying a road model to the lane converted into the inverse IPM,
The step of detecting a lane in the processed IPM image comprises:
The lane candidate group is selected using the cumulative histogram in the processed IPM image, the selected lane candidate group is localized, and the B-Spline and RANSAC algorithms are applied to the localized lane candidates, respectively, Based lane identification method. The method according to claim 6,
Wherein the step of setting an area of interest including a driving lane in a road image from the vision sensor comprises:
Based on the heuristic method, the portion corresponding to the first set value of the disappearance line is set to the top and the attention area is set to the area corresponding to the second set value at the bottom. delete The method according to claim 6,
Fitting the detected lane comprises:
Image based lane recognition method using GVF SNAKE algorithm. The method according to claim 6,
The step of outputting the final lane generated by deriving the road equation by applying the road model to the lane converted into the inverse IPM,
And generating and outputting the final lane by deriving the road equation by applying the road model to a starting point, a midpoint of the lane converted into the reverse IPM, and a lowest point of the ROI, and outputting the final lane.

Images