KR20140104516A - Lane detection method and apparatus - Google Patents

Abstract

The present invention relates to a lane detection method and a device for the same capable of improving lane detection performance. The lane detection method according to the present invention comprises a step of obtaining an image taken around a user′s vehicle and distance data sensing obstacles around the user′s vehicle; a step of detecting a sign within the image; a step of detecting a situation around the user′s vehicle based on content of the detected sign; and a step of detecting a lane from the image based on the detected surrounding situation information and the distance data.

Description

[0001] Lane detection method and apparatus [0002]

The present invention relates to a lane recognition method and apparatus, and more particularly, to a lane recognition method and apparatus capable of improving lane recognition performance.

Until electronic technology became popular, automobiles were controlled only by mechanical principles. However, as information and communication technology develops, automobiles are also being combined with information and communication technologies. Accordingly, research on safety technologies related to automobile surroundings as well as safety technologies for automobiles is increasing.

The automobile safety control system is installed in the recently released automobiles. The automotive safety control system senses the surroundings of the vehicle using sensors mounted on the vehicle and car safety control such as lane keeping, collision avoidance and headlight control based on the detection result.

In the case of lane recognition, a sensor installed in front of the automobile is used. Examples of the sensor include a Vision Sensor, a Radar, and a Lidar. Vision sensors are widely used because they can utilize various image processing algorithms.

However, since the conventional lane recognition method uses one vision sensor, there is a problem in that when the lane in which the vehicle is traveling is merged with another lane, the recognition of the situation such as the crosswalk and the sharp curve is poor.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a lane recognition method and apparatus capable of improving lane recognition performance.

However, the problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems which are not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a lane recognition method including: obtaining distance data of an image of a periphery of a car and an obstacle around the car; Recognizing a sign in the image; Recognizing a peripheral situation of the vehicle based on the recognized contents of the sign; And recognizing a lane from the image based on the recognized surrounding information and the distance data.

According to an aspect of the present invention, there is provided a lane recognizing apparatus including: an image capturing unit for capturing an image of a periphery of a car; A distance sensor for sensing an obstacle around the vehicle and obtaining distance data; A sign recognition unit for recognizing a sign in the image; A situation recognition unit for recognizing the surroundings of the vehicle on the basis of the contents of the recognized sign; And a lane recognition unit for recognizing the lane from the image based on the recognized surrounding information and the distance data.

The method and apparatus for recognizing a lane according to the present invention have the following effects.

By recognizing the signs in the image obtained through the vision sensor and using it to recognize the surrounding situation, it is possible to improve lane recognition performance for situations where lanes are merged, crosswalks, sharp curves, and the like.

1 is a diagram illustrating a configuration of a lane recognizing apparatus according to an embodiment of the present invention.
2 is a diagram illustrating an image in which a region of interest is set.
3 is a view showing the contents of a sign and the surrounding situation of the car classified according to the content of the sign.
4 is a view illustrating an image taken when the vehicle is traveling in the middle of the road and an image taken when the vehicle is traveling on an uphill road.
5 is a view illustrating an image taken when an image taken when the vehicle is traveling in the middle of the road and when the vehicle is traveling in the lane-narrowing section.
6 is a diagram illustrating an image taken when the vehicle is traveling on a road curved rightward.
Fig. 7 is a view showing an image taken when a vehicle is traveling on a confluence road; Fig.
8 is a diagram illustrating an image including a crosswalk.
FIG. 9 is a flowchart illustrating a lane recognition method performed by the lane recognizing device of FIG. 1. FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, like reference numerals designate like elements.

1 is a diagram illustrating a configuration of a lane recognizing apparatus according to an embodiment of the present invention.

1, the lane recognizing apparatus 100 includes an image photographing unit 110, a sign recognizing unit 120, a situation recognizing unit 130, a lane recognizing unit 140, a control unit 150, (Not shown).

The image capturing unit 110 captures an image of a lane in which the vehicle is traveling (hereinafter referred to as 'the present lane') and a forward vehicle traveling in the present lane. The image capturing unit 110 may include a vision sensor. Examples of the vision sensor include a vision sensor capable of acquiring a two-dimensional image, and a three-dimensional vision sensor capable of acquiring a three-dimensional image.

The sign recognition unit 120 may include a region of interest setting unit 122 and a content recognition unit 124. [

The ROI setting unit 122 sets a region of interest (ROI) for detecting a sign in an image captured by the image capturing unit 110. [ The area of interest may be set to include a sign, but to minimize the search range for sign detection. The region of interest can be set based on the Adaboost algorithm. The AdaBoost algorithm is an algorithm that generates a strong classifier with high detection performance by linearly combining sets of weak classifiers. 2 is a view illustrating a state in which a region of interest is set for detecting a sign. Referring to FIG. 2, it can be seen that two areas of interest are set to include signs.

Referring again to FIG. 1, the content recognition unit 124 can recognize the position of the signboard in the image based on the set region of interest, and recognize the content in the signboard. Here, recognizing the content means recognizing at least one of the numbers, letters, symbols, and graphics displayed on the sign. For content recognition, the content recognition unit 124 may use Neural Networks technology.

Neural network is a field of research that aims to realize complex information processing ability that human beings intelligently process through computer by imitating the structure of human brain connected with numerous neurons and information processing method performed in human brain. Neural networks consist of units modeling biological neurons and weighted connections between the units, and have various structures and learning rules according to each neural network model.

The situation recognition unit 130 can recognize the peripheral situation of the vehicle based on the recognition result of the content recognition unit 124. [ The surrounding situation of the vehicle can be classified according to the contents of the sign. For a more detailed description, FIG. 3 will be referred to.

FIG. 3 is a view showing the surroundings of a car classified according to the contents of a sign. Referring to FIG. 3, it can be seen that the surroundings of the car are classified into six categories such as 'lane reduction', 'curved road', 'double curved road', 'joining road', 'crosswalk', and 'tunnel' have.

When the content of the sign recognized by the content recognition unit 124 is a sign indicating the reduction of the left and right two lanes, a sign indicating the reduction of the right one lane, or a sign indicating the reduction of the left one lane, 130) perceives the surroundings of the vehicle as 'lane reduction'.

If the content of the sign recognized by the content recognition unit 124 is an arrow curved to the right or a curved arrow to the left, the situation recognition unit 130 recognizes the surroundings of the car as a 'curved road'.

If the content of the sign recognized by the content recognition unit 124 is a double curved arrow to the right or a double curved arrow to the left and right, the situation recognition unit 130 recognizes the surroundings of the car as a 'double curved road' do.

If the content of the sign recognized by the content recognition unit 124 is a symbol representing a right junction road or a symbol representing a left junction road, the situation recognition unit 130 recognizes the surrounding situation of the vehicle as a & do.

If the content of the sign recognized by the content recognition unit 124 is a sign or a character indicating a crosswalk, the situation recognition unit 130 recognizes the surrounding situation of the vehicle as a 'crosswalk'.

If the content of the sign recognized by the content recognition unit 124 is a symbol representing a tunnel, the situation recognition unit 130 recognizes the circumference of the car as a 'tunnel'.

How the surrounding information described above can be utilized for lane recognition will be described below.

In the case where the surrounding situation information is 'lane reduction', this surrounding situation information can be used to distinguish whether the vanishing point detected in the image meets both lanes of the current lane or a side lane due to lane reduction. Reference will be made to FIGS. 4 and 5 for a detailed description thereof.

FIG. 4A shows an image taken when the vehicle is traveling in the middle of the road, and FIG. 4B shows the image taken when the vehicle is traveling on an uphill road. 5A shows an image taken while the vehicle is traveling in the middle of the road, and FIG. 5B shows an image taken when the vehicle is traveling in a lane-reducing section.

When the two images in FIG. 4 are compared, it can be seen that the point where the two lanes meet when the vehicle is traveling on the uphill road is located below the point where the two lanes meet when the vehicle is traveling in the middle of the road. That is, it can be seen that the red line shown in the image of FIG. 4 (B) is located below the red line shown in the image (A). Similarly, when comparing two images of FIG. 5, it can be seen that the point where the two lanes meet when the vehicle is running in the lane merging section is located below the point where the two lanes meet when the vehicle is traveling in the middle of the road. That is, it can be seen that the red line shown in the image (B) of FIG. 5 is located below the red line shown in the image (A). In this case, by using the vehicle periphery information called " lane reduction ", it is possible to distinguish whether the vehicle is traveling in an uphill or a lane junction. That is, it is possible to distinguish whether the vanishing point detected in the image is the point at which both lanes of the current lane meet, or the point at which it meets the next lane.

If the surrounding information is a 'curved road' or a 'double curved road', this surrounding information can be used to extend the lane recognition range. Reference will be made to Fig. 6 for a more detailed description thereof.

6 shows an image taken when the vehicle is traveling on a road curved rightward. As shown in Fig. 6, the curved road has a curve in which the lane is abrupt. Therefore, there is a high probability of departing from the lane recognition range. Accordingly, when the surrounding information is 'curved road' or 'double curved road', lane recognition errors can be reduced by extending the lane recognition range to left and / or right before performing lane recognition.

If the surrounding situation information is a " confluence road ", this surrounding situation information can be used to set a weight for the inner lane among a plurality of lanes detected in the image. Reference will be made to Fig. 7 for a more detailed description thereof.

7 shows an image taken when the vehicle is traveling on a confluence road. When a lane recognition process is performed on an image as shown in FIG. 7, a plurality of lanes are detected. When a plurality of lanes are detected, a lane recognition error may occur. Therefore, when the surrounding information is the 'confluence road', lane recognition can be reduced by applying the lane recognition only to the inner lane.

If the surrounding situation information is 'crosswalk', this surrounding situation information can be used to remove the crosswalk from the image. For example, as shown in FIG. 8, when a lane recognition process is performed on an image including a pedestrian crossing, a lane recognition error may occur due to the pedestrian crossing. Therefore, when the surrounding situation information is 'crosswalk', if the lane recognition is performed by removing the crosswalk from the image, it is possible to prevent the crosswalk from being recognized as a lane.

If the surrounding situation information is a 'tunnel', the surrounding situation information can be used to recognize the illumination change around the car in advance. Specifically, when the vehicle travels through the tunnel, the lane recognition error may occur because the light around the vehicle suddenly becomes dark. Accordingly, when the surrounding information is a 'tunnel', it is possible to improve the overall brightness of the image and prevent a lane recognition error from occurring.

Referring again to FIG. 1, the surrounding information recognized by the situation recognition unit 130 is provided to the lane recognition unit 140. FIG.

The lane recognition unit 140 performs lane recognition based on the surrounding situation information provided from the situation recognition unit 130. [ The lane recognition process is as follows.

First, the lane recognition unit 140 removes the obstacle from the image based on the distance data obtained by the distance sensor, and sets the area from which the obstacle is removed to the free space. Here, the obstacle refers to an object that causes a lane recognition error, for example, a forward vehicle or a vehicle located in a side lane.

When the free space is set, the lane recognition unit 140 detects an edge in the free space. For this, the lane recognition unit 140 may use a Sobel filter. The Sobel filter is effective for edge detection in a diagonal direction than a vertical edge or a horizontal edge, and is generally defined as a 3 × 3 mask window. However, the size of the mask is not limited to 3 × 3, and may have a size of 5 × 5 or 7 × 7.

Thereafter, the lane recognizing unit 140 detects a linear component from the edge information using a Hough Transform.

Thereafter, the lane recognition unit 140 sets at least one region of interest necessary for lane recognition based on the linear component detection result in the free space. The area of interest necessary for lane recognition is set to include a lane but minimize the image search range for lane recognition. Generally, the lane is located at the bottom of the image, so the region of interest is set on the lower side of the image. Further, since the lane becomes smaller as the distance lengthens, the size of the ROI becomes smaller as the lane becomes smaller. Further, the lane includes the left lane and the right lane, so that the ROI is set for the left lane and the right lane, respectively.

When the area of interest for lane recognition is set, the lane recognition unit 140 extracts the lane in the set area of interest. At this time, the lane extraction may be performed sequentially from the region of interest located at the lower end of the image. That is, a lane is extracted from the left and right ROs located at the bottom of the image, from ROIs of interest and ROIs. Then, a lane is extracted from the ROI located above the ROIs.

When the lane is extracted from the ROIs, the lane recognition unit 140 can detect the vanishing point of the lane. The position of the detected vanishing point may be provided to the controller 150.

The control unit 150 compares the position of the vanishing point detected in the image with the position of the center point of the image. Then, it is determined whether or not the lane departure of the vehicle is based on the comparison result. As a result of the determination, if the vehicle does not deviate from the lane, the lane recognition process is continuously performed. If the vehicle leaves the lane, the warning is given to the driver through the warning unit 160.

9 is a flowchart illustrating a lane recognition method according to an embodiment of the present invention.

First, an image of the periphery of the vehicle is acquired through the image capturing unit 110, and distance data is acquired through the distance sensor (500).

Thereafter, the sign recognition unit 120 recognizes the sign in the image (600). The step of recognizing the sign includes the step of setting a region of interest for detecting the sign in the image taken by the image capturing unit 110, the step of confirming the position of the sign in the set region of interest, . ≪ / RTI >

If the sign is recognized, the situation recognition unit 130 recognizes the surroundings of the vehicle according to the contents of the recognized sign (700). Depending on the contents of the sign, the surrounding situation of the vehicle can be classified into six categories such as lane reduction, curved road, double curved road, confluent road, crosswalk and tunnel.

When the surrounding situation of the vehicle is recognized, the lane recognition unit 140 performs lane recognition based on the recognized surrounding condition information distance data (800). Here, the lane recognition step 800 will be described in more detail.

For example, if the recognized surrounding situation is 'lane reduction', performing the lane recognition may include removing obstacles from the image based on the distance data, setting the area from which the obstacle is removed to free space, Detecting a straight line component from a detected edge; setting a region of interest necessary for lane recognition based on a linear component detection result; extracting a lane in a set region of interest; Detecting a vanishing point based on the detected lane; and recognizing the detected vanishing point as a point where the lane departure causes the lane to collide with the adjacent lane.

As another example, when the recognized surrounding situation is a 'curved road' or a 'double curved road', performing the lane recognition may include extending a lane recognition range, removing an obstacle from the image based on the distance data, Setting an area in which an obstacle is removed to a free space; detecting an edge in the free space; detecting a linear component from the detected edge; setting a region of interest necessary for lane recognition based on the linear component detection result Extracting a lane in the set area of interest, and detecting a vanishing point based on the detected lane.

As another example, if the recognized surrounding situation is a 'confluence road', performing the lane recognition may include removing obstacles from the image based on the distance data, setting the area from which the obstacle has been removed to free space, The method comprising the steps of: detecting an edge in a space; detecting a linear component from the detected edge; setting a region of interest necessary for lane recognition based on a linear component detection result; Extracting a lane from the detected lane, and detecting a vanishing point based on the detected lane.

As another example, when the recognized surrounding situation is 'crosswalk', performing the lane recognition may include removing obstacles from the image based on the distance data, setting the area from which the obstacle has been removed to free space, Setting an area of interest necessary for lane recognition based on a linear component detection result and setting a region of interest such that a pedestrian crossing is excluded; , Extracting a lane in the set region of interest, and detecting a vanishing point based on the detected lane.

As another example, if the recognized surrounding situation is a 'tunnel', performing the lane recognition may include adjusting brightness of the image, removing an obstacle from the image based on the distance data, A step of detecting an edge in the free space, a step of detecting a straight line component from the detected edge, a step of setting a region of interest necessary for lane recognition based on the straight line component detection result, Extracting a lane in the area, and detecting a vanishing point based on the detected lane.

When the lane recognition step is completed, the control unit 150 issues a lane departure warning through the warning unit 160 according to the lane recognition result (900).

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood that the invention may be practiced. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: lane recognition device
110:
120: Sign recognition unit
122: region of interest setting section
124:
130:
140: lane recognition section
150:
160: Warning section

Claims (15)

A step of acquiring distance data of an image taken around the subject vehicle and an obstacle around the subject vehicle;
Recognizing a sign in the image;
Recognizing a peripheral situation of the vehicle based on the recognized contents of the sign; And
And recognizing a lane from the image based on the recognized surrounding information and the distance data. The method according to claim 1,
The step of recognizing the sign
Setting a region of interest for the sign detection; And
Determining a position of the signboard in the image based on the set area of interest, and recognizing contents in the signboard. The method according to claim 1,
If the recognized surrounding situation is lane-narrowing, the step of recognizing the lane
Removing an obstacle from the image based on the distance data, and setting an area in which the obstacle is removed as a free space;
Detecting an edge in the free space;
Detecting a linear component from the detected edge;
Setting a region of interest necessary for lane recognition based on the linear component detection result;
Extracting a lane in the area of interest;
Detecting a vanishing point based on the detected lane; And
And recognizing the detected vanishing point as a point where it meets a side lane. The method according to claim 1,
If the recognized surrounding situation is a curved road or a double curved road, the step of recognizing the lane
Extending a lane recognition range;
Removing an obstacle from the image based on the distance data, and setting an area in which the obstacle is removed as a free space;
Detecting an edge in the free space;
Detecting a linear component from the detected edge;
Setting a region of interest necessary for lane recognition based on the linear component detection result;
Extracting a lane in the area of interest; And
And detecting a vanishing point based on the detected lane. The method according to claim 1,
If the recognized surrounding situation is a confluence road, the step of recognizing the lane
Removing an obstacle from the image based on the distance data, and setting an area in which the obstacle is removed as a free space;
Detecting an edge in the free space;
Detecting a linear component from the detected edge;
Setting a region of interest necessary for lane recognition based on the linear component detection result;
Extracting a lane in the area of interest based on a weight set for each lane; And
And detecting a vanishing point based on the detected lane. The method according to claim 1,
If the recognized surrounding situation is a pedestrian crossing, the step of recognizing the lane
Removing an obstacle from the image based on the distance data, and setting an area in which the obstacle is removed as a free space;
Detecting an edge in the free space;
Detecting a linear component from the detected edge;
Setting a region of interest necessary for lane recognition based on the linear component detection result, and setting a region of interest such that a pedestrian crossing is excluded;
Extracting a lane in the area of interest; And
And detecting a vanishing point based on the detected lane. The method according to claim 1,
If the recognized surrounding situation is a tunnel, the step of recognizing the lane
Adjusting brightness of the image;
Removing an obstacle from the image based on the distance data, and setting an area in which the obstacle is removed as a free space;
Detecting an edge in the free space;
Detecting a linear component from the detected edge;
Setting a region of interest necessary for lane recognition based on the linear component detection result, and setting a region of interest such that a pedestrian crossing is excluded;
Extracting a lane in the area of interest; And
And detecting a vanishing point based on the detected lane. An image capturing unit for capturing an image of the surroundings of the vehicle;
A distance sensor for sensing an obstacle around the vehicle and obtaining distance data;
A sign recognition unit for recognizing a sign in the image;
A situation recognition unit for recognizing the surroundings of the vehicle on the basis of the contents of the recognized sign; And
And a lane recognition unit for recognizing a lane from the image based on the recognized surrounding information and the distance data. 9. The method of claim 8,
The sign recognition section
A region of interest setting unit for setting a region of interest for detecting the sign; And
And a content recognition unit for confirming the position of the signboard in the image based on the set region of interest and recognizing contents in the signboard. 9. The method of claim 8,
The lane recognition unit
Removing an obstacle from the image based on the distance data, setting an area from which the obstacle is removed to a free space,
Detecting an edge in the free space,
Detecting a linear component from the detected edge,
Setting a region of interest necessary for lane recognition based on the linear component detection result,
Extracting a lane in the area of interest,
And detects a vanishing point based on the detected lane. 11. The method of claim 10,
The lane recognition unit
And recognizes the detected vanishing point as a point where it meets a side lane when the recognized surrounding situation is lane reduction. 11. The method of claim 10,
The lane recognition unit
And extends the lane recognition range when the recognized surrounding situation is a curved road or a double curved road. 11. The method of claim 10,
The lane recognition unit
Wherein the lane recognition unit extracts a lane in the ROI based on a weight set for each lane when the lane is extracted from the ROI when the recognized surrounding situation is a confluence road. 11. The method of claim 10,
The lane recognition unit
And sets the area of interest such that the pedestrian crossing is excluded when setting the area of interest when the recognized surrounding situation is a pedestrian crossing. 11. The method of claim 10,
The lane recognition unit
And adjusts the brightness of the image when the recognized surrounding situation is a tunnel.

Images