KR20150138898A - Method and system for warning lane departure using turn sign recognition - Google Patents

Abstract

The present invention is to warn the departure of a vehicle from a driving lane due to the carelessness of a driver by recognizing a guidance sign in the front video of the vehicle and synthesizing driving information such as rate, wheel speed and a steering angle coming from sensors of the vehicle. The method for warning a lane departure using guidance recognition according to the present invention comprises the steps of: (a) obtaining a front video by photographing the front using a camera; (b) recognizing a lane from the obtained front video; (c) recognizing a guidance from the obtained front video; (d) comparing the result of the recognizing the lane with the result of recognizing the guidance to find difference; (e) if the result of recognizing the lane is different from the result of recognizing the guidance, using the result of recognizing the guidance if the confidence level of the lane is smaller than a threshold value and otherwise using the result of recognizing the lane; (f) predicting the proceeding direction of the vehicle by analyzing speed, a steering angle and yawing motion using the obtained front video; (g) determining whether the vehicle drives normally or abnormally, based on the result of recognizing the lane, the result of recognizing the guidance and the result of predicting the proceeding direction of the vehicle; and (h) warning lane departure if the vehicle is determined to drive abnormally.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a warning system

More particularly, the present invention relates to a system and method for warning of lane departure by recognizing a line-of-sight derivative, in which a vehicle traveling on a curved road passing through a side surface of a cliff is set in a corner of a curve (Yaw Rate, Wheel Speed, Steering Angle, etc.) coming from the sensor of the vehicle are recognized, and the vehicle does not travel normally due to the carelessness of the driver And more particularly, to a warning system and method for warning of departure of a lane through recognition of a line-of-sight guidance enabling a driver to warn the driver to depart from the lane.

Drivers are often exposed to accidents due to drowsiness or other lack of attention. In particular, if the driver's attention is distracted on a curved road that passes through the side of a cliff, it can lead to a major accident such as a car crash or rollover.

The Lane Departure Warning System (LDWS) is an Active Driver Assistance System that reduces the risk of accidents by warning these careless drivers that they are leaving the drive lane. A Lane Keeping Assistance System (LKAS) / Lane Keeping System (LKS) is also on the market to help drivers steer clear of the Lane Departure Warning System (LDWS).

The conventional LDWS / LKAS / LKS technology recognizes the lane in the image coming from the camera and compares it with the expected travel route of the vehicle to determine whether or not the driver is in a normal driving state.

Therefore, if the lane is not clearly visible to the image, such as a nighttime situation, a section where dust is accumulated in a lot, a lane is cleared, or other bad weather, the operation of the system is degraded .

Korean Patent Laid-Open Publication No. 10-2013-0067463 (Publication date: June 24, 2014)

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-described drawbacks in that a vehicle running on a curved road passing through a side surface of a cliff, (Yaw Rate, Wheel Speed, Steering Angle, etc.) coming from the sensor of the vehicle are synthesized to recognize that the vehicle does not travel normally and travels away from the driving lane due to carelessness of the driver And to provide a warning system and method for lane departure warning through recognition of a line-of-sight guidance enabling warning.

According to an aspect of the present invention, there is provided an apparatus for detecting a line-of-sight of a system including an image obtaining unit, a lane recognizing unit, a line derivative recognizing unit, a traveling direction predicting unit, The method of claim 1, further comprising: (a) capturing a forward image with a camera to acquire a forward image; (b) recognizing a lane in the obtained forward image; (c) recognizing a line-of-sight derivative in the obtained forward image; (d) comparing whether the lane recognition result is different from the sight line derivative recognition result; (e) if the recognition result of the lane is different from the recognition result of the line-of-sight derivative, the recognition result of the line-of-sight derivative is used if the confidence level of the lane is less than the threshold value, Using the result; (f) analyzing a speed, a steering angle, and a yaw motion using the obtained forward image to predict a traveling direction of the vehicle; (g) determining normal driving or abnormal driving based on the lane recognition result, the sight line derivative recognition result, and the vehicle traveling direction prediction result; And (h) warning the driver of departure from the lane during the abnormal driving as a result of the determination.

The step (b) uses a Hough transform or a cubic curve form, and does not execute the lane recognition operation when the lane is not recognized.

In the step (b), the recognition result of the lane includes an offset, an angle, a curvature and a confidence level of the lane to the lane based on the automobile coordinate system.

Also, in the step (c), the recognition result of the line-of-sight derivative includes a curvature radius according to a speed and a rule.

In the step (c), a normalized histogram Hn of a predetermined size is generated for the obtained forward image, and a normalized gray value of each pixel for the normal histogram is calculated The aspect ratio is calculated using Histogram of Oriented Gradient (HOG) based on the normal gray value. If the calculated aspect ratio is below a certain standard, it is excluded as an outlier, And recognizes the line-of-sight derivative based on an inlier.

The step (c) may be performed by dividing the histogram H of the sample by the number of sample constituent pixels Ns with respect to the normal histogram Hn.

The step (c) may further include a step of performing Nomalization (I (x, y)) on the normal gray value (NGV) of the current pixel I '(x, y) with respect to the normal histogram Hn The value obtained by subtracting the average of the patches is divided by the value twice the variance (2 * sigma), which is then multiplied by 255.

In the step (e), when the lane recognition result does not differ from the sight line derivative recognition result, the lane recognition result is used.

In the step (g), when the radius of curvature of the lane recognized through the lane recognition unit is compared with the radius of curvature according to the aspect ratio of the line of sight recognized through the line-of-sight recognition unit, , And the radius of curvature of the lane is corrected based on the radius of curvature in accordance with the aspect ratio of the line-of-sight derivative.

In the step (g), if the lane recognition fails in the lane recognition result through the lane recognition unit, or if the confidence level of the lane recognition result is lower than the threshold range, If the radius of curvature of the lane is larger than a certain standard despite the recognition of the derivative, the radius of curvature of the lane of the scope of application of the line of sight specified by law shall be applied.

In the step (g), if the curvature of the lane through the lane recognition unit is different from the curvature of the lane through the lane recognition unit, the controller determines that the vehicle is traveling abnormally.

According to another aspect of the present invention, there is provided an image processing apparatus including an image acquiring unit for acquiring a forward image by capturing a forward image with a camera; A lane recognition unit for recognizing a lane in the obtained forward image; A visual line derivative recognizing unit for recognizing a visual line derivative in the obtained front image; A traveling direction predicting unit for predicting a traveling direction of the vehicle by analyzing a speed, a steering angle, and a yaw motion using the obtained forward image; The recognition result of the lane marker is different from the recognition result of the lane marker, and if the confidence level of the lane is less than the threshold value, the recognition result of the line indicator is used and the reliability of the lane marker is less than the threshold value A control unit for using the recognition result of the lane, and controlling the warning to depart from the lane; A normal driving determination unit for determining a normal driving or an abnormal driving based on the lane recognition result, the sight line derivative recognition result, and the vehicle traveling direction prediction result; And a warning unit for outputting a warning sound or displaying a warning according to the control of the control unit.

The recognition result of the lane includes an offset, an angle, a curvature and a confidence level of a lane on the basis of the automobile coordinate system.

The visual line derivative recognizing unit may generate a normalized histogram Hn of a predetermined size for the obtained forward image and calculate a normalized gray value of each pixel with respect to the normal histogram , HOG (Histogram of Oriented Gradient) is used to calculate the Aspect Ratio. If it is below a certain threshold, it is excluded as an outlier, and the sight line derivative is recognized based on the inlier which is above a certain standard.

The visual line derivative recognizing unit calculates the histogram H by dividing the histogram H of the sample by the number of sample constituent pixels Ns for the normal histogram Hn.

The gaze derivative recognizing unit recognizes a nominalization patch at the previous pixel I (x, y) with respect to the normal gray value NGV of the current pixel I '(x, y) with respect to the normal histogram Hn. (2 * sigma), which is then multiplied by 255 to calculate the value.

Further, the control unit uses the lane recognition result when the lane recognition result does not differ from the sight line derivative recognition result.

The control unit compares the radius of curvature of the lane recognized through the lane recognition unit with the radius of curvature according to the aspect ratio of the line of sight recognized through the line of sight recognition unit, The radius of curvature of the lane is corrected based on the curvature radius according to the aspect ratio of the derivative.

If the lane recognition is not successful in the lane recognition result through the lane recognition unit or the confidence level of the lane recognition result is lower than the threshold range or the visual line derivation is recognized through the lane recognition unit, If the radius of curvature of the lane is larger than a certain standard, the radius of curvature of the lane of the scope of application of the line of sight specified by law shall be applied.

If the curvature of the lane through the traveling direction predicting unit and the curvature of the lane through the lane recognizing unit are different within the threshold range, the normal traveling determining unit determines that the vehicle is traveling abnormally.

According to the present invention, a lane is recognized on the basis of a forward image acquired through a camera, and a gaze derivative is recognized, so that a departure of a lane on a curved road can be warned.

Accordingly, it is possible to prevent the vehicle from falling or overturning accident by warning that the vehicle leaves the lane due to carelessness of the driver on the curved road or curved road surface.

1 is a block diagram schematically showing functional blocks of a lane departure warning system according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a method of warning a departure warning through a line-of-sight recognition according to an exemplary embodiment of the present invention.
3 is a diagram illustrating an example of an original image and an NGV image according to an embodiment of the present invention.
FIG. 4 is a graph illustrating an example of an aspect ratio in the case of Width 8 or more according to an embodiment of the present invention.
FIG. 5 is a view showing an example of the configuration of a gull flag signboard according to an embodiment of the present invention.
FIG. 6 is a view showing a range of chromaticity coordinates with respect to hues of a gull sign according to an embodiment of the present invention.
FIG. 7 is a view showing an installation example of a gull sign according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that it is not intended to be limited to the particular embodiments of the invention but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a lane departure warning system and method using a line-of-sight recognition according to the present invention will be described in detail with reference to the accompanying drawings. In the following description with reference to the accompanying drawings, the same or corresponding components are denoted by the same reference numerals, and a redundant description thereof will be omitted.

1 is a block diagram schematically showing functional blocks of a lane departure warning system according to an embodiment of the present invention.

1, a lane departure warning system 100 according to an embodiment of the present invention includes an image obtaining unit 110, a lane recognizing unit 120, a line derivative recognizing unit 130, a traveling direction predicting unit 140 A normal driving determination unit 150, a storage unit 160, a control unit 170, and a warning unit 180.

The image acquiring unit 110 acquires a forward image by taking a forward image with a camera.

The lane recognition unit 120 recognizes a lane in the acquired forward image. Here, the recognition result of the lane includes an offset, an angle, a curvature and a confidence level of the lane to the lane based on the automobile coordinate system.

The line-of-sight derivation recognizing unit 130 recognizes the line-of-sight derivative in the obtained forward image. That is, the gaze derivative recognizing unit 130 generates a normalized histogram Hn of a predetermined size for the obtained forward image, and calculates a normalized gray value of each pixel with respect to the normal histogram The aspect ratio is calculated using a Histogram of Oriented Gradient (HOG). If it is below a certain threshold, it is excluded as an outlier, and the line-of-sight derivative is recognized based on an inlier that is above a certain standard.

The visual line derivative recognizing unit 130 calculates the histogram H by dividing the histogram H of the sample by the number of sample constituent pixels Ns with respect to the normal histogram Hn.

The visual line derivative recognizing unit 130 recognizes the normal gray value NGV of the current pixel I '(x, y) with respect to the normal histogram Hn at the previous pixel I (x, y) The value obtained by subtracting the average of the nominalization patch is divided by the value twice the variance (2 * sigma), which is then multiplied by 255.

The traveling direction prediction unit 140 predicts the traveling direction of the vehicle by analyzing the velocity, the steering angle, and the yaw motion using the obtained forward image.

The normal travel determiner 150 determines normal travel or abnormal travel based on the lane recognition result, the sight line derivative recognition result, and the vehicle travel direction prediction result.

The storage unit 160 stores a forward image acquired through the image acquisition unit 110 or a warning sound for alerting the vehicle to a lane departure.

If the lane recognition result is different from the recognition result of the sight line derivative, the control unit 170 uses the sight line derivative recognition result if the confidence level of the lane is less than the threshold value, If the reliability is greater than the threshold value, the recognition result of the lane is used.

The warning unit 180 outputs a warning sound or displays a warning according to the control of the control unit.

Here, when the lane recognition result does not differ from the sight line derivative recognition result, the control unit 170 uses the lane recognition result.

The control unit 170 compares the curvature of the lane recognized by the lane recognition unit with the curvature of the line-of-sight derivative recognized through the line-of-sight line recognition unit, and when there is a difference of more than a certain standard, The curvature of the lane is corrected based on the curvature according to the ratio.

If the lane recognition fails in the lane recognition result through the lane recognition unit or the confidence level of the lane recognition result is lower than the threshold range or the visual line derivation is recognized through the visual line recognition unit, If the curvature of the lane is larger than a certain standard, the curvature of the lane of the scope of the line of sight specified by the law is applied.

If the curvature of the lane through the traveling direction predicting unit and the curvature of the lane through the lane recognizing unit are different within the threshold range, the normal travel determining unit 150 determines that the vehicle is traveling abnormally.

FIG. 2 is a flowchart illustrating a method of warning a departure warning through a line-of-sight recognition according to an exemplary embodiment of the present invention.

Referring to FIG. 2, a lane departure warning system 100 according to an embodiment of the present invention captures a forward image by using a camera (S210).

That is, the image acquiring unit 110 acquires the forward image by photographing the forward view through the forward camera installed at a position where the forward of the vehicle can be photographed. Therefore, the forward image includes the image of the road and a lane marked on the road.

Next, the lane recognition unit 120 recognizes the lane in the acquired forward image (S220). That is, the lane recognition unit 120 performs the same functions as an active driver assistance system such as a commonly known LDWS (Lane Departure Warning System), LKAS (Lane Keeping Assistances System), LKS (Lane Keeping System) It is possible to recognize the lane in the forward image.

At this time, the lane recognition unit 120 may use a Hough transform or a cubic curve form, and may not execute the lane recognition operation when the lane is not recognized. The lane recognition results include an offset to the lane, an angle, a curvature and a confidence level of the lane based on the automobile coordinate system.

Then, the line-of-sight derivative recognizing unit 130 recognizes the line-of-sight derivative in the obtained front image (S230).

That is, the line-of-sight derivation unit 130 generates a normalized histogram Hn of a predetermined size with respect to the obtained forward image. At this time, the visual line derivative recognizer 130 calculates the histogram H of the sample by dividing the sample histogram Hn by the sample constituent pixel count Ns according to the following equation (1).

Here, H denotes a histogram of a sample, and Ns denotes a number of pixels constituting a sample.

The recognition result of the line-of-sight derivative includes the velocity and the curvature radius according to the law.

The visual line derivative recognizing unit 130 calculates the normalized gray value of each pixel with respect to the normal histogram. At this time, the visual line derivative recognizing unit 130 recognizes the normal gray value (NGV) of the current pixel I '(x, y) with respect to the normal histogram Hn according to the following equation (2) , y)) is calculated by dividing the value obtained by subtracting the average of the nominalization patch by the value twice the variance (2 * sigma) and then multiplying by 255.

Where μ denotes the average of the nominalization patches, ie the average of the Crop region, and σ denotes the variance of the Crop region.

In case of normal gray value (NGV), NGV value is calculated by normalizing blob with width 8 or more to 8 ㅧ 8, 16 ㅧ 16 according to size. Since the visual line derivative recognizing unit 130 uses the Integral Image on the HBA (High Beam Assistance) algorithm, the mean and standard deviation of the crooked region can be quickly calculated using the NGV. Therefore, the gaze derivative recognizing unit 130 can obtain the right NGV image with respect to the left original image as shown in FIG. 3 is a diagram illustrating an example of an original image and an NGV image according to an embodiment of the present invention.

The visual line derivative recognizing unit 130 calculates the aspect ratio using the Histogram of Oriented Gradient based on the normal gray value. That is, the visual line derivative recognizing unit 130 obtains the Blob aspect ratio as shown in FIG. 4 using HOG based on the normal gray value. FIG. 4 is a graph illustrating an aspect ratio when the width is 8 or more according to an embodiment of the present invention. In FIG. 4, in case of Width 8 or more, the aspect ratio is 0.7474 and the standard deviation is 0.2236. Aspect Ratio is 1.2 in the case of the sight line derivative or gull flag sign, and 0.74 in the case of the light source / reflector. HOG is a method of extracting the local gradient (edge direction) distribution (histogram) characteristic of the image and identifying the object.

In addition, the sight line derivative recognizing unit 130 normalizes the size of the normalized histogram gull flag with a size of 8 ㅧ 8 and 16 ㅧ 16 in consideration of the aspect ratio of the two signs, and the width is 8 to 15 Normalization is performed at 8, and when it is 16 or more, normalization is performed at 16.

Then, the line-of-sight derivation unit 130 recognizes the line-of-sight derivative based on an inlier that is not less than a predetermined standard, if the computed aspect ratio is less than a predetermined standard. That is, as shown in FIG. 3, when the aspect ratio is 2.0 or more as shown in FIG. 3, the line-of-sight recognition unit 130 regards the outline as an outlier and excludes a total of 8 generated outliers.

Here, SVM (Support Vector Machine) can be used when classifying into Inlier and Outlier classes. Two groups can be distinguished by measuring the distance between each data in two groups, finding the two centers, and finding the optimal pixel among them.

When HOG is used, 16 ㅧ 16 has more than 95% G / E / M recognition rate, less than 5% false recognition performance, and 8 ㅧ 8 has similar performance to NGV. Seagull signs of 16 pixels or more can distinguish seagulls and light sources when HOG is applied, and when NGV and HOG are applied in case of 8 pixels or more and 15 pixels or less.

In the case of a gull sign that is a sight line derivative, a sign of a gull sign is installed in a poorly sighted area such as an urgent plane curved line as a road appendage of the Article 2 of the Road Traffic Act, so that the driver can clearly know the linearity and bend It is a sight-inducing facility that promotes safe driving.

Seagull signs are installed in areas where the driver is given special emphasis on providing prior information about the road situation, such as the area where the plane linearity of the road changes rapidly.

The need for distinction between the sight guidance mark and the installation location of the seagull sign is that the driver recognizes the linearity of the road by the sight guidance mark installed on the left and right sides of the road in the case of the urgent plane curved portion . For this reason, it is possible to reduce the driver 's confusion by placing the gull mark on the inside of the road in the case of the urgent flat curved line inside the road.

As shown in FIG. 5, the gull sign consists of a gull, a sign, and a post. FIG. 5 is a view showing an example of the configuration of a gull flag signboard according to an embodiment of the present invention. The standard size of the plate is 45 cm width and 60 cm length. The number of signs of seagulls shall be one. The standard size can be reduced in places where standard specifications can not be easily installed by road structures such as median separators and bridges. Seagull signs used in the construction section can be adjusted according to the overall safety installation plan considering the situation of roads and traffic. It shall be of two-sided type on the secondary road and shall have a sectional type on the four-lane or more road separated by the median separator.

In FIG. 5, the seagulls symbol indicates a gull-shaped cross symbol indicating to the driver that the road section in front of the road section is curved and the road is curved. The sign is a plate which is attached to the pillar by attaching a seagull symbol to the sign to increase visibility and readability. Therefore, the signs should be able to securely affix seagull emblems and be firmly assembled to the support. The landing is to fix the sign in the required position. Therefore, the gull sign should not be moved due to the slight external pressure. The background of the seagull cover is yellow, and the breaking mark is black. The chromaticity of the reflector should fall within the range of chromaticity coordinates as shown in FIG. 6 when measured according to the chromaticity measuring method. FIG. 6 is a view showing a range of chromaticity coordinates with respect to hues of a gull sign according to an embodiment of the present invention.

The installation position of the gull sign is located near the outside of the roadway facility limit. Generally, it should be installed to the terrain at 0 ~ 200 Cm from the shoulder edge. Mounting height is 120 Cm from the road surface to the bottom of the sign.

As shown in FIG. 7, the seagull signs are continuously installed in a curved section to provide a smooth visual guidance effect, and installation intervals according to the curve radius of the road are as shown in Table 1 below.

Curve radius Installation Interval Curve radius Installation Interval Less than 50 8 246 to 320 25 51 ~ 80 12 321 to 405 30 81 ~ 125 15 406-500 35 126-180 20 501 to 650 38 181 to 245 22 651 to 900 45

Seagull covers must be visible to the driver at least 150 m in front of them, and two or more signs should be visible in the driver's field of view. In a highway or highway that has a high driving speed, a gull sign is additionally installed at a position twice as long as the interval between the curved portions in the entrance transition section. FIG. 7 is a view showing an installation example of a gull sign according to an embodiment of the present invention.

The angle of installation should be well taken into account for the visibility of the sign and the direction of the car, and should be installed at a right angle or within 10 에 of the direction of travel. At night, a proper angle should be set so that the reflected light from the surface of the cover is placed on the operator's line of sight by the automotive headlight so that momentary glare does not occur.

Table 2 below shows the minimum curve radius for the gull-cover sign.

Design speed (km / h) Minimum Curve Radius (m) Seagull Cover Application Curve Radius (m) 120 710 770 110 600 650 100 460 550 90 380 420 80 280 340 70 200 250 60 140 180 50 90 120 40 60 80 30 30 45

The standard installation spacing S along the curve radius of the road is given by the following equation (3).

Here, S denotes the installation interval (m), and R denotes the radius of curvature (m).

Next, the control unit 170 compares the recognition result of the lane line with the recognition result of the sight line derivative (S240).

If the reliability of the lane is less than the threshold value (YES in S252), the control unit 170 determines whether the recognition result of the sight line derivative is (S254). If the reliability of the lane is greater than the threshold value (S252-No), the recognition result of the lane is used (S256).

Further, the control unit 170 uses the lane recognition result (S256) even when the lane recognition result is not different from the sight line derivative recognition result (S250-No).

The traveling direction predicting unit 140 analyzes the velocity, the steering angle, and the yaw motion using the obtained forward image to predict the traveling direction of the vehicle (S260).

Then, the normal travel determiner 150 determines normal travel or abnormal travel based on the lane recognition result, the sight line derivative recognition result, and the vehicle travel direction prediction result (S270).

That is, the normal driving determination unit 150 estimates the lane curvature of the front road according to the predicted traveling direction of the vehicle, the recognition result of the lane, and the recognition result of the sight line derivative, and determines the normal driving or the abnormal driving.

The normal driving determination unit 150 determines whether the lane recognition is unsuccessful in the lane recognition result through the lane recognition unit or the confidence level of the lane recognition result is lower than the threshold range, The curvature of the lane of the line-of-sight range covered by the law is applied as shown in Tables 1 and 2 when the curvature of the lane is greater than a certain standard.

In addition, the normal travel determiner 150 determines that the vehicle is traveling abnormally when the curvature of the lane through the traveling direction predicting unit and the curvature of the lane through the lane recognizing unit are different within a threshold range.

The control unit 170 alerts the driver to departure of the vehicle through the warning unit 180 during the abnormal driving (S280).

That is, the warning unit 180 alerts the driver of abnormal driving by outputting an alarm or by displaying an abnormal driving on a screen that the driver can see.

Therefore, it is possible to prevent a safety accident by preventing the vehicle from traveling in the driving lane.

The control unit 170 compares the curvature of the lane recognized by the lane recognition unit with the curvature of the line of sight recognized through the line-of-sight recognition unit, and if the difference is greater than a predetermined reference, the aspect ratio of the line- The curvature of the lane is corrected based on the curvature of the lane.

As described above, according to the present invention, when a vehicle running on a curved road passing through a side surface of a cliff faces a crooked (" or ") gaze standing on a corner of a curved line If the sign of the derivative is recognized, the driver can be alerted that the vehicle does not travel normally and travels away from the driving lane due to the carelessness of the driver by integrating driving information (Yaw Rate, Wheel Speed, Steering Angle, etc.) It is possible to realize the lane departure warning system and method through the recognition of the line-of-sight derivative.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents. Only. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

When a vehicle traveling on a curved road passing through a side surface of a cliff recognizes a line-shaped indicator sign of a crooked shape (>> or <<) standing on a corner of a curve from an image obtained by photographing the camera through the front side, A vehicle lane recognition system that collects driving information (rate, wheel speed, steering angle, etc.) coming from a sensor of a vehicle to alert the driver that the vehicle does not normally run and depart from the driving lane due to carelessness Can be applied to a departure warning system and method.

100: lane departure warning system 110:
120: lane recognition unit 130: visual line derivative recognition unit
140: traveling direction predicting unit 150: normal traveling determining unit
160: Storage unit 170: Control unit
180: Warning part

Claims (20)

1. A lane departure warning method for recognizing a line derivative of a system including an image acquiring section, a lane recognizing section, a line derivative recognizing section, a traveling direction predicting section, a normal travel judging section, and a warning section,
(a) capturing a forward image by capturing a forward image with a camera;
(b) recognizing a lane in the obtained forward image;
(c) recognizing a line-of-sight derivative in the obtained forward image;
(d) comparing whether the lane recognition result is different from the sight line derivative recognition result;
(e) if the recognition result of the lane is different from the recognition result of the line-of-sight derivative, the recognition result of the line-of-sight derivative is used if the confidence level of the lane is less than the threshold value, Using the result;
(f) analyzing a speed, a steering angle, and a yaw motion using the obtained forward image to predict a traveling direction of the vehicle;
(g) determining normal driving or abnormal driving based on the lane recognition result, the sight line derivative recognition result, and the vehicle traveling direction prediction result; And
(h) warning a departure from the lane during abnormal driving as a result of the determination;
Wherein the vehicle is a vehicle.
The method according to claim 1,
Wherein the step (b) uses a Hough transform or a cubic curve form, and does not execute the lane recognition operation when the lane is not recognized.
The method according to claim 1,
Wherein the recognition result of the lane in the step (b) includes an offset, an angle, a curvature, and a confidence level of a lane to a lane on the basis of a vehicle coordinate system. A method of warning of departure by lane through recognition of a derivative.
The method according to claim 1,
Wherein the recognition result of the line-of-sight derivative in the step (c) includes a curvature radius according to a speed and a rule.
The method according to claim 1,
In the step (c), a normalized histogram Hn of a predetermined size is generated for the obtained forward image, a normalized gray value of each pixel of the normal histogram is calculated , The Aspect Ratio is calculated using Histogram of Oriented Gradient (HOG) based on the normal gray value. If the calculated aspect ratio is below a certain standard, it is excluded as an outlier, Inlier, the line-of-sight derivative is recognized based on the line-of-sight recognition.
The method of claim 5,
Wherein the step (c) is performed by dividing the histogram (H) of the sample by the number of sample constituent pixels (Ns) for the normal histogram (Hn).
The method of claim 5,
Wherein the step (c) comprises: comparing the normal gray value (NGV) of the current pixel (I '(x, y)) with the normal histogram (Hn) Dividing the value obtained by subtracting the average value by a value twice the variance (2 * sigma), and multiplying the value by 255, thereby calculating the line departure warning through visual line derivative recognition.
The method according to claim 1,
Wherein the recognition result of the lane is used when the lane recognition result and the sight line derivative recognition result do not differ in the step (e).
The method according to claim 1,
Wherein the step (g) comprises comparing the radius of curvature of the lane recognized by the lane recognition unit with the radius of curvature of the line of sight recognized by the line-of-sight recognition unit, And correcting the radius of curvature of the lane based on the curvature radius according to the aspect ratio of the line-of-sight derivative.
The method according to claim 1,
The step (g) may include a step of determining whether the lane recognition has failed in the lane recognition result through the lane recognition unit or the confidence level of the lane recognition result is lower than the threshold range, When the radius of curvature of the lane is greater than a predetermined standard, the radius of curvature of the lane of the lane of the scope of application of the line of sight specified by the law is applied.
The method according to claim 1,
Wherein the step (g) determines that the vehicle is in an abnormal driving when the curvature of the lane through the lane recognition unit is different from the curvature of the lane through the lane recognition unit, Leak warning method.
An image acquiring unit for acquiring a forward image by capturing a forward image with a camera;
A lane recognition unit for recognizing a lane in the obtained forward image;
A visual line derivative recognizing unit for recognizing a visual line derivative in the obtained front image;
A traveling direction predicting unit for predicting a traveling direction of the vehicle by analyzing a speed, a steering angle, and a yaw motion using the obtained forward image;
The recognition result of the lane marker is different from the recognition result of the lane marker, and if the confidence level of the lane is less than the threshold value, the recognition result of the line indicator is used and the reliability of the lane marker is less than the threshold value A control unit for using the recognition result of the lane, and controlling the warning to depart from the lane;
A normal driving determination unit for determining a normal driving or an abnormal driving based on the lane recognition result, the sight line derivative recognition result, and the vehicle traveling direction prediction result; And
An alarm unit for outputting a warning sound or displaying a warning according to the control of the control unit;
Of the vehicle is detected.
The method of claim 12,
Wherein the recognition result of the lane includes an offset, an angle, a curvature and a confidence level of the lane to the lane on the basis of the automobile coordinate system. Exit warning system.
The method of claim 12,
The gaze derivative recognizing unit generates a normalized histogram Hn of a predetermined size for the obtained forward image and calculates a normalized gray value of each pixel with respect to the normal histogram, Wherein the aspect ratio is calculated using a histogram of an orientated gradient and is excluded as an outlier if it is less than a predetermined standard and the visual line derivative is recognized based on an inlier of a predetermined standard or more, Lane Departure Warning System through Derivative Recognition.
15. The method of claim 14,
Wherein the visual line derivative recognizing section calculates the histogram derivative H of the sample by dividing the histogram H of the sample by the number of sample constituent pixels Ns with respect to the normal histogram Hn.
15. The method of claim 14,
The visual line derivative recognizing unit recognizes an average of the nominalization patch in the previous pixel I (x, y) with respect to the normal gray value NGV of the current pixel I '(x, y) with respect to the normal histogram Hn (2 * sigma), and multiplying this value by 255. The system according to claim 1, wherein the difference is calculated by dividing the value by 2 times the variance (2 * sigma).
The method of claim 12,
Wherein the control unit uses the lane recognition result when the lane recognition result does not differ from the sight line derivative recognition result.
The method of claim 12,
Wherein the controller compares the radius of curvature of the lane recognized by the lane recognition unit with the radius of curvature of the line of sight derivative recognized through the line of sight recognition unit, And corrects the radius of curvature of the lane based on the radius of curvature in accordance with the aspect ratio.
The method of claim 12,
Wherein the control unit determines that the lane recognition is unsuccessful in the lane recognition result through the lane recognition unit or that the confidence level of the lane recognition result is lower than the threshold range or the visual line derivation is recognized through the visual line recognition unit And when the radius of curvature of the lane is larger than a predetermined standard, the radius of curvature of the lane of the lane of the scope of application of the line of sight specified by the law is applied.
The method of claim 12,
Wherein the normal driving determination unit determines that the vehicle is in an abnormal driving when the curvature of the lane through the lane recognition unit is different from the curvature of the lane through the lane recognition unit. Exit warning system.

Images