KR20170104089A

KR20170104089A - Real time Lane Departure Warning Method and Warning System for Vehicle with Improving Processing Speed

Info

Publication number: KR20170104089A
Application number: KR1020160026457A
Authority: KR
Inventors: 박주홍; 박효진
Original assignee: 아이디어주식회사
Priority date: 2016-03-04
Filing date: 2016-03-04
Publication date: 2017-09-14
Also published as: KR101805883B1

Abstract

The present invention relates to a real-time lane departure warning system and an alarm method with improved processing speed, and a real-time lane departure warning method with improved processing speed according to the present invention, Claims [1] A lane departure warning method for warning a lane departure warning to a driver, comprising: receiving road image information formed of an RGB or YUV image format as raw image data without additional operation; Generating an image for image processing for real-time lane recognition of a region of interest for detecting lanes by preprocessing road image information; Removing noises of the generated image for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm; Adjusting, in real time, the position value of the ROI extracted from the road image information and the lane width value referred to for generating the warning signal using the driving lane data; And generating a lane departure warning signal when the detected driving lane data satisfies a warning generating condition.

Description

TECHNICAL FIELD [0001] The present invention relates to a real-time lane departure warning method and an alarm system,

The present invention relates to a real-time lane departure warning method and alarm system that improves the processing speed, and more particularly, to a real time lane departure warning method and alarm system that improves the processing speed by correcting the position of a region of interest extracted from a road image taken by a camera installed in a vehicle, And more particularly, to a lane departure warning method and an alarm system in which a processing speed is improved so that a departure can be warned.

Recently, a black box, which is used as an event data recorder (EDR) for an automobile, is mounted on a vehicle in order to find an exact cause of a damage in the event of a traffic accident.

Such a vehicle black box can record the accident before and after a collision in case of a traffic accident, provide information necessary for the identification of the accident situation, and has a convenient function for real life .

On the other hand, a driver assist system (DAS) such as a lane departure warning system (LDWS) that warns a lane departure warning such that a driver can operate without leaving a lane in a vehicle black box as IT technology rapidly develops. Driver Assistance Systems).

Currently, most of the technologies used in lane departure warning systems are combined with sensor-based technologies such as position-based technologies such as GPS (Global Positioning System) and driving sensors to the image information on the front of the vehicle. However, And it is difficult to apply to various road situations. In addition, it requires various data besides image data, and therefore, the amount of data processing is increased, so that it is difficult to apply it to a black box using low-end hardware.

In order to solve these problems, rather than searching for lanes in all areas of the image data collected by the camera, it is possible to search the lane only in a specific area of interest by setting the area of interest in the collected image data, Interest is growing.

On the other hand, among technologies related to a lane departure warning system for finding a conventional lane, a technique using a Hough Transform is simpler than a method using a histogram or using edge connection information, The number of pixels to be processed increases, so that a lot of processing time is required. Since the lane is separated from the entire original image by using a complicated operation, the amount of computation is excessive Unnecessary information is also included in the lane image, resulting in an increase in the overall operation speed.

Accordingly, it is possible to extract only the region of interest from the image data collected through the camera to increase the efficiency in the processing process of the processor, and at the same time, A realistic lane departure warning system and a lane departure warning method which improve the processing speed so as to be able to warn a departure are required to be practically applicable.

Published Japanese Patent Application No. 10-2013-0077074 (published on July 30, 2013) Patent Registration No. KR 10-149956 (Published date 2015.03.06)

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a navigation system capable of alerting a user using a vehicle black box having low- A lane departure warning method, and an alarm system.

The lane departure warning system according to an embodiment of the present invention is a lane departure warning system that alerts a driver to a lane departure when a lane departure occurs by using image information of a camera mounted on the vehicle, An image information input unit receiving road image information through the image information input unit; A preprocessing unit for preprocessing road image information input to the image information input unit to generate an image for image processing for real time lane recognition of a region of interest for detecting a lane; An image processing unit for removing noises of an image for image processing generated through the preprocessing unit for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm; A warning signal unit for generating a lane departure warning signal when the driving lane data detected by the image processing unit satisfies a warning generating condition; And a calibration unit for adjusting the position value of the ROI extracted by the preprocessing unit for image processing and the lane width value transmitted to the warning signal unit using the driving lane data detected by the image processing unit.

In addition, the image information input unit may receive the image information formed by any one of the RGB format and the YUV format as raw image data without additional operation, and may transmit the raw image data to the image processing preprocessing unit.

In addition, the image processing preprocessing unit may include a ROI image extracting unit for extracting a panoramic ROI image corresponding to a region of interest for detecting a lane from the road image information, the horizontal length of which is relatively longer than the vertical length; An ROI size reconstructing unit for reconstructing an ROI image of a ROI extracted through the ROI image extractor to a specific size corresponding to a minimum size capable of lane recognition; And a black-and-white image generating unit for generating a black-and-white image so that an image can be processed from the resized ROI image through the ROI size resizing unit.

The ROI image extracting unit may extract the ROI image of the panoramic shape from the central portion of the lower side of the image screen constituting the road image information, and then extract the ROI image corresponding to the position value of the ROI transmitted from the calibration unit Can be changed in real time.

The ROI size resizing unit may resize the panoramic ROI image into a rectangle ROI image whose length is fixed to a reference value and the width is entirely reduced to be relatively shorter than the vertical length have.

In addition, the re-adjusted ROI image may be formed in a size of 80 × 120 pixels.

If the image information input through the image information input unit is a YUV image format having a Y value as a luminance signal and a U value and a V value as color signals, the monochrome image generating unit extracts only a Y value as a luminance signal If the image information input through the image information input unit is an RGB image format, a black and white image may be generated using the following equation.

(Where r represents the R value of the RGB system and g represents the G value).

The image processing unit may include: a Gaussian blur image generating unit for removing noise of an image for image processing generated through the preprocessing unit for image processing; A candidate lane extracting unit for extracting a candidate lane using a horizontal gradient in the Gaussian blur image generated by the Gaussian blur image generating unit; And a driving lane data processor for detecting driving lane data from the candidate lane using the RANSAC algorithm and stably tracking the driving lane and setting a lane search area for applying the RANSAC algorithm.

The candidate lane extracting unit may include a horizontal gradient processing unit for generating a horizontal gradient image using the following equation and extracting a candidate lane by using a relationship between a minimum value and a maximum value of a horizontal gradient value; A candidate lane forming unit for forming a candidate lane having a minimum thickness by leaving a median value for the candidate lane extracted by the horizontal gradient processing unit and removing the median value; And a lane departure section for eliminating a candidate lane formed by the lane departure section when there is no change with time.

(Where Gv = horizontal gradient image, I (x, y) = x, y coordinate input image value)

If the difference between the minimum value and the maximum value of the horizontal gradient values constituting the generated horizontal gradient image is equal to or greater than a predetermined value and the difference between the minimum value and the maximum value of the horizontal gradient value is equal to or less than a predetermined distance, It can be judged as a candidate lane.

The driving lane data processing unit may further include: a search area setting unit for setting an initial search area for the candidate lane; A driving lane detecting unit for detecting driving lane data by finding points forming lanes by using the RANSAC algorithm in the search area and connecting respective points by a straight line; A lane-tracking unit for stably tracking the driving lane comprising the detected driving lane data using a Kalman filter; And a search area resetting unit for resetting the search area for the candidate lane in accordance with the result of the tracking by the lane-finding unit.

The search area setting unit may set the search width W of the reference line corresponding to the initial search area, the slope θ of the reference line, and the distance rho between the reference line and the reference line using the following equation.

(here,

,

)

The driving lane detecting unit may detect a driving lane that forms a single straight line using randomly sampled data corresponding to lanes in the search range of the candidate lane.

Also, the search area re-setting unit can regard the search area as an initial search area if the width (W) value of the search area is equal to the initial value for a predetermined time, and can reset the search area to the initial search area.

When the lane-tracking by the lane-tracking unit succeeds for a predetermined time or longer, the warning signal unit may be set to a lane-detecting unit when the x-coordinate of the tracking lane approaches a position set to a threshold value with respect to the lane- The lane departure warning signal can be generated when the x coordinate of the detected lane is more than the position set as the threshold value of the lane in the ROI.

The calibration unit may further include: an ROI position adjustment unit that generates a variation value for adjusting a position value of a ROI using the driving lane data detected by the image processing unit; A lane width adjusting unit that estimates a lane width value using the driving lane data detected by the image processing unit and generates a variation value for adjusting the lane width transmitted to the warning signal unit; And a data stabilization unit that stabilizes the variation value data generated by the ROI position adjustment unit and the lane width adjustment unit using a Kalman filter or an average value.

A lane departure warning method for improving a processing speed according to an embodiment of the present invention is a lane departure warning method for warning a lane departure to a driver in lane departure by using image information of a camera mounted on the vehicle, Receiving road image information formed of any one of RGB and YUV image formats as raw image data without additional operation; Generating an image for image processing for real-time lane recognition of a region of interest for detecting lanes by preprocessing road image information; Removing noises of the generated image for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm; And generating a lane departure warning signal when the detected driving lane data satisfies a warning generating condition.

In addition, the real-time lane departure warning method of improving the processing speed according to the embodiment of the present invention is characterized in that the lane departure warning method further comprises: And adjusting the value in real time.

The step of generating an image for image processing for real-time lane recognition of a region of interest for detecting a lane by preprocessing the road image information may include generating an image for lane detection based on a lower center of the image screen constituting the road image information, Extracting an ROI image corresponding to a region of interest; Rearranging the ROI image of the ROI to a specific size corresponding to a minimum size capable of lane recognition; And generating a monochrome image from the re-adjusted ROI image.

In addition, in the real-time lane departure warning method with improved processing speed according to the embodiment of the present invention, when the position value of the ROI is adjusted using the driving lane data, the ROI image is extracted from the image screen constituting the road image information And changing the position in real time.

The step of rearranging the ROI image of the ROI to a specific size corresponding to the minimum size capable of recognizing the lane may include fixing the ROI image of the panoramic form to a reference value with the vertical length being fixed to a reference value, The size of the ROI image may be reduced.

The step of generating a black-and-white image from the re-adjusted ROI image may include: generating a black-and-white image from the re-adjusted ROI image by using a Y value as a luminance signal when the input image information is a YUV- If the image information input through the image information input unit is an RGB image format, a process of generating a monochrome image (gray) using the following equation is performed .

In addition, the step of removing the noise of the generated image processing image, extracting the candidate lane, and detecting the driving lane data using the RANSAC algorithm may include: generating a Gaussian blur image for noise removal of the image for image processing; Extracting a candidate lane using a horizontal gradient in a Gaussian blur image; Setting an initial search area for the candidate lane; Detecting driving lane data by finding points forming lanes using the RANSAC algorithm in the search area and connecting the points to each other by a straight line; Stably tracking a driving lane comprising the detected driving lane data using a Kalman filter; And resetting the search area for the candidate lane in response to the tracking result.

The step of extracting a candidate lane using the horizontal gradient in the Gaussian blur image may include: Generating a horizontal gradient image using the following equation and extracting a candidate lane using the relationship between the minimum value and the maximum value of the horizontal gradient value; Forming a candidate lane of minimum thickness by removing a median value for the extracted candidate lane; And removing the candidate lane when there is no change with passage of time.

The step of extracting the candidate lane may include a case where the difference between the minimum value and the maximum value of the horizontal gradient value constituting the generated horizontal gradient image is greater than or equal to a predetermined value and the case where the distance difference between the minimum value and the maximum value of the horizontal gradient value is constant If the distance is less than the predetermined distance, it is determined to be the candidate lane.

Also, the step of setting the initial search area for the candidate lane may include calculating the search width (W) of the reference line corresponding to the initial search area for the candidate lane, the slope (?) Of the reference line, And setting the distance rho.

(here,

,

)

The step of detecting driving lane data by finding a lane forming point by using the RANSAC algorithm in the search area and connecting each of the points by a straight line is characterized by comprising the steps of: Detecting a driving lane that forms a single straight line using the sampled data.

The step of resetting the search area for the candidate lane in response to the tracking result may include determining that the search area is an initial search area if the W value of the search area is equal to the initial value for a predetermined time, And resetting.

The step of generating the lane departure warning signal when the detected lane departure data satisfies the warning generation condition may include a step of, when the lane tracking by the lane departure unit succeeds for a predetermined time or longer, And when the x coordinate of the lane detected by the lane detecting unit approaches the position set by the threshold value of the lane in the area of interest, if the lane departure warning signal . &Lt; / RTI >

The step of adjusting, in real time, the position value of the ROI extracted from the road image information using the driving lane data and the lane width value referred to for generating the warning signal, Generating a variation value for adjusting a position value; Estimating a lane width value using the driving lane data detected by the image processing unit and generating a variation value for adjusting the lane width transmitted to the warning signal unit; And stabilizing the variation value data generated by using the Kalman filter and the average value.

As described above, according to the present invention, the position of a region of interest extracted from a road image photographed by a camera is corrected, so that a user who uses a vehicle black box having low- The lane departure warning system according to the present invention provides a real time lane departure warning system.

Further, the present invention has an effect of improving the processing speed roll by rearranging the size of the ROI image corresponding to the ROI in the black box to the minimum size capable of recognizing the lane in the case of generating the image for image processing.

In the present invention, an image having a panoramic shape in a wide area is extracted from the road image information so that a lane can be detected when the size of the ROI image is readjusted. Then, in the extracted image, There is an effect of improving the lane detection efficiency while improving the processing speed.

In the present invention, when a candidate lane is extracted from the image processing unit, a horizontal gradient image is generated, and a candidate lane is extracted using the relationship between the minimum value and the maximum value. Then, a candidate lane is formed using only the median value, It has the effect of improving the processing speed.

Further, in the present invention, when driving lane data detected by the image processing unit is detected, a lane detecting unit detects a lane forming point by using the RANSAC algorithm, and connects each point by a single straight line to detect a lane among a plurality of lines It is possible to reduce the size of data to be processed, thereby improving the processing speed and realizing real-time recognition.

Further, according to the present invention, by generating a variation value for adjusting the position of the ROI in the road image information by using the driving lane data detected by the image processing section in the calibration section, the position of the vehicle front dashboard on which the black box is mounted The optimum lane detection position can be automatically set regardless of the lane departure warning signal, and the lane departure warning signal can be generated in real time.

The present invention further includes a search area resetting unit for resetting the search area for the candidate lane to the driving lane data processing unit constituting the image processing unit so as to track the lane in real time in response to the position change of the candidate lane in the area of interest There is an effect.

In the present invention, the warning signal is limited to the case where the lane-tracking is successful and the condition of both the tracking lane and the approaching within the threshold value with respect to the X coordinate of the detection lane is satisfied in order to generate the warning signal in the warning signal part, It is possible to enhance the accuracy and improve the safety.

FIG. 1 is a diagram showing the overall configuration of a real-time lane departure warning system for a car black box, which improves the processing speed according to an embodiment of the present invention.
Fig. 2 is a detailed configuration diagram of the preprocessing section for image processing of Fig. 1. Fig.
FIG. 3 is a diagram for explaining the ROI image extracting unit of FIG. 2. FIG.
FIG. 4 is a view for explaining the ROI size resizing unit of FIG. 2. FIG.
5 is a view for explaining the monochrome image generating unit of FIG.
6 is a detailed configuration diagram of the image processing unit of FIG.
FIG. 7 is a diagram for explaining a Gaussian blur image generating unit of FIG. 6. FIG.
8 to 9 are views for explaining the candidate lane extracting unit of FIG.
Figs. 10 to 12 are diagrams for explaining the driving lane data processing unit of Fig. 6. Fig.
13 is a detailed configuration diagram of the calibration unit of FIG.
Figs. 14 to 16 are diagrams for explaining the calibration unit of Fig. 13. Fig.
17 is a flowchart showing a real-time lane departure warning method for a car black box, which improves the processing speed according to another embodiment of the present invention.
18 to 20 are flowcharts for explaining the flow chart of FIG. 17 in more detail.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas.

Meanwhile, the meaning of the terms described in the present invention should be understood as follows.

The terms "first "," second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It should be understood that the singular " include "or" have "are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, the identification code does not describe the order of each step, Unless otherwise stated, it may occur differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present invention.

Meanwhile, the ROI described in the specification of the present invention indicates a ROI required for image processing, and data of a ROI corresponding to ROIs may be referred to as ROI data (ROI).

Here, the region of interest is used in a case where a desired result can be obtained even if the image processing is performed only on a specific region of a necessary portion without applying the image processing to the entire image. In this case, instead of processing the entire region of the image, The time required for the image processing can be shortened.

In the specification of the present invention, it can be seen that the region of interest is an area extracted mainly from an image frame constituting a road image, and in the embodiment of the present invention, it is limited to a panorama or rectangular shape in detail using an XY coordinate range. Not only the coordinate range of the frame but also various information can be extracted and used as needed.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a diagram showing the overall configuration of a real-time lane departure warning system for a car black box, which improves the processing speed according to an embodiment of the present invention.

As shown in the figure, a real-time lane departure warning system for a car black box according to the present invention is a lane departure warning system for warning a lane departure warning to a driver in lane departure using image information of a camera mounted on a vehicle black box An image information input unit 100, an image processing preprocessor 200, an image processing unit 300, a warning signal unit 400, and a calibration unit 500.

More specifically, the image information input unit 100 receives road image information through the camera, and the image processing preprocessor 200 preprocesses the road image information input to the image information input unit 100 It is possible to generate an image for image processing for real-time lane recognition of a region of interest for detecting a lane.

In addition, the image processor 300 can remove noise of the image for processing image generated through the image processing preprocessing unit 200, extract candidate lanes, and detect driving lane data using the RANSAC algorithm.

The warning signal unit 400 generates a lane departure warning signal when the driving lane data detected by the image processing unit 300 satisfies the warning generation condition, 300, the position value of the ROI extracted from the image processing preprocessing unit 200 and the lane width value transmitted to the warning signal unit 400 can be adjusted.

Meanwhile, in the embodiment of the present invention, the image information input unit 100 receives the image information formed by any one of the RGB format and the YUV format as raw image data without any additional operation, (200).

Fig. 2 is a detailed configuration diagram of the preprocessing section for image processing of Fig. 1, and Figs. 3 to 5 are views for explaining the detailed configuration of Fig. 2 in detail.

As shown in the figure, in the embodiment of the present invention, the image processing preprocessing unit 200 includes an ROI image extracting unit 210, an ROI size reconstruction unit 210, And a black-and-white image generating unit 230. The black-

Here, the ROI image extracting unit 210 may extract a panoramic ROI image corresponding to a region of interest for detecting a lane from the road image information and having a relatively long length compared to the vertical length.

In addition, the ROI size resizing unit 220 may readjust the ROI image of the ROI extracted through the ROI image extracting unit 210 to a specific size corresponding to the minimum size that can be recognized through the vehicle black box have.

In addition, the monochrome image generating unit 230 may generate a monochrome image so that the monochrome image can be processed from the resized ROI image through the ROI size resizing unit 220.

3 to 5, a preprocessing unit for image processing according to an embodiment of the present invention will be described in more detail as follows.

3, the ROI image extracting unit 210 may extract a panoramic ROI image corresponding to a region of interest from a central portion of a lower side of an image screen constituting road image information, It is possible to improve the data processing speed by extracting only the possible specific area as a region of interest.

Meanwhile, in the embodiment of the present invention, the ROI image extracted first from the ROI image extracting unit 210 is a position at which the ROI image is extracted corresponding to the position value of the ROI transmitted from the calibration unit 500 of FIG. 1 It can be changed in real time.

FIG. 4 is a diagram illustrating the adjustment of the size of the ROI image extracted by the ROI image extracting unit 210. FIG.

As described above, since the size of the area of interest of the road image information corresponding to the ROI image extracted by the ROI image extracting unit 210 is large enough to be processed in real time, it is converted into a smallest size .

That is, as shown in FIG. 4, the ROI size resizing unit 220 fixes the ROI image of the panoramic shape as a reference value and reduces the horizontal length as a whole, so that the horizontal length is relatively short It is possible to resize the image into a formed rectangular ROI image.

In the embodiment of the present invention, the size of the initially extracted ROI image has a size of 320 × 100 pixels to detect a lane, and the re-adjusted ROI image can be formed to have a size of 80 × 120 pixels.

In this way, the ROI size refinement unit 220 reduces the horizontal length of the ROI image and fixes the vertical length, because the direction in which the lane appears indicates the longitudinal characteristic rather than the lateral characteristic.

5 is a diagram showing the result processed by the monochrome image generating unit 230. As shown in FIG.

As shown in the figure, a monochrome image for image processing is required. The monochrome image generating unit 230 generates a monochrome image using the image information inputted through the image information input unit, In the case of the image format of YUV system consisting of the U value and the V value, only the Y value, which is the brightness signal, is extracted to generate a monochrome image. When the image information input through the image information input unit is an RGB image format, A monochrome image (gray) can be generated using Equation (1).

FIG. 6 is a detailed configuration diagram of the image processing unit of FIG. 1, and FIGS. 7 to 12 are views for explaining FIG. 6 in more detail.

The image processing unit 300 according to the embodiment of the present invention may include a Gaussian blur image generating unit 310, a candidate lane extracting unit 320, and a driving lane data processing unit 330 have.

More specifically, the Gaussian blur image generating unit 310 may generate a Gaussian blur image in which the noise of the image for processing the image generated through the image processing preprocessing unit 200 is removed as shown in FIG.

The candidate lane extracting unit 320 may extract a candidate lane using the horizontal gradient in the Gaussian blur image generated by the Gaussian blur image generating unit 310. [

That is, as shown in FIGS. 8 to 9, the candidate lane extracting unit 320 generates a horizontal gradient image using Equation (2) below and calculates a candidate lane by using the relationship between the minimum value and the maximum value of the horizontal gradient value. A candidate lane forming unit 322 for forming a candidate lane having a minimum thickness by leaving and removing a median value for the candidate lane extracted from the horizontal gradient processing unit 321, And a lane eliminator 323 for eliminating a candidate lane formed at the unit 322 when there is no change with the lapse of time.

More specifically, the present invention extracts a candidate lane using the value of the horizontal gradient, as described above, and can determine whether the lane is a lane by using the local minimum / maximum value relationship of the horizontal gradient value.

8, when the difference between the minimum value and the maximum value of the horizontal gradient value constituting the generated horizontal gradient image is greater than or equal to a predetermined value, the horizontal gradient value processing unit 321 outputs the horizontal gradient value And the distance difference between the minimum value and the maximum value of the distance is less than or equal to a certain distance, it can be judged as a candidate lane.

The conditions for the above-described candidate lane judgment are expressed by the following equations.

(Where Gv = horizontal radial image flattened gradient image, I (x, y) = x, y coordinate input image value)

In the meantime, in the embodiment of the present invention, the candidate lane can be eliminated when the vehicle is in the stop state. In order to eliminate the erroneous expression in the stopped state of the vehicle without changing the image, can do.

The image T corresponding to the change of time corresponds to Equation (4) below and the image A in which the candidate lane is removed according to the change of time is expressed by Equation (5) below.

(Here, T = image T, L- (x, y) according to the change in the previous time = median value extracted image of the candidate lane)

Also, in the embodiment of the present invention, the driving lane data processing unit 330 described above detects the driving lane data from the candidate lane using the RANSAC algorithm, stably tracks the driving lane, and applies the RANSAC algorithm A lane search area can be set.

6, the driving lane data processing unit 330 includes a search area setting unit 331 for setting an initial search area for the candidate lane, A driving lane detecting unit 332 for detecting driving lane data by finding points forming lanes by using the RANSAC algorithm and connecting the respective points by a straight line, and a driving lane detecting unit 332 for detecting driving lane data by using a Kalman filter for the driving lane comprising the detected driving lane data And a search area resetting unit 334 for resetting the search area for the candidate lane in accordance with the tracking result of the lane-finding unit 333.

On the other hand, the search area setting unit 331 sets the search area on the condition shown in FIG. 10, and uses the following equation (6) to search the initial search area The slope θ of the reference line and the distance rho between the reference line and the reference line corresponding to the reference line can be set and the condition of the search area can be changed according to the result of tracking the lane to be described later.

(here,

,

)

12, the driving lane detecting unit 332 detects a driving lane by using randomly sampled data corresponding to lane-forming points in the search range of the candidate lane, It is possible to detect a driving lane forming a straight line. This indicates how the maximum inlier is determined as a consensus parameter using randomly sampled data using RANSAC (RAN domain Sample Consensus) theory.

In addition, it is preferable to use a generalized expression for the expression of the Kalman filter used in the lane-tracking unit 333, and the expression of the Kalman filter applied to the calibration unit shown below may be referred to.

Also, the search region resetting unit may regard the search region as a failure and reset the search region to the initial search region if the W value of the search region is equal to the initial value for a predetermined time, The RANSAC search area can be reset using the following Equation (7).

(here,

, Wt = current W value, Wt-1 = previous W value, inlier = number of inliers)

In the embodiment of the present invention, when the lane-tracking by the lane-tracking unit succeeds for a predetermined time or more, the warning signal unit 400 shown in Fig. 1 determines that the x- The lane departure warning signal can be generated when both the lane departure detection unit and the lane departure detection unit approach the threshold value and the x coordinate of the lane detected by the lane detection unit approaches the threshold value of the lane in the ROI .

13 is a detailed configuration diagram of the calibration unit of FIG.

As shown in the figure, the calibration unit 500 constituting the system according to the embodiment of the present invention may include an ROI position adjustment unit 510, a lane width adjustment unit 520, and a data stabilization unit 530 have.

In the meantime, the present invention does not detect lanes in the entire input image area for real-time processing even in a low-cost, low-rise black box, but detects a lane by setting a ROI of a certain area, Detection and the quality of the system accordingly.

That is, it is practically impossible to install the black box camera in a position where the lane detection can be smoothly performed in the process of installing the black box camera. Further, the installation height and the position of the vehicle are different from each other, It may also vary depending on the installation location.

In other words, since the ROI position information and the lane width information are different from each other, it is necessary to perform a correction operation to automatically find the ROI position that can smoothly detect the lane and automatically find the lane width information at the same time And the calibration unit 500 of the present invention will be referred to as a configuration for the above correction operation.

Figs. 14 to 16 are diagrams for explaining the calibration unit of Fig. 13. Fig.

As shown in Fig. 14, in the embodiment of the present invention, it can be known that the position shift reference value of the ROI is ROI [left (x), top (y)].

The calibration unit 500 according to the embodiment of the present invention adjusts the position value of the ROI using the ROI position adjustment unit 510 shown in FIG. 13 using the driving lane data detected by the image processing unit 300 of FIG. It can be seen that a variation value is generated.

On the other hand, FIG. 15 can adjust the top position of the ROI by adjusting the left position of the ROI first by using the x coordinate at the position of y = ROIheight in the lane and by using the y coordinate.

ROIleft = ROIleft + Leftx + margin, ROIleft = ROIleft + (ROIwidth-Rightx) + margin, and ROIleft = Leftx + ((Rightx-Leftx) / 2) - (ROIwidth / 2) can be applied.

15, it is possible to adjust the top value based on the position where the y coordinate of the inlier data of the RANSAC on the left and right sides is largest (near the car bonnet). If Leftbottom> Rightbottom, ROItop = ROItop - (ROIheight-Leftbottom) + margin, otherwise, ROItop = ROItop- (ROIheight-Rightbottom) + margin.

In addition, the calibration unit 500 according to the embodiment of the present invention estimates the lane width value using the driving lane data detected by the image processing unit 300 of Fig. 1, and adjusts the lane width by being transmitted to the warning signal unit The lane-width adjusting unit 520 shown in Fig. 13 may be provided to generate the variation value for the lane-width adjusting unit 520 shown in Fig.

At this time, as shown in FIG. 16, the width of the lane can be estimated by using the x-coordinate at the position of y = ROIheight of two lanes, and width = Right x-Left x can be applied.

The data stabilizing unit 530 shown in FIG. 13 stabilizes the variation value data generated by the ROI position adjusting unit 510 and the lane width adjusting unit 520 using a Kalman filter or an average value .

Equation (8) and Equation (9) below express the equations using the Kalman filter and the mean value, respectively.

A covariance matrix, A: state trasition matrix, K: Kalman gain, Q: Covariance matrix of the prediction process, R: Covariance of measured values Matrix, z: measurement data)

(Where X is the average of the current data, Xk-1 is the previous average, K is the number of measured data, and Xk is the current data)

Hereinafter, a lane departure warning method using a real time lane departure warning system for a car black box will be described in which the above-described processing speed is improved with reference to a flowchart.

Description of the same configuration as described above in Figs. 1 to 16 will be omitted.

17 is a flowchart showing a real-time lane departure warning method for a car black box, which improves the processing speed according to another embodiment of the present invention.

A lane departure warning method for improving a processing speed according to an embodiment of the present invention is a lane departure warning method for warning a lane departure warning to a driver in lane departure using image information of a camera mounted on a vehicle black box, Receiving road image information formed of any one of an RGB method and a YUV method through a camera as raw image data without additional operation; Generating an image for image processing for real-time lane recognition of a region of interest for detecting lanes by preprocessing road image information; Removing noises of the generated image for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm; And generating a lane departure warning signal when the detected driving lane data satisfies a warning generating condition.

18 to 20 are flowcharts for explaining the flow chart of FIG. 17 in more detail.

The step of generating an image for image processing for real-time lane recognition of a region of interest for detecting a lane by preprocessing the road image information may include the steps of: Extracting an ROI image corresponding to the ROI image; Rearranging the ROI image of the ROI to a specific size corresponding to a minimum size capable of lane recognition; And generating a monochrome image from the re-adjusted ROI image.

The step of generating a black-and-white image from the re-adjusted ROI image may include: generating a black-and-white image from the re-adjusted ROI image by using a Y value as a luminance signal when the input image information is a YUV- If the image information input through the image information input unit is an RGB image format, a process of generating a monochrome image (gray) using Equation (1) is performed can do.

The step of extracting a candidate lane using the horizontal gradient in the Gaussian blur image may include: Generating a horizontal gradient image using Equation (2) and extracting a candidate lane using a relation between a minimum value and a maximum value of a horizontal gradient value; Forming a candidate lane of minimum thickness by removing a median value for the extracted candidate lane; And removing the candidate lane when there is no change with passage of time.

The step of setting the initial search area for the candidate lane may include calculating the search width W of the reference line corresponding to the initial search area for the candidate lane and the slope θ of the reference line using Equation (6) And the distance rho of the reference line.

As described above, according to the present invention, the position of a region of interest extracted from a road image photographed by a camera is corrected, thereby improving the processing speed for alerting a lane departure in real time to a user using a vehicle black box having low- Thereby providing a real-time lane departure warning system.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, It is within the scope of the present invention that component changes to such an extent that they can be coped evenly within a range that does not deviate from the scope of the present invention.

100: image information input unit 200: preprocessing unit for image processing
210: ROI image extracting unit 220: ROI size resizing unit
230: a monochrome image generating unit 300:
310: Gaussian blur image generator 320: candidate lane extractor
321: Horizontal gradient processing unit 322: Candidate lane forming unit
323: Lane departure rejection 330: Driving lane data processing section
331: Search area setting section 332: Driving lane detecting section
333: lane-tracking unit 334: search area resetting unit
400: Warning signal unit 500: Calibration unit
510: ROI position adjustment unit 520: Lane width adjustment unit
530: Data Stabilization Unit

Claims

A lane departure warning method for warning a driver of a lane departure when a lane departure occurs by using image information of a camera mounted on the vehicle,
Receiving road image information formed of any one of an RGB method and a YUV method through the camera as raw image data without additional operation;
Generating an image for image processing for real-time lane recognition of a region of interest for detecting lanes by preprocessing road image information;
Removing noises of the generated image for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm;
Adjusting, in real time, the position value of the ROI extracted from the road image information and the lane width value referred to for generating the warning signal using the driving lane data; And
And generating a lane departure warning signal when the detected lane of travel data satisfies a warning generating condition.

The method according to claim 1,
And changing a location where the ROI image is extracted on the image screen constituting the road image information in real time when the position value of the ROI is adjusted using the driving lane data. Real-time lane departure warning method

The method as claimed in claim 1, wherein the step of generating an image for image processing for real-time lane recognition of a region of interest for detecting a lane by preprocessing the road image information comprises:
Extracting an ROI image corresponding to a region of interest for lane detection based on a lower center portion of an image screen constituting road image information;
Rearranging the ROI image of the ROI to a specific size corresponding to a minimum size capable of lane recognition; And
And generating a black-and-white image from the re-adjusted ROI image.

4. The method of claim 3, wherein rearranging the ROI image of the ROI to a specific size corresponding to a minimum size capable of lane recognition comprises:
The ROI image of the panoramic shape is a step of re-sizing the ROI image into a rectangle-shaped ROI image in which the vertical length is fixed as a reference value and the horizontal length is entirely reduced to make the horizontal length relatively shorter than the vertical length. Real time lane departure warning method

The method as claimed in claim 1, wherein the noise of the generated image processing image is removed, a candidate lane is extracted, and then the driving lane data is detected using the RANSAC algorithm,
Generating a Gaussian blur image for noise removal of an image for image processing;
Extracting a candidate lane using a horizontal gradient in a Gaussian blur image;
Setting an initial search area for the candidate lane;
Detecting driving lane data by finding points forming lanes using the RANSAC algorithm in the search area and connecting the points to each other by a straight line;
Stably tracking a driving lane comprising the detected driving lane data using a Kalman filter; And
And resetting the search area for the candidate lane in response to the result of the tracking.

The method of claim 5, wherein extracting a candidate lane using the horizontal gradient in the Gaussian blur image comprises:
Generating a horizontal gradient image using the following equation and extracting a candidate lane using the relationship between the minimum value and the maximum value of the horizontal gradient value;
Forming a candidate lane of minimum thickness by removing a median value for the extracted candidate lane;
And removing the candidate lane when there is no change with passage of time.

The method according to claim 6, wherein the extracting of the candidate lane comprises:
When the difference between the minimum value and the maximum value of the horizontal gradient values constituting the generated horizontal gradient image is equal to or more than a predetermined value and when the distance difference between the minimum value and the maximum value of the horizontal gradient value is equal to or less than a predetermined distance, The lane departure warning method comprising the steps of:

6. The method of claim 5, wherein the setting of the initial search area for the candidate lane comprises:
Wherein the step of setting the search width (W) of the reference line, the slope (?) Of the reference line, and the distance (rho) between the reference line corresponding to the initial search area for the candidate lane is performed using the following equation Improved real-time lane departure warning method

(here,

,

)

6. The method of claim 5, wherein detecting the lane data by using the RANSAC algorithm in the search area and then connecting the respective points with a straight line,
And detecting a driving lane that corresponds to the lane forming points in the search range of the candidate lane and forms a single straight line using randomly sampled data.

The method as claimed in claim 5, wherein the step of resetting the search area for the candidate lane in response to the tracking result comprises:
And a step of resetting the search area to an initial search area when the width (W) value of the search area is equal to the initial value for a predetermined period of time.

The method as claimed in claim 1, wherein the step of generating the lane departure warning signal when the detected driving lane data satisfies the warning generating condition comprises:
When the lane-tracking by the lane-finding unit succeeds for a predetermined time or longer, when the x-coordinate of the lane of tracing approaches a position set as a threshold value of the lane width in the area of interest and the x- And the lane departure warning signal is generated when both of the lane departure warning signal and the lane departure warning signal are both satisfied at the same time.

2. The method as claimed in claim 1, wherein real-time adjustment of the position value of the ROI extracted from the road image information using the driving lane data and the lane width value referred to for generating the warning signal,
Generating a variation value for adjusting a position value of the ROI using the driving lane data;
Estimating a lane width value using the driving lane data detected by the image processing unit and generating a variation value for adjusting the lane width transmitted to the warning signal unit; And
And a step of stabilizing the variable value data generated by using the Kalman filter and the average value.

A lane departure warning system for warning a lane departure to a driver when a lane departure occurs by using image information of a camera mounted on the vehicle,
An image information input unit receiving road image information through the camera;
A preprocessing unit for preprocessing road image information input to the image information input unit to generate an image for image processing for real time lane recognition of a region of interest for detecting a lane;
An image processing unit for removing noises of an image for image processing generated through the preprocessing unit for image processing, extracting candidate lanes, and detecting driving lane data using the RANSAC algorithm;
A warning signal unit for generating a lane departure warning signal when the driving lane data detected by the image processing unit satisfies a warning generating condition; And
And a calibration unit for adjusting a position value of the ROI extracted by the preprocessing unit for image processing and a lane width value transmitted to the warning signal unit using the driving lane data detected by the image processing unit Real-time lane departure warning system with improved processing speed

14. The image processing apparatus according to claim 13,
An ROI image extracting unit for extracting a panoramic ROI image corresponding to an area of interest for detecting a lane from the road image information and having a relatively long length relative to a vertical length;
An ROI size reconstructing unit for reconstructing an ROI image of a ROI extracted through the ROI image extracting unit to a specific size corresponding to a minimum size capable of recognizing a lane through the vehicle black box; And
And a monochrome image generating unit for generating a monochrome image so that an image can be processed from the resized ROI image through the ROI size resizing unit,
Wherein the ROI image extracting unit extracts the ROI image of the panoramic shape from the central portion of the lower side of the image screen constituting the road image information and then extracts the ROI image corresponding to the position value of the ROI transmitted from the calibration unit Wherein the real time lane departure warning system

15. The apparatus of claim 14, wherein the ROI size re-
The ROI image of the panoramic form is fixed to a reference value and the width of the ROI image is entirely reduced to form a rectangular ROI image having a size of 80 × 120 pixels, the width of which is relatively short compared to the vertical length. Real-time lane departure warning system

14. The image processing apparatus according to claim 13,
A Gaussian blur image generating unit for removing noise of an image for image processing generated through the preprocessing unit for image processing;
A candidate lane extracting unit for extracting a candidate lane using a horizontal gradient in the Gaussian blur image generated by the Gaussian blur image generating unit; And
And a driving lane data processing unit for detecting driving lane data from the candidate lane using the RANSAC algorithm and stably tracking the driving lane and setting a lane search area for applying the RANSAC algorithm. Real-time lane departure warning system

The navigation system according to claim 16,
A search area setting unit for setting an initial search area for the candidate lane;
A driving lane detecting unit for detecting driving lane data by finding points forming lanes by using the RANSAC algorithm in the search area and connecting respective points by a straight line;
A lane-tracking unit for stably tracking the driving lane comprising the detected driving lane data using a Kalman filter; And
And a search area resetting unit resetting a search area for the candidate lane in response to a result of the lane-finding unit tracking the real time lane departure warning system

14. The apparatus according to claim 13,
An ROI position adjustment unit for generating a variation value for adjusting a position value of a ROI using the driving lane data detected by the image processing unit;
A lane width adjusting unit that estimates a lane width value using the driving lane data detected by the image processing unit and generates a variation value for adjusting the lane width transmitted to the warning signal unit; And
And a data stabilization unit that stabilizes the variation value data generated by the ROI position adjustment unit and the lane width adjustment unit using a Kalman filter and an average value, Alarm system