KR20180101029A - Apparatus and method for estimating curvature of road - Google Patents

Abstract

The present invention provides an apparatus and method for estimating a curvature of a road using an eye-guided object disposed along a road. The apparatus for estimating a curvature of a road according to the present invention includes an image obtaining unit for obtaining a front image of the road on which a vehicle runs, an image processing unit for extracting a plurality of eye-induced objects installed along the road or lane components of the road from the obtained front image, calculating the curvature of the road by applying curve fitting to the plurality of eye-induced objects or the lane components which are extracted, and determining a coefficient of a curve equation with an arbitrary dimension using the calculated curvature, and a lane outputting unit for outputting lanes of the road based on a calculation result of the curve equation. Accordingly, the present invention can estimate the curvature of the road regardless of securing the visibility of a running image.

Description

[0001] APPARATUS AND METHOD FOR ESTIMATING CURVATURE OF ROAD [0002]

The present invention relates to an apparatus and method for estimating road curvature. More particularly, the present invention relates to an apparatus and method for estimating a road curvature using a line-guided object disposed on a road.

Recently released vehicles are equipped with a driving support system. They use the sensors mounted on the vehicle to detect the surroundings of the vehicle and conduct driving safety control such as lane keeping, collision avoidance, and headlight control based on the sensed information do.

In this case, the sensor installed in the front of the vehicle recognizes the lane. Examples of the sensor include a vision sensor, a radar, and a lidar. Particularly, the vision sensor uses a variety of image processing algorithms It is widely used because it can be used.

That is, a lane recognition method for acquiring a running road as an image and recognizing lanes in an acquired image is performed by mapping a lane position in the acquired image to two-dimensional coordinates and detecting and displaying the position of the lane. At this time, when the lane is formed of a curve, the curve is recognized from the calculated lane position coordinates by using the curve equation or by using the width of the lane.

However, this method can significantly reduce the accuracy of the recognized lane when the visibility is significantly low, that is, when it is raining or foggy, when the camera lens is stained or when the lane is erased.

That is, the conventional curve recognition method uses a curve fitting method for the lane. If the visibility of the lane is not secured, the curvature of the lane can not be correctly recognized. Therefore, it is possible to estimate an offset indicating lateral deviation of the lane and a heading angle, which is direction information of the lane corresponding to the vehicle, from the feature points located in the vicinity of the acquired image, but it is also possible to calculate the curvature There is no problem.

In this case, the accuracy of the lane recognition is degraded, so that errors may occur in various driving support systems mounted on the vehicle. In particular, errors in the lane keeping assistance system (LKAS) A phenomenon may occur.

The present invention, which is devised from the background described above, proposes an apparatus and method for estimating a curvature of a road using an eye-guided object disposed along a road.

In addition, the present invention proposes a road curvature estimation apparatus and method for estimating a curvature of a highly reliable curved road by using a cubic curve equation.

According to another aspect of the present invention, there is provided an apparatus for estimating a curvature of a road, comprising: an image acquisition unit for acquiring a forward image of a road on which the vehicle is traveling; a lane component of the road from the obtained forward image; The curvature of the road is calculated by applying curvilinear fitting to the extracted lane component or the plurality of line-of-sight guidance objects, and an arbitrary dimension is calculated using the calculated curvature And a lane output unit for outputting a lane on the road based on the calculation result of the curve equation.

The image processing unit may further include an interest area setting unit that sets a road area located at a lower end of the horizon line in the forward image and a line-guiding area located at left and right sides as an area of interest, an edge detecting unit for detecting edges of the plurality of line-guided objects, extracting a plurality of feature points based on pixel information or color information from the detected edges, and extracting the plurality of feature points based on the position of the vehicle And a curvature calculating unit for calculating the curvature of the road using the curve equation by performing a curve fitting method by merging the extracted feature points and the extracted plurality of feature points.

According to another aspect of the present invention, there is provided a road curvature estimation method comprising: an image acquisition step of acquiring a forward image of a road on which a vehicle is traveling; a lane component of the road or a road A curvature of the road is calculated by applying curve fitting to the extracted lane component or the plurality of line-of-sight guidance objects, and the curvature of the road is calculated using an arbitrary dimension And a lane-outputting step of outputting a lane of the road based on the calculation result of the curve equation.

In addition, the image processing step may include: a ROI setting step of setting a road area located at the lower end of the horizon in the forward image and a gaze guidance area located at left and right sides of the ROI as an ROI; An edge detecting step of detecting edges of the plurality of line-of-sight guiding objects, extracting a plurality of feature points based on pixel information or color information from the detected edges, A feature point extracting step of extracting a feature point from the feature points and a feature point extracting step of extracting feature points from the feature point, .

Other specific details of the invention are included in the detailed description and drawings.

The present invention can estimate the curvature of a curved road regardless of securing the visibility of the traveling image.

In addition, the present invention provides the driver with the curvature of the curved road with improved reliability, thereby improving the convenience of traveling.

1 is a block diagram of a road curvature estimating apparatus according to an embodiment of the present invention.
FIG. 2 is a view illustrating a curvature of the obtained front image along a lane and a curvature according to an eye-guiding object.
3 is a flowchart briefly explaining a road curvature estimation method according to an embodiment of the present invention.
4 is a flowchart for explaining a road curvature estimation method according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Although the first, second, etc. are used to describe various elements, components and / or sections, it is needless to say that these elements, components and / or sections are not limited by these terms. These terms are only used to distinguish one element, element or section from another element, element or section. Therefore, it goes without saying that the first element, the first element or the first section mentioned below may be the second element, the second element or the second section within the technical spirit of the present invention.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms "comprises" and / or "made of" means that a component, step, operation, and / or element may be embodied in one or more other components, steps, operations, and / And does not exclude the presence or addition thereof.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram of a road curvature estimation apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 1, a road curvature estimation apparatus 100 includes an image acquisition unit 110 for acquiring a forward image of a road on which a vehicle is traveling, a lane component of the road from the obtained forward image, And calculating a curvature of the road by applying a curve fitting method to the extracted lane component or the plurality of line-of-sight guidance objects, and calculating an arbitrary And a lane output unit 130 for outputting a lane of the road based on the calculation result of the curve equation.

At this time, the visual line-guiding object includes at least one or more than one or more road markers, street lamps, street lamps, guide lamps, columnar caps, reflectors, and guide rods disposed along one or more of the roads, Is an object that aims to drive safely by guiding the line of sight to follow the curve of the road.

Specifically, the image obtaining unit 110 may obtain the forward image of the curved road ahead of the vehicle from a photographing device capable of photographing still images and moving images provided in the vehicle.

At this time, the photographing apparatus may include a vision sensor. Examples of the vision sensor include a vision sensor capable of acquiring a two-dimensional image and a three-dimensional vision sensor capable of acquiring a three-dimensional image. The photographing apparatus may be configured to include at least one of a stereo camera and a moving stereo camera for photographing all directions around the vehicle and acquiring an image.

The image acquiring unit 110 may further include a communication unit (not shown) for performing a communication function with the photographing apparatus to acquire the forward image. At this time, the communication unit may be a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a World Interoperability for Microwave Access (Wimax), a High Speed Downlink Packet (HSDPA), IEEE 802.16, Long Term Evolution (LTE), Wireless Mobile Broadband Service (WMBS), Bluetooth, Radio Frequency Identification (RFID) Wireless communication technologies such as Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, CAN communication, automotive Ethernet, flexray, and LIN (Local Interconnect Network) .

The image processing unit 120 includes a region of interest setting unit 121 that sets a road region located at a lower end of the horizon in the forward image and a line-guiding region located at left and right as an ROI, An edge detection unit 122 for detecting an edge of a lane component or an edge of the plurality of line-guided objects, a plurality of feature points based on pixel information or color information from the detected edge, A feature point extraction unit (123) for converting the plurality of feature points into a coordinate system based on the extracted feature points, and a curve fitting method by merging the extracted feature points and calculating a curvature of the road using the curve equation (124). ≪ / RTI >

Specifically, the ROI setting unit 121 receives the forward image acquired by the image obtaining unit 110 and removes the obstacle. The obstacle refers to an object that causes a lane recognition error, And the like.

Then, the ROI setting unit 121 sets the ROI and the ROI as the ROI (ROI) in the forward image from which the obstacle has been removed, And is set so as to minimize an image search range for lane recognition, including an eye-guided object.

The method of setting the region of interest may be based on an Adaboost algorithm. The Adaboost algorithm combines a set of weak classifiers linearly to generate a strong classifier with high detection performance. .

The edge detecting unit 122 may detect the lane component and the edges of the plurality of line-guided objects by receiving the region of interest set by the region-of-interest setting unit 121. [ At this time, the edge detector 122 may use a Hough Transform.

In addition, the edge detecting unit 122 stores edge components of various lane-guiding objects as well as lanes so that the edge of the line-guiding object can be easily detected.

The feature point extracting unit 123 can extract the plurality of feature points through various algorithms using the edge information received from the edge detecting unit 122. [ At this time, the feature point extracting unit 123 can extract the feature point for the lane component and the feature point for the eye-directed object, respectively.

For example, the feature point extracting unit 123 may extract the feature points based on pixel information. That is, the feature point extracting unit 123 may designate the pixel as a feature point and extract all the feature points for the designated pixel if the brightness or color hue between neighboring pixels of the obtained forward image changes abruptly.

The feature point extracting unit 123 may extract the feature points based on the color information. In other words, the general lane of the road is white, the center lane is yellow, the non-lane part is black, the road guide sign, which is a guide object, is yellow, and the street lamp and tunnel guide light are yellow, It is possible to select only the region having the color that can be classified as the lane component or the line-guided object in the image into one object. That is, the feature point may be extracted from the object based on the lane component or the color information previously stored on the eye-directing object.

In addition, the feature point extracting unit 123 may convert the position of the feature point extracted based on the degree of freedom of the six-axis relative to the vehicle to the coordinate value of the coordinate system. That is, the feature point extracting unit 123 may perform a calibration process of calculating a conversion relation between the photographing apparatus provided in the vehicle and the coordinate system based on the position of the vehicle.

Here, the six-axis degree of freedom refers to the number of mutually independent variables necessary to determine the geometric position of the body with respect to the three axes, i.e., the x, y, and z axes, relative to the vehicle.

The method of converting the feature points into the coordinate values of the coordinate system may be performed by using a transformation matrix or using a coordinate transformation equation, and the transformation matrix may be a homographic maxtix.

The curvature calculator 124 may perform the curve fit method to receive the feature points having coordinate values in the feature point extractor 123 and express them as one line. That is, the curve fitting method can be performed based on a multidimensional curve equation, and a coefficient for the degree of the curve equation can be obtained by merging the coordinate values corresponding to the characteristic points. For example, the curvature can be calculated by substituting the coordinate values of the minutiae points into the curvilinear equation to calculate the curvilinear equation to be concatenated.

At this time, each of the curve fitting methods may be performed on the lane information or the line-guided object.

일 수 있으며, 이는 [수학식 1]과 같이 나타낼 수 있다.Here, the curve fitting method, that is, the curve fitting method, refers to a technique of calculating and calculating a curve for the point with respect to a point scattered on a plane, and the curve equation for the curve fitting method is a cubic curve equation

, Which can be expressed as [Equation 1].

Here, the 0th-order coefficient C ₀ is a heading angle which is direction information of the lane corresponding to the vehicle, and the first-order coefficient C ₁ is an offset indicating the lateral deviation of the lane, and 2 The difference coefficient C ₂ and the third order coefficient C ₃ may be the curvature.

Further, the offset indicates whether the lane of the road is located on the left side or the right side with respect to the vehicle, and refers to a value corresponding to the first order coefficient of the cubic curve equation. That is, when the primary coefficient has a positive value, the lane is located on the right side of the vehicle, and conversely, when the constant has a negative value, the lane indicates that the lane is located on the left side of the vehicle.

That is, if the visibility of the lane in the forward image is not secured, the curvature of the lane can not be correctly recognized. Therefore, only the offset and the direction angle can be estimated by performing the curve fitting method with respect to the lane component. That is, the curvature of the road curve can not be calculated. Therefore, in this case, the curvature of the road can be easily calculated by applying a characteristic point to the curvilinear equation and performing a curve fitting method.

Therefore, the curvature calculator 124 can generate the lane by calculating both the constants and the first to third-order coefficients of the cubic curve equation, so that the lane of the curved road on which the vehicle travels can be recognized.

The lane output unit 130 generates the lane through the cubic curve equation calculated by the curvature calculator 124 and outputs it to a blind spot detecting system (BSD), a lane departure avoidance system And a driving support system including a control unit (LDWS & LKAS), a smart cruise control (SCC), a smart parking assist system (SPAS), and the like.

Further, the lane-output section 130 may output the generated lane to the driver of the vehicle. For example, when the vehicle is running, the lane can be displayed through a UI (User Interface) or GUI (Graphic User Interface) related to driving.

FIG. 2 is a view illustrating a curvature of the obtained front image along a lane and a curvature according to an eye-guiding object.

Fig. 2 (a) is a front image, which is a road guidance marker, and Fig. 2 (b) is a front image, in which a gaze guidance object is a tunnel guidance light.

2 (a) and 2 (b), it can be seen that the curve OC for the eye-guided object has a similar curvature to the curved line LC for the lane.

At this time, the visual line-guiding object includes at least one or more than one or more road markers, street lamps, street lamps, guide lamps, columnar caps, reflectors, and guide rods disposed along one or more of the roads, Is an object that aims to drive safely by guiding the line of sight to follow the curve of the road.

Therefore, when the visibility is not ensured to such an extent that the curvature of the lane can not be properly recognized, the curvature of the road can be calculated by extracting the minutiae of the visual line-guiding object instead of the lane component and performing the curve fitting method. The road curvature estimation apparatus 100 can accurately generate the lane of the road even if the visibility of the lane of the front image of the vehicle is not secured.

3 is a flowchart briefly explaining a road curvature estimation method according to an embodiment of the present invention.

Referring to FIG. 3, the road curvature estimation method includes an image acquisition step (S300) of acquiring a forward image of a road on which the vehicle is traveling, a lane component of the road from the obtained front image, The curvature of the road is calculated by applying a curve fitting method to the extracted lane component or the plurality of line-of-sight guidance objects, and the curvature of the road is calculated using the calculated curvature An image processing step of determining coefficients of the curve equation, and a lane output step (S350) of outputting a lane of the road based on the calculation result of the curve equation.

In this case, the image processing step may include an interest area setting step (S310) of setting a road area located at the lower end of the horizon in the forward image and a sight line guidance area located at left and right sides as an area of interest (S310) An edge detection step (S320) of detecting an edge of a lane component or an edge of the plurality of line-guided objects, extracting a plurality of feature points based on pixel information or color information from the detected edge, A feature point extracting step of converting the plurality of feature points into a coordinate system based on the plurality of feature points extracted in step S330 and a step of performing a curve fitting method by merging the extracted plurality of feature points and calculating a curvature of the road using the curve equation And a calculating step S340.

4 is a flowchart for explaining a road curvature estimation method according to an embodiment of the present invention.

Referring to FIG. 4, in an image acquisition step (S300), a forward image of a road on which the vehicle is traveling is acquired (S400). At this time, it is possible to acquire the forward image of the curved road ahead of the vehicle from a photographing device capable of photographing still images and moving images provided in the vehicle.

In the interest area setting step S310, the road area located at the bottom of the horizon in the front image from which the obstacle is removed, and the sight line guidance area located at the right and left sides are set as the area of interest (S410). At this time, the ROI includes the lane component and the line-guided object, and is set to minimize an image search range for lane recognition.

In operation S420, an edge of the lane component or an edge of the plurality of line-guided objects located within the set region of interest is detected in operation S420.

In the feature point extraction step S330, a plurality of feature points are extracted based on the pixel information or the color information from the edge (S430).

In operation S450, the plurality of feature points extracted based on the position of the vehicle are converted into coordinate systems. At this time, the method of converting the feature point into the coordinate value of the coordinate system may be performed by using a transformation matrix or using a coordinate transformation equation, and the transformation matrix may be a homographic maxtix.

In step S440, the curve fitting method is performed by merging the plurality of feature points extracted from the lane information (S450), and it is determined whether the lane component is detected to be equal to or greater than a threshold (S460). That is, the visibility of the lane of the forward image is secured and it is determined whether the curvature can be calculated with the detected lane component.

Specifically, the curve fitting method can be performed based on a multidimensional curve equation, and a coefficient for the degree of the curve equation can be obtained by merging the coordinate values corresponding to the characteristic points. That is, the curvature can be calculated by substituting the coordinate values of the minutiae points into the curvilinear equation and calculating the curvature equation to be concatenated.

The curve equation for the curve fitting method may be a cubic curve equation, which is expressed by the following equation (1): f (x) = C ₃ x ³ + C ₂ x ² + C ₁ x + C ₀ .

Here, the 0th-order coefficient C ₀ is a heading angle which is direction information of the lane corresponding to the vehicle, and the first-order coefficient C ₁ is an offset indicating the lateral deviation of the lane, and 2 The difference coefficient C ₂ and the third order coefficient C ₃ may be the curvature.

If the visibility of the lane of the front image is not secured, the curvature of the lane can not be correctly recognized. Therefore, only the offset and the direction angle can be estimated by performing the curve fitting method with respect to the lane component. That is, the curvature of the road curve can not be calculated. Therefore, in this case, the curvature of the road can be easily calculated by applying a characteristic point to the curvilinear equation and performing a curve fitting method.

Therefore, when the lane component is detected to be equal to or greater than the threshold value, that is, when the visibility of the lane is good in the forward image, the curve fitting method is performed using only the feature points of the lane component (S450). That is, the feature point of the lane component is calculated by the curve equation, the curvature is calculated, and the coefficient is determined to output the curve lane of the road (S480).

On the other hand, when the lane component is detected to be less than the threshold value, that is, when the visibility of the lane is poor in the forward image, the curvature can not be calculated as the feature point of the lane component, And performs a fitting method (S470). In operation S480, the curved lane of the road is calculated by computing the characteristic points of the line-guided object by the curve equation, calculating the curvature, and determining the coefficient.

As described above, the road curvature estimating apparatus and method according to the present invention can estimate the curvature of a curved road regardless of securing the visibility of the traveling image, and provide the driver with the curvature of the curved road with improved reliability, Can be improved.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or essential characteristics thereof. . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Road curvature estimation device: 100
Image acquisition unit: 110
Image processor: 120
A region of interest setting section: 121 edge detecting section: 122
Feature point extracting section: 123 Curvature calculating section: 124
Lane output section: 130

Claims (10)

An image acquisition unit for acquiring a forward image of a road on which the vehicle is traveling;
Extracting a lane component of the road or a plurality of line-guided objects installed along the road from the obtained forward image, applying a curve fitting method to the extracted lane component or the plurality of line-guiding objects An image processing unit for calculating a curvature of the road and determining a coefficient of a curvilinear equation having an arbitrary dimension by using the calculated curvature; And
A lane output unit for outputting a lane on the road based on the calculation result of the curve equation; The road curvature estimating device comprising: The method according to claim 1,
Wherein the image processing unit comprises:
A ROI setting unit that sets ROIs located at the bottom of the horizon in the forward image and ROIs located at right and left sides as ROIs;
An edge detection unit detecting an edge of the lane component or an edge of the plurality of line-guided objects within the set area of interest;
Extracting a plurality of feature points based on pixel information or color information from the detected edge, and converting the plurality of feature points into a coordinate system based on the position of the vehicle; And
And a curvature calculator for calculating the curvature of the road using the curvilinear equation by performing the curve fitting method by merging the extracted plurality of feature points. 3. The method of claim 2,
In the curvature calculating section,
Wherein the curve equation for the curve fitting method is a cubic curve equation. The method of claim 3,
In the cubic curve equation,
Order coefficient is a heading angle that is direction information of the lane corresponding to the vehicle, the first coefficient is an offset indicating lateral deviation of the lane, the second-order coefficient and the third-order coefficient are And the curvature of the road. 3. The method of claim 2,
The feature point extracting unit may extract,
And converting the position of the minutiae point into a coordinate value of the coordinate system based on a degree of freedom of the vehicle with respect to the six axes. The method according to claim 1,
Wherein the line-of-sight guiding object includes at least one of a road sign, a roadside lamp, a street lamp, an induction lamp, a strut cap, a reflector, and an induction rod arranged along at least one road along the road. An image acquiring step of acquiring a forward image of a road on which the vehicle is traveling;
Extracting a lane component of the road or a plurality of line-guided objects installed along the road from the obtained forward image, applying a curve fitting method to the extracted lane component or the plurality of line- An image processing step of calculating a curvature of the road and determining a coefficient of a curvilinear equation having an arbitrary dimension by using the calculated curvature; And
A lane output step of outputting a lane of the road based on a calculation result of the curve equation; And estimating the curvature of the road. 8. The method of claim 7,
Wherein the image processing step comprises:
Setting an ROI located at a lower end of the horizon in the forward image and a ROI located at right and left sides of the ROI as an ROI;
An edge detecting step of detecting an edge of the lane component or an edge of the plurality of line-guided objects within the set region of interest;
A feature point extracting step of extracting a plurality of feature points based on pixel information or color information from the detected edge and converting the plurality of feature points into a coordinate system based on the position of the vehicle; And
Calculating a curvature of the road using the curve equation by performing a curve fitting method by merging the extracted plurality of feature points; and calculating a curvature of the road using the curve equation. 9. The method of claim 8,
In the curvature calculating step,
Wherein the curve equation for the curve fit method is a cubic curve equation. 10. The method of claim 9,
In the cubic curve equation,
Order coefficient is a heading angle which is direction information of the lane corresponding to the vehicle, the first coefficient is an offset indicating a lateral deviation of the lane, and the second-order coefficient and the third- Road curvature estimation method.

Images