KR101578434B1 - Apparatus for detecting lane and method thereof - Google Patents

Abstract

The present invention relates to a camera module for acquiring an image; A controller for extracting a plurality of feature points from the obtained image, performing lane fitting to connect the extracted plurality of feature points with one line, and tracking the adapted lane; And a display unit for displaying the tracked lane, wherein the lane adaptation comprises: obtaining minutia information extracted at an arbitrary past time point, extracting from the past time point based on the running information from the past past time point to the present time, Estimating a position of the current point of time with respect to the feature point that has been determined, determining an offset indicating the lateral deviation of the lane based on the estimated position of the current point of time, and performing curve fitting based on the determined offset To a lane recognizing device.
Further, the present disclosure relates to a method of acquiring images, Extracting a plurality of feature points from the acquired image; Performing lane fitting to connect the extracted plurality of feature points by one line; Tracking the adapted lane; And displaying the tracked lane, wherein performing the lane fitting comprises: obtaining minutia information extracted at an arbitrary past time; Estimating a position of a current point of time with respect to the minutiae extracted from the past time point based on the traveling information from the past past time point to the present; Determining an offset indicating a lateral deviation of the lane based on the estimated current position; And performing a curve fitting based on the determined offset.

Description

[0001] APPARATUS FOR DETECTING LANE AND METHOD THEREOF [0002]

The present specification relates to a lane recognition apparatus and a method thereof.

Generally, when a traveling device moves along a line, it is necessary to recognize the line by using an image recognition device or the like in order to detect a departure from the line and issue an alarm to the outside.

That is, a recognition device using an image acquires an image of a target to be recognized by a traveling device using a camera or the like, extracts a characteristic of the target using a digital image processing technique, and confirms the target using the extracted characteristic In order for the image recognition apparatus to perform its original function smoothly, it is necessary to extract the target line more accurately from the moving object.

Such image recognition apparatuses can be widely applied in fields such as automobiles, RT (Robot Technology), and AGV (Automated Guided Vehicle). Especially, in the automobile field, since accurate lane recognition must be ensured even when traveling at high speed, The difficulty is high. Here, the lane is the baseline of driving, and it is a standard for all driving behaviors such as forward, backward, lane change, career change, forward parking, reverse parking and in-line parking.

The Advanced Safety Vehicle (ASV), which is currently being sophisticated and intelligent, is mainly applied to the lane identification method, which includes a lane departure warning device to prevent drowsiness driving, a rear parking guide to help beginners park, Devices, and lane keeping devices that maintain the lane by applying torque to the handle in a dangerous situation.

The lane recognition methods according to the technique of acquiring the running road as an image as described above and recognizing lanes in the obtained image are described as follows. The lane points for the coordinates of the acquired image are mapped to the two-dimensional coordinates and the position of the lane is detected .

At this time, when the lane is formed of a curve, the curve is recognized from the obtained lane points using the curve equation or the width of the lane. However, this method can significantly reduce the accuracy of the recognized lane when curved roads are formed by dotted lines and the roads are cut off and restarted from the front of the vehicle.

The present invention provides an apparatus and method for accurately recognizing a lane even when road changes are severe, such as when a curved road with a large curvature is formed by a dotted line and the road is cut off and then restarted from the front of the vehicle.

The lane recognizing apparatus disclosed in this specification includes a camera module for acquiring an image, a lane matching function for extracting a plurality of feature points from the obtained image, connecting the extracted plurality of feature points by one line, And a display unit for displaying the tracked lane, wherein the lane adaptation comprises: acquiring minutia information extracted at an arbitrary past time point; and calculating the driving information from the arbitrary past time point to the present time Estimating a position of the current point of time with respect to the minutiae extracted at the past point in time, determining an offset indicating the lateral offset of the lane based on the estimated position of the current point of time, And performs the adaptation.

Further, the curve fit is characterized by determining coefficients of the curve equation based on a curve equation having an arbitrary dimension.

The driving information may be at least one of speed information, acceleration information, and steering information.

In addition, the camera module may include at least one pair of cameras or a single camera, which are spaced apart from each other on the same central axis of the same plane.

In addition, the plurality of feature points may be extracted only for the region of interest, with only the road portion corresponding to the lower end centered on the horizon in the obtained image set as the region of interest.

The plurality of minutiae are extracted based on gradient information or color information of the obtained image.

The control unit may convert the extracted plurality of feature points into a world coordinate system, and fit the lane based on the converted plurality of feature points.

Further, the tracking of the lane is performed with respect to all of the adapted lanes.

The control unit may further include a storage unit for storing information on the extracted minutiae, wherein the control unit fits the lane based on the information about the minutiae stored in the storage unit.

Further, the lane recognizing apparatus disclosed in the present specification includes a camera module for acquiring an image, a lane matching function for extracting a plurality of feature points from the obtained image, connecting the extracted plurality of feature points with one line, And a display unit for displaying the tracked lane, wherein the lane adaptation obtains an offset representing a lateral deviation of a lane of the lane adaptation result at any past point in time, Estimates an offset of the current point of time based on the travel information from the past point of time to the present point of time, and performs curve fitting based on the estimated offset of the current point of time.

According to another aspect of the present invention, there is provided a lane recognition method including: obtaining an image; extracting a plurality of feature points from the acquired image; performing lane fitting to connect the extracted plurality of feature points with one line; The method comprising the steps of: tracking the adapted lane; and displaying the tracked lane, wherein performing the lane fitting comprises: obtaining minutia information extracted at an arbitrary past time; Estimating a current position of the feature point extracted from the past point of view based on the current driving information from the current point of view, determining an offset indicating lateral displacement of the lane based on the estimated position of the current point of time And performing a curve fit based on the determined offset.

In addition, the step of performing the curve fitting may include determining coefficients of the curve equation based on a curve equation having an arbitrary dimension.

The driving information may be at least one of speed information, acceleration information, and steering information.

In addition, the camera module may include at least one pair of cameras or a single camera, which are spaced apart from each other on the same central axis of the same plane.

The step of extracting the plurality of feature points may further include the steps of setting only a road portion corresponding to a lower end of a center of a horizon in the obtained image as a region of interest and extracting a plurality of feature points only with respect to the region of interest .

The plurality of minutiae are extracted based on gradient information or color information of the obtained image.

The step of fitting the lane may include converting the extracted plurality of feature points into a world coordinate system, and fitting the lane based on the converted plurality of feature points.

In addition, the step of tracking the lane is performed on all of the adapted lanes.

The step of extracting the feature points may further include storing information on the extracted feature points, and the step of fitting the lanes fits the lane based on the information on the stored feature points. do.

According to the lane recognizing apparatus disclosed in this specification, it is possible to accurately recognize a lane even in a changing road situation such as an interchange by tracking the entire lane appearing in the traveling route and recognizing the lane from the tracking result, I can recognize the lane.

Further, according to the lane recognizing apparatus disclosed in this specification, when a curved road with a large curvature is formed by a dotted line, curve fitting to the current lane is performed based on past feature points or past lane fitting results, It is possible to obtain a close result and to determine an accurate off set.

1 is a block diagram showing a configuration of a lane recognizing apparatus according to an embodiment disclosed herein.
2 is a flowchart illustrating a lane recognition process according to an embodiment disclosed herein.
3 is a diagram illustrating an image according to an embodiment disclosed herein.
FIG. 4 is a diagram showing a feature point extraction result according to the embodiment disclosed herein.
FIG. 5 is a diagram showing a world coordinate system transformation result of extracted minutiae in accordance with the embodiment disclosed herein.
6 is a diagram showing lane-fitting results according to the embodiment disclosed herein.
7 is a diagram illustrating the display of lane-tracking results in accordance with the embodiment disclosed herein.
8 is a flowchart illustrating a curve fitting process according to the embodiment disclosed herein.
9 is a diagram showing an example of a road on which a lane recognizing device according to the embodiment disclosed herein runs.
Fig. 10 is a diagram showing the comparison between the actual lane and the curve fitting result lane.
11 is a diagram illustrating a result of obtaining minutia information extracted in the past according to the embodiment disclosed herein.
12 is a diagram illustrating a result of estimating a changed position of a minutiae on the basis of travel information according to the embodiment disclosed herein.
13 is a diagram showing the curve fitting result according to the embodiment disclosed herein.

The embodiments disclosed herein may be configured not only as a stand alone device but also as a mobile terminal, a telematics terminal, a smart phone, a portable terminal, A personal digital assistant (PDA), a portable multimedia player (PMP), a notebook computer, a tablet PC, a Wibro terminal, an Internet Protocol Television (IPTV) terminal, , A 3D television, an image device, a telematics terminal, a navigation terminal, an AVN (Audio Video Navigation) terminal, and the like.

The embodiments disclosed herein may be implemented in the form of program instructions that may be executed on various computer means and recorded in a computer readable medium. The computer-readable medium may include program instructions, file data, data structures, and the like, alone or in combination. Program instructions to be recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as a hard disk, a floppy disk, and a magnetic tape; optical media such as CD-ROM and DVD; magnetic recording media such as a floppy disk; Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The above hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and so forth.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. Also, the technical terms used herein should be interpreted in a sense that is generally understood by those skilled in the art to which the present disclosure relates, unless otherwise specifically defined in the present specification, Should not be construed to mean, or be interpreted in an excessively reduced sense. In addition, when a technical term used in this specification is an erroneous technical term that does not accurately express the concept of the technology disclosed in this specification, it should be understood that technical terms which can be understood by a person skilled in the art are replaced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising" or "comprising" and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

Further, the suffix "part" for a component used in the present specification is given or mixed in consideration of ease of specification, and does not have a meaning or role that is different from itself.

Further, in the description of the technology disclosed in this specification, a detailed description of related arts will be omitted if it is determined that the gist of the technology disclosed in this specification may be obscured. It is to be noted that the attached drawings are only for the purpose of easily understanding the concept of the technology disclosed in the present specification, and should not be construed as limiting the spirit of the technology by the attached drawings.

Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a lane recognition apparatus according to an embodiment disclosed herein.

1, the lane recognizing apparatus 100 may include a camera module 110, a control unit 120, a storage unit 130, an output unit 140, and a sensor unit 150.

The camera module 110 is a camera system for taking an omnidirectional, a rear direction and / or a side direction at a time using a rotating body reflector, a focusing lens, and an image pickup element, and can be applied to a security facility, a surveillance camera, a robot vision, The shape of the rotating body reflector may be hyperbolic, spherical, conical, or hybrid. The camera module 110 includes at least one pair of cameras (a stereo camera, a stereo co-peak camera (hereinafter, referred to as " a stereoscopic camera), or a single camera. At this time, the horizontal interval may be set considering the distance between two eyes of a general person, and may be set when the lane recognition device 100 is constructed. In addition, the camera module 110 may be any camera module capable of image capturing.

A CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) may be used as the imaging element of the camera module 110. An image (i.e., an omni-directional image) projected on the imaging surface of the imaging element may be a distorted image that is reflected by the rotating mirror and is not suitable for human observation. Accordingly, the camera module 110 can convert the coordinates of the output of the image pickup device through a microprocessor or the like to create a new panoramic image for accurate observation of the image.

The camera module 110 may be configured to include at least one of a stereo camera and a moving stereo camera to capture an image in all directions and acquire an image.

The stereo camera is a video device composed of a plurality of cameras. The image obtained through the camera module 110 may provide two-dimensional information about the periphery of the camera module 110. Dimensional information about the periphery of the camera module 110 can be obtained by using a plurality of images taken from different directions through a plurality of cameras.

The mobile stereo camera refers to a camera in which the position of the stereo camera is actively changed according to the distance of the obstacle to fix the main time of the observation obstacle. The stereo camera generally arranges two cameras in parallel and acquires an image, and calculates the distance to the obstacle according to the stereo parallax of the acquired image.

The stereo camera is a passive camera in which the optical axis is always parallel and fixed. On the other hand, the mobile stereo camera can actively change the geometrical position of the optical axis to fix the principal time.

Control of the viewing angle of the stereo camera according to the distance of the obstacle is called main vision control. The main vision control stereo camera maintains a stereoscopic difference with respect to a moving obstacle at all times, thereby providing a stereoscopic image more natural to a stereoscopic vision observer, and can provide useful information in distance measurement of a obstacle or stereo image processing .

The control unit 120 may control the overall operation of the lane recognition apparatus 100. [ For example, the lane recognizing apparatus 100 can control various power driving units for driving.

According to an embodiment of the present invention, the control unit 120 performs processes relating to processing of images received from the camera module 110, lane recognition, and other operations. Also, according to the embodiment disclosed herein, the controller 120 may use the travel information collected by the sensor unit 150 for the lane recognition.

The lane recognition process of the controller 120 will be described in detail with reference to FIG. 2 to FIG.

The control unit 120 may control the position of the lane recognition device 100 (or the vehicle equipped with the lane recognition device 100) identified through an arbitrary GPS module (not shown) (Including warning message function, automatic lane keeping function, etc.) related to lane keeping based on the traffic information.

The storage unit 130 may store data and programs for the operation of the controller 120, and temporarily store input / output data.

According to an embodiment of the present invention, the storage unit 120 may temporarily store an image received through the camera module 110, processing information related to the image, and lane identification information. In addition, the storage unit 120 may store an arithmetic expression (e.g., a curve equation) for processing on the image.

In addition, according to the embodiment disclosed herein, the storage unit 120 may store the feature point or the lane fitting result extracted at an arbitrary time in the lane recognition process. Feature points or lane fitting results extracted from any stored past time can be used to perform the fit of the current lane.

In some embodiments, the storage unit 130 may include an operating system (not shown), a module (not shown) for performing a wireless communication function, a module (not shown) Software components may be stored, including a module (not shown) that operates in conjunction with an input unit, and a module that operates in conjunction with the output unit 140. The operating system (e.g., LINUX, UNIX, OS X, WINDOWS, Chrome, Symbian, iOS, Android, VxWorks or other embedded operating systems) provides various software for controlling system tasks such as memory management, power management, Components and / or drivers.

The storage unit 130 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or xD memory) A random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM) , A magnetic disk, or an optical disk. The lane recognition apparatus 100 may operate in association with a web storage that performs a storage function of the storage unit 130 on the Internet.

The output unit 140 is for generating output related to visual, auditory or tactile sense, and may include a display unit 141 and an audio output module 142.

The display unit 141 may output information processed by the lane recognition apparatus 100. [ For example, when the lane recognition apparatus 100 is running, a UI (User Interface) or a GUI (Graphic User Interface) associated with the driving can be displayed.

The display unit 141 may display an image obtained through the camera module 110 of the lane recognition device 100 and / or a lane recognized through the control unit 120, Can be displayed. The display unit 141 may display the image and information about the recognized lane at the same time, and may display the divided image up and down or left and right, or overlap and display the information about the recognized lane in the image.

The display unit 141 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) a flexible display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 141 may also be of a light transmission type.

There may be two or more display units 141 according to the embodiment of the lane recognition apparatus 100. [ For example, in the lane recognizing apparatus 100, a plurality of display units may be spaced apart from one another, or may be arranged integrally with each other, or may be disposed on different planes.

In addition, the display unit 141 may have a mutual layer structure with a touch sensor (not shown) for sensing a touch operation. In this case, the display unit 141 may be used as an input device in addition to the output device. The touch sensor may take the form of, for example, a touch film, a touch sheet, a touch pad, a touch panel, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific part of the display part 141 or a capacitance generated in a specific part of the display part 141 into an electrical input signal. The touch sensor may be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal is sent to the touch controller. The touch controller processes the signal and transmits corresponding data to the controller 120. [ Thus, the control unit 120 can know which area of the display unit 141 is touched or the like.

The audio output module 142 may output the audio data stored in the storage unit 130 in a recording mode, a voice recognition mode, and the like. The sound output module 142 outputs a lane recognition result (for example, notification of the type of the recognized lane or the like) performed by the lane recognition device 200 and a lane recognition function (e.g., An automatic lane keeping notification, etc.). The sound output module 142 may include a receiver, a speaker, a buzzer, and the like.

The sensor unit 150 is for collecting driving information of the lane recognition device 100 and may include a speed sensor 151 and a steering sensor 152. [

The speed sensor 151 detects the speed of the lane recognizing device 100. [ Since the gear ratio of the differential gear and the size of the tire are determined in the lane recognizing device 100, the speed sensor 151 can calculate the speed of the lane recognizing device 100 based on the speed of the transmission output shaft or the wheel . In addition, the speed sensor 151 may be configured of any one of a reed switch type, a magnetic type, and a hole type.

In addition, the speed sensor 151 can detect the acceleration of the lane recognizing device 100 based on the change in the speed of the lane recognizing device 100. Alternatively, the lane recognizing device 100 may include a separate acceleration sensor (not shown) for detecting the acceleration of the lane recognizing device 100.

The steering sensor 152 senses the steering movement of the lane recognizing device 100, that is, the wheel angle. Therefore, the steering sensor 152 is connected to the steering wheel of the lane recognizing apparatus 100, and detects the phase change due to the rotation of the rotor, the gear rotating integrally with the rotor, An operation unit for calculating and outputting an input of the sensing unit, and a PCB substrate and a housing for mounting the operation unit.

The lane recognition apparatus 100 may further include a communication unit (not shown) that performs a communication function with an arbitrary terminal or a server under the control of the control unit 120. [ At this time, the communication unit may include a wired / wireless communication module. Herein, the wireless Internet technology may be a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a World Interoperability for Microwave Access (HSDPA), IEEE 802.16, Long Term Evolution (LTE), Wireless Mobile Broadband Service (WMBS), and the like. The Short Range Communication Technology may include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like. The wired communication technology may include USB (Universal Serial Bus) communication, and the like.

The communication unit may include CAN communication, a vehicle Ethernet, a flexray, a LIN (Local Interconnect Network), or the like for communication with any vehicle provided with the lane recognition device 100. [

In addition, the communication unit may receive the image photographed through the arbitrary camera module from the arbitrary terminal or server. The communication unit may transmit lane identification information for an arbitrary image to the arbitrary terminal or server under the control of the control unit 120. [

The lane recognizing apparatus 100 shown in Fig. 1 is not necessarily an essential component, and the lane recognizing apparatus 100 can be realized by a constituent element that is more or less than the constituent elements shown in Fig.

2 is a flowchart illustrating a lane recognition process according to an embodiment disclosed herein.

Referring to FIG. 2, the lane recognition apparatus 100 acquires an image (s21).

The camera module 110 is mounted on the same lane recognition device 100 as the lane recognizing device 100. The lane recognizing device 100 includes a lane recognizing device 100, 1 image and the second image, or acquire an image photographed through a single camera. Here, the first image may be a left image captured by a left camera included in the pair of cameras, and the second image may be a right image captured by a right camera included in the pair of cameras . In addition, the camera module 110 may acquire any one of a first image and a second image captured through the pair of cameras.

For example, the camera module 110 may acquire an image 310 as shown in FIG. 3 through the single camera.

Then, the lane recognizing device 100 extracts feature points of the lane (s22).

The control unit 120 extracts a plurality of edge points 410 existing in the image as shown in FIG. 4 in order to distinguish the lane from the image 310 obtained from the camera module 110 . At this time, the controller 120 sets only the ROI of the lower part of the horizon as a region of interest (ROI), and extracts the feature points 410 only in the ROI.

Feature point 410 extraction may be performed through various algorithms.

For example, the control unit 120 may extract the feature point 410 based on the gradient information of the obtained image 310. That is, the controller 120 can recognize that the brightness or color intensity between neighboring pixels of the obtained image 310 is not the characteristic point 410 in a stepwise manner. On the contrary, when the brightness or color intensity between adjacent pixels changes abruptly, the controller 120 recognizes the brightness or color hue as a feature point 410 and extracts corresponding pixel information. In this case, the minutiae point 410 may be a discontinuous point for the boundary of the two regions in which the brightness of the pixel or the color intensity is distinguished.

Alternatively, the control unit 120 may extract the feature point 410 based on the color information of the obtained image 310. [ Typically, in a road, the normal lane is white, the center lane is yellow, and the non-lane part is black. Accordingly, the controller 120 can extract the feature point 410 based on the color information of the lane. That is, the controller 120 makes only an area having a color that can be classified into a lane from the image 310 as an object, and excludes only objects corresponding to the road as interest areas in order to exclude other objects running on the road The feature point 410 can be extracted from the object based on the color information in the region of interest.

In the above description, the controller 120 extracts the minutiae point 410. However, the minutiae point extractor 410 extracts the minutiae point 410 through various minutiae point extraction algorithms or filters such as Edge Tracing or Boundary Flowing algorithm. .

According to an embodiment of the present invention, the control unit 120 may store information on the extracted minutiae in the storage unit 130 and use it to fit the lane from the obtained image. That is, the controller 120 may first determine the offset of the lane based on the information of the extracted minutia points, and may perform the curve fitting on the current lane based on the determined offset value.

The curve fitting process of the current lane based on the information of the feature points extracted in the past will be described in detail below with reference to FIG. 8 to FIG.

5, the control unit 120 may convert the extracted plurality of feature points 410 into a world coordinate system after extracting the feature points 410. In this case, . At this time, the controller 120 may use a transformation matrix or a coordinate transformation equation. The transformation matrix may be a homographic maxtix stored in advance in the storage unit 130. In addition, the control unit 120 can easily detect an error occurring during coordinate conversion by keeping the vertical and horizontal intervals of the plurality of feature points 510 converted to the world coordinate system the same.

Thereafter, the lane recognition device 100 fits the lane (s23).

6, the control unit 120 performs lane fitting to represent the extracted plurality of feature points 510 as one line 610. [0054] FIG. The control unit 120 may be any of a least square method, a random sample consensus (RANSAC) method, a general hough transform method, and a spline interpolation method to extract a straight line or a curve from an image. One method can be used.

According to an embodiment of the present invention, when the extracted plurality of feature points 510 correspond to a curve, the controller 120 may perform lane fitting based on a curve equation. Specifically, the control unit 120 may substitute the extracted plurality of feature points 510 into a curve equation to obtain coefficients thereof, and perform curve fitting based on the result of the calculated curve equation. At this time, the curve equation is stored in the storage unit 130 and may be any arbitrary multi-dimensional equation.

For example, when the curvilinear equation is a quadratic equation, the controller 120 multiplies the plurality of feature points 510 by a quadratic equation, for example, y = ax ² + bx + c (where a is a curvature, b The heading angle, and C is the offset) to perform the curve fitting. At this time, if the substitution result a is 0, the controller 120 recognizes it as a straight line, and if a is not 0, it can recognize it as a curve.

In another example, when the curve equation is a cubic equation, the controller 120 multiplies the plurality of feature points 510 by a cubic equation, for example, y = ax ³ + bx ² + cx + d, Curve derivative, b is curvature, c is the heading angle, and d is offset). When a is 0, b is the curvature, c is the heading angle, d is the offset, and when both a and b are 0, the straight line detection, c is the heading angle, and d is the offset.

Also, the controller 120 may store the fit result in the storage unit 130 and use the fit result for the next lane fit.

According to an embodiment of the present invention, the controller 120 may perform a curve fit for the current lane based on past feature points or past lane adaptation results.

If the road changes severely due to a curvature curved road having a dotted line, for example, the road is cut off and restarted from the front of the vehicle, the accuracy of the offset may deteriorate when the curve is fitted. Accordingly, the controller 120 may first determine the offset of the lane based on the extracted feature points or past lane adaptation results, and may perform the curve fitting on the current lane based on the determined offset value.

The curve fitting process of the current lane based on the past lane-fitting result will be described in detail below with reference to Figs. 8 to 13.

Finally, the lane recognizing device 100 tracks the lane (s24).

The control unit 120 performs tracking based on the plurality of feature points 510 corresponding to the adapted lane in order to reduce the calibration time and reduce the noise. At this time, the calibration means calculating the conversion relation between the camera module 110 and the world coordinate system.

According to the embodiment disclosed herein, the control unit 120 may perform tracking on all of the plurality of lanes that are suitable. That is, the control unit 120 may track the surrounding lanes existing on the traveling road in addition to the lane in which the lane recognition apparatus 100 is running.

Accordingly, even when the lane recognizing apparatus 100 changes the lane, the control unit 120 can quickly recognize the lane from the existing track information without having to track the lane newly.

In addition, even when some lanes are missed, lost, or a video is missed due to a temporary operation failure of the camera module 110, the control unit 120 may also store tracking information for a plurality of lanes (e.g., Width, curve equation of curved lane, etc.), the lane can be estimated and recognized.

In this case, the control unit 120 may store the tracking result in the storage unit 130. Accordingly, the control unit 120 can correct the error based on the stored tracking result even when some errors occur in the lane fitting.

According to an embodiment of the present invention, the control unit 120 may store the tracking result in the storage unit 130 and may use the obtained result to fit the lane from the obtained image. That is, the controller 120 may first determine the offset of the lane based on the past lane-tracking result, and may perform curve fitting on the current lane based on the determined offset value.

In addition, the lane recognition device 100 can display the lane-following result (s25).

The output unit 140 may display the tracked result through the lane recognition process as shown in FIG.

The output unit 140 displays the image 310 obtained through the camera module 110 of the lane recognition device 100 and / or the tracking result 710 of the lane recognized through the control unit 120 can do. The output unit 140 displays the image 310 and the tracked result 710 of the recognized lane at the same time and divides the image 310 up or down or right and left to display the image 310. The output unit 140 overlaps information on the recognized lane Can be displayed.

Alternatively, the output unit 140 may output the lane recognition result (for example, a notification about the type of the recognized lane or the like) and a function related to the lane recognition (for example, a lane departure warning, It is possible to output a sound signal.

In the following, the lane fitting process, particularly the curve fitting process, in the lane recognition process will be described in more detail.

8 is a flowchart illustrating a curve fitting process according to the embodiment disclosed herein.

Referring to FIG. 8, the lane recognition apparatus 100 obtains minutia information extracted at a past time (s231).

According to the embodiment of the present invention, when the extracted plurality of feature points 510 correspond to a curve, the controller 120 may perform lane fitting using a curve equation. In this case, if the road changes severely, for example, the curved road having a curvature is formed by a dotted line and the road is cut off and restarted from the front of the vehicle, the accuracy of the offset of the recognized lane may be significantly deteriorated.

Here, the offset indicates whether the lane is located on the left side or the right side with respect to the lane recognition device 100, and refers to a value corresponding to a constant (coefficient of the first-order term) when the lane is expressed by an arbitrary equation. That is, when the constant has a positive value, the lane is located on the right side of the lane recognition device 100, and when the constant has a negative value, the lane is located on the left side of the lane recognition device 100 .

For example, as shown in FIG. 9, the lane recognizing apparatus 100 can travel on an actual lane 910 having a large curvature and a dotted line. The actual lane 910 is divided into a broken lane 901 and a lane 902 because the actual lane 910 is a dotted line. Since the lane recognizing apparatus 100 is running, the lane recognizing apparatus 100 is located at t-1 at an arbitrary time in the past and at t at the present time.

10, since there are only a plurality of feature points 510 extracted from the area 902 in which the lane exists, when the lane fitting is performed at the current point of time t, The control unit 120 performs the lane adaptation based on this. Accordingly, the controller 120, which recognizes the plurality of feature points 510 extracted from the curved road with a large curvature as a curve, performs the lane fitting based on the curve equation. As a result, the recognition result for the lane is curved Gt; 920 < / RTI > In this case, although the lane recognition can be accurate for the area 902 in which the lane exists, the lane recognition becomes inaccurate for the area 901 in which the lane with a large curvature is broken. 10, an offset of the curve fitting result 920 lane exists on the left side of the lane recognizing device 100 with respect to the actual lane 910 existing on the right side of the lane recognizing device 100 (the origin point of Fig. 10) An inaccurate offset occurs, thereby making safe running difficult.

Accordingly, the controller 120 may obtain the feature point 520 information extracted at the previous time point t-1 as shown in FIG. 11 (a) to find an accurate offset. The feature point 520 information may be obtained from the storage unit 130 as an extracted at an arbitrary time in the past, that is, t-1, when the current time is t. The past arbitrary point of time can be predetermined or can be changed in a fluid manner so that the lane recognition device 100 can prevent the lane of the road on which it is traveling from being too long from the past.

When the control unit 120 obtains the feature point 520 information at the current point of time t, the original position information remains as shown in FIG. 11 (b), so that the current point of time can not be used to fit the lane. Therefore, the controller 120 estimates the minutia information 520 as a correct value for the current time t and corrects the information.

Therefore, the lane recognizing device 100 estimates the changed position of the minutiae based on the travel information (s232).

Since the lane recognizing apparatus 100 in motion is running on the road, the minutia information 520 should be estimated to be a correct value for the current point of time t.

For example, when the lane recognizing apparatus 100 travels on a curved road as shown in Fig. 9, the lane recognizing apparatus 100 obtains an image obtained at a past time t-1 by the lane recognizing apparatus 100 (The result of the decomposition of the curved motion), and is moved to the opposite side and backward with respect to the current time point t.

Accordingly, the controller 120 may estimate the changed position of the feature point 520 in order to correctly correct the position of the feature point 520 at the time point t-1 with respect to the current point of time t.

At this time, the control unit 120 can estimate the changed position based on the traveling information of the lane recognition device 100 collected through the sensor unit 150. [ The traveling information may include speed information, acceleration information, or steering motion information of the lane recognition device 100 collected from the sensor unit 150. [

The control unit 120 can determine whether the feature point 520 has changed in the fore-and-aft direction or in the lateral direction based on the collected travel information. In addition, the control unit 120 may change the feature point 520 to the estimated position 520 'based on the determination result, as shown in FIG.

Thereafter, the lane recognizing device 100 determines an offset based on the estimated feature point (s233).

The controller 120 determines a constant indicating the offset of the lane among the coefficients of the curve equation based on the estimated feature point 520 '.

For example, when curve fitting is performed using a quadratic equation, i.e., y = ax ² + bx + c, for a curve that is a curve, the c value indicating the offset of the lane is calculated as the feature point 520 ' As shown in FIG.

Accordingly, even when the actual lane 910 having a large curvature is formed by a dotted line and it is difficult to estimate an accurate offset due to a break in the front of the lane recognizing apparatus 100, the control unit 120 determines that the lane- It is possible to accurately determine the offset of the lane based on the travel information of the recognition apparatus 100. [

According to the embodiment disclosed herein, the controller 120 may determine the offset based on the lane adaptation result at the past time point rather than the estimated feature point 520 '. For example, the control unit 120 estimates a change in c value based on the running information by taking a value c representing an offset of the lane in an equation representing a lane recognized from the lane-fitting result at the past time point, The value can be determined as the offset at the current point in time.

Finally, the lane recognizing apparatus 100 performs curve fitting at the present time (s234).

The control unit 120 may calculate a residual coefficient of the curve equation by performing a curve fitting on the lane at the current time point of the lane on the basis of the determined offset.

For example, when curve fitting is performed using a two-dimensional equation, that is, y = ax ² + bx + c, for a curved road at the current point, the c value is a feature point 520 ', and determines the values of the remaining constants a and b by performing a curve fit based on the extracted plurality of feature points 510. [0050] FIG.

13, even if a portion of the curve fitting result lane is cut off or lost, and the lane located on the right side of the lane recognizing apparatus 100 is erroneously recognized as being located on the left side, 120 may determine the exact offset of the actual lane 910 using the minutiae information 520 extracted from the past time (see FIG. 9). Also, the lane adaptation result 930 at the current time point can be accurately recognized as a curved line based on the correctly determined offset.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: lane recognition device 110: camera module
120: control unit 130:
140: output unit 150: sensor unit

Claims (19)

A camera module for acquiring an image;
A controller for extracting a plurality of feature points from the obtained image, performing lane fitting to connect the extracted plurality of feature points with one line, and tracking the adapted lane; And
And a display unit for displaying the tracked lane,
The lane-
Obtaining an offset indicating a lateral deviation of a lane of a lane adaptation result at an arbitrary past time point and estimating an offset of a current time point with respect to an offset based on the traveling information from the arbitrary past time point, And performs curve fitting based on the offset of the current point of time. 2. The method of claim 1,
Wherein the coefficients of the curve equation are determined based on a curve equation having an arbitrary dimension. The information processing apparatus according to claim 1,
Speed information, acceleration information, and steering information. The camera module according to claim 1,
And at least one pair of cameras or a single camera provided at equal intervals on the same central axis of the same plane. The image processing apparatus according to claim 1,
Wherein only the road portion corresponding to the lower end of the horizon in the obtained image is set as a region of interest and extracted only for the region of interest. The image processing apparatus according to claim 1,
Wherein the extracted lane recognition information is extracted based on gradient information or color information of the obtained image. The apparatus of claim 1,
Converts the extracted plurality of feature points into a world coordinate system, and fits the lane based on the converted plurality of feature points. 2. The method according to claim 1,
Wherein the lane identification is performed with respect to all of the adapted lanes. The method according to claim 1,
And a storage unit for storing information on the extracted minutiae,
Wherein,
Wherein the lane recognizing unit matches the lane based on the information about the minutiae stored in the storage unit. delete Acquiring an image;
Extracting a plurality of feature points from the acquired image;
Performing lane fitting to connect the extracted plurality of feature points by one line;
Tracking the adapted lane; And
And displaying the tracked lane,
Wherein performing the lane fitting comprises:
Obtaining an offset indicative of a lateral deviation of a lane of lane fitting result at any past time point;
Estimating an offset of the current point of time with respect to the offset based on the traveling information from the arbitrary past point of time; And
And performing a curve fitting based on the estimated offset of the current time point. 12. The method of claim 11, wherein performing curve fitting comprises:
Determining coefficients of the curvilinear equation based on a curvilinear equation having an arbitrary dimension. 12. The method according to claim 11,
Speed information, acceleration information, and steering information. delete The method of claim 11, wherein the extracting of the plurality of feature points comprises:
Setting only a road portion corresponding to a lower end of the obtained image centered on the horizon as a region of interest; And
And extracting a plurality of minutiae only for the area of interest. 12. The image processing apparatus according to claim 11,
Wherein the extracted lane recognition information is extracted based on gradient information or color information of the obtained image. 12. The method of claim 11, wherein fitting the lane comprises:
Converting the extracted plurality of feature points into a world coordinate system; And
And fitting the lane based on the converted plurality of feature points. 12. The method of claim 11, wherein tracking the lane comprises:
Wherein the step of performing the lane recognition is performed for all of the adapted lanes. 12. The method of claim 11, wherein extracting the feature points comprises:
Further comprising storing information on the extracted feature points,
The step of fitting the lane comprises:
And wherein the lane recognition unit fits the lane based on the information about the stored feature points.

Images