KR20220063433A - Method and apparatus for recognizing and tracking lane using avm image, vehicle data and map information - Google Patents

Abstract

The present invention relates to a method and apparatus for recognizing and tracking a lane using an AVM image, vehicle data, and map information. According to one embodiment of the present invention, the method for recognizing and tracking the lane using the AVM image, vehicle data, and map information comprises: a step of generating an around view monitoring (AVM) image around a vehicle through calibration and matching of a camera installed in the vehicle; a step of recognizing a lane through processing and fitting of the generated AVM image; and a step of tracking the lane by fusing the extended Kalman filter-based AVM image and vehicle data.

Description

Lane recognition and tracking method and apparatus using AVM image, vehicle data and map information

The present invention relates to a method and apparatus for lane recognition and tracking using AVM images, vehicle data, and map information.

A camera sensor commonly used for road environment recognition has the advantage of being able to detect an environment far away from the vehicle in the driving direction. However, there are disadvantages in that it is sensitive to changes in the sun or artificial lighting depending on the installation location and angle, and the lane recognition result is greatly affected by image distortion caused by a vehicle heading or pitch change. In addition, since the lane recognition result in front of the vehicle is extended to the vehicle to derive the position within the lane of the vehicle, a greater error in lane recognition may occur.

Embodiments of the present invention recognize a linearized lane model using an AVM (Around View Monitoring) image, vehicle data, and map information, reduce a processing load through an extended Kalman filter, and perform stable lane tracking, an AVM image, An object of the present invention is to provide a method and apparatus for lane recognition and tracking using vehicle data and map information.

However, the problem to be solved by the present invention is not limited thereto, and may be variously expanded in an environment within the scope not departing from the spirit and scope of the present invention.

According to an embodiment of the present invention, there is provided a lane recognition and tracking method, comprising: generating an AVM (Around View Monitoring) image around a vehicle through calibration and matching of a camera installed in the vehicle; recognizing a lane through the generated AVM image processing and fitting; and tracking the lane by fusing the extended Kalman filter-based AVM image and vehicle data, the lane recognition and tracking method using the AVM image, vehicle data, and map information may be provided.

The disclosed technology may have the following effects. However, this does not mean that a specific embodiment should include all of the following effects or only the following effects, so the scope of the disclosed technology should not be understood as being limited thereby.

Embodiments of the present invention recognize a linearized lane model using an AVM image, fuse it with lane data using an extended Kalman filter, and reduce processing load and perform stable lane tracking through an extended Kalman filter that converges two sensors. can Embodiments of the present invention can secure a processing speed of 10 ms or less from lane recognition to tracking.

Embodiments of the present invention can help to improve technical robustness and precision, such as recognition of the surrounding environment and location estimation of the own vehicle, which are major technologies of an autonomous vehicle.

1 is a flowchart illustrating a lane recognition and tracking method using an AVM image, vehicle data, and map information according to an embodiment of the present invention.
2 is a diagram illustrating a process of generating a top view image around a vehicle according to an embodiment of the present invention.
3 is a diagram illustrating a lane recognition process according to an embodiment of the present invention.
4 is a diagram illustrating a lane tracking process according to an embodiment of the present invention.
5 is a block diagram illustrating an apparatus for recognizing and tracking a lane using an AVM image, vehicle data, and map information according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it can be understood to include all transformations, equivalents or substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention, precedent, or emergence of new technology of those of ordinary skill in the art. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present invention, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of addition or existence of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. do.

1 is a flowchart illustrating a lane recognition and tracking method using an AVM image, vehicle data, and map information according to an embodiment of the present invention.

As shown in FIG. 1 , in step S101, the lane recognition and tracking apparatus according to an embodiment of the present invention performs AVM image registration. The lane recognition and tracking device generates a top view image around the vehicle through calibration and matching of cameras installed at four front, rear, left, right, and right places of the vehicle to generate an AVM image.

In step S102, the lane recognition and tracking device performs lane recognition. The lane recognition and tracking apparatus performs image processing and fitting for lane recognition.

In step S103, the lane recognition and tracking device performs lane tracking. The lane recognition and tracking device fuses the AVM image and vehicle data based on the extended Kalman filter for lane tracking.

2 is a diagram illustrating a process of generating a top-view image around a vehicle according to an embodiment of the present invention.

As shown in FIG. 2 , the lane recognition and tracking apparatus according to an embodiment of the present invention uses the internal and external calibration of cameras installed at four front, rear, left, right, and right places of the vehicle to generate an AVM image, and four image matching. Creates a top-view image around the vehicle.

3 is a diagram illustrating a lane recognition process according to an embodiment of the present invention.

Looking at the lane recognition process with reference to FIG. 3 , the lane recognition and tracking apparatus according to an embodiment of the present invention applies a Sobel filter that is an edge detection filter to derive candidate points of a lane from the AVM original image. The noise component is removed by using the lane width information specified in the Road Traffic Act. The lane recognition and tracking apparatus uses a random sample consensus (RANSAC) algorithm that is robust to outliers by using the lane center candidate points derived through the preprocessing process to recognize the final lane model.

The lane recognition and tracking device obtains the left and right lane candidate models and uses the heading and lane width data of the current driving lane among the information on the precision map as the final lane determination criteria. Confirm the model as a result of recognition.

In an embodiment of the present invention, a linear model crossing the center of the left and right straight line models is required in order to correct a lateral position error using a precision map and reduce the complexity and computational resources of the lane tracking model. The lane recognition and tracking device can be obtained as follows [Equation 1], which is obtained through the conditional expression that the distance between two straight lines that do not belong to one plane and do not coincide with each other is the same.

4 is a diagram illustrating a lane tracking process according to an embodiment of the present invention.

Referring to the lane tracking process with reference to FIG. 4 , the lane recognition and tracking apparatus according to an embodiment of the present invention detects a change in the position of a lane on the vehicle coordinate system through a change in the position and attitude of the vehicle because the lane is fixed to the road surface. can be estimated The model expression of the extended Kalman filter can be expressed by applying the vehicle movement change amount as shown in Equations 2 and 3 below and FIG. 4 through the vehicle CAN (Controller Area Network) data updated at a rate of 100Hz. .

는 후륜 좌, 우 휠에서 추정된 속도의 평균값,

는 편향 오차가 제거된 요레이트(yawrate),

는 샘플링 시간을 의미한다.here

is the average value of the estimated speeds from the rear left and right wheels,

is the yaw rate at which the bias error is removed,

is the sampling time.

5 is a block diagram illustrating an apparatus for recognizing and tracking a lane using an AVM image, vehicle data, and map information according to an embodiment of the present invention.

As shown in FIG. 5 , the lane recognition and tracking apparatus 100 using an AVM image, vehicle data and map information according to an embodiment of the present invention includes a camera 110 , a memory 120 , and a processor 130 . include However, not all illustrated components are essential components. The lane recognition and tracking apparatus 100 may be implemented by more components than the illustrated components, and the lane recognition and tracking apparatus 100 may be implemented by fewer components.

Hereinafter, a detailed configuration and operation of each component of the lane recognition and tracking apparatus of FIG. 5 will be described.

The camera 110 is installed in four places on the front, rear, left, and right of the vehicle to capture images around the vehicle.

The memory 120 stores one or more programs.

The processor 130 executes one or more programs stored in the memory 120 . The processor 130 generates a top-view image around the vehicle through calibration and matching of cameras installed at four front, rear, left, right, and right places of the vehicle to generate an AVM image, performs image processing and fitting for lane recognition, and performs lane tracking It fuses the extended Kalman filter-based AVM image and vehicle data for

On the other hand, an embodiment of the present invention performs stable and continuous lane recognition and tracking to improve the performance and safety of autonomous driving technology, and by utilizing this, it can be expected to improve the lateral position of the vehicle.

Meanwhile, according to an embodiment of the present invention, the various embodiments described above are implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). can be The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device A) according to the disclosed embodiments. When the instruction is executed by the processor, the processor may perform a function corresponding to the instruction by using other components directly or under the control of the processor. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

In addition, according to an embodiment of the present invention, the methods according to the various embodiments described above may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

In addition, according to an embodiment of the present invention, the various embodiments described above are stored in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. can be implemented in In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing the processing operation of the device according to the above-described various embodiments may be stored in a non-transitory computer-readable medium. The computer instructions stored in the non-transitory computer-readable medium, when executed by the processor of the specific device, cause the specific device to perform the processing operation in the device according to the various embodiments described above. The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, a cache, a memory, and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, each of the components (eg, a module or a program) according to the above-described various embodiments may be composed of a singular or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other Sub-components may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity, so that functions performed by each corresponding component prior to integration may be performed identically or similarly. According to various embodiments, operations performed by a module, program, or other component are executed sequentially, parallel, iteratively, or heuristically, or at least some operations are executed in a different order, are omitted, or other operations are added. can be

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and is commonly used in the technical field pertaining to the present disclosure without departing from the gist of the present invention as claimed in the claims. Various modifications are possible by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: lane recognition and tracking device
110: camera
120: memory
130: processor

Claims (1)

A lane recognition and tracking method comprising:
generating an AVM (Around View Monitoring) image around the vehicle through calibration and matching of a camera installed in the vehicle;
recognizing a lane through the generated AVM image processing and fitting; and
A lane recognition and tracking method using AVM image, vehicle data, and map information, comprising the step of tracking a lane by fusing the extended Kalman filter-based AVM image and vehicle data.

Images