KR20150061445A - Apparatus for building 3D Load map and recognizing lane - Google Patents

Abstract

The present invention relates to an apparatus for building a three-dimensional map and recognizing a lane. According to an embodiment of the present invention, the apparatus for building a three-dimensional map and recognizing a lane comprises: a laser scanner for collecting distance data and angle data respectively; a data processing unit for generating two-dimensional information using the angle data and distance data; a map building unit for building a three-dimensional map by combining the two-dimensional information and information such as a location, a position and the like of an actuator; a lane characteristic point extraction unit for extracting characteristic points of a lane from the three-dimensional map; and a curvature estimation unit for estimating a curved path through a curvature algorithm when a road lane is blocked.

Description

Technical Field [0001] The present invention relates to a three-

More particularly, the present invention relates to a three-dimensional road map creation and lane recognition apparatus capable of recognizing a lane while generating a three-dimensional road map by a driveable drive body .

Over the past several decades, automotive safety systems have evolved significantly. Starting with the seat belt in the 1960s, it was mainly influenced by electrical systems such as air bag, anti-lock brake system (ABS) and electric power steering (EPS) in order to increase the safety of passengers. However, since these systems operate according to the internal state of the vehicle system, they provide only passive safety. Recently, the United States and Japan have been conducting a lot of researches to build intelligent and luxurious systems in order to lead a more comfortable and affluent life in developed countries.

This system is called an Advanced Safety Vehicle (ASV). The ASV is a system that prevents the accident of the driver beforehand and informs the driver of the risk of the accident so that the driver actively copes. This is an artificial intelligent advanced safety vehicle that maximizes driver's stability and convenience by applying various intelligent safety technologies to vehicles.

ASV's core technologies are the collision warning system and the lane recognition system. The vehicle collision warning system is a technology that notifies the driver when there is a possibility of a collision due to the front-to-rear distance sensor and the distance detection of the front and rear cars. The lane departure warning system checks the driving lane of the car with the lane departure detection sensor and informs the driver of the lane departure when the lane departure is expected. Vision System, LRF, etc. are mainly used to recognize the lane by ultrasonic sensor and RADAR to recognize the obstacle ahead in ASV, and to recognize the road situation such as traffic light. Based on the perceived information, it is used for obstacle recognition, mapping, and driving. It is also used for position estimation and collision avoidance, route tracking, lane recognition in driving.

One of the core technologies of ASV is lane recognition, which is based on Vision System and laser sensor technology. The Vision System has the advantage that it can extract information of abundant amount of roads with low cost equipment. However, since it is very sensitive to the illumination conditions, it is very difficult to process due to the saturation of the light caused by the light or the difference of the illumination. In the previous research, various approaches such as hough, template matching, splines, and polynomial estimation have been proposed based on the characteristics of various roads in vision-based lane recognition. In order to compensate for this problem, we have been studying road modeling considering the characteristics of horizontal and vertical clothoid curves, polynomial curves and splines. However, this method is also sensitive to noise or false detection, And so on. In addition, since it is necessary to keep the number of frames of image more than a certain number in order to stably recognize, it is a great burden to maintain sophisticated control and stable frame number. And, like the misty weather, the Vision System is also influenced by the weather. Since dense fogs obscure the vision of the Vision System, it is difficult to judge the road information, and it is also difficult to recognize the lane.

KR 10-0440387 B1

SUMMARY OF THE INVENTION It is an object of the present invention to provide a three-dimensional road map creation and lane recognition apparatus for recognizing a lane without being influenced by the surrounding environment and running the lane.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

According to an aspect of the present invention, there is provided a three-dimensional road map generating and lane recognizing apparatus including a laser scanner for collecting distance data and angle data, A lane feature point extracting unit for extracting feature points of a road lane from a three-dimensional map, a lane feature point extracting unit for extracting feature points of a road lane from the three-dimensional map, And a curvature estimator for estimating a curved path through the cover-collapse algorithm when the frame collapses.

According to the present invention, a three-dimensional road map making and lane recognizing device obtains two-dimensional information through a laser scanner and combines information such as the position and attitude of the driving body with an IMU (Inertia Measurement Unit) A map is formed in which the asphalt is higher than the lane by the reflectance of the asphalt and the lane of the road in the three-dimensional map, and the lane can be recognized by extracting the characteristic points of the lane.

Also, in the case of a dotted lane, it is advantageous that the driver can predict the broken portion of the dotted lane by using the cover-shooter algorithm (curvature detection algorithm) so that the driver can drive the straight lane, the dotted lane, and the dotted lane.

1 is a diagram illustrating a scanning method of an LRF according to an embodiment of the present invention.
2 is a diagram for explaining coordinate transformation of map data according to an embodiment of the present invention;
3 is a view showing a three-dimensional road environment map according to an embodiment of the present invention;
4 is a view showing a dotted lane map according to an embodiment of the present invention;
Fig. 5 is a diagram showing characteristic points of a dotted lane including noises in a dotted lane map; Fig.
6 is a view for explaining feature point noise elimination in a dotted lane according to an embodiment of the present invention;
FIG. 7 is a diagram for explaining minutiae extraction of a lane according to an embodiment of the present invention; FIG.
FIG. 8 is a diagram for explaining curve estimation using a covercher algorithm according to an embodiment of the present invention; FIG.
9 is a diagram for explaining a system configuration according to an embodiment of the present invention.
10 is a view showing a structure of a robot according to an embodiment of the present invention.
11 illustrates a structure of a tracking controller according to an embodiment of the present invention.
12 is a diagram showing a result of a lane-following test according to an embodiment of the present invention.
13 shows a tracking error according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described below, but may be embodied in various different forms, and these embodiments are not intended to be exhaustive or to limit the scope of the present invention to the precise form disclosed, It is provided to inform the person completely of the scope of the invention. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

SUMMARY OF THE INVENTION An object of the present invention is to recognize a lane without being influenced by the surrounding environment so as to drive the vehicle. For this purpose, in the present invention, two-dimensional information is obtained through a laser scanner, and a three-dimensional map is formed by combining information such as position and attitude of an actuator through an IMU (Inertia Measurement Unit) and an encoder, In the map, asphalt and lane reflectance of the road are used to form a map with the asphalt rising above the lane to extract the feature points of the lane and recognize the lane. In the case of a dotted lane, a broken portion of a dotted line is predicted using a covercher algorithm (curvature detection algorithm) so that the driver can drive a straight lane, a dotted lane, and a dotted lane.

In short, the present invention is characterized in that: (1) a traveling robot provided with a laser scanner (LRF), an IMU and an encoder; 2) a lane feature point extracting unit for extracting feature points of a road lane; and 3) And a curvature estimator for estimating a curvature of the image.

On the other hand, IMU is an electronic device that measures acceleration and rotational motion and measures the posture and direction of a drivable body such as a mobile robot.

The point of the present invention is to recognize a lane by making a three-dimensional map so that the road can be traveled even when there is no foggy weather or a curb of the road. In the existing study, when using the camera, the weather system is affected by the weather as well as the weather with heavy fog. Since dense fogs obscure the vision of the Vision System, it is difficult to judge the road information, and it is also difficult to recognize the lane. In addition, it is difficult to cope with various situations of the road when the laser scanner recognizes the curb of the road and moves along the curb. For example, it is difficult to travel because there are many curved roads, such as no curbs, roads, lane-length roads, road width standards and multiple roads. In order to solve these disadvantages, it is necessary to recognize the lane without any influence of the environment and to drive the lane existing anywhere on the road.

In the present invention, the information coming into the system through the laser scanner is two-dimensional information. This is combined with information such as the position and posture of the robot, and then processed into a three-dimensional map. In this 3D map, asphalt and lane reflections are generated for the road asphalt. Here, by extracting the characteristic points of the lane and recognizing the lane, it is possible to travel along the lane. In the case of a dotted line lane, a broken lane of a dotted line can be predicted using a coverer algorithm, so that a straight lane, a dotted lane, and a dotted line lane can be driven. Although it is possible to create a map in two dimensions, there is a difficulty in processing data in a complex manner by using various filters in order to process a noise in a two-dimensional map. On the other hand, when the map is three-dimensionally formed, the height at which the lane is formed is limited, and the noise is generated mainly by deviating from the lane height, so that the noise can be easily removed.

1. 3D road environment map

1.1 Three-dimensional mapping

In the embodiment of the present invention, the laser scanning sensor used in the mobile robot provides distance and angle data to the data processing unit. The data processing unit forms the two-dimensional map by forming the X and Y coordinates using the angle data and the distance data as shown in Equation (6).

(1)

,

,

는 LRF와 물체 사이의 거리고,

는 LRF의 스캔하는 점의 각도를 나타낸다. LRF는 0~240도를 스캔 할 수 있으며 0도에서부터 순차적으로 0.36도씩 회전하며 LRF 주위를 스캔한다.(1)

Is the distance between the LRF and the object,

Represents the angle of the scanning point of the LRF. LRF can scan from 0 to 240 degrees, and it rotates from 0.degree. To 0.36 degrees sequentially and scans around LRF.

는 0도 에서부터 0.36도씩 누적되어 가는 스캔 각도를 표현한 것이다. 이렇게 얻어진 2차원 스캔 좌표 데이터를 이용하여 2차원 지도를 만들 수 있다. 1 is a diagram illustrating an LRF sequentially scanning an object. In Fig. 1, A1, A2, and A3 indicate that the laser is emitted from the LRF and reflected back. And

Represents the scan angle accumulated from 0.degree. To 0.36deg. A two-dimensional map can be created using the obtained two-dimensional scan coordinate data.

FIG. 2 shows coordinate conversion of two-dimensional scan data of the road scanned by the LRF and scan data through the moving coordinates of the mobile robot. X0, Y0, and Z0 denote relative coordinate axes that serve as a reference for creating a map. X1, Y1, and Z1 are the movement coordinate axes of the mobile robot. The change of the mobile coordinates and the posture of the mobile robot changes the mobile coordinate axis of the mobile robot. In order to stack the measured data in the LRF, the LRF data must be coordinate based on X0, Y0, and Z0. In order to coordinate the LRF data on the reference coordinate axis, coordinate conversion of a rectangular coordinate system such as (Equation 2) is used. It is possible to create a road environment map in three dimensions by laminating the coordinates of the LRF obtained through the coordinate transformation.

(2)

,

,

는 모바일 로봇의 좌표이며,

,

는 LRF가 스캔한 데이터의 좌표이다. 그리고

는 X0을기준으로 변화된 모바일 로봇의 이동좌표의 회전각도를 의미한다.

,

,

는 지도를 구성할 점들의 좌표다. (수학식 2)을 통해 변환된 LRF 데이터의 좌표점은 계속적인 적층을 통해 3차원의 지도를 만들어 낸다.(2)

,

,

Is the coordinates of the mobile robot,

,

Is the coordinate of the data scanned by the LRF. And

Represents the rotation angle of the mobile coordinates of the mobile robot changed with reference to X0.

,

,

Is the coordinates of the points that make up the map. The coordinate points of the LRF data transformed through (Equation 2) produce a three-dimensional map through continuous stacking.

3 is an image obtained by restoring scanned data using LRF to a three-dimensional map image using a three-dimensional map algorithm. The test environment at the upper left is represented by a three-dimensional map on the right side, and the lower left figure is a view showing a lane.

The left-hand side of FIG. 4 is a dotted line lane of an intersection, which can be represented by a three-dimensional road map as shown on the right.

In FIG. 5, although the lane can be found by using the feature point algorithm of the lane, noise is generated as shown in the left part of the above drawing. Generally, this noise is generated in a range of a height higher than a lane characteristic point or a range of a lower height. Noise can be removed easily by arranging data outside the minutiae height range of the lane.

6, only the lane data can be obtained by removing the noise shown in the left drawing as shown in the right drawing. In the case of 2-D, it is necessary to use various complicated filters, so there is a limitation in the calculation processing in real time. However, if this method is used, it can be suitable for real-time processing because the calculation processing is simple.

2. Detection of lane

In order to determine the lane that is the boundary between the lane and the asphalt, it is necessary to extract the corner points of the lane lower than the road based on the road surface. (Equation 3) is used to obtain the boundary points.

(3)

,

,

는 도로의 횡방향 좌표고,

는 도로의 높이 좌표를 나타낸다. (수학식 3)을 통해 두 점 사이의 높이 차와, 두 점의 기울기를 구할 수 있다. 높이 차가 있고, 일정 범위를 벗어난다는 것은 지형의 변화가 있다는 것이고, 두 점의 기울기가 일정 범위를 벗어난다는 것은 변화가 심한 부분 즉 모서리이기 때문에 특이점이 될 수 있다. 이 식을 통해 차선과 아스팔트의 경계가 되는 모서리점을 보다 용의하게 얻을 수 있다. (수학식 3)에서

는 특이점이 될 수 있는 두 점 사이의 최소 높이 차를 의미하며,

는 특이점이 될 수 있는 두 점 사이의 최소 각도를 의미한다.(3)

Is the lateral coordinate of the road,

Represents the height of the road. (Equation 3), the height difference between two points and the slope of two points can be obtained. There is a difference in height, and a deviation from a certain range means that there is a change in the terrain, and the deviation of the two points from a certain range may be a singularity because the change is a serious part or edge. Through this equation, it is possible to obtain the edge point which is the boundary between the lane and the asphalt more easily. (3)

Means the minimum height difference between two points that can be a singular point,

Means the minimum angle between two points that can be a singular point.

The upper left diagram in Fig. 7 is the experimental environment of the misty road. The lower left of FIG. 7 is a view showing a result of processing the image by the vision method. The right side is the 3D environment map scanned by LRF in the experimental environment.

3. Control algorithm

3.1 Dotted Curve Lane Estimation Using Coverage Algorithm

If the lane is a dotted line, the part where the lane is not drawn should be estimated. The cover - up algorithm was used to estimate the area where the lane was not drawn. Generally, the cover algorithm uses the distance and the rotation angle of two points when it knows the current position and the final position, and creates the radius of curvature that the robot will move.

 In the embodiment according to the present invention, when a broken lane is generated, the tendency of the lane is grasped by using a part of the recent lane, and the curvature trajectory is created by using the influence.

The coverage algorithm uses the distance between the two points and the attitude at the current position and the attitude at the target point to determine the curvature radius when knowing the current position and the target point.

(4)

,

,

은 현재 위치를 나타내며 현재 직교좌표의 x축 위치를

, y축 위치를

이라 한다. 그리고 현재 로봇이 바라보는 방향을

이라 한다. 목표점의 위치는

로 나타낸다.(Equation 4) represents the wedge in terms of orthogonal coordinates and angles.

Represents the current position and the x-axis position of the current Cartesian coordinate

, the y-axis position

Quot; And the current direction of the robot

Quot; The position of the target point is

Respectively.

는 목표점에서 로봇이 바라보는 방향을 나타낸다.

Represents the direction in which the robot looks at the target point.

(5)

,

,

(Equation 5) to calculate the distance and rotation angle of the two points.

(6)

(6). ≪ / RTI > In Equation (6), R represents a radius.

Applying this radius to (Equation 7), the curve path estimation coordinates are created.

(7)

,

,

Fig. 8 is a view showing a route created by creating a curvature locus with respect to a portion without a lane by using a cover-up algorithm for a dotted line lane in the left part. In the right figure, the red part represents the lane and the green dotted line represents the travel path.

3.2 Application of stable tracking control algorithm

,

,

로 표현이 가능하며,

는 (수학식 8)에 의해 얻을 수 있다.This algorithm is based on a stable follow-up control algorithm to control the Cartesian trajectory gradually from the starting point of the robot which is not on the trajectory to the target point of the robot by using the difference between the current position and attitude of the mobile robot, to be. The position of the robot is expressed in a 3-degree-of-freedom form as shown in Equation (4). And the elements of (2) are all time functions. therefore

,

,

Can be expressed as,

Can be obtained by Equation (8).

(8)

The movement of the mobile robot is controlled by the linear velocity and the angular velocity. The kinematics of the mobile robot is defined by Jacobian matrix J.

(9)

를 얻게 된다.This kinematic equation is common to all non-directional robots. For the control of the mobile robot, this algorithm uses two positions: target position and current position. ≪ / RTI > (10) < RTI ID = 0.0 >

.

(10)

는 현재 로봇의 방향각을 의미한다. 최종적으로 로봇이 목표위치까지의 요구되는 선속도와 각속도는 (수학식 11)을 이용해 계산할 수 있다. (10)

Means the direction angle of the current robot. Finally, the required linear velocity and angular velocity of the robot up to the target position can be calculated using Equation (11).

(11)

,

,

,

,

,

은 목표점까지 이동하는데 필요한 선속도와 각속도고,

,

는 현재의 선 속도와 각 속도를 나타낸다.

,

는 목표점의 좌표를 나타낸다.
(11)

,

Is the linear velocity and angular velocity necessary to move to the target point,

,

Represents the current linear velocity and angular velocity.

,

Represents the coordinates of the target point.

In Fig. 9, the left portion of the mobile platform represents the sensor portion and the right portion represents the driving portion. And the lower data control platform part represents the data processing part. The sensor unit consists of a laser scanner that can scan the road surface, an encoder to determine the posture and position of the mobile robot, and a sensor controller that sends data received from the IMU to the data processing unit. The data processing unit creates a map using the data received from the sensor unit, and extracts feature points. Then, a control command is issued to the driving unit to follow the lane. The driving unit is composed of a motor controller for interpreting the command sent from the data processing unit and giving a motor control command to the motor drive, a motor drive for driving the mobile robot, and a motor.

10 is a three-dimensional road map creation and lane recognition and traveling robot according to an embodiment of the present invention. The robot scans the information two-dimensionally to make it a laser scanner called an LRF (Laser Range Finder). And we use the sensor called IMU (Inertia Measurement Unit) to obtain information about the robot's position and orientation. And the position of the robot is grasped by using the encoder information of the motor. These three kinds of information are processed to create a three-dimensional road map.

11 is a diagram illustrating a structure of a tracking controller according to an embodiment of the present invention.

Referring to FIG. 11, p _r (t) denotes a target position and P _c (t) denotes a current position of the mobile robot. The error metric, P _e , the difference between the current position and the target position, yields a q composed of the required linear velocity and angular velocity. The third box T is converted to hardware in consideration of the hardware structure. The coordinates of the mobile robot that reflects the required velocity are combined with the Jacobian J to change to the current velocity, which is accumulated and fed back to the current position.

12 is a view showing a result of a lane-following test according to an embodiment of the present invention. In FIG. 12, red dots indicate points having the highest density among the minutiae of the lane. The blue dashed line represents the path traveled by the mobile robot, and the purple lines on both sides represent actual lanes. As a result of this experiment, it can be seen that the detected lane is driven stably using the stable tracking algorithm. The zigzag feature points of the lane can be irregularly displayed in the lane according to the density of the detected feature points because the feature point density of both lane edges is calculated.

13 is a diagram illustrating a tracking error according to an embodiment of the present invention. In Fig. 13, the blue solid line indicates the characteristic point of the actual lane, and the mobile robot must follow the line, but an error occurs as indicated by a red dotted line. The maximum tracking error was 147.65mm. However, as shown in Fig. 13, it can be seen that the traveling path of the robot stably travels without losing the lane when the whole traveling is viewed.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalents should be construed as being included in the scope of the present invention.

Claims (1)

A laser scanner for collecting distance data and angle data, respectively;
A data processing unit for generating two-dimensional information using the distance angle data and the distance data;
A map forming unit for forming a three-dimensional map by combining the two-dimensional information and information such as position and orientation of the driving body;
A lane feature point extracting unit for extracting feature points of a road lane from a three-dimensional map; And
A curvature estimator for estimating a curved path through a cover image algorithm when a road lane is broken;
Dimensional road map creation and lane recognition device.

Images