KR20130035959A - Driable area providindg apparatus, method and system - Google Patents

Abstract

PURPOSE: A drivable area providing apparatus and a method and system thereof are provided to stably perform a drivable area detection and provide an unmanned vehicle driving technology capable of giving a reliable road curve and gradient information to an area reaching a far distance further than an existing distance in the progress direction of a vehicle by analyzing and properly using three or more laser scan results obtained by a multilayer laser scan method in one time of scan. CONSTITUTION: A drivable area providing apparatus includes a scan data analyzing unit extracting a first drivable area by utilizing 'an analysis by histogram and statistical mahalanobis distance classification' and 'a differentiation about the scan result data of multiple multilayer laser sensors about the driving road of a vehicle while driving'; a camera acquisition image information analyzing unit obtaining a boundary and traffic sign information by analyzing by reflecting information related to the first drivable area in an analysis range; and a final drivable area information acquisition unit obtaining final drivable area information by connecting the scan data analysis result with a camera acquisition image information analysis result. [Reference numerals] (AA) Scanning a multilayer; (BB) Analyzing an individual scan layer; (CC) Analyzing upper plane(X-Y) data histogram; (DD) Analyzing front plane(X-Y) data histogram; (EE) Detecting a drivable horizontal smoothing line; (FF) Generating a polygon by the connection of individual smoothing line edge points; (GG) Generating a drivable area; (HH) Obtaining a camera image; (II) Limiting an image processing range in the inner area of the drivable area; (JJ) Separating a color image to gray; (KK) Setting a threshold intensity database in advance to make a separated object area as a purpose recognition target; (LL) Converting into a black and white image by threshold intensity and an area expansion of a separated pixel; (MM) Recognizing a strong drivable area;

Description

Device, method and system for providing driving range {DRIABLE AREA PROVIDINDG APPARATUS, METHOD AND SYSTEM}

The present invention relates to an apparatus, a method and a system for providing a movable area.

The prior art has used a method using a laser sensor or a vision individually in the steering control process by recognition of the driving range of the unmanned vehicle, which has a disadvantage that is not robust to a variety of lighting environment and road environment.

SUMMARY OF THE INVENTION An object of the present invention is a technology for recognizing a road area using a conventional laser sensor, by analyzing and appropriately using three or more laser scan results obtained in one scan by an advanced multi-layer laser scan method. The present invention provides a driverless vehicle technology capable of stably performing area detection and giving reliable road curve and gradient information to an area that reaches a farther distance in the direction of travel of the vehicle.

In addition, an object of the present invention is to detect lanes, crosswalks, speed bumps, etc. while analyzing the image obtained by the camera installed toward the road surface inside the area after the runnable area obtained by the multilayer laser sensor information is determined. By doing so, it is to provide a wireless driving vehicle technology that can increase the stability of unmanned driving technology applicable to the actual road environment.

The apparatus for providing a runnable area according to the present invention for solving the above problems is based on a differential, histogram, and statistical Mahalanobis distance classification analysis on scan result data of a plurality of multi-layer laser sensors of a driving road of a driving vehicle. A scan data analyzer configured to extract one driveable region; A camera acquisition image information analyzer configured to obtain information about a boundary line and traffic marker by analyzing the image acquired by the camera and reflecting the information related to the first travelable area in an analysis range; And a final driving region information acquisition unit which connects the scan data analysis result and the camera acquired image information analysis result to obtain final driving region information.

The scan data analysis unit may perform a derivative on the individual scan layers obtained through the plurality of multilayer laser sensors to obtain driving road width information; Histogram analysis and Mahalanobis distance analysis to find candidate range of the first travelable region by performing histogram analysis on each axis of the individual scan layer and Mahalanobis distance analysis for statistically classifying a set of laser detection points. Execution unit; And a comparator configured to compare a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.

The derivative performing unit extracts a maximum value by performing a derivative in the right direction of the horizontal axis on the individual scan layer to reduce the information on the first travelable region; And a road width information acquisition unit that detects both ends of the road whose slope is rapidly changed based on the peak value extraction result and obtains the road width information.

The comparison unit may include: a boundary position acquisition unit configured to obtain an accurate boundary position of a common part by comparing a differential performance result of the individual scan layers with a candidate range of the first travelable region; A smoothing line display unit displaying a horizontal smoothing line of each individual scan layer from the obtained boundary position information; And an acquisition unit for connecting the corner points of the horizontal smooth line of each scan layer in a quadrangular shape to obtain the first travelable region between adjacent scan layers.

The camera acquisition image analyzer may include a plane image converter configured to convert an image acquired while the camera faces the front road surface into a planar image viewed from the top of the road surface by inverse perspective mapping. And an information obtaining unit obtaining the boundary line and the traffic mark information from the plane image.

The information acquisition unit includes a color channel separation unit for separating the image on the plane into an RGB color channel; A basic feature information extraction unit for extracting basic feature information related to a point where a brightness intensity rapidly changes in an adjacent pixel set with respect to an R channel image among the separated RGB color channels; A black and white image acquisition unit for obtaining a black and white image by forcibly converting colors lighter than a threshold intensity to a color value of all 1 based on the basic feature information and colors darker than a threshold intensity to a color value of 0; And a traffic marker information acquisition unit for acquiring the boundary surface and the traffic marker information based on the obtained black and white image.

The basic feature information extractor may extract the basic feature information by applying a first and second differential mask kernel after performing a Gaussian smoothing process on the R channel image.

The basic feature information may be distinguished by a difference between the road surface of the asphalt color and the color representing the traffic sign.

The traffic marking information obtaining unit may include: an information converting unit converting the same color pixels within a predetermined distance into continuous line or polygon information by applying a region growing technique to a white region of the black and white image; And converting the continuous line into boundary line information on a road surface, and the polygonal information may include a boundary line and a traffic mark information converting unit converting the continuous line into traffic mark information by using a predetermined pattern and threshold strength information that is database.

The boundary line and traffic message information converting unit may convert the continuous line into the boundary line information by linear fitting by RANSAC (RANdom SAmple Consensus) / kmeans clustering algorithm processing.

According to an aspect of the present invention, there is provided a method of providing a driveable area according to the present invention, using differential, histogram, and statistical Mahalanobis distance classification analysis on scan result data of a plurality of multilayer laser sensors of a driving road of a driving vehicle. A scan data analysis step of extracting a driving range; A camera acquired image information analyzing step of acquiring boundary lines and traffic marker information by analyzing the image acquired through the camera and reflecting the information related to the first travelable area in an analysis range; And acquiring final driving region information by connecting the scan data analysis result and the camera acquired image information analysis result to obtain final driving region information.

The scanning data analyzing step may include performing derivatives on individual scan layers obtained through the plurality of multilayer laser sensors to obtain boundary side information; Performing a histogram analysis and a Mahalanobis distance analysis to find a candidate range of a first travelable region by performing histogram analysis and statistical Mahalanobis distance classification analysis on each axis of the individual scan layer; And comparing a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.

The camera image analysis step may include converting an image acquired while the camera faces the front road surface into a planar image viewed from the top of the road surface by inverse perspective mapping. And an information acquiring step of acquiring the boundary line and traffic marker information from the plane image.

The traveling area providing system of the present invention for solving the above problems is a plurality of multi-layer laser sensor for scanning with a direction angle gradually increasing in the vertical direction; A camera for photographing driving roads; And extracting a first travelable region by using differential, histogram, and statistical Mahalanobis distance classification analysis of the scan result data of the plurality of multilayer laser sensors, and extracting an image acquired by the camera from the first travelable region. The apparatus may include a driving area providing apparatus for obtaining boundary line and traffic marker information by reflecting relevant information in an analysis range and analyzing the result, and obtaining final driving area information by connecting the scan data analysis result and the camera acquired image information analysis result. Can be.

The apparatus for providing a travelable area may include a differential performing unit configured to obtain boundary side information by performing differentiation on individual scan layers obtained through the plurality of multilayer laser sensors; The histogram analysis and the Mahalanobis distance analysis for finding the candidate range of the first travelable region by performing histogram analysis and the Mahalanobis distance analysis for statistically classifying the set of laser detection points for each axis of the individual scan layer. Execution unit; And a comparator configured to compare a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.

The apparatus for providing a movable area includes a plane image converting unit which converts an image acquired while the camera faces a front road surface into an image on a plane viewed from above on a road surface by inverse perspective mapping; And an information obtaining unit obtaining the boundary line and the traffic mark information from the plane image.

Analyzing and processing three or more scan data results obtained from a single multilayer laser scan result to secure driving range, and continuously extending it over time in the driving process to stably recognize the driving range ahead. Thus, the steering direction prediction and route planning of the driving can be facilitated, and the maximum safety speed of the vehicle can be increased. Obstacles, traffic signs, or hazards within the driveable area are roughly identified by multi-layer laser scan analysis, and additional information on the driveable area is supplemented through the acquisition and analysis of camera image information, thereby providing robust driving. Possible area recognition performance.

1 is an order relationship with the information processing step according to the present invention,
2 is a data output result seen from the top plane and a data output result seen from the front plane, and the third is a result of expressing these data in three-dimensional space.
3 and 4 illustrate a process of extracting road area information that can be driven by performing analysis on the obtained individual scan data.
5 is a result of performing a height direction (Z axis) histogram analysis,
6 is a result of performing a width direction (Y axis) histogram analysis.
7 shows a horizontal smooth line of each individual scan data,
8 shows the result of extracting line information of a general road after conversion by inverse perspective projection matching method,
FIG. 9 is a process and results of obtaining robust driving region information by connecting an image obtained by analysis between two adjacent laser scan layers and an image information analysis result to each other.
Fig. 10 shows the range of travel angles and the like of each layer of the multilayer laser sensor used.

In Fig. 1, the order processing and information processing steps related to the present invention are shown. The multi-layer laser system analyzes a plurality of pieces of scan layer data obtained by one scan. In the first figure on the left side of FIG. 2, the data output result from the upper plane, that is, the XY plane, and the second front plane, that is, the YZ plane Analyze the data output you see at the same time. The final third figure is the result of representing these data in three-dimensional space. 3 and 4 illustrate a process of extracting road area information that can be driven by performing analysis on the obtained individual scan data.

FIG. 3 may give information about the travelable area by extracting a peak value by performing a derivative in the right direction of the horizontal axis on the result obtained for the individual floors. In other words, by detecting both ends of a flat surface such as a curb or a center dividing line, the slope is rapidly changed, it is possible to obtain the road width information. However, the process of obtaining boundary-side information by the differential method is possible only when the laser measurement result data has no noise component, so that the overall soft property is maintained, and when the noise component is included in the middle, an error occurs in detecting both ends. Can be done. Therefore, area detection is required to reinforce it, and histogram analysis of data is added.

4 is a result of performing a moving direction (X axis) histogram analysis on one layer of the first sensor data output result of FIG. 2, and FIG. 5 is a result of performing a height direction (Z axis) histogram analysis. Is the result of the width direction (Y axis) histogram analysis. Using the above histogram analysis results for each individual floor, the candidate range of the travelable area can be found, and the differential results obtained above can be compared to obtain an accurate boundary position of the common part. From this information, a horizontal smooth line of each individual scan data can be displayed as shown in FIG. 7, and the corner points of the horizontal smooth line of each floor can be connected in a quadrangular shape to obtain a rectangular area of the movable area between adjacent floors. When these areas accumulate as they progress forward in the course of traveling of the vehicle, the multi-layer laser sensor data results may provide map information of the driving region on the traveled route. The information about the driveable area is reflected in the image analysis range of the camera, and the image acquired in the state of facing the front road surface of the vehicle is converted into the planar image viewed from above by Inverse Perspective Mapping.

FIG. 8 shows the result of extracting line information of a general road after conversion by inverse near perspective matching method, and FIG. 9 shows robustness by connecting the image information analysis result with the area obtained by analysis between two adjacent laser scan layers obtained above. It shows the process of obtaining the driveable region information. The process of acquiring the necessary information from the image obtained by the inverse near-projection matching method is as follows. First, the obtained color image is separated into RGB color channels, the Gaussian smoothing process is performed on the R channel (red) image, and then primary and second differential mask kernels are applied to the basic feature information, that is, the adjacent image. Extract points whose brightness intensity changes rapidly in a set of pixels. This information is finally distinguished by the difference between the road surface of dark gray asphalt color and the color representing the traffic sign, and the traffic signs on the road surface displayed in yellow or white color are identified and identified by subsequent image processing. . That is, the threshold intensity is set so that colors brighter than the threshold intensity of the image pixels are all converted to 1, and colors darker than the threshold intensity are converted to 0, thereby converting them into black and white images, and the converted images are pure dark gray. It is divided into road surfaces and other traffic markers. The obtained white areas are transformed into continuous line or polygon information by applying the same region growing technique, and the same color pixels within a certain distance are merged with each other, and the continuous line is converted into boundary information on the road surface. In order to be able to be transformed, line fitting is performed by RANSAC (RANdom SAmple Consensus) / kmeans clustering algorithm processing. The traffic signs on various road surfaces are databased by a predetermined pattern and critical intensity values by pre-experimental experiments, and then applied, enabling prediction of forward driving direction as well as information at the time of sensor measurement about the driving range. By doing so, it is possible to recognize the strong driving range.

Claims (16)

A scan data analyzer extracting a first driveable region using analysis based on differential, histogram, and statistical Mahalanobis distance classification on scan result data of a plurality of multi-layer laser sensors of a driving road of a driving vehicle;
A camera acquisition image information analyzer configured to obtain information about a boundary line and traffic marker by analyzing the image acquired by the camera and reflecting the information related to the first travelable area in an analysis range; And
And a final driveable area information acquisition unit which connects the scan data analysis result and the camera acquired image information analysis result to obtain final driveable area information.
The method of claim 1, wherein the scan data analysis unit
A differential performing unit configured to obtain driving road width information by performing differentiation on individual scan layers obtained through the plurality of multilayer laser sensors;
A histogram analysis and a Mahalanobis distance analysis unit for performing a histogram analysis and statistical Mahalanobis distance classification analysis on each axis of the individual scan layer to find a candidate range of a first driveable region; And
And a comparator configured to compare a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.
The method of claim 2, wherein the differential execution unit
A maximum value extracting unit configured to extract a maximum value by performing differential operation on the individual scan layers in a right direction along a horizontal axis to reduce information on a first travelable region; And
And a road width information acquisition unit for detecting both ends of a road whose slope is rapidly changed based on the highest value extraction result and obtaining road width information. .
The method of claim 2, wherein the comparison unit
A boundary position acquisition unit for obtaining an accurate boundary position of a common part by comparing a differential performance result of the individual scan layers with a candidate range of the first travelable region;
A smoothing line display unit displaying a horizontal smoothing line of each individual scan layer from the obtained boundary position information; And
And an acquisition unit for connecting the corner points of the horizontal smooth line of each scan layer in a quadrangular shape to obtain the first travelable region between adjacent scan layers.
The method of claim 1, wherein the camera acquisition image analysis unit
A planar image converting unit converting an image acquired while the camera faces the front road surface into a planar image viewed from the top of the road surface by Inverse Perspective Mapping; And
And an information acquisition unit for acquiring the boundary line and the traffic mark information from the planar image.
The method of claim 5, wherein the information acquisition unit
A color channel separator that separates the image on the plane into an RGB color channel;
A basic feature information extraction unit for extracting basic feature information related to a point where a brightness intensity rapidly changes in an adjacent pixel set with respect to an R channel image among the separated RGB color channels;
A black and white image acquisition unit for obtaining a black and white image by forcibly converting colors lighter than a threshold intensity to a color value of all 1 based on the basic feature information and colors darker than a threshold intensity to a color value of 0; And
And a traffic marking information obtaining unit obtaining the boundary surface and the traffic marking information based on the obtained black and white image.
The method according to claim 6,
And the basic feature information extractor extracts the basic feature information by applying first and second differential mask kernels after performing a Gaussian smoothing process on the R channel image.
The method according to claim 6,
The basic feature information is distinguished by the difference between the road surface of the asphalt color and the color representing the traffic sign.
The method of claim 6, wherein the traffic sign information acquisition unit
An information conversion unit converting the same color pixels within a predetermined distance into continuous line or polygon information by applying a region growing technique to a white region of the black and white image; And
The continuous line is converted into boundary information on the road surface, and the polygonal information includes a boundary line and a traffic marker information converting unit for converting the traffic marker information by using a constant pattern and threshold strength information that is databased. Possible area provision device.
10. The apparatus of claim 9, wherein the boundary line and traffic message information converting unit is used.
And converting the continuous line into the boundary line information by linearly fitting the random lines by a RANSAC (random SAmple consensus) / kmeans clustering algorithm process.
A scan data analysis step of extracting a first travelable region by using differential, histogram, and statistical Mahalanobis distance classification analysis on scan result data of a plurality of multilayer laser sensors of a driving road of a driving vehicle;
A camera acquired image information analyzing step of acquiring boundary lines and traffic marker information by analyzing the image acquired through the camera and reflecting the information related to the first travelable area in an analysis range; And
And obtaining final driving region information by connecting the scan data analysis result and the camera acquired image information analysis result to obtain final driving region information.
The method of claim 11, wherein analyzing the scan data
Performing differentiation on individual scan layers obtained through the plurality of multilayer laser sensors to obtain boundary side information;
Performing a histogram analysis and a Mahalanobis distance analysis to find a candidate range of a first travelable region by performing histogram analysis and statistical Mahalanobis distance classification analysis on each axis of the individual scan layer; And
And comparing a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.
The method of claim 11, wherein the analyzing of the camera acquired image comprises:
A plane image conversion step of converting an image acquired while the camera faces the front road surface into a planar image viewed from the top of the road surface by inverse perspective mapping; And
And an information obtaining step of acquiring the boundary line and the traffic mark information from the plane image.
A plurality of multi-layer laser sensor for scanning with a direction angle gradually increasing in the vertical direction;
A camera for photographing driving roads; And
The first driving region is extracted by using differential, histogram, and statistical Mahalanobis distance classification analysis of the scan result data of the plurality of multilayer laser sensors, and the image acquired by the camera is associated with the first driving region. Reflecting the information in the analysis range to obtain boundary lines and traffic marker information, and connecting the scan data analysis result and the camera acquired image information analysis result to obtain a driving area providing apparatus for obtaining final driving area information. A drive area providing system characterized in that.
The apparatus of claim 14, wherein the driving area providing apparatus
A differential performing unit configured to obtain boundary side information by performing differentiation on individual scan layers obtained through the plurality of multilayer laser sensors;
A histogram analysis and mahalanobis distance analysis unit that performs a histogram and statistical Mahalanobis distance classification analysis on each axis of the individual scan layer to find a candidate range of a first driveable region; And
And a comparator configured to compare a result of performing the differential of the individual scan layers with a candidate range of the first travelable area to obtain a rectangular area of the at least one travelable area.
The apparatus of claim 14, wherein the driving area providing apparatus
A planar image converting unit converting an image acquired while the camera faces the front road surface into a planar image viewed from the top of the road surface by Inverse Perspective Mapping; And
And a data acquisition unit for acquiring the boundary line and traffic marking information from the plane image.

Images