KR102039801B1 - Road Gradient Estimation Method and System based on Stereo Camera - Google Patents

Abstract

As a method for proactively predicting the slope of a road and actively using it for chassis control of a vehicle, a stereo camera-based road gradient prediction method and system are provided. Gradient prediction method according to an embodiment of the present invention comprises the steps of extracting depth information on the road ahead using a stereo camera; And estimating gradient information of the road based on the extracted depth information.
As a result, the gradient information of the road can be predicted in advance while driving, and thus the presence or absence of the bump, the height of the bump, the presence of the pothole, the height of the pothole, the inclination of the slope, the slope of the downhill, etc. can be determined in advance. Based on this, it can be used for active chassis control to further improve vehicle stability and ride comfort.

Description

Road Gradient Estimation Method and System based on Stereo Camera

The present invention relates to vehicle imaging technology, and more particularly, to a method and system for predicting gradient information of a road ahead using a stereo camera.

In recent years, as the number of high-end vehicles increases, system devices for securing ride comfort, driving convenience, and driving stability are increasing.In addition, the road information such as bumps and portholes in front of the vehicle is predicted and used for chassis control. There is a demand for a technology that increases stability and ride comfort.

Conventional methods are made of a method of correcting the attitude of the vehicle by mechanically controlling the chassis by recognizing that the attitude of the vehicle is changed by a bump or a port hole.

However, since this is due to the method of performing the trailing control, the driver's feeling of riding comfort, driving comfort, and driving stability is not high.

The present invention has been made to solve the above problems, an object of the present invention, a method for predicting the slope of the road to actively utilize in the chassis control of the vehicle, etc., a stereo camera-based road gradient prediction method and system In providing.

According to an embodiment of the present invention, a gradient prediction method includes: extracting depth information of a road ahead using a stereo camera; And estimating gradient information of the road based on the extracted depth information.

The extracting step may include extracting depth information of the road ahead using the camera reference coordinate system, and estimating gradient information of the road may include converting depth information of the road ahead from the camera reference coordinate system to the road reference coordinate system. .

The gradient prediction method according to the present invention may further include estimating gradient information of the road, estimating a direction angle of the stereo camera facing the road surface and a height of the stereo camera, and the converting step includes the estimated direction of the stereo camera. The angle and height may be used to convert depth information of the road ahead from the camera reference coordinate system to the road reference coordinate system.

The estimating angle and height estimating step of the stereo camera may include: modeling depth information of the road ahead as a plane; And estimating a direction angle and height of the stereo camera using coefficients of the equation of the modeled plane.

The depth information on the road ahead may be depth information of the region from which the obstacle is excluded.

The coefficients of the equation of the modeled plane may be those determined by RANSAC or least squares method.

The road reference coordinate system may represent the height of the ground based on the vehicle wheel.

The gradient prediction method according to the present invention may further include estimating the trajectory of the wheel, and the estimating may include estimating the gradient information of the road with respect to the predicted trajectory of the wheel.

The predicting step may be to predict the trajectory of the wheel based on the steering information.

On the other hand, according to another embodiment of the present invention, the gradient prediction system comprises a stereo camera for extracting depth information on the road ahead; And a processor configured to estimate gradient information of the road based on the extracted depth information.

On the other hand, the vehicle control system according to another embodiment of the present invention, the step of estimating the gradient information of the road, based on the depth information on the road ahead; And controlling the chassis using the estimated road gradient information.

On the other hand, a vehicle control system according to another embodiment of the present invention, the processor for estimating the gradient information of the road, based on the depth information on the road ahead; And a controller for controlling the chassis using the estimated road gradient information.

As described above, according to the embodiments of the present invention, it is possible to predict the slope information of the road ahead while driving, so that the presence of the bump, the height of the bump, the presence of the porthole, the height of the porthole, the slope of the slope, the slope of the downhill Since the slope can be found in advance, it can be used for active chassis control based on this to further improve the stability and ride comfort of the vehicle.

In addition, according to embodiments of the present invention, a more accurate road gradient value may be obtained by continuously predicting a height and a direction angle of the camera to correct a road gradient value.

1 is a conceptual diagram of an active chassis control system using a stereo camera according to an embodiment of the present invention.
FIG. 2 is a view provided to explain a stereo camera based road gradient prediction method by the system shown in FIG. 1.
3 is a view provided to explain the concept of converting a camera coordinate system into a road reference coordinate system;
4 is a view provided for further explanation of a method of estimating a height and a direction angle of a camera;
5 is a diagram illustrating a modified road reference coordinate system;
6 is a diagram illustrating an estimation result of a road gradient value with respect to a predicted wheel trajectory, and
7 is a block diagram of an active chassis control system according to another embodiment of the present invention.

In an embodiment of the present invention, a stereo camera based road gradient prediction method and system are provided.

The road slope prediction system according to an embodiment of the present invention may extract depth information on a road ahead using a stereo camera, estimate road slope information based on the depth information, and use the same to control a vehicle suspension. System to ensure that

Road gradient prediction system according to an embodiment of the present invention, it is possible to predict the gradient information for the road in the tens of meters ahead through the stereo camera, active chassis control is possible to improve the ride comfort and convenience stability.

As described above, the road gradient prediction system according to the embodiment of the present invention is a system for predicting the gradient information on the road which has not yet passed from the stereo camera monitoring the front.

In addition, the road gradient prediction system according to an embodiment of the present invention, based on the current steering angle information to predict the trajectory that the wheel actually passes on the road ahead to extract the gradient information for this trajectory.

Since the stereo camera mounted on the vehicle may change the angle and position toward the ground while the vehicle is driving, and the gradient value of the road may be changed based on the stereo camera, the road slope prediction system according to an embodiment of the present invention may be used while driving the vehicle. By predicting the angle of the camera to the ground and the height of the camera by itself, the gradient is corrected by continuously reflecting the change value, thereby predicting the road slope with high accuracy.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a conceptual diagram of an active chassis control system using a stereo camera according to an embodiment of the present invention.

In the active chassis control system according to an exemplary embodiment of the present invention, as shown in FIG. 1, a three-dimensional coordinate value of a road ahead is acquired through a stereo camera, and a current steering wheel value is applied to track a trajectory of the left and right wheels of the vehicle. Predict.

Active chassis control is performed by adjusting the active damper by estimating the height information (gradient) corresponding to the wheel trajectory from the three-dimensional coordinate values of the road ahead.

FIG. 2 is a view provided to explain a stereo camera based road gradient prediction method by the system shown in FIG. 1.

The stereo camera shoots looking at an object with two cameras corresponding to left and right aligned eyes (S110), and estimates a three-dimensional position by estimating their parallax. The same object appears simultaneously in the left and right cameras and goes through a process of extracting matching points in pixel-by-pixel (S120).

Stereo matching finds a matching point in left and right images and measures the disparity. An example of a stereo matching algorithm used in automobiles is a semi-global matching algorithm (SGM).

The 3D position of the pixel may be estimated for each image pixel based on the parallax values extracted through stereo matching (S130). When the parallax value at each pixel coordinate (u, v) with respect to the camera origin is d, the three-dimensional position is expressed by the following equation. Here, b is the length of the baseline of the stereo camera, (px, py) is the coordinates of the principal point of the camera, f is the focal length.

However, as shown in FIG. 3, the three-dimensional position based on the camera coordinate system may be converted to the gradient information of the road by converting the road reference coordinate system. In order to convert to road reference coordinates, the camera needs to know the direction angle θ of the road surface and the height h of the camera.

Even if the direction and height of the camera are accurately measured at the initial stage, the position of the camera is often changed by the vibration of the vehicle while the vehicle is traveling, and the orientation angle is changed instantaneously when driving uphill or downhill. In addition, when a lot of people ride or load in the rear seats, the camera is slightly lifted and the orientation angle decreases. It is not realistic for the driver to measure the camera's height and orientation angle whenever there are external factors.

Therefore, in the embodiment of the present invention, the height and the orientation angle of the camera are not measured in advance by using a mechanism, and the height and the orientation angle of the camera are estimated through the three-dimensional position of the road measured through the stereo camera (S140).

Specifically, as shown in FIG. 4, it is assumed that a predetermined area (eg, 320 pixels x 170 pixels size) at the lower front side of the image is defined, and an area excluding an obstacle from this area is a road. The remaining area after removing the obstacle is considered a road, and the road area is modeled as a plane. The equation of this plane is estimated by setting the equation of the road plane Yc = aZc + b, and there are representative methods using RANSAC and least square method.

Accordingly, it is assumed that each of the points in the lower area in front of the image has three-dimensional position coordinates (Xc, Yc, Zc) based on the camera coordinates, so that these points form a road surface, and the equation of the plane of the road surface is Yc. Modeling is done with = aZc + b, and a and b are calculated using RANSAC or least-squares method, and ultimately, θ and h can be obtained using the following equation.

Then, using the height h and θ of the camera, the camera reference surface system is converted into a road reference coordinate system according to the following equation (S150). Yw means the height of the ground with respect to the vehicle wheels.

Meanwhile, the trajectory of the wheel predicted by the steering wheel information S160 may be represented by an ordered pair of (t, Xw, Zw), and may be referred to as (Xw, Zw) at the position of time t (S170). The area of the image corresponding to the position of (Xw, Zw) is shown in a box form with margins, and there are a plurality of pixels in the box. Extract a representative value Yw of one of these multiple pixels. Median Value is selected among the plurality of pixels and the representative value Yw is selected. 5 illustrates a specific example.

Next, by estimating the road gradient value for the trajectory of the wheel predicted in step S170 (S180), the vehicle suspension and the like are controlled. 6 illustrates an example of estimating a road gradient value with respect to the predicted wheel trajectory.

7 is a block diagram of an active chassis control system according to another embodiment of the present invention. An active chassis control system according to an embodiment of the present invention includes a stereo camera 210, an image processor 220, a controller 230, and a storage 240.

The stereo camera 210 is a binocular camera that photographs the road ahead, and generates depth information about the road ahead.

The image processor 220 estimates the gradient information of the road on the track of the wheel based on the depth information extracted by the stereo camera 210. The storage unit 240 provides a storage space necessary for the image processor 220 in estimating road gradient information.

The controller 230 controls the chassis based on the gradient information of the road estimated by the image processor 220, thereby improving riding comfort, driving convenience, and driving stability.

So far, preferred embodiments of the stereo camera-based road gradient prediction method and system have been described in detail.

In the exemplary embodiment of the present invention, the vehicle predicts the slope of the road ahead of the vehicle and actively uses the vehicle to control the chassis. The stereo camera is used to accurately estimate the slope information of the road to 15 to 20 meters. Stereo cameras can accurately estimate gradient information at a lower cost than other sensors.

In addition, in the embodiment of the present invention, in order to predict the trajectory of the wheel in conjunction with the steering wheel information to extract the road gradient information corresponding to the trajectory of the wheel, to compensate for the change of the measured value according to the camera's orientation angle The height of the camera and the camera's orientation angle were estimated in real time.

According to an embodiment of the present invention, it is possible to predict the slope information of the road ahead while driving, and find out in advance the presence or absence of the bump, the height of the bump, the presence of the pothole, the height of the pothole, the degree of inclination of the slope, the slope of the downhill, and the like. And it can be used for active chassis control to ensure the stability and ride comfort of the vehicle.

In addition, in an embodiment of the present invention, a more accurate road slope value can be obtained by continuously predicting a height and a direction angle of the camera to correct a road slope value.

On the other hand, the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present disclosure may be implemented in the form of computer readable codes recorded on a computer readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. In addition, the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between the computers.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

210: stereo camera
220: image processor
230: control unit
240: storage unit

Claims (12)

Extracting depth information of the road ahead using a camera reference coordinate system;
Estimating, by the stereo camera, the heading angle toward the road surface and the height of the stereo camera;
Converting depth information on the road ahead from the camera reference coordinate system to the road reference coordinate system using the estimated angle and height of the stereo camera; And
Estimating gradient information of the road based on the converted depth information;
Estimating angle and height of stereo camera,
Modeling depth information on the road ahead as a plane; And
Estimating a direction angle and a height of the stereo camera using coefficients of the equation of the modeled plane.
delete delete delete In claim 1,
For depth information on the road ahead,
Gradient prediction method characterized in that the depth information of the region from which the obstacle.
The method according to claim 1,
The coefficients of the equation of the modeled plane are
Gradient prediction method characterized in that determined by the RANSAC or least-squares method.
The method according to claim 1,
The road reference coordinate system
Gradient prediction method characterized in that the height of the ground expressed on the basis of the vehicle wheel.
The method according to claim 1,
Predicting a trajectory of the wheel;
The gradient information estimation step of the road,
Gradient prediction method comprising estimating the gradient information of the road for the predicted trajectory of the wheel.
The method according to claim 8,
The prediction phase is
Gradient prediction method for predicting the trajectory of the wheel based on the steering information.
A stereo camera extracting depth information of the road ahead using a camera reference coordinate system; And
And a processor configured to estimate gradient information of the road based on the extracted depth information.
The processor,
The stereo camera estimates the heading angle and the height of the stereo camera facing the road surface, and converts the depth information of the road ahead from the camera reference coordinate system to the road reference coordinate system using the estimated angle and height of the stereo camera. Based on the information, estimate the slope information of the road,
Gradient prediction system, characterized in that to model the depth information on the road ahead, the direction and height of the stereo camera using the coefficients of the equation of the modeled plane.
Estimating gradient information of the road, based on the depth information of the road ahead; And
And controlling the chassis using the estimated road gradient information.
The estimating step is
The stereo camera converts the depth information of the road ahead extracted from the camera reference coordinate system into the road reference coordinate system using the directivity angle facing the road surface and the height of the stereo camera. Based on depth information for, estimate road grades,
And modeling depth information of the road ahead as a plane, and estimating a direction and height of the stereo camera using coefficients of the modeled plane equation.
A processor for estimating gradient information of the road based on the depth information of the road ahead; And
And a controller configured to control the chassis using the estimated road gradient information.
The processor,
The stereo camera converts the depth information of the road ahead extracted from the camera reference coordinate system into the road reference coordinate system using the directivity angle facing the road surface and the height of the stereo camera. Based on the depth information for, estimate the slope of the road,
And modeling depth information of the road ahead as a plane and estimating the direction and height of the stereo camera using coefficients of the modeled plane equation.

Images