DE102012009577A1 - Method for calibrating squint angle of frame camera of stereo camera arrangement in vehicle, involves calculating travel distance covered by vehicle, and determining deviation between distance and reference distance to calibrate angle - Google Patents

Abstract

The method involves determining reference-travel distance covered by a vehicle in a time period between two time points. Two positions of the vehicle are determined from stereoscopically determined disparity of image points of images detected by frame cameras (1.1, 1.2), focal length of a camera arrangement (1) and base width between the cameras. Stereoscopically determined travel distance covered by the vehicle is calculated from difference between the positions. Deviation between the reference and stereoscopically determined distances is determined to calibrate squint angle of the cameras. An independent claim is also included for a method for adjusting a frame camera of a camera system.

Description

The invention relates to a method for calibrating a squint angle of individual cameras of a camera arrangement arranged on a vehicle, comprising at least two individual image cameras, wherein correspondingly consecutively detected sets of respectively at least two images detected simultaneously by the at least two individual cameras are detected, at least two successive times in each case a disparity of the corresponding pixels in the simultaneously acquired images is determined stereoscopically.

The invention further relates to a method for adjusting single-frame cameras of a camera arrangement arranged on a vehicle using a method for calibrating a camera arrangement arranged on the vehicle.

From the DE 10 2004 062 275 A1 A method for determining a calibration parameter for a vehicle-mounted stereo camera is described. The stereo camera comprises two single-frame cameras whose optical axes are aligned at a given yaw angle to each other. On the basis of data of a navigation device and on the basis of data recorded by means of a wireless communication network, a separate position of the vehicle, ie the stereo camera, is determined in space and provided with a time stamp. On the basis of the own position it is checked whether a distance measurement carried out by means of the stereo camera leads to an object located in the surroundings of the vehicle to the same result. From this, it is deduced whether a current yaw angle corresponds to a nominal value and corrected in the event of a deviation. Furthermore, an object tracking of a stationary object is carried out over a plurality of pairs of individual images acquired temporally one after the other.

Object tracking is used to determine a change in position or a change in the speed of the object in space, the yaw angle being determined from the change in position or speed, compared with the desired value and correspondingly corrected in the event of a deviation.

The invention is based on the object to provide a comparison with the prior art improved method for calibrating a squint angle of frame cameras of a camera assembly and an improved method for adjusting single-frame cameras of a camera arrangement.

With regard to the method for calibration, the object is achieved by the features specified in claim 1 and in terms of the method for adjustment by the features specified in claim 6.

Advantageous embodiments of the invention are the subject of the dependent claims.

In a method for calibrating a squint angle of individual cameras of a camera arrangement arranged on a vehicle, comprising at least two single-frame cameras, corresponding pixels are determined in chronologically successively detected sets of at least two images simultaneously acquired by the at least two individual cameras and at at least two consecutive times respectively a disparity of the corresponding pixels in the simultaneously captured images detected stereoscopically.

According to the invention, a reference travel distance covered by the vehicle in a period between the at least two times is determined from odometry data of the vehicle.

In this case, odometry data is understood as meaning data describing a movement of the vehicle and in particular comprising measured variables of a chassis, a drive train, a steering system and measured variables of a navigation device of the vehicle.

Furthermore, according to the invention, positions of the vehicle are determined for the at least two consecutive times from the stereoscopically determined disparity, a focal length of the camera arrangement and a base width between the single-frame cameras, wherein a difference of the positions results stereoscopically from the vehicle in a time interval between the at least two times determined route is determined. To calibrate the squint angle of the individual cameras, a deviation between the reference distance and the stereoscopically determined distance is determined.

The method according to the invention enables a continuous and robust calibration of the squint angle of the single-frame cameras of the camera arrangement, for example of a stereo camera arrangement.

Embodiments of the invention are explained in more detail below with reference to drawings.

Showing:

1 schematically a trained as a stereo camera assembly camera assembly with two single-frame cameras,

2 schematically an epipolar geometry of a camera arrangement designed as a stereo camera arrangement,

3 schematically the camera arrangement according to 1 and associated angles which describe an alignment of the single-frame cameras to one another and to a reference system,

4 schematically two simultaneously captured images of the image cameras of the camera arrangement according to 1 and a plan view of a traffic situation shown in the pictures,

5 schematically a means of a single camera according to the camera arrangement 1 recorded and divided into several sectors image.

Corresponding parts are provided in all figures with the same reference numerals.

In 1 is a camera arrangement designed as a stereo camera arrangement 1 with two single-frame cameras 1.1 . 1.2 and their detection ranges EB1, EB2 shown. The camera arrangement 1 is in a manner not shown on an in 4 arranged vehicle F and provided for detecting an in the direction of travel in front of the vehicle F environment.

In the camera arrangement 1 are the single-frame cameras 1.1 . 1.2 aligned such that their main optical axes HA1, HA2 extending from a so-called camera main point, not shown, through the center of a lens in the direction of the single-frame cameras 1.1 . 1.2 move. That is, both main optical axes HA1, HA2 of the camera array are aligned parallel to each other and point in the same direction. Thus, the camera arrangement 1 an axis-parallel camera coordinate system.

In such axis-parallel camera coordinate systems, the so-called epipole E1, E2, which in 2 is shown in detail, moved to infinity, making small changes to an in 3 shown squint angle γ have a great influence on the position of the epipoles E1, E2. In numerics or computer science, the epipole E1, E2 is located far outside on an epipolar line L1, L2, since infinity can not be displayed by means of data processing units.

2 shows, by way of example, an epipolar geometry of a camera arrangement designed as a stereo camera arrangement 1 in which the epipolar geometry determines the geometric relationships between the single-frame cameras 1.1 . 1.2 captured images B1, B2 of the same object O represents. The object O shows 4 closer.

The epipolar geometry shows the single-frame cameras 1.1 . 1.2 , in particular their projection centers, as well as an epipolar plane A and the epipoles E1, E2 lying on the epipolar lines L1, L2.

Furthermore, in each case one image plane by means of the single-frame cameras 1.1 . 1.2 captured images B1, B2 shown, which in front of the projection centers of the single-frame cameras 1.1 . 1.2 worked out.

An object point OP1 forms in the image B1 of the first single-frame camera 1.1 to a pixel BP1. Starting from this pixel BP1, it is possible to determine an associated ray on which the object point OP1 lies. Possible further object points OP2 to OP4, which are shown on the pixel BP1, lie on the same beam. This beam and thus all possible object points OP1 to OP4 become in the detection of the object O from another position by means of the second single-frame camera 1.2 in their captured image B2 on a straight line, the epipolar line L2, mapped. The search for the pixel BP2 corresponding to the pixel BP1 in the second image B2 is reduced to this epipolar line L2.

In 3 are the camera arrangement 1 according to 1 and associated angles .alpha., .beta.,. gamma. which indicate an alignment of the single-frame cameras 1.1 . 1.2 to each other and to a reference system, in particular the vehicle F describe. The angles α, β, γ are exemplary only for the single-frame camera 1.1 are shown, however, are for the further single-frame camera 1.2 formed analog.

In order to optimally capture the environment of the vehicle F with the camera arrangement 1 as well as an optimal stereoscopic evaluation by means of the single-frame cameras 1.1 . 1.2 Captured images B1, B2, it is necessary that the orientation of the single-frame cameras 1.1 . 1.2 to each other and the vehicle F corresponds to a desired value. For this purpose, it is necessary to monitor the alignment and in case of deviations from the setpoint adjustment of the single-frame cameras 1.1 . 1.2 perform.

The orientation of the single-frame cameras 1.1 . 1.2 is described by a first angle α, which is a rotation of the single-frame cameras 1.1 . 1.2 describes a horizontal and perpendicular to the main axes HA1, HA2 extending first axis X.

The orientation of the single-frame cameras 1.1 . 1.2 is further described by a second angle β, which is a rotation of the single-frame cameras 1.1 . 1.2 about a horizontal and parallel to the main axes HA1, HA2 extending second axis Z describes.

Further, the orientation of the still cameras 1.1 . 1.2 is described by a squint angle γ, which is a rotation of the single-frame cameras 1.1 . 1.2 about a vertically extending and perpendicular to the main axes HA1, HA2 aligned third axis Y describes.

By means of the method for calibrating a squint angle γ of the single-frame cameras 1.1 . 1.2 the arranged on the vehicle F camera arrangement 1 become the single-frame cameras 1.1 . 1.2 online, ie during their operation on the vehicle F, calibrated to each other or to a defined world coordinate system. The first angle α and the second angle β are thereby estimated and calibrated in particular.

4 shows two simultaneously captured images B1, B2 of the single-frame cameras 1.1 . 1.2 the camera arrangement 1 and a plan view of a traffic situation shown in the images B1, B2, wherein the vehicle F moves at a distance x behind the also formed as a vehicle object O.

To calibrate the squint angle γ odometry data of the vehicle F are determined, the odometry data include data describing a movement of the vehicle F and in particular measures of a chassis, a powertrain, a steering and measured variables of a navigation device of the vehicle F include.

From the odometry data, a distance covered by the vehicle between two times is determined. This route is used as a reference route.

Furthermore, the corresponding pixels BP1, BP2 are ascertained in sets of two images B1, B2 recorded simultaneously in succession, these correlations being timed with a camera arrangement 1 , or spatially with two or more camera arrangements 1 be collectivized. The method is described below for the stereo camera arrangement. However, the method is also for camera arrangements 1 with more than two single-frame cameras 1.1 . 1.2 applicable. In these so-called multi-camera systems, it is necessary that the orientation of the single-frame cameras 1.1 . 1.2 That is, the camera coordinate systems to each other and the coordinate system against which is calibrated known.

To calibrate the squint angle γ, a disparity of the corresponding pixels BP1, BP2 in the simultaneously acquired images B1, B2 is also determined stereoscopically at two successive times. That is, a distance between the correspondences is determined in a first time step and a subsequent next time step. The distance between the time steps is characterized by a recording speed of the single-frame cameras 1.1 . 1.2 Are defined.

mit:

x

= Abstand zwischen dem Fahrzeug F und dem Objekt O,

f

= Brennweite der Kameraanordnung 1,

b

= Basisbreite, d. h. Abstand, zwischen den Einzelbildkameras 1.1, 1.2,

x_L

= Bildpunkt BP1 im linken Bild B1 und

x_R

= Bildpunkt BP2 im rechten Bild B2.

Due to the determination of the reference route from the odometry data of the vehicle F, it is possible to determine in the set of corresponding pixels BP1, BP2 those correspondences, ie corresponding pixels BP1, BP2, which satisfy a backprojection function shown in the following equation (1) :

With:

x

= Distance between the vehicle F and the object O,

f

= Focal length of the camera arrangement 1 .

b

= Base width, ie distance, between the single-frame cameras 1.1 . 1.2 .

x _L

= Pixel BP1 in the left image B1 and

x _R

= Pixel BP2 in the right image B2.

From the difference of the pixels BP1, BP2 (x _L - x _R) of the rear projection divisor function formed thereby represents the disparity of the pixels BP1, BP2.

The rear projection function is calculated for each determined pair of corresponding pixels BP1, BP2 in the consecutive times, wherein a position of the vehicle F is derived from the respective distance between the vehicle F and the object O. That is, that of the vehicle F at the respective time is determined from the quotient of the product formed from the focal length and the base width and the disparity of the corresponding pixels BP1, BP2.

mit:

Dist_Fahrzeug

= Referenz-Fahrstrecke,

₁

= erster Zeitpunkt und

₂

= zweiter Zeitpunkt.

A difference between the rear projection functions calculated at the two times results in a distance traveled stereoscopically between the two times and is compared with the reference distance traveled by the vehicle F and determined from its odometry data according to equation (2) :

With:

Dist _vehicle

= Reference route,

₁

= first time and

₂

= second time.

For calibration of the squint angle γ of the single-frame cameras 1.1 . 1.2 a deviation between the reference route and the stereoscopically determined route is determined. If there is a deviation, the single-frame cameras become 1.1 . 1.2 adjusted by the squint angle γ, but also the other angles α, β of the single-frame cameras 1.1 . 1.2 be changed until the value of the deviation between the reference route and the stereoscopically determined route is zero.

Is there a possibility that an object O over a longer distance by means of the camera arrangement 1 track or that the camera arrangement 1 not moving, then is a calibration of the single-frame cameras 1.1 . 1.2 by additional use of sensors which provide an exact absolute distance x of the vehicle F to the object O possible. In this case, by means of such a sensor, for example by means of a lidar or radar sensor, a point in the surroundings of the vehicle F is aimed, which also in the images B1, B2 of the single-frame cameras 1.1 . 1.2 is visible. The distance x becomes with the backprojection of this point out of the camera arrangement 1 adjusted using equation (1). The squint angles γ, but also the other angles α, β are adjusted until the values of the absolute distance x coincide with the back-projected distances from the images B1, B2.

In order to increase the robustness of the calibration, a plurality of pairs of corresponding pixels BP1, BP2 and their respective disparities are preferably determined stereoscopically at a respective time, and the calibration is carried out in accordance with equations (1) and (2).

In a particularly advantageous manner, by changing the equation (2) to the disparity, it is possible to make a statement about the quality of the pairs of corresponding pixels BP1, BP2, d. H. correspondence. Thus, it is possible to use high quality correspondence for calibration and discard low quality correspondences for calibration.

In 5 is one by means of the single-frame camera 1.1 the camera arrangement 1 captured image B1, wherein the image B1 is divided into four sectors S1 to S4.

In each sector S1 to S4, a plurality of pixels BP1 are selected and together with a plurality of pixels BP2 in the image B2 of the further single-frame camera 1.2 formed a plurality of pairs of corresponding pixels BP1, BP2 and determined their respective disparity stereoscopically.

This gathering of correspondences is referred to as "binning", wherein in addition to subdividing the images B1, B2 into a plurality, for example four sectors S1 to S4, a threshold, i. H. a maximum number of pairs of corresponding pixels BP1, BP2 to be determined is specified. Thus, a certain amount and a distribution of the correspondences are simulated.

LIST OF REFERENCE NUMBERS

1

camera assembly

1.1

Still camera

1.2

Still camera

A

Epipolarfläche

B1, B2

image

BP1, BP2

pixel

E1, E2

epipole

EB1, EB2

detection zones

F

vehicle

HA1, HA2

main optical axis

L1, L2

epipolar

O, O1 to On

object

OP1 to OP4

object point

S1 to S4

sectors

x

distance

X, Y, Z

axis

α

first angle

β

second angle

γ

squint

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102004062275 A1 [0003]

Claims (6)

Method for calibrating a squint angle (γ) of single-frame cameras ( 1.1 . 1.2 ) a camera arrangement arranged on a vehicle (F) ( 1 ) comprising at least two single-frame cameras ( 1.1 . 1.2 ), In which at least two sets of at least two simultaneously recorded by means of the at least two single-frame cameras ( 1.1 . 1.2 corresponding pixels (BP1, BP2) are determined, wherein at least two successive points in time respectively a disparity of the corresponding pixels (BP1, BP2) in the simultaneously acquired images (B1, B2) is determined stereoscopically, characterized in that - from odometry data of the vehicle (F) is determined by the vehicle (F) in a period between the at least two times reference travel distance is determined, - at least two successive times from the stereoscopically determined disparity, a focal length of Camera arrangement ( 1 ) and a base width between the single-frame cameras ( 1.1 . 1.2 ) Positions of the vehicle (F) are determined, - a difference of the positions determines a stereoscopically determined travel distance covered by the vehicle (F) in a time interval between the at least two times, and - calibration of the squint angle (γ) of the single-frame cameras ( 1.1 . 1.2 ) a deviation between the reference route and the stereoscopically determined route is determined. A method according to claim 1, characterized in that the position at a respective time from the quotient of a product formed from the focal length and the base width product and the disparity of the corresponding pixels (BP1, BP2) is determined. A method according to claim 1 or 2, characterized in that at a respective time a plurality of pairs of corresponding pixels (BP1, BP2) and their respective disparity are determined stereoscopically. Method according to one of the preceding claims, characterized in that the of the single-frame cameras ( 1.1 . 1.2 ) (B1, B2) are divided into a plurality of sectors (S1 to S4), wherein in each sector (S1 to S4) a plurality of pairs of corresponding pixels (BP1, BP2) and their respective disparity are determined stereoscopically. Method according to Claim 3 or 4, characterized in that a maximum number of pairs of corresponding pixels (BP1, BP2) to be determined is specified. Method for adjusting single-frame cameras ( 1.1 . 1.2 ) a camera arrangement arranged on a vehicle (F) ( 1 ) using the method for calibrating the camera arrangement arranged on the vehicle (F) ( 1 ) according to one of the preceding claims, wherein at least the squint angle (γ) of the single-frame cameras ( 1.1 . 1.2 ) is changed until the value of the deviation between the reference route and the stereoscopically determined route is zero.

Images