WO2011063785A1

WO2011063785A1 - Method for estimating the roll angle in a travelling vehicle

Info

Publication number: WO2011063785A1
Application number: PCT/DE2010/001332
Authority: WO
Inventors: Michael Walter
Original assignee: Conti Temic Microelectronic Gmbh Patente & Lizenzen
Priority date: 2009-11-25
Filing date: 2010-11-16
Publication date: 2011-06-03
Also published as: KR20130004895A; CN102648115B; US20120281881A1; DE102009055776A1; CN102648115A; JP2013512150A; DE112010003422A5; EP2504209A1; US8824741B2; EP2504209B1; KR101729912B1

Abstract

The invention relates to a method for estimating the roll angle in a travelling vehicle (7), with the following steps. In step a), a camera (8) is used to record a sequence of images from the surroundings of the vehicle, in particular from the carriageway (1) located in front. In step b), at least one signature (S1-S6) on the carriageway surface is extracted from the camera images, i.e. is determined and tracked. From the changed position of the at least one signature (S1-S6) in one or more subsequent camera images, it is determined, in step c), in which direction the camera (8) has been rotated with regard to the roll angle. The amount of roll angle is estimated in step d). For this purpose, the roll angle is either directly estimated in step d1) taking into consideration the vehicle speed(v) and an imaging model of the camera (8), or, in step d2), the roll angle is iteratively increased or reduced by a predetermined correction angle until the roll angle adequately compensates for the rotation of the camera (8). The estimated roll angle arises therefrom in the form of an overall correction value.

Description

Method for estimating the roll angle in a moving

vehicle

The invention relates to a method for estimating the roll angle with a camera in a moving vehicle and finds e.g. Application in vehicles equipped with driver assistance functions.

Driver assistance functions include, for example, traffic sign recognition, automatic light control, vehicle and pedestrian recognition, night vision systems, adaptive cruise control (ACC), parking assistance or automatic control and tracking.

Track finding systems require precise knowledge of the A ^¬ installation position of the camera in the vehicle to estimate track width, track storage and the yaw angle. In particular, these are the height, as well as the roll (or roll), pitch and yaw angle of the camera, which usually have to be determined by a complex tape end or service calibration.

Although tracking systems are able to estimate the pitch and yaw angles, only in the event that the roll angle is known, errors in the roll angle will result in pitch and yaw errors. The roll angle is also referred to as roll angle.

CONFIRMATION COPY A basic problem is that, a small error in the roll angle occur in a Carefully prepared ^¬ term calibration of the camera mounting position after installation in a vehicle to the one that is then retained, and on the other subsequent changes in the installation position of the camera or at ^¬ play, a Uneven loading of the vehicle can lead to a deviation of the roll angle.

The DE 10 2006 018 978 Al shows a method for determining the dynamic roll angle using an apparatus for determining the yaw rate and the specifi ^¬ rule Wankfedersteifigkeit of the vehicle.

It is the object of the presented invention is to provide a method for estimating the roll angle, which enables a current, accurate Rollwin ^¬ kelschätzung in a moving vehicle.

This object is achieved by a procedural ^¬ ren according to the independent claims. Advantageous developments can be found in the dependent claims.

It is a method for estimating the roll angle indicated in egg ^¬ nem vehicle with the following Schrit ^¬ th. In step a) a sequence of images of the vehicle environment, in particular of the forward roadway is taken with a camera. In step b) is determined from the Ka ^¬ merabildern is at least one signature on the roadway surface ^¬ extracted, so ermit- their shape and position telt and tracked (tracked). A signature is a structure on the road surface, such as the beginning or the end of a mark on the road. From the changed position of the at least one signature in one or more subsequent camera image (s), it is determined in step c) in which direction the camera is rotated with respect to the roll angle.

The amount of roll angle is estimated in step d). For this purpose, either in step dl) taking into account the vehicle speed (of the own vehicle) and ei ^¬ Nes imaging model of the camera, the roll angle is estimated directly or in step d2) the roll angle is iteratively increased or decreased by a correction angle until the roll angle, the rotation of the camera compensated. In the iterative roll angle estimation, the roll angle at the beginning of the method is preferably estimated to be zero. In the course of the method, the (currently estimated) roll angle is corrected by a correction angle corresponding to the direction of rotation determined in step c) until it is concluded from the changed position of the at least one signature in one or more subsequent camera images that the roll angle sufficiently compensated for the rotation of the camera. A sufficient compensation can be seen in the fact that the corrected roll angle in the context of the angular resolution no longer deviates from zero (eg deviation less than 0.1 ° or 0.5 °). From this, the absolutely estimated Roil angle as the total correction value by the number of performed corrections with the given Correction angle is multiplied or with varying Kor- correction values as the sum of the total corrections made.

The advantage of the invention is that the roll angle in the moving vehicle can be estimated up to date and precisely. No device for yaw rate determination is needed for the method according to the invention. In test runs, a roll angle deviation of up to seven degrees could be estimated and compensated by the method.

The invention is based on the consideration that in the event of a roll angle of zero all points within one of a shot by the camera of the image representing the surface of the roadway, having an identical distance to the camera in the projection of the vehicle longitudinal direction in the world coordinate Zei ^¬ le. This consideration is based on the assumption that the road is essentially flat. This corresponds to a "fiat earth geometry" assumption for the road, where world coordinates are the coordinates in real space, as opposed to the image coordinates given, for example, by lines and columns of the camera image Points whose image coordinates are known can be projected back onto object points in world coordinates when a complete imaging model of the camera is known, since the signatures can by definition lie on the road surface and assuming that the road surface is flat, it suffices to have a simple rear projection to the Determine location of a signature in world coordinates. In the invention, in particular the compo nent ^¬ interested in the vehicle longitudinal direction of the distance between a point on the road surface and the camera in the real space.

A non-zero roll angle results in points ^within a line of an image taken by the camera representing the surface of the roadway having an increasing or decreasing distance in the projection of the vehicle's longitudinal direction in world coordinates. Are ^{beispielswei ¬} se lane points that lie in the image in a row in real space from left to right more and more distant, so the camera is rotated relative to their viewing direction to the left.

In order to determine the distance in the projection of the vehicle longitudinal direction for pixels, the length of the flow vectors can be analyzed in particular from the optical flow of the pixels. For example, if the vertical components of the flux vectors of road points that are in the picture in a row, from left to right increas ^¬ ingly, the camera is turned to the right.

An advantageous embodiment of the invention provides that in step dl) taking into account an imaging model of the camera, the distance of a signature to the camera in the vehicle longitudinal direction is determined. For this example, a representation of the camera image ermit ^¬ telten signature (s) in bird's eye view of rear projection ekti- be created on and the distance between the signature and camera in the vehicle longitudinal direction are determined from this representation.

Preferably, a distance of the signature after a time Δτ is predicted from the determined distance of the signature (to the camera in the vehicle longitudinal direction) and the vehicle speed.

Preferably, the distance of the same signature to the camera in the vehicle longitudinal direction is measured by rear projection from a subsequent image, which was taken a time Δt after the first image.

Preferably, the roll angle is estimated from a deviation of the measured distance from the predicted distance, taking into account an imaging model of the camera. In this case, it is assumed that the prediction provides a distance with roll angle zero and in the event of a deviation the roll angle can be directly determined, at which the camera would have measured a distance which corresponds to the predicted distance.

An uncertainty in the value of the vehicle speed can lead to consequential errors in the roll angle estimation. In a preferred embodiment of the invention, therefore, a known error in terms of a previous measurement uncertainty in the vehicle speed is taken into account in the estimation of the roll angle and the resulting estimated roll angle error, in particular by means of ei ^{¬ a} Kalman filter.

According to an advantageous embodiment of the invention, in step b) at least one first signature in the left half of the image and in the same or a subsequent image at least one second signature in the right half of the image is determined and tracked.

In addition, the vehicle speed is stored at the time of the respective taking of the camera image.

In step c), a prediction of the signature spacing is performed separately for each signature taking into account the respective vehicle speed and the imaging model with the currently estimated roll angle of the camera. The signature spacing is measured in a subsequent image. It is determined in which direction the camera (the roll angle) is rotated by comparing the deviation of the predicted signature spacing with the measured signature spacing for the first with the deviation for the signature spacing of the second signature.

The roll angle is iteratively estimated according to step d2). This embodiment offers the advantage that errors in the vehicle speed can be compensated by comparing left and right signature deviations. Thus, this method offers a high degree of accuracy.

In a preferred embodiment, in step b) at least one first signature in the left and in the same or a subsequent image, a second signature in the lower right image quadrant is determined and tracked. In addition, will stored the vehicle speed at the time of each recording of the camera image.

In step c), for each of the signatures, the change of the vertical signature position in the image is related to the respective vehicle speed. This ratio is compared for both halves of the image, it being concluded that the camera is rotated in the direction of the image half in relation to its viewing direction in which this ratio is greater.

The roll angle is iteratively estimated according to step d2). This method has the advantage that errors in the absolute vehicle speed hardly affect.

The invention also relates to a device for estimating the roll angle in a moving vehicle with a camera and with means for estimating the roll angle according to the invention.

The invention will be explained with reference to Ausführungsbei ^¬ play and figures.

Fig. 1: Diagram of a method for roll angle estimation

Fig. 2: Camera image of a roadway with signatures

Fig. 3: Representation of the reproduced by the camera image scene in bird's eye view

The flowchart in Fig. 1 shows an embodiment of a method for estimating the roll angle. In a moving vehicle, in step a), a camera (8) is used to take a picture of the vehicle surroundings, which includes the road ahead. In step b), at least one signature (S1-S6) on the road surface is determined from the camera image and its position in the image determined. Sine signature (S1-S6) is a structure on the road surface, for example, the beginning (S2, S4) or the end (S3) of a mark on the road. In a following step a) another image is taken, from which the current position of the signature (S1-S6) in the image is determined. From the changed position of the at least one signature (S1-S6) it is determined in step c) whether and in which direction the camera (8) is ^{rotated relative} to the roll angle. If the camera (8) is rotated, the roll angle is corrected in step d). The estimate of the roll angle he ^¬ are from the sum of the corrections in step d).

Fig. 2 shows an example of a camera image of the vehicle environment Conversely ^¬ as it is received in step a). The vehicle is traveling on a carriageway (1) bounded by left (2) and right (3) continuous lane markings. The middle of the lane (1) is marked by a dashed median strip. On the road surface, a variety of signatures can be seen. The signatures

(S1-S6) are indicated in the figure. The detection of the signatures (S1-S6) takes place in step b). The signature (SI) is an additional mark on the left lane side. The signatures (S2) and (S) or (S3) are the beginning and the end of center strip marks. The signature

(S5) is a crack of the road surface and the signature (S6) is a structure on the right continuous pavement marking (3), for example, a stain or a hole thereof. The auxiliary lines (4) and (5) mark the horizontal or vertical center of the image. The image rows and columns run parallel to the auxiliary lines (5) and (4). The fact that the horizon (6) is not horizontal gives the human observer an indication that the roll angle under which this image was taken deviates from zero and the camera (8) is turned to the left.

A representation of the scene represented by the camera image in bird's eye view is shown in FIG. 3.

The creation of this representation of the represented by the camera image scene in bird's eye view (birdview or top down view) with markers and signatures (S1-S6) in world coordinates can be done from the 3ildkoordinaten with a step size of eg 5cm by a back projection, the from the image model the camera (8) was ermit ^¬ telt taking into account the actual estimated roll angle.

The signatures (S1-S6) on the road surface are extracted from this representation. The distance (d) of a signature (S1-S6) to the camera (8) in the direction of the vehicle longitudinal axis in world coordinates is determined therefrom.

The position of this signature (S1-S6) in world coordinates is predicted for a later time considering the vehicle's airspeed (v). The later time At is chosen so that at this later time At a subsequent image with the Ka mera (8) is recorded. At the later time is al so ^¬ by a multiple of the reciprocal of 3ildfrequenz later (eg 40ms) than the time t = 0, the recording of the first image (Fig. 2). The image taken at the later time At is not shown.

A prediction at a time Δt for a signature (S1-S6) whose distance from the camera (8) at the time t = 0 in world coordinates d_0 is obtained, for example, from: d_est = d_0-v-At

where v is the speed of the vehicle determined at time t = 0.

The actual distance (d_meas) of this signature (S1-S6) in world coordinates is measured from the corresponding representation of the image taken at the later time dt.

The ratio of measured to prädiziertem Signaturab ^¬ was determined: d_meas / d_est.

In this way, as long as signatures in the left (S1-S3) and in the computational ^¬ th (S4-S6) half of the image to a suffi ^¬ sponding number N (in predictions and measurements, for example, can be analyzed according to a further embodiment, N = 10 or N = 50) of signature pitch changes between every two consecutive pictures for the left (N_links> N) and for the right (N_right> N) picture half. The ratio of the predicted to measured Signa ^¬ turabstände is averaged for the left and the right half (d_meas_links / d_est and d_meas_rechts / d_est). Then, an update of the estimated roll angle value may be performed as follows: • If d_meas_links / d_est> d_meas_right / d_est, the roll angle is increased by a correction angle.

• If d_meas_links / d_est <d_meas_right / d_est, the roll angle is reduced by a correction angle.

Otherwise, the roll angle is retained.

The correction angle may be a predetermined constant value (e.g., 0.05 ° or 0.1 °) that determines the resolution of the roll angle estimate, or the correction angle may assume a value proportional to the amount of (d_meas_links / d_est-d_meas_right / d_est).

By forming the ratio of measured (d_meas) to prädiziertem (d_est) Signature far the Einflüs ^¬ se a change in velocity of the vehicle are taken into account.

reference numeral

1 lane

2 Left continuous lane marking

3 rights continuous road markings

4 Horizontal image center

5 Vertical image center

6 horizon

7 vehicle

8 camera

S1-S6 Signature 1 to 6

d distance signature to camera in the direction of

Vehicle longitudinal axis in world coordinates v Vehicle speed

Claims

claims

Method for estimating the roll angle in a moving vehicle (7) with a camera (8) comprising the following steps:

a) the camera (8) records a sequence of images of the vehicle surroundings,

b) at least one signature (S1-S6) on the road surface is determined from the camera images and tracked (tracked)

c) from the changed position of the at least one signature (S1-S6) in one or more subsequent camera images (ern) it is determined in which rolling angle direction the camera (8) is rotated d) the amount of the roll angle

1) taking into account the vehicle speed (v) and an imaging model of the camera (8) directly estimated or

2) iteratively corrects by a correction angle until the estimated roll angle sufficiently compensates for the rotation of the camera (8).

2. The method of claim 1, wherein in step dl), taking into account an imaging model of the camera (8) of the distance (d) of at least one signature

(S1-S6) to the camera (8) in the vehicle longitudinal direction is determined.

3. The method of claim 2, wherein from the determined distance (d) of the signature (S1-S6) and the vehicle speed (v) a distance (d_est) of the signature (sl-S6) is predicted after a time Δt.

4. The method of claim 3, wherein the distance (d_meas) of the same signature (S1-S6) to the camera (8) in the vehicle longitudinal direction by rear projection is measured from a subsequent image, which was taken a time At after the first.

Is 5. The method of claim 4, wherein from a ^¬ deviate deviation of the measured distance (d_meas) for predicative ed distance (d_est) using a mapping scheme of the camera (8) ge ^¬ estimates the roll angle.

6. The method of claim 1, wherein

in step b) at least one first signature (S1-S3) in the left half of the image and in the same or a subsequent image a second signature (S4-S6) in the right half of the image is detected and tracked, also the vehicle speed (v) at the time of respective recording of the camera image is stored,

in step (c) for each signature S1-S6) is performed separately, taking into account the respective Fahrzeugge ^¬ speed (v) and an imaging model of the camera (8) is a prediction of the signature distance (d_est) and the signature distance (d_meas) in a subsequent image is measured and determines in which direction the camera (8) is rotated by the deviation of the prädizier- th signature distance (d_est) with the measured taking into account the ^¬ current estimated roll angle signature distance (d_meas) is compared in the two halves of the picture, and

the roll angle is estimated iteratively according to step d2).

7. The method of claim 1, wherein

in step b) at least one first signature (S1-S3) in the left and in the same or a subsequent image a second signature (S4-S6) is determined and tracked in the lower right quadrant and also the vehicle speed (v) at the time of each recording the camera image is saved,

in step c) for each of the signatures (S1-S6) the change in the signature position in the image is compared to the respective vehicle speed (v) and this ratio is compared for both image halves, it being concluded that the camera (8) in the direction of the picture half is turned, in which this ratio is larger, and

the roll angle is estimated iteratively according to step d2).

8. A device for estimating the roll angle in a moving vehicle (7) comprising a camera (8) and an evaluation unit with means for estimating the roll angle according to one of claims 1 to 7.