DE102017207609A1 - Construction and method for height calibration of a monocular vehicle camera - Google Patents

Abstract

The invention relates to a structure, a method and a device for height calibration of a monocular vehicle camera (2) and can be used in particular for vehicle cameras for driver assistance systems.
The method for height calibration of the monocular vehicle camera (2) comprises the steps:
- taking a sequence of images (20) by means of the monocular vehicle camera (2);
Driving the vehicle (1) according to a defined sequence on a drivable surface (3) which has two markings (4, 5) at a predetermined distance d, the markings (4, 5) visually contrasting the other drivable surface (3). 3);
- determining the ratio d / h by an evaluation of the sequence of images (20), where h is the height of the vehicle camera; and
Calculating the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d.

Description

The invention relates to a structure, a method and a device for height calibration of a monocular vehicle camera and can be used in particular for vehicle cameras for driver assistance systems.

• - auf Basis von CAD-Nominal-Daten kann die Verbauhöhe in der Fahrzeugkamera kodiert werden, d.h. als Verbauhöhe wird die in den CAD (computer-aided design, „rechnerunterstütztes Konstruieren“)-Plänen angegebene Verbauhöhe übernommen;
• - Target-basierte End-of-line Calibration (im Folgenden durch ECM abgekürzt), also eine Kalibrierung, die nach Einbau der Kamera und Fertigstellung des Fahrzeugs am Band-Ende mittels einer Kalibriertafel durchgeführt wird; oder
• - Auto-Kalibrierung auf Basis von Odometer-Werten (s. unten).

The exact mounting height of the camera is important for many camera-based driver assistance functions and vehicle environment data. The installation height has hitherto been determined by various methods:

• - Based on CAD nominal data, the shoring height can be coded in the vehicle camera, ie as shoring height, the shoring height specified in the CAD (computer-aided design) plans is adopted;
• Target-based end-of-line calibration (hereafter abbreviated to ECM), ie a calibration performed after installation of the camera and completion of the vehicle at the end of the belt by means of a calibration panel; or
• - Auto calibration based on odometer values (see below).

• - CAD-Nominal-Daten sind mitunter:
  • o für ein einzelnes Kameraexemplar aus einer Serienproduktion eines Kameratyps nicht genau, da sie die Produktionsvariation (Abweichungen, die sich bei der Kamera- bzw. Fahrzeugproduktion ergeben) nicht berücksichtigen, bzw.
  • o teuer, da die CAD-Nominal-Daten für jede die Höhe beeinflussende Fahrzeug-Variation (z.B. unterschiedliche Radgrößen) adaptiert werden müssen;
• - Target-basierte ECM:
  • o Hierzu muss ein Target und jedes Fahrzeug in definierte Position relativ zueinander gebracht werden. Das benötigt Zeit und erzeugt hohe Kosten.
• - Auto-Kalibrierung auf Basis von Odometer-Werten:
  • o Es ist möglich die Kamerahöhe durch Techniken, wie Structure-from-Motion (SFM) oder Simultaneous Localization and Mapping (SLAM), im Verhältnis zur zurückgelegten Strecke zu bestimmen. Die zurückgelegte Strecke wird üblicherweise durch den Odometer im Fahrzeug bestimmt und damit die Höhe errechnet. Ein Problem ist dabei, dass Odometer extrem ungenau sind und z.B. ebenfalls vom Radumfang abhängen.

The known methods can bring about the following problems:

• - CAD nominal data may include:
  • o for a single camera copy from a series production of a camera type not exactly because they do not consider the production variation (deviations that result in the camera or vehicle production), or
  • o expensive, since the CAD nominal data must be adapted for each height varying vehicle variation (eg different wheel sizes);
• Target-based ECM:
  • o For this purpose, a target and each vehicle must be brought into a defined position relative to each other. This takes time and generates high costs.
• - Auto calibration based on odometer values:
  • o It is possible to determine the camera height in relation to the distance covered by techniques such as Structure-from-Motion (SFM) or Simultaneous Localization and Mapping (SLAM). The distance traveled is usually determined by the odometer in the vehicle and thus calculates the height. One problem is that odometers are extremely inaccurate and, for example, also depend on the wheel circumference.

It is an object of the invention to provide an improved structure and method for height calibration of a monocular vehicle camera.

A first aspect relates to an arrangement that allows a measurement of the camera height after mounting a monocular camera in the vehicle at the end-of-line (EOL) of the vehicle production.

A second aspect relates to a method for height calibration of a monocular vehicle camera using a corresponding arrangement.

A third aspect relates to a device for carrying out a corresponding method, in particular one or more control devices.

A fourth aspect relates to a vehicle with a corresponding device.

An arrangement according to the invention for height calibration of a monocular vehicle camera comprises a surface which can be driven by a vehicle. The surface is preferably very flat, at least sufficiently even (e.g., differences in height of less than one centimeter, preferably less than 5 millimeters or maximum height differences of less than 1% relative to the predetermined distance d).

The drivable surface has two markings at a predetermined distance d. The markings are such that they have a visual contrast to the rest of the drivable surface.

Preferably, the markings are mutually parallel stripes.

Advantageously, a virtual straight line which defines the distance d between the two strips is perpendicular to the direction of the parallel strips.

Furthermore, the arrangement advantageously comprises objects arranged in the region of the road surface (that is to say on the drivable surface or in its immediate surroundings) which can be observed as distinctive features in images recorded with the vehicle camera. A feature is said to be distinctive if it is distinguishable from other features in the image and if it is in another image a sequence of images is retrievable (ie identifiable).

The objects preferably have a defined object height. The distinctive features can be assigned a known altitude in the vehicle coordinate system. The remaining geometric conditions of the arrangement of the features are assumed to be unknown.

Preferably, the objects or features are arranged in space so that they cover a large area of the image.

Advantageously, the drivable surface is such that features that are already recognizable are already present on it. Preferably, the surface has an asphalt pattern. Thus, well-recognizable features of known height in the vehicle coordinate system that cover a large portion (the lower half of the image) of the image result.

• - Aufnehmen einer Folge von Bildern mittels der monokularen Fahrzeugkamera;
• - Fahren des Fahrzeugs gemäß einem definierten Ablauf auf einer befahrbaren Oberfläche, die in einem vorgegebenen Abstand d zwei Markierungen aufweist, wobei die Markierungen einen visuellen Kontrast zur übrigen befahrbaren Oberfläche aufweisen;
• - Bestimmen des Verhältnisses d/h durch eine Auswertung der Folge von Bildern, wobei h die Höhe der Fahrzeugkamera ist; und
• - Berechnen der Höhe der monokularen Fahrzeugkamera aus dem bestimmten Verhältnis d/h und dem vorgegebenen Abstand d.

An inventive method for height calibration of a monocular vehicle camera comprises the steps:

• - taking a sequence of images by means of the monocular vehicle camera;
• - Driving the vehicle according to a defined sequence on a drivable surface having two markings at a predetermined distance d, wherein the markings have a visual contrast to the rest of the drivable surface;
• Determining the ratio d / h by evaluating the sequence of images, where h is the height of the vehicle camera; and
• Calculating the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d.

Advantageously, the vehicle is guided so that it feeds unaccelerated and straight with a known orientation to the markings. The ride should follow a defined schedule in this regard with sufficient accuracy to achieve good calibration results.

Preferably, the vehicle is guided along the above-mentioned virtual line perpendicular to the converted as a strip marks. In this respect too, the journey should take place in accordance with a defined procedure.

According to a preferred embodiment, the proper motion of the vehicle is taken into account in the evaluation of the sequence of images.

Preferably, the self-motion of the vehicle is determined by an evaluation of the sequence of images, e.g. reconstructed by optical flow or an analysis of feature correspondences and detecting the passage of the markers.

Preferably, the ratio d / h is determined from the spatial reconstruction of the above-noted distinctive features of known height considering the markings.

Preferably, the homography is calculated, which corresponds to the drivable surface. From this, the ratio d / h can be determined directly.

Advantageously, therefore, the distinctive features in the sequence of images taken with the vehicle camera are taken into account in the reconstruction of the intrinsic motion or in the calculation of the homography.

• - eine Folge von Bildern als Eingangsdaten zu empfangen, wobei die Folge von Bildern mittels einer monokularen Fahrzeugkamera während einer definierten Fahrt des Fahrzeugs auf einer befahrbaren Oberfläche, die in einem vorgegebenen Abstand d zwei Markierungen aufweist, wobei die Markierungen einen visuellen Kontrast zur übrigen befahrbaren Oberfläche aufweisen, aufgenommen sind;
• - das Verhältnis d/h zu bestimmen wird durch eine Auswertung der Folge von Bildern, wobei h die Höhe der Fahrzeugkamera ist; und
• - die Höhe der monokularen Fahrzeugkamera aus dem bestimmten Verhältnis d/h und dem vorgegebenen Abstand d zu berechnen.

A device according to the invention for the height calibration of a monocular vehicle camera comprises at least one control device, which is designed to

• receiving a sequence of images as input data, wherein the sequence of images by means of a monocular vehicle camera during a defined travel of the vehicle on a drivable surface which has two markings at a predetermined distance d, wherein the markings in visual contrast to the rest of the drivable surface have been included;
• to determine the ratio d / h by an evaluation of the sequence of images, where h is the height of the vehicle camera; and
• to calculate the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d.

• - Verwendung von einfachen Markierungen auf dem Boden (z.B. zwei dicke schwarze oder weiße Quer-Linien die überfahren werden) in einem bekannten Abstand (z.B. 10m). Diese Markierungen können auf einfache Weise in der Kamera detektiert werden. So kann die tatsächlich zurückgelegte Wegstrecke (Odometrie unabhängig) bestimmt werden und somit die Kamera-Einbau-Höhe exakt vermessen werden (siehe Ausführungsbeispiel).

In other words, the following solution can be realized:

• - Use of simple markings on the ground (eg two thick black or white cross lines to be crossed) at a known distance (eg 10m). These markings can be easily detected in the camera. Thus, the actually traveled distance (odometry independent) can be determined and thus the camera installation height can be measured exactly (see embodiment).

• - Robust gegenüber Verschmutzung/Beschädigung
• - Großes Kosteneinsparungspotenzial
• - Zeitersparnis in der Produktion
• - Verbesserung der Genauigkeit (durchschnittlicher Fehler gegenüber dem wahren Wert, sowie die Streuung)

Advantages of the invention are:

• - Robust against dirt / damage
• - Great cost saving potential
• - Time savings in production
• - Improvement of accuracy (average error against the true value, as well as the dispersion)

• 1 schematisch eine Anordnung von Markierungen auf einer Fahrbahn zur Höhenkalibrierung einer monokularen Fahrzeugkamera;
• 2 schematisch ein von der Fahrzeugkamera aufgenommenes Bild mit einem Messfeld;
• 3 schematisch einen in einem Messfeld gemessenen durchschnittlichen Intensitätsverlauf während einer Folge von Bildern der Fahrzeugkamera; und
• 4 schematisch eine Seitenansicht zur Veranschaulichung eines beispielhaften Verfahrens der Höhenermittlung.

Further features, advantages and effects of the invention will become apparent from the following description of preferred embodiments of the invention. Showing:

• 1 schematically an arrangement of markings on a roadway for height calibration of a monocular vehicle camera;
• 2 schematically an image taken by the vehicle camera with a measuring field;
• 3 schematically an average intensity profile measured in a measuring field during a sequence of images of the vehicle camera; and
• 4 schematically a side view illustrating an exemplary method of height determination.

Using structure-from-motion methods it is possible to reconstruct the environment down to a scale ambiguity based on a sequence of images and known intrinsic camera parameters. Assuming additionally that only the ground in front of the vehicle is visible in a certain image area (region-of-interest, ROI), the camera height (except for a scale ambiguity) can be determined from this.

The method presented here is used at the end of the tape after vehicle production to resolve this scale ambiguity and thus to determine the camera height clearly.

1 shows a bird's-eye view of an arrangement of two markings ( 4 . 5 ) on a roadway ( 3 ) for height calibration of a monocular camera ( 2 ) of a vehicle ( 1 ).

For measuring the installation or installation height of the camera ( 2 ) in the vehicle ( 1 ), two markers ( 4 . 5 ) with a defined distance d (eg 10 meters) on the ground. In the 1 pictured lines ( 6 . 7 and 8th ) limit the carriageway ( 3 ), on which the markings ( 4 . 5 ) are applied. They serve mainly to illustrate the lane area in the schematic vehicle camera image ( 20 ) in 2 ,

In 1 is shown with an arrow that the vehicle moves forward (v) (to the right in 1 ), preferably straight ahead at a constant speed. As a result, the vehicle passes over ( 1 ) the markings ( 4 . 5 ) at a right angle. Such a defined course of travel (direction perpendicular to markings and at constant speed) ensures optimum results of the altitude calibration.

2 schematically shows one of the vehicle camera ( 2 ) recorded image ( 20 ) the in 1 illustrated situation. Due to the camera angle, the roadway ( 3 ) a trapezoidal shape ( 6 . 7 . 8th ) in the camera image ( 20 ) one.

In the camera picture ( 20 ) is a measuring field ( 10 ) in the form of a rectangle, shown with dashed lines, provided. The measuring field ( 10 ) can also be called a scan area. When moving the vehicle ( 1 ) the measuring field ( 10 ) the ground or the roadway ( 3 ) in front of the vehicle ( 1 ).

Within the measuring field ( 10 ) an average intensity value is determined. The average intensity value may be, for example, the average gray value. For this it is advantageous that the markings ( 4 . 5 ) a high contrast to the rest (as homogeneous as possible) surface of the road ( 3 ) exhibit. For example, the markers may be black (as in 1 and 2 shown) or be white when the roadway ( 3 ) is gray. This makes the method also suitable for monochrome cameras ( 2 ) that do not provide color resolution. Alternatively, colored markers ( 4 . 5 ), one or more color channels of a color-resolving camera ( 2 ) strongly respond. Even so, the average intensity value (of this color channel) changes significantly when the colored marking ( 4 ; 5 ) in the measuring field ( 10 ) of a color camera comes to rest.

Advantageously, it should be checked with image processing methods, whether the vehicle is approximately straightforward on the markers moves (direction of movement perpendicular to the markers), so that the drive over the calibration distance according to a defined procedure. If the latter is not the case, then an error can be output and the approach can be repeated. Alternatively, the direction of travel over the calibration path may be e.g. be defined by attaching pylons or the vehicle is moved along rails.

3 schematically shows one in a measuring field ( 10 ) measured average intensity profile I during a sequence of images with index n of the vehicle camera ( 2 ).

When crossing a mark ( 4 ; 5 ) (or if the mark ( 4 ; 5 ) at a certain distance in front of the moving vehicle ( 1 ) it's so that they are from the measuring field ( 10 ) is detected), the average intensity / gray value changes I within the measuring field ( 10 ) significantly. If the marks ( 4 . 5 ) as in 1 and 2 are black, the curve goes through the average intensity I one minimum each.

This allows the image index n1 respectively. n2 when crossing the first ( 4 ) or second ( 5 ) Mark. Since the distance d between the first mark ( 4 ), Image index n1 and the second mark ( 5 ), Image index n2 is known, the between the image index n1 and picture index n2 the distance covered by the camera can be used to determine the camera height above ground.

4 illustrates the process of height determination.

At the time A detects the vehicle camera ( 2 ), which are at an angle α inclined (lowered) relative to the horizontal, the first mark ( 4 ) on the road ( 3 ). The angle α is essentially due to the installation position of the camera ( 2 ) in the vehicle ( 1 ) and can be determined from the observation of the distinctive features. Different tire diameters, eg due to different tire pressures, can lead to variations in the angle α to lead. In general, these variations are for a vehicle ( 1 ) while driving over the calibration assembly or bottom surface to be considered constant. By contrast, dynamic changes in the angle α due to a pitching movement of the vehicle ( 1 ) play a role that negatively affects height determination. The roadway ( 3 ) should be as level as possible. The movement of the vehicle ( 1 ) should be as uniform as possible, accelerating or decelerating during a test drive should be avoided, otherwise the vehicle body will vibrate, resulting in particular in pitching movements of the camera. For this reason, an unaccelerated straightforward approach to the markings is advantageous.

der gefahrenen Strecke d zur Kamerahöhe h zwischen den Markierungen (4, 5) entsprechend den Zeitpunkten A und B.Basically, known methods for estimating the natural motion (ego-motion, EMO) of a monocular camera ( 2 ) The relationship $$\text{c}=\text{d / h}$$

the distance traveled d to the camera height h between the marks ( 4 . 5 ) according to the times A and B.

The ratio c = d / h may preferably be calculated by determining a homography which is the movement of points on the roadway plane between the times A and B (given by the minima n1 respectively. n2 in 3 ) describes. A homography describes the correspondence of points on a plane between two camera positions or the correspondence of two points in two consecutive images of the vehicle ( 1 ) moving camera ( 2 ). Thus, the ground level homography can be determined by using coordinates of distinctive features on the road surface in a first image and a subsequent second image.

If distinctive features of known height are present in the vehicle coordinate system as described (e.g., on the ground) and therefore observable with the vehicle camera, the observed distinctive features can be used in the reconstruction of the self-motion or, in particular, to compute the homography from the sequence of images.

The special method depends on the spatial arrangement of the features in space and simplifies the classical homography estimation according to Hartley and Zisserman if the features lie on the ground level and / or on a plane parallel to the ground.

hierbei ist n' der transponierte Normalenvektor der Bodenebene mit Länge 1.The homography H of the road or ground plane has a factorization into intrinsic camera parameters (matrix K), in rotation (R) and on length 1 normalized translation t of the camera between times A and B , as well as the normal vector n of the plane and the constant c (ie the ratio d / h) as follows $$H=K\left[R-t*n\text{'}\frac{d}{H}\right]K^{-1}$$

where n 'is the transposed normal vector of the ground plane with length 1 ,

It is preferably assumed above that the intrinsic camera parameters (matrix K) are known.

From equation (2), the unknown or the ratio c can thus be determined.

Overall, the factorization of the homography between the images at times of detecting both marks by substituting the ratio c determined according to equation (2) and the known distance d in equation (1) gives the camera height H when $$\text{H}=\text{d / c}$$

Claims (12)

Arrangement for the height calibration of a monocular vehicle camera (2) which comprises a flat surface (3) which can be driven by a vehicle (1), characterized in that the drivable surface (3) has two markings (4, 5) at a predetermined distance d, wherein the markings (4, 5) have a visual contrast to the rest of the drivable surface (3). Arrangement according to Claim 1 , where the marks are mutually parallel stripes. Arrangement according to Claim 2 wherein a virtual straight line defining the distance d between both strips is perpendicular to the direction of the parallel strips. Arrangement according to one of the preceding claims, wherein in the region of the roadway objects are arranged, which are observable as distinctive features in images taken with the vehicle camera. Arrangement according to Claim 4 , wherein the objects have a defined object height. A method of height calibration of a monocular vehicle camera (2), comprising the steps of: - taking a sequence of images (20) by means of the monocular vehicle camera (2); Driving the vehicle (1) according to a defined sequence on a drivable surface (3) which has two markings (4, 5) at a predetermined distance d, the markings (4, 5) visually contrasting the other drivable surface (3). 3); - determining the ratio d / h by an evaluation of the sequence of images (20), where h is the height of the vehicle camera; and Calculating the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d. Method according to Claim 6 , wherein in the evaluation of the sequence of images, the proper motion of the vehicle is taken into account. Method according to Claim 6 or 7 wherein the self-motion of the vehicle is reconstructed by evaluating the sequence of images by optical flow and detecting the passage of the markers. Method according to one of Claims 6 to 8th in which the homography corresponding to the drivable surface (3) is calculated. Method according to Claim 8 or 9 , wherein in the reconstruction of the self-motion or in the calculation of the homography distinctive features are taken into account in the sequence of images. Device for height calibration of a monocular vehicle camera (2) comprising a control device which is designed receiving a sequence of images (20) as input data, wherein the sequence of images by means of a monocular vehicle camera (2) during a defined travel of the vehicle (1) on a drivable surface (3), at a predetermined distance d two markings (4, 5), wherein the markings (4, 5) have a visual contrast to the rest of the drivable surface (3) are received; - determine the ratio d / h by an evaluation of the sequence of images (20), where h is the height of the vehicle camera; and to calculate the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d. A vehicle (1) with a vehicle camera (2) and a device according to Claim 11 ,

Images