WO2018202259A1

WO2018202259A1 - Assembly and method for calibrating the height of a monocular vehicle camera

Info

Publication number: WO2018202259A1
Application number: PCT/DE2018/200041
Authority: WO
Inventors: Thomas Ruland; Aless Lasaruk
Original assignee: Conti Temic Microelectronic Gmbh
Priority date: 2017-05-05
Filing date: 2018-04-27
Publication date: 2018-11-08
Also published as: DE102017207609A1

Abstract

The invention relates to an assembly, a method and a device for calibrating the height of a monocular vehicle camera (2) and can be used, in particular, in vehicle cameras for driver assistance systems. The method for calibrating the height of the monocular vehicle camera (2) comprises the steps: - capturing a sequence of images (20) using the monocular vehicle camera (2); - driving the vehicle (1) according to a defined course on a motorable surface (3) which has two markings (4, 5) at a predetermined distance d from one another, the markings (4, 5) having a visual contrast to the rest of the motorable surface (3); - determining the ratio d/h by evaluating the sequence of images (20), wherein h is the height of the vehicle camera; and - calculating the height of the monocular vehicle camera using the determined ratio d/h and the predetermined distance d.

Description

Construction and method for height calibration of a monocular vehicle camera

The invention relates to a structure, a method and a device for height calibration of a monocular vehicle camera and can be used in particular for vehicle cameras for driver assistance systems.

The exact mounting height of the camera is important for many camera-based driver assistance functions and vehicle environment data. The shoring height has hitherto been determined by various methods: on the basis of CAD nominal data, the shoring height can be coded in the vehicle camera, i. the installation height is taken as the installation height specified in the CAD (computer-aided design) plans;

Target-based end-of-line calibration (abbreviated hereafter by ECM), ie a calibration that is carried out after installation of the camera and completion of the vehicle at the end of the belt by means of a calibration panel; or

- Auto calibration based on odometer values (see below).

The known methods can bring about the following problems:

CAD nominal data may include:

o for a single copy of a camera of a camera Serienpro- type production is not exactly as it does not take into account the Pro ^¬ duktionsvariation (deviations which arise in the camera or vehicle production) or

This is expensive because the nominal CAD data for each height-varying vehicle variation (e.g.

Wheel sizes) must be adapted;

Target-based ECM: o For this purpose, a target and each vehicle must be brought into a defined position relative to each other. This takes time and generates high costs.

Auto calibration based on odometer values:

o It is possible to determine the camera height in relation to the distance traveled using techniques such as Structure-from-Motion (SFM) or Simultaneous Localization and Mapping (SLAM). The distance traveled is usually be ^¬ true by the odometer in the vehicle and calculates the height. One problem is that odometers are extremely inaccurate and eg

also depend on the wheel circumference.

It is an object of the invention to provide an improved structure and method for height calibration of a monocular vehicle camera.

A first aspect relates to an arrangement that allows a measurement of the camera height after mounting a monocular camera in the vehicle at the end-of-line (EOL) of the vehicle production.

A second aspect relates to a method for height calibration of a monocular vehicle camera using a corresponding arrangement.

A third aspect relates to a device for carrying out a corresponding method, in particular one or more control devices.

A fourth aspect relates to a vehicle with a corresponding device. An arrangement according to the invention for height calibration of a monocular vehicle camera comprises a surface which can be driven by a vehicle. The surface is preferably very flat, at least sufficiently flat (eg height differences of less than one centimeter, preferably less than 5 millimeters or maximum height differences of less than 1% relative to the predetermined distance d).

The drivable surface has two markings at a predetermined distance d. The markings are such that they have a visual contrast to the rest of the drivable surface.

Preferably, the markings are mutually parallel stripes.

Advantageously, a virtual straight line which defines the distance d between the two strips is perpendicular to the direction of the parallel strips. Furthermore, the arrangement includes advantageously in the range of the road surface (ie, on the passable surface, or in its immediate vicinity) arranged objects that are be ^¬ obachtbar as distinctive features captured in the vehicle camera images. A feature is said to be distinctive if it is distinguishable from other features in the image and if it is retrievable (ie identifiable) in another image of a sequence of images.

The objects preferably have a defined object height. The distinctive features can be assigned a known altitude in the vehicle coordinate system. The remaining geometric conditions of the arrangement of the features are assumed to be unknown. Preferably, the objects or features are arranged so ^¬ is in space, that they cover a large area of the image.

Advantageously, the drivable surface is such that features that are already recognizable are already present on it. Preferably, the surface has an asphalt pattern. Thus, well-recognizable features of known height in the vehicle coordinate system that cover a large portion (the lower half of the image) of the image result.

An inventive method for height calibration of a monocular vehicle camera comprises the steps:

- taking a sequence of images by means of the monocular vehicle camera;

- Driving the vehicle according to a defined sequence on a drivable surface having two markings at a predetermined distance d, wherein the markings have a visual contrast to the rest of the drivable surface;

Determining the ratio d / h by evaluating the sequence of images, where h is the height of the vehicle camera; and

Calculating the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d.

Preferably, the markings may be arranged or formed as described above.

Advantageously, the vehicle is guided so that it feeds unaccelerated and straight with a known orientation to the markings. The journey should follow a defined process in this regard with sufficient accuracy to achieve good Ka ^¬ librierungsergebnisse.

Preferably, the vehicle is perpendicular to the virtual line shown above as converted into strips Markings led. In this respect too, the journey should take place in accordance with a defined procedure.

According to a preferred embodiment, the proper motion of the vehicle is taken into account in the evaluation of the sequence of images.

Preferably, the self-motion of the vehicle is determined by an evaluation of the sequence of images, e.g. is reconstructed by means of optical flow or an analysis of feature correspondences and the detection of passage through the markings.

Preferably, the ratio d / h from the spatial Re ^¬ construction of the above distinctive features of known height is determined under the consideration of the marks.

Preferably, the homography is calculated, which corresponds to the drivable surface. From this, the ratio d / h can be determined directly.

Advantageously, therefore, the distinctive features in the sequence of images taken with the vehicle camera are taken into account in the reconstruction of the intrinsic motion or in the calculation of the homography.

A device according to the invention for the height calibration of a monocular vehicle camera comprises at least one control device, which is designed to

receiving a sequence of images as input data, wherein the sequence of images by means of a monocular vehicle camera during a defined travel of the vehicle on a be ^¬ mobile surface having two markings at a predetermined distance d, wherein the markings a visual Contrast to the rest of the drivable surface, are included;

to determine the ratio d / h by an evaluation of the sequence of images, where h is the height of the vehicle camera; and

to calculate the height of the monocular vehicle camera from the determined ratio d / h and the predetermined distance d.

In other words, the following solution can be realized: Use of simple markings on the ground (for example, two thick black or white cross-lines crossed) at a known distance (e.g., 10 m). These markings can be easily detected in the camera. Thus, the actually traveled distance (odometry independent) can be determined and thus the camera installation height can be measured exactly (see embodiment).

Advantages of the invention are:

Robust against dirt / damage

- Great cost saving potential

Time savings in production

Improvement in Accuracy (Average Error vs. True Value, and Scattering) Further features, advantages and effects of the invention will become apparent from the following description of preferred embodiments of the invention. Showing:

1 shows schematically an arrangement of markings on a roadway for height calibration of a monocular vehicle camera; Figure 2 shows schematically a set of the vehicle camera ^¬ exempt image with a measuring field.

3 schematically shows an average intensity profile measured in a measuring field during a sequence of images of the vehicle camera

FIG. 4 schematically shows a side view for illustrating an exemplary method of FIG

Altitude determination.

Using structure-from-motion methods it is possible to reconstruct the environment down to a scale ambiguity based on a sequence of images and known intrinsic camera parameters. Assuming additionally that only the ground in front of the vehicle is visible in a certain image area (region-of-interest, ROI), the camera height (except for a scale ambiguity) can be determined from this.

The method presented here is used at the end of the tape after vehicle production to resolve this scale ambiguity and thus to determine the camera height clearly.

Fig. 1 shows a bird's eye view of an arrangement of two markings (4, 5) on a roadway (3) for height calibration of a monocular camera (2) of a vehicle (1).

To measure the installation or installation height of the camera (2) in the vehicle (1) two markings (4, 5) with a defined distance d (eg 10 meters) are mounted on the ground. The lines (6, 7 and 8) shown in Fig. 1 define the roadway (3) on which the markings (4, 5) are applied. They serve mainly the illustration of the roadway area in the schematic vehicle camera image (20) in FIG. 2.

In Fig. 1 is shown with an arrow that the vehicle moves forward (v) (to the right in Fig. 1), preferably straight ahead at a constant speed. As a result, the vehicle (1) passes over the markings (4, 5) at a right angle. Such a defined course of travel (direction perpendicular to markings and at constant speed) ensures optimum results of the altitude calibration.

FIG. 2 schematically shows an image (20) taken by the vehicle camera (2) of the situation illustrated in FIG. Due to the camera perspective, the roadway (3) assumes a trapezoidal shape (6, 7, 8) in the camera image (20).

A measuring field (10) in the form of a rectangle, shown with dashed lines, is provided in the camera image (20). The measuring field (10) can also be referred to as a scanning area. When the vehicle (1) is moving, the measuring field (10) scans the ground or the roadway (3) in front of the vehicle (1).

Within the measuring field (10) is an average

Intensity value determined. The average intensity value may be, for example, the average gray value. For this purpose, it is advantageous that the markings (4, 5) have a high contrast to the rest (as homogeneous as possible) surface of the roadway (3). For example, For example, the markers may be black (as shown in Figures 1 and 2) or white if the roadway (3) is gray. As a result, the method is also suitable for monochrome cameras (2), which offer no color resolution.

Alternatively, colored markings (4, 5) can be used which strongly respond to one or more color channels of a color-resolving camera (2). Even so, the average intensity value (of this color channel) changes markedly when the colored marking (4; 5) comes to lie in the measuring field (10) of a color camera. _n

Advantageously, it should be checked with image processing methods, whether the vehicle is approximately straightforward on the markers moves (direction of movement perpendicular to the markers), so that the drive over the calibration distance according to a defined procedure. If the latter is not the case, then an error can be output and the approach can be repeated. Alternatively, the direction of travel over the calibration path may be e.g. be defined by attaching pylons or the vehicle is moved along rails.

FIG. 3 shows schematically an average intensity profile I measured in a measuring field (10) during a sequence of images with index n of the vehicle camera (2).

When crossing a marking (4; 5) (or when the marking (4; 5) is at a certain distance in front of the moving vehicle (1), so that it is detected by the measuring field (10)), the average intensity changes - / gray value I within the measuring field (10) significantly. When the marks (4, 5) are black as shown in Figs. 1 and 2, the average intensity I curve passes through a minimum.

As a result, the image index n1 or n2 can be determined when the first (4) or second (5) mark is passed over. Since the distance d between the first marker (4), image index nl and the second mark (5), image index is known n2 talking ent ^¬ the distance between image index nl and image index n2 distance, from the camera height can be determined over the ground.

Fig. 4 illustrates the process of height determination.

At time A, the vehicle camera (2), which is inclined (lowered) by an angle relative to the horizontal, detects the first marking (4) on the roadway (3). The angle is essentially determined by the mounting position of the camera (2) in the vehicle (1) and can be based on the observation of the distinctive features be determined. Different tire diameters, eg due to different tire pressures, can lead to variations in the angle. In general, these variations for a vehicle (1) during driving over the calibration assembly or bottom surface are considered to be constant. In contrast, dy namic ^¬ changes in the angle due to a pitch motion of the vehicle (1) may play a role, which adversely affect the height determination. The roadway (3) should be as level as possible. The movement of the vehicle (1) should be as uniform as possible, acceleration or deceleration during a test drive should be avoided, otherwise the vehicle body oscillates, which in particular leads to pitching movements of the camera. For this reason, an unaccelerated straightforward approach to the markings is advantageous.

Basically, known methods for estimating the natural motion (ego-motion, EMO) of a monocular camera (2) provide the ratio c = d / h (1) of the distance traveled d to the camera height h between the marks (4, 5) corresponding to the times A. and B. The ratio c = d / h may preferably be calculated by determining a homography describing the movement of points on the roadway plane between times A and B (given by minima n1 and n2 in FIG. 3, respectively). A homography describes the correspondence of points on a plane between two camera positions or the correspondence of two points in two consecutive images of the camera (2) moved with the vehicle (1). Thus, the homography for the Bo ^¬ dene bene may be determined by coordinates of distinctive Features on the road surface in a first image and a subsequent second image.

If distinctive features of known height are present in the vehicle coordinate system as described (for example on the ground) and can therefore be observed with the vehicle camera, the observed distinctive features can be used in the reconstruction of the self-motion or in particular for the calculation of the homography from the image sequence.

The particular method depends on the spatial arrangement of the features in space and simplifies the classical Ho- mographieschätzung by Hartley and Zisserman, if the features on the ground plane and / or lie on a pa to the ground ^¬ rallelen level.

The homography H of the road or ground plane has a factorization into intrinsic camera parameters (matrix K), in rotation (R) and normalized to length 1 translation t of the camera between times A and B, and the normal vector n of the plane and the constant c (ie the ratio d / h) as follows

Ή = K [Rt * n ' _i ] K- ¹ (2), where η ^{λ is} the transposed normal vector of the ground plane with length 1.

It is preferably assumed above that the intrinsic camera parameters (matrix K) are known. From equation (2), the unknown or the ratio c can thus be determined.

Overall, the factorization of the homography between the images at each point in time of the detection results both markings by inserting the ratio c determined according to equation (2) and the known distance d in equation (1) the camera height h as h = d / c (3)

Claims

claims

1. Arrangement for height calibration of a monocular vehicle camera (2), which comprises a vehicle surface (1) passable flat surface (3),

characterized in that the drivable surface (3) at a predetermined distance d two markings (4, 5), wherein the markings (4, 5) have a visual contrast to the rest of the drivable surface (3).

2. Arrangement according to claim 1, wherein the markings are mutually parallel strips.

3. Arrangement according to claim 2, wherein a virtual straight line defining the distance d between the two strips is perpendicular to the direction of the parallel strips.

4. An arrangement according to one of the preceding claims, in which objects are arranged in the region of the roadway, which are as distinctive features captured in the vehicle camera images be ^¬ obachtbar.

5. Arrangement according to claim 4, wherein the objects have a defined object height.

6. A method for height calibration of a monocular vehicle camera (2), comprising the steps:

- taking a sequence of images (20) by means of the monocular vehicle camera (2);

- Driving the vehicle (1) according to a defined sequence on a drivable surface (3) having at a predetermined distance d two markings (4, 5), wherein the Mar ^¬ kierungen (4, 5) a visual contrast to the other passable Having surface (3); - determining the ratio d / h by an evaluation of the sequence of images (20), where h is the height of the vehicle camera; and

7. The method of claim 6, wherein the defined sequence comprises an unaccelerated ride of the vehicle.

8. The method of claim 6 or 7, wherein the defined sequence comprises a straight ahead of the vehicle with a known orientation with respect to the markings.

9. The method of claim 8, wherein the defined sequence comprises driving the vehicle along a virtual line perpendicular to two markers formed as parallel strips.

10. The method according to any one of claims 6 to 9, wherein in the evaluation of the sequence of images, the proper motion of the vehicle is taken into account.

11. The method according to any one of claims 6 to 10, wherein the self-motion of the vehicle is reconstructed by an evaluation of the sequence of images by means of optical flow and detecting the passage of the markings.

12. The method according to any one of claims 6 to 11, wherein the homography is calculated, which corresponds to the drivable surface (3).

13. The method of claim 11 or 12, wherein in the reconstruction of the intrinsic motion or in the calculation of the homography distinctive features in the sequence of images be taken into ^account .

14. A device for height calibration of a monocular vehicle camera (2) comprising a control unit which is designed to

receiving a sequence of images (20) as input data, wherein the sequence of images by means of a monocular vehicle camera (2) during a defined travel of the vehicle (1) on a drivable surface (3), at a predetermined distance d two markings having (4, 5), which are Mar ^¬ markings (4, 5) have a visual contrast with the rest of navigable surface (3) was added;

- determine the ratio d / h by an evaluation of the sequence of images (20), where h is the height of the vehicle camera; and

15. A vehicle (1) with a vehicle camera (2) to be calibrated and a device according to claim 14.