DE102017117594A1 - Automated detection of headlight misalignment - Google Patents

Abstract

The present invention relates to a method for automated detection of a headlight misalignment, in which a vehicle in the front region at least one headlamp and at least one camera whose calibration data are known, wherein the at least one camera images (400) in the form of pixels receives, wherein the epi-polar geometry is used to calculate an epipolar line (404) for each pixel (404) of the image (400), the at least one headlight projecting at least one characteristic feature into a vehicle apron, the at least one camera capturing two images at a predetermined time interval (FIG. 400) of the vehicle apron and forwards to a memory, wherein image processing software in the first of the camera searches for at least one characteristic feature (302), wherein a respective pixel (402) associated with the at least one characteristic feature (302) is identified, wherein the Bildver in the second image searches for the same characteristic feature (302) and determines if it is located on the epipolar line (404) of the respective associated pixel (402) of the first image, wherein a deviation from the epipolar line (404) an error function is calculated, and wherein when the calculated value of the error function is exceeded by a predetermined value of the error function, an automatic headlight calibration is triggered, which generates new calibration data.

Description

The present invention relates to a method and a system for an automated detection and correction of a headlight misalignment by means of epipolar geometry.

A 3D reconstruction of a space in front of an optical system forms the basis for a computer-aided image analysis and is of essential importance for autonomously moving systems whose surrounding space has to be explored. A standard method of 3D reconstruction is triangulation, i. a distance measurement by exact angle determination within a triangle, which form two points of the optical system with a respective point of an object in space. One distinguishes between passive and active triangulation. In passive triangulation, there is one observer at each of two points of the optical system, which is used, for example, in a stereo camera which takes an object in space at different angles. In contrast, in the case of active triangulation, which is a measuring principle frequently used in optical distance measurement, an observation point is replaced by a light source, ie the triangle is formed by the light beam, the illuminated object and the camera.

In a motor vehicle, the light source is advantageously already given by a headlight, which forms the optical system together with a built-in front camera. By means of the headlamp, a pattern can be projected into a vehicle apron, which is absorbed by the camera as a function of a surface structure in the vehicle apron or of objects located there. Subsequently, an image processing system performs a feature search, for example, the identification of several characteristic changes of light-dark boundaries within the projected pattern. For the quality of the image processing, the exact knowledge of a vehicle-specific light distribution of the headlight is of great importance, which makes a calibration of a headlight camera system indispensable.

The calibration takes place initially during the production of the vehicle and is then generally no longer adjusted. It includes the position, orientation and internal structure of the headlight and the camera as calibration data. Only the automated adjustment of the angle of inclination taken by the headlamp remains variable. For this purpose, two methods from the prior art are mentioned below.

An adjustment of a headlight on the basis of the detected by a camera dipped beam whose illuminated area forms a light-dark boundary with a darker environment is, for example, in the document DE 10 2011 109 440 A1 disclosed. By means of linear model equations, a determination of the cut-off line is supported, from which a tilt angle of the headlamp is calculated and compared with a desired position.

Another method for checking the adjustment of a headlight in a motor vehicle is shown in the document DE 10 2013 211 876 A1 on. As a result, the adjustment can also take place with different, not necessarily planar projection surfaces. A position of the projection surface relative to a vehicle position, or the headlight position and orientation, must be known.

With a headlight whose exact position and orientation has been calibrated relative to the camera installed in the front area, active triangulation can be used to create a depth map of the area in front of the vehicle. It is problematic if, for example, adjusting screws of the headlight assembly by maintenance in the engine compartment, if necessary. Be casually adjusted, and then by using non-corrected calibration data then erroneous distance values come to pass.

A basic, theoretical description of the existing of two observation points and an object optical system is given by the Epipolargeometrie, which describes a relationship between different shots of the same object. A pixel formed by the object, which only changes its distance in a line of sight to a first observation point and thus appears invariable as a point, corresponds to a pixel which, viewed from the second observation point, only appears on one line, the so-called epipolar line. changed. Accordingly, with known epipolar geometry for finding the pixel only along this epipolar line must be searched. In the field of machine vision, the epipolar geometry is of great importance. The publication DE 10 2012 218 390 A1 discloses a method that is intended to enable a vehicle using the Epipolargeometrie, for example. Traffic signs to recognize.

Against this background, it is an object of the present invention to provide a method for on-the-go calibration of the headlights, which detects changes in the calibration data, corrects and continuously performs distance measurements to objects in front of the vehicle by means of active triangulation allows. Furthermore, it is an object of the present invention to provide a corresponding system for carrying out such a method, and a vehicle using the method and the system.

To achieve the object mentioned above, a method is provided for an automated detection of a headlight misalignment, in which a vehicle in the front region has at least one headlight and at least one camera whose calibration data are known, wherein the at least one camera takes pictures in the form of pixels an epipolar line is calculated from the calibration data by means of epipolar geometry for each pixel of the image, the at least one headlight projecting at least one characteristic feature into a vehicle apron, wherein the at least one camera takes two images of the vehicle apron at a predetermined time interval and forwards them to a memory, wherein image processing software in the first image of the camera searches for at least one characteristic feature identifying a respective pixel associated with the at least one characteristic feature, wherein the image processing means oftware in the second image searches for the same characteristic feature and determines on discovery whether it is on the epipolar line of the respective associated pixel of the first image, wherein a deviation from the epipolar line an error function is calculated, and wherein when exceeding the calculated Value of the error function of a predetermined value of the error function, an automatic headlight calibration is triggered, which generates new calibration data.

welche Rotation und Translation eines Objektpunktes in die jeweilige Bildebene umfasst, und einer jeweiligen den inneren Aufbau, also bspw. im Fall einer Digital-Kamera für die Abbildung in die diskreten Pixelkoordinaten eines Bildsensors wichtige Parameter wie Brennweite (f_x, f_y), Positionsvektor in Bildebene (c_x, c_y, 1)^T, und sogenannter Skew s, beschreibenden intrinsischen Matrix K, $$K=\left(\begin{array}{ccc}{f}_x& s& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right).$$

A course of the same feature, detected in several images preceding the current image, along an epipolar line is given in the case of a calibrated headlight and camera when the headlight or the vehicle is moving toward a stationary object onto which the pattern is projected. An assignment of image planes of the headlight and of the camera necessary for a calculation of the epipolar lines takes place by means of a respective extrinsic matrix R describing the outer geometry, ie position and orientation in space. $$R=\left(\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\end{array}\right).$$

which rotation and translation of an object point into the respective image plane comprises, and a respective the internal structure, so for example in the case of a digital camera for the mapping into the discrete pixel coordinates of an image sensor important parameters such as focal length (f _x , f _y ), position vector in image plane (c _x , c _y , 1) ^T , and so-called skew s, descriptive intrinsic matrix K, $$K=\left(\begin{array}{ccc}{f}_x& s& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right),$$

For calculations by means of epipolar geometry, only a knowledge of a relationship of calibrated systems, here the headlight and the camera, is necessary.

Due to this relationship, the epipolar lines can be calculated, for example, for each pixel of the image sensor of a digital camera.

In a further embodiment of the method according to the invention, a checkerboard pattern of light and dark surface elements is selected as the pattern. Other patterns that have light-dark transitions, such as stripes or concentric circles, are conceivable, but should advantageously have horizontal and vertical differences in structure throughout the projected pattern.

In yet another embodiment of the method according to the invention, the characteristic feature is selected as a pixel having two transitions from light to dark in pixels of its immediate vicinity in an order from upper left to upper right to lower right to lower left to upper left. In particular, this is the case for a checkerboard pattern in all vertices of those surface elements that are not on the edge of the checkerboard pattern. In an embodiment of the method according to the invention, four features detected in the respective corners of a dark or bright surface element of the checkerboard pattern lead to an assignment of the center of the dark or bright area element to a pixel of the camera and the epipolar line connected therewith.

If a characteristic feature is detected in the current image, further occurrences are searched for in the preceding images on the epipolar line of the pixel which is associated with the feature or, as mentioned above, a pixel derived from a plurality of detected features. In this case, a wrong decision due to a first occurrence of a detected feature, such as when an object is first illuminated by the headlamp and thus the feature can not be present in previous images, thereby excluded that only in a predetermined number of images, the temporally when the feature was first created by the camera. A statistic of the detection of the at least one characteristic feature on the corresponding at least one epipolar line in preceding images is mapped into a value by means of an error function. If this value of the error function falls below a predetermined value, ie a threshold value, which means that the feature on the epipolar line is rarely or not detected in previous images, a headlight orientation deviating from the initial calibration must be assumed.

The error function can be selected, for example, as the sum of all characteristic features detected on an epipolar line, divided by the total number of all detected characteristic features.

In a further embodiment of the method according to the invention, the predetermined time interval between the first and second image of the at least one camera is selected as a time interval of two images produced or recorded directly after one another by the at least one camera. The inventive method is thus carried out for each so-called camera frame.

If the value of the error function has exceeded a predetermined value, and so erroneous calibration data are assumed, automatic headlight calibration is initiated. In this case, in a further embodiment of the method according to the invention, the epipolar line is calculated for each pixel from the new calibration data.

In yet another embodiment of the method according to the invention, the modified calibration data are made available to a distance measurement by active triangulation.

It would also be conceivable as an error function, for example, to select a reciprocal of the error function mentioned above by way of example, in which case an exceeding of a predetermined threshold value would suggest a headlight orientation deviating from the initial calibration. Any other suitable error function with a suitably selected threshold value is conceivable.

Furthermore, a system for automated recognition of a headlight misalignment is claimed which has at least one headlight, at least one camera, a computing unit with a memory unit and image processing software and is configured to carry out the method according to the invention.

Finally, a vehicle equipped with the system according to the invention is claimed, which carries out the method according to the invention.

Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

It is understood that the features mentioned above and those yet to be explained below can be used not only in the particular combination indicated, but also in other combinations or in isolation, without departing from the scope of the present invention.

• 1 zeigt in schematischer Darstellung Bilder eines sich einer Mauer nähernden Scheinwerfer-Kamera-Systems, wobei der Scheinwerfer einen Lichtpunkt auf die Mauer projiziert, gemäß einer Ausführungsform des erfindungsgemäßen Verfahrens.
• 2 zeigt in schematischer Darstellung ein Bild aus einer Kamera mit einem in ein Fahrzeugvorfeld projizierten Schachbrettmuster und detektierten charakteristischen Merkmalen gemäß einer Ausführungsform des erfindungsgemäßen Verfahrens.
• 3 zeigt in schematischer Darstellung ein Bild aus einer Kamera mit einem in ein Fahrzeugvorfeld projizierten Schachbrettmuster mit einer Suche nach einem charakteristischen Merkmal entlang einer dejustierten Epipolarlinie gemäß einer Ausführungsform des erfindungsgemäßen Verfahrens.
• 4 zeigt in schematischer Darstellung ein Bild aus einer Kamera mit einem in ein Fahrzeugvorfeld projizierten Schachbrettmuster mit einer Suche nach einem charakteristischen Merkmal entlang einer mit neuen Kalibrierdaten berechneten Epipolarlinie gemäß einer Ausführungsform des erfindungsgemäßen Verfahrens.

The figures are described coherently and comprehensively, the same components are assigned the same reference numerals.

• 1 shows a schematic representation of images of a wall approaching headlight camera system, the headlight projecting a point of light on the wall, according to an embodiment of the method according to the invention.
• 2 shows a schematic representation of an image from a camera with a projected into a vehicle apron checkerboard pattern and detected characteristics in accordance with an embodiment of the method according to the invention.
• 3 shows a schematic representation of an image from a camera with a projected into a vehicle apron checkerboard pattern with a search for a characteristic feature along a misaligned epipolar line according to an embodiment of the method according to the invention.
• 4 shows a schematic representation of an image from a camera with a projected into a vehicle apron checkerboard pattern with a search for a characteristic feature along a calculated with new calibration data epipolar line according to an embodiment of the method according to the invention.

In 1 become schematic pictures 102 . 104 . 106 . 108 . 110 a headlight camera system approaching a wall, the headlight a point of light 118 projected onto the wall, according to an embodiment of the method according to the invention. Through the point of light located in the picture of the camera 118 is a respective epipolar line 116 of the spot of light 118 associated pixels, wherein the respective epipolar line 116 calculated using the epipolar geometry from the calibration data of the headlight camera system. Approaching the headlight camera system of the wall, as in one by arrow 112 displayed image sequence 102 . 104 . 106 . 108 . 110 , the associated pixel travels up the right along the epipolar line. The headlight camera system removes itself from the wall, as in an arrow 114 displayed image sequence 110 . 108 . 106 . 104 . 102 , the pixel travels down the epipolar line to the left.

In 2 is a schematic representation of an image 200 from a camera with a checkerboard pattern projected into a vehicle apron 202 and detected characteristic features 210 according to an embodiment of the method according to the invention. Become four features within a circular mask by an image analysis software 212 determined, these characteristics can be a pixel 204 and the associated epipolar line 206 be assigned. If the respective features are found along the epipolar line in previous images, a feature recognition agrees with the epipolar lines and a current calibration is valid.

In 3 is a schematic representation of an image 300 from a camera with a checkerboard pattern projected into a vehicle apron 202 with a search for a characteristic feature 302 along a misaligned epipolar line 304 according to an embodiment of the method according to the invention. A misalignment in the camera headlight system turns the pixel into a center of the characteristic feature 302 , a dark surface element enclosed by four detected features, a false epipolar line 304 assigned. A search along the epipolar line 304 after a characteristic feature within a circular mask 306 , indicated by several in the direction of the vector 310 shifted circular masks 308 is inconclusive. Thus, there must be a headlight misalignment deviating from an initial calibration and an automatic headlight calibration is initiated.

In 4 is a schematic representation of an image 400 from a camera with a checkerboard pattern projected into a vehicle apron 202 with a search for a characteristic feature along an epipolar line calculated with new calibration data 404 according to an embodiment of the method according to the invention. The epipolar line 404 leads through the pixel 402 which is a center of a characteristic feature consisting of four detected features 302 assigned. A search for the characteristic feature indicated by several along the epipolar line 404 shifted masks 406 , runs successfully. There are correct calibration data available for active triangulation.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102011109440 A1 [0005]
• DE 102013211876 A1 [0006]
• DE 102012218390 A1 [0008]

Claims (9)

Method for an automated detection of a headlight misalignment, in which a vehicle in the front region has at least one headlight and at least one camera whose calibration data are known, wherein the at least one camera images (102, 104, 106, 108, 110, 200, 300, 400 ) in the form of pixels, wherein an epipolar line (116, 206, 404) is calculated from the calibration data by means of epipolar geometry for each pixel (204, 404) of the image (102, 104, 106, 108, 110, 200, 300, 400) is, wherein the at least one headlight projected at least one characteristic feature in a vehicle apron, wherein the at least one camera at a predetermined time interval, two images (102, 104, 106, 108, 110, 200, 300, 400) of the vehicle apron receives and on forward a memory, wherein image processing software in the first image (102, 104, 106, 108, 200) of the camera searches for at least one characteristic feature (212, 302), a respective one of identifying the pixel (204, 402) associated with the at least one feature (212, 302), the image processing software searching for the same feature (302) in the second image and determining if found on the epipolar line (304, 404) of the respective associated pixel (402) of the first image (102, 104, 106, 108, 200), wherein an error function is calculated in the event of a deviation from the epipolar line (404), and if the calculated value of the error function is exceeded from a predetermined value of the error function, an automatic headlight calibration is triggered, which generates new calibration data. Method according to Claim 1 in which the headlamp and camera are described by applying an epitome geometry by an extrinsic and an intrinsic matrix. Method according to one of the preceding claims, wherein a checkerboard pattern (202) is selected as the pattern. Method according to Claim 3 in which the at least one characteristic feature (212, 302) is selected as a pixel which in pixels in its immediate vicinity in an order from upper left to upper right to lower right to lower left to upper left two transitions from light to dark having. Method according to one of the preceding claims, in which the predetermined time interval between the first and second image of the at least one camera is selected as a time interval of two images produced directly in succession by the at least one camera. Method according to one of the preceding claims, in which the epipolar line to each pixel is calculated from the new calibration data by automatic headlight calibration. Method according to Claim 6 in which the modified calibration data are made available to a distance measurement by active triangulation. A system for automated detection of a headlight misalignment comprising at least one headlamp, at least one camera, a computing unit having a memory unit, and image processing software and configured to perform a method according to any one of the preceding claims. Vehicle following a system Claim 8 and configured to perform a method according to any one of Claims 1 to 7 perform.

Images