DE102021005335B4 - Method for compensating measurement errors - Google Patents

Abstract

Method for compensating for measurement errors (sx, sy) in a distance measurement carried out by means of a sensor system of a vehicle having at least two cameras (1, 2), wherein an image (B1, B2) is captured by means of each of the at least two cameras (1, 2). - a time deviation between recording times (tn, tn+1) of the two captured images (B1, B2) is determined, - an object speed (v) of an object in a vehicle environment is determined, - a position of an object point (P) of the object in the image (B1, B2) of one of the cameras (1, 2) used as a reference image is determined as a reference position, - a position of the object point (P ') in the image (B1, B2) of the other camera (1, 2) is determined - the position of the object point (P') in the image (B1, B2) of the other camera (1, 2) based on the object speed (v) of the object and the time deviation to a corrected position at which the object point ( P') would have been at the time of recording (tn, tn+1) of the reference image, is recalculated and a distance to the object is determined by triangulation based on the reference position of the object point (P) and the corrected position of the object point (P'). becomes.

Description

The invention relates to a method for compensating for measurement errors in a distance measurement carried out by means of a vehicle sensor system having at least two cameras.

From the DE 102 15 009 A1 a method for determining a speed of a vehicle is known, in which speed-dependent driving state variables are recorded and an approximate speed of the vehicle is calculated from these using a transfer function. To correct a transfer function, a relative speed of the vehicle to the landmark is calculated from the distance measured at different points in time from a point on the vehicle to a landmark. The error in determining the speed is then recorded by comparing the speed of the vehicle with the relative speed and the transfer function is adjusted so that the error is minimized.

From the DE 10 2006 005 231 A1 a method for determining the distance of an object by evaluating stereoscopic camera images is known. Parallaxes of images of an image pair that were recorded from different measurement positions are determined by counting pixels of a lateral offset of the images and the distance of the object is calculated based on this. A requirement for a small measurement error can be taken into account by a sufficiently large distance between the measuring positions.

From US 2013/0 194 419 A1 a method for tracking a moving object and for determining its current distance from a vehicle is known, the method being based on stereo image processing. The determination of the distance is based on filtering and refining an image of the object, using a Kalman filter to remove noise errors.

From US 2015/0 062 304 A1 an imaging system for a vehicle is known which includes two image capture devices that work according to the rolling shutter principle. The two image capture devices have different and at least partially overlapping fields of view. The sampling rates of the two image capture devices differ from each other, but are coordinated with one another in such a way that the image capture devices record the overlapping area of the fields of view within the same period of time.

From WO 2015/186 002 A2 a method for detecting an object in front of a vehicle is known, in which a large number of images of an area are captured by means of an image capture device, with displacement vectors between pixels being determined by comparing two images in order to create upright objects based on this to identify.

The invention is based on the object of specifying a novel method for compensating for measurement errors in a distance measurement carried out using a sensor system having at least two cameras.

The object is achieved according to the invention by a method which has the features specified in claim 1.

Advantageous embodiments of the invention are the subject of the subclaims.

In the method according to the invention for compensating for measurement errors in a distance measurement carried out by means of a vehicle sensor system having at least two cameras, an image is captured by each of the at least two cameras. A time deviation between recording times of the two captured images, an object speed of an object in a vehicle environment and a position of an object point of the object in the image of one of the cameras used as a reference image are determined as a reference position. Furthermore, a position of the object point in the image of the other camera is determined, the position of the object point in the image of the other camera based on the object speed of the object and the time deviation to a corrected position at which the object point was located at the time the reference image was recorded would have been recalculated. A distance to the object is determined by triangulation based on the reference position of the object point and the corrected position of the object point.

Multi-camera systems are able to determine the position of at least one object using the measuring principle of triangulation. However, with moving objects, measurement errors in position determination can occur due to temporally asynchronous recordings by the multi-camera system. This results from the fact that the object is recorded at different positions by the cameras of the corresponding multi-camera system due to a movement of the object and/or the multi-camera system and a resulting time offset in the image recordings. Using the present method, the position of the object can be corrected and can therefore be determined with greater accuracy.

Using the method, it is possible to improve the measurement accuracy of existing measurement systems that use so-called software or hardware triggers for dynamic measurements. The software triggers are usually less precise than hardware triggers and often use a so-called bus synchronization, in which a recording pulse is output to the multi-camera systems. When using hardware triggers, a direct conductive connection is used between the individual cameras of the multi-camera systems, with an output of one camera, for example a so-called flash output, usually being connected to an input of another camera, for example a trigger. If the first camera is triggered, all other cameras or camera systems are then triggered by the impulse on the input. The synchronization of the recordings is usually better with hardware triggers than with software triggers, but still not completely synchronous. This still leads to errors in determining the position of objects. Especially with camera systems from different manufacturers or different types in a network, the software trigger does not work or only works to a limited extent. In this case, even with hardware triggers, there is the problem of the necessary assembly of suitable hardware. By means of the present method, however, it is possible to improve the dynamic measurement accuracy of multi-camera systems of different types or manufacturers, which are difficult or impossible to synchronize due to their structure or configuration. This means that the method can be used in a particularly advantageous manner without the need for additional hardware and without being dependent on a type and manufacturer of a camera.

In a possible embodiment of the method, a common time system for the cameras is specified by a component of an image processing system that processes data from the cameras, for example an image evaluation unit. This makes it possible to achieve a particularly precise determination of the time deviation and, as a result, a reliable compensation for measurement errors.

In a further possible embodiment of the method, a transmission time between the component and other components of the image processing system in a system comprising at least the components and the cameras is determined from a latency between the component and the other components, the transmission time being used in determining the time deviation between the recording times is taken into account. Taking the transmission time into account enables a further increase in the accuracy when determining the time deviation.

In a further possible embodiment of the method, the time deviation is determined from the recording time of the reference image minus the recording time of the image of the other camera minus the transmission time. Such a determination is particularly reliable and accurate.

In a further possible embodiment of the method, a middle of an exposure time when recording the respective image is used as the recording time. This enables easy recording of the recording times based on a common basis and different exposure times can be compensated for.

In a further possible embodiment of the method, an average value of the speeds of the corresponding object is determined for several recording times, the average value of the speeds of the object being used as the object speed.

In a further possible embodiment of the method, the object speed is multiplied by the time deviation between the recording times, a correction value being determined from the result of the multiplication and the corrected position being determined based on the correction value. This enables a simple and reliable determination of the corrected position.

Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

• 1 schematisch eine Darstellung einer Anordnung zur Bestimmung einer Position eines Objektpunkts bei einer zeitlich synchronen Aufnahme desselben mittels zwei Kameras,
• 2 schematisch eine Darstellung einer Anordnung zur Bestimmung einer Position eines Objektpunkts bei einer zeitlich asynchronen Aufnahme desselben mittels zwei Kameras,
• 3 schematisch eine Darstellung eines mittels einer Kamera zu unterschiedlichen Aufnahmezeitpunkten erfassten Objektpunkts,
• 4 schematisch eine Messung einer Bewegung und Asynchronität eines Objektpunkts in einem Bild sowie eine Zuordnung zu einem Referenzbild und
• 5 schematisch eine Korrektur einer Position eines Objektpunkts anhand der ermittelten Bewegung und Asynchronität.

Show:

• 1 schematically a representation of an arrangement for determining a position of an object point when recording it synchronously using two cameras,
• 2 schematically a representation of an arrangement for determining a position of an object point when recording it asynchronously in time using two cameras,
• 3 schematically a representation of an object point captured by a camera at different recording times,
• 4 schematically a measurement of a movement and asynchrony of an object point in an image as well as an assignment to a reference image and
• 5 schematically a correction of a position of an object point based on the determined movement and asynchrony.

Corresponding parts are provided with the same reference numbers in all figures.

In 1 is a representation of an arrangement for determining a position of an object point P during a synchronous recording of the same using two cameras 1, 2.

The cameras 1, 2 are, for example, part of a sensor system, for example a multi-camera system, which is provided, for example, in or on a vehicle for detecting a vehicle environment.

Environmental data recorded by the cameras 1, 2 is intended in particular for use in driver assistance systems, for example for automated, in particular highly automated or autonomous operation of the vehicle. When used in this way, objects B1, B2 are detected in images B1, B2 captured by the cameras 1, 2, such an object being represented by the object point P in the present case.

For this purpose, several images B1, B2 of an object point P are recorded from different perspectives using the cameras 1, 2 and associated recording times t _n , t _n+1 . The cameras 1, 2 are calibrated to one another, with a distance d between the cameras 1, 2 and a respective focal length f1, f2 of the cameras 1, 2 being known. If the position of at least one object is calculated when the images B1, B2 are evaluated, the same object point P is determined in each image B1, B2. A position of the object point P is calculated from each image B1, B2 by triangulation. That is, in the captured images B1, B2, a distance of the object point P to the cameras 1, 2 is determined based on an intersection of lines of sight S1, S2 of the cameras 1, 2, the distance d of the cameras 1, 2 to one another and the focal lengths f1, f2 of cameras 1, 2 determined.

However, if the object moves while the images B1, B2 are being recorded and these are not recorded at the same time but asynchronously, then the object point P in an image B1, B2 shifts by a distance s, which depends on a time deviation between the recording times t _n , t _n+1 of the images B1, B2 and the movement of the object depends. This is in 2 shown in more detail, which shows a representation of an arrangement for determining a position of an object point P during a temporally asynchronous recording of the same using two cameras 1, 2.

The image B2 captured by the camera 2 was recorded with a time offset from the image B1 captured by the camera 1. The temporal offset and the movement of the object point P cause a displacement V of the object point P in image B2. A resulting line of sight S2' intersects the line of sight S1 of the camera 1 at point P'.

Without taking into account the asynchrony of the recording times t _n , t _n+1, the position calculation of the object point P is subject to errors. A true position of the object point P in space can deviate more or less from the measured position, depending on how the object moves relative to the cameras 1, 2 and how large the time deviation between the recording times t _n , t _{n +1} of the two recorded Pictures B1, B2 is. The path s by which the object point P' is shifted relative to the object point P results from a measurement error s _y in the longitudinal direction Y and a measurement error s _x in the transverse direction X.

Will be an in 5 If the correction value K shown in more detail is included, the resulting measurement error s _x , s _y can at least be reduced or completely compensated for in the form of the path s. For this correction, knowledge of the time deviation, one in 4 Direction of movement R of the object point P 'shown in more detail and an object speed v of the object point P' are required.

The possible exemplary embodiment _of a method for compensating for measurement errors _s in this case run through it several times.

Each time an image B1, B2 is recorded, the recording time t _n , t _n+1 is determined in a common time system. The common time system is defined by a component of the image processing system that processes data from the cameras 1, 2, for example an image evaluation unit.

In addition, in order to determine the time deviation as precisely as possible, a transmission time is required between the time-defining component and other components of the image processing system. For this purpose, a latency time, which describes a transmission time between the component and other components in a system comprising at least the components and the cameras, is determined and this time difference is included in the calculation of the asynchrony.

Likewise, to compensate for different exposure times, the middle of the respective exposure time is defined as the recording time.

To correct the _measurement errors _s A special case in which a movement of the object point P 'takes place exactly in the direction of the line of sight S1, S2 of a camera 1, 2 is solved by assigning the camera 1, 2 to the reference image.

First, to determine the object speed v of the object point P ', a movement of the object point P 'around the respective recording time t _n , t _{n + 1} of each camera 1, 2 is analyzed and the image B1, B2 with the smallest movement of the object point P is set as a reference image.

aus einer zeitlichen Differenz des Aufnahmezeitpunkts t_Referenz des Referenzbilds zu weiteren Aufnahmezeitpunkten t_n, korrigiert um die durch die jeweilige Latenz t_Latenz,n beschriebene Übertragungszeit zwischen der zeitdefinierenden Komponente und anderen Komponenten. Dabei kann die berechnete Zeitabweichung positiv oder negativ sein, was sich später auf die Korrekturrichtung auswirkt.The time deviation t _Async describing the asynchrony is determined according to $$t_{Async}=t_{Referen\mathrm{e.g}}-t_n-t_{Laten\mathrm{e.g},n}$$

from a time difference between the recording time t _reference of the reference image and further recording times t _n , corrected by the transmission time between the time-defining component and other components described by the respective latency t latency _,n . The calculated time deviation can be positive or negative, which later affects the correction direction.

The example in 4 The object speed v shown and the direction of movement R of the object point P' are calculated by measurements, for example by the cameras 1, 2 to be corrected, around the recording time t _n , t _n + ₁ to be corrected. The measurement of the object speed v and the direction of movement R is carried out around the image B1, B2 to be corrected and by at least one further image recording. This is in 3 based on a representation of an object point P' -1, P ^' , P' ⁺¹ captured by a camera 1, 2 at different recording times t _n-1 , t _n , t _n+1 .

Here, the determination of the object speed v and the direction of movement R of the object point P' is shown at a recording time t _n-1 before and a recording time t _n+1 after a recording time t _n .

aus einer Differenz Δx der Koordinaten des Objektpunkts P in Querrichtung X geteilt durch die zugehörige Zeitabweichung Δt_Async zwischen den Bildern B1, B2.An object speed in the transverse direction X is calculated according to $$v_{xn}=\frac{\mathrm{\Delta }x}{\mathrm{\Delta }{t}_{Async}}=\frac{x\left(P{\text{'}}^{\pm n}\right)-x\left(P\text{'}\right)}{t_n\left(P{\text{'}}^{\pm n}\right)-t_{Referen\mathrm{e.g}}\left(P\text{'}\right)}$$

from a difference Δx of the coordinates of the object point P in the transverse direction X divided by the associated time deviation Δt _Async between the images B1, B2.

aus einer Differenz Δy der Koordinaten des Objektpunkts P in Längsrichtung Y geteilt durch die zugehörige Zeitabweichung At_Async zwischen den Bildern B1, B2.An object speed in the longitudinal direction Y is calculated according to $$v_{yn}=\frac{\mathrm{\Delta }y}{\mathrm{\Delta }{t}_{Async}}=\frac{y\left(P{\text{'}}^{\pm n}\right)-y\left(P\text{'}\right)}{t_n\left(P{\text{'}}^{\pm n}\right)-t_{Referen\mathrm{e.g}}\left(P\text{'}\right)}$$

from a difference Δy of the coordinates of the object point P in the longitudinal direction Y divided by the associated time deviation At _Async between the images B1, B2.

und die Längsrichtung Y gemäß $$v_y=\frac{v_{y1}+v_{y2}+\cdots +v_{yn}}{n}$$

jeweils ein Mittelwert der bestimmten Objektgeschwindigkeiten berechnet. Dies wird anhand einer Bildaufnahme oder mit beiden Bildaufnahmen durchgeführt, welche zeitlich am nächsten mit dem kleinsten Zeitintervall vor und nach der Messung aufgenommen wurden.For several measurements, X is then used for the transverse direction $$v_x=\frac{v_{x1}+v_{x2}+\cdots +v_{xn}}{n}$$

and the longitudinal direction Y according to $$v_y=\frac{{\mathrm{<figure-callout\; id="1"\; label="v\; y"\; filenames="DE102021005335B4\_0010.png,DE102021005335B4\_0011.png"\; state="\{\{state\}\}">v</figure-callout>}}_y+{\mathrm{<figure-callout\; id="2"\; label="v\; y"\; filenames="DE102021005335B4\_0010.png,DE102021005335B4\_0011.png"\; state="\{\{state\}\}">v</figure-callout>}}_y+\cdots +v_{yn}}{n}$$

an average of the specific object speeds is calculated. This is carried out using one image recording or both image recordings, which were taken closest in time with the smallest time interval before and after the measurement.

und in Längsrichtung Y gemäß $$y_{Korrektur}=t_{Async}\ast v_y$$

berechnet. _A respective _{correction value} in the transverse direction $$x_{KOrrektur}=t_{Async}\ast v_x$$

and in the longitudinal direction Y according to $$y_{KOrrektur}=t_{Async}\ast v_y$$

calculated.

und $$y_{korrigiert}=y_{unkorrigiert}+y_{Korrektur}.$$

The correction values determined in this way are then added to the respective uncorrected coordinate of the object point P' $$x_{kOrriGiert}=x_{unkOrriGiert}+x_{KOrrektur}$$

and $$y_{kOrriGiert}=y_{unkOrriGiert}+y_{KOrrektur}.$$

This means that the offset of the object point P' is calculated back to the reference image by the positive time deviation Δt _Async of the corresponding image B1, B2 and the measurement error s _x , s _y is corrected by the newly calculated object point P.

Such a correction using a correction value K formed from the correction values x _correction , y _correction is exemplified in 5 shown.

Finally, the coordinates of the object point P, P' are determined using the triangulation method.

Claims (7)

Method _for compensating for measurement errors ( _s ) is recorded, - a time deviation between recording times (t _n , t _n+1 ) of the two captured images (B1, B2) is determined, - an object speed (v) of an object in a vehicle environment is determined, - a position of an object point ( P) of the object in the image (B1, B2) of one of the cameras (1, 2) used as a reference image is determined as a reference position, - a position of the object point (P ') in the image (B1, B2) of the other camera (1 , 2) is determined, - the position of the object point (P ') in the image (B1, B2) of the other camera (1, 2) based on the object speed (v) of the object and the time deviation to a corrected position at which the object point (P') would have been at the time of recording (t _n , t _n+1 ) of the reference image, is recalculated and - a distance to the object by triangulation, based on the reference position of the object point (P) and the corrected position of the Object point (P') is determined. Procedure according to Claim 1 , characterized in that a common time system for the cameras (1, 2) is specified by a component of an image processing system that processes data from the cameras (1, 2). Procedure according to Claim 2 , characterized in that - a transmission time between the component and other components in an image processing system comprising at least the components and the cameras (1, 2) is determined from a latency between the component and the other components of the image processing system and - the transmission time during the determination the time deviation between the recording times (t _n , t _n+1 ) is taken into account. Procedure according to Claim 3 , characterized in that the time deviation from the recording time (t _n , t _n+1 ) of the reference image minus the recording time (t _n , t _n+1 ) of the image (B1, B2) of the other camera (1, 2) minus the Transmission time is determined. Method according to one of the preceding claims, characterized in that a middle of an exposure time when recording the respective image (B1, B2) is used as the recording time (t _n , t _n+1 ). Method according to one of the preceding claims, characterized in that - an average value of velocities of the corresponding object is determined for several recording times (t _n , t _n+1 ) and - the average value of the velocities of the object is used as the object speed (v). Method according to one of the preceding claims, characterized in that - the object speed (v) is multiplied by the time deviation between the recording times (t _n , t _n+1 ), - a correction value (K) is determined from the result of the multiplication and - The corrected position is determined based on the correction value (K).

Images