DE102020207248A1 - Method and device for determining a time offset between a reference time of a first signal and a reference time of a second signal - Google Patents

Abstract

The invention relates to a method (500) for determining a time offset (δ) between a reference time (t) of a first signal (125, 135) and a reference time (t) of a second signal (155). The method (500) comprises a step of reading in (510) the first signal (125, 135) and the second signal (155), a sequence of images (125, 135) being read in as the first signal (125, 135) , which were recorded at different recording times (t) by an optical sensor (105) and a sequence of a one-dimensional measured variable (155) is read in as the second signal (155), which is recorded by a measuring sensor (150) at different recording times (t) were, wherein the optical sensor (105) and the measuring sensor (155) are arranged on a common mobile carrier platform (100). The method (500) further comprises a step of calculating (520) first scaling parameters (σ1), the scaling information between several images (125, t2, t3, t4) acquired by the optical sensor (105) at different acquisition times. 135) and are each assigned to a reference point in time (t) of the first signal (125, 135), and calculation of second scaling parameters (σ2) from the sequence of the one-dimensional measured variable (155) whose reference points (t) compared to the reference points in time (t) of the first signal (125, 135) are shifted according to a time parameter (Δ). The method also includes a step of comparing (530) the calculated first scaling parameters (σ1) with second scaling parameters (σ2) from the second signal (155). Finally, the method (500) comprises a step of determining (540) that time parameter (Δ) as the time offset (δ) between the first (125, 135) and second signal (155) for which in the step (530) of comparing the highest agreement of the compared scaling parameters (σ1, σ2) was recognized.

Description

State of the art

The invention is based on a method or a device according to the preamble of the independent claims. The present invention also relates to a computer program.

Modular solutions are already standard in the automotive industry. Sensors and control devices from various suppliers are used here. A preprocessing control device thus processes part of the sensor measurements directly and delivers the preprocessed information to a further processing control device via the public data bus. This merges the data with other sensor information, which, for example, enables precise determination of the vehicle's own movement. Since there is generally no cross-control unit time base, the processing control unit initially only knows the time it was read in for the sensor signal. However, this often does not correspond to the time of the physical event recorded by the measuring sensor, but to a later point in time. A temporal synchronization of different sensor signals is usually limited to providing incoming signals in the merging control unit with a time stamp. Possible delays between the arrival of the signal and the actual physical event are neglected. Due to the unknown delay of the signal, a correct fusion with sensor data of a different time base is not possible or only with difficulty.

For this reason, the incorrect time base should be corrected for efficient signal processing, this being made possible as a rule by determining the time offset between the two signals.

Disclosure of the invention

Against this background, with the approach presented here, a method, furthermore a device that uses this method, and finally a corresponding computer program according to the main claims are presented. The measures listed in the dependent claims make advantageous developments and improvements of the device specified in the independent claim possible.

• - Einlesen des ersten Signals und des zweiten Signals, wobei als das erste Signal eine Sequenz von Bildern eingelesen wird, die zu unterschiedlichen Erfassungszeitpunkten von einem optischen Sensor erfasst wurden und als das zweite Signal eine Sequenz einer eindimensionalen Messgröße eingelesen wird, die zu unterschiedlichen Erfassungszeitpunkten von einem Messsensor erfasst wurden, wobei der optische Sensor und der Messsensor auf einer gemeinsamen mobilen Trägerplattform angeordnet sind;
• - Berechnen von ersten Skalierungsparametern, die Skalierungsinformationen zwischen mehreren von dem optischen Sensor zu je unterschiedlichen Erfassungszeitpunkten erfassten Bildern repräsentieren und je einem zeitlichen Bezugszeitpunkt des ersten Signals zugeordnet sind, und Berechnen von zweiten Skalierungsparametern aus der Sequenz der eindimensionalen Messgröße deren Bezugspunkte gegenüber den Bezugszeitpunkten des ersten Signals entsprechend eines Zeitparameters verschoben sind;
• - Vergleichen der berechneten ersten Skalierungsparameter aus dem ersten Signal mit den zweiten Skalierungsparametern aus dem zweiten Signal; und
• - Ermitteln desjenigen Zeitparameters als den Zeitversatz zwischen dem ersten und zweiten Signal, für den im Schritt des Vergleichens die höchste Übereinstimmung der verglichenen Skalierungsparameter erkannt wurde.

A method is proposed for determining a time offset between a reference time of a first signal and a reference time of a second signal, the method having the following steps:

• - Reading in the first signal and the second signal, with a sequence of images being read in as the first signal, which were acquired at different acquisition times by an optical sensor, and a sequence of a one-dimensional measured variable is read in as the second signal, which is read in at different acquisition times of a measuring sensor were detected, wherein the optical sensor and the measuring sensor are arranged on a common mobile carrier platform;
• Calculation of first scaling parameters, which represent scaling information between a plurality of images recorded by the optical sensor at different acquisition times and are each assigned to a time reference time of the first signal, and calculation of second scaling parameters from the sequence of the one-dimensional measured variable whose reference points compared to the reference times of the first Signals are shifted according to a time parameter;
• - comparing the calculated first scaling parameters from the first signal with the second scaling parameters from the second signal; and
• Determination of that time parameter as the time offset between the first and second signal for which the highest correspondence of the compared scaling parameters was recognized in the step of comparing.

A time offset can be understood to mean, for example, a time span or time difference by which a reference point in time of the first signal is shifted with respect to a reference point in time of the second signal. This time offset can be caused, for example, by different transit times in lines of different lengths or by different intensities of preprocessing of the corresponding signals and the associated different preprocessing times. An optical sensor can be understood to mean, for example, a camera, including a mono camera. A one-dimensional measured variable can be understood to mean, for example, a variable that can be recorded as an absolute value. For example, such a one-dimensional measured variable can be a speed or a distance in the form of a spatial depth. A mobile carrier platform can be understood to mean, for example, a vehicle on or in which the optical sensor and the measuring sensor are installed. In particular, the scaling parameters of the optical sensor and the measuring sensor should be able to be converted into a common reference at least temporarily. A scaling parameter can be understood as a parameter that represents information about how the Change scales or relationships between elements from the images on the one hand or between the measured variables on the other hand.

The approach presented here is based on the knowledge that a time offset between the first signal and the second signal can also be recognized by the fact that characteristic features from the two signals can be compared with regard to their temporal course or temporal occurrence in the two signals, so that even without the knowledge of the actual acquisition time of the relevant values for the respective signals, a synchronization of these signals can now be carried out. For this purpose, however, values must first be provided or delivered with the scaling parameters which can be compared with one another if the sensors delivering the first and second signals detect or provide different physical parameters that are actually not comparable with one another. For this reason, with the approach proposed here, the first scaling parameters can be calculated from the sequence of images of the first signal, for example using approaches for evaluating an optical flow, whereas the second scaling parameters can be determined from the sequence of measured variables of the second signal. The scaling parameters each depict information on how the scales or relationships between elements from the images on the one hand or between the measured variables on the other hand change, so that dimensionless, comparable information is now available for each of the two signals. By comparing these scaling parameters it can then be determined, for example, at which points in time or at which reference point in time of a first signal or second signal the scaling parameters have the same value or a similar curve, so that it can be recognized at which time segments in the different signals the the same features are shown in terms of content. This allows the time offset between the individual signals to be easily identified so that, for example, in a subsequent step, a time shift or delay of one of the signals by the time offset can be carried out in order to synchronize or calibrate the first signal with the second signal vice versa.

The approach proposed here offers the advantage of making it possible to draw conclusions about the time offset solely from the evaluation of the content of signals that are already available, which can then be used, for example, for a very efficient calibration or synchronization of the two signals. No great additional technical effort is therefore required in order to synchronize or calibrate different signals. An important aspect of the approach presented here can be seen in the fact that the non-metric signal of the (mono) camera can be correlated with the metric signal over time or registered with one another via the detour of the dimensionless size of the scaling parameter.

An embodiment of the approach proposed here is particularly favorable, in which, in the step of comparing, a time profile of first scaling parameters within a time window is compared with a time profile of the second scaling parameters within a time window shifted by the time parameter. Furthermore, in the step of determining the time parameter as the time offset between the first and second signal, that parameter is determined in which the temporal course of the second scaling parameter shifted by the time parameter has the greatest similarity to the temporal course of the first scaling parameter. Such an embodiment of the approach proposed here offers the advantage of not only using a single time parameter at which the first scaling parameter is within a tolerance range around the second scaling parameter, but of considering the course of the respective scaling parameters within a time window. In this way, it can be avoided that if the value of the first scaling parameter happens to be the same with the second scaling parameter shifted by the time parameter, but completely different further temporal progression of the respective scaling parameters, the time parameter is interpreted as the actual time prefix. In this way, the time offset can advantageously be determined very robustly and reliably.

An embodiment of the approach proposed here is also conceivable in which the first and second signals are read in in the reading-in step, in which the sequence of images captured by the optical sensor and / or the sequence of the measured variable captured by the measuring sensor are each identical the time interval between acquisition times was recorded. As a result, the first and second signals are provided by cyclical scanning of the environment around the mobile carrier platform. Such an embodiment offers the advantage that the respective sensors, which provide the information on which the first and second signals are based, scan the respective relevant physical variable at the same time interval. In this way, a time offset that has been determined once has a significantly longer validity than if the sampling rates of the individual sensors drift apart in time.

One embodiment of the approach proposed here is technically very easy to implement, in which, in the step of reading in the second signal, a metric and / or physical variable that can be absolutely detected by the measuring sensor is read in as the measured variable. Such an embodiment offers the advantage that a corresponding measuring sensor is often already available in a corresponding sensor environment and usually already delivers a very precise measurement result for the measured variable, for example compared to a scaling parameter to be derived from different images. In this way, a very reliable detection of the time offset can be implemented, in which the more precise measured value is used as the basis for shifting the scaling parameter.

An embodiment of the approach proposed here is also advantageous, in which, in the reading-in step, a sequence of the measured variable over time is read in for a speed of the mobile carrier platform. Such an embodiment offers the advantage that, in particular, a speed value can already be recorded very precisely by different sensors, so that the determination of the time offset can also be designed very precisely.

An embodiment of the approach presented here is particularly favorable in which, in the calculation step, the second scaling parameters are each calculated using an integral value formation over at least a time segment of the speed used as the measured variable. In this way, the second scaling parameter can be determined very easily, but nevertheless precisely.

Alternatively or additionally, according to a further embodiment, in the step of reading in, a spatial depth to a recognized object that is detected by the measuring sensor of the mobile carrier platform can be read in as a measured variable. Such an embodiment offers the advantage of a simplified calculation of the scaling parameters compared to z. B. a speed measuring sensor.

According to a further embodiment of the approach presented here, the first scaling parameters can be calculated in the calculation step using a determination of relative positions (of scene points recognized in the images and assigned to one another) in the images. Such an embodiment offers the advantage of approaches for determining the optical flow, which are often already known from image processing, of also obtaining a very precise result for the first scaling parameter.

According to a further embodiment of the approach presented here, in a shifting step, a time base of the first and / or second signal can be shifted using the determined time offset, in particular wherein the first and / or second signal is delayed by the determined time offset. Such an embodiment offers the advantage of having signals available for further subsequent processing, for example for driver assistance systems or emergency driving functions, which have a common time base with regard to a recognized event. In this way, a significant improvement in the precision of the information obtained from the corrected signals or the signals whose time base has been shifted can be opened up.

This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control device.

The approach presented here also creates a device which is designed to carry out, control or implement the steps of a variant of a method presented here in corresponding devices. The object on which the invention is based can also be achieved quickly and efficiently by means of this embodiment variant of the invention in the form of a device.

For this purpose, the device can have at least one processing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or an actuator for reading in sensor signals from the sensor or for outputting data or control signals to the Have an actuator and / or at least one communication interface for reading in or outputting data, which are embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller or the like, wherein the storage unit can be a flash memory, an EEPROM or a magnetic storage unit. The communication interface can be designed to read in or output data wirelessly and / or wired, a communication interface that can read in or output wired data, for example, can read this data electrically or optically from a corresponding data transmission line or output it into a corresponding data transmission line.

In the present case, a device can be understood to mean an electrical device that processes sensor signals and outputs control and / or data signals as a function thereof. The device can have an interface that is hard and / or can be designed in software. In the case of a hardware design, the interfaces can, for example, be part of a so-called system ASIC which contains a wide variety of functions of the device. However, it is also possible that the interfaces are separate, integrated circuits or at least partially consist of discrete components. In the case of a software-based design, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules.

A computer program product or computer program with program code, which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk or an optical memory, and for performing, implementing and / or controlling the steps of the method according to one of the embodiments described above is also advantageous is used, especially when the program product or program is executed on a computer or device.

• 1 ein Szenario, in dem der hier vorgestellte Ansatz gemäß einem Ausführungsbeispiel im Zusammenhang mit einer mobilen Trägerplattform und einem Blockschaltbild einer Vorrichtung gemäß einem Ausführungsbeispiel näher erläutert wird;
• 2 eine Darstellung zur Erläuterung der prinzipiellen Funktionsweise des hier vorgestellten Ansatzes gemäß einem Ausführungsbeispiel;
• 3 eine Blockschaltbilddarstellung eines Sensorsystems zur Verarbeitung von Sensorsignalen zur Verdeutlichung der dem hier vorgestellten Ansatz zugrundeliegenden Problematik;
• 4 eine Diagramm-Darstellung einer exemplarischen Vorgehensweise gemäß dem vorgestellten Ansatz zum Bestimmen eines Zeitversatzes zwischen einem Bezugszeitpunkt eines ersten Signals und einem Bezugszeitpunkt eines zweiten Signals; und
• 5 ein Ablaufdiagramm eines Ausführungsbeispiels eines Verfahrens.

Embodiments of the approach presented here are shown in the drawings and explained in more detail in the description below. It shows:

• 1 a scenario in which the approach presented here is explained in more detail according to an exemplary embodiment in connection with a mobile carrier platform and a block diagram of a device according to an exemplary embodiment;
• 2 a representation to explain the basic functionality of the approach presented here according to an embodiment;
• 3 a block diagram representation of a sensor system for processing sensor signals to illustrate the problem on which the approach presented here is based;
• 4th a diagram representation of an exemplary procedure according to the presented approach for determining a time offset between a reference time of a first signal and a reference time of a second signal; and
• 5 a flowchart of an embodiment of a method.

In the following description of advantageous exemplary embodiments of the present invention, identical or similar reference symbols are used for the elements shown in the various figures and having a similar effect, a repeated description of these elements being dispensed with.

1 shows a scenario in which the approach presented here according to an exemplary embodiment in connection with a mobile carrier platform 100 is explained in more detail, which is implemented here as a vehicle. The mobile carrier platform 100 comprises one embodiment of an apparatus 102 for determining a time offset between a reference time of a first signal and a reference time of a second signal. The mobile carrier platform 100 has an optical image sensor 105 on, which in the present case is designed as a camera, more precisely as a mono camera, and the images or images of immobile or static objects 110 in the neighborhood 111 the carrier platform 100 (e.g. trees 112 or buildings 114 ) or moving or non-static objects 115 (such as other vehicles) and these images of a facility 120 for calculating a first scaling parameter σ ₁ . More specifically, it is from the image pickup sensor 105 a first pair of images as the first signal 125 to a read-in interface 130 the facility 120 transmitted, whereby images of the first image pair 125 each at a beginning and an end of a first time interval by the image recording sensor 105 were recorded. At the same time is from the image recording sensor 105 also a second pair of images in the first signal 135 to the read-in interface 130 the facility 120 transmitted, with images of the second pair of images 135 each at a beginning and an end of a second time interval by the image recording sensor 105 were recorded. In the device 120 are then in a recognition unit 140 at least one scene point, its image points in the images or image pairs 125 , 135 the two time intervals are identified. Here, two corresponding image points can be taken from the images of the first image pair recorded at the beginning and at the end of a first time interval 125 and of at least two corresponding image points each from the images of the second image pair recorded at the beginning and at the end of a second time interval 135 be recognized. Finally, in a determination unit 145 a first relation parameter using a relation between positions of the image points from the first image pair 125 and a second relationship parameter using a relationship between positions of the image points from the second image pair 135 and the first and second relation parameters are linked in order to determine _{the first scaling parameter (s) σ 1.} This process of ascertaining first scaling parameters σ ₁ is carried out cyclically for a sequence of several temporally successive images, so that a sequence of first scaling parameters σ _{1 is also} obtained.

An imaging optical sensor like the optical image pickup sensor 105 provides a two-dimensional image or a projection of the scene. I. E. it usually does not provide any metric information, such as the depth of the individual scene points. Thus, a scaled movement always creates the same image in a correspondingly scaled world. The scale is usually not measurable.

It is also on the mobile carrier platform 100 another measuring sensor 150 arranged, the second signal is a sequence of a one-dimensional measured variable 155 provides, which also by means of the interface 130 is read. For example, the measuring sensor 150 a speed of the mobile carrier platform 100 or a distance / depth of the objects such as the vehicle 115 , the tree 1124 or the building 114 capture and as the measurand 155 provide. The measurand 155 or the sequence of the measured variables 155 then becomes a unit 160 to calculate at least one second scaling parameter σ ₂ supplied, in which a scale or relative scale of the measured variable 155 is calculated as the second scaling parameter σ ₂ , as will be described in more detail below. The first scaling parameters σ ₁ and the second scaling parameters σ ₂ then become one unit 165 for comparison, in which a first scaling parameter σ _{1 of} a predetermined reference time t is compared with (several) second scaling parameters σ ₂ , the reference times of which of the recorded measured variable each by a different predetermined time parameter Δ compared to the reference time t _{on which the first scaling parameter σ 1 is based} was moved. Using the comparison result obtained from the comparison 170 will then be in one unit 175 to determine those time parameters as the time offset δ between the first 125 , 135 and second signal 155 determined for the one in the unit 165 for comparison, it was recognized that the first scaling parameter σ ₁ lies within a tolerance range by the second scaling parameter σ ₂ shifted by the time parameter Δ. Another unit is optional 180 the determined time offset δ is used to correct the second signal or the measured variable 155 , and / or the first signal in the form of the first pair of images 125 and / or the second pair of images 135 or generally from the camera 105 to delay the data supplied to a delayed measured variable 155 ' , a delayed first pair of images 120 ' and / or a delayed second pair of images 135 ' which are then provided in a time-synchronized manner with one another, so that this data is now synchronized as time for the subsequent processing, for example in a 1 Not shown driver assistance system or an emergency driving system of a vehicle as a mobile carrier platform 100 can be used. Essentially, this procedure means that the time stamps of one signal are corrected and then both signals have the same time base.

As a measured variable 155 In the present case, metric information can generally be understood, which in the following can also be referred to as sensor information, for example, and which enables conclusions to be drawn about the scaling of an observed scene or a movement in this scene with respect to a unit of measurement. As examples of such measuring sensors 150 As metric sensors, a depth-measuring laser scanner (e.g. Velodyne), a depth-measuring stereo camera, a speed-measuring inertial sensor, a sensor for detecting the number of wheel revolutions of a vehicle (ESP sensor), which are converted into a Speed information are transferred.

The metric information can be disturbed by an unknown scale error that changes slowly over time, this scale error being able to be recognized and compensated for by the determined second scale parameter. In the example of the ESP sensor system, this could be a deviation of the actual tire size from the assumed tire size.

2 shows a representation to explain the basic functionality of the approach presented here according to an embodiment. The representation from the 2 is only shown schematically in order to be able to explain in more detail the mode of operation of the approach presented here. As above with reference to 1 represented or described is by the image pickup sensor 105 the first pair of images 130 output which is a first picture 200 and a second picture 210 , includes. The first picture 200 was recorded at a first point in time t1, so that in this picture 200 the objects such as the moving objects 115 or the static objects 112 or here 114 are shown. The representation of the objects 114 , here as a building and 115 , here as a vehicle, are included in the 2 shown in solid lines to make it clear that these are objects from the first image 200 acts. Analog are in the second picture 210 , which was recorded at the second point in time t2, the objects 114 or. 115 Shown in dashed lines to again distinguish between the objects from the first picture 200 to enable. After the on the mobile carrier platform 100 located image acquisition sensor 105 The objects in question are thus moved 114 or. 115 in different pictures 200 or. 210 arranged at different times t1 and t2 at different locations or positions. It should also be noted here that the first point in time t1 by a predefined first time interval is before the second point in time t2. At the same time there can also be a proper movement of the moving object 115 , here of the vehicle, so that there is no longer any change in the position of objects in the images of the first image pair 130 by the movement of the image pickup sensor 105 is caused.

In addition, as above with reference to 1 also shown or described by the image pickup sensor 105 the second pair of images 135 output which a third image 220 and a fourth picture 230 , includes. The third picture 220 was recorded at a third point in time t3, so that also in this third image 220 the objects such as the moving objects 115 or the static objects 112 or here 114 are shown. The representation of the objects 114 , here as a building and 115 , here as a vehicle, are included in the 2 shown in solid lines to make it clear that these are objects of the third image 220 acts. Analog are in the fourth picture 230 , which was recorded at the fourth point in time t4, the objects 114 or. 115 shown in dashed lines to again distinguish between the objects from the third picture 220 to enable. After the on the mobile carrier platform 100 located image acquisition sensor 105 The objects in question are thus also moved 114 or. 115 in different pictures 220 or. 230 arranged at different locations or positions at the different times t3 and t4. It should also be noted here that the third point in time t3 is a predefined second time interval before the fourth point in time t4. It is also conceivable that the third point in time and the second point in time coincide, so that the fourth point in time can then also be referred to as the third point in time. At the same time there can also be a proper movement of the moving object 115 , here the vehicle, take place, so that there is no longer any change in the position of objects in the images of the second image pair 135 by the movement of the image pickup sensor 105 is caused.

The first picture 200 and the second picture 210 as the first pair of images 130 are subsequently sent to the recognition unit 140 transmitted, in which at least two corresponding image points A and A 'from the images recorded at the beginning (t1) and at the end (t2) of the first time interval 200 or. 210 be recognized. In the present case, this concerns the roof top of the property 114 as a building in the first picture 200 as pixel A, with the corresponding pixel A 'in the second image 210 also as the rooftop of the building 114 is recognized. In addition, another pixel B can also be located on the rear wheel of the vehicle 115 in the first picture 200 can be recognized, whereas the corresponding further image point B 'in the second image 210 also on the rear wheel of the vehicle 115 is recognized.

The third picture can also be analogous 220 and the fourth picture 230 as a second pair of images 135 subsequently to the recognition unit 140 are transmitted, in which at least two corresponding image points A and A 'from the images recorded at the beginning (t3) and at the end (t4) of the second time interval 220 or. 230 be recognized. In the present case, this also applies to the roof top of the property 114 as a building in the third picture 220 as pixel A, with the corresponding pixel A 'in the fourth image 230 also as the rooftop of the building 114 is recognized. In addition, another pixel B can also be located on the rear wheel of the vehicle 115 in the first picture 200 can be recognized, whereas the corresponding further image point B 'in the fourth image 210 also on the rear wheel of the vehicle 115 is recognized.

The information obtained in this way is then sent to the determination unit 145 transmitted, in which a first relation parameter s (t1, t2) using a ratio between positions of the image points A, A ', B and B' from the images from the first image pair 130 and a second relation parameter s (t3, t4) using a ratio between positions of the image points A, A ', B and B' from the images from the second image pair 135 is determined. This first relation parameter can for example be a relation, length or a distance which lies between the corresponding image points A and A 'and / or the image points B and B'. About the different characteristics of the first relation parameter and the second relation parameter at the different positions or locations in the images of the first image pair 130 and the second pair of images 135 To be able to take into account, the corresponding relation parameters can also have several dimensions or contain further information. This can also be used, for example, to evaluate an intrinsic movement of the object in the first time interval between the first time t1 and the second time t2 and / or in the second time interval between the third time t3 and the fourth time t4. As will be shown in more detail below, then in a linking unit 150 the first relation parameter is linked to the second relation parameter in order to determine the scaling parameter σ.

In the context used here, the term “scale” or scaling parameter relates to the scaling information inherent in a scene or a movement. This can be a distance, extension or distance traveled. The relative scale denotes a change in the scale with respect to two time intervals.

The approach presented here uses an imaging optical sensor that delivers a two-dimensional image or projection of the scene. This means that it does not provide any metric information, such as the depth of the individual scene points. Thus, a scaled movement always creates the same image in a correspondingly scaled world. The scale itself is usually not measurable.

The approach presented here relates to moving sensor platforms that are equipped with an optical sensor. An important aspect of the approach presented here can be seen in the fact that a method for estimating the relative scale with respect to two time intervals is presented here, based on the images of an optical sensor. The method is based purely on the movement of projected static scene points and does not require any further information.

The method proposed here or the approach presented here is distinguished by its low complexity and, at the same time, universality. Contrary to the approaches known up to now, the relative scale is determined for each scene point, which additionally provides relative movement information for non-static scene points.

3 shows a block diagram representation of a sensor system for processing sensor signals to clarify the problem on which the approach presented here is based. How sensor components can be recognized, here for example the optical image recording sensor 105 and the measuring sensor 150 represent. An event 300 in the vicinity of the mobile carrier platform 100 , which can consist, for example, in the appearance of a third party vehicle, is detected both by the optical image recording sensor 105 as well as the measuring sensor 150 , which is designed here as a depth-measuring radar sensor, for example. This means that at the same point in time t this occurrence of the event 300 both by the optical image pickup sensor 105 as well as the depth-measuring radar sensor 150 recognized and as a feature in the corresponding output signals 125 , 135 or. 155 is mapped via a data bus 310 to a processing unit such as the device 105 is transmitted. However, this can be done by preprocessing the signals of varying intensity 155 or. 125 or 135 a time offset δ arises, which the measured variable 155 compared to the signals or images 125 or. 135 suffers. In order to achieve the most precise possible recognition of information from these signals 155 or. 125 or 135 to enable is now in the device 105 the approach presented here is carried out in order, for example, to compensate for this time offset δ and thereby enable an improvement in the signal evaluation.

wobei K die intrinsische Kalibrierungsmatrix des optischen Sensors darstellt. Ein entsprechendes Punktpaar im normalisierten Bild die Koordinaten zu zwei aufeinanderfolgenden Zeitpunkten t1 und t2 angegeben werden als ^C(t1)p̃ und ^C(t2)p̃.The determination of the time offset will now be discussed in greater detail below, with a mathematical nomenclature for the description being explained first. Matrices and vectors are given here in bold (capital and small letters). A frame of reference is added as a superscript to the right and the relative position or orientation as a superscript left and subscript right. A rotation matrix ^A R _B ∈ SO (3) describes a rotation sequence R _z ( ^A ψ _B ) · R _y ( ^A θ _B ) · R _x ( ^A ϕ _B ) of the transmission from frame A to frame B. The vector ^A t _B ∈ ℝ ³ , encodes the vector from A to B with reference to the coordinate frame A. See also equation (2) to clarify how the notation relates transformation entities and points to be transformed in ℝ ³ . Measured point coordinates ^l p = [ ^l u, ^l v] ^T ∈ ℝ ² are displayed in normalized image coordinates as $${\text{c}}_{\tilde{\text{p}}}={\text{K}}^{-1}\cdot \left[\begin{array}{c}{}^{\text{I.}}\text{p}\\ 1\end{array}\right]=\left[\begin{array}{c}\left({}^{\text{I.}}u{-}^{\text{I.}}{u}_0\right)\frac{1}{{ƒ}_u}\\ \left({}^{\text{I.}}v{-}^{\text{I.}}{v}_0\right)\frac{1}{{ƒ}_v}\\ 1\end{array}\right]\text{=}\left[\begin{array}{c}\begin{array}{l}{}^{\text{c}}\tilde{x}\\ {\text{c}}_{\tilde{y}}\end{array}\\ 1\end{array}\right],$$

where K represents the intrinsic calibration matrix of the optical sensor. A corresponding pair of points in the normalized image the coordinates at two successive times t1 and t2 are given as ^{C (t1)} p̃ and ^{C (t2)} p̃.

The approach presented here relates to moving sensor platforms 100 that are equipped with multiple, non-synchronized sensors. I. E. the sensor information is not given with respect to a common time base, which is why calibration with respect to a common time base is necessary for correct merging of the data. The core of the invention is a method for the relative calibration of an optical sensor to a metric sensor over time. The metric information can be disturbed by an unknown scale error that changes slowly over time. A correctly scaled measurement of the scene is therefore not necessary for the method.

The method described here enables the determination of the time offset between metric and optical and sensor measurements, and thus a synchronization of any derived information.

A mathematical derivation of the process to be protected is given below.

First, the estimation of the relative scale over time is discussed in more detail.

A monocular optical sensor such as the optical image pickup sensor delivers here 105 a two-dimensional image, or a projection, of the scene. I. E. it does not provide any metric information, such as the depth of the individual scene points. Even over time, depth information can only be determined up to a common, unknown scale factor. The relative scale, ie the change in the scale factor between successive image pairs, can nevertheless be determined (for example by analyzing the optical flow).

$${}^{\text{c}\left(t3\right)}\text{p}{=}^{\text{c}\left(t3\right)}{\text{R}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\text{p}{+}^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\text{. }{s}_{\left(t\mathrm{2,}t3\right)}\text{,}$$

wobei die Translation zwischen zwei Aufnahmezeitpunkten ^C(t1)t_C(t2) ∈ ℝ³ zerlegt ist als normierter Richtungsvektor ^C(t1)t̅_C(t2) und seine Länge bzw. Skalierung s_(t1,t2). Werden die Korrespondenzen in normalisierten Bildkoordinaten ^C(t)p̃ und Tiefe entlang der Z-Achse ^C(t)z dargestellt, ergibt sich $${}^{\text{c}\left(t1\right)}\text{z}{\cdot }^{\text{c}\left(t1\right)}\tilde{\text{p}}{=}^{\text{c}\left(t1\right)}{\text{R}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}{+}^{\text{c}\left(t1\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{1,}t2\right)}$$

$${}^{\text{c}\left(t3\right)}\text{z}{\cdot }^{\text{c}\left(t1\right)}\tilde{\text{p}}{=}^{\text{c}\left(t3\right)}{\text{R}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}{+}^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{2,}t3\right)}\text{.}$$

Below is an estimate of the relative scale based on optical flow. The movement of the projected scene points over time can be determined from the image pairs of a monocular sensor. This is a standard problem in image processing and is known as optical flow. Based on the correspondence of the optical flow, the rotation of the sensor and the direction of translation can then be determined. This is also a standard problem and is often referred to as visual odometry. Due to the lack of depth information, the length or scaling of the translation vector cannot be determined, only its direction. The relative scale, ie the change in the unknown scaling of the translation vector, can now be determined from the point correspondences of three successive sensor images and the proper movement of the image pairs. For pairwise correspondences ( ^{C (t1)} p, ^{C (t2)} p) and ( ^{C (t2)} p, ^{C (t3)} p) ∈ ℝ ^{3 of} a static three-dimensional scene point, the following applies for the transformation with respect to the sensor coordinate system C at the respective imaging times ( t1; t2) or (t3; t2) $${}^{\text{c}\left(t1\right)}\text{p}{=}^{\text{c}\left(t1\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\text{p}{+}^{\text{c}\left(t1\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\text{.}{s}_{\left(t\mathrm{1,}t2\right)}$$

$${}^{\text{c}\left(t3\right)}\text{p}{=}^{\text{c}\left(t3\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\text{p}{+}^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\text{.}{s}_{\left(t\mathrm{2,}t3\right)}\text{,}$$

where the translation between two recording ^{times C (t1)} t _{C (t2)} ∈ ℝ ^{3 is} broken down as a normalized direction vector ^{C (t1)} t̅ _{C (t2)} and its length or scaling s _{(t1, t2)} . If the correspondences are shown in normalized image coordinates ^{C (t)} p̃ and depth along the Z-axis ^{C (t)} z, this results $${}^{\text{c}\left(t1\right)}\text{z}{\cdot }^{\text{c}\left(t1\right)}\tilde{\text{p}}{=}^{\text{c}\left(t1\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}{+}^{\text{c}\left(t1\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{1,}t2\right)}$$

$${}^{\text{c}\left(t3\right)}\text{z}{\cdot }^{\text{c}\left(t1\right)}\tilde{\text{p}}{=}^{\text{c}\left(t3\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}{+}^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{2,}t3\right)}\text{.}$$

$$\begin{array}{l}{\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^T{=}^{\text{c}\left(t3\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t3\right)}{\text{R}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}+\\ {}^{\text{c}\left(t3\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{2,}t3\right)}\end{array}$$

By multiplying both sides by [ ^{C (t1)} p̃] × or [ ^{C (t1)} p̃] × one obtains the multi-dimensional conditional equation $$\begin{array}{l}{\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^T{=}^{\text{c}\left(t1\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t1\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}+\\ {}^{\text{c}\left(t1\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t1\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{1,}t2\right)}\end{array}$$

$$\begin{array}{l}{\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^T{=}^{\text{c}\left(t3\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t3\right)}{\text{R.}}_{\text{c}\left(t2\right)}{\cdot }^{\text{c}\left(t2\right)}\tilde{\text{p}}{\cdot }^{\text{c}\left(t2\right)}\text{z}+\\ {}^{\text{c}\left(t3\right)}\tilde{\text{p}}{\times }^{\text{c}\left(t3\right)}{\overline{\text{t}}}_{\text{c}\left(t2\right)}\cdot s_{\left(t\mathrm{2,}t3\right)}\end{array}$$

verbleiben drei Bedingungsgleichungen mit einer Unbekannten, der relativen Skala ^(t2,t3)σ_(t1,t2).By solving 6 and 7 for ^{C (t2)} z and then equating them, one obtains three conditional equations with the unknowns s ₁₂ and s ₂₃ . All other sizes are known. By substitution $${}^{\left(t\mathrm{2,}t3\right)}{\sigma }_{\left(t\mathrm{1,}t2\right)}=\frac{s_{\left(t\mathrm{2,}t3\right)}}{s_{\left(t\mathrm{1,}t2\right)}}$$

This leaves three conditional equations with one unknown, the relative scale ^{(t2, t3)} σ _{(t1, t2)} .

$${}^{\text{c}\left(t2\right)}{\tilde{\text{z}}}_{23}=\frac{{}^{\text{c}\left(t2\right)}\text{z}}{s_{\left(t\mathrm{2,}t3\right)}}\text{.}$$

This can now be estimated in different ways. One or more equations of all point correspondences can be solved according to ^{(t2, t3)} σ _{(t1, t2) in} order to then apply the median or mean value, for example. A histogram solution is also possible. The conditional equations of all point correspondences can also be converted into a common, overdetermined linear system of equations in order to then be solved as a common optimization problem. Without loss of generality, adjacent time intervals (t1; t2) and (t2; t3) were selected in this example. The relative scale can, however, actually be determined for any, even overlapping time intervals. Instead of a direct calculation of the relative scale from the point correspondences, three-dimensional scene information can alternatively be determined for both time intervals, assuming a scale. This then contains an inherent scale which is proportional to the unknown scale. If both depth information relates to the same reference coordinate system, the calculation of the relative scale ^{(t2, t3)} σ _{(t1, t2)} is done by comparing the inherent scales of the two depth information. Assuming s _{(t1, t2)} = 1 and s _{(t2, t3)} = 1, normalized depths along the z-axis ^{C (t2)} z̃ ₁₂ and ^{C (t2)} z̃ ₂₃ can be calculated with respect to frame c _(t2) . For this applies $${}^{\text{c}\left(t2\right)}{\tilde{\text{z}}}_{\mathrm{12th}}=\frac{{}^{\text{c}\left(t2\right)}\text{z}}{s_{\left(t\mathrm{1,}t2\right)}}$$

$${}^{\text{c}\left(t2\right)}{\tilde{\text{z}}}_{23}=\frac{{}^{\text{c}\left(t2\right)}\text{z}}{s_{\left(t\mathrm{2,}t3\right)}}\text{.}$$

The relative scale is then calculated as $${}^{\left(t\mathrm{2,}t3\right)}{\sigma }_{\left(t\mathrm{1,}t2\right)}=\frac{s_{\left(t\mathrm{2,}t3\right)}}{s_{\left(t\mathrm{1,}t2\right)}}=\frac{{}^{\text{c}\left(t2\right)}{\tilde{\text{z}}}_{\mathrm{12th}}}{{}^{\text{c}\left(t2\right)}{\tilde{\text{z}}}_{23}}\text{.}$$

In the same way, other scene elements such as three-dimensional points or planes can also be calculated (e.g. ground plane). In the case of three-dimensional points, the radial distance can serve as depth information, in the case of the plane, the distance between this and the origin.

Correspondingly, the relative scale between the information from two metric sensors can also be determined directly.

Finally, the time offset is determined in the case of a speed-measuring sensor as a measuring sensor 150 accomplished.

To determine the time offset between an optical sensor and a sensor measuring a speed, the relative scale, as determined above, can be interpreted as the quotient of two integrals over the speed. Without limiting the generality, equidistant recording times, at a distance Δ, are assumed here for the optical sensor. Thus applies to the relative scale with respect to the optical sensor $${}^{\left(\text{t}-\Delta \text{, t}\right)}{\sigma }_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}\left(t\right)=\frac{s_{\left(\text{t}-\Delta \text{, t}\right)}}{s_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}}\text{.}$$

For the sake of simplicity, it is assumed here that the direction of translation of the optical sensor is parallel to the axis of the speed measurement. This is the case, for example, when the sensor platform moves in a straight line. The relative scale is calculated for the speed signal $$\begin{array}{l}{}^{\left(\text{t}-\Delta \text{, t}\right)}{\widehat{\sigma }}_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}\left(t\right)\\ \text{as}\\ {}^{\left(\text{t}-\Delta \text{, t}\right)}{\widehat{\sigma }}_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}\left(t\right)=\frac{{\displaystyle \int {}_{t-\Delta }^{t}v\left(t\right)dt}}{{\displaystyle \int {}_{t-2\Delta }^{t-\Delta }v\left(t\right)dt}}\end{array}$$

This gives two equivalent signals ^{(t-Δ, t)} σ _{(t-2Δ, t-Δ)} and ^{(t-Δ, t)} σ̂ _{(t-2Δ, t-Δ)} . If the speed signal is registered with a time offset δ, then the following applies $${}^{\left(\text{t}-\Delta \text{, t}\right)}{\sigma }_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}{=}^{\left(\text{t}-\Delta -\mathrm{\delta }\text{, t}-\mathrm{\delta }\right)}{\widehat{\sigma }}_{\left(\text{t}-2\Delta -\mathrm{\delta }\text{, t}-\Delta -\mathrm{\delta }\right)}\left(t-\mathrm{\delta }\right)$$

In 4th this relationship is illustrated graphically in a diagram. The time offset δ can now be determined using conventional methods (for example, considering the correlation). This can be done in that first the first scale parameter σ1, which is also referred to here simply with the variable σ, with a second scale parameter σ2 shifted by the time parameter Δ, which is also referred to here with the variable σ̂. It can be seen that the shape of the course of the first and second scale parameters σ1 and σ2 in the case of the time offset δ is within a tolerance range 400 is the same, with this tolerance range 400 for example represents a band in which the values of the second scaling parameter σ2 differ by a maximum of 20 percent from the first scaling parameter σ1. In this way, a very robust and reliable detection of the time offset δ can be made possible, in which the scale parameters are approximately the same. In another embodiment, the course of the first scaling parameter σ1 can also be matched with the course of the second scaling parameter σ2 within a time window 410 be compared. In this way, it can also be recognized very well whether the curves of the two scaling parameters are of the same type, so that the time offset δ, for example, not with the same values but with opposite curves (for example, with a falling curve of the first scaling parameter σ1 and an increasing curve of the second scaling parameter σ2) will. In the 4th a relationship between a relative time offset of the relative scale signals of the metric and the optical sensor is thus shown.

As an alternative or in addition, the time offset is also determined in the case of a depth-measuring sensor as the measuring sensor 150 presented. To determine the time offset δ between an optical and a depth-measuring sensor 150 can proceed in an equivalent manner will. Again applies to the relative scale with respect to the optical sensor $${}^{\left(\text{t}-\Delta \text{, t}\right)}{\sigma }_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}\left(t\right)=\frac{s_{\left(\text{t}-\Delta \text{, t}\right)}}{s_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}}\text{.}$$

The change in depth or scale information of a scene element (e.g. point or plane) is considered as a comparison variable. Let d (t) be depth or scale information of a scene element at time t, which is measured with a time offset, then applies $${}^{\left(\text{t}-\Delta \text{, t}\right)}{\widehat{\sigma }}_{\left(\text{t}-2\Delta \text{, t}-\Delta \right)}=\frac{d\left(t\right)-d\left(t-\Delta \right)}{d\left(t-\Delta \right)-d\left(t-2\Delta \right)}\text{.}$$

Again, two equivalent signals ^{(t-Δ, t)} σ _{(t-2Δ, t-Δ)} and ^{(t-Δ, t)} σ _{(t-2Δ, t-Δ) are obtained} , for which conventional methods (e.g. B. considering the correlation) the time offset can be determined.

5 shows a flowchart of an exemplary embodiment of a method 500 for determining a time offset between a reference time of a first signal and a reference time of a second signal. The procedure 500 includes one step 510 of reading in the first signal and the second signal, with a sequence of images being read in as the first signal that were acquired by an optical sensor at different acquisition times, and a sequence of a one-dimensional measured variable that was acquired at different acquisition times of a measuring sensor were detected, wherein the optical sensor and the measuring sensor are arranged on a common mobile carrier platform. The method also includes 500 one step 520 calculating first scaling parameters that represent scaling information between several images captured by the optical sensor at different capture times and each assigned to a reference time of the first signal, and calculating second scaling parameters from the sequence of the one-dimensional measured variable, whose reference points are compared to the reference times of the first signal are shifted according to a time parameter. Also includes the procedure 500 one step 530 comparing the calculated first scaling parameters with second scaling parameters from the second signal. Finally, the procedure includes 500 one step 540 determining that time parameter as the time offset between the first and second signal for which the highest correspondence of the compared scaling parameters was recognized in the step of comparing.

• - Tiefe/Entfernung der Szenenpunkte
• - Skalierung bzw. Geschwindigkeit der eigenen Bewegung
• - relative Geschwindigkeit fremdbewegter Objekte

It can also be noted that no depth information can be measured in monocular sensor systems (e.g. cameras) due to the projective detection of the scene. I. E. the scale of the scene is unknown (an arbitrarily scaled movement of the camera through a correspondingly scaled scene would produce the same image). The following information is therefore missing:

• - Depth / distance of the scene points
• - Scaling or speed of your own movement
• - Relative speed of objects moving outside

The camera movement can be determined from an image sequence over time, using the movement of the scene points in the image. This movement includes all three rotational degrees of freedom, but the translational movement can only be determined up to the unknown scale. You only know the direction of movement, not the distance or scale covered. If the above-mentioned scale-dependent information is to be received, the unknown scale should also be fed into the system. This can e.g. B. come from a speed-measuring or depth-measuring sensor.

• - wie verändert sich die Bewegung/Geschwindigkeit der Sensorplattform über die Zeit (relative Skala statischer Szenenpunkte verhält sich umgekehrt proportional)
• - sind Szenenpunkte statisch oder fremdbewegt
• - wie ist die relative Bewegung fremdbewegter Szenenpunkte (nähern oder entfernen sich)

One aspect of the approach presented here is to determine a change in the scale (relative scale) over time from the monocular sensor data. This is done point by point independently, so it is calculated separately for each scene point. The absolute scale remains unknown, but important questions can be answered, such as: B .:

• - How does the movement / speed of the sensor platform change over time (relative scale of static scene points is inversely proportional)
• - Scene points are static or externally moved
• - what is the relative movement of externally moved scene points (approaching or moving away from each other)

• - Erkennung fremdbewegter Szenenpunkte/Objekte wodurch eine semantische Segmentierung der Szene realisiert wird und zusätzlich die Bestimmung des Kollisionszeitpunkts.
• - Ist die relative Skala bekannt, kann sie mit der gemessenen relativen oder absoluten Skala (die absolute Skala lässt sich in eine relative umwandeln) eines anderen Sensors verglichen werden, wodurch ein zeitlicher Versatz der beiden Sensorströme erkannt und ausgeglichen werden kann.

If the measurement fails temporarily, the current speed can still be deduced from the relative scale of the proper movement.

• - Detection of scene points / objects that have been moved outside of the vehicle, whereby a semantic segmentation of the scene is implemented and, in addition, the determination of the time of the collision.
• - If the relative scale is known, it can be compared with the measured relative or absolute scale (the absolute scale can be converted into a relative) of another sensor, whereby a time offset between the two sensor currents can be recognized and compensated for.

The special thing is that the relative scale functions here as a universal measure, which can be derived robustly and efficiently for the optical sensor and is directly accessible for every metric sensor (e.g. measuring speed / depth). In fact, it is possible to solve complex problems that traditionally rely on depth / scale information in the solution, also in this relative representation. An example of this would be a driver assistance system for cars (emergency braking function, adaptive cruise control, ...).

The relative scale is determined in a very efficient way for each scene point separately, which opens up further possibilities for use (see above). Other methods do not use the fact that the problem can be separated into individual points, but rather calculate them using restricted models or as a change in a global variable.

The approach presented here enables the point-by-point, independent determination of spatially relative movement information from the temporal data stream of a monocular imaging sensor. Furthermore, a further embodiment enables a time offset to be determined in an efficient manner between a monocular optical sensor and one or more further sensors (optical or metric).

If an exemplary embodiment comprises an “and / or” link between a first feature and a second feature, this is to be read in such a way that the exemplary embodiment according to one embodiment has both the first feature and the second feature and, according to a further embodiment, either only the has the first feature or only the second feature.

Claims (12)

A method (500) for determining a time offset (δ) between a reference time (t) of a first signal (125, 135) and a reference time (t) of a second signal (155), the method (500) having the following steps: Reading (510) the first signal (125, 135) and the second signal (155), with a sequence of images (125, 135) being read in as the first signal (125, 135), which at different acquisition times (t) from an optical sensor (105) and, as the second signal (155), a sequence of a one-dimensional measured variable (155) is read in, which was recorded at different acquisition times (t) by a measuring sensor (150), the optical sensor (105) and the measuring sensor (155) are arranged on a common mobile carrier platform (100); - Calculation (520) of first scaling parameters (σ ₁ ), which represent scaling information between a plurality of images (125, 135) recorded by the optical sensor (105) at different recording times (t1, t2, t3, t4) and each a reference time ( t) of the first signal (125, 135) are assigned, and calculation of second scaling parameters (σ ₂ ) from the sequence of the one-dimensional measured variable (155) whose reference points (t) are compared to the reference times (t) of the first signal (125, 135) ) are shifted according to a time parameter (Δ); - Comparing (530) the calculated first scaling parameters (σ ₁ ) from the first signal (125, 135) with second scaling parameters (σ ₂ ) from the second signal (155); and - determining (540) that time parameter (Δ) as the time offset (δ) between the first (125, 135) and second signal (155) for which, in the step (530) of comparing, the highest agreement of the compared scaling parameters (σ ₁ , σ ₂ ) was recognized. Method (500) according to Claim 1 in which, in the step (530) of comparing, a time profile of first scaling parameters (σ ₁ ) within a time window (410) with a time profile of the second scaling parameters (σ ₂ ) within a time window (410) shifted by the time parameter (Δ) is compared and wherein in step (540) of determining that time parameter (Δ) as the time offset (δ) between the first (125, 135) and second signal (155) is determined, in which the time parameter (Δ) shifted by the time parameter (Δ) The course of the second scaling parameter (σ ₂ ) has the greatest similarity to the temporal course of the first scaling parameter (σ ₁ ). Method (500) according to one of the preceding claims, in which in the reading-in step (510) the first (125, 135) and second (155) signals are read in, in which the sequence of images (105) captured by the optical sensor (105) is read in. 125, 135) and / or the sequence of the measured variable (155) recorded by the measuring sensor (150) was recorded at the same time interval from recording times (t). Method (500) according to one of the preceding claims, in which, in the step (510) of reading in the second signal (155), a metric and / or physical quantity that can be absolutely recorded by the measuring sensor (150) is read in as the measured quantity. Method (500) according to Claim 4 , in which, in the step (510) of reading in, a sequence of the measured variable (155) is read in a time profile of a speed (v) of the mobile carrier platform (100). Method (500) according to Claim 5 , in which, in the step (520) of calculating, the second scaling parameters (σ ₂ ) are each calculated using an integral value formation over at least a time segment of the speed (v) used as the measured variable (155). Method (500) according to Claim 4 , in which, in the reading-in step (510), a spatial depth of a recognized object (112, 114, 115) recorded by the measuring sensor (150) of the mobile carrier platform (100) is read in as a measured variable (155). Method (500) according to one of the preceding claims, in which in the step (520) of calculating the first scaling parameters (σ ₁ ) using a determination of relative positions of associated scene points recognized in the images (125, 135) in the images (125, 135) can be calculated. Method (500) according to one of the preceding claims, with a step (550) of shifting a time base of the first (125, 135) and / or second signal (155) using the determined time offset (δ), in particular wherein the first (125 , 135) and / or the second signal (155) is delayed by the determined time offset (δ). Device (102) which is set up to execute and / or control the steps of the method (500) according to one of the preceding claims in corresponding units. Computer program which is set up to carry out the steps of the method (500) according to one of the preceding Claims 1 until 9 execute and / or control. Machine-readable storage medium on which the computer program is based Claim 11 is stored.

Images