WO2021121896A1

WO2021121896A1 - Method for synchronization and system

Info

Publication number: WO2021121896A1
Application number: PCT/EP2020/083498
Authority: WO
Inventors: Istvan SZELL
Original assignee: Robert Bosch Gmbh
Priority date: 2019-12-18
Filing date: 2020-11-26
Publication date: 2021-06-24
Also published as: EP4078334A1; DE102019219971A1

Abstract

The invention relates to a method for synchronization of a time system (28) of a main device (12) with a reference time system (30) of a reference device (14), wherein a light signal (22) is emitted using a light signal apparatus (20) of the reference device (14), a light signal section (24) of the light signal (22) being detected in at least one image (26) using a detection apparatus (18) of the main device (12), wherein the light signal section (24) can be associated with a point in time t_HG of the time system (28) of the main device (12), the light signal (22) comprising a specified, time-dependent signal sequence (32, 32a, 32b) which has a defined reference to a point in time t_RG of the reference time system (30), a time offset Δt (42, 42a, 42b) being determined from a comparison of the detected light signal section (24) with the specified time-dependent signal sequence (32, 32a, 32b) of the light signal (22), wherein a correction by the time offset Δt (42, 42a, 42b) is carried out for the purpose of synchronization. The invention also relates to a system (10) comprising at least one main device (12) and at least one reference device (14) for carrying out the method.

Description

description

title

Procedure for synchronization and system

The invention relates to a method for synchronizing a time system of a main device with a reference time system of a reference device. The invention further relates to a system comprising at least one main device and at least one reference device, the main device and the reference device being set up to carry out the method for synchronization.

State of the art

A large number of modern applications and technologies, for example applications on a smartphone in the field of “virtual reality” or “computer vision”, require an operating system operating the application or technology and / or the application itself to be real-time capable. Widely used operating systems such as Android, iOS, Windows 10 Mobile, BlackBerry 10, Tizen or Sailfish OS are not real-time capable, so that the implementation of time-critical tasks or applications on such operating systems is often impossible.

Real-time capability of an otherwise non-real-time capable device or operating system can be made possible, for example, by a time synchronization of the time system of the device with a reference time system, for example a reference time system of a reference device. Methods for synchronizing time systems are already known from the prior art, for example from WO 03/093856 A2, which discloses a method for synchronizing a main device with an external reference device, the main device in particular having a non-real-time operating system. Furthermore, the aspect of synchronization is discussed in detail in “Software development for real-time systems”, edited by Juliane T. Benra.

These prior art methods are based on the disadvantage that interfaces used for synchronization ("input / output interface") such as WLAN, Bluetooth or NFC are also non-real-time capable, since these are based on the application level. "Application layer") are not directly accessible without the interposition of software abstraction layers. This interposition of software abstraction levels causes a non-deterministic behavior of the device executing the method, in particular the operating system, with the consequence of unpredictable latency times. Consequently, using prior art methods, precise synchronization - for example in the millisecond range - is not possible.

Disclosure of the invention

A method is proposed for a time synchronization of a time system of a main device with a reference time system of a reference device, in particular for a time synchronization of a time system of a non-real-time capable operating system of the main device with the reference time system of the reference device, a light signal using a light signal device of the reference device is emitted, with a light signal section of the light signal being detected in at least one image recording using a detection device of the main unit, the light signal section being assignable to a point in time ΪH _{Q of} the time system of the main unit, wherein the light signal comprises a predetermined time-dependent signal sequence which has a defined reference to a point in time t _{RG of} the reference time system, a time offset At being determined from a comparison of the detected light signal section with the given time-dependent signal sequence of the light signal, with a correction being made for synchronization the time offset At is carried out.

Time synchronization is understood to mean that a reference is made between the time system of the main device and the reference time system of the reference device. The reference allows a point in time (in particular a time specification) of the time system of the main device to be converted into a point in time (in particular a time specification) of the reference time system of the reference device - or vice versa. If the reference time system is real-time capable, a reference to an, in particular external, real-time capable reference time system can be made from the non-real-time capable time system of the main system. In particular, the main device can also be made real-time capable in this way.

In one embodiment, the main device is implemented by a mobile device such as a smartphone, a tablet, a motion controller of a game console or the like. In one embodiment, the reference device is implemented by an embedded system. The reference device can in particular be implemented as a device external to the main device. Furthermore, the reference device can be implemented as an embedded system (“embedded system” or “embedded device”). In an application example, the method for synchronization can be used in a method for positioning a mobile device (reference device), with relative position data provided by an inertial sensor system of the device (from the movement of the device) being compared with absolute position data, with the absolute Position data can be recorded with a camera system of another device (main device), for example a smartphone. The inventive method for synchronization allows a latency period (also: latency) of the other device in the Evaluation (so-called "sensor fusion") of the relative and absolute position data must be taken into account.

It should be noted that in the following no explicit distinction is made between a time system (reference time system) of a main device (reference device) and a time system (reference time system) of an operating system that is operated on the device (reference device). Both are used synonymously here.

The light signal device is used to generate and emit a light signal that can be detected by the detection device. The light signal can in principle be implemented in any desired spectral range, for example in the visual light spectrum, in the ultraviolet range and / or in the infrared range. In one embodiment of the method, the light signal device comprises at least one LED. In particular, the light signal device can also comprise several LEDs, in particular special ones for generating different colors. In an alternative or additional embodiment of the method, the light signal device comprises at least one laser diode and / or at least one laser diode for generating a laser line. A laser diode for generating a laser line is to be understood in particular as a laser diode with an optical device, for example egg ner cylinder lens, the optical device splitting a laser beam emitted by the laser diode into a laser plane in such a way that the projection of the laser plane onto an object a laser line is generated.

The detection device is sensitive at least in the spectral range of the light signal used and is set up to at least partially detect the light signal. The at least partially detected section of the light signal is referred to as the light signal section. In particular, the detection device is spatially resolving. The detection device has in particular an array of light-sensitive elements (pixels). In one embodiment of the method, the detection device has a rolling shutter effect. The “rolling shutter effect” means that the exposure of all light-sensitive elements of the detection device does not take place at exactly the same point in time, but that an area of simultaneous exposure - typically a column or a line - moves over the detection device column by column or line by line. The area needs a period of time that cannot be neglected when it moves column-wise or line-wise, so that, especially in the case of rapidly changing motifs, each successively traversed area - ie every column or line - depicts a changed motif. In one embodiment of the method, the detection device is implemented as an optical image sensor, in particular as an optical image sensor with a rolling shutter effect, in particular as a CMOS image sensor. The optical image sensor can be referred to as a camera.

The light signal denotes a signal emitted by the light signal device, for example a continuously or intermittently emitted light of one or more colors and / or a continuously or intermittently emitted light of one or more intensities. The light signal is emitted, i.e. transmitted, by the light signal device and detected, i.e. received, by the detection device - at least partially as a light signal section. The light signal in connection with the light signal device and the detection device serves as the interface used for synchronization of the system comprising the main device and the reference device.

The light signal comprises a predetermined time-dependent signal sequence. A signal sequence denotes a sequence of distinguishable states of the emitted light signal. The distinguishable states can be detected in particular by means of the detection device. The distinguishable states - for example different colors or intensities of the light signal - each cause a change of state between successive states - for example from red to green. In particular, the time-dependent signal sequence can also be implemented as a predefined time-dependent signal modulation of the light signal.

“Preset” is to be understood as meaning that the signal sequence is defined and reproducible, whereby it can be provided, for example, by reading out from a memory device or from a data communication interface of the control device of the main device and / or the control device of the reference device. In particular, the signal sequence can also be transmitted to a further device, for example as a code or protocol, so that the signal sequence can in principle also be made available to the further device as a predetermined time-dependent signal sequence. A change of state is to be understood in particular as a transition or a change or a sequence from a first state to a second state that is distinguishable from the first state (state change or state transition or state sequence).

In one embodiment of the method, no repetition of a state change, in particular a state change or a state transition or a state sequence, occurs within the predetermined time-dependent signal sequence of the light signal. Consequently, an unambiguous (in particular unambiguous) assignment of a change in state - at least relatively within the signal sequence - can be made possible at a defined point in time if the rate of change in the state of the states is known. The "rate of change of state" is to be understood in particular as the frequency with which the states of the time-dependent signal sequence change, i.e. with which a change in state occurs. "Rate of change of state of the signal sequence" and "Rate of change of state of the states" are used synonymously. The rate of change of state (in Hz) can be converted into a state duration (corresponding to the period duration associated with the frequency). If the states change, for example, at a frequency of 100 Hz, a respective state duration (i.e. the duration during which the light signal is emitted in red, for example) is 10 ms. According to this duration of state, for example, the third change of state in the time-dependent signal sequence occurs after 30 ms from the start of the signal sequence.

In one embodiment of the method, the emitted light signal comprises a predefined time-dependent signal sequence with regard to a state of a color of the light signal. For example, using three LEDs in the colors red (R), green (G) and blue (B), a signal sequence can be given by the sequence of the states of the colors RGRBGBR. The signal sequence includes six clearly identifiable color transitions (RG, GR, RB, ..., BR) as changes in status, with no regression within the signal sequence RGRBGBR. repetition of a change of state occurs. In an alternative embodiment example, three LEDs in the colors red (R), green (G) and blue (B) as well as the colors orange (0 = R + G), violet (V = R + B), which can be mixed therefrom, can also be used ) and turquoise (T = G + B) a signal sequence can be given that includes thirty color transitions and thus thirty changes in state.

Alternatively or additionally, the emitted light signal comprises a predetermined time-dependent signal sequence with regard to a state of an intensity of the light signal. It is conceivable that the light signal includes distinguishable states such as “light” (H), “dark” (D) and “off” (A). A sequence of signals can also be given with these states, for example through the sequence of the states of the intensities H-D-H-A-D-A-H. The signal sequence comprises six clearly identifiable intensity transitions (H-D, D-H, ..., A-H) as changes in state, with no repetition of a state change occurring within the signal sequence H-D-H-A-D-A-H.

Furthermore, a combined light signal is conceivable which comprises a predetermined time-dependent signal sequence with regard to a state of an intensity and color. Further states of the emitted light signal are in principle conceivable, for example states of a contrast or states of a polarization angle. The selection of the states - i.e. the selection of a type of the states of the light signal - can be matched to the detection device used. For example, the selection of the type of states of the light signal can be dependent on the resolution capability of the detection device with regard to a brightness difference and / or a color nuance. A stringing together of the signal sequence to generate a continuous light signal comprising a, in particular periodically, repeated signal sequence is conceivable.

The predefined time-dependent signal sequence has a defined (ie deterministic or determinable or calculable) reference to a point in time t _{RG of} the reference time system of the reference device (index “RG”). In particular, the predetermined time-dependent signal sequence can begin or be started _{at the point in time t RG of the reference time signal.} A defined time offset - of, for example, 10 ms (e.g. as a result of a Latency) - or a determinable (calculable or determinable) and thus defined time offset conceivable. Through this clear (in particular special one-to-one) reference, the predetermined time-dependent signal sequence emitted by the light signal device by means of the light signal and transmitted in this way can be linked to the reference time system of the reference system. By using a signal sequence with unambiguous (that is, one-off) state changes, a time reference to the time t _{RG of} the reference time system can be established for each state change if - as already explained above - the rate of state change of the time-dependent signal sequence is known. Consequently, a point in time at which an unambiguous change in state occurs - at least within the specified signal sequence - can be determined unambiguously with reference to the reference time system if the rate of change in state of the signal sequence is known. If the states change, for example, with a frequency of 100 Hz (corresponding to a state duration of 10 ms) and the signal sequence begins at time t _RG ° based on the reference time system, the third change in state in the signal sequence occurs at time t _RG ³ = t _RG ° + 30 ms, based on the reference time system.

The number of unambiguous (i.e. non-repeating) changes in the state of the signal sequence provided can be used to influence or specifically select a time resolution of the signal sequence (in particular in relation to the reference time system) and thus a time resolution of the synchronization. A higher number of state changes enables, for example, a higher temporal resolution. In contrast, a smaller number of state changes enables the implementation of comparatively long emission periods of a respective state, so that the detection device simplifies the detection and differentiation of the state and a particularly robust method can be achieved. A more robust method can consequently be implemented with reduced requirements for the light signal device and the detection device.

The light signal segment is to be understood as an at least section-wise part of the light signal that is detected at least temporarily or temporarily by the detection device. The light signal section also includes a section of the predetermined time-dependent signal sequence emitted with the light signal - hereinafter also referred to as “detected signal sequence”. The light signal section is formed by the components of the light signal that are detected by the detection device. The detection takes place in the form of image recordings (images or frames) by means of the detection device. It should be noted that a section of an image recording (for example captured through one half of the light-sensitive surface of the detection device) can also be viewed as an image recording. The light signal section can be assigned at least to a point in time ΪH _{Q of} the time system of the main device (index "HG"). This can be implemented, for example, via a time stamp of the image recording in the metadata. The time stamp advantageously originates from a physical preprocessing unit of the detection device and therefore has no latency due to interposed software abstraction levels. This assignment can be used to link the point in time at which the light signal section is detected with the time system of the main device.

A time offset Δt is determined from a comparison of the detected light signal section, in particular the detected signal sequence of the light signal, with the predetermined and consequently known time-dependent signal sequence of the light signal. This time offset is the key to synchronizing the time system of the main device and the reference time system of the reference device. “Comparison” is to be understood in particular to mean that the detected signal sequence contained in the detected light signal segment is compared with the predefined time-dependent signal sequence in such a way that an identification and / or assignment of states is possible. For example, a detected light signal section comprising the detected signal sequence “R-B” can be discovered in the signal sequence “R-G-R-B-G-B-R” comprised in the emitted light signal. The time offset At can be determined via a look-up table or a calculation rule.

With knowledge of the time offset At, a synchronization of the time system of the main device and the reference time system of the reference device can finally be carried out. For example, the time system of the main device can be adapted to the reference time system of the reference device by correcting ture of the time system of the main device by the time offset At: ΪHQ: = At + t _RG °. Alternatively, the reference time system of the reference device can be adapted to the reference time system of the reference device by correcting the reference time system by the time offset At: t _RG : = t _HG - At. The correction denotes in particular offsetting, compensating, equating, subtracting, multiplying or adding at least one of the time systems with the aim of establishing a clear (in particular one-to-one) relationship between the time systems. In particular, it is also conceivable to correct both time systems.

As an example, the light signal introduced above includes the unambiguous signal sequence of the statuses of the colors "RGRBGBR". By using the signal sequence with unambiguous (ie each unique) status changes - as described above - in principle a one-to-one time change can be made for each status change Reference can be made at a point in time t _{RG of} the reference time system. It is assumed here that the specified time-dependent signal sequence begins or is started _{at time t RG} ^{0 of the reference time signal.} Furthermore, the rate of change of state of the time-dependent signal sequence is assumed to be known. While the method is being carried out, the detection device can for example detect the light signal segment including the signal sequence “RB”, for example in at least one image recording, the at least one image recording having a time stamp t _HG and thus a clear (in particular one-to-one) reference to the time system of the main device having. The time offset At can now be determined from the comparison of the detected light signal segment, in particular the detected signal sequence “RB”, with the predefined time-dependent signal sequence “RGRBGBR” of the light signal. From the comparison, the “position” of the change in status can be recognized as the third change in status (first position “R-G”; second position “GR”; third position “RB”). Given an assumed rate of change of state of 100 Hz with a state duration of 10 ms, a time offset of At = 3 * 10 ms = 30 ms can be calculated from the knowledge of the third position - relative to the start of the signal sequence, i.e. relative to t _RG ⁰ . By referring to the predetermined time-dependent signal sequence at the point in time t _{RG of} the reference time system, the following applies: t _HG = t _RG ⁰ + At, ie t _HG = t _RG ⁰ + 30 ms. t _RG ⁰ is off known to the reference time system and can, for example, be transmitted from the reference device to the main device using a data communication interface.

The method according to the invention makes it possible to carry out a synchronization based on time information that originates from a peripheral device that stores this time information in a hardware layer and / or in such software layers. ), which can be accessed with a determinable (calculable or determinable) latency period. In this way, it is possible to synchronize a time system of a non-real-time capable device, in particular a time system of an operating system of a non-real-time capable device, with a reference time system of a reference device.

Depending on a ratio of an image recording rate of the detection device and a rate of change of state of the time-dependent signal sequence, different embodiments of the method with different effects are conceivable, which will be outlined briefly below.

In one embodiment of the method, an image recording rate of the detection device essentially corresponds to a rate of change of state of the time-dependent signal sequence. Alternatively, an image recording rate of the detection device is greater than a state change rate of the time-dependent signal sequence (ie the image recording frequency is faster). In these embodiments of the method, in order to determine the time offset At, a light signal section can be evaluated which is detected in a plurality of, in particular sequential, image recordings. In this context, “essentially corresponds” is to be understood as meaning that the image recording rate and the rate of change of state of the time-dependent signal sequence do not differ from one another by more than 10%, in particular do not differ from one another by more than 5%, in particular no more than 2% from one another. “Greater” is to be understood here in particular to mean that the image recording rate is at least 5%, in particular at least 10%, very particularly at least 15% greater than the rate of change of state. In one embodiment, the image capture rate and the state change rate are identical to the time-dependent signal sequence. A light signal section which is detected in a plurality of image recordings thus has a changed state after a period of time which elapses for the detection of an image recording. If the emitted light signal comprises, for example, a predetermined time-dependent signal sequence with regard to a state of a color, then in sequentially detected image recordings, a change in state, ie a different color, is to be expected per image recording. This sequence of the states detected in a plurality of image recordings - ie the detected signal sequence - can then be compared with the predetermined time-dependent signal sequence. The plurality of image recordings can in particular be understood to mean sequentially detected image recordings. Alternatively, the image recordings can also be selected from a series of sequentially detected image recordings.

In an alternative embodiment of the method, an image recording rate of the detection device is less than a state change rate of the time-dependent signal sequence (ie the image recording frequency is slower). In this embodiment of the method, to determine the time offset At, a light signal segment can be evaluated, which is detected in a plurality of, in particular sequential, image recordings. “Smaller” here is to be understood in particular to mean that the image recording rate is at least 5%, in particular at least 10%, very particularly at least 15% less than the rate of change of state. A light signal section that is detected in a plurality of image recordings thus typically has a plurality of detected states per image record. If the emitted light signal comprises, for example, a predetermined time-dependent signal sequence with regard to a state of a color, a plurality of changes in state, ie several color changes, can be expected for sequentially detected image recordings. This sequence of the detected states - ie the detected signal sequence - can then be compared with the predetermined time-dependent signal sequence. Furthermore, in this embodiment of the method for determining the time offset At, alternatively or in addition, a light signal excerpt can also be evaluated which is detected in (in particular also: within) a single image recording of the detection device. Since a single image capture is already a If it comprises a plurality of detected states, the light signal segment detected in this way - ie the signal sequence detected in a single image recording - can be compared with the predetermined time-dependent signal sequence of the light signal. It is particularly advantageous that a rolling shutter effect of the detection device generates a type of “barcode”. The "barcode" refers to areas of states arranged next to one another in the image recording (column by column or line by line) in which, due to a simultaneous exposure - a respective column or a respective line - a state that was present in the light signal section at the time of exposure (e.g. color ) was shown.

In one embodiment of the method, at least one further time offset At, is determined from a comparison of at least one further detected light signal section with the predetermined time-dependent signal sequence of the light signal, a relative time drift being determined from the time offsets At and At Correction for the relative time drift is carried out. It is known that time systems generated using different clocks can drift, ie diverge, over time. For example, a quartz can be subject to a time drift under the influence of temperature fluctuations, ie time passes faster or slower. If the time system of the main unit and the reference time system of the reference system have different clocks and / or different influencing parameters (e.g. temperature), the time systems can drift apart _{despite initial synchronization - for example at time t HG} ^s = t _RG ^s. It should be noted that in order to synchronize the time system of the main device and the reference time system of the reference device, the correction for the relative time drift can be carried out again on the time system of the main device and / or on the reference time system of the reference device. For example, it is conceivable to apply a scaling factor to a clock rate of the time system of the main device, which is used to compensate and thus correct the relative time drift. The scaling factor can be determined, for example, from the ratio of the specific time offsets s = At / Ati. It is also conceivable to determine further time offsets continuously or quasi-continuously and to implement a continuous synchronization or at least a continuous control of the synchronization of the time systems in this way. In one embodiment of the method, in a preceding process step, at least one setting parameter, in particular relating to the detection device and / or the light signal device and / or the predefined time-dependent signal sequence, is transmitted from the main device to the reference device and / or from the reference device to the main device, wherein an adaptation of the light signal device and / or the detection device and / or the predetermined time-dependent signal sequence is carried out as a function of the at least one transmitted setting parameter. A setting parameter relating to the detection device can be, for example, a recording rate, an exposure duration, an ISO value, a focus setting, a white balance setting and / or the like. A setting parameter relating to the light signal device can be, for example, a luminosity setting. A setting parameter relating to the signal sequence can, for example, be a setting of the type of states of the light signal (color, intensity, ...) and / or a rate of change in the state of the states. For example, the rate of change of state can be adjusted by transferring a corresponding setting parameter to a detection device used, in particular its image recording rate. A transmission can in particular be implemented using a wired or wireless data communication interface, for example a WLAN, Bluetooth or an NFC interface. The at least one transmitted setting parameter enables advantageous adaptations to be made in order to carry out the method. For example, an adaptation of the light signal device can include a selection of the light signal device and / or a selection of a type of the states of the light signal (color, intensity, ...) or, furthermore, an adjustment of the luminosity or the like. An adaptation of the detection device can relate to an image recording rate or an exposure duration, for example. An adaptation of the predetermined time-dependent signal sequence can in particular include the selection of the signal sequence. In an alternative embodiment of the method, the setting parameters are permanently predefined and, in particular, are made available to the main device and the reference device, in particular invariably. In a further aspect of the invention, a system comprising at least one main device and at least one reference device is proposed, the main device and the reference device being set up to carry out the method according to the invention. In particular, the main device and the reference device, in particular each, have a control device which is set up to carry out the method, in particular to carry out the synchronization. A control device has a processor device and a memory device (not shown here in more detail). The method is at least partially stored in the memory device as a machine-readable computer program. The computer program contains instructions which, when executed by the processor device, cause the processor device to, at least partially, execute the synchronization method.

In one embodiment of the system, the light signal device is implemented as at least one LED and / or a laser diode and / or a laser diode for generating a laser line. In particular, the light signal device is assigned to the reference device.

In one embodiment of the system, the detection device is implemented as an optical image sensor, in particular as an optical image sensor with a rolling shutter effect, in particular as a CMOS image sensor.

In the following, “provided” or “set up” should be understood specifically as “programmed”, “designed”, “designed” and / or “equipped”. The fact that an object is “provided” or “set up” for a specific function is to be understood in particular to mean that the object fulfills and / or performs this specific function in at least one application and / or operating state or is designed to do so Function.

drawings

The invention is explained in more detail in the following description with reference to Ausführungsbei shown in the drawings. The drawings that The description and the claims contain numerous features in combination. The person skilled in the art will expediently also consider the features individually and combine them into useful further combinations. The same reference characters in the figures denote the same elements.

Show it:

Figure 1 is a schematic view of a system according to the invention;

FIG. 2 shows an embodiment of the method according to the invention, shown in a method diagram;

FIG. 3 shows a schematic representation of a time system of a main device and a reference time system of a reference device;

FIG. 4 shows schematic views of image recordings recorded during the implementation of the method according to the invention at different times (a) and (b);

FIG. 5 shows a schematic view of an image recording acquired while the method according to the invention is being carried out.

FIG. 1 shows a schematic view of an embodiment of the system 10 for carrying out the method 100 according to the invention. The system 10 comprises a main device 12 and a reference device 14. The main device 12 is implemented, for example, as a smartphone. The reference device is implemented, for example, as a mobile device external to the main device 12. A time synchronization of both devices is desired in order to be able to convert precisely times of the time system 28 of the main device 12 into times of the reference time system 30 of the reference device 14 - and vice versa.

The main device 12 and the reference device 14 each have a control device 16 (which should also be understood to mean an evaluation device). The respective control device 16 is specially set up to carry out the method 100. For this purpose, the respective control device 16 has a processor device and a storage device (not shown in detail here). In the storage device, the method 100 is at least partially as a machine n-readable computer program stored. The computer program contains instructions which, when executed by the processor device, cause the processor device to, at least partially, execute the synchronization method. The respective control device 16 is also set up to operate the assigned device and is connected in terms of signaling to functional components of the respective device.

The reference device 14 has a light signal device 20 which is implemented as at least one LED. The LED is provided to generate at least the different colors red, green and blue and to emit them as light signal 22. The main device 12 has a detection device 18 which is implemented as a CMOS image sensor. The CMOS image sensor has a rolling shutter effect (not shown in detail here). The detection device 18, in particular the CMOS image sensor, is set up to detect a light signal section 24 of the light signal 22 in at least one image recording 26 (cf. FIGS. 4 and 5).

In FIG. 2, an exemplary embodiment of the method 100 according to the invention is shown in a method diagram. The method 100 is used to synchronize the time system 28 of the main device 12 with the reference time system 30 of the reference device 14. The method 100 is explained below in connection with FIG Reference time system 30 of the reference device 14 reproduces. Each time system is shown as a vertical timeline with lines as time units. As can be seen in FIG. 3, the two time systems are not synchronous - represented here by inconsistent time units (lines) and different intervals between the lines in the two time systems. In particular, it should be pointed out that the arrow 40, which starts from a time unit (line) in the reference time system 30, does not fall on a time unit (line) in the time system 28. Furthermore, in Figure 3 - to clarify the interrelationships discussed below - signal sequences 32, 32a, 32b emitted with a light signal 22 and detected signal sequences 34, 34a, 34b of a detected light signal section 24 are shown. In method step 102, a communication link is established between the main device 12 and the reference device 14 using a data communication interface assigned to the main device 12 (not shown here) and a data communication interface assigned to the reference device 14 (also not shown here). The data communication interfaces are exemplified as Bluetooth data communication interfaces, the communication link being a Bluetooth communication link. Using the communication connection, setting parameters relating to the detection device 18 and relating to the predefined time-dependent signal sequence 32, 32a are transmitted from the main device 12 to the reference device 14. The reference device 14 confirms receipt of the setting parameters. The setting parameters include a recording rate and an exposure duration of the detection device 18 as well as a type of status of the light signal (here “color”, see below) and a status change rate of the signal sequence 32, 32a to be emitted with the light signal 22 (here 100 Hz). This specifies for the reference device 14 that the light signal 22 should include a predetermined time-dependent signal sequence 32, 32a with regard to a state 36 of a color of the light signal 22. The control device 16 of the reference device 14 selects and / or adjusts the light signal device 20 as a function of the transmitted setting parameters. In particular, the type of states 36 of the light signal 22 is set to “color” (for example, in contrast to “intensity”) and the rate of change of state of the states 36 is set to 100 Hz. Furthermore, a luminosity setting of the LED is adapted to the transmitted exposure duration. The control device 16 of the main device 12 selects and / or adjusts the detection device 18 as a function of the transmitted setting parameters. In particular, the image recording rate and the exposure duration of the detection device 18 are adapted or set. Furthermore, an evaluation algorithm adapted to the type of states 36 for recognizing and differentiating colors in detected image recordings 26 is selected.

In method step 104, a light signal 22 is emitted using the light signal device 20 of the reference device 14, the light signal 22 comprising a given time-dependent signal sequence 32, 32a. The signal sequence 32, 32a is implemented as a sequence of color states 36 of the emitted light signal 22 of the colors red (R), green (G) and blue (B), the signal sequence 32, 32a here comprising the sequence RGRBGBR. The signal sequence 32, 32a comprises six clearly identifiable (that is, each unique) color transitions as state changes 38, with no repetition of a state change 38 (ie of a specific color transition such as “RG” or “GR”) occurring within the signal sequence RGRBGBR. To emit the light signal 22 comprising the signal sequence 32, 32a, the control device 16 of the reference device 14 controls the LED in accordance with the given signal sequence 32, 32a. In an alternative embodiment, instead of a sequence of color states, the signal sequence can also comprise a sequence of intensity states (not shown in more detail here). The signal sequence 32, 32a has a defined reference to the point in time t _RG ° of the reference time system 30, in that the start of the emission of the signal sequence 32, 32a (not necessarily the light signal 22) _{falls here on the point in time t RG} °. The light signal 22 comprising the signal sequence 32, 32a is sent out once here. A one-to-one assignment of a state change 38 to a specific point in time t _{RG is} made possible. Since the states 36 change here with a state change rate of 100 Hz, the third state change 38 “RB” occurs, for example, in the signal sequence 32, 32a after a time offset At (reference number 42, 42a) of 30 ms from the start of the signal sequence, ie after time t _RG ⁰ .

In method step 106, a light signal section 24 of the light signal 22 is detected in at least one image recording 26 using the detection device 18 of the main device 12. The light signal section 24 also includes a section of the predetermined time-dependent signal sequence 32, 32a emitted with the light signal 22, here referred to as “detected signal sequence 34, 34a”. The detected light signal segment 24 can be clearly assigned to a point in time t _{HG of} the time system 28 of the main device 12 by means of the at least one image recording 26. Here, the unambiguous assignment is implemented using a time stamp of at least one image recording 26 in the metadata. The light signal segment 24 detected in this exemplary embodiment comprises the detected signal sequence 34, 34a, which is shown in FIG. 3 as “RB”. Consequently, the color transition or, more generally, the change of state 38 in the detected signal sequence 34, 34a can also be clearly assigned to the point in time t _{HG of} the time system 28. A possible latency period between the point in time ΪH _Q and the image recording 26 is known in principle as a deterministic (ie firmly defined, determinable or calculable) variable for carrying out the method 100 and is not considered further here.

In the exemplary embodiment shown, the setting parameters of the light signal device 20 and / or the detection device 18 are selected such that an image recording rate of the detection device 18 essentially corresponds to the rate of change of state of the time-dependent signal sequence 32, 32a (or, alternatively or additionally, is also greater than the rate of change of state the time-dependent signal sequence 32, 32a). FIGS. 4a and 4b each show an image recording 26 as it was recorded by the light signal device 20, in particular the LED, by means of the detection device 18. FIGS. 4a and 4b show a light signal section 24, in particular a detected signal sequence 34, 34a, which is detected in a plurality of - here sequential - image recordings 26 of the detection device 18. Here, FIG. 4a reproduces an image recording 26 which was detected immediately before the image recording 26 of FIG. 4b. It can be seen that due to the comparatively slow image recording rate in sequentially detected image recordings 26, a change in status 38, ie a different color than status 36, is detected per image recording 26. In FIG. 4a, the image recording 26 depicts the state 36 red, while the image recording 26 in FIG. 4b depicts the state 36 blue (see also FIG. 3). This sequence of the detected states 36 can then be compared with the predefined time-dependent signal sequence 32, 32a. Each image recording 26 is given a unique time stamp so that the state change 38 can be clearly assigned to a point in time ΪH _{Q of} the time system 28.

In an alternative or additional embodiment, the setting parameters of the light signal device 20 and / or of the detection device 18 are selected as described above such that an image recording rate of the detection device 18 is less than a rate of change of state of the time-dependent signal sequence 32, 32a. FIG. 5 shows an image recording 26 as recorded by the light signal device 20, in particular the LED, by means of the detection device 18. was taken. A light signal excerpt 24, in particular a detected signal sequence 34, 34a, can be seen here, which is detected in a (single) image recording 26 of the detection device 18. It can be seen that due to the rolling shutter effect (indicated by the arrow 44, in the direction of which the column-by-column exposure proceeds during an image recording 26), a time-dependent splitting of the light signal section 24 into the different states 36 in an image recording 26 takes place, so that the one (only) image recording 26 already comprises a plurality of detected states 36 (can be referred to as a “barcode”). It should be noted that the illustration in FIG. 5 differs in the number of detected states 36 to clarify the effect from the schematic illustration in FIG. 2 - there only two states are entered in 34, 34a. Furthermore, it is of course also conceivable to evaluate a light signal segment 24 (ie several “barcodes”) detected in several image recordings 26.

In method step 108, a comparison of the detected light signal segment 24, in particular the detected signal sequence 34, 34a, with the predefined time-dependent signal sequence 32, 32a of the light signal 22 is carried out and a time offset At (reference numeral 42, 42a) is determined therefrom. In the illustrated embodiment, the time offset At 42, 42a of the light signal section 24 is evaluated, which was detected in two sequential image recordings 26 (see. Figu ren 4a and 4b). In particular, the color transition of the detected signal sequence 34, 34a is matched with the predefined time-dependent signal sequence 32, 32a, with an identification and assignment of states 36 being possible. The above-mentioned evaluation algorithm is set up to carry out this identification and assignment based on image recognition algorithms and / or image analysis algorithms. In this exemplary embodiment, the detected light signal section 24 comprising the signal sequence “RB” in the signal sequence 32, 32a “RGRBGBR” included in the emitted light signal 22 can be recognized as a third change in state 38. With the known rate of change of state of 100 Hz (duration of state 10 ms), the time offset 42, 42a can therefore be calculated from the knowledge of the third state change 38 to At = 3 * 10 ms = 30 ms - relative to the start of the signal sequence at t _RG ° of the reference time system 30. In the alternative or additional exemplary embodiment, it is conceivable to carry out a comparison of the detected light signal section 24, in particular the detected signal sequence 34, 34a, with the predetermined time-dependent signal sequence 32, 32a of the light signal 22 in a method step 108, the time offset At 42, 42a is then evaluated from the light signal excerpt 24 which was detected in the majority of the - here sequential - image recordings 26.

In method step 110, a correction by the time offset At 42, 42a is carried out for synchronization, whereby in this embodiment a correction value is temporarily stored in the control device 16 of the main device 12, which is used to convert the time system 28 of the main device 12 into the reference time system 30 of the Reference device 14 - and vice versa - is used. The calculation rule for the correction is then given by ΪH _Q = At + t _RG , ie here by ΪH _Q = t _RG + 30 ms.

The method 100 can be carried out repeatedly - shown here by a dashed arrow 112. By performing the method 100 again, it is possible, from a comparison (method step 108), to determine at least one further detected light signal segment 24, in particular a further detected signal sequence 34, 34b (cf. FIG. 3), with a further predetermined time-dependent signal sequence 32, 32b (which can also be identical to the first signal sequence 32, 32a) of the light signal 22 to determine at least one further time offset At, 42, 42b. A relative time drift can then be determined from the time offsets At 42, 42a and At, 42, 42b in sub-method step 110a, where a correction for the relative time drift is also carried out for synchronization. In method step 110 in this exemplary embodiment, a correction value is temporarily stored in the control device 16 of the main device 12, which is used to convert the time system 28 of the main device 12 into the reference time system 30 of the reference device 14 and to correct the time drift. After the method 100 according to the invention has been carried out, a temporal reference between the time system 28 of the main device 12 and the reference time system 30 of the reference device 14 is advantageously established. The reference allows a point in time of the time system 28 of the main device 12 to be converted into a point in time of the reference time system 30 of the reference device 14 - and vice versa.

Claims

Expectations

1. A method for synchronizing a time system (28) of a main device (12) with a reference time system (30) of a reference device (14),

• a light signal (22) being emitted using a light signal device (20) of the reference device (14),

• using a detection device (18) of the main device (12) a light signal section (24) of the light signal (22) is detected in at least one image recording (26), the light signal section (24) at a point in time ΪH _{Q of} the time system ( 28) of the main unit (12) can be assigned, wherein

• the light signal (22) comprises a predetermined time-dependent signal sequence (32, 32a, 32b) which has a defined reference to a point in time t _{RG of} the reference time system (30),

• where a time offset At (42, 42a, 42b) is determined from a comparison of the detected light signal section (24) with the predetermined time-dependent signal sequence (32, 32a, 32b) of the light signal (22),

• a correction by the time offset At (42, 42a, 42b) being carried out for synchronization.

2. The method according to claim 1, wherein from a comparison of at least one further detected light signal section (24) with the predetermined time-dependent signal sequence (32, 32a, 32b) of the light signal (22) at least one further time offset At, (42, 42a, 42b) is determined, with a relative time drift being determined from the time offsets At and At, with a correction for the relative time drift also being carried out for synchronization.

3. The method according to any one of claims 1-2, wherein the emitted light signal (22) comprises a predetermined time-dependent signal sequence (32, 32a, 32b) with regard to a state (36) of a color of the light signal (22).

4. The method according to any one of claims 1-3, wherein the emitted light signal (22) comprises a predetermined time-dependent signal sequence (32, 32a, 32b) with regard to a state (36) of an intensity of the light signal (22).

5. The method according to any one of claims 1-4, wherein within the given time-dependent signal sequence (32, 32a, 32b) of the light signal (22) no repetition of a change of state (38) occurs.

6. The method according to any one of claims 1-5, wherein the detection device (18) has a rolling shutter effect (44).

7. The method according to any one of claims 1-6, wherein an image recording rate of the detection device (18) corresponds essentially to a rate of change of state of the time-dependent signal sequence (32, 32a, 32b) or wherein an image recording rate of the detection device (18) is greater than a rate of change of state of the time-dependent signal sequence (32, 32a, 32b).

8. The method according to any one of claims 1-6, wherein an image recording rate of the detection device (18) is less than a rate of change of state of the time-dependent signal sequence (32, 32a, 32b).

9. The method according to claim 7 or 8, wherein to determine the time offset At (42, 42a, 42b), a light signal section (24) is evaluated, which is detected in a plurality of image recordings (26).

10. The method according to claim 8, wherein to determine the time offset At (42, 42a, 42b), a light signal section (24) is evaluated, which is detected in an image (26) of the detection device (18).

11. The method according to any one of claims 1-10, wherein in a preceding method step at least one setting parameter, in particular be relevant to the detection device (18) and / or the light signal device (20) and / or the predetermined time-dependent signal sequence (32, 32a, 32b) , is transmitted from the main device (12) to the reference device (14) and / or from the reference device (14) to the main device (12), with an adaptation of the light signal device (20) and / or depending on the at least one transmitted setting parameter the detection device (18) and / or the predetermined time-dependent signal sequence (32, 32a, 32b) is carried out.

12. System (10) comprising at least one main device (12) and at least one reference device (14), wherein the main device (12) and the reference device (14) are set up to carry out a method according to any one of claims 1-11.

13. System (10) according to claim 12, wherein the light signal device (20) is implemented as at least one LED and / or a laser diode and / or a laser diode for generating a laser line and / or wherein the Detektionsvor device (20) as an optical Image sensor, in particular as an optical image sensor with a rolling shutter effect (44), in particular as a CMOS image sensor, is implemented.