DE102018131580B4 - Time-of-flight distance measuring system and method for operating such a system - Google Patents

Abstract

Time-of-flight distance measuring system (28) with lighting (14) for emitting and a time-of-flight sensor (22) for receiving modulated light and with a modulator (30) for generating a modulation signal (M0', M0") for the lighting (14) and the Light transit time sensor (22) consisting of at least one base PN sequence, the base PN sequence being designed as a maximum sequence for selecting a distance range, and this base PN sequence (44) being in the form of sub-bit sequences (46, 48), which result from a substitution rule, is output to the lighting (14) and the time-of-flight sensor (22), with sub-bit sequences (46, 48) for both the lighting (14) and the time-of-flight sensor (22). ) are generated, in which a logical one of the base PN sequence (44) is represented by a first sub-bit sequence (46; 48) and a logical zero of the base PN sequence is represented by a second sub-bit sequence ( 48; 46) is substituted and the time-of-flight sensor (22) is designed as a photonic mixing element sensor (26) with modulation channels (A, B), characterized in that the sub-bit sequences (46) for the illumination (14) of each base PN sequence differ from the corresponding sub-bit sequences (48) for the time-of-flight sensor (22) in that the sub-bit sequences (48) for the photonic mixing element sensor (26) are designed in such a way that a substantially symmetrical distribution of charge carriers the modulation channels (A, B) are successful, and that the sub-bit sequences (46) for the lighting (14) are designed such that the number of ONE bits is smaller than the number of ZERO bits.

Description

The invention relates to a time-of-flight distance measuring system, in particular a time-of-flight camera system, with lighting for emitting and a time-of-flight sensor for receiving modulated light and with a modulator for generating a modulation signal for the lighting and the time-of-flight sensor from at least one basic PN sequence, this basic PN sequence. PN sequence in the form of sub-bit sequences, which result according to a substitution rule, is output to the lighting and the time-of-flight sensor. In this context, the term PN sequence means a pseudo-random binary sequence, where the abbreviation PN stands for pseudo noise.

The invention further relates to a corresponding method for operating such a time-of-flight distance measuring system as well as a corresponding computer program product and a corresponding use of the method.

The use of such pseudo-random binary sequences (PN sequences) in time-of-flight measuring systems is quite common and the substitution of an underlying basic PN sequence using sub-bit sequences is also well known in connection with such measuring systems. Due to their advantageous autocorrelation properties have decisive advantages over conventional modulation sequences in the form of simply periodic rectangular or sinusoidal signal sequences.

However, direct or unchanged use of these known PN sequences for lighting and sensor modulation is unsuitable for certain light transit time measurement methods due to the unequal number of logical “zeros” and “ones”. There are negative effects on the measurement system performance, especially with regard to noise behavior and resistance to ambient light.

From the DE 10 2014 210 750 B3 For example, a time-of-flight camera system is known in which a PN word for modulating lighting and a time-of-flight sensor is specified based on a PN base word and a substitution rule, with bit states of the PN base word being substituted by sub-bit sequences.

The DE 600 09 565 T2 discloses a LIDAR distance measuring system in which, in order to avoid ambiguity when measuring large distances, a coarse and a fine cross-correlation is carried out, with PN signal sequences of maximum length (MLS) preferably being used.

The DE 3 616 950 A1 deals with a noise radar in which a pseudo-random sequence, preferably larger than 10,000 bits, is repeatedly generated and used to modulate the phase of an emitted signal in order, for example, to avoid interference or external detection of the signals.

The DE 19 704 496 A1 describes the basic principle of phase detection using photonic mixing elements. Among other things, an embodiment is also proposed in which the modulation takes place with a pseudo-noise signal.

A time-of-flight distance measuring system of the type mentioned at the beginning is, for example, a time-of-flight camera system from the patent DE 10 2014 210 750 B3 known. This time-of-flight camera system has lighting for emitting modulated light and a time-of-flight camera based on the photomixing element principle for receiving modulated light, as well as a modulator for generating a modulation signal for the lighting and the time-of-flight camera from at least one basic PN sequence. The basic PN sequence is output to the lighting and the time-of-flight camera in the form of sub-bit sequences, which result from a substitution rule.

The object of the invention is to provide measures that make it possible to use PN sequences for other time-of-flight measurement systems.

The object is achieved according to the invention by the features of the independent claims.

In the light transit time distance measuring system according to the invention with the features mentioned in the preamble of claim 1, it is provided that the sub-bit sequences for the illumination of each basic PN sequence differ from the corresponding sub-bit sequences for the light transit time sensor. The time-of-flight distance measuring system is preferably a time-of-flight camera system, the time-of-flight sensor of which is designed to produce images, i.e. as a time-of-flight camera.

The object is achieved according to the invention in that the sequence resulting from the substitution for the time-of-flight sensor can be selected at least to a certain extent independently of the sequence resulting from the substitution for the lighting. This makes it possible to optimize the two sequences according to different criteria.

According to a preferred embodiment of the invention, sub-bit sequences are generated for both the lighting and the time-of-flight sensor, in which a logical one of the base PN sequence is replaced by a first sub-bit sequence and a logical zero of the base PN sequence is substituted by a second sub-bit sequence. At least one of the two sub-bit sequences differs for the lighting and the time-of-flight sensor.

According to a further preferred embodiment of the invention, the time of flight sensor is designed as a photonic mixing element sensor with modulation channels. PN sequences (pseudo-random binary sequences) for modulating a depth image sensor, which works according to the principle of the photomixing element or “Photonic Mixer Device” (PMD), offer decisive advantages over conventional modulation sequences in the form of simply periodic rectangular or sinusoidal signal sequences. However, direct or unchanged use of these sequences for sensor modulation is unsuitable due to the uneven distribution of logical “zeros” and “ones” and the resulting uneven distribution of charge carriers on the two modulation channels (gates) of the PMD depth image sensor. There are negative effects on camera performance, especially with regard to noise behavior and resistance to extraneous light, since with asymmetrical modulation, background light is distributed unequally across the two modulation channels (gates) and the distance measurement value is therefore falsified. This in turn results in a reduction in the dynamic range. It is therefore provided for such a photonic mixing element sensor that the sub-bit sequences are designed in such a way that a substantially symmetrical distribution of charge carriers across the modulation channels occurs.

In the method according to the invention with the features mentioned in the preamble of claim 5, it is provided that the sub-bit sequences for the illumination of each basic PN sequence differ from the corresponding sub-bit sequences for the time-of-flight sensor.

According to a preferred embodiment of the method according to the invention, a logical one of the base PN sequence is represented by a first sub-bit sequence and a logical zero of the base PN sequence is represented by a second sub-bit for both the lighting and the time-of-flight sensor -Sequence substituted, whereby in the sequence for the sensor these two sub-bit sequences are inverse to each other.

According to yet another preferred embodiment of the method according to the invention, several individual measurements are carried out with a respective PN sequence for a light transit time distance measurement within a defined distance measurement range, the sub-bit sequences being varied.

It is advantageously provided that the sub-bit sequences are varied by shifting or inverting the sub-bit sequences.

The invention further relates to a computer program product comprising program parts which are loaded into a processor of a computer-based system control and are set up to carry out the method mentioned above.

1. (i) Streulicht, welches durch optische Komponenten des Systems eingebracht wird,
2. (ii) eines als Multi Path Interference bekannten Effekts der Mehrwegeausbreitung bei der Messung und
3. (iii) eines Hintergrund-Signals bei Gesichtserkennungs-Anwendungen.

Finally, the invention also relates to the use of the above-mentioned method for avoiding or at least reducing at least one disruptive influence during operation or when using the time-of-flight distance measuring system from the following list:

1. (i) scattered light, which is introduced by optical components of the system,
2. (ii) an effect of multipath propagation in measurement known as Multi Path Interference and
3. (iii) a background signal in facial recognition applications.

The invention is explained in more detail below using exemplary embodiments with reference to the drawings.

• 1 eine schematische Darstellung eines als Lichtlaufzeitkamerasystem ausgebildeten Lichtlaufzeit-Entfernungsmesssystems gemäß einer Ausführungsform der Erfindung,
• 2 eine modulierte Integration der erzeugten Ladungsträger,
• 3 einen Schnitt durch ein Pixel eines als photonischer Mischelemente-Sensor ausgebildeten Lichtlaufzeitsensors des Lichtlaufzeitkamerasystems,
• 4 eine Prinzipdarstellung der Basis PN-Folge (oben), der resultierenden Folge für eine Beleuchtung des Lichtlaufzeitkamerasystems (Mitte) und der resultierenden Folge für einen der Modulationskanäle des photonischen Mischelemente-Sensors (unten),
• 5 resultierende Korrelationsfunktionen mehrerer Einzelmessungen bei denen eine Modulationssequenz der Beleuchtung gegenüber einer entsprechenden Modulationssequenz des Lichtlaufzeitsensors gemäß einer ersten Vorschrift variiert werden und
• 6 resultierende Korrelationsfunktionen mehrerer Einzelmessungen bei denen eine Modulationssequenz der Beleuchtung gegenüber einer entsprechenden Modulationssequenz des Lichtlaufzeitsensors gemäß einer zweiten Vorschrift variiert werden.

Show it:

• 1 a schematic representation of a time-of-flight distance measuring system designed as a time-of-flight camera system according to an embodiment of the invention,
• 2 a modulated integration of the generated charge carriers,
• 3 a section through a pixel of a time-of-flight sensor of the time-of-flight camera system designed as a photonic mixing element sensor,
• 4 a schematic representation of the basic PN sequence (top), the resulting sequence for illumination of the time-of-flight camera system (middle) and the resulting sequence for one of the modulation channels of the photonic mixing element sensor (bottom),
• 5 resulting correlation functions of several individual measurements in which a modulation sequence of the lighting is varied compared to a corresponding modulation sequence of the time-of-flight sensor according to a first rule and
• 6 resulting correlation functions of several individual measurements in which a modulation sequence of the lighting is varied compared to a corresponding modulation sequence of the time-of-flight sensor according to a second rule.

The 1 shows a schematic representation of a time-of-flight camera system 10. The time-of-flight camera system 10 includes a transmitting unit or an illumination module 12 with lighting 14 and associated beam shaping optics 16 as well as a receiving unit or time-of-flight camera 18 with a receiving optics 20 and a time-of-flight sensor 22. The time-of-flight sensor 22 has a pixel -Array of a large number of light travel time pixels 24 (one of which is in 3 is explicitly shown) and is designed in the example as a photonic mixing element sensor 26, also called a PMD sensor. If the light guide sensor 22 is not imaging, the time of flight sensor 22 can also have a few or even just a single time of flight pixel 24. In this context, one no longer speaks of a time-of-flight camera system 10 but generally of a time-of-flight distance measuring system 28. The receiving optics 20 typically consist of several optical elements to improve the imaging properties. The beam shaping optics 16 of the lighting module 12 can be designed, for example, as a reflector or lens optics. In a very simple embodiment, optical elements can also be dispensed with on both the receiving and transmitting sides.

The measuring principle of this arrangement is essentially based on the fact that the transit time and thus the distance traveled by the received light can be determined based on the phase shift of the emitted and received light. For this purpose, the lighting 14 and the time-of-flight sensor 22 are supplied with modulation signals M ₀₁ , M ₀₂ via a modulator 30, which are based on common basic PN sequences.

In the example shown, a phase shifter 32 is also provided between the modulator 28 and the lighting 14, with which the base phase φ ₀ of the modulation signal M ₀₁ of the light source 12 can be shifted by defined phase positions φ _var . For typical phase measurements, phase positions of φ _var = 0°, 90°, 180°, 270° are preferably used.

According to the set modulation signal, the light source 12 sends out an intensity-modulated signal Sp ₁ with the first phase position P ₁ or P ₁ = φ ₀ + φ _var . In the case shown, this signal Sp ₁ or the electromagnetic radiation is reflected by an object 34 and, due to the distance traveled, arrives correspondingly phase-shifted Δφ(t _L ) with a second phase position p ₂ = φ ₀ + φvar + Δφ(t _L ) as a received signal Sp ₂ on the time-of-flight sensor 22. In the time-of-flight sensor 22, the modulation signal M ₀ is mixed with the received signal Sp ₂ , the phase shift or the object distance d being determined from the resulting signal.

Furthermore, a modulation control device 36 is provided, with which the shape and in particular the pulse and pause ratios of the modulation signal are specified. The phase shifter 32 can also be controlled via the modulation control device 36 depending on the measurement task to be carried out.

Infrared light-emitting diodes and laser diodes are preferably suitable as the lighting or light source 12. Of course, other radiation sources in other frequency ranges are also conceivable; in particular, light sources in the visible frequency range also come into consideration.

The basic principle of phase measurement is shown schematically in 2 shown. The upper curve shows the time profile of the basic modulation signal M ₀ with which the lighting 14 and the time-of-flight sensor 22 are controlled. The light reflected from the object 32 hits the light transit time sensor 22 as a received signal Sp ₂ in accordance with its light transit time t _L with a phase shift Δφ(t _L ).

The 3 shows the cross section of a single pixel 24 of a photonic mixing element sensor 26 using the example of a CCD structure. In addition to the pixel 24, the photonic mixing element includes the structures necessary for the power supply and the signal derivations. The outer gates Gsep only serve to electrically delimit this pixel 24 from neighboring structures.

In the 3 The embodiment shown is carried out on a p-doped silicon substrate 38. The modulation gates Ga, Gb of the two modulation channels A, B represent the light-sensitive part 40 and are in the inversion state. In addition to a positive bias voltage U ₀ on the conductive but optically partially transparent upper cover, for example made of poly-silicon, they are operated with the superimposed push-pull voltages Um(t). The resulting modulation voltages multiplicatively cause a separation of the minority charge carriers generated by the photons of the incident light in the space charge zone immediately below the insulator layer 42, for example made of silicon oxide or silicon nitride. These charge carriers (electrons in the example) drift under the influence of the modulating push-pull voltage to the closely adjacent accumulation gates Ga or Gb and are integrated there while the majority charge carriers or holes flow to the ground connection of the p-Si substrate.

In other words, the time-of-flight pixel 24 of the time-of-flight sensor 22 typically has a first and second accumulation gate Ga, Gb, in which the photonically generated charges q are collected alternately over several modulation periods depending on the potential profile in the light-sensitive area. The charges q generated in the non-shifted phase position are collected in the first accumulation gate Ga and those in the 180° shifted phase position M ₀ + 180° in the second accumulation gate Gb. The phase shift Δϕ(t _L ) and thus a distance d of the object can be determined from the ratio of the charges qa, qb collected in the first and second gates Ga, Gb.

Basically, the number of logical ones in a pseudo-random binary sequence called a maximum sequence (MLS) is one greater than the number of logical zeros, each having an even number of logical ones and the odd number of logical zeros. When using an MLS directly for the modulation of a PMD depth image sensor 26, this fundamentally leads to an uneven charge carrier distribution on the two modulation gates Ga, Gb or modulation channels A, B. In order to achieve a symmetrical charge carrier distribution and thus MLS for the modulation of a PMD depth image sensor To make 26 usable, the number of ones and zeros must be identical to realize the pulse-shaped autocorrelation property.

This compensation is now brought about by an adequate substitution of the individual bits of the basic MLS.

• Ausgehend von einer Basis-PN-Folge, die man auch als Basis-Modulationssignal M₀ auffassen kann, wird ein Modulationssignal M₀' für die Beleuchtung 14 und ein Modulationssignal M₀" für den Lichtlaufzeitsensor 22 in Form von Sub-Bit-Folgen generiert, wobei sich die Sub-Bitfolgen für die Beleuchtung 14 einer jeden Basis-PN-Folge von den entsprechenden Sub-Bitfolgen für den Lichtlaufzeitsensor 22 unterscheidet.

The following procedure now results with regard to the resulting modulation signals M ₀ ', M ₀ '' for the lighting 14 and the time-of-flight sensor 22:

• Starting from a basic PN sequence, which can also be interpreted as a basic modulation signal M ₀ , a modulation signal M ₀ ' for the lighting 14 and a modulation signal M ₀ " for the time-of-flight sensor 22 are generated in the form of sub-bit sequences , whereby the sub-bit sequences for the illumination 14 of each base PN sequence differ from the corresponding sub-bit sequences for the time-of-flight sensor 22.

Table 1 shows a particularly simple scheme for a system 28 in which the time-of-flight sensor 22 is designed as a photonic mixing element sensor 26. In this example, each bit of the base PN sequence in the modulation signals is replaced by a 2-bit word. Basic PN bit Exemplary substitution Lighting modulation PMD modulation (channel A) PMD modulation (channel B) 0 00 10 0 1 1 0 1 0 1 10

Since each basic PN sequence has at least one zero, in which the lighting modulation differs from the PMD modulation of each of the modulation channels A, B, the sub-bit sequences for the lighting 14 of each basic PN sequence necessarily differ from the corresponding sub-bit sequences for the light transit time sensor 22, designed here as a PMD sensor 26.

The substitution is of course not limited to 2-bit words, but can also have more than 2-bits, in particular 4-bits, as shown as an example in the table below. Basic PN bit Exemplary substitution Lighting modulation PMD modulation (channel A) PMD modulation (channel B) 0 0000 1100 0011 1 1000 0011 1100

This substitution is characterized by the fact that two ONE bits are provided in the lighting modulation and only a single ONE bit in the lighting modulation. The position of the set ONE bit can be chosen arbitrarily.

The sub-bit sequences advantageously have an even number of bits. The number of ONE and ZERO bits is the same for sensor or PMD modulation. For the lighting modulation, however, it is preferred to select the number of ONE bits smaller than the number of ZERO bits and in particular smaller than the number of ONE bits of the sensor or PMD modulation.

The 4 is divided into three sections. Above is a schematic representation of the base PN sequence 44, in the middle is a schematic representation of a resulting sub-bit sequence 46 for the illumination 14 of the time-of-flight camera system 10 and below is a schematic representation of the resulting sub-bit sequence 48 for one of the modulation channels A of the photonic mixing element sensor 26 shown. In the substitution of the base PN bits that is the basis here, each of these base PN bits is substituted by a 4-bit word.

How well in the illustrations of the 4 can be seen, the signals for the sensor modulation (bottom) and the lighting modulation (middle) differ significantly.

The 5 and 6 show resulting correlation functions (50, 52, 54, 56) of four individual measurements in which a modulation sequence of the lighting 14 is varied compared to a corresponding modulation sequence of the time-of-flight sensor 22 according to a first rule or a second rule. The amplitude I is plotted against the distance d between system 10, 28 and object 34.

In the 5 and 6 the correlation functions between optical modulation signal M ₀ ' and the modulation signal M ₀ ", with which the PMD depth image sensor 26 is modulated, are shown for two exemplary modulation schemes. In the first example modulation ( 5 ), the illumination modulation sequence was shifted four times successively by one pulse each time compared to the modulation sequence of the PMD depth image sensor 26. In the second example modulation ( 6 ), on the other hand, with two correlation functions, the modulation signal of the PMD depth image sensor 26 was inverted and the lighting modulation sequence was only shifted by up to one pulse.

It can be seen that a successive relative shift of the modulation sequences relative to one another leads to a successive shift of the selective distance measurement range. In the present example, a relative shift of the two modulation sequences by one pulse each corresponds to one Shifting the selected distance measuring range by 0.75 m. Furthermore, it can be seen that, in accordance with the autocorrelation property of MLS, the AKF only delivers a signal within the selected distance measuring range, while outside of this range it assumes the value ZERO.

On the one hand, the pulse-shaped autocorrelation function of a maximum sequence offers the possibility of precisely limiting the desired distance measurement range so that objects that are outside this selected distance range are not detected. On the other hand, the area in which a clear distance measurement is possible (unique range) can be expanded practically arbitrarily by varying the length of an MLS.

• Mehrzielfähigkeit, beispielsweise zur Separation von Vorder- und Hintergrund einer Szene.

This “fading property” offers a variety of new possibilities and advantages over conventional modulation schemes, in particular:

• Multi-target capability, for example to separate the foreground and background of a scene.

Reduction of mixed phases, which can be attributed to multipath interference or scattered light, for example. By appropriately configuring the modulation sequences, scattered light can even be almost completely eliminated.

Reference symbols

10

Time of flight camera system

12

Lighting module

14

lighting

16

Beam shaping optics

18

Time of flight camera

20

receiving optics

22

Time of flight sensor

24

Light travel time pixels

26

Photonic mixed element sensor

28

Time of flight distance measuring system

30

modulator

32

Phase shifter

34

object

36

Modulation control unit

38

Silicon substrate

40

light sensitive part

42

insulator layer

44

Basic PN sequence

46

Sub-bit sequences for lighting

48

Sub-bit sequences for the sensor

50

first correlation function

52

second correlation function

54

third correlation function

56

fourth correlation function

d

Distance

A

first modulation channel

b

second modulation channel

Δφ(tL)

Transit time-related phase shift

φvar

Phase position

φ0

Basic phase

Ga

modulation gate

Gb

modulation gate

Gaa, Gba, Gsep

more gates

Claims (8)

Time-of-flight distance measuring system (28) with lighting (14) for emitting and a time-of-flight sensor (22) for receiving modulated light and with a modulator (30) for generating a modulation signal (M ₀ ', M ₀ ") for the lighting (14) and the time of flight sensor (22) from at least one base PN sequence, wherein the base PN sequence is designed as a maximum sequence for selecting a distance range, and wherein this base PN sequence (44) is in the form of sub-bit sequences ( 46, 48), which result from a substitution rule, is output to the lighting (14) and the time-of-flight sensor (22), with sub-bit sequences (46 , 48) are generated, in which a logical one of the base PN sequence (44) is represented by a first sub-bit sequence (46; 48) and a logical zero of the base PN sequence is represented by a second sub-bit sequence. Sequence (48; 46) is substituted and the time-of-flight sensor (22) is designed as a photonic mixing element sensor (26) with modulation channels (A, B), characterized in that the sub-bit sequences (46) for the lighting (14) of each basic PN sequence from the corresponding sub-bit sequences (48) for the time-of-flight sensor (22), that the sub-bit sequences (48) for the photonic mixing element sensor (26) are designed such that a substantially symmetrical Distribution of charge carriers to the modulation channels (A, B) is successful, and that the sub-bit sequences (46) for the lighting (14) are designed such that the number of ONE bits is smaller than the number of ZERO bits. Time of flight distance measuring system Claim 1 , in which the sub-bit sequences (46, 48) have an even number of bits. Method for operating a time-of-flight distance measuring system (28) having lighting (14) and a time-of-flight sensor (22), in particular a time-of-flight camera system (10), in which starting from a base PN sequence (44), which is used as a maximum sequence for selection a distance range, a modulation signal (M ₀ ', M ₀ ") for the lighting (14) and the time-of-flight sensor (22) is generated in the form of sub-bit sequences, both for the lighting (14) and for the time of flight sensor (22) a logical one of the base PN sequence (44) is substituted by a first sub-bit sequence and a logical zero of the base PN sequence is substituted by a second sub-bit sequence, with the sequence for the time-of-flight sensor (22), the two sub-bit sequences are inverse to one another, characterized in that the sub-bit sequences (46) for the illumination (14) of each base PN sequence (44) differ from the corresponding sub-bit sequences. Bit sequences (48) for the time-of-flight sensor (22) differ in that the sub-bit sequences (48) for the photonic mixing element sensor (26) are designed in such a way that there is a substantially symmetrical distribution of charge carriers across the modulation channels (A, B). , and that the sub-bit sequences (46) for the lighting (14) are designed such that the number of ONE bits is smaller than the number of ZERO bits. Procedure according to Claim 3 , characterized in that for a light transit time distance measurement within a defined distance measurement range, several individual measurements are carried out with a respective PN sequence, in which the sub-bit sequences (46, 48) are varied. Procedure according to Claim 4 , characterized in that the sub-bit sequences (46, 48) are varied by shifting or inverting the sub-bit sequences (46, 48). Procedure according to one of the Claims 3 until 5 , in which the sub-bit sequences (46, 48) have an even number of bits. Computer program product comprising program parts that are loaded into a processor of a computer-based system control for carrying out the method according to one of the Claims 3 until 6 are set up. Using the method according to one of the Claims 3 until 6 to avoid or at least reduce at least one disruptive influence from the following list when operating the time-of-flight distance measuring system (28): - scattered light, which is introduced by optical components of the system (28), - an effect of multi-path propagation known as multi-path interference Measurement of a background signal in facial recognition applications.

Images