DE102014205586A1 - Time of flight camera system - Google Patents

Abstract

Method for operating a time-of-flight camera system in which the lighting is operated in a modulated manner in a modulation measurement sequence and correlation values of an auto-correlation function are determined, the lighting not being operated in a reference measurement sequence and correlation values bi being determined only with the background light present. The two correlation values are calculated and an actual phase shift or a distance value is calculated from the corrected correlation values.

Description

The invention relates to a light transit time camera system and a method for operating such according to the preamble of the independent claims.

With the time of flight camera system, not only systems are to be included which determine distances directly from the light transit time, but in particular also all the time of flight or 3D TOF camera systems which acquire transit time information from the phase shift of an emitted and received radiation. In particular, PMD cameras with photonic mixer detectors (PMD) are suitable as the light transit time or 3D TOF cameras, as described, inter alia, in the applications EP 1 777 747 . US Pat. No. 6,587,186 and also DE 197 04 496 described and, for example, by the company 'ifm electronic GmbH' or 'PMD Technologies GmbH' as a frame grabber O3D or as CamCube relate. In particular, systems are also to be included under the time of flight camera system in which the light transit time sensor has only one pixel or a small number of pixels.

For determining a distance or a corresponding phase shift of the reflected light is, as in the DE 197 04 496 described in detail, mixed in the PMD sensor, the reflected light with the modulating signal. This mixture provides an in-phase signal (0 °) and a signal offset by 180 °, from which a distance can be determined in a known manner. To improve the quality of the distance measurement, it may be provided to shift the transmission modulation selectively by 90 °, 180 ° or 270 °, for example, and preferably to determine a phase angle of the reflected signal in relation to the transmitted signal by means of IQ (Inphase, Quadrature) demodulation. This approach is particularly useful for obtaining redundant information, for example, to compensate for various parasitic effects such as fixed pattern noise (FPN), backlight, or sensor asymmetries.

The object of the invention is to improve the distance measurement of a light transit time camera system.

The object is achieved in an advantageous manner by the inventive time of flight camera system and method according to the preamble of the independent claims.

Advantageously, a method is provided for operating a light transit time camera system with a light transit time sensor having at least one light transit time pixel, in which a distance received is correlated to a light received at the time of flight pixel with a modulation signal, which is used to drive an illumination, wherein for determining a phase shifts between the Modulation signal and received light signal correlation values are determined. In particular, it is provided to determine the correlation values both for a modulation measurement sequence in which the illumination is actively operated with the modulation signal and for a reference measurement sequence in which the illumination is inactive, the phase shifts and / or the distance values corrected correlation values are calculated, which result from a calculation of the correlation values determined in the modulation and reference measurement sequence.

This procedure makes it possible to adapt a photoelectric time camera to different ambient light situations during operation. While both the reflected modulated light and the background light are detected by the light transit time sensor or its pixels in the modulation measurement sequences, only the background light is detected in the reference measurement sequence. In both measurement sequences, the camera is electrically operated the same, in contrast, only the illumination in the reference measurement sequence is not actively operated. The electrical signals, and in particular the electrical quantities at the light transit time pixel, which are detected in this sequence can be used for the correction and, in particular, compensation of the variables determined in the modulation measurement sequence.

By virtue of this procedure, system-related deviations due to a given background light can advantageously be compensated.

In a preferred refinement, to determine the corrected correlation values, a difference is formed from the correlation values determined in the modulation and reference measurement sequence.

Furthermore, it is provided to determine the correlation values on the basis of the charges accumulated at the integration node of the light-propagation time pixel.

It is particularly advantageous if, in the modulation and reference measurement sequence, the modulation signal is shifted at least twice in the phase position and correlation values for each of these phase positions are determined in both measurement sequences.

The invention will be explained in more detail by means of embodiments with reference to the drawings.

Show it:

1 schematically the basic principle of photomix detection,

2 a modulated integration of the generated charge carriers,

3 two temporal courses of the charge integration with different phase positions,

4 Relation of integration in an IQ-diagram,

5 schematically influencing the phase position by background light,

6 a superposition of the phases in the phasor diagram.

In the following description of the preferred embodiments, like reference characters designate like or similar components.

1 shows a measurement situation for an optical distance measurement with a light time camera, as for example from the DE 197 04 496 is known.

The light transit time camera system 1 comprises a transmitting unit or a lighting module 10 with a lighting 12 and associated beam shaping optics 15 as well as a receiving unit or light runtime camera 20 with a receiving optics 25 and a light transit time sensor 22 ,

The light transit time sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is designed in particular as a PMD sensor. The receiving optics 25 typically consists of improving the imaging characteristics of multiple optical elements. The beam shaping optics 15 the transmitting unit 10 may be formed for example as a reflector or lens optics. In a very simple embodiment, if necessary, optical elements can also be dispensed with both on the receiving side and on the transmitting side.

The measuring principle of this arrangement is essentially based on the fact that, based on the phase shift of the emitted and received light, the transit time and thus the distance covered by the received light can be determined. For this purpose, the light source 12 and the light transit time sensor 22 via a modulator 30 together with a certain modulation signal M _o with a base phase position φ ₀ applied. In the example shown is also between the modulator 30 and the light source 12 a phase shifter 35 provided with the base phase φ _{0 of} the modulation signal M _{0 of} the light source 12 can be moved by defined phase positions φ _var . For typical phase measurements, phase positions of φ _var = 0 °, 90 °, 180 °, 270 ° are preferably used.

The light source transmits according to the set modulation signal 12 an intensity-modulated signal S _p1 with the first phase position p1 or p1 = φ ₀ + φ _var . This signal S _p1 or the electromagnetic radiation is in the illustrated case of an object 40 reflects and hits due to the distance traveled corresponding phase-shifted Δφ (t _L ) with a second phase position p2 = φ ₀ + φ _var + Δφ (t _L ) as a received signal S _p2 on the light transit time sensor 22 , In the time of flight sensor 22 the modulation signal M _{o is} mixed with the received signal S _p2 , wherein the phase shift or the object distance d is determined from the resulting signal.

To improve the measurement accuracy and / or to extend the uniqueness range, it is advantageous to perform the light transit time measurements with different modulation frequencies. For this purpose, the modulator 30 with a modulation control unit 38 connected, which can preferably specify within a predetermined frequency spectrum modulation frequencies.

The modulator 30 could for example be designed as a frequency synthesizer, via the modulation control unit 38 is controlled for the respective measuring task. It is also conceivable to switch between quartz oscillators with fixed frequencies.

Furthermore, the receiving unit 20 with an evaluation unit 27 connected. The evaluation unit 27 may also be part of the receiving unit 20 and in particular also part of the light transit time sensor 22 be. Task of the evaluation unit 27 It is to determine based on the received signals in relation to the modulation frequency phase shifts and / or evaluate. The mixture of the received light beams with the modulation frequency is preferably carried out in the light transit time sensor 22 or PMD sensor. Furthermore, the modulation control unit 38 also part of the evaluation unit 27 be. In particular, it may also be provided that the evaluation unit 27 the function of the modulation control unit 38 completely or partially takes over.

As illumination source or light source 12 are preferably infrared light emitting diodes. Of course, other radiation sources in other frequency ranges are conceivable, in particular, light sources in the visible frequency range are also considered.

As already in the DE 197 04 496 is described in detail, has a light transit time sensor 22 at least one light transit time pixel with a light-sensitive and light-insensitive area. The modulation photogates Gam, Gbm are arranged to be translucent in a photosensitive region adjacent to the accumulation gate Ga, Gb in adjacent light-insensitive regions. In accordance with the modulation signal M ₀ applied to the modulation gates Gam, Gbm, the photoelectrically generated charges q are directed either to one or the other accumulation gate or integration node Ga, Gb. The charges q collected there can then be tapped, for example, as voltage U. It should be noted that the accumulation gates can also be designed as diodes.

To determine a phase shift, the charge Δq or voltage difference ΔU between the two integration nodes Ga, Gb of a light transit time pixel is of particular interest. This difference may also be considered as a correlation value of a correlation or autocorrelation function, as will be described later.

In general, the integration nodes Ga, Gb can also be referred to as A and B readout channels, wherein the difference between the two channels, that is to say Δq or ΔU, can also be described as difference channel "A - B".

The basic principle of phase measurement is schematically in 2 shown. The upper curve shows the time profile of the modulation signal M ₀ with the illumination 12 and the light transit time sensor 22 be controlled. The object 40 Reflected light impinges on the light transit time sensor as received signal S _{p2 in} accordance with its light transit time t _L phase-shifted Δφ (t _L ) 22 , The light transit time sensor 22 collects the photonically generated charges q over several modulation periods in the phase position of the modulation signal M ₀ in a first accumulation gate Ga and in a 180 ° shifted phase position M ₀ + 180 ° in a second accumulation Gb. From the ratio of the charges qa, qb collected in the first and second gate Ga, Gb, the phase shift Δφ (t _L ) and thus a distance d of the object can be determined.

3a and 3b show curves of the charge difference Δq = q _a - q _b / (q _a + q _b ) as a function of the phase shift Δφ (t _L ) of the received light signal S _p2 with different phase angles. The 3a shows a curve for an unshifted modulation phase M ₀ with a phase angle φ _var = 0 °.

When the signal S _{p2 strikes} without a phase shift, ie Δφ (t _L ) = 0 °, for example when the transmission signal S _{p1 is directed} directly to the sensor, the phases of the modulation M ₀ and of the received signal S _{p2 are} identical that all generated charge carriers are detected synchronously at the first gate Ga and thus a maximum difference signal with Δq = 1 is applied.

As the phase shift increases, the charge on the first accumulation gate Ga decreases and on the second accumulation gate Gb. With a phase shift of Δφ (t _L ) = 90 °, the charge carriers qa, qb are equally distributed at both gates Ga, Gb and the difference is thus zero and after 180 ° phase shift "-1". With further increasing phase shift, the charge at the first gate Ga increases again, so that as a result the charge difference increases again in order then to reach a maximum again at 360 ° or 0 °.

Mathematically, this is a correlation of the received signal S _p2 with the modulating signal M ₀ according to the correlation function:

In the case of a modulation with a square-wave signal, as already described, a triangular function results as the correlation function. For a modulation with, for example, a sine signal, the result would be a cosine function.

As 3a shows, a phase phase measurement is unique only up to a phase shift Δφ (t _L ) ≤ 180 °.

For maximum detection of the phase shift of the IQ (in-phase quadrature) method is known for example in which two measurements are performed with shifted by 90 ° phase positions, that is, for example, with the phase position φ _var = 0 ° and φ _var = 90 °. The result of a measurement with the phase angle φ _var = 90 ° is in 3b shown.

The relationship of these two curves can be in a known manner, for example, for sinusoidal waveforms in an IQ diagram gem. 4 represent. In a first approximation, this representation is readily applicable to the triangular functions shown.

The phase angle can then be determined in a known manner via an arctan function: φ = arctan Δq (90 °) / Δq (0 °)

For example, to compensate for asymmetry of the sensor, additional 180 ° shifted phase measurements are carried out so that the result is the phase angle can be determined as follows. φ = arctan Δq (90 °) - Δq (270 °) / Δ (0 °) - Δq (180 °)

Generally speaking, the charge differences Δq (0 °), Δq (90 °), Δq (180 °), Δq (270 °) are the different values of correlation c ₁ , c ₂ , c ₃ , c ₄ Phase angles, namely 0 °, 90 °, 180 ° and 270 °, corresponding to autocorrelation.

The determination of the phase shift over the arctan relation is an example only one way of evaluation, of course, other calculation methods are conceivable. In particular, the procedure is not limited to two or four phase positions, but measurements with three phase positions or more than four different phase positions can also be carried out and evaluated.

From the in 2 shown runtime-dependent phase shift Δφ (t _L ) can be determined for object distances d, which are smaller than half the wavelength λ of the modulation frequency d ≤ λ / 2 in a known manner a distance. d = Δφ (t _L ) λ / 2π × 1/2

For distances d> λ / 2, there is generally no possibility of absolutely measuring the phase shift, so that the determined phase shift can no longer be unambiguously assigned to a distance value.

Phase and amplitude errors caused by unmodulated backlight and systematic phase-dependent modulation differences are adaptively corrected by a dark measurement or other background light measurement.

The phase, amplitude and distance calculations for ToF image sensors are made by evaluating several sequentially recorded phase images. The desired difference of the phase images is exclusively a well-defined phase shift from optical modulation to electrical modulation of the sensor. This allows the portions of the active optical modulation to be extracted from the overall signal because the unmodulated light (e.g., sunlight) provides identical proportions in all phase images.

In real camera systems, this ideal case often does not occur, so that the phase images also differ systematically from one another by the proportion of unmodulated light or other influences. Causes for this can be, for example, variations in the duty cycle of the modulation or deviating modulation levels. Without correction, this results in an error of all measured variables, which is dependent on the intensity of the background light and the useful signal and can, for example, as a so-called black-and-white error show, since the effect on dark and bright signal components is different. Weak signal levels are more affected by a superimposed interference due to weighting than strong signals.

The diagrams in the 5 show a simulation of a four-phase measurement under three different lighting conditions. The four measured values of the difference channel "A - B" are plotted analogously to the illustration in accordance with 3a , Amplitude, phase and offset then result from the fits of the measured values. Shown in the diagrams in each case as a sine-fit without further corrections.

5 (a) shows an autocorrelation function AKF only for the modulated light without being affected by a background light. In the example shown there is a phase shift of a = 30 °.

5 (b) shows an example of an autocorrelation function, which could only arise from background light. Although the backlight has no fixed phase relationship to the modulated light and is typically unmodulated, the backlight produces an apparent phase shift that affects the accuracy of the phase measurement of the modulated light. The apparent phase shift is essentially due to so-called imperfections in the system.

In a real measurement would result under these conditions as in 5 (c) show a mixed phase of (a) and (b). It applies to the measured values c1, c2, c3 and c4: c _i = a _i + b _i , i ε {1, 2, 3, 4}

The apparent phase shift is essentially a systematic error and can preferably be determined once by a suitable calibration for a system and stored in a characteristic table or lookup table LUT. If necessary, a correction function can also be calculated from this.

In measuring mode, only the intensity of the background light has to be determined, eg by a single phase measurement without modulated light. This now allows the noise component to be calculated and compensated for the other phases. For the upstream determination of the often linear relationship, a measurement series without an active, ie modulated signal is ideally carried out. The total signal may be, for example, a variation of an uncorrelated radiation, in particular a Blurred light or a variation of the integration time can be changed.

For example, a correction can be determined as follows:
Let I be the intensity of the background light. For a linear relationship of the values b1 to b4: b _i = F _i (I) = m _i I + o _i , i ε {1, 2, 3, 4} With the constants m _i and o _i , the unadulterated values a1 to a4 can be calculated by: a _i = c _i - b _i = c _i - F _i (I) = c _i - m _i I + o _i .

In a non-linear relationship, the function F _i (I) can be replaced by a characteristic table LUT _i (I).

A further possibility for determining the apparent phase shift would be conceivable as follows: In measurement mode, a measurement with N sequential phase images with optical modulation is followed by a further measurement with N phases, but without optical modulation, ie only with background light. N here represents the number of phase measurements for a range measurement, i.d.R. Four phase measurements are used.

If the second measurement in the camera is electrically identical to the first, one obtains directly the respective background light component of the phase images of the first measurement. The advantages of this method are that calibration with storage of the correction data is eliminated, and that non-linearities with much background light, e.g. can be compensated adaptively by non-linear characteristics, asymmetry of system components or time-variant sources of interference.

To calculate the apparent phase shift, the disturbance values b1 to b4 are measured directly. The unadulterated values a1 to a4 actual autocorrelation or phase shift result with the measured, disturbed values c1 to c4: a _i = c _i - b _i

In a modulation measurement sequence, the illumination is operated in a modulated manner and correlation values c _{i of} the autocorrelation function are determined. In a reference measurement sequence, the illumination is not operated and correlation values b _{i are determined} only with the existing background light. The two correlation values c _i , b _i are calculated and an actual phase shift or a distance value is calculated from the corrected correlation values a _i .

In a further preferred embodiment, the compensation of the interference components takes place without additional measurements in that the determination of the interference components takes place from the raw data of the actual distance measurement and is thus not associated with a reduction in the measurement rate. After a suitable calibration with combination of the unmodulated light component, as described in the first method, this is done in the simplest case with PMD sensors directly by calculating the total signal as the sum of the two output signals at each phase.

Furthermore, there are TOF sensor concepts in which it is attempted to reduce the influence of the background light by electronic circuits, so-called SBI circuits (SBI: suppression of background illumination) or by determining a difference signal between the two readout channels (A, B) , However, the information about the total amount of charge is basically lost.

Under real circumstances, however, the uncorrelated background signal is not completely compensated, so that, for example, an average over all raw data, the unmodulated amount of light can be estimated.

Alternatively, the determination of the total signal may, for example, also take place from a measurement of the time until the SBI becomes active during the integration phase or a parallel determination via photodiodes or neighboring pixels or alternative methods.

In the case of a linear SBI asymmetry, the following applies for the intensity I of the background light:

The rest of the calculation is analogous to the above-mentioned procedure.

Alternatively, the influence of the noise components on the phase measurements for all three methods described can also be corrected from the calculated distance and amplitude. Since unmodulated light in a naive conventional uncorrected calculation apparently produces an auto-correlation function (ACF) with phase and amplitude as desired by the modulated light, the effect is also referred to as a shamAKF. The calculated AKF during a measurement with background light and useful signal represents a superimposition of the two pointers of the useful signal AKF and ScheinAKF. The superimposition can be resolved by first determining the angle and amplitude of the apparent COF via the possibly reconstructed total signal. Subsequently, by means of calculated suitable trigonometric addition theorems of the pointer in phase and amplitude of the useful signal from the superimposed signal.

Phase and amplitude of measurements (a), (b) and (c) are in 6 represented as vectors a →, b → and c →. The length of the vectors corresponds to the amplitude, the phase corresponds to the angle of the vectors with the real part axis. c → results directly from a faulty measurement. Assuming that phase_b is constant and the ratio of amplitude_b to intensity I of the background light is known, b → can be completely reconstructed from I. The correction calculation to determine a → is then only: a → = c → - b →.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• EP 1777747 [0002]
• US 6587186 [0002]
• DE 19704496 [0002, 0003, 0021, 0030]

Claims (5)

Method for operating a light transit time camera system with a light transit time sensor ( 22 ) with at least one light-propagation time pixel, in which, for the purpose of distance determination, a light received at the light-propagation time pixel is provided with a modulation signal (M ₀ ) which is used to trigger a lighting ( 10 ) is determined, wherein to determine a phase shifts (Δφ (t _L )) between the modulation signal and received light signal correlation values (b _i , c _i , ΔU, Δq) are determined, characterized in that the correlation values ( b _i , c _i , ΔU, Δq) for a modulation measurement sequence in which the illumination is actively operated with the modulation signal, and for a reference measurement sequence in which the illumination is inactive, and the phase shifts (Δφ (t _L )) and / or the distance values (d) from corrected correlation values (a _i ) are calculated, which result from a calculation of the correlation values (c _i , b _i ) determined in the modulation and reference measurement sequence , The method of claim 1, wherein a difference for determining the corrected correlation values (a _i) from the determined in the modulation and reference measurement sequence correlation values (c _i, b _i) is formed. Method according to one of the preceding claims, in which the correlation values (b _i , c _i , ΔU, Δq) are determined on the basis of the charges (q) accumulated at integration nodes (Gam, Gbm) of the light-propagation time pixel. Method according to one of the preceding claims, wherein in the modulation and reference measurement sequence, the modulation signal (M ₀ ) at least twice in the phase position (φ ₀ + φ _var ) is shifted and correlation values (b _i , c _i , ΔU, Δ q) are determined for each of these phase positions (φ ₀ + φ _var ) in both measurement sequences. Time of flight camera system with illumination ( 10 ) and a light transit time sensor ( 22 ) having at least one light-propagation time pixel, wherein the light-propagating time camera system is adapted to perform one of the aforementioned methods.

Images