DE102014205585B4 - Method for operating a time of flight camera and time of flight camera system - Google Patents

Abstract

Method for operating a photoflash camera comprising the steps of: - operating the illumination without modulation to emit an unmodulated light in a calibration step, - emitting an unmodulated light with different light intensities - detecting an apparent phase shift and / or autocorrelation function and / or correlation values for the different light intensities, - Deposit of the apparent phase shift and / or autocorrelation function and / or correlation values as a table of values or correction function in a memory.

Description

The invention relates to a light transit time camera system and a method for operating such according to the preamble of the independent claims.

With the time of flight camera system, not only systems should be included which determine distances directly from the light transit time, but in particular also all the time of flight or 3D TOF camera systems which acquire transit time information from the phase shift of an emitted and received radiation. In particular, PMD cameras with photonic mixer detectors (PMD) are suitable as the light transit time or 3D TOF cameras, as described, inter alia, in the applications EP 1 777 747 A1 . US Pat. No. 6,587,186 B2 and also DE 197 04 496 A1 described and, for example, by the company 'ifm electronic GmbH' or 'PMD Technologies GmbH' as a frame grabber O3D or as CamCube relate. In particular, the PMD camera allows a flexible arrangement of the light source and the detector, which can be arranged both in a housing and separately. In particular, systems are also to be included under the time of flight camera system in which the light transit time sensor has only one pixel or a small number of pixels.

For determining a distance or a corresponding phase shift of the reflected light is, as in the DE 197 04 496 A1 described in detail, mixed in the PMD sensor, the reflected light with the modulating signal. This mixture provides an in-phase signal (0 °) and a signal offset by 180 °, from which a distance can be determined in a known manner. To improve the quality of the distance measurement, it may be provided to shift the transmission modulation selectively by 90 °, 180 ° or 270 °, for example, and preferably to determine a phase angle of the reflected signal in relation to the transmitted signal by means of IQ (Inphase, Quadrature) demodulation. This approach is particularly useful for obtaining redundant information, for example, to compensate for various parasitic effects such as fixed pattern noise (FPN), backlight, or sensor asymmetries.

From the WO 2011/042 290 A1 Furthermore, a distance measuring device in the form of a photon counter SPAD with a calibration device is known in which a distance is determined by transmitting and receiving a periodically modulated optical measuring radiation. In a calibration measurement, it is intended to use uncorrelated radiation, for example also in the form of normal ambient light. After a certain lighting time or measuring time, counter readings result which are directly proportional to the effective bin widths and can be calibrated accordingly.

From the DE 10 2005 045 555 A1 a method for measuring error correction of non-linear systems is known in which the measurement error correction of an optical radar sensor, the non-linear properties of the measuring system are stored in a look-up table and used to correct the measurement results.

The object of the invention is to improve the distance measurement of a light transit time camera system.

The object is achieved in an advantageous manner by the inventive time of flight camera system and method according to the preamble of the independent claims.

• – Betreiben der Beleuchtung ohne Modulation zur Aussendung eines unmodulierten Lichts in einem Kalibrierschritt,
• – Aussendung eines unmodulierten Lichts bzw. Gleichlichts mit unterschiedlichen Lichtstärken
• – Erfassung einer scheinbaren Phasenverschiebung und/oder Autokorrelationsfunktion und/oder Korrelationswerte für die unterschiedlichen Lichtstärken,
• – Hinterlegung der scheinbaren Phasenverschiebung, Autokorrelationsfunktion und/oder Korrelationswerte als Wertetabelle oder Korrekturfunktion in einem Speicher.

Advantageously, a method is provided for operating a time of flight camera with the following steps:

• Operating the illumination without modulation to emit an unmodulated light in a calibration step,
• - Emission of unmodulated light or direct light with different light levels
• Detecting an apparent phase shift and / or autocorrelation function and / or correlation values for the different light intensities,
• - Deposit of the apparent phase shift, autocorrelation function and / or correlation values as a table of values or correction function in a memory.

In addition, it is provided to correct the distance values determined during a distance measurement on the basis of the stored value table or correction function with such a calibrated time of flight camera.

By this procedure, system-related deviations due to a present background light are advantageously compensated.

The invention will be explained in more detail by means of embodiments with reference to the drawings.

Show it:

1 schematically the basic principle of photomix detection,

2 a modulated integration of the generated charge carriers,

3 two temporal courses of the charge integration with different phase positions,

4 Relation of integration in an IQ-diagram,

5 schematically influencing the phase position by background light,

6 a superposition of the phases in the phasor diagram.

In the following description of the preferred embodiments, like reference characters designate like or similar components.

1 shows a measurement situation for an optical distance measurement with a light time camera, as for example from the DE 197 04 496 A1 is known.

The light transit time camera system 1 comprises a transmitting unit or a lighting module 10 with a lighting 12 and associated beam shaping optics 15 as well as a receiving unit or light runtime camera 20 with a receiving optics 25 and a light transit time sensor 22 ,

The light transit time sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is designed in particular as a PMD sensor. The receiving optics 25 typically consists of improving the imaging characteristics of multiple optical elements. The beam shaping optics 15 the transmitting unit 10 may be formed for example as a reflector or lens optics. In a very simple embodiment, if necessary, optical elements can also be dispensed with both on the receiving side and on the transmitting side.

The measuring principle of this arrangement is essentially based on the fact that, based on the phase shift of the emitted and received light, the transit time and thus the distance covered by the received light can be determined. For this purpose, the light source 12 and the light transit time sensor 22 via a modulator 30 together with a certain modulation signal M _o with a base phase position φ ₀ applied. In the example shown is also between the modulator 30 and the light source 12 a phase shifter 35 provided with which the base phase φ ₀ of the modulation signal M ₀ of the light source 12 can be moved by defined phase positions φ _var . For typical phase measurements, phase positions of φ _var = 0 °, 90 °, 180 °, 270 ° are preferably used. The light source transmits according to the set modulation signal 12 an intensity-modulated signal S _p1 with the first phase position p1 or p1 = φ ₀ + φ _var . This signal S _p1 or the electromagnetic radiation is in the illustrated case of an object 40 reflects and hits due to the distance traveled corresponding phase-shifted Δφ (t _L ) with a second phase position p2 = φ ₀ + φ _var + Δφ (t _L ) as a received signal S _p2 on the light transit time sensor 22 , In the time of flight sensor 22 the modulation signal M _{o is} mixed with the received signal S _p2 , wherein the phase shift or the object distance d is determined from the resulting signal.

To improve the measurement accuracy and / or to extend the uniqueness range, it is advantageous to perform the light transit time measurements with different modulation frequencies. For this purpose, the modulator 30 with a modulation control unit 38 connected, which can preferably specify within a predetermined frequency spectrum modulation frequencies.

The modulator 30 could for example be designed as a frequency synthesizer, via the modulation control unit 38 is controlled for the respective measuring task. It is also conceivable to switch between quartz oscillators with fixed frequencies.

Furthermore, the receiving unit 20 with an evaluation unit 27 connected. The evaluation unit 27 may also be part of the receiving unit 20 and in particular also part of the light transit time sensor 22 be. Task of the evaluation unit 27 It is to determine based on the received signals in relation to the modulation frequency phase shifts and / or evaluate. The mixture of the received light beams with the modulation frequency is preferably carried out in the light transit time sensor 22 or PMD sensor. Furthermore, the modulation control unit 38 also part of the evaluation unit 27 be. In particular, it may also be provided that the evaluation unit 27 the function of the modulation control unit 38 completely or partially takes over.

As illumination source or light source 12 are preferably infrared light emitting diodes. Of course, other radiation sources in other frequency ranges are conceivable, in particular, light sources in the visible frequency range are also considered.

As already in the DE 197 04 496 A1 is described in detail, has a light transit time sensor 22 at least one light transit time pixel with a light-sensitive and light-insensitive area. The modulation photogates Gam, Gbm are translucently arranged in a photosensitive region at the accumulation gate Ga, Gb in adjacent light-insensitive ones Areas adjacent. In accordance with the modulation signal M ₀ applied to the modulation gates Gam, Gbm, the photoelectrically generated charges q are directed either to one or the other accumulation gate or integration node Ga, Gb. The charges q collected there can then be tapped, for example, as voltage U. It should be noted that the accumulation gates can also be designed as diodes.

To determine a phase shift, the charge Δq or voltage difference ΔU between the two integration nodes Ga, Gb of a light transit time pixel is of particular interest. This difference may also be considered as a correlation value of a correlation or autocorrelation function, as will be described later.

In general, the integration nodes Ga, Gb can also be referred to as A and B readout channels, wherein the difference between the two channels, that is to say Δq or ΔU, can also be described as difference channel "A - B".

The basic principle of phase measurement is schematically in 2 shown. The upper curve shows the time profile of the modulation signal M ₀ with the illumination 12 and the light transit time sensor 22 be controlled. The object 40 Reflected light impinges on the light transit time sensor as received signal S _{p2 in} accordance with its light transit time t _L phase-shifted Δφ (t _L ) 22 , The light transit time sensor 22 collects the photonically generated charges q over several modulation periods in the phase position of the modulation signal M ₀ in a first accumulation gate Ga and in a 180 ° shifted phase position M ₀ + 180 ° in a second accumulation Gb. From the ratio of the charges qa, qb collected in the first and second gate Ga, Gb, the phase shift Δφ (t _L ) and thus a distance d of the object can be determined.

3a and 3b show curves of the charge difference Δq = q _a - q _b / (q _a + q _b ) as a function of the phase shift Δφ (t _L ) of the received light signal S _p2 with different phase angles. The 3a shows a curve for an unshifted modulation phase M ₀ with a phase angle φ _var = 0 °.

When the signal S _{p2 strikes} without a phase shift, ie Δφ (t _L ) = 0 °, for example when the transmission signal S _{p1 is directed} directly to the sensor, the phases of the modulation M ₀ and of the received signal S _{p2 are} identical, so that all generated charge carriers are detected synchronously at the first gate Ga and thus a maximum difference signal with Δq = 1 is applied.

As the phase shift increases, the charge on the first accumulation gate Ga decreases and on the second accumulation gate Gb. With a phase shift of Δφ (t _L ) = 90 °, the charge carriers qa, qb are equally distributed at both gates Ga, Gb and the difference is thus zero and after 180 ° phase shift "-1". With further increasing phase shift, the charge at the first gate Ga increases again, so that as a result the charge difference increases again in order then to reach a maximum again at 360 ° or 0 °.

Mathematically, this is a correlation function of the received signal S _p2 with the modulating signal M ₀ .

In the case of a modulation with a square-wave signal, as already described, a triangular function results as the correlation function. For a modulation with, for example, a sine signal, the result would be a cosine function.

As 3a shows, a phase phase measurement is unique only up to a phase shift Δφ (t _L ) ≤ 180 °.

For maximum detection of the phase shift, for example, the IQ (in-phase quadrature) method is known in which two measurements are performed with shifted by 90 ° phase angles, so for example with the phase φ _var = 0 ° and φ _var = 90 °. The result of a measurement with the phase angle φ _var = 90 ° is in 3b shown.

The relationship of these two curves can be in a known manner, for example, for sinusoidal waveforms in an IQ diagram gem. 4 represent. In a first approximation, this representation is readily applicable to the triangular functions shown.

The phase angle can then be determined in a known manner via an arctan function: φ = arctan Δq (90 °) / Δq (0 °) In order to compensate, for example, for asymmetry of the sensor, additional phase measurements shifted by 180 ° can be performed so that, as a result, the phase angle can be determined as follows. φ = arctan Δq (90 °) - Δq (270 °) / Δ (0 °) - Δq (180 °)

Generally speaking, the charge differences Δq (0 °), Δq (90 °), Δq (180 °), Δq (270 °) about correlation values c ₁ , c ₂ , c ₃ , c ₄ correspond to the different phase positions, namely 0 °, 90 °, 180 ° and 270 °, of the autocorrelation.

The determination of the phase shift over the arctan relation is an example only one way of evaluation, of course, other calculation methods are conceivable. In particular, the procedure is not limited to two or four phase positions, but measurements with three phase positions or more than four different phase positions can also be carried out and evaluated.

From the in 2 shown runtime-dependent phase shift Δφ (t _L ) can be determined for object distances d, which are smaller than half the wavelength λ of the modulation frequency d ≤ λ / 2 in a known manner a distance. d = Δφ (t _L ) λ / 2π × 1/2

For distances d> λ / 2, there is generally no possibility of absolutely measuring the phase shift, so that the determined phase shift can no longer be unambiguously assigned to a distance value.

Phase and amplitude errors caused by unmodulated backlight and systematic phase-dependent modulation differences are adaptively corrected by a dark measurement or other background light measurement.

The phase, amplitude and distance calculations for ToF image sensors are made by evaluating several sequentially recorded phase images. The desired difference of the phase images is exclusively a well-defined phase shift from optical modulation to electrical modulation of the sensor. This allows the portions of the active optical modulation to be extracted from the overall signal because the unmodulated light (e.g., sunlight) provides identical proportions in all phase images.

In real camera systems, this ideal case often does not occur, so that the phase images also differ systematically from one another by the proportion of unmodulated light or other influences. Causes for this can be, for example, variations in the duty cycle of the modulation or deviating modulation levels. Without correction, this results in an error of all measured variables, which is dependent on the intensity of the background light and the useful signal and can, for example, as a so-called black-and-white error show, since the effect on dark and bright signal components is different. Weak signal levels are more affected by a superimposed interference due to weighting than strong signals.

The diagrams in the 5 show a simulation of a four-phase measurement under three different lighting conditions. The four measured values of the difference channel "A - B" are plotted analogously to the illustration in accordance with 3a , Amplitude, phase and offset then result from the fits of the measured values. Shown in the diagrams in each case as a sine-fit without further corrections.

5 (a) shows an autocorrelation function AKF only for the modulated light without being affected by a background light. In the example shown there is a phase shift of a = 30 °.

5 (b) shows an example of an autocorrelation function, which could only arise from background light. Although the backlight has no fixed phase relationship to the modulated light and is typically unmodulated, the backlight produces an apparent phase shift that affects the accuracy of the phase measurement of the modulated light. The apparent phase shift is essentially due to so-called imperfections in the system.

In a real measurement would result under these conditions as in 5 (c) show a mixed phase of (a) and (b). It applies to the measured values c1, c2, c3 and c4: c _i = a _i + b _i , i ∈ {1, 2, 3, 4}

The apparent phase shift is essentially a systematic error and can preferably be determined once by a suitable calibration for a system and stored in a characteristic table or lookup table LUT. If necessary, a correction function can also be calculated from this.

In measurement mode, only the intensity of the background light needs to be determined, e.g. by a single phase measurement without modulated light. This now allows the noise component to be calculated and compensated for the other phases. For the upstream determination of the often linear relationship, ideally a measurement series without active, i. modulated signal performed. The total signal can be varied, for example, by varying an uncorrelated irradiation, in particular a direct light, or a variation of the integration time.

For example, a correction can be determined as follows:
Let I be the intensity of the background light. For a linear relationship of the values b1 to b4: b _i = F _i (I) = m _i I + o _i , i ∈ {1, 2, 3, 4} With the constants m _i and o _i , the unadulterated values a1 to a4 can be calculated by: a _i = c _i - b _i = c _i - F _i (I) = c _i - m _i I + o _i . In a non-linear relationship, the function F _i (I) can be replaced by a characteristic table LUT _i (I). In addition to the phase shift, an autocorrelation function and / or the associated correlation values can also be stored in the characteristic value table or as a function in a memory.

A further possibility for determining the apparent phase shift would be conceivable as follows: In measurement mode, a measurement with N sequential phase images with optical modulation is followed by a further measurement with N phases, but without optical modulation, ie only with background light. N here represents the number of phase measurements for a range measurement, i.d.R. Four phase measurements are used.

If the second measurement in the camera is electrically identical to the first, one obtains directly the respective background light component of the phase images of the first measurement. The advantages of this method are that calibration with storage of the correction data is eliminated, and that non-linearities with much background light, e.g. can be compensated adaptively by non-linear characteristics, asymmetry of system components or time-variant sources of interference.

To calculate the apparent phase shift, the disturbance values b1 to b4 are measured directly. The unadulterated values a1 to a4 actual autocorrelation or phase shift result with the measured, disturbed values c1 to c4: a _i = c _i - b _i

In a further preferred embodiment, the compensation of the interference components takes place without additional measurements in that the determination of the interference components takes place from the raw data of the actual distance measurement and is thus not associated with a reduction in the measurement rate. After a suitable calibration with combination of the unmodulated light component, as described in the first method, this is done in the simplest case with PMD sensors directly by calculating the total signal as the sum of the two output signals at each phase.

Furthermore, there are TOF sensor concepts in which it is attempted to reduce the influence of the background light by electronic circuits, so-called SBI circuits (SBI: suppression of background illumination) or by determining a difference signal between the two readout channels (A, B) , However, the information about the total amount of charge is basically lost.

Under real circumstances, however, the uncorrelated background signal is not completely compensated, so that, for example, an average over all raw data, the unmodulated amount of light can be estimated.

Alternatively, the determination of the total signal may, for example, also take place from a measurement of the time until the SBI becomes active during the integration phase or a parallel determination via photodiodes or neighboring pixels or alternative methods.

In the case of a linear SBI asymmetry, the following applies for the intensity I of the background light: I α Σ 4 / i = 1b _i .

The rest of the calculation is analogous to the above-mentioned procedure.

Alternatively, the influence of the noise components on the phase measurements for all three methods described can also be corrected from the calculated distance and amplitude. Since unmodulated light in a naive conventional uncorrected calculation apparently produces an auto-correlation function (ACF) with phase and amplitude as desired by the modulated light, the effect is also referred to as a sham AKF. The calculated AKF in a measurement with background light and useful signal represents a superposition of the two pointers of AKF of the useful signal and bill AKF. The superimposition can be resolved by first determining the angle and amplitude of the apparent AKF via the possibly reconstructed total signal. Subsequently, the pointer is calculated in phase and amplitude of the useful signal from the superimposed signal by means of suitable trigonometric addition theorems.

Phase and amplitude of measurements (a), (b) and (c) are in 6 represented as vectors a →, b →, and c →. The length of the vectors corresponds to the amplitude, the phase corresponds to the angle of the vectors with the real part axis. c → results directly from a faulty measurement. Assuming that phase_b is constant and the ratio of amplitude_b to intensity I of the background light is known, b → can be completely reconstructed from I. The correction calculation to determine a → is then only: a → = c → - b →.

Claims (3)

 Method for operating a time of flight camera with the steps: Operating the illumination without modulation to emit an unmodulated light in a calibration step, - Emission of an unmodulated light with different light intensities Detecting an apparent phase shift and / or autocorrelation function and / or correlation values for the different light intensities, - Deposit of the apparent phase shift and / or autocorrelation function and / or correlation values as a table of values or correction function in a memory. A method for operating a calibrated according to claim 1 light runtime camera, in which the distance values determined during a distance measurement are corrected on the basis of the stored value table or correction function. Time of flight camera system designed to carry out one of the aforementioned methods.

Images