DE102017205745A1 - Time of flight camera - Google Patents

Abstract

Method for determining a point spread function (PSF) for a light runtime camera (20) of a light transit time camera system (1) in which a single image I (x) of a reference scene with an object (40) in the foreground is detected by means of the light runtime camera (20) the condition that the reference scene is formed as a flat surface, wherein the single image I (x) is corrected by means of a first Punkttspreizfunktion, wherein to determine a correction point spread function parameters of the first point spread function are changed until a difference between the corrected image I '(x) and an expected image I ₀ (x) is minimal and / or falls below a threshold.

Description

The invention relates to a time-of-flight camera and to a method for detecting a point spreading function for correcting the detected signals of a light transit time sensor.

Time-of-flight camera or time-of-flight camera systems relate, in particular, to all of the time-of-flight or 3D TOF camera systems which obtain transit time information from the phase shift of an emitted and received radiation. PMT cameras with photonic mixer detectors (PMD) are particularly suitable as the time of flight or 3D TOF cameras, as they are used in the DE 197 04 496 A1 and can be obtained from the company 'ifm electronic GmbH' or 'pmdtechnologies ag' as a frame grabber O3D or as a CamCube. In particular, the PMD camera allows a flexible arrangement of the light source and the detector, which can be arranged both in a housing and separately.

The object of the invention is to further improve the compensation of phase errors.

The object is achieved in an advantageous manner by the inventive time of flight camera system according to the preamble of the independent claims.

Advantageously, a method is provided for determining a point spread function for a light runtime camera of a light transit time camera system in which a single image I (x) of a reference scene with an object in the foreground is detected with the aid of the light runtime camera, with the prerequisite that the reference scene is formed as a flat surface in which the individual image I (x) is corrected by means of a first point spread function, wherein parameters of the first point spread function are changed until a difference between the corrected image I '(x) and an expected image I is determined to determine a correction point spread function ₀ (x) is minimal and / or falls below a threshold.

This procedure has the advantage that essential parameters of a point spreading function can be determined on the basis of a simply designed reference scene.

The reference scene can advantageously be further developed in that the reference scene has defined properties, therefore, a method for determining a point spread function for a light runtime camera of a light time camera system is preferably provided, in which using the light runtime camera, a 3D image I (x) a stage of Reference object is detected, wherein the reference object has a level of defined height and the surfaces of the step are planar and arranged plane-parallel to each other, and the reference object is arranged in relation to the time of flight camera so that at the edge of the step, a distance jump to the more distant level level takes place for determining a correction point spreading function, parameters of the first point spreading function are changed until a difference between the corrected image I '(x) and an expected image I ₀ (x) is minimal and / or falls below a limit value.

A light transit time camera for a light transit time camera system is particularly advantageously provided, with a light transit time sensor with a plurality of light runtime pixels for determining a phase shift of a transmitted and received light, wherein distance values are determined based on the detected phase shifts, wherein the runtime camera has a memory in which at least parameters of a point spread function, which has been determined according to one of the aforementioned methods, are stored, wherein the point spread function takes into account scattered light behavior and signal crosstalk of the light transit time camera and the light transit time sensor, with an evaluation unit which is designed in such a way that a detected image unfolds and a corrected image is determined on the basis of the stored point spread function and that the determination of the phase shifts or distance values is based on the corrected image.

Preferably, the point spreading function is complex.

It is also useful if the unfolding of the acquired image and the stored point spreading function takes place in Fourier space.

The point spread function is preferably stored in the memory as a matrix or lookup table and / or as a Fourier transform.

In a further embodiment, it is provided to store the point spreading function on an external device and to carry out the correction of the phase shifts or distance values on the external device.

The invention will be explained in more detail by means of embodiments with reference to the drawings.

• 1 schematisch ein Lichtlaufzeitkamerasystem,
• 2 eine modulierte Integration erzeugter Ladungsträger,
• 3 ein Aufbau zur Ermittlung einer Punktspreizfunktion,
• 4 einen Querschnitt von Bildern zu Ermittlung einer Punktspreizfunktion
• 5 eine Erfassung einer Referenzszene,
• 6 eine Erfassung eines Objekts vor der Referenzszene,
• 7 gemessene Distanzwerte nach 6 in Relation zu den tatsächlichen Distanzen,
• 8 eine Erfassung von zwei Referenzflächen mit unterschiedlichem Abstand,
• 9 gemessene Distanzwerte nach 8 in Relation zu den tatsächlichen Distanzen,
• 10 einen möglichen schematischen Ablauf der Streulichtkorrektur im Sinne der Erfindung.

Show it:

• 1 schematically a light transit time camera system,
• 2 a modulated integration of generated charge carriers,
• 3 a structure for determining a point spread function,
• 4 a cross-section of images to determine a point spread function
• 5 a capture of a reference scene,
• 6 a detection of an object in front of the reference scene,
• 7 measured distance values after 6 in relation to the actual distances,
• 8th a detection of two reference surfaces with different distances,
• 9 measured distance values after 8th in relation to the actual distances,
• 10 a possible schematic sequence of stray light correction in the context of the invention.

In the following description of the preferred embodiments, like reference characters designate like or similar components.

1 shows a measurement situation for an optical distance measurement with a light time camera, as for example from the DE 197 04 496 A1 is known.

The light transit time camera system 1 comprises a transmitting unit or a lighting module 10 with a lighting 12 and associated beam shaping optics 15 as well as a receiving unit or light runtime camera 20 with a receiving optics 25 and a light transit time sensor 22.

The light transit time sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is designed in particular as a PMD sensor. The receiving optic 25 typically consists of improving the imaging characteristics of a plurality of optical elements. The beam shaping optics 15 the transmitting unit 10 may be formed for example as a reflector or lens optics. In a very simple embodiment, if necessary, optical elements can also be dispensed with both on the receiving side and on the transmitting side.

The measurement principle of this arrangement is essentially based on the fact that, based on the phase shift of the emitted and received light, the transit time and thus the distance covered by the received light can be determined. For this purpose, the light source 12 and the light transit time sensor 22 via a modulator 30 together with a specific modulation signal M ₀ with a base phase situation φ ₀ applied. In the example shown is also between the modulator 30 and the light source 12 is a phase shifter 35 provided with which the base phase φ ₀ the modulation signal M ₀ the light source 12 around defined phase positions φ _var can be moved. For typical phase measurements, preferably phase angles of φ _var = 0 °, 90 °, 180 °, 270 ° used.

The light source transmits according to the set modulation signal 12 an intensity modulated signal S _p1 with the first phase position p1 respectively. p1 = φ ₀ + φ _var out. This signal S _p1 or the electromagnetic radiation is reflected in the case shown by an object 40 and hits due to the distance traveled 2d, or the light transit time t _L , out of phase Δφ (t _L ) with a second phase position p2 = φ ₀ + φ _var + Δφ (t _L ) as a received signal S _p2 on the light transit time sensor 22 , In the time of flight sensor 22 becomes the modulation signal M ₀ with the received signal S _p2 mixed, wherein from the resulting signal, the phase shift or the object distance d is determined.

As illumination source or light source 12 Preferably, infrared light emitting diodes or surface emitters (VCSEL) are suitable. Of course, other radiation sources in other frequency ranges are conceivable, in particular, light sources in the visible frequency range are also considered.

The basic principle of phase measurement is exemplified schematically in FIG 2 shown. The upper curve shows the time course of the modulation signal M ₀ with the lighting 12 and the light transit time sensor 22 be controlled. The object 40 reflected light hits as received signal S _p2 according to its light-time t _L phase Δφ (t _L ) on the light transit time sensor 22 , The light transit time sensor 22 collects the photonically generated charges q over several modulation periods in the phase position of the modulation signal M ₀ in a first integration node ga and in a phase angle shifted by 180 ° in a second integration node gb , For this steering of the charges to the integration nodes, the pixels 23 of the light transit time sensor 22 at least two modulation gates Gam, Gbm on, depending on the applied modulation signals, the charges to the first or second integration node ga . gb to steer. From the difference in the first and second integration nodes ga . gb collected charges qa, qb taking into account all phases φ _var can be the phase shift Δφ (t _L ) and thus a distance d of the object.

3 schematically shows a structure for determining a point spread PSF. Here, the light source 112 and the light transit time sensor 22 be operated unmodulated or with at least one predetermined modulation frequency. When using an unmodulated light, it is advantageous, although the light transit time sensor 22 or the pixels 23 are operated unmodulated. Here it is helpful if a constant voltage is applied to the modulation gates Yarn, Gbm of the pixels 23 in such a way that the photogenerated charges are primarily only in an integration node ga . gb to be collected.

To determine the PSF, it is favorable if the light source 112 substantially only a single pixel 23, preferably less than 3 × 3 and in particular less than 5 × 5 pixels 23 of the light transit time sensor 22 illuminated. To provide such a light spot, a diaphragm 150 with a sufficiently small diaphragm opening 152 is provided in front of the light source 112. The originating from the aperture 150 original light signal I ₀ is influenced by a variety of influences on the way to the sensor up to the detected image signal I (x), for example by properties of the optical system or optics 25 or reflections between sensor 22 and optics 25. Also play intrinsic properties of the sensor 22 As a result, the image signal I (x) detected at the sensor may be a convolution between the incoming light I ₀ and a point spreading function PSF that includes substantially all the characteristics of the overall system , to be viewed as. Due to the singular illumination of one or a few fewer pixels, the detected image signal I (x) essentially corresponds to the point spreading function PSF. For the determination of the point spreading function, preferably all pixels are evaluated. In principle, however, it is also conceivable to evaluate only a partial area around the singularly illuminated pixel.

The quality of the point spreading function PSF can be improved if necessary, if several point spreading functions are determined on the basis of a plurality of singularly illuminated pixels 23. For example, it makes sense to also illuminate pixels 23 outside the optical axis in order to determine further point spread functions at these positions. On the basis of the determined point spread functions, a point spread function can then be determined which is to be used for the later corrections.

Since the mentioned electron diffusion typically occurs at a diffusion rate that is significantly lower than the propagation of light, the electrons reach adjacent pixels with a time delay, so that the influence of electron diffusion also manifests itself as a phase shift. The point spread PSF thus also receives complex valued shares. For more accurate determination of these quantities, it is therefore advantageous to operate the light source 112 in different phase angles.

Since a point spread function typically has high dynamics over several orders of magnitude, it is further advantageous for detecting the PSF, the point light source 112 with different intensities and / or the sensor 22 operate with different integration times.

To compensate for dark currents, it is helpful to capture image signals I (x) both when the light source 112 is switched on and off.

From the sum of all measurements, a model of a point spread function can be generated, which is applicable to all pixels 23.

wobei beispielsweise $$A\left(\overrightarrow{x}\right)=A_0\text{ exp}\left(-s\left({\Vert \overrightarrow{x}\Vert }_p\right)\right)$$

gewählt werden kann.Such a model can be generated according to the following considerations: Since the measured PSF is noisy and may contain, for example, artifacts that are very specific to the pixel position on the sensor, a "clean" PSF is obtained, for example, by fitting the measured PSF to a suitable one Model. As a model are suitable, for example $$\text{PSF}\left(\overrightarrow{x}\right)=A\left(\overrightarrow{x}\right)+B\left({\Vert \overrightarrow{x}\Vert }_{p_B}\right)$$

for example $$A\left(\overrightarrow{x}\right)=A_0\text{exp}\left(-s\left({\Vert \overrightarrow{x}\Vert }_p\right)\right)$$

can be chosen.

den Entfernungsvektor vom Zentralpixel ${\overrightarrow{x}}_0$

der PSF in Pixeln und ${\Vert \overrightarrow{x}\Vert }_p={\left(x_1^p+x_2^p\right)}^{1/p}$

die p-Norm von $\overrightarrow{x}\cdot p=2$

ergäbe z.B. eine exakt radialsymmetrische PSF. Da die PSF nicht unbedingt radialsymmetrisch ist, sondern z.B. rautenförmig sein kann, kann p ≠ 2 bessere Ergebnisse liefern. Durch geeignete Wahl der p-Norm können somit Anisotropien der PSF berücksichtigt werden.Hereby designated $\overrightarrow{x}$

the distance vector from the central pixel ${\overrightarrow{x}}_0$

the PSF in pixels and ${\Vert \overrightarrow{x}\Vert }_p={\left(x_1^p+x_2^p\right)}^{1/p}$

the p-norm of $\overrightarrow{x}\cdot p=2$

For example, would result in an exactly radially symmetric PSF. Since the PSF is not necessarily radially symmetric, but may be diamond-shaped, for example, p ≠ 2 can give better results. By suitable choice of the p-standard anisotropies of the PSF can thus be taken into account.

Since most of the light falls on the central pixel of the PSF, it is helpful to add to the model a locally narrow function B (r) that reflects that fraction. This can e.g. a Dirac delta, or a Gaussian function describing, for example, the lens blurring.

For reasons of efficiency, it is advantageous to describe the PSF in the form of a spline curve, for example. In order to describe phase shifts with this PSF, the spline may, for example, have a complex-valued component in addition to the real part. This also makes the PSF complex. Suitable fitting parameters are then, for example, the values at the nodes of the splines, the norm parameters p and p _B , as well as parameters that specify the shape of B (r). Instead of the complete PSF It is advantageous to save only the necessary parameters in order to generate the PSF from these parameters when the software is initialized.

During operation of the time of flight camera, it is then possible to use the stored parameters and the resulting PSF to clean up the distance values with regard to scattered light influences.

mit kurzer Belichtungszeit t_k aufgenommen. Konkret ist die Belichtungszeit so zu wählen, dass keiner der Pixel in Sättigung ist. Für den Fall, dass moduliertes Licht verwendet wird, darf kein Pixel der erhaltenen Rohbilder gesättigt sein.With the help of the described structure is preferably a first image ${\tilde{I}}_k\left(\overrightarrow{x}\right)$

taken with a short exposure time t _k . Specifically, the exposure time should be chosen so that none of the pixels is in saturation. In the case where modulated light is used, no pixel of the obtained raw images may be saturated.

mit langer Belichtungszeit t_l aufgenommen. Hier ist die Belichtungszeit so zu wählen, dass der von Streulicht und/oder crosstalk/Signalübersprechen verursachte Anteil der PSF möglichst vollständig sichtbar, also nicht von Rauschen beeinträchtigt ist. Die Belichtungszeit ist hier typischerweise 1000-10000 mal größer als im ersten Bild.There will also be a second picture ${\tilde{I}}_l\left(\overrightarrow{x}\right)$

recorded with a long exposure time t _l . Here, the exposure time should be selected so that the proportion of the PSF caused by scattered light and / or crosstalk / signal crosstalk is as completely as possible visible, ie not affected by noise. The exposure time here is typically 1000-10000 times greater than in the first image.

wie üblich komplexwertig, enthalten also Phaseninformationen, die die Zeit vom Aussenden des Lichts bis zum Aufnehmen der erzeugten Elektronen an den Gates des Sensors wiederspiegelt.When recording, either unmodulated light can be used or the light source and sensor can be modulated in the usual way. In the latter case are the pictures ${\tilde{I}}_k\left(\overrightarrow{x}\right)\text{and}{\tilde{I}}_l\left(\overrightarrow{x}\right)$

as usual complex-valued, thus contain phase information, which reflects the time from the emission of light to the recording of the generated electrons at the gates of the sensor.

For both images, it may be helpful to take a series of images instead of an image and average them to further reduce the noise.

In order to obtain consistent values between the first and second image, for example, the brightnesses (or amplitudes) with the different integration times are normalized: $$\begin{array}{c}{I}_l\left(\overrightarrow{x}\right)={\tilde{I}}_l\left(\overrightarrow{x}\right)/t_l\\ I_k\left(\overrightarrow{x}\right)={\tilde{I}}_k\left(\overrightarrow{x}\right)/t_k\end{array}$$

noch unbekannt. Um die Position des Zentralpixels ${\overrightarrow{x}}_0$

zu bestimmen, wird das erste Bild $I_k\left(\overrightarrow{x}\right)$

beispielsweise mit Hilfe eines Schwellwertverfahrens binarisiert, wodurch der helle LED-Spot in einer zusammenhängenden Fläche resultieren sollte.In the pictures obtained iA is the exact position of the illuminated central pixel ${\overrightarrow{x}}_0$

still unknown. To the position of the central pixel ${\overrightarrow{x}}_0$

to determine, becomes the first picture $I_k\left(\overrightarrow{x}\right)$

for example, using a thresholding method, which should result in the bright LED spot in a contiguous area.

auf dem Sensor, auf den die Lichtquelle gerichtet ist. Dieser Zentralpunkt ${\overrightarrow{x}}_0$

muss nicht notwendigerweise auf die Mitte eines Pixels fallen, d.h. die gefundene Position für den Zentralpunkt ${\overrightarrow{x}}_0$

muss nicht ganzzahlig sein.The center of the contiguous surface is a good estimate of the central pixel or center point ${\overrightarrow{x}}_0$

on the sensor to which the light source is directed. This central point ${\overrightarrow{x}}_0$

does not necessarily fall to the center of a pixel, ie the position found for the central point ${\overrightarrow{x}}_0$

does not have to be an integer.

der kurzen Belichtung an das Modell eines scharfen Spots gefittet. Ein solches Modell ist beispielsweise in Gleichung (1) mit $B\left({\Vert \overrightarrow{x}\Vert }_p\right)$

bezeichnet. Konkret wird hierbei $$\left(P_B,p_B\right)=\underset{P_B,p_B}{{\displaystyle \text{arg min}}}\left({{\displaystyle {\sum }_{\overrightarrow{x}}\left|I_k\left(\overrightarrow{x}\right)-B\left({\Vert \overrightarrow{x}-{\overrightarrow{x}}_0\Vert }_{p_B}\right)\right|}}^2\right)$$

bestimmt, wobei P_B die Parameter der Funktion B(r) und p_B der Parameter der Norm sind. Beispielsweise könnte B(r) = B₀ exp(-br²) gewählt werden, wobei dann P_B = (B₀, b) würde.Now the picture will be $I_k\left(\overrightarrow{x}\right)$

short exposure to the model of a sharp spot. Such a model is for example in equation (1) with $B\left({\Vert \overrightarrow{x}\Vert }_p\right)$

designated. It becomes concrete here $$\left(P_B.p_B\right)=\underset{P_B.p_B}{{\displaystyle \text{arg min}}}\left({{\displaystyle {\Sigma }_{\overrightarrow{x}}\left|I_k\left(\overrightarrow{x}\right)-B\left({\Vert \overrightarrow{x}-{\overrightarrow{\mathrm{<figure-callout\; id="0"\; label="x\; \to "\; filenames="DE102017205745A1\_0040.png"\; state="\{\{state\}\}">x\; </figure-callout>}}}_0\Vert }_{p_B}\right)\right|}}^2\right)$$

where P _{B are} the parameters of function B (r) and p _{B are} the parameters of the norm. For example, B (r) = B ₀ exp (-br ² ) could be chosen, then P _B = (B ₀ , b).

For the numerical minimization according to equation (4), there are numerous algorithms, e.g. the Nelder-Mead procedure.

der Lichtquelle mit in die Optimierung aufzunehmen. Dann würde der zuvor gefundene Wert aus dem binarisierten Bild sich als Startwert eignen.In addition to P _B and p _{B it} can also give better results, in equation (4) the center ${\overrightarrow{x}}_0$

to include the light source in the optimization. Then the previously found value from the binarized image would be suitable as a seed.

mit der langen Belichtungszeit betrachtet. Analog zu Gleichung (4) wird das Bild an das Modell der Streulichtsignatur, $A\left(\overrightarrow{x}\right)$

in Gleichung (1), gefittet: $$\left(P_A,p_A\right)=\underset{P_A,p_A}{{\displaystyle \text{arg min}}}\left({{\displaystyle \sum _{\overrightarrow{x}}\left|I_l\left(\overrightarrow{x}\right)-A\left(\overrightarrow{x}-{\overrightarrow{x}}_0\right)\right|}}^2\right)$$

Now the second picture $I_l\left(\overrightarrow{x}\right)$

considered with the long exposure time. Analogously to equation (4), the image is applied to the model of the scattered light signature, $A\left(\overrightarrow{x}\right)$

in equation (1), fitted: $$\left(P_A.p_A\right)=\underset{P_A.p_A}{{\displaystyle \text{arg min}}}\left({{\displaystyle \underset{\overrightarrow{x}}{\Sigma }\left|I_l\left(\overrightarrow{x}\right)-A\left(\overrightarrow{x}-{\overrightarrow{x}}_0\right)\right|}}^2\right)$$

If necessary, the central portion of the PSF described by B (r) can be disregarded.

Als zweckdienlich erweist sich z.B. eine Funktion der Form: $$A\left(\overrightarrow{r}\right)=A_0\text{ exp}\left(-s\left({\Vert \overrightarrow{r}\Vert }_{p_A}\right)+i\mathrm{\Phi }\left(\overrightarrow{r}\right)\right)$$

wobei s(r) und eine (reelle) Spline-Kurve darstellt. Die Funktion $\mathrm{\Phi }\left(\overrightarrow{r}\right)$

beschreibt eine Phasenverzögerung des einfallenden Lichtspots, wie sie beispielsweise durch Phase-Crosstalk/Signalübersprechen zwischen den Pixel verursacht werden kann. Da dieser nicht zwingend isotrop ist, kann es nötig sein, $\mathrm{\Phi }\left(\overrightarrow{r}\right)$

als zweidimensionale Funktion (z.B. einen 2D-Spline oder eine 2D-Look-up-Table) zu modellieren, anstatt wie für s(r) eine radial symmetrische Funktion anzunehmen.Analogous to the first fit, here P _{A are} the parameters of the model function $A\left(\overrightarrow{r}\right),$

For example, a function of the form proves to be useful: $$A\left(\overrightarrow{r}\right)=A_0\text{exp}\left(-s\left({\Vert \overrightarrow{r}\Vert }_{p_A}\right)+i\mathrm{\Phi }\left(\overrightarrow{r}\right)\right)$$

where s represents (r) and a (real) spline curve. The function $\mathrm{\Phi }\left(\overrightarrow{r}\right)$

describes a phase delay of the incident light spot, such as may be caused by phase crosstalk / signal crosstalk between the pixels. Since this is not necessarily isotropic, it may be necessary $\mathrm{\Phi }\left(\overrightarrow{r}\right)$

as a two-dimensional function (eg a 2D spline or a 2D look-up table) instead of assuming a radially symmetric function as for s (r).

The fit parameters P _A in this case are A ₀ , p _A , as well as the function values of the splines at the nodes. If necessary, the location of the nodes can also be part of the fit parameter P _A.

With the obtained parameters P _A and P _B , as well as the PSF model, for example according to equation (1), it is now possible to generate an artifact-free and non-noisy PSF. Instead of storing the complete PSF, it is advantageous to store only these or other suitable parameters from which the PSF can be generated when the software is initialized.

It is preferably provided that the recordings recorded with different exposure times are processed separately from one another: $$\text{PSF}\left(\overrightarrow{x}\right)=A_1\left(\overrightarrow{x}\right)+A_2\left(\overrightarrow{x}\right)+\cdots$$

können beispielsweise unterschiedlichen Dynamikbereichen der PSF entsprechen und jeweils getrennt voneinander an Aufnahmen $I_1\left(\overrightarrow{x}\right),I_2\left(\overrightarrow{x}\right),\dots$

gefittet werden. Ausgehend von diesen Fit-Parameter kann dann die PSF gemäß Gleichung (7) zusammengefasst werden.The part models $A_1\left(\overrightarrow{x}\right).A_2\left(\overrightarrow{x}\right)....$

For example, they may correspond to different dynamic ranges of the PSF and to recordings separately from each other $I_1\left(\overrightarrow{x}\right).I_2\left(\overrightarrow{x}\right)....$

be fit. Based on these fit parameters, the PSF can then be summarized according to equation (7).

In the foregoing, the calibration has been described with reference to a point light source with aperture as the light source or light source system. Of course, the calibration is not limited to such a light source, but all light sources or light systems are considered, which can generate a suitable light spot.

4 to 9 show further methods for determining a suitable point spread function PSF. In the method according to 4 become a first 3D image I ₁ (x) of a reference scene and a second 3D image I ₂ (x) with an object 40 captured in the foreground of the reference scene. As already discussed, a change in the distance values known from the first 3D image I ₁ (x) is to be expected due to systemic influences. In order to determine a point spread function suitable for the correction, parameters of a first model PSF are then varied until differences between the first and second image, in particular distance errors, are minimal or smaller than a tolerated limit value. Here, preferably only the image areas or a subregion thereof are taken into account, in which the reference scene is visible in both images.

For example, the pictures can be like in 5 and 6 be shown. In a first step, a first 3D image I ₁ (x) of a reference scene is acquired ( 5 ). As a reference scene, for example, a wall or a floor can be detected in a simple manner, but in principle any scenes with an arbitrary height profile can also be detected. In the second step according to 6 For example, an object is placed above the reference scene, such as a hand or other object, and captures a second range image I ₂ (x). Again, the properties of the object are essentially uncritical. How to 4 described can then be generated on the basis of the difference between the two images, a correction PSF.

In 7 a variant is shown in which the reference scene and the object are arranged plane and plane parallel to each other. With such prior knowledge, optimizing the PSF may be easier.

Alternatively, for example, instead of two images, it is possible to capture only one image of a target at a sufficient distance from a flat reference scene (e.g., wall, table, floor). To determine the PSF, the parameters are now varied until the reference surface behind the target is as flat as possible, or the deviations of the corrected reference surface from one plane are smaller than a tolerated limit.

It is particularly advantageous if the dimensions and distances of the reference scene and / or of the introduced target are known in advance.

8th and 9 show a further variant of the above procedure. This in 8th represented object 40 has a level defined in height. The height Δd = d _T2 -d _T1 is preferably previously known. As in the above example, the parameters of a PSF model are varied until the distance error is minimal or below a tolerated limit.

mit der PSF: $$D_j\left(\mathbf{x}\right)={\displaystyle {\sum }_{\Delta \text{x}}{D}_j^0\left(\mathbf{x}-\Delta \mathbf{x}\right)}\cdot \text{PSF}\left(\Delta \mathbf{x}\right)$$

The raw images D _j measured by the sensor (eg j = 0,1,2,3 corresponding to the phase positions 0 °, 90 °, 180 °, 270 °) are, mathematically speaking, a convolution of the unknown, not stray-light-distorted raw images $D_j^{}$${}_0^{}$

with the PSF: $$D_j\left(\mathbf{x}\right)={\displaystyle {\Sigma }_{\Delta \text{x}}{D}_j^0\left(\mathbf{x}-\Delta \mathbf{x}\right)}\cdot \text{PSF}\left(\Delta \mathbf{x}\right)$$

Interesting for further processing are the complex-valued images $$I\left(\mathbf{x}\right):=\left(D_0\left(\mathbf{x}\right)-D_2\left(\mathbf{x}\right)\right)+i\left(D_1\left(\mathbf{x}\right)-D_3\left(\mathbf{x}\right)\right)$$

bzw. $$I\left(\mathbf{x}\right)=I_0\left(\mathbf{x}\right)\ast \text{PSF}\left(\mathbf{x}\right)$$

Since the convolution is a linear operation, the same holds for I (x) and the non-scattered complex-valued image I ₀ (x): $$I\left(\mathbf{x}\right)={\displaystyle {\Sigma }_{\Delta \text{x}}{I}_0}\left(\mathbf{x}\text{-}\Delta \mathbf{x}\right)\cdot \text{PSF}\left(\Delta \mathbf{x}\right)$$

respectively. $$I\left(\mathbf{x}\right)=I_0\left(\mathbf{x}\right)*\text{PSF}\left(\mathbf{x}\right)$$

The unfolding is performed in Fourier space. For this I (x) and the PSF are Fourier transformed $\left(F[\cdot ]\right):\widehat{I}\left(\mathbf{k}\right)=F\left[I\left(\mathbf{x}\right)\right]\text{and}\widehat{\text{PSF}}\left(\mathbf{k}\right)=F\left[\text{PSF}\left(\mathbf{x}\right)\right],$

und daher $${\widehat{I}}_0\left(\mathbf{k}\right)=\frac{\widehat{I}\left(\mathbf{k}\right)}{\widehat{\text{PSF}}\left(\mathbf{k}\right)}$$

Equation 4 becomes: $$\widehat{I}\left(\mathbf{k}\right)={\widehat{I}}_0\left(\mathbf{k}\right)\cdot \widehat{\text{PSF}}\left(\mathbf{k}\right)$$

and therefore $${\widehat{I}}_0\left(\mathbf{k}\right)=\frac{\widehat{I}\left(\mathbf{k}\right)}{\widehat{\text{PSF}}\left(\mathbf{k}\right)}$$

This gives the non-scattered image $$I_0\left(\mathbf{x}\right)=F^{-1}\left[{\widehat{I}}_0\left(\mathbf{k}\right)\right]$$

LIST OF REFERENCE NUMBERS

1

Time of flight camera system

10

lighting module

12

lighting

15

Beam shaping optics

20

Receiver, light time camera

22

Transit Time Sensor

30

modulator

35

Phase shifter, lighting phase shifter

40

object

φ, Δφ (t _L )

term-related phase shift

φ _var

phasing

φ ₀

base phase

M ₀

modulation signal

p1

first phase

p2

second phase

S _p1

Transmission signal with first phase

S _p2

Received signal with second phase

t _L

Transit Time

Ga, Gb

integration node

d

Object distance

q

charge

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 19704496 A1 [0002, 0016]

Claims (8)

Method for determining a point spread function (PSF) for a light runtime camera (20) of a light transit time camera system (1) in which a single image I (x) of a reference scene with an object (40) in the foreground is detected by means of the light runtime camera (20) the condition that the reference scene is formed as a flat surface, wherein the single image I (x) is corrected by means of a first Punkttspreizfunktion, wherein to determine a correction point spread function parameters of the first point spread function are changed until a difference between the corrected image I '(x) and an expected image I ₀ (x) is minimal and / or falls below a threshold. A method for determining a point spreading function for a light runtime camera (20) of a light time camera system (1), in which using the light time camera (20) a 3D image I (x) of a stage of a reference object is detected, wherein the reference object has a level of defined height and the surfaces of the step are planar and arranged plane-parallel to each other, and the reference object is arranged in relation to the time of flight camera (20) so that at the edge of the stage a distance jump to the more distant level level takes place, wherein for determining a correction Punkttspreizfünktion parameters of the first Punkttspreizfunktion be changed until a difference between the corrected image I '(x) and an expected image I ₀ (x) is minimal and / or falls below a threshold. A light transit time camera (20) for a light transit time camera system (1), comprising a light transit time sensor (22) with a plurality of light runtime pixels (23) for determining a phase shift of a transmitted and received light (Sp2), wherein distance values (d) are determined based on the detected phase displacements (Δφ) are characterized in that the light runtime camera (20) comprises a memory in which at least parameters of a point spreading function (PSF), which according to a method according to Claims 1 or 2 was determined, the point spread function (PSF) a scattered light behavior and a signal crosstalk of the light runtime camera (20) and the light transit time sensor (22) considered, with an evaluation that is designed such that based on the stored point spread function (PSF) a captured image (I (x)) is unfolded and a corrected image (I ₀ (x)) is determined, and that the determination of the phase shifts (Δφ) and distance values (d) takes place on the basis of the corrected image (I ₀ (x)). Photocell camera (20) after Claim 3 in which the point spread function (PSF) is complex-valued. The light transit time camera (20) according to one of the preceding device claims, wherein the unfolding of the captured image (I (x)) and the stored point spread function (PSF) is performed in Fourier space. The time of flight camera (20) according to one of the preceding device claims, wherein the point spread function (PSF) is stored as a matrix or lookup table in the memory. The time of flight camera (20) according to one of the preceding device claims, in which the point spread function (PSF) is stored in the memory as a Fourier transform. A time of flight camera (20) according to any one of the preceding apparatus claims, wherein the point spreading function (PSF) is stored on an external device and the correction of the phase shifts (Δφ) or distance values (d) is done on the external device.

Images