EP4445170A1 - Electronic device and method - Google Patents

Abstract

An electronic device comprising circuitry configured to a determine, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

Description

ELECTRONIC DEVICE AND METHOD

TECHNICAL FIELD

The present disclosure generally pertains to the field of Time-of- Flight imaging, and in particular to devices and methods for Time-of-Flight image capturing.

TECHNICAL BACKGROUND

A Time-of-Flight (ToF) camera is a range imaging camera system that determines the distance of objects by measuring the time of flight of a light signal between the camera and the object for each point of the image. Generally, a ToF camera has an illumination unit (a LED or VCSEL, Vertical- Cavity Surface-Emitting Laser) that illuminates a scene with modulated light. A pixel array in the ToF camera collects the light reflected from the scene and measures phase-shift which provides information on the travelling time of the light, and hence information on distance.

Currently, acquisitions of the light signal at short distances is typically covered by indirect Time of Flight (iToF) systems and the acquisitions at mid-to-long distances is typically covered by direct Time of Flight (dToF) systems.

In indirect Time-of-Flight (iToF), three-dimensional (3D) images of a scene are captured. These images are also commonly referred to as “depth map”, or “depth image”, wherein each pixel of the image is attributed with a respective depth measurement. The depth image can be determined directly from a phase image, which is the collection of all phase delays determined in the pixels of the iToF camera.

Current iToF systems comprise image sensors with pixels of the so called "2-tap" type, which allow for a VGA resolution or higher and ensure accuracy in the millimetre range at object distances below Im. Between Im and 5m an increase in noise may occur which may be mediated with the help of binning. However, the downside of binning is a reducing the resolution, typically to QVGA. Above 5m the resolution may be further reduced to QQVGA. Moreover, scattering effects from highly reflective objects at close distances may make long-range objects difficult to detect.

Additionally, de-aliasing of data within the detectable range may become difficult because single- frequency methods are often prone to high depth noise, while double-frequency methods are often prone to aliasing errors. Still further, temporal accumulation (i.e. micro frame-based acquisition) may make the iToF system prone to artefacts due to motion of objects spanning multiple microframes. dToF systems typically offer a strongly reduced resolution at distances below Im. They provide an effective resolution of QQQVGA or lower and only accuracy in the centimetre range. Between Im and 5m, there may be no decrease in resolution or increase in accuracy. However, above 5m one dTof may have good ambient light rejection, while still preserving a good resolution which is often achieved by analysing the histogram of the received number of photons and identifying corresponding objects as peaks in these histograms. Thereby, also scattering and multi-path may be mitigated.

Therefore, it is generally desirable to provide techniques which improve the determining of depths images with an iToF camera.

SUMMARY

According to a first aspect the disclosure provides an electronic device comprising circuitry configured to a determine, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

According to a further aspect the disclosure provides a method comprising determining, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

Further aspects are set forth in the dependent claims, the following description and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are explained byway of example with respect to the accompanying drawings, in which:

Fig. 1 schematically shows the basic operational principle of an indirect Time-of-Flight imaging;

Fig. 2 schematically shows an exemplifying circuit layout of an N-tap ToF pixel with more than two taps (N > 2), here N = 7 seven taps;

Fig. 3 shows a timing diagram concerning the control of transfer gates TG1,..., TG7 and overflow gate OFG of the N-tap pixel described in Fig. 2.

Fig. 4a schematically shows an example of gate rotary switching with an activated transfer gate TG1;

Fig. 4b schematically shows an example of gate rotary switching with an activated transfer gate TG2;

Fig. 5 shows in a schematic way a process of sampling of a correlation waveform between the reference (mix) signal and the received signal for an N-tap pixel with N = 8;

Fig. 6 shows in a schematic way the determination of the phase value Φ between the emitted and the received light from the IQ measurement F(1);

Fig. 7 schematically shows a timing diagram for the control of the transfer gates of an N-tap pixel (here N = 7 + OFG); Fig. 8 shows an embodiment of a frame structure of a 7 -tap iToF pixel;

Fig. 9 shows a timing diagram for a sub -integration cycle of a range gated iToF image sensor with seven taps;

Fig. 10 shows a radial detection range of an iToF camera with close-range and a far-range gated areas;

Fig. 11 shows another embodiment of radial detection range of an iToF camera with close-range and a far-range gated areas;

Fig. 12 shows two timing diagrams for a sub-integration cycle of a range gated iToF image sensor with seven taps using a tap skipping and a high precision method;

Fig. 13 shows four timing diagrams for four sub-integration cycles of an iToF image sensor with seven taps using dynamic range extension;

Fig. 14 shows a graph depicting the acquired current in the different taps of a 7-tap system (using dynamic range extension) depending on the distance of a detected object;

Fig. 15 shows a timing diagram for a sub-integration cycle of an iToF image sensor with seven taps using range specific binning.

Fig. 16 shows radial detection range of an iToF camera with range specific binning;

Fig. 17 schematically describes an embodiment of an iToF device that can implement the processes of determining the depth value of the object with an N-tap pixel.

DETAILED DESCRIPTION OF EMBODIMENTS

The embodiments disclose an electronic device comprising circuitry configured to a determine, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

The electronic device may for example be an imaging sensor of an imaging camera, in particular a sensor of a ToF camera. For example, the electronic device may be an imaging sensor with N-tap pixels, where N denotes the number of taps provided in a pixel of the imaging sensor. The electronic device may also comprise additional circuitry. For example, circuitry of the electronic device may include electronic components such as switching elements (gates, transistors, etc.), resistors, memory elements (capacitors, RAM, ROM or the like), pixel circuitry, a storage, input means (mouse, keyboard, camera, etc.), output means (display (e.g. liquid crystal, (organic) light emitting diode, etc.), loudspeakers, etc., a (wireless) interface, etc., as it is generally known for electronic devices (computers, smartphones, etc.). Moreover, it may include sensors for sensing still image or video image data (image sensor, camera sensor, video sensor, etc.), for sensing a fingerprint, for sensing environmental parameters (e.g. radar, humidity, light, temperature), etc.

A tap (also referred to as a collection node) may be any structure or configuration a camera sensor, respectively a pixel of a camera sensor uses to output data. A pixel may be any single cell of an image sensor. When light is captured by an image sensor, a photo sensor (e.g. a photodiode) in each pixel produces an electrical charge. The electrical charges accumulated in the pixels of an image sensor are transferred by read-out electronics from the pixels to a processer for post-processing of an image.

The circuitry of the electronic device described below in more detail may be configured to determine a discrete Fourier transform based on the three or more measurements related to the taps of the pixel, and to determine the depth measurement based on the discrete Fourier transform. The term Fourier transform refers to both the frequency domain representation and the mathematical operation that associates the frequency domain representation to a function of space or time. The Fourier transform of a function of time is typically a complex-valued function of frequency.

The circuitry may for example be configured to determine the discrete Fourier transform based on the total number of taps of the pixel and based on measurements provided by the respective taps of the pixel.

The circuitry may be configured to determine the depth measurement based on a real part and an imaginary part of a Fourier transform. The discrete Fourier transform may for example be a first harmonic estimate which is also referred to as an IQ measurement (with I and Q being the real, respectively, imaginary part of the first harmonic estimate) which yields the phase delay between the emitted light and the reflected light. The phase delay yields the distance of the object. That means the phase delay information and depth information (as well as confidence and amplitude) may be obtained in the Fourier domain and, once the (discrete) Fourier transform is computed on the samples of the correlation waveform (i.e. the measurements provided by the taps), the first harmonic will contain I and Q information as its real and imaginary part, respectively.

The circuitry further may comprise three or more transfer gates each corresponding to one of the three or more taps. A transfer gate may for example be implemented as a transistor. However, other switching elements may be used as an alternative to transistors.

The circuitry may be configured to open the transfer gates of the respective taps of the pixel one after the other for a predetermined time.

The circuitry may be configured to open the transfer gates according to a predetermined pulse width which is equal to the pulse width of pulses of an illumination pulse train. In another embodiment illumination pulse width may be wider or narrower than predetermined pulse width opening the transfer gates.

The circuitry may be configured to generate an illumination pulse train which has a duty cycle equal or lower than 1%. In another embodiment the typical duty cycle may be 1/N, wherein N is the number of taps, which may yield a duty cycle of 12.5% for an 8-tap system.

The circuitry may further comprise an overflow gate which is activated in a sub-integration cycle after the transfer gates have been activated.

The circuitry may be configured to determine ambient light corrected measurements based on the measurements obtained by respective taps of the pixel.

The circuitry may be configured to determine empty taps to perform ambient light suppression.

An empty tap may be a tap without illumination reflected light determined after peak detection.

The circuitry may be configured to determine ambient light corrected measurements by subtracting an average signal over empty taps from the measurements obtained by the respective taps of the pixel.

The circuitry may be configured to set measurements that correspond to empty taps to zero.

The circuitry may be configured to perform close-range gating.

The circuitry may be configured to perform far-range gating.

The circuitry may be configured to perform dynamic range extension.

The circuitry may be configured to perform range-specific binning.

The embodiments also disclose a method comprising determining, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

Embodiments are now described by reference to the drawings.

Operational principle of an indirect Time-of-Flight imaging system (iToF)

Fig. 1 schematically shows the basic operational principle of an indirect Time-of-Flight imaging system which can be used for depth sensing. The iToF imaging system 101 includes an iToF camera with an imaging sensor 102 having a matrix of pixels and a processor (CPU) 105. A scene 107 is actively illuminated with amplitude-modulated infrared light LMS at a predetermined wavelength using an illumination device 110, for instance with some light pulses of at least one predetermined modulation frequency DML generated by a timing generator 106. The amplitude-modulated infrared light LMS is reflected from objects within the scene 107. A lens 103 collects the reflected light 109 and forms an image of the objects within the scene 107 onto the imaging sensor 102. In indirect Time-of-Flight (iToF) the CPU 105 determines for each pixel a phase delay between the modulated signal DML and the reflected light RL. Based on these correlations a so called in-phase component value (“I value”) and a so called quadrature component value (“Q value”) can be determined (see below for a detailed description) for each pixel.

Fig. 2 schematically shows an exemplifying circuit layout of an N-tap ToF pixel with more than two taps (N > 2), here N = 7 seven taps. The ToF pixel (also called here a variable-tap photo-electric converter) comprises a photodiode 201 which collects incident photons and generates a photo current. The photodiode 201 is connected to first transfer gates TG1 to TG7, each first transfer gate TG1 to TG7 being related to a respective tap of the pixel. By opening one of the transfer gates TG1 to TG7, the charge produced by photodiode 201 is transferred onto a respective memory MEM1 to MEM7 where it is stored. By opening a respective second transfer gate TX1 to TX7, the charge collected onto a memory MEM1 to MEM7 is transferred to amplifier AMP and select transistor SEL to read out the amplified signal on a read out line VSL. A reset transistor RST is provided to reset the pixel. The photodiode 201 is also connected to an overflow gate OFG. The first transfer gates TG1 to TG7 and the overflow gate OFG are controlled according to the timing diagram as shown in Fig. 3. The activation of a new tap corresponds to an acquisition with a different phase shift applied on the reference/ emitted signal (a train of pulses, a sinusoid, etc). As in 2-tap iToF, the received signal will include both this phase shift and an additional phase shift induced by the time of flight.

It should be noted that according to techniques familiar to the skilled person, the first transfer gates TG1,..., TG7, the second transfer gates TX1,..., TX7 and the overflow gate OFG may be implemented e.g. as transistors, and the memories MEM1 to MEM7 may for example be implemented as capacitors.

It should further be noted that in Fig. 2, only the transfer gates TG1, TG7, the memories MEM1, MEM7 and the transfer gates TX1, TX7 of the first and seventh tap are shown. The respective elements of the second to sixth tap are indicated by dashed lines and are configured in the same way as the first and seventh tap.

It should further be noted that despite the embodiment of Fig. 2 shows a 7 -tap (N = 7) ToF pixel, in alternative embodiments, other numbers of taps may be foreseen in an N-tap pixel. For example there may be pixels with N = 3, 4, 5, 6, 8, 9, 10, 11, 12 or more taps.

Fig. 3 shows a timing diagram concerning the control of transfer gates TG1,..., TG7 and overflow gate OFG of the N-tap pixel described in Fig. 2. As shown at reference sign ILLUM, a laser pulse for illuminating a scene is activated for a predefined illumination period. Transfer gates TG1 to TG7 of an N-tap pixel with seven taps (N = 7) are controlled by respective control signals (also called "demodulation signals" or "mix signals"). According to this specific timing diagram of Fig. 3, each of the transfer gates TG1, ..., TG7 of the pixel is activated one after the other for a predetermined activation time window of a window length that is identical to the illumination period. The overflow gate OFG is opened after the last transfer gate TG7 has been closed. The described process (called "sub-integration cycle") is repeated multiple times (see also Fig. 8) to achieve measurements over a predefined integration cycle (the sensor may be also configured without use of OFG, for example the transfer gate TG1 may be opened right after the transfer gate TG7).

In the embodiment of Fig. 2 the length of the time window is equal for each transfer gate TG1, ..., TG7 and identical in length to the illumination period. It should however be noted that in alternative embodiments, there may be other time periods chosen. For example, the predetermined activation time window may be different for all or some of the transfer gates TG1,...,TG7. For example, the first activation time window may be longest for the first transfer gate TG1 and decreasing down to the last transfer gate TG7, or it may be shortest for the shortest for the first transfer gate TG1 and increasing up to the last transfer gate TG7. In another embodiment the activation time window may be equal for all transfer gates, while the width of the illumination signal may be lower or higher (though do not changed during integration period)

According to the timing diagram shown in Fig. 3, the mixing signals that are applied to transfer gates TG1,...,TG7 each have a respective phase delay Φ_i (i = 1, ..., 7) with respect to the illumination signal ILLUM. The respective iToF measurements provides N different measurements collected at the N taps corresponding to N electronic transmit delays (Τ_{E, n} in Eq. 9 below). These N measurements are used to determine an approximation (estimate) of a first harmonic of the correlation waveform with a Fourier Transform as described below in more detail. The tap information is considered as component information, i.e. samples of the correlation waveform between the reference/ emitted signal and the received signal. That is, obtaining depth information (as well as confidence and amplitude) is done in the Fourier domain.

After the integration cycle has ended, the read-out period (not depicted in Fig. 3) starts. The read- out of the charges that are collected at the seven taps 2031,...,203-7 (corresponding to the transfer gates TG1,...,TG7), which are stored in the capacitors MEM1,. . ,,MEM7, yields the corresponding signals q₀,..., q₆ is performed. The signals q₀,..., q₆ are then used for determining the phase value between the emitted and the received light.

The control signals according to Fig. 3 may be generated by a controller or any other means known to the skilled person. Gate rotary switching as depicted in a schematic way in Figs. 4a and 4b may symbolize this generation of the control signals. The gate rotary switching opens (i.e. activates) the transfer gates TG1, ... TG7 and the overflow gate OFG one after each other for a predefined period of time. Fig. 4a shows that the transfer gate TG1 is activated and the photo-electrons are directed to the transfer gate TG1. Fig. 4b shows that the transfer gate TG2 is activated and the photo-electrons are directed to the transfer gate TG2. Gate rotary switching as shown in Fig. 4 may also be used to control the transfer gates TX1, ... TX7 at the read-out side of the N-tap pixel.

Model for N-tap phase delay calculation

Consider an iToF camera pixel imaging an object at a distance D. A (differential) iToF pixel measurement ν(Τ_E, T_D) as obtained in the iToF pixel is a variable whose expected value μ (Τ_E, T_D) is given by where, t is the time variable, T_I is the exposure time (integration time), m(t) is the in-pixel reference signal which corresponds to the modulation signal (i.e. the emitted light signal) or a phase shifted version of the modulation signal (for example one of the control signals applied to transfer transistors TG1,...,TG7 as shown in Figs. 2 and 3). Φ_R (t, Τ_E, T_D) is the pixel irradiance signal which represents the reflected light (RL in Fig. 1) captured by the pixel. Τ_E represents a time variable indicative of the time delay between the in-pixel reference signal (modulation signal) and the emitted light ( in Fig. 1), and T_D is a time variable representing the time that it is required for the light to travel from the iToF camera (101 in Fig. 1) to the object (107 in Fig. 1) and back. Neglecting the parallax effect, the time variable T_D is given by: (Eq. 2) where D is the distance between the ToF camera and the object, and c is the speed of light.

The reflected light signal Φ _R(t, Τ_E, T_D) is a scaled and delayed version of the emitted light Φ_E(t — Τ_E). The pixel irradiance signal Φ _R(t, Τ_E, T_D) is given by: where Φ(T_D) is a real value scaling factor that depends on the distance D between the ToF camera and the object, and Φ_E(t — Τ_E — Τ_D) is the emitted light Φ_E(t — Τ_E) (16 in Fig. 1) additionally delayed with the time variable T_D .

In the context of iToF, both m(t) and Φ_E (t) are typically periodical signals with period (f_M being the fundamental frequency or modulation frequency generated by the timing generator (106 in Fig. 1). As T_I >> T_M, the expected signal μ(T_E, T_D) is also a periodical function with respect to the electronic delay T_E between in-pixel reference signal m(t) and optical emission Φ_E(t — Τ_E) with the same fundamental frequency f_M .

Writing μ(T_E, T_D) in terms of its Fourier Coefficients M_k yields (Eq. 4)

Note that due to the distance-dependent scaling of the light (factor Φ(Τ_D)), the expected signalμ(T_E, T_D) is not periodical with respect to the time-of-flight T_D.

From the above it is clear that the time-of-flight, and hence depth, can be estimated from the first harmonic H₁ (T_D) of μ(T_E, T_D): (Eq. 5)

From the first harmonic H_1,μ (T_D) the phase angle θ_1,μ(Τ_D) is obtained as (Eq. 6) with (Eq. 7)

Here, denotes the phase of a complex number z = re^iΦ (Eq. 8)

In practice, it is not feasible to evaluate H_1,μ (T_D) due to the presence of noise and due to the number of transmit delays.

Concerning the presence of noise, H_1,μ (T_D) is formulated in terms of the expected value μ(T_E, T_D) of signal measurements ν(T_E, T_D). Estimating this expected value from measurements may be performed by multiple repeated acquisitions (of static scene) to average out noise.

Concerning the number of transmit delays, H_1,μ (T_D) is given as an integral over all possible transmit delays T_E. Approximating this integral may require a high number of transmit delays.

Due to these reasons iToF systems measure an approximation of this first harmonic H_1,μ (T_D). This approximation uses N signal measurements (i.e. N different measurements collected at the N taps) v(Τ_{E, n}, T_D) (n = 0, ... , N — 1) corresponding to N electronic transmit delays Τ_{E, n}. A vectorized representation of this set of transmit delays is: t_E = [Τ_E,0 ... Τ_E,N-1]^T (Eq- 9) The approximation of the first harmonic H_1,μ (T_D) is obtained by an N-point EDFT (Extended Discrete Fourier Transform), according to (Eq.10) with n being the N-point EDFT bin considered. In standard iToF, n = 1. However, depending on the transmit delays selected, different values of n could be more appropriate. For simplicity and without loss of generality, we will assume n = 1 in the remainder of this disclosure: (Eq.11) This first harmonic estimate H_1,ν(Τ_D; t_E) is also referred to as IQ measurement (with I and Q the real resp. imaginary part of the first harmonic estimate). In order to stay close to iToF nomenclature, in the following H_1,ν(Τ_D; t_E) is denoted as “IQ measurement”. However, it is important to remember that an IQ measurement is an estimate of the first harmonic H_1,μ (T_D) of the expected measurement (as function of transmit delay). From the first harmonic estimate H_1,ν(Τ_D; t_E) of equation Eq.11, the phase value ^ between the emitted and the received light is obtained as (Eq.12) with Im() and Re() being respectively the imaginary part and the real part operator, andarctan2 being the 4-quadrant inverse tangent function. Due to the statistical nature of the signal measurements ν(Τ_E,n, Τ_D)_, the IQ measurement H_1,ν(Τ_D; t_E) is a random variable with the following expected value (Eq.13) This expected value is here referred to as expected IQ measurement. With F(1) = E (H_1,ν(Τ_D; t_E)) (Eq.14) denoting the IQ measurement, and q_n = μ(Τ_E,n, Τ_D) (Eq.15) denoting the N measurements (samples) obtained by the pixel at respective phases, this gives:

Specifically, once the Fourier transform is computed on the samples of the correlation waveform (tap values in the N-tap case), the first harmonic will contain I and Q information as its real and imaginary part, respectively.

Based on equation Eq. 12 the phase value Φ between the emitted and the received light is obtained as:

Fig. 5 shows in a schematic way a sample of a correlation waveform between the reference (mix) signal and the received signal for an N-tap pixel with N = 8, obtained according to a timing diagram as shown in Fig. 3 (but with N=8 instead of N=7). The correlation waveform 502 (solid line) between reference (mix) signal and received signal is approximated by the acquired samples q₀, ... , q₇ via the measurements obtained at the eight taps of the pixel. According to the timing diagram as shown in Fig. 3, the samples q₀, ... , q₇ are distributed equidistant in time and phase. The correlation waveform 501 (dashed line) between reference (mix) signal and emitted signal has no time of flight contribution. The time difference Τ_D between the correlation waveforms 501 and 502 corresponds to the distance between the ToF camera and the object (see Eq. 2).

Fig. 6 shows in a schematic way the determination of the phase value Φ between the emitted and the received light from the IQ measurement F(1) as set out in Eq. 17. With the acquired samples q₀, ... , q₇ (see Fig. 5) a N-point Fast Fourier Transform (here: N=8) is performed as described in Eq. 16 based on the complex valued summands 601,.., 607. Here a “split of identity” is applied which makes the time-of-flight estimation independently of scale and offset. This yields the IQ measurement F(1). The imaginary part Im(F(1)) of the IQ measurement F(1) is denoted as Q component and the real part Re(F(1)) of the IQ measurement F(1) is denoted as the I component of the IQ measurement F(1). The phase value Φ is obtained from the Q component and I component of the IQ measurement F(1) according to trigonometric principles.

Based on the phase value Φ obtained from the measurements q₀, ... , q_N-1 at a pixel according to equation Eq. 17 the corresponding depth value d for the pixel is determined as follows: with f_M being the modulation frequency of the emitted signal and c being the speed of light. The amplitude value amp for the N-tap pixel, which is an indicator of the signal strength and, implicitly, of the noise level, is determined as follows: (Eq. 19)

The confidence value conf for the N-tap pixel is determined as follows: conf = |lm(F(1))| + |Re(F(1))| (Eq. 20)

Still further, in other embodiments other scene descriptors may be determined from the measurements.

Ambient light suppression based on “empty” taps

Providing N-tap pixels with more than two taps (N > 2) as described above allows for new approaches to ambient light suppression.

The embodiments described below in more detail refer to a background light suppression algorithm. One basic means of achieving the background light suppression is to take the average value across so-called “empty” taps (i.e. taps not containing active light) and subtracting this from each tap value.

After this subtraction, the “empty” taps may still contain various types of noise (system noise, shot noise, etc.) which may influence the results of the Fourier transform and, implicitly, the first harmonic. To counter this, once the average across “empty” taps has been subtracted from all taps, the remaining value corresponding to each tap may be set to zero (and thus eliminating measurement noise produce by ambient light). Finally, the resulting measurement values provided by the taps are processed with the Fourier transform of Eq. 16 above and the depth computation is performed according to Eq. 20 above.

Fig. 7 schematically shows a timing diagram for the control of the transfer gates of an N-tap pixel (here N = 7 + OFG). The timing diagram shows nine rows, where the first row shows the illumination signal ILLUM over time t, the second to eights rows show the control signals applied to transfer gates TG1,...,TG7 over time t, and the nineth row shows the control signal applied to the overflow gate OFG over time t.

The illumination signal ILLUM (also called emitted signal) as emitted by an illumination unit of a iToF camera (110 in Fig. 1) is a pulse train which comprises pulses 301 with a predetermined pulse width t_p. The pulse width may for example be 5ns and the duty cycle of the pulse train may for example be 1%. The emitted light pulse 301 is reflected by the scene and received as incident light INCIDENT by the N-tap pixel as a light pulse 303 with a certain time delay as compared to the emitted pulse 301.

Together with the light pulse 303 the incident light INCIDENT comprises an ambient light contribution 304. Ambient light 304 may for example be background light from the sun or a room illumination, or scattered light from an illumination unit (110 in Fig. 1). The ambient light 304 is considered here as constant over time with an amplitude smaller than the emitted light pulse.

The first to seventh transfer gate control signals control the activation and deactivation of first to seventh transfer gates TG1,...,TG7, respectively, as described above with regard to Fig. 3. When a transfer gate control signal has a zero amplitude the corresponding transfer gate is closed and when a transfer gate control signal has a non-zero amplitude the corresponding transfer gate is open. As described with regard to Fig. 3 above, the control signals for the first to seventh transfer gates TG1,...,TG7 are respective pulse train signals. The respective pulses 302-1 to 302-7 have a predetermined pulse width which is equal to the pulse width t_p of the illumination signal and the pulses 302-1 to 302-7 are all phase-shifted with respect to each other. The time shift between the control signals for transfer gates TG1,...,TG7 is equal to the activation pulse width t_p and translates to a respective phase shift. The control signal for transfer gate TG1 is in synchronization with the illumination signal ILLU. Accordingly, its phase shift is zero. In the embodiment of Fig. 7, the pulse trains applied to transfer gates TG1,...,TG7 also have the same duty cycle. According to the timing diagram described in Fig. 7 for the control of the N-tap pixel (with N = 7, i.e. more than two taps) the transfer gates are opened for a predetermined time period one after the other. This can be seen as adding a concept a time-domain histogram to the classical 2-tap iToF. The acquisition of the signals is done in the time domain. The processing of the signals is however (see description below) done in the frequency domain (Fourier domain).

After t_max = 7* t_p (this is equal to the modulation period) the overflow gate OFG is activated. The overflow gate control signal OFG controls the activation and deactivation of the overflow gate OFG. The overflow gate control signal OFG is also a pulse train signal and it comprises a pulse 302- 8. The overflow gate OFG accumulates and collects all remaining and unnecessary charges generated due to received light by the pixel outside the time period t_max (i.e. the time period where all transfer gates TG1, ...,TG7 are closed). The pulse width of pulse 302-8 may be much longer than the pulse width t_p of the emitted light, for example 465ns, and the duty cycle of the overflow flow control signal may accordingly also be higher than the duty cycle of the transfer gate control signals, for example 90%. One sub-integration cycle is defined as lasting from the beginning of the activation of the first transfer gate TG1 until the deactivation of the overflow gate OFG. The sub- integration cycle may be repeated several times, for example 500, or 800 times, depending on the signal-to-noise (SNR) target. All sub-integration cycles together form one integration cycle. After one integration cycle is finished the read-out of the stored charges in the capacitors is read out (see also Fig. 12) as corresponding measurements q₀,..., q₆ (samples) for further processing and depth calculation (see Eq. 16 to 20 above).

The ambient light 304 is received by the N-tap pixel is constant. Therefore, each tap of the N-tap pixel receives a respective part 304-1, ..., 304-7 of the ambient light 304 when transfer gate TG1,...,TG 7 is open.

A first part 303-1 of the light pulse 303 related to the reflected light is received by the N-tap pixel while transfer gate TG3 is open. The charges generated due to this first part 303-1 of the light pulse 303. A second part 303-2 of the light pulse 303 is received by the N-tap pixel while transfer gate TG4 is open. The transfer gates TG1, TG2, TG5, TG6 and TG 7 are closed while the light pulse 303 is received by the N-tap pixel. Therefore, the corresponding taps 203-1, 203-2, 203-5, 203-6 and 203-7 do not collect any charges caused by light pulse 303. They receive only ambient light 304. When the pulse width of the emitted light t_p is smaller or equal to the transfer gate activation pulse width t_p, there are at least N-2 taps which do not collect any charges generated by the emitted light pulse (so called active light) but only ambient light. These at least N-2 taps which do not collect any active light are called "empty" taps. The remaining taps (i.e. the taps containing charges generated by the emitted light pulse) are called "non-empty" taps. As long as the number of taps N is greater than or equal to three, there is always at least one empty tap.

The ambient light contributions 304-1, ..., 304-7 received at the taps of the N-tap pixel may degenerate the quality of the depth measurement that is obtained based on the signals generated by the taps of the N-tap pixel.

Taps may be classified as empty taps by determining which of the taps do provide a corresponding measurement q_i which is below a predetermined threshold qthreshold- That is each tap whose corresponding measurement q_i ∈ q₀, ... , q_N-1 fulfils the condition q_i < q_threshold is classified as empty tap. Otherwise, a tap is classified as non-empty tap. Alternatively, empty taps may be identified if no peak is observed in the number of detected photons as compared to all tap information.

Then, the average signal over all empty taps produced by the ambient light is (Eq. 21) with N_empty being the total number of empty taps: (Eq.22) Then, ambient light corrected measurement values are determined by subtracting the average signal from the measurements q₀, ... , q_N-1: (Eq.23)

These ambient light corrected measurement values can be used to determine the Fourier transform of Eq. 16 as: (Eq. 24) and in accordance with Eq. 17 to obtain an ambient light corrected phase value according to: (Eq. 25).

As the empty taps may still contain various types on noise (system noise, shot noise, etc.) which may influence the results of the Fourier transform and, implicitly, the first harmonic the ambient light, the measurements for the empty taps may be set to zero (Eq.26) and the resulting set of measurement values may be used in Eqs. 24 and 25 to compute the ambient light corrected phase value .

The N-tap pixel that applies the ambient light suppression algorithm described above which is based on empty taps has an improved SNR as compared to a 2-tap pixel without ambient light suppression based on empty taps.

In other embodiment the ambient light suppression may be based on techniques other than subtracting the average of the empty taps, for example techniques which involve passing the “empty” tap information through a Fourier transform.

As described above the acquisition of the signals is done in the time domain while the processing of the signals is done in the frequency domain (Fourier domain). Therefore, the N-tap solution combines the strengths of dTof and 2-tap iToF.

The N-tap pixel preserves a good resolution (such as VGA). It may provide a mm-accuracy below Im. Between Im and 5m, the N-tap pixel solution described above has extended de-aliasing capabilities, for example via peak detection in the histogram of the signal for peak or range gating (see below) etc., while still working in single frequency. Moreover, the N-tap pixel has a better SNR than for example the 2-tap iToF due to ambient light rejection based on information from the empty taps (i.e. taps where active light is not detected). Above 5m, close-object scattering can be mitigated using the histogram operating principle in dToF.

Frame structure

Fig. 8 shows an embodiment of a frame structure of a 7 -tap iToF pixel. Signal 1201 comprises an alternating structure of depth frames (depth frames n, n+1, n+2, ...) and idle periods at a frame rate of 30 frames per second (t_f = 33.3 ms).

A depth frame (here for example depth frame n+1) comprises a reset period, followed by an integration period, again followed by a read-out period. During the integration period a sequence 1204 of light pulses are emitted by the illumination unit of the iToF imaging system. The illumination period may for example last 400μs and may comprise 800 pulses of 5ns, which yields a duty cycle of 1%. The read-out period may last 5.3ms and the read-out may be performed MIPI standard compliant.

Each pulse 1205 of the sequence 1204 of light pulses defines a sub-integration cycle. Within a sub- integration cycle the first to the seventh transfer gates (TSG1, ..., TSG7 in Fig. 2) which correspond to taps 0 to 6 are active one after the other followed by an activation of the overflow gate OFG. One sub-integration cycle is defined as lasting from the beginning of the activation of the first transfer gate TG1, corresponding to tap 0, until the deactivation of the overflow gate OFG. The sub-integration cycle has a time span t_sub which may be t_sub=500ns, which equals the period of one emitted light pulse (although the pulse width is only 1% of the pulse period). The activation pulse width t_p of the transfer gates may be equal and may for example be t_p =5ns. The pulse width of the emitted light may be equal or smaller than the activation pulse width t_p of the transfer gates. The transfer gates may be activated directly each of the other, which yields a combined activation time of as t_max = 35ns per sub-integration cycle. The overflow gate may be activated the remaining time of the sub-integration cycle which may be 465ns. The combined activation time t_max of all transfer gates and corresponding taps defines the (radial) range in which the iToF camera can record objects. The emitted light pulse 1205 may have a time delay t_b (phase shift) with respect to the activation of the first transfer gate, corresponding to tap 0. The time delay Rmay for example bet_b= Ins. The number of sub -integration cycles per integration period may be determined according to the SNR target.

Other than with previously known iToF pixels (for example a 2-tap pixel), in the case of the N-tap pixel a single integration period may be enough for full depth acquisition. Therefore, less or even no motion blurring (due to a movement of a recorded object during read-out periods) may occur. And the N-tap pixel is therefore natively motion robust. The number of taps and the pulse width t_p of the transfer gates determines the range of the iToF camera. Therefore, as compared to the 2-tap pixel with the same range, in the N-tap pixel the pulse width and/ or the emitted light pulse width t_p may be shorter and therefore the duty cycle may be smaller. Therefore, with the N-tap pixel it is possible to get a higher peak optical power without increasing the size of the illumination unit (for example a VCSEL array), while avoiding saturation at close distances and while keeping eye safety. Therefore, a better SNR can be achieved.

Additional Features enabled by the N-Tap pixel structure

There are a series of system features unique to N-tap pixels. These additional system features unique to N-tap pixels include close-range gating (which may yield reduced multipath interference and scattering for close objects) and far-range gating (also covering suppression of echo from objects located further than a certain distance), dynamic range extension for short distances. Still further, depth dependent selective/adaptative filtering and binning are also enabled and self-calibration of a mismatch between taps (in terms of gain, phase, and correlation waveform) may be improved by using the scattering in the cover glass as a calibration setup.

In the following close-range gating and far-range gating, dynamic range extension, and range-specific binning are described in more detail.

Close and far-range gating and selective range depth acquisition

With the N-tap pixel, a so called range gating is possible due to the capturing of depth information in time domain. During a gating period, all unwanted photoelectrons are drained by the overflow gate OFG. Range gating is performed on a radial detection range of the iToF camera. Both, close and far-ranges can be gated.

Fig. 9 shows a timing diagram for a sub -integration cycle of a range gated iToF pixel with seven taps. A light pulse 1301 is emitted onto a scene. The emitted light pulse 1301 is received by the image sensor as close object rejection light pulse 1302, as far object rejection light pulse 1304 and as a usable range object reflection 1303.

Close-range gating is achieved by advancing (delaying) the illumination pulse 1301 by the time t_b from the sub -integration start moment, that is before the transfer gate TG1 corresponding to the tap 0 is activated. During the time t_b, the overflow gate OFG is enabled such that it can drain out the photoelectrons which stem from close object rejection light pulse 1302 and ambient light reflected by objects in the close-range. The resulting close-range distance d_cr which is gated-out is determined as (with c being the speed of light). For example, if t_b = 1ns, the gated-out close-range distance is d_cr-0.15m. Far-range gating is achieved by enabling the overflow gate OFG for a predetermined time t_e overlapping with the end of a standard sub -integration time (here, the nomenclature “standard” implies that no range gating is performed). That is, while for close-range gating the overflow gate OFG is enabled beyond the beginning of the standard sub-integration time, for far-range gating the overflow gate OFG is enabled beyond the end of the standard sub-integration time. This is equivalent to shortening the sub-integration time by the predetermined time t_e. The resulting far- range distance which is gated-out is determined as: .

The usable range d_ur is defined by the pulse width t_p of the transfer gate opening and the number of taps n, i.e. .

Fig. 10 shows a radial detection range of an iToF camera with close-range and a far-range gated areas. An illumination unit 1401 is illuminating a scene along a radial direction 1403. An image sensor 1402 is capturing reflected light from the scene. The usable range d_ur with a pulse width t_p = 5ns and seven taps is d_ur = 4.5m. The close-range distance is d_cr = 0.15m. This means that reflections from objects within the close-range gated radial area 1406, from distances of 0cm to 15cm, are not recorded by the taps. The total operating time for the overflow gate OFG is (t_e + t_b) and the corresponding close + far-range distance which is gated-out is determined as: . With t_e = 200ns this yields, d_cfr = 30.15m. This means that reflections from objects within the far-range gated radial area 1407, i.e. from a radius of 4.65m to 30.15m, are not recorded by the taps.

Some applications such as AR gaming may benefit from a selective range depth acquisition, for example an even further reduced usable range d_ur. This would allow for faster and more targeted processing as well as preventing foreground objects from appearing in the depth map. Occlusions appearing in the depth map will not be solved through selective depth ranging. That is, any objects in the usable range that are occluded by the eliminated foreground objects will not appear in the depth map even after depth ranging, as they are still behind the foreground objects in the real scene.

A desired reduced usable range may be obtained by close-range gating either with less active taps (tap skipping) or by shortening the activation pulse width of the transfer gate and thereby the overall sub-integration time (“high precision”).

Fig. 11 shows another embodiment of radial detection range of an iToF camera with close-range and far-range gated areas. An illumination unit 1501 is illuminating a scene along a radial direction 1503. An image sensor 1502 is capturing reflected light from the scene. The usable range d_ur is set to be in a range between 3.0m and 5.25m. The close-range distance is d_cr = 3m, which means reflections from objects within the close-range gated radial area 1504, from distances of 0m to 3m, are not recorded by the taps. The far-range gated radial area 1505 begins at 5.25m.

Fig. 12 shows two timing diagrams for a sub-integration cycle of a range gated iToF image sensor with seven taps using a tap skipping and a high precision method. In the upper timing diagram 1600, a light pulse 1601 is emitted a predetermined time t_b before the sub-integration time starts. The emitted light pulse 1601 is received by the image sensor as close object rejection light pulse 1602 and as a usable range object reflection 1603. Between the end of the sub-cycle and the emitting of new light pulse 1609 a time span t_e the overflow gate OFG is activated. The time delay between the emitting of the light pulse and the activation of the first transfer gate corresponding to tap 0, is set as t_b=20ns. Thereby, from 7 existing taps only 4 taps, i.e. taps 3 to 6, are active taps (“tap skipping method”). The resulting close-range distance is d_cr = 3m. The usable range is d_ur = 2.25m within a radial area from 3m to 5.25m. The reduced number of active taps allows for less power consumption for the MIX drivers and the depth processing.

In the lower timing diagram 1606 a light pulse 1607 is emitted at the same predetermined time t_b before the sub-integration time starts, just as in the upper timing diagram 1600 with the reduced number of taps. The emitted light pulse is received by the image sensor as usable range object reflection 1608 at same time as it is received in upper timing diagram 1600 with the reduced number of taps and also the time t_e between sub-integration end and emitting of new light pulse 1609 and the time delay between the emitting of the light pulse and the activation of the first transfer gate corresponding to tap 0 (set as t_b = 20ns) is the same as in upper case. However, in this case all seven taps are active but the activation time length t_p = 2.5ns of the transfer gates is shorter than in the upper case, such that the overall sub -integration time in both case is equal (“high precision method”). This results in the same close-range distance is d_cr = 3m. and usable range distance is d_ur = 2.25m. However, in this case with all seven taps being active an increased precision within the usable range d_ur is achieved while depth noise is halved. This still results in decreased power consumption due to the shorter integration time.

Dynamic range extension

Another system feature achievable with the N-tap pixel is dynamic range extension. This is particularly useful when dealing with saturation due to close foreground objects. Such objects may generate a very strong reflected signal leading to fast saturation of the taps. The ability to measure the distance of far-range objects is constrained by the minimum measurable distance and maximum pixel saturation Q_sat, which is limited. Dynamic range extension is achieved by opening taps which are measuring close distances only for a part of the integration time, thus significantly extending the equivalent dynamic range by ensuring that theses taps do not get saturated. The process results in more data to describe objects in the usable range. Thus, more details can be seen for objects in the far-range while fine details remain clear for objects in the close-range while at the same time avoiding saturation.

Fig. 13 shows four timing diagrams for four respective sub -integration cycles of an iToF pixel with seven taps using dynamic range extension. The first timing diagram 1701 for the first sub-integration cycle shows that initially all seven taps 0,...,7 are activated (one after the other) during the sub- integration cycle. A second timing diagram 1702 shows a new acquisition during a second sub- integration cycle, where tap 0 stays closed all the time and the overflow gate OFG is instead open. A third timing diagram 1703 shows a new acquisition during a third sub-integration cycle where taps 0 and 1 stay closed all the time and the overflow OFG is open instead. A fourth timing diagram 1704 shows a fourth sub -integration cycle where taps 0, 1 and 2 stay closed all the time and the overflow OFG is open instead.

This leads to tap 0 being active only 1/16 of its normal active time of the total integration time T_int, tap 1 being active only 1/4 of its normal active time of the total integration time T_int, tap 2 and 3 being active only 1/2 of their normal active time of the total integration time T_int, and only taps 4, 5 and 6 being active the full time of their normal active time of the total integration time T_int.

In order to enable high dynamic range extension, the tap signal values need to be normalized during depth processing. This above described case would imply x16 for tap 0, x4 for tap 1, x2 for taps 2 and 3.

Depending on the minimum and the maximum distance requirements, other partitions in time may be applied.

Fig. 14 shows a graph depicting the acquired current in the different taps of a 7-tap system depending on the distance of a detected object. The x-axis of the graph shows the distance in m and the y-axis shows the acquired charge in the taps in C (Coulomb). The seven bins 0 to 6 correspond to the seven taps 0 to 6. The dashed line 1801 shows the acquired charge in taps 0 to 6 during normal (non-range extended) operations. The dotted line 1802 shows the acquired charge in the taps 0 to 6 during range extended operations. In both cases the first tap 0 acquires the most charge and the acquired charge decreases from the first tap 0 to the last tap 6. The line 1803 shows the maximal tap saturation Q_sat that still can be resolved by the pixel. Any acquired charge that is above the maximal tap saturation Q_sat leads to objects not being detectable anymore.

The acquired charge in normal (non-range extended) operations shown by the dashed line 1801 is above the maximal tap saturation Q_sat in tap 0. By applying the range extension mode, shown as the dotted line 1802, the acquired charge in tap 0 can be reduced below the maximal tap Q_sat, so that tap saturation can be prevented.

Range-specific binning

Another system feature achievable with the N-tap pixel is the support for an implementation of range-specific binning. More specifically, since individual taps collect information from different ranges, certain taps can be binned independently from others. Depending on the range, different binning levels can be set for different (groups of) taps.

Fig. 15 shows a timing diagram for a sub-integration cycle of an iToF image sensor with seven taps using range specific binning. An emitted light pulse 1901 is received as usable range object reflection 1902 by the image sensor. The taps 0 to 6 are activated one after the other. The first three taps 0 to 2 which acquire close-range objects are grouped as a first group 1906. The fourth and fifth tap 3 and 4, which acquire mid-range objects, are grouped as a second group 1907. The sixth and seventh tap 5 and 6, which acquire far-range objects, are grouped as a third group. The first group 1906 is set to acquire a full resolution depth frame by not applying any binning. The second group 1907 applies a 2x2 binning (meaning that 2x2 pixels are considered as one pixel) to obtain a higher SNR with a small impact on resolution. The third group 1908 applies 4x4 binning (to opt for background removal) .

Fig. 16 shows a radial detection range of an iToF camera with range specific binning. An illumination unit 2001 is illuminating a scene along a radial direction 2003. An image sensor 2002 is capturing reflected light from the scene. The usable range d_ur may be set to be in a range between 0.15m and 4.65m. Due to the applied range-specific binning, there are different areas where objects are detected with different resolutions within usable range d_ur. In a first area 2005 objects may be detected with a high resolution, for example VGA. In a second area 2006 objects may be detected with a reduced resolution, for example QVGA. In a third area 2007 objects may be detected with a low resolution, for example QQVGA.

Implementation

Fig. 17 schematically describes an embodiment of an iToF device that can implement the processes performing depth measurement (and other scene descriptors) with an N-tap pixel and perform ambient light suppression, close-range gating, dynamic range extension, and range-specific binning. The electronic device 2100 may further implement all other processes of a standard iToF/ spot ToF system. The electronic device 2100 comprises a CPU 1201 as processor. The electronic device 2100 further comprises an iToF sensor 2106 connected to the processor 2101. The processor 2101 may for example implement performing a depth measurement of a N-tap pixel, as described with regard to Figs. 7 to 11. The electronic device 2100 further comprises a user interface 2107 that is connected to the processor 2101. This user interface 2107 acts as a man-machine interface and enables a dialogue between an administrator and the electronic system. For example, an administrator may make configurations to the system using this user interface 2107. The electronic device 2100 further comprises a Bluetooth interface 2104, a WLAN interface 2105, and an Ethernet interface 2108. These units 2104, 2105 act as 1/ O interfaces for data communication with external devices. For example, video cameras with Ethernet, WLAN or Bluetooth connection may be coupled to the processor 2101 via these interfaces 2104, 2105, and 2108. The electronic device 2100 further comprises a data storage 2102, and a data memory 2103 (here a RAM). The data storage 2102 is arranged as a long-term storage, e.g. for storing parameters for one or more use-cases, for recording iToF sensor data obtained from the iToF sensor 2106 the like. The data memory 2103 is arranged to temporarily store or cache data or computer instructions for processing by the processor 2101.

It should be noted that the description above is only an example configuration. Alternative configurations may be implemented with additional or other sensors, storage devices, interfaces, or the like.

It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is, however, given for illustrative purposes only and should not be construed as binding.

It should be noted that despite that in the embodiment of Fig. 7, the pulse widths of the transfer gate control signals are described as being equal to the pulse width t_p, of the emitted light pulse, in alternative embodiments the pulse widths of the transfer gate control signals may also be shorter or longer than the pulse width t_p of the emitted light pulse. However, to achieve a high number of "empty" taps, it is preferred that the pulse widths of the transfer gate control signals are equal to or shorter than the pulse width t_p of the emitted light pulse. In yet another embodiment all or some pulse widths of the transfer gate control signals may be different from each other.

In yet alternative embodiments the phase shift between the first transfer gate control signal and the illumination signal ILLU may be non-zero. For example, the emitted light pulse 301 and the pulse 302-1 of the first transfer gate transistor may have a time/ phase shift with respect to each other. This time span may be used for close-range gating (see Fig. 12).

In another embodiment the duty cycle of the illumination signal ILLU may be shorter or longer than 5ns for example Ins or 10ns. In yet another example the duty cycle may be below or above 1% for example 0.3% or 0.5% or 5% or 10%. In anomer embodiment the illumination signal may be a sinusoid.

In yet alternative embodiments there may be other numbers of taps (for example 3, 4, 5, 6, 8, 9, 10, 11, 12 or more) and the timing diagram provided in Fig. 7 may be adapted correspondingly.

The embodiments of Figs. 2, 3 and 7 relate to an N-tap pixel with seven taps with respective transfer gate plus one overflow gate. It should however be noted that the aspect of providing an overflow gate in the embodiments described above is an optional aspect. In alternative embodiments, for example an eighth tap with respective transfer gate may be foreseen instead of the overflow gate.

It should also be noted that the division of the electronic device of Fig. 17 into units is only made for illustration purposes and that the present disclosure is not limited to any specific division of functions in specific units. For instance, at least parts of the circuitry could be implemented by a respectively programmed processor, field programmable gate array (FPGA), dedicated circuits, and the like.

All units and entities described in this specification and claimed in the appended claims can, if not stated otherwise, be implemented as integrated circuit logic, for example, on a chip, and functionality provided by such units and entities can, if not stated otherwise, be implemented by software.

In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be appreciated that a computer program providing such software control and a transmission, storage or other medium by which such a computer program is provided are envisaged as aspects of the present disclosure.

Note that the present technology can also be configured as described below:

(1) An electronic device comprising circuitry configured to a determine, according to a Time-of- Flight, ToF, principle, a depth measurement (d) in the frequency domain based on three or more measurements (q₀,..., q_{N -1}) which correspond to respective three or more taps of a pixel.

(2) The electronic device of (1), wherein the circuitry is configured to determine a discrete Fourier transform (F(1)) based on the three or more measurements (q₀,..., q_{N -1}) related to the taps of the pixel, and to determine the depth measurement (d) based on the discrete Fourier transform (F(1)).

(3) The electronic device of (1) or (2), wherein the circuitry is configured to determine a discrete Fourier transform (F(1)) according to where N is the total number of taps of the pixel and q_i are the corresponding measurements provided by the respective taps of the pixel.

(4) The electronic device of anyone of (1) to (3), wherein the circuitry is configured to determine the depth measurement (d) based on a real part (Re(F(1))) and an imaginary part (Im(F(1))) of a Fourier transform (F(1)).

(5) The electronic device of anyone of (1) to (4), wherein the circuitry is configured to determine a depth measurement (d) as where Im(F(1)) and Re(F(1)) are respectively the imaginary part and the real part of a Fourier transform (F(1)) and arctan2 is the is the 4-quadrant inverse tangent function.

(6) The electronic device of anyone of (1) to (5), wherein the circuitry further comprises three or more transfer gates (TG1,...,TG7) each corresponding to one of the three or more taps (208-

1,..., 208-7).

(7) The electronic device of (6), wherein the circuitry is configured to open the transfer gates (TG1,...,TG7) of the respective taps of the pixel one after the other for a predetermined time.

(8) The electronic device of (6) or (7), wherein the circuitry is configured to open the transfer gates (TG1,...,TG7) according to a predetermined pulse width which is equal to the pulse width (t_p) of pulses of an illumination pulse train.

(9) The electronic device of anyone of (1) to (8), wherein the circuitry is configured to generate an illumination pulse train which has a duty cycle equal or lower than 1%.

(10) The electronic device of (6) to (8), wherein in the circuitry further comprises an overflow gate (OFG) which is activated in a sub-integration cycle after the transfer gates (TG1,...,TG7) have been activated.

(11) The electronic device of anyone of (1) to (10), wherein the circuitry is configured to determine ambient light corrected measurements based on the measurements (q₀,..., q_{N -1}) obtained by the respective taps of the pixel.

(12) The electronic device of anyone of (1) to (11), wherein the circuitry is configured to determine empty taps (302-1, 302-2, 302-5, 302-6, 302-7) of the pixel to perform ambient light suppression. (13) The electronic device of anyone of (1) to (12), wherein the circuitry is configured to determine ambient light corrected measurements by subtracting an average signal over empty taps from the measurements (q₀,..., q_{N -1}) obtained by the respective taps of the pixel. (14) The electronic device of anyone of (1) to (13), wherein the circuitry is configured to set measurements that correspond to empty taps to zero.

(15) The electronic device of anyone of (1) to (14), wherein the circuitry is configured to perform close-range gating.

(16) The electronic device of anyone of (1) to (15), wherein the circuitry is configured to perform far-range gating.

(17) The electronic device of anyone of (1) to (16), wherein the circuitry is configured to perform dynamic range extension.

(18) The electronic device of anyone of (1) to (17), wherein the circuitry is configured to perform range-specific binning. (19) A method comprising determining, according to a Time-of-Flight, ToF, principle, a depth measurement (d) in the frequency domain based on three or more measurements (q₀, ... , q_N-1) which correspond to respective three or more taps of a pixel.

Claims

1. An electronic device comprising circuitry configured to a determine, according to a Time-of- Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.

2. The electronic device of claim 1, wherein the circuitry is configured to determine a discrete Fourier transform based on the three or more measurements related to the taps of the pixel, and to determine the depth measurement based on the discrete Fourier transform.

3. The electronic device of claim 2, wherein the circuitry is configured to determine a discrete Fourier transform according to where N is the total number of taps of the pixel and q_i are the corresponding measurements provided by the respective taps of the pixel.

4. The electronic device of claim 1, wherein the circuitry is configured to determine the depth measurement based on a real part and an imaginary part of a Fourier transform.

5. The electronic device of claim 1, wherein the circuitry is configured to determine a depth measurement as where Im(F(1)) and Re(F(1)) are respectively the imaginary part and the real part of a Fourier transform (F(1)) and arctan2 is the is the 4-quadrant inverse tangent function.

6. The electronic device of claim 1, wherein the circuitry further comprises three or more transfer gates each corresponding to one of the three or more taps.

7. The electronic device of claim 6, wherein the circuitry is configured to open the transfer gates of the respective taps of the pixel one after the other for a predetermined time.

8. The electronic device of claim 6, wherein the circuitry is configured to open the transfer gates according to a predetermined pulse width which is equal to the pulse width of pulses of an illumination pulse train.

9. The electronic device of claim 1, wherein the circuitry is configured to generate an illumination pulse train which has a duty cycle equal or lower than 1%.

10. The electronic device of claim 6, wherein in the circuitry further comprises an overflow gate (OFG) which is activated in a sub-integration cycle after the transfer gates have been activated.

11. The electronic device of claim 1 wherein the circuitry is configured to determine ambient light corrected measurements based on the measurements obtained by the respective taps of the pixel.

12. The electronic device of claim 1, wherein the circuitry is configured to determine empty taps of the pixel to perform ambient light suppression.

13. The electronic device of claim 1, wherein the circuitry is configured to determine ambient light corrected measurements by subtracting an average signal over empty taps from the measurements obtained by the respective taps of the pixel.

14. The electronic device of claim 1, wherein the circuitry is configured to set measurements that correspond to empty taps to zero.

15. The electronic device of claim 1, wherein the circuitry is configured to perform close-range gating.

16. The electronic device of claim 1, wherein the circuitry is configured to perform far-range gating.

17. The electronic device of claim 1, wherein the circuitry is configured to perform dynamic range extension.

18. The electronic device of claim 1, wherein the circuitry is configured to perform range- specific binning.

19. A method comprising determining, according to a Time-of-Flight, ToF, principle, a depth measurement in the frequency domain based on three or more measurements which correspond to respective three or more taps of a pixel.