WO2014146978A1 - Distance determination method - Google Patents

Distance determination method Download PDF

Info

Publication number
WO2014146978A1
WO2014146978A1 PCT/EP2014/055068 EP2014055068W WO2014146978A1 WO 2014146978 A1 WO2014146978 A1 WO 2014146978A1 EP 2014055068 W EP2014055068 W EP 2014055068W WO 2014146978 A1 WO2014146978 A1 WO 2014146978A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
unwrapped phase
combination
phase hypotheses
hypotheses
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2014/055068
Other languages
French (fr)
Inventor
Laurent Lamesch
Bruno Mirbach
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
IEE International Electronics and Engineering SA
Original Assignee
IEE International Electronics and Engineering SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by IEE International Electronics and Engineering SA filed Critical IEE International Electronics and Engineering SA
Priority to DE112014001541.5T priority Critical patent/DE112014001541T5/en
Priority to CN201480016730.9A priority patent/CN105143916B/en
Priority to US14/778,918 priority patent/US10795021B2/en
Publication of WO2014146978A1 publication Critical patent/WO2014146978A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06Systems determining position data of a target
    • G01S17/08Systems determining position data of a target for measuring distance only
    • G01S17/32Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G01S17/36Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/491Details of non-pulse systems
    • G01S7/4912Receivers
    • G01S7/4915Time delay measurement, e.g. operational details for pixel components; Phase measurement
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C15/00Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
    • G01C15/002Active optical surveying means

Definitions

  • the present invention generally relates to a method for determining a distance based upon a plurality of phase measurements, made at different wavelengths, each of the phase measurements, taken individually, being ambiguous in the sense that it indicates the distance only up to an integer multiple of the respective wavelength.
  • Distance measuring equipment often relies on the time-of-flight (TOF) measurement principle.
  • Distance (also: depth) information is in this case obtained by emitting modulated light into the direction of the target and measuring the phase shift of the modulation between the emitted and the reflected light.
  • the measured phase is proportional to the distance between the measurement equipment and the target, modulo the wavelength of the modulation. This relationship may be expressed as: d - A - f
  • is the modulation phase in radians
  • d is the distance
  • f is the modulation frequency
  • c is the speed of light.
  • the extra factor 2 (4 ⁇ instead of 2 ⁇ ) in the first argument of the mod function is due to the fact that the light travels twice the distance between the distance measuring equipment and the target.
  • the target may be located at greater distances from the target than d u , any phase measurement will be comprised in the interval [0, 2 ⁇ [: one says that the measurements are "wrapped" into the unambiguity range.
  • phase unwrapping retrieves the actual distance from the ambiguous measurement, or, otherwise stated, determining the integer number of modulation wavelengths to add to the distance value d' in order to arrive at the actual distance.
  • the usable distance measurement range is bounded by the unambiguity range unless assumptions are made on the smoothness of the scene (neighbouring pixels) and on the smoothness of the evolution in time of a distance measurement. Such assumptions cannot be made in all circumstances and for all applications. Specifically, in an adaptive driver assistance system, such assumptions are normally not permitted.
  • the usable unambiguity range may not be sufficient when employing a single modulation frequency.
  • n (n > 1 ) measurements of the same distance may be made with n different modulation frequencies.
  • the usable distance measurement range can thereby be extended by combining the results of the different measurements and finding, or estimating (in the case of presence of measurement noise), the unknown distance by trying to invert the modulo function.
  • US patent 7,791 ,715 B1 relates to a method and system for dealiasing (unwrapping) in TOF systems.
  • the system uses at least two close-together modulation frequencies fi and h that are close to the maximum modulation frequency supported by the system.
  • a method for determining a distance comprises o providing at least two phase measurements made with modulated light of different modulation wavelengths, each phase measurement being indicative of the distance up to an integer multiple of a respective modulation wavelength; o providing a set of possible wraparound count combinations; o for each one of the possible wraparound count combinations, calculating a combination of unwrapped phase hypotheses corresponding to the at least two phase measurements; and o selecting a most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses and calculating the distance based upon the selected most plausible combination of unwrapped phase hypotheses.
  • the distance may e.g. be calculated as an average over the distance values associated with the unwrapped phase hypotheses of the most plausible combination of unwrapped phase hypotheses.
  • the selecting of the most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses comprises o for each combination of unwrapped phase hypotheses, calculating a combination of distance values associated with the unwrapped phase hypotheses of the respective combination of unwrapped phase hypotheses, o for each combination of unwrapped phase hypotheses, calculating a variance or standard deviation of the respective combination of distance values, and o selecting as the most plausible combination of unwrapped phase hypotheses the one for which the variance or standard deviation is smallest.
  • the selecting of the most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses comprises o for each combination of unwrapped phase hypotheses, calculating a gap between a point having the respective unwrapped phase hypotheses as coordinates and an origin-crossing straight line having a direction vector with coordinate representation [1/ ⁇ ,..., 1/ ⁇ ⁇ ], where ⁇ ,..., ⁇ ⁇ designate the different modulation wavelengths, and o selecting as the most plausible combination of unwrapped phase hypotheses the one for which the gap is smallest.
  • the distance may advantageously be calculated as 4 ⁇ ⁇
  • the calculating of the gap may be carried out in a rotated coordinate system, in which the origin-crossing straight line is a coordinate axis.
  • the distance is preferably calculated as calculated as
  • ( ⁇ ⁇ ⁇ opt ) l is a coordinate of the point having the most plausible combination of unwrapped phase hypotheses as coordinates in the rotated coordinate system, and ⁇ is the direction vector [c/Ai , ... , c/A n ].
  • a preferred aspect of the invention concerns a computer program, comprising computer-implementable instructions, which, when executed by a computer, cause the computer to carry out the method described hereinabove.
  • Another preferred aspect of the invention relates to a time-of-flight distance- measuring device, comprising an electronic control unit with a memory, the memory having stored therein a computer program as described hereinabove, the electronic control unit being configured to execute the computer program when determining a distance.
  • Yet another preferred aspect of the invention relates to a to a time-of-flight distance-measuring device, comprising an FPGA (field-programmable gate array) or ASIC (application-specific integrated circuit) configured and arranged to carry out the method described hereinabove.
  • Fig. 1 is a graph illustrating phase couples ( ⁇ , ⁇ 2) obtainable through ideal phase measurements at two different modulation frequencies for distances between 0 and a maximum distance;
  • Fig. 2 is a graph illustrating phase triples ( ⁇ , ⁇ 2, ⁇ 3) obtainable through ideal phase measurements at three different modulation frequencies for distances between 0 and a maximum distance;
  • Fig. 3 is a schematic flowchart of a first embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
  • Fig. 4 is a schematic flowchart of a second embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
  • Fig. 5 is a schematic flowchart of a third embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
  • Fig. 6 is a schematic illustration of a preferred embodiment of a 3D-TOF camera
  • Embodiments of the invention are going to be discussed, for illustration purposes, in the context of a 3D TOF camera that delivers, for each of its pixels, a phase information ⁇ in rad which is indicative of the distance d in meters between the camera and the target, and is given by: d - A - f
  • mod is the modulo function (which returns the remainder of the division of the first argument by the second argument), f is the modulation frequency in Hz, and c is the speed of light in m/s.
  • the phase ⁇ repeats itself periodically with the distance, implying that there is a limited unambiguity range of the distance measurement.
  • the unambiguity range is given by:
  • the usable distance measurement range is bounded by the unambiguity range.
  • the usable distance measurement range can be extended by combining the results of the different measurements and finding, or estimating in the case of presence of measurement noise, the unknown distance by "inverting" the modulo function.
  • a first problem that needs to be addressed is how to select the frequency values so as to maximize the allowed measurement noise when a defined usable distance measurement range is required. When sufficient noise is present, the modulo function inversion leads to a distance estimation error.
  • Fig. 1 shows the relation between ideally measured (i.e. not affected by noise) phases ⁇ and ⁇ 2, respectively corresponding to modulation frequencies fi and f2, when the unknown distance ranges between 0 and the required usable distance measurement range.
  • ⁇ and ⁇ 2 have been assumed to be 10 MHz and 1 1 MHz, respectively, and the maximum usable distance measurement range has been set to 100 m. It can be seen that ⁇ and ⁇ 2 "wrap around" each time they arrive at 2 ⁇ (illustrated by dashed lines).
  • the point ( ⁇ , ⁇ 2 ) defined by the measured phases ⁇ and ⁇ 2 lies on one of the line segments shown in Fig. 1 .
  • the measured point ( 3 ⁇ 4 , ⁇ 2 ) may differ from the a priori unknown ideal point ( ⁇ , ⁇ 2 ) and be located in the "forbidden areas" between the line segments. If the noise is sufficiently high, the measured point (3 ⁇ 4 , 3 ⁇ 4 ) may be closer to a line segment that corresponds to a distance that is completely different from the actual distance. Therefore, the distance between the line segments shown on Fig. 1 must be maximized for a given required usable distance measurement range by appropriately selecting the frequencies fi and f 2 .
  • Fig. 2 shows a graph similar to the one of Fig. 1 for the frequencies 10 MHz, 1 1 MHz and 13 MHz. The three frequencies result in a 3D plot.
  • the maximization of the distances between the line segments may be achieved by a brute force approach.
  • the range in which the modulation frequencies can lie is known beforehand, and is defined for example by the maximum operating frequency of the active illumination and by the minimum frequency which still yields an acceptable imager noise level. Within this range all n frequencies are varied, for example by linearly sweeping or randomly selecting. For each set of n frequencies, (frequency n-tuple), the distance trajectories are drawn in the n-dimensional phase space, and the minimum distance between each possible line segment pair is searched. Finally, the optimal frequency n-tuple that achieves the maximum of the minimum line segment distances is kept and used for the application. The minimum line segment distance value hi for the optimal frequency n-tuple is also kept and is used in the application to check the confidence level of the measured phase n-tuple.
  • n frequencies are assumed to be known. For illustration purposes, they will be given certain values, but it shall be understood that these frequencies are not necessarily optimal with regard to the above criterion of maximal separation between the line segments.
  • a first embodiment of a method for estimating the unknown distance from n phase measurements at n different frequencies will now be described, with reference to Fig. 3.
  • nwrapi is the wraparound count of ⁇ for frequency fi
  • nwrap 2 the wraparound count of ⁇ 2 for frequency
  • nwrap 3 the wraparound count of ⁇ 3 for frequency ?. It is worthwhile noting that each column in the above table corresponds to one segment in Fig. 2.
  • the table of phase wraparound counts can be calculated offline, that is to say only once during the configuration of the measurement system, and its results can be permanently stored in the application's non volatile memory, as the only information needed are the n frequencies and the required usable distance measurement range, which are known beforehand.
  • a distance hypothesis may be calculated as: ( ⁇ ⁇ + nwrapy ⁇ 2 ⁇ ) ⁇ c
  • a phase n-tuple [ ⁇ , ... ⁇ date] is measured (step 303).
  • a distance hypothesis is estimated for each cell in the table using the above formula for d (step 304).
  • the optimal column j op t in the table is searched for. This column corresponds to the most plausible combination of phase wraparound counts.
  • the optimal column is the one for which the variance or the standard deviation of the n distances dy in the column is minimal.
  • a second embodiment of the method for estimating the unknown distance from n phase measurements at n different frequencies will now be described with reference to Fig. 4.
  • This embodiment takes advantage from the fact that the actual required distance measurement range is typically smaller than the target detection range from which no folding back into the required distance measurement range must occur, but where the actual target distance does not need to be measured.
  • the TOF measurement device is able to measure the unknown distance over a restricted range. Targets beyond this range and still within the range where the TOF measurement device detects a target must not fold back into the required distance measurement range, but their distance does not need to be known.
  • the distances to targets which are within, e.g.
  • the detection range is 100 m, but the required distance measurement range is only 20 m. This implies that the frequency selection process (step 401 ) takes into account
  • the minimum line segment distance value (hereinafter noted h 2 ) that is found (step 403) is kept and used in the application to check the confidence level of the measured phase n-tuple.
  • nwrapi is again the wraparound count of ⁇ for frequency fi
  • nwrap2 the wraparound count of ⁇ 2 for frequency f2
  • nwrap 3 the wraparound count of ⁇ 3 for frequency ?.
  • the set of frequencies and the table of wraparound count can be calculated offline (steps 401 and 402), that is only once during the configuration of the measurement system and its results can be permanently stored in the application's non volatile memory, as the only information needed are the n frequencies, the wanted measurement range and the required extended unambiguity distance, which are known beforehand.
  • the wraparound count table above the column which best unwraps the n-tuple of measured phases [ ⁇ ⁇ ⁇ ], leading to an optimal estimation of the unknown distance, is searched for.
  • phase measurements which may be represented as phase n-tuple [ ⁇ , ..., ⁇ date] or vector ⁇ .
  • This can for example be done by calculating the length of the vector cross product of the unwrapped phase n-tuple [ ⁇ ... ⁇ p 'êt, ⁇ ], which we will designate by ⁇ ' , for simplicity, and the normalized vector U : where x designated the vector cross product and
  • the vector U is preferably determined beforehand (step 404).
  • gpopt is compared (step 409) with h 2 /2 found in the frequency selection process described above. If gp opt ⁇ h 2 /2, then the distance to be determined lies within the required measurement range, if gp op t ⁇ h 2 /2, then it lies outside the required measurement range (but within the detection range, for otherwise there would be no phase measurement values) and is not considered a valid distance (step 410). [0040] Preferably, the squares of the gj are computed instead because doing this is computationally less expensive. gj_ op t 2 is then compared with the offline computed parameter ⁇ /4.
  • the distance can be estimated (step 412) as:
  • FIG. 5 A preferred implementation of the second embodiment, which is computationally inexpensive will now be explained with reference to Fig. 5.
  • the y and z-axis unit vectors would allow the same inexpensive implementation, but for the sake of simplicity only the case for the x axis unit vector will be described.
  • There is a virtually infinite number of ways to achieve this transformation and for simplicity the rotation will be performed around the unity-length normal vector n of the plane spanned by ⁇ and 1 , and the angle of rotation will be the angle a between ⁇ and 1 :
  • n w /
  • , n w /
  • the rotation matrix R which performs the rotation operation described above is defined by: n x n x (1 - cosa) + cos a n 2 n y (1 - cosa) - n 3 sin a n 3 n y (1 - cos a) + n 2 sin a
  • the offline computation comprises the selection of the modulation frequencies (step 501 ), the calculation of the wraparound count table (step 502) and the calculation of the rotation matrix R (step 503), as indicated above, and the calculation of the parameter h 2 (step 505).
  • the gap gj depends only on the second and third coordinates of ⁇ " j , which we denote by ( ' ⁇ ) 2 and ( ' ⁇ ) 3 .
  • ⁇ " j the second and third coordinates of ⁇ " j .
  • the following equation holds: [0051 ] The smallest gap is searched for among all the gj. The smallest gap found, g opt, is compared with h 2 /2 found in the offline frequency selection process.
  • the distance to be determined lies within the required measurement range, if gj 0 pt ⁇ h 2 /2, then it lies outside the required measurement range (but within the detection range, for otherwise there would be no phase measurement values) and is not considered a valid distance. It is possible to compute only the squares of the gj (step 508). In this case, the smallest squared gap is searched for among all the gf (step 509), and the value found, gj_ op t 2 , is compared with the offline computed parameter ⁇ /4 (step 510).
  • the unknown distance d es t is computed by orthogonally projecting the rotated unwrapped phase n-tuple ⁇ " j opt onto the x-axis (i.e. the rotated origin-crossing line with direction vector U ). Noting ( ⁇ ' ⁇ opt the first coordinate of ⁇ " j opt , the projected point has ( ( ' ⁇ opt , 0, 0) as coordinates.
  • the actual value of the measurement phase noise is preferably known or estimated during the measurement. Indeed, if the phase noise is too large, it may happen that the noisy unknown distance n-tuple in the n-phase space jumps from the vicinity of the line segment, which the actual unknown distance phase n-tuple belongs to, into the vicinity of a completely different line segment in the phase space. Eventually, this leads to wrong distance estimation. Therefore, measurements, which have too large phase noise, are preferably declared invalid.
  • the phase noise can be estimated or measured by measuring the amplitude, or by measuring the amplitude and intensity of the received modulated light and combing the amplitude and intensity. How this can be done is, for example, described in '3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD-Technology' by R. Lange.
  • Fig. 6 shows a preferred embodiment of a 3D TOF camera, generally identified by reference numeral 10.
  • the 3D TOF camera 10 comprises an illumination unit 12 emitting light onto a scene, and an imaging sensor 14 imaging the scene.
  • the imaging sensor 14 comprises, in a manner known per se, the required optical accessories such as a focusing lens (not shown) and an electronic camera chip 16 executed in any suitable technology, such as CCD, CMOS, TFA or the like.
  • the imaging sensor 14 comprises a two-dimensional array of individual lock-in pixel sensor cells 18, each of which receives light from a small portion of the scene to be imaged for creating a pixel-by-pixel image thereof.
  • the illumination unit 12 comprises several individual light emitting devices 20 such as light emitting diodes (LEDs), which are collectively driven by means of an illumination driver 22.
  • a signal source 24 provides the input signals for the illumination driver 22 and a photo gate driver 26.
  • the output of the photo gate driver 26 is connected to the imaging sensor 14.
  • An evaluation unit 28 comprising a suitable electronic calculation device, e.g. a digital signal processor (DSP), is connected to the output of the imaging sensor 14.
  • DSP digital signal processor
  • the signal source 24 When operating, the signal source 24 generates a modulation signal E1 on its output and feeds this modulation signal E1 to the illumination driver 22.
  • the latter drives the illumination unit 12 with a drive signal E2 to emit an intensity-modulated light signal L1 into a target scene comprising an object 30 (for illustration purposes).
  • the modulated light signal L1 is partially reflected by the object 30 so as to form a returning light signal L2 which is received as incident light by the imaging sensor 14.
  • the modulation signal E1 is also fed to the photo gate driver 26, which provides a demodulation signal E3.
  • the imaging sensor 14 receives this demodulation signal E3 and produces a phase information signal E4, which is fed to the evaluation unit 28.
  • the signal source 24 cycles through the plurality of different modulation frequencies fi , f n .
  • the 3D TOF camera 10 further comprises a reference pixel 32 mounted in the illumination unit 12 contiguous to the light emitting devices 20 in such a way that it gathers light from the light emitting devices 18.
  • the reference pixel 32 produces an output signal E5, which corresponds essentially to a modulation phase reference (comparable to a "zero" distance measurement).
  • a reference subtraction stage 34 within the evaluation unit 28 outputs, for each pixel, phase information ⁇ , in rad corresponding to the modulation frequency f,.
  • the distance information for each camera pixel 18 is then calculated in a distance calculation stage 36, in accordance with the method of the present invention.
  • the evaluation unit 28 comprises a nonvolatile memory 38, in which the calibration data, in particular the parameters that may be calculated offline, are stored.
  • the distance calculation stage 36 has access to these parameters for executing the method.
  • phase subtraction stage 34 and the distance calculation stage 36 are not necessarily physically separate components.
  • the different operations may, in particular, be carried out by a single (multi-purpose) processor.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

A method for determining a distance comprises: providing at least two phase measurements made with modulated light of different modulation wavelengths, each phase measurement being indicative of the distance up to an integer multiple of a respective modulation wavelength; providing a set of possible wraparound count combinations; for each one of the possible wraparound count combinations, calculating a combination of unwrapped phase hypotheses corresponding to the at least two phase measurements; and selecting a most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses and calculating the distance based upon the selected most plausible combination of unwrapped phase hypotheses.

Description

DESCRIPTION
DISTANCE DETERMINATION METHOD
Technical field
[0001 ] The present invention generally relates to a method for determining a distance based upon a plurality of phase measurements, made at different wavelengths, each of the phase measurements, taken individually, being ambiguous in the sense that it indicates the distance only up to an integer multiple of the respective wavelength.
Background Art [0002] Distance measuring equipment often relies on the time-of-flight (TOF) measurement principle. Distance (also: depth) information is in this case obtained by emitting modulated light into the direction of the target and measuring the phase shift of the modulation between the emitted and the reflected light. The measured phase is proportional to the distance between the measurement equipment and the target, modulo the wavelength of the modulation. This relationship may be expressed as: d - A - f
φ = mod( — , 2π) ,
c where φ is the modulation phase in radians, d is the distance, f is the modulation frequency, and c is the speed of light. The modulation wavelength λ is given as A=c/f. The extra factor 2 (4π instead of 2π) in the first argument of the mod function is due to the fact that the light travels twice the distance between the distance measuring equipment and the target. The above relationship expresses that the phase measurement is ambiguous, i.e. the distance value d' calculated as d' = cp-c/(4TT-f) may be different from the actual distance by an a priori unknown integer number of modulation wavelengths. [0003] The distance du = c/(2-f) is called the "unambiguity range", because if it is known beforehand that the measured distance is less than du, the modulo function may be inverted, yielding an unambiguous distance. However, if the target may be located at greater distances from the target than du, any phase measurement will be comprised in the interval [0, 2π[: one says that the measurements are "wrapped" into the unambiguity range. As a consequence, retrieving the actual distance from the ambiguous measurement, or, otherwise stated, determining the integer number of modulation wavelengths to add to the distance value d' in order to arrive at the actual distance, is called "phase unwrapping". [0004] In a single-frequency system, the usable distance measurement range is bounded by the unambiguity range unless assumptions are made on the smoothness of the scene (neighbouring pixels) and on the smoothness of the evolution in time of a distance measurement. Such assumptions cannot be made in all circumstances and for all applications. Specifically, in an adaptive driver assistance system, such assumptions are normally not permitted. When operating a 3D TOF camera or other distance measuring equipment, the usable unambiguity range may not be sufficient when employing a single modulation frequency. To overcome that drawback, n (n > 1 ) measurements of the same distance may be made with n different modulation frequencies. The usable distance measurement range can thereby be extended by combining the results of the different measurements and finding, or estimating (in the case of presence of measurement noise), the unknown distance by trying to invert the modulo function. When using two frequencies (n=2), the combination is easily performed by calculating the difference between the measured phases at the first and second modulation frequency, but when using three frequencies or more in order to further reduce the measurement error in presence of noise compared to using only two frequencies, a method which is as simple as calculating the phase difference does not exist.
[0005] The paper "Multi-frequency Phase Unwrapping for Time-of- Flight Cameras", D. Droeschel, D. Holz and S. Behnke, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, October 2010, discloses a probabilistic phase unwrapping approach, which takes into account the neighbourhood of each pixel. The paper also mentions the possibility of using two modulation frequencies for detecting phase wrapping. It is proposed to compare the two distance values obtained from the two measured phases. If these distance values coincide, the distance lies in the unambiguity range; if they do not coincide, at least one of the phases has been wrapped into the [0,2π[ interval and the phase has to be unwrapped. [0006] Document DE 100 39 422 C2 discloses a photoelectronic mixing device (PMD) operating at plural frequencies for carrying out measurements of speed and/or distance.
[0007] US patent 7,791 ,715 B1 relates to a method and system for dealiasing (unwrapping) in TOF systems. The system uses at least two close-together modulation frequencies fi and h that are close to the maximum modulation frequency supported by the system. The distance Z is then calculated as: z = c^
4π - Α/ where Δφ is the difference of the measured phases at fi and h, respectively, and where Δί = fi-f2. When more than two different frequencies are available, the phase measurements are combined in pairs and the same formula is applied.
[0008] Whereas it is known from the prior art to combine plural frequencies in order to achieve a greater (combined) unambiguity range, the known methods have drawbacks when measurement noise comes into play. Furthermore, the computational effort necessary to carry these methods out may be too important for low-cost implementations.
Technical problem
[0009] It is an object of the present invention to provide a practically implementable method for estimating the unknown distance when using two or more modulation frequencies. This object is achieved by a method as claimed in claim 1 .
General Description of the Invention
[0010] According to the invention, a method for determining a distance comprises o providing at least two phase measurements made with modulated light of different modulation wavelengths, each phase measurement being indicative of the distance up to an integer multiple of a respective modulation wavelength; o providing a set of possible wraparound count combinations; o for each one of the possible wraparound count combinations, calculating a combination of unwrapped phase hypotheses corresponding to the at least two phase measurements; and o selecting a most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses and calculating the distance based upon the selected most plausible combination of unwrapped phase hypotheses.
[001 1 ] The distance may e.g. be calculated as an average over the distance values associated with the unwrapped phase hypotheses of the most plausible combination of unwrapped phase hypotheses.
[0012] According to preferred possible embodiment the selecting of the most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses comprises o for each combination of unwrapped phase hypotheses, calculating a combination of distance values associated with the unwrapped phase hypotheses of the respective combination of unwrapped phase hypotheses, o for each combination of unwrapped phase hypotheses, calculating a variance or standard deviation of the respective combination of distance values, and o selecting as the most plausible combination of unwrapped phase hypotheses the one for which the variance or standard deviation is smallest.
[0013] According to a preferred embodiment of the invention, the selecting of the most plausible combination of unwrapped phase hypotheses among the combinations of unwrapped phase hypotheses comprises o for each combination of unwrapped phase hypotheses, calculating a gap between a point having the respective unwrapped phase hypotheses as coordinates and an origin-crossing straight line having a direction vector with coordinate representation [1/λι,..., 1/λη], where λι,..., λη designate the different modulation wavelengths, and o selecting as the most plausible combination of unwrapped phase hypotheses the one for which the gap is smallest.
[0014] The distance may advantageously be calculated as 4π · ||« ||
where 'est is an orthogonal projection onto the origin-crossing straight line of a point having the most plausible combination of unwrapped phase hypotheses as coordinates, and ΰ is the direction vector [c/Ai , ... , c/An]. [0015] Yet more advantageously, the calculating of the gap may be carried out in a rotated coordinate system, in which the origin-crossing straight line is a coordinate axis. In this case, the distance is preferably calculated as calculated as
Figure imgf000006_0001
where (φη } opt )l is a coordinate of the point having the most plausible combination of unwrapped phase hypotheses as coordinates in the rotated coordinate system, and ΰ is the direction vector [c/Ai , ... , c/An].
[0016] A preferred aspect of the invention concerns a computer program, comprising computer-implementable instructions, which, when executed by a computer, cause the computer to carry out the method described hereinabove. [0017] Another preferred aspect of the invention relates to a time-of-flight distance- measuring device, comprising an electronic control unit with a memory, the memory having stored therein a computer program as described hereinabove, the electronic control unit being configured to execute the computer program when determining a distance. [0018] Yet another preferred aspect of the invention relates to a to a time-of-flight distance-measuring device, comprising an FPGA (field-programmable gate array) or ASIC (application-specific integrated circuit) configured and arranged to carry out the method described hereinabove.
Brief Description of the Drawings [0019] Preferred, non-limiting, embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings in which: Fig. 1 is a graph illustrating phase couples (φι, ψ2) obtainable through ideal phase measurements at two different modulation frequencies for distances between 0 and a maximum distance;
Fig. 2 is a graph illustrating phase triples (φι, ψ2, ψ3) obtainable through ideal phase measurements at three different modulation frequencies for distances between 0 and a maximum distance;
Fig. 3 is a schematic flowchart of a first embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
Fig. 4 is a schematic flowchart of a second embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
Fig. 5 is a schematic flowchart of a third embodiment of a method for estimating the unknown distance from n phase measurements made at n different frequencies;
Fig. 6 is a schematic illustration of a preferred embodiment of a 3D-TOF camera
Description of Preferred Embodiments [0020] Embodiments of the invention are going to be discussed, for illustration purposes, in the context of a 3D TOF camera that delivers, for each of its pixels, a phase information φ in rad which is indicative of the distance d in meters between the camera and the target, and is given by: d - A - f
φ = mod( — , 2π) ,
c where mod is the modulo function (which returns the remainder of the division of the first argument by the second argument), f is the modulation frequency in Hz, and c is the speed of light in m/s. The modulation wavelength λ is given as A=c/f.
[0021 ] As can be seen, the phase φ repeats itself periodically with the distance, implying that there is a limited unambiguity range of the distance measurement. The unambiguity range is given by:
Figure imgf000007_0001
[0022] The usable distance measurement range is bounded by the unambiguity range. When performing n measurements with n different frequencies of the same scene, the usable distance measurement range can be extended by combining the results of the different measurements and finding, or estimating in the case of presence of measurement noise, the unknown distance by "inverting" the modulo function. [0023] A first problem that needs to be addressed is how to select the frequency values so as to maximize the allowed measurement noise when a defined usable distance measurement range is required. When sufficient noise is present, the modulo function inversion leads to a distance estimation error.
[0024] To illustrate this problem for a 2-frequency measurement, Fig. 1 shows the relation between ideally measured (i.e. not affected by noise) phases φι and ψ2, respectively corresponding to modulation frequencies fi and f2, when the unknown distance ranges between 0 and the required usable distance measurement range. For Fig.1 , /j and /2 have been assumed to be 10 MHz and 1 1 MHz, respectively, and the maximum usable distance measurement range has been set to 100 m. It can be seen that φι and ψ2 "wrap around" each time they arrive at 2π (illustrated by dashed lines). If no noise is present on the measurements, the point (φι, φ2) defined by the measured phases φι and ψ2 lies on one of the line segments shown in Fig. 1 . However, if noise is present on the phase measurements, the measured point ( ¾ , φ2 ) may differ from the a priori unknown ideal point (φι, φ2) and be located in the "forbidden areas" between the line segments. If the noise is sufficiently high, the measured point (¾ , ¾ ) may be closer to a line segment that corresponds to a distance that is completely different from the actual distance. Therefore, the distance between the line segments shown on Fig. 1 must be maximized for a given required usable distance measurement range by appropriately selecting the frequencies fi and f2. The same applies for an n-frequency measurement. For the illustration of a 3- frequency measurement, Fig. 2 shows a graph similar to the one of Fig. 1 for the frequencies 10 MHz, 1 1 MHz and 13 MHz. The three frequencies result in a 3D plot.
[0025] The maximization of the distances between the line segments may be achieved by a brute force approach. The range in which the modulation frequencies can lie is known beforehand, and is defined for example by the maximum operating frequency of the active illumination and by the minimum frequency which still yields an acceptable imager noise level. Within this range all n frequencies are varied, for example by linearly sweeping or randomly selecting. For each set of n frequencies, (frequency n-tuple), the distance trajectories are drawn in the n-dimensional phase space, and the minimum distance between each possible line segment pair is searched. Finally, the optimal frequency n-tuple that achieves the maximum of the minimum line segment distances is kept and used for the application. The minimum line segment distance value hi for the optimal frequency n-tuple is also kept and is used in the application to check the confidence level of the measured phase n-tuple.
[0026] In the following, the n frequencies are assumed to be known. For illustration purposes, they will be given certain values, but it shall be understood that these frequencies are not necessarily optimal with regard to the above criterion of maximal separation between the line segments.
[0027] A first embodiment of a method for estimating the unknown distance from n phase measurements at n different frequencies will now be described, with reference to Fig. 3. In a first step, the modulation frequencies fi , fn are fixed (step 301 ) and for the unknown distance ranging between zero and the usable distance measurement range, a list with all the possibly occurring distinct combinations of phase wraparound counts of each of the n phases is established (step 302). For example, using a 3-frequency measurement with the same parameters as for Fig. 2 (frequencies fi=10 MHz, f2=1 1 MHz and f3=13 MHz) the following table results:
Figure imgf000009_0001
where nwrapi is the wraparound count of φι for frequency fi, nwrap2 the wraparound count of ψ2 for frequency and nwrap3 the wraparound count of ψ3 for frequency ?. It is worthwhile noting that each column in the above table corresponds to one segment in Fig. 2. The table of phase wraparound counts can be calculated offline, that is to say only once during the configuration of the measurement system, and its results can be permanently stored in the application's non volatile memory, as the only information needed are the n frequencies and the required usable distance measurement range, which are known beforehand.
[0028] For each row and column in the table, a distance hypothesis may be calculated as: (φί + nwrapy 2π) c
[0029] During the measurement, a phase n-tuple [φι, ... φ„] is measured (step 303). Using the wraparound count table above, a distance hypothesis is estimated for each cell in the table using the above formula for d (step 304). [0030] Then, the optimal column jopt in the table is searched for. This column corresponds to the most plausible combination of phase wraparound counts. According to the first embodiment of the method, the optimal column is the one for which the variance or the standard deviation of the n distances dy in the column is minimal. The variance 2 of each wraparound count combination j may be calculated (step 305) as: σ) =
Figure imgf000010_0001
and the minimum variance be determined
(step 306).
[0031 ] The ratio of the standard deviation (found for j opt) to the minimum line segment distance hi can be used as an indicator of the confidence level of the distance measurement (the smaller the ratio the better). [0032] Finally, the mean (average) dj opt = ^i=xdv opt of the distances dy_opt of the column j_opt serves as an estimation of the unknown distance (step 307).
[0033] A second embodiment of the method for estimating the unknown distance from n phase measurements at n different frequencies will now be described with reference to Fig. 4. This embodiment takes advantage from the fact that the actual required distance measurement range is typically smaller than the target detection range from which no folding back into the required distance measurement range must occur, but where the actual target distance does not need to be measured. In many application it is necessary only that the TOF measurement device is able to measure the unknown distance over a restricted range. Targets beyond this range and still within the range where the TOF measurement device detects a target must not fold back into the required distance measurement range, but their distance does not need to be known. For example, in an automotive 'advanced driver assistance system', only the distances to targets which are within, e.g. 20 m in front of the car shall be measured, but at the same time highly reflective targets up to 100 m in front of the car must not be folded back into the required distance measurement range. For this example, the detection range is 100 m, but the required distance measurement range is only 20 m. This implies that the frequency selection process (step 401 ) takes into account
1 ) the distances between any two line segments representing distances within the required measurement range,
2) the distances between any pair of one line segment representing a distance within the required measurement range and one line segment representing a distance that is outside the required measurement range but still within the equipment's detection range but does not take into account the distances between any two line segments representing distances, which are outside the required measurement range but still within the equipment's detection range. The minimum line segment distance value (hereinafter noted h2) that is found (step 403) is kept and used in the application to check the confidence level of the measured phase n-tuple.
[0034] For the unknown distance ranging between zero and the required measurement range, a list with all the possibly occurring distinct combinations of phase wraparound counts of each of the n phases is established. For example, using 3-frequency measurement with the same parameters as for Fig. 2 and a wanted measurement range of 20 m, the following table results:
Figure imgf000011_0001
where nwrapi is again the wraparound count of φι for frequency fi, nwrap2 the wraparound count of ψ2 for frequency f2 and nwrap3 the wraparound count of ψ3 for frequency ?. The set of frequencies and the table of wraparound count can be calculated offline (steps 401 and 402), that is only once during the configuration of the measurement system and its results can be permanently stored in the application's non volatile memory, as the only information needed are the n frequencies, the wanted measurement range and the required extended unambiguity distance, which are known beforehand. [0035] Using the wraparound count table above, the column which best unwraps the n-tuple of measured phases [φι φη], leading to an optimal estimation of the unknown distance, is searched for.
[0036] The online calculation starts with the providing of the phase measurements (step 405), which may be represented as phase n-tuple [φι, ..., φ„] or vector φ . For each row and column in the list and for a combination of measured phases (n-tuple [φι, ..., φ„]), the unwrapped phase is calculated (step 406) using the formula:
Figure imgf000012_0001
= (pi + nwrapij - 2n .
[0037] Then, for each column j, the gap gj in the n-dimensional phase space between the unwrapped phase n-tuple [φΊρ ... (p'nj\ and the origin-crossing straight line having the direction vector = \fi ,...,fn ] = [c/ l ,...,c/ n ] is calculated (step 407). This can for example be done by calculating the length of the vector cross product of the unwrapped phase n-tuple [φΊρ ... <p '„,·], which we will designate by φ' , for simplicity, and the normalized vector U :
Figure imgf000012_0002
where x designated the vector cross product and || . || the length of vector a vector. It is worthwhile noting that the vector U is preferably determined beforehand (step 404). [0038] Then, the smallest gj (hereinafter denoted gj_opt) is searched among all the gp (step 408) and then the unwrapped phase n-tuple j opt = [ (p j opt ,... , (p j opt ] corresponding to gpopt is selected.
[0039] gpopt is compared (step 409) with h2/2 found in the frequency selection process described above. If gpopt < h2/2, then the distance to be determined lies within the required measurement range, if gpopt≥h2/2, then it lies outside the required measurement range (but within the detection range, for otherwise there would be no phase measurement values) and is not considered a valid distance (step 410). [0040] Preferably, the squares of the gj are computed instead because doing this is computationally less expensive. gj_opt2 is then compared with the offline computed parameter Λ /4.
[0041 ] The unknown distance dest is finally computed by projecting the found unwrapped phase n-tuple j ορ, = [ φ'υ ορ, ,...,φ'η] opt ] onto the origin-crossing line with direction vector ΰ (step 41 1 ), measuring the distance of the projection to the origin. [0042] Noting
Figure imgf000013_0001
the projection of the found unwrapped phase n-tuple ορ, = [ φ'υ ορ, ,...,φ'η] opt ] onto the origin-crossing line with direction vector ΰ (" · " designated the dot product between two vectors), the distance can be estimated (step 412) as:
Figure imgf000013_0002
[0043] A preferred implementation of the second embodiment, which is computationally inexpensive will now be explained with reference to Fig. 5. This implementation is similar to the implementation of Fig. 4, except that the n- dimensional phase space is rotated around the origin in such a way that the vector ΰ = f2, f3] will be rotated into alignment with the x-axis unit vector 1 = [1 ,0,0]. The y and z-axis unit vectors would allow the same inexpensive implementation, but for the sake of simplicity only the case for the x axis unit vector will be described. There is a virtually infinite number of ways to achieve this transformation, and for simplicity the rotation will be performed around the unity-length normal vector n of the plane spanned by ΰ and 1 , and the angle of rotation will be the angle a between ΰ and 1 :
[0044] With v = w /||w|| , n may be expressed as:
Figure imgf000013_0003
\\v xi \\ [0045] The angle can be calculated as: a = arccos(v · i ) . [0046] The rotation matrix R, which performs the rotation operation described above is defined by: nxnx (1 - cosa) + cos a n2ny (1 - cosa) - n3 sin a n3 ny (1 - cos a) + n2 sin a
R = ηγη2 (1 - cos ) + n3 sin a n2n2 (1 - cosa) + cos a /73/72(l - cosa) - /?1 sin a
ηγη3 (1 - cosa) - n2 sin a n2n3 (1 - cosa) + ηγ sin a «3«3 (1 - cosa) + cosa with n = {nx , n2 ,n3 ) . [0047] The offline computation comprises the selection of the modulation frequencies (step 501 ), the calculation of the wraparound count table (step 502) and the calculation of the rotation matrix R (step 503), as indicated above, and the calculation of the parameter h2 (step 505).
[0048] The online calculation begins with providing the n phase measurements (step 506). Instead of calculating, for each column j, the gap gj in the n-dimensional phase space between the unwrapped phase n-tuple (φΊρ ... cp'nj) and the origin-crossing straight line having the direction vector ΰ = [fl ,...,f„], in this implementation, one calculates, for each column j, the gap g'j between the rotated unwrapped phase n- tuple R[(p'ip ...
Figure imgf000014_0001
= R(p'j and the x-axis (i.e. the rotated origin-crossing straight line having ΰ = [fl ,...,fn ] as direction vector). Since the rotation does not change the measures, g'j = gj.
[0049] The rotated unwrapped phase n-tuple φ" j may be calculated (step 507) as: φ" j = Κφ' j = Rq> + R nwrap . · 2π , Where nwrap . is the j-th column vector of the phase wraparound table. It i is worthwhile noting that R nwrap . · 2π can be calculate upfront (as step 504 of the offline calculations), which means that this calculation does not take any processing time in the online calculations.
[0050] It shall also be noted that the gap gj depends only on the second and third coordinates of φ" j , which we denote by ( '^ )2 and ( '^ )3 . For the square of the gap gj, the following equation holds: [0051 ] The smallest gap is searched for among all the gj. The smallest gap found, g opt, is compared with h2/2 found in the offline frequency selection process. If gj_oPt < h2/2, then the distance to be determined lies within the required measurement range, if gj 0pt≥ h2/2, then it lies outside the required measurement range (but within the detection range, for otherwise there would be no phase measurement values) and is not considered a valid distance. It is possible to compute only the squares of the gj (step 508). In this case, the smallest squared gap is searched for among all the gf (step 509), and the value found, gj_opt2, is compared with the offline computed parameter Λ /4 (step 510). If gj_opt 2≥ (h2/2)2 , the rotated unwrapped phase n-tuple φ" j opt is considered to represent the most plausible combination of unwrapped phase hypotheses, otherwise the target is considered to lie outside the measurement range (step 51 1 ).
[0052] The unknown distance dest is computed by orthogonally projecting the rotated unwrapped phase n-tuple φ" j opt onto the x-axis (i.e. the rotated origin-crossing line with direction vector U ). Noting (φ'^ opt the first coordinate of φ" j opt , the projected point has ( ( '^ opt , 0, 0) as coordinates.
[0053] The distance can finally be estimated (step 512) as: est 4π · ||«|| [0054] The difference between the computational efforts for the first implementation of the method described above, the second implementation and the computationally inexpensive version described above is now illustrated by means of an example. Only the online computation effort is estimated, since the offline computations do not consume any processing time during the measurements. Only the multiplications and square root operations are counted as they are the most expensive to implement.
[0055] When using the first implementation and using the parameters given as example, an initial scaling of [ψι, φ2, ψ3] in order to reduce the computational effort in calculating the distance hypotheses d requires 3 scalar multiplications for each column in the wraparound list (there are 22 columns in the example), one multiplication for the computation of the mean value, 3 multiplications for the variance (easier to compute than the standard deviation but fulfils the same purpose), and for the calculation of the final result (distance) one scalar multiplication, resulting in: 3 + 22 * (1 + 3) + 1 = 92 multiplications and no square root operation.
[0056] When using the second method described above and using the parameters given as example, for each column in the wraparound list (there are 5 columns), a vector cross product (6 multiplications), a vector squared length calculation (3 multiplications), and for the final result, a vector dot product (3 multiplications), a vector scalar product (3 multiplications), a vector length calculation (3 multiplications and a square root) and a scalar multiplication are required, resulting in: 5 * (6 + 3) + 3 + 3 + 3 + 1 = 55 multiplications and one square root operation. [0057] When using the computationally inexpensive version described above and using the parameters given as example, the initial rotation requires 9 multiplications, for each column in the wraparound list (there are 5 columns), a 2-dimensional vector squared length calculation (2 multiplications), and for the final result, a scalar multiplication are required, resulting in: 9 + 5 * 2 + 1 = 20 multiplications and no square root operation.
[0058] In the above-disclosed distance estimation methods the actual value of the measurement phase noise is preferably known or estimated during the measurement. Indeed, if the phase noise is too large, it may happen that the noisy unknown distance n-tuple in the n-phase space jumps from the vicinity of the line segment, which the actual unknown distance phase n-tuple belongs to, into the vicinity of a completely different line segment in the phase space. Eventually, this leads to wrong distance estimation. Therefore, measurements, which have too large phase noise, are preferably declared invalid. The phase noise can be estimated or measured by measuring the amplitude, or by measuring the amplitude and intensity of the received modulated light and combing the amplitude and intensity. How this can be done is, for example, described in '3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD-Technology' by R. Lange.
[0059] Fig. 6 shows a preferred embodiment of a 3D TOF camera, generally identified by reference numeral 10. The 3D TOF camera 10 comprises an illumination unit 12 emitting light onto a scene, and an imaging sensor 14 imaging the scene. The imaging sensor 14 comprises, in a manner known per se, the required optical accessories such as a focusing lens (not shown) and an electronic camera chip 16 executed in any suitable technology, such as CCD, CMOS, TFA or the like. The imaging sensor 14 comprises a two-dimensional array of individual lock-in pixel sensor cells 18, each of which receives light from a small portion of the scene to be imaged for creating a pixel-by-pixel image thereof. [0060] The illumination unit 12 comprises several individual light emitting devices 20 such as light emitting diodes (LEDs), which are collectively driven by means of an illumination driver 22. A signal source 24 provides the input signals for the illumination driver 22 and a photo gate driver 26. The output of the photo gate driver 26 is connected to the imaging sensor 14. An evaluation unit 28 comprising a suitable electronic calculation device, e.g. a digital signal processor (DSP), is connected to the output of the imaging sensor 14.
[0061 ] When operating, the signal source 24 generates a modulation signal E1 on its output and feeds this modulation signal E1 to the illumination driver 22. The latter drives the illumination unit 12 with a drive signal E2 to emit an intensity-modulated light signal L1 into a target scene comprising an object 30 (for illustration purposes). The modulated light signal L1 is partially reflected by the object 30 so as to form a returning light signal L2 which is received as incident light by the imaging sensor 14. The modulation signal E1 is also fed to the photo gate driver 26, which provides a demodulation signal E3. The imaging sensor 14 receives this demodulation signal E3 and produces a phase information signal E4, which is fed to the evaluation unit 28. The signal source 24 cycles through the plurality of different modulation frequencies fi , fn.
[0062] The 3D TOF camera 10 further comprises a reference pixel 32 mounted in the illumination unit 12 contiguous to the light emitting devices 20 in such a way that it gathers light from the light emitting devices 18. The reference pixel 32 produces an output signal E5, which corresponds essentially to a modulation phase reference (comparable to a "zero" distance measurement). A reference subtraction stage 34 within the evaluation unit 28, outputs, for each pixel, phase information φ, in rad corresponding to the modulation frequency f,. The distance information for each camera pixel 18 is then calculated in a distance calculation stage 36, in accordance with the method of the present invention. The evaluation unit 28 comprises a nonvolatile memory 38, in which the calibration data, in particular the parameters that may be calculated offline, are stored. The distance calculation stage 36 has access to these parameters for executing the method.
[0063] It is worthwhile noting that in an actual implementation of the evaluation unit 28, the phase subtraction stage 34 and the distance calculation stage 36 are not necessarily physically separate components. The different operations may, in particular, be carried out by a single (multi-purpose) processor.
[0064] While specific embodiments have been described in detail, those skilled in the art will appreciate that various modifications and alternatives to those details could be developed in light of the overall teachings of the disclosure. Accordingly, the particular arrangements disclosed are meant to be illustrative only and not limiting as to the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalents thereof.

Claims

Claims
1 . Method for determining a distance, comprising:
providing at least two phase measurements made with modulated light of different modulation wavelengths, each phase measurement being indicative of said distance up to an integer multiple of a respective modulation wavelength; providing a set of possible wraparound count combinations;
for each one of said possible wraparound count combinations, calculating a combination of unwrapped phase hypotheses corresponding to said at least two phase measurements; and
selecting a most plausible combination of unwrapped phase hypotheses among said combinations of unwrapped phase hypotheses and calculating said distance based upon said selected most plausible combination of unwrapped phase hypotheses,
said method being characterized in that said selecting of the most plausible combination of unwrapped phase hypotheses among said combinations of unwrapped phase hypotheses comprises
for each combination of unwrapped phase hypotheses, calculating a gap between a point having the respective unwrapped phase hypotheses as coordinates and an origin-crossing straight line having a direction vector with coordinate representation [c/λι , . .. , c/An], where λι , . .. , λη designate the different modulation wavelengths and c designates the speed of light, and
selecting as said most plausible combination of unwrapped phase hypotheses the one for which said gap is smallest.
2. Method as claimed in claim 1 , wherein said distance is calculated as an average over distance values associated with the unwrapped phase hypotheses of said most plausible combination of unwrapped phase hypotheses.
3. Method as claimed in claim 1 or 2, wherein said selecting of the most plausible combination of unwrapped phase hypotheses among said combinations of unwrapped phase hypotheses comprises
for each combination of unwrapped phase hypotheses, calculating a combination of distance values associated with the unwrapped phase hypotheses of the respective combination of unwrapped phase hypotheses, for each combination of unwrapped phase hypotheses, calculating a variance or standard deviation of the respective combination of distance values, and selecting as said most plausible combination of unwrapped phase hypotheses the one for which said variance or standard deviation is smallest.
4. Method as claimed in any one of claims 1 to 3, wherein said distance is calculated as
4π · ||« ||
where 'est is an orthogonal projection onto said origin-crossing straight line of a point having said most plausible combination of unwrapped phase hypotheses as coordinates, and ΰ is said direction vector [c/Ai , ... , c/An].
5. Method as claimed in any one of claims 1 to 3, wherein said calculating of the gap is carried out in a rotated coordinate system, in which said origin-crossing straight line is a coordinate axis.
6. Method as claimed in claim 5, wherein said distance is calculated as
Figure imgf000020_0001
where ( '^ opt is a coordinate of said point having said most plausible combination of unwrapped phase hypotheses as coordinates in said rotated coordinate system, and ΰ is said direction vector [c/Ai , ... , c/An].
7. Computer program, comprising computer-implementable instructions, which, when executed by a computer, cause said computer to carry out the method as claimed in any one of claims 1 to 6.
8. Time-of-f light distance measuring device, comprising an electronic control unit with a memory, said memory having stored therein a computer program as claimed in claim 7, said electronic control unit being configured to execute said computer program when determining a distance.
9. Time-of-f light distance measuring device, comprising an electronic control unit with and FPGA and/or an ASIC, said FPGA and/or ASIC configured and arranged to carry out the method as claimed in any one of claims 1 to 6.
PCT/EP2014/055068 2013-03-20 2014-03-14 Distance determination method Ceased WO2014146978A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
DE112014001541.5T DE112014001541T5 (en) 2013-03-20 2014-03-14 Method for determining a distance
CN201480016730.9A CN105143916B (en) 2013-03-20 2014-03-14 distance determination method
US14/778,918 US10795021B2 (en) 2013-03-20 2014-03-14 Distance determination method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
LU92173A LU92173B1 (en) 2013-03-20 2013-03-20 Distance determination method
LULU92173 2013-03-20

Publications (1)

Publication Number Publication Date
WO2014146978A1 true WO2014146978A1 (en) 2014-09-25

Family

ID=47998496

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2014/055068 Ceased WO2014146978A1 (en) 2013-03-20 2014-03-14 Distance determination method

Country Status (5)

Country Link
US (1) US10795021B2 (en)
CN (1) CN105143916B (en)
DE (1) DE112014001541T5 (en)
LU (1) LU92173B1 (en)
WO (1) WO2014146978A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017018399A1 (en) * 2015-07-30 2017-02-02 株式会社デンソー Time-of-flight distance measuring device
WO2017022152A1 (en) * 2015-07-31 2017-02-09 パナソニックIpマネジメント株式会社 Range imaging device and solid-state imaging device
JP2019504326A (en) * 2016-02-08 2019-02-14 株式会社デンソー Time-of-flight distance measuring apparatus and method for detecting multipath errors
JPWO2021124918A1 (en) * 2019-12-18 2021-06-24

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9702976B2 (en) * 2014-10-27 2017-07-11 Microsoft Technology Licensing, Llc Time of flight camera
US10024966B2 (en) * 2015-01-28 2018-07-17 Texas Instruments Incorporated Efficient implementation of distance de-aliasing for ranging systems using phase domain computation
US10598783B2 (en) 2016-07-07 2020-03-24 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
US10884109B2 (en) * 2018-03-30 2021-01-05 Microsoft Technology Licensing, Llc Analytical-adaptive multifrequency error minimization unwrapping
CN109782262A (en) * 2019-01-30 2019-05-21 Oppo广东移动通信有限公司 Phase ranging method and device, split type electronic equipment and readable storage medium
US10929956B2 (en) * 2019-07-02 2021-02-23 Microsoft Technology Licensing, Llc Machine-learned depth dealiasing
DE102019121686A1 (en) * 2019-08-12 2021-02-18 Basler Ag Measurement and compensation of phase errors in time-of-flight cameras
DE102019122891A1 (en) * 2019-08-27 2021-03-04 Basler Ag Multipath and noise reduction for time-of-flight cameras
US11619723B2 (en) * 2019-09-23 2023-04-04 Microsoft Technology Licensing, Llc Multiple-mode frequency sharing for time-of-flight camera
US11651503B2 (en) 2020-06-16 2023-05-16 Microsoft Technology Licensing, Llc Determining depth in a depth image
US12228650B2 (en) * 2020-08-10 2025-02-18 Luminar Technologies, Inc. Master-oscillator power-amplifier (MOPA) light source with optical isolator
FR3115887B1 (en) * 2020-10-30 2022-12-02 St Microelectronics Grenoble 2 Method for acquiring depth mapping by indirect time of flight and corresponding sensor
US20220252729A1 (en) * 2021-02-08 2022-08-11 Microsoft Technology Licensing, Llc DETERMINING PHASE ORDERS IN ToF IMAGING

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10039422A1 (en) * 2000-08-11 2002-02-28 Siemens Ag Methods and devices for operating a PMD system
US7791715B1 (en) 2006-10-02 2010-09-07 Canesta, Inc. Method and system for lossless dealiasing in time-of-flight (TOF) systems

Family Cites Families (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU715284B2 (en) * 1996-09-05 2000-01-20 Rudolf Schwarte Method and apparatus for determining the phase and/or amplitude information of an electromagnetic wave
US6288776B1 (en) * 1999-11-24 2001-09-11 Eastman Kodak Company Method for unambiguous range estimation
DE60130968T2 (en) * 2000-01-10 2008-07-10 Massachusetts Institute of Technology Inc., Cambridge APPARATUS AND METHOD FOR SURFACE CONTOUR MEASUREMENT
AU2002241784A1 (en) * 2000-11-02 2002-05-27 Zygo Corporation Height scanning interferometry method and apparatus including phase gap analysis
GB0103669D0 (en) * 2001-02-15 2001-03-28 Central Research Lab Ltd A method of estimating the carrier frequency of a phase-modulated signal
WO2003093759A2 (en) * 2002-05-02 2003-11-13 Zygo Corporation Phase gap analysis for scanning interferometry
AU2003294323A1 (en) * 2002-11-19 2004-06-15 Radatec, Inc. Method and system for calibration of a phase-based sensing system
US7268889B2 (en) * 2004-09-22 2007-09-11 Corning Incorporated Phase-resolved measurement for frequency-shifting interferometry
GB0520332D0 (en) * 2005-10-06 2005-11-16 Roke Manor Research Calibration of phased array antennas
EP1793243A1 (en) * 2005-12-05 2007-06-06 Leica Geosystems AG Method for resolving a phase ambiguity
US20070127009A1 (en) * 2005-12-07 2007-06-07 Su-Ling Chen Multi-modulation frequency laser range finder and method for the same
EP1860462A1 (en) * 2006-05-23 2007-11-28 Leica Geosystems AG Distance measuring method and distance meter for determining the spatial dimension of a target
US7471376B2 (en) * 2006-07-06 2008-12-30 Canesta, Inc. Method and system for fast calibration of three-dimensional (3D) sensors
US8629976B2 (en) * 2007-10-02 2014-01-14 Microsoft Corporation Methods and systems for hierarchical de-aliasing time-of-flight (TOF) systems
EP2073035A1 (en) * 2007-12-18 2009-06-24 IEE INTERNATIONAL ELECTRONICS &amp; ENGINEERING S.A. Recording of 3D images of a scene
US20090322859A1 (en) * 2008-03-20 2009-12-31 Shelton Damion M Method and System for 3D Imaging Using a Spacetime Coded Laser Projection System
US8194233B2 (en) * 2008-04-11 2012-06-05 Microsoft Corporation Method and system to reduce stray light reflection error in time-of-flight sensor arrays
US8203699B2 (en) * 2008-06-30 2012-06-19 Microsoft Corporation System architecture design for time-of-flight system having reduced differential pixel size, and time-of-flight systems so designed
JP5448617B2 (en) * 2008-08-19 2014-03-19 パナソニック株式会社 Distance estimation device, distance estimation method, program, integrated circuit, and camera
US8010316B2 (en) * 2008-12-11 2011-08-30 Intermec Ip Corp. System and method for laser range-finding
US8988661B2 (en) * 2009-05-29 2015-03-24 Microsoft Technology Licensing, Llc Method and system to maximize space-time resolution in a time-of-flight (TOF) system
US8848171B2 (en) 2009-12-22 2014-09-30 Leica Geosystems Ag Highly accurate distance measurement device
EP2378310B1 (en) * 2010-04-15 2016-08-10 Rockwell Automation Safety AG Time of flight camera unit and optical surveillance system
US8294610B2 (en) * 2010-04-22 2012-10-23 L-3 Communications Integrated Systems L.P. Systems and methods for resolving interferometric angle-of-arrival ambiguities due to local multipath reflections
US8587771B2 (en) * 2010-07-16 2013-11-19 Microsoft Corporation Method and system for multi-phase dynamic calibration of three-dimensional (3D) sensors in a time-of-flight system
US9753128B2 (en) * 2010-07-23 2017-09-05 Heptagon Micro Optics Pte. Ltd. Multi-path compensation using multiple modulation frequencies in time of flight sensor
GB201013896D0 (en) * 2010-08-19 2010-10-06 Isis Innovation Apparatus and method for measuring distance
US8648702B2 (en) 2010-08-20 2014-02-11 Denso International America, Inc. Combined time-of-flight and image sensor systems
DE102010043768B3 (en) * 2010-09-30 2011-12-15 Ifm Electronic Gmbh Time of flight camera
US9194953B2 (en) * 2010-10-21 2015-11-24 Sony Corporation 3D time-of-light camera and method
FR2966933B1 (en) * 2010-11-02 2013-05-24 Thales Sa METHOD FOR REALIZING HIGH RESOLUTION ANALYSIS OF A SPACE AREA USING A FREQUENCY AGILE PULSE WAVE
US10571571B2 (en) * 2010-11-12 2020-02-25 Texas Instruments Incorporated Method and apparatus for controlling time of flight confidence map based depth noise and depth coverage range
US9151842B2 (en) * 2010-11-12 2015-10-06 Texas Instruments Incorporated Method and apparatus for time of flight sensor 2-dimensional and 3-dimensional map generation
EP2477043A1 (en) * 2011-01-12 2012-07-18 Sony Corporation 3D time-of-flight camera and method
KR101854188B1 (en) * 2011-10-25 2018-05-08 삼성전자주식회사 3D image acquisition apparatus and method of acqiring depth information in the 3D image acquisition apparatus
JP6246131B2 (en) * 2012-01-10 2017-12-13 ソフトキネティック センサーズ エヌブイ Improvements in or relating to processing of time-of-flight signals
KR101862199B1 (en) * 2012-02-29 2018-05-29 삼성전자주식회사 Method and Fusion system of time-of-flight camera and stereo camera for reliable wide range depth acquisition
US8686367B2 (en) * 2012-03-01 2014-04-01 Omnivision Technologies, Inc. Circuit configuration and method for time of flight sensor
US20140049767A1 (en) * 2012-08-15 2014-02-20 Microsoft Corporation Methods and systems for geometric phase unwrapping in time of flight systems
US9175957B2 (en) * 2012-09-24 2015-11-03 Alces Technology, Inc. Grayscale patterns from binary spatial light modulators
US9602807B2 (en) * 2012-12-19 2017-03-21 Microsoft Technology Licensing, Llc Single frequency time of flight de-aliasing
US9497440B2 (en) * 2013-04-05 2016-11-15 Microsoft Technology Licensing, Llc Burst-mode time-of-flight imaging
US9488722B2 (en) * 2013-06-05 2016-11-08 Samsung Electronics Co., Ltd. Time-of-flight ranging system and method with extended range
US10061028B2 (en) * 2013-09-05 2018-08-28 Texas Instruments Incorporated Time-of-flight (TOF) assisted structured light imaging
US9778363B2 (en) * 2013-10-24 2017-10-03 Massachusetts Institute Of Technology Methods and apparatus for coded time-of-flight camera
EP3063585B1 (en) * 2013-11-01 2022-01-05 iRobot Corporation Scanning range finder
US9542749B2 (en) * 2014-01-06 2017-01-10 Microsoft Technology Licensing, Llc Fast general multipath correction in time-of-flight imaging
US9897699B2 (en) * 2014-07-09 2018-02-20 Massachusetts Institute Of Technology Methods and apparatus for virtual sensor array
EP2966475B1 (en) * 2014-07-09 2016-05-25 Softkinetic Sensors N.V. A method for binning time-of-flight data
US9702976B2 (en) * 2014-10-27 2017-07-11 Microsoft Technology Licensing, Llc Time of flight camera
US10024966B2 (en) * 2015-01-28 2018-07-17 Texas Instruments Incorporated Efficient implementation of distance de-aliasing for ranging systems using phase domain computation
US9864048B2 (en) * 2015-05-17 2018-01-09 Microsoft Technology Licensing, Llc. Gated time of flight camera
EP3118651B1 (en) * 2015-07-17 2021-04-21 Hexagon Technology Center GmbH Time-of-flight measuring device and time-of-flight measuring method with ambiguity solution in real time
JP6558122B2 (en) * 2015-07-30 2019-08-14 株式会社デンソー Optical flight rangefinder
US10190983B2 (en) * 2016-04-15 2019-01-29 Massachusetts Institute Of Technology Methods and apparatus for fluorescence lifetime imaging with pulsed light
US10598783B2 (en) * 2016-07-07 2020-03-24 Microsoft Technology Licensing, Llc Multi-frequency unwrapping

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10039422A1 (en) * 2000-08-11 2002-02-28 Siemens Ag Methods and devices for operating a PMD system
DE10039422C2 (en) 2000-08-11 2002-08-01 Siemens Ag Methods and devices for operating a PMD system
US7791715B1 (en) 2006-10-02 2010-09-07 Canesta, Inc. Method and system for lossless dealiasing in time-of-flight (TOF) systems

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
D. DROESCHEL; D. HOLZ; S. BEHNKE: "Multi-frequency Phase Unwrapping for Time-of-Flight Cameras", PROCEEDINGS OF IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS, October 2010 (2010-10-01)
DORRINGTON A A ET AL: "Achieving sub-millimetre precision with a solid-state full-field heterodyning range imaging camera; Sub-millimetre precision with a heterodyning range imaging camera", MEASUREMENT SCIENCE AND TECHNOLOGY, IOP, BRISTOL, GB, vol. 18, no. 9, 20 July 2007 (2007-07-20), pages 2809 - 2816, XP020118817, ISSN: 0957-0233, DOI: 10.1088/0957-0233/18/9/010 *
DROESCHEL D ET AL: "Multi-frequency Phase Unwrapping for Time-of-Flight cameras", INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2010 IEEE/RSJ INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 18 October 2010 (2010-10-18), pages 1463 - 1469, XP031920128, ISBN: 978-1-4244-6674-0, DOI: 10.1109/IROS.2010.5649488 *
PAYNE ANDREW D ET AL: "Multiple frequency range imaging to remove measurement ambiguity", 2 July 2009 (2009-07-02), pages 139 - 148, XP002660844, Retrieved from the Internet <URL:http://researchcommons.waikato.ac.nz/bitstream/10289/4032/1/Multiple%20Frequency%20Range%20Imaging.pdf> *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017018399A1 (en) * 2015-07-30 2017-02-02 株式会社デンソー Time-of-flight distance measuring device
JP2017032342A (en) * 2015-07-30 2017-02-09 株式会社デンソー Optical flight rangefinder
WO2017022152A1 (en) * 2015-07-31 2017-02-09 パナソニックIpマネジメント株式会社 Range imaging device and solid-state imaging device
JPWO2017022152A1 (en) * 2015-07-31 2018-05-17 パナソニックIpマネジメント株式会社 Ranging imaging device and solid-state imaging device
JP2019504326A (en) * 2016-02-08 2019-02-14 株式会社デンソー Time-of-flight distance measuring apparatus and method for detecting multipath errors
JPWO2021124918A1 (en) * 2019-12-18 2021-06-24
WO2021124918A1 (en) * 2019-12-18 2021-06-24 ソニーグループ株式会社 Signal processing device, signal processing method, and range finding device
JP7517349B2 (en) 2019-12-18 2024-07-17 ソニーグループ株式会社 Signal processing device, signal processing method, and distance measuring device

Also Published As

Publication number Publication date
CN105143916A (en) 2015-12-09
LU92173B1 (en) 2014-09-22
US20160047913A1 (en) 2016-02-18
DE112014001541T5 (en) 2015-12-03
CN105143916B (en) 2017-05-31
US10795021B2 (en) 2020-10-06

Similar Documents

Publication Publication Date Title
US10795021B2 (en) Distance determination method
US10430956B2 (en) Time-of-flight (TOF) capturing apparatus and image processing method of reducing distortion of depth caused by multiple reflection
US12298395B2 (en) Velocity estimation using doppler per point lidar systems
US10878589B2 (en) Time-of-flight depth measurement using modulation frequency adjustment
US10091492B2 (en) Imaging apparatuses and a time of flight imaging method
EP2596321B1 (en) Methods and systems for hierarchical de-aliasing time-of-flight (tof) systems
EP3814797A1 (en) Phase noise and methods of correction in multi-frequency mode lidar
US20170184704A1 (en) Apparatus for making a distance determination
US10684121B2 (en) Distance-measuring apparatus which uses different calculations depending on whether the object is dynamically changing
US20160216376A1 (en) Efficient Implementation of Distance De-Aliasing for Ranging Systems using Phase Domain Computation
WO2017138032A1 (en) Time-of-flight distance measuring device and method for detecting multipath error
US20150310622A1 (en) Depth Image Generation Utilizing Pseudoframes Each Comprising Multiple Phase Images
US20200182984A1 (en) Apparatuses and Methods for Determining Depth Motion Relative to a Time-of-Flight Camera in a Scene Sensed by the Time-of-Flight Camera
CN112904305B (en) LiDAR device and method of operating the same
JP7149941B2 (en) Apparatus and method
US11686828B2 (en) Method and apparatus for compensating stray light caused by an object in a scene that is sensed by a time-of-flight camera
US11849223B2 (en) Time of flight camera data processing system
JP2023091179A (en) Range finder, sensor device
US20200175706A1 (en) Three-dimensional position detecting device, three-dimensional position detecting system and method for detecting three-dimensional positions
CN114599999A (en) Movement amount estimation device, movement amount estimation method, movement amount estimation program, and movement amount estimation system
EP4485007A1 (en) Information processing device, information processing method, and program
Langmann et al. Real-time image stabilization for ToF cameras on mobile platforms
CN116320667A (en) Depth camera and method for eliminating motion artifact
JP7264474B2 (en) Object detection device and object detection method
US12498467B2 (en) Distance-measuring apparatus and distance-measuring method

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201480016730.9

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14710545

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 14778918

Country of ref document: US

Ref document number: 112014001541

Country of ref document: DE

Ref document number: 1120140015415

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14710545

Country of ref document: EP

Kind code of ref document: A1