JP6261681B2 - Improvements in or relating to processing of time-of-flight signals - Google Patents

Description

  The present invention relates to improvements in or relating to processing of time-of-flight (TOF) signals, and in particular to de-aliasing of such time-of-flight signals. In particular, the present invention relates to TOF sensors, cameras, etc., including de-aliasing of time-of-flight signals.

  Time of flight (TOF) camera systems are known for providing information about the distance to an object by analyzing the time of flight from the light source to the object and back. One particular application is a camera that can capture a three-dimensional (3-D) image of a scene, ie, two-dimensional information and depth or distance information. Such a camera system is utilized in many applications where depth or distance information from a fixed point needs to be determined. In general, depth or distance information is measured from a TOF camera system.

  The basic operating principle of the TOF camera system is to illuminate the scene using modulated light such as pulses. The modulated light such as these light pulses is reflected back from the object in the scene, and the lens collects the reflected light, and the object in the scene is focused on the image sensor, particularly on the sensor surface of the sensor. The image is formed. Depending on the distance of the object from the camera, a delay occurs between the emission of the modulated light, for example a pulse, and the reception of its reflection at the camera, for example an object located 2.5 m away from the camera A delay of 16.66 ns is generated. However, when using pulses, the pulse width of each light pulse determines the camera range, for example with a pulse width of 50 ns, the range is limited to 7.5 m. As a result, scene illumination becomes essential to the operation of the TOF camera system and the requirement for the illumination unit is the use of a dedicated light source, such as a light emitting diode (LED) or laser, to generate such short light pulses. Need.

  Another major component of the TOF camera system is an imaging sensor. An imaging sensor typically includes an array of pixels that form an image of a scene. In addition, the output of the pixel may be used to determine the time of flight of light from the illumination unit to the object in the scene and light reflected back from the object to the imaging sensor. The time of flight may be calculated in a separate processing unit coupled to the sensor, or the processing unit may be incorporated in the sensor. Various methods are known for measuring the timing at which light travels from the illumination unit to the object and returns from the object to the imaging sensor.

  As is well known, each pixel substantially includes a photosensitive element that receives incident light and converts it into an electrical signal, eg, a current signal. In an analog timing imaging sensor, the pixel is connected to a switch that directs the current signal to a memory that acts as an adder. In digital timing imaging sensors, a high frequency counter is used to detect the current signal. The presence of background light can cause the distance to be erroneously determined when the pixel receives additional light that is not reflected from objects in the scene.

  Of course, the imaging optics and processing electronics also form part of the TOF camera. The imaging optics collects light reflected from objects in the scene and removes light that is not the same wavelength or frequency as the light emitted by the illumination unit. By removing light at unnecessary wavelengths or frequencies, background light can be effectively suppressed. The processing electronics include drivers for both the illumination unit and the imaging sensor so that these components can be precisely controlled to ensure that a high resolution image of the scene can be acquired.

  TOF camera systems tend to cover a wide range from several millimeters to several kilometers, depending on the type of imaging sensor utilized. Such TOF camera systems may have various distance resolutions from less than a centimeter to several centimeters, or even meters, depending on the configuration of the TOF camera system used in a particular application. Technologies that may be used with a TOF camera system include pulsed light sources with digital time counters, radio frequency (RF) modulated light sources with phase detectors, and range-gated imagers.

  Although it may be relatively easy to extract distance information from the output of an imaging sensor for an object in the scene, the TOF system has the background light described above and two or more TOF camera systems overlap the scene. It is affected by interference and multiple reflection when operating with a field of view. Interference between different TOF camera systems can be reduced by using time division multiplexing techniques or by utilizing different modulation frequencies for each TOF camera system. Since the TOF camera system illuminates the entire scene, multiple reflections may be obtained as the illuminating light reaches each object along several paths. This has the effect that the measured distance of the object from the TOF camera system may be determined as greater or less than its actual distance from the TOF camera system. This is a major drawback of the conventional TOF camera system. Measurement ambiguity in a range imaging system is D. Payne (AD Pine) et al., Described in a paper entitled “Multiple Frequency Range Imaging to Remove Measurement Ambiguity”, in which: To overcome the phase ambiguity problem that leads to inaccurate distance measurements, two different modulation frequencies are superimposed in a single acquisition (http://researchcommons.waikato.ac.nz/bitstram/10289/ 4032/1 / Multiple% 20Frequency% 20Range% 20Imaging.pdf).

  S. H. McClure (SH McClure) et al., “Resolving Depth Measurement Ambientity with Commercially Available Range Imaging Cameras” In Electronic Imaging, SPIE vol. 7538, pp. 75380K, 2010), software post-processing is used to resolve depth ambiguity. Range data is processed to divide the scene into separate objects, and the average intensity of each object is used to determine which pixels are outside the unambiguous range. This method can be used with any range imaging camera, such as robot vision, where the camera may be moving during image capture and has reduced sensitivity to reflectance differences from objects in the scene. Can be used in the system. In signal processing and related fields, aliasing means the effect of causing different signals to become indistinguishable (ie, alias each other). Temporal aliasing is when a sample becomes indistinguishable in time. Temporal aliasing can occur when a periodically sampled signal also has periodic content. In TOF systems, aliasing introduces an ambiguity regarding the distance between the light source and the object that reflects the light.

  “Phase unwrapping” is due to David Droeschel, Dirk Holz, and Sven Behnke, “Probabilistic Phase Unwrapping for Time-of-Flight” Proceedings of Joint 41st International Symposium on Robotics and 6th German Conference on Robotics, Munich, June 2010. ”, Proceedings of Joint 41st International Symposium on Robotics and 6th German Conference on Robotics, Munich, June 2010. Multiple relative phase shifts are inferred from the wrapped phase signal, where a probabilistic approach is used to detect phase jumps or phase shifts based on discontinuities in the depth image containing distance information. The The authors said, “Multi-frequency Phase Unwrapping for Time-of-Flight Cameras (Multi-frequency phase unwrapping for time-of-flight cameras)” (Processings of IEEE / RSJ International Conf. (Taipei, Taiwan, October 2010) also describes phase unwrapping.

  R. Schwarte et al., “A New Active 3D-Vision System Based on RF-Modulation Interferometry of Incoherent Light” (New Active 3D System Based on RF Modulation Interferometry of Incoherent Light) Photonics East-Intelligent Systems and Advanced Manufacturing, Proceedings of the SPIE, Vol. 2588, Philadelphia, 1995), a method for processing signals from a TOF camera system is disclosed. Use phase shift algorithm Where the phase is proportional to the distance from the TOF camera system, where the measurement uncertainty depends on the number of phase measurements, the modulation contrast, the electrons in the active and noise signals, and the wavelength of the modulation signal. Can be determined. Such a system has been described as being useful for both pedestrian and vehicle safety systems.

  U.S. Pat. No. 7,791,715 (B) describes a method of dealiasing TOF phase-derived data by obtaining phase data using at least two modulation frequencies. To perform de-aliasing, pixel detection information is captured in at least two, preferably four separate phases. These distinct phases represent the shift between the modulator in the TOF camera system and the illumination unit. This is different from the desired phase shift detected between the emitted light and the reflected light from the object received by the imaging sensor. The separate phases are subtracted from each other to cancel the fixed pattern offset between the modulator and the illumination unit. By using two modulation frequencies, the system behaves as if phase data is being collected while operating at a very slow modulation frequency proportional to the difference between the two modulation frequencies. . The use of multiple modulation frequencies provides better depth measurement certainty and accuracy than can be achieved using only one modulation frequency.

  The use of multiple modulation frequencies is described in Canesta Inc. (Kanesta, Inc.) “A Time-of-Flight Depth Sensor Depth Sensor Depth Sense Sensor, Sir Burak Gokturk, Hakan Yalcin, and Cyrus Bamji. -System description, problems and solutions) ". In one embodiment, two modulation frequencies are used and the results are combined to provide a depth value for the object that is more accurate compared to a single modulation frequency. In addition, the non-ambiguous or unambiguous range is extended. In another embodiment, multiple measurements are made using the modulation frequencies at f, 2f, 4f, 8f, etc., and the results at each modulation frequency are combined to provide an accurate value for the distance over a long unambiguous or obvious range. Can be provided. In each successive measurement, the resolution is doubled and the range is halved.

  Due to the periodic nature of the irradiation source used, i.e. due to the modulation of the irradiation source, the distance range intervals tend to repeat. This phenomenon is known as TOF aliasing as described above. Currently solving this TOF aliasing problem, allowing the TOF camera system to provide a sufficiently long unambiguous or obvious distance for the particular application in which the TOF camera system is implemented. There are a limited number of technologies. However, these techniques have several known limitations because they are very complex to implement, require intensive calculations, are not robust, generate bandwidth overhead in data transmission, or This is because it tends to reduce depth accuracy or overall camera performance.

  Accordingly, it is an object of the present invention to provide an alternative apparatus and method for processing time-of-flight (TOF) signals, and in particular, an apparatus and method for de-aliasing of such time-of-flight signals. In particular, it is an object of the present invention to provide alternative TOF sensors, cameras, etc. that include time-of-flight signal de-aliasing.

One advantage of the present invention is that de-aliasing of the TOF signal can be performed in an efficient manner. Aspects of embodiments of the present invention are as follows.
a) Two correlation processes are used, the first correlation process is for determining distance or depth values, and the second correlation process is whether aliasing is present in the first correlation It is for judging. These two processes may be performed independently of each other.
b) The first correlation of a) above may be given higher priority than the second correlation, for example, the amount of computation for the first correlation is greater for the first than for the second And / or the integration time for the signal in the first correlation may be longer than for the second.
c) Whereas one correlation process is used to obtain depth or distance information, two methods or processes may be used to determine if aliasing exists. The first of these processes preferably reduces the number of possible aliased values, and the second process then reduces the number of possible aliased values to 1 or, eg, by a removal process. Concentrate or reduce to around 1.
d) The decision process, i.e. whether aliasing is present, is preferably based on simple arithmetic, e.g. over or under decision.

According to a first aspect of the present invention, in the correlation measurement in a time-of-flight system for use with an illumination unit having an imaging sensor, the presence of aliasing is determined within certain limits or aliasing is avoided. A method is provided, wherein the illumination unit is adapted to illuminate the scene with the modulated light, and the system is adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from the scene. And
This method
a) detecting reflected light from at least one object in the scene;
b) determining a first correlation between the first correlation signal and the reflected light received at the imaging sensor by means of one or more first correlation measurements; Having a periodic waveform of one frequency, the first correlation measurement provides a value of a parameter relating to the distance from the imaging sensor to at least one object;
c) determining a second correlation between the reflected light received at the imaging sensor and the second correlation signal by means of one or more second correlation measurements; A periodic waveform of a second frequency lower than the frequency of 1,
d) For the value of the parameter, including determining the presence of aliasing from the second correlation measurement.

The first and second signals may be used to modulate light for illuminating the scene, and the second frequency is different from the first frequency. Alternatively, the third signal used to modulate the light from the illumination unit may have a different duty cycle than the second signal used for the second correlation. The second signal may be offset compared to the first signal. A third process is provided to determine distance related parameters, i.e. distance between sensor and scene, to remove any possible aliasing that may still exist after the second correlation step. May be. This distance related parameter may be a confidence level, such as a reflectance map.

According to another aspect of the present invention, there is provided a method for determining the presence of aliasing in phase measurements in a time-of-flight system, for example within at least certain limits, the system comprising an imaging sensor and with an illumination unit The illumination unit is adapted to illuminate the scene with modulated light having a periodic waveform, and the system correlates by applying a correlation signal to the modulated light reflected from the scene. Adapted to get measurements,
This method
a) providing a modulated signal used by the illumination unit to illuminate the scene with light modulated at the first frequency and the second frequency;
b) detecting reflected light from at least one object in the scene at a first frequency;
c) determining a first correlation between the modulated signal and the reflected light received at the imaging sensor;
d) determining a second correlation between the illumination light emitted by the illumination unit and the reflected light received at the imaging sensor;
e) determining from the first and / or second correlation the in-phase and / or quadrature component of the detected reflected light and using the in-phase and / or quadrature component to determine the presence of aliasing. Including.

The presence or ambiguity of aliasing may be determined within certain distance limits, for example, as determined by the first and second frequencies and the relationship between them. This ambiguity means that the ToF distance can be reliably determined up to a certain distance. The effects or ambiguity of aliasing can be further reduced in some embodiments if confidence levels are determined for distance measurements based on distance dependent factors or processes other than TOF. Such distance-related factors can be passive, such as the amount of IR light reflected from the object, the size of the speckle in the returning image, or projected by a divergence pattern and seen by the sensor. It may be active, such as measuring the size of the pattern. According to an embodiment of the present invention, a reliability threshold is applied. This confidence threshold may be based, for example, on the distance-dependent characteristics of objects in the scene that can be detected. A threshold may be applied on a pixel-by-pixel basis. In alternative embodiments, the confidence threshold is based on a portion of the complete image, for example, based on object (s) that can be detected in the image by image processing techniques, or for the entire image. May be applied. For example, the reflectance of objects detected in the scene may be used to confirm that the measured distance may be correct. By using an intelligent threshold process where the confidence threshold is distance dependent, the maximum distance that can actually be supported by the TOF method of the present invention is increased, or the maximum that can be reliably detected It is possible to reduce the minimum reflectivity so that the distance can be extended.

  The term quadrature refers to a signal having phase information. In general, such signals are 90 ° out of phase, but the invention is not limited to implying 90 ° quadrature. Other angles located within 360 ° may be used, such as 60, 30 °, etc. It is used by embodiments of the present invention that two signals at different frequencies fall within different sectors of the constellation diagram in such a way that aliasing can be determined at least within certain limits. The aliasing determination may be made by examining a decision region in the constellation diagram.

  Preferably, the presence of aliasing is determined from the sign of the detected in-phase and / or quadrature component of the reflected light.

Preferably, phase measurements are determined from correlations at various phase offsets for the first frequency. Preferably, phase measurements are determined from correlations at various phase offsets for the second frequency. Optionally, the second frequency is 2- ⁿ of the first frequency, where n = 1, 2,. . . Etc.

  Preferably, the step of determining the I value and the Q value is performed at the first frequency and / or the second frequency. The phase measurement step using the first and / or second frequency may include determining the phase measurement using the I and Q values.

  In an embodiment of the present invention, determining the presence of aliasing comprises, for example, comparing the I value and / or the Q value with 0 at the first frequency and / or the second frequency, and / or Including performing simple arithmetic operations, such as determining the sign of the Q value. The sectors of the constellation circle may be used to associate various phase offsets with the decision algorithm to determine the presence of aliasing.

In another aspect, the present invention provides a time-of-flight or range-finding sensor having means for determining the presence of aliasing within specified limits or for avoiding aliasing, the sensor comprising: And for use with a second signal, also for use with an illumination unit, the illumination unit being adapted to illuminate the scene with modulated modulated light, the system comprising: Adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from
The sensor
a) means for providing a first signal used by the illumination unit to illuminate the scene;
b) means for detecting reflected light from at least one object in the scene at a first frequency;
c) means for determining a first correlation between the modulated signal and the reflected light received in the imaging;
d) means for determining a second correlation between the reflected light received at the imaging sensor and the second signal;
e) means for determining the distance from the imaging sensor to the scene from the first correlation and determining the presence of aliasing from the second correlation.

Means may be provided such that the second signal may be used to modulate the light for illuminating the scene with a second frequency that is different from the first frequency. Or means are provided such that the third signal used to modulate the light from the illumination unit may have a different duty cycle than the second signal used for the second correlation. Also good. Means may be provided such that the second signal may be offset compared to the first signal. Means are provided for determining a distance related parameter, i.e. a parameter relating to the distance between the sensor and the scene, to remove any possible aliasing that may still exist after the second correlation step. May be. This distance related parameter may be a confidence level, such as a reflectance map.

In yet another aspect, the present invention provides a time-of-flight or range detection sensor having means for determining the presence of aliasing within certain limits or for avoiding aliasing, the sensor comprising an imaging sensor, For use with an illumination unit, the illumination unit being adapted to illuminate the scene with modulated light having a periodic waveform,
a) means for providing a modulated signal used by the illumination unit to illuminate the scene with light modulated at the first frequency and the second frequency;
b) means for detecting reflected light from at least one object in the scene;
c) first means for determining, for a first frequency, a first correlation between the modulated signal and the reflected light received at the imaging sensor;
d) a second means for determining, for a second frequency, a second correlation between the illumination light sent by the illumination unit and the reflected light received at the imaging sensor;
e) determining from the first and / or second correlation an in-phase and / or quadrature component of the detected reflected light and using the in-phase and / or quadrature component to determine the presence of aliasing; 3 means.

  The means for detecting the reflected light may be a sensor having pixels. The means for determining the correlation and / or the means for determining the in-phase and / or quadrature component, or the means for determining the presence of aliasing are processing units connected to or integrated with the sensor. May be. For example, these means may be provided by a computer and the output of the sensor is connected to the computer.

The presence or ambiguity of aliasing may be determined within certain limits by the first and second frequencies and their separation, or the relationship between them. This ambiguity means that the ToF distance can be reliably determined up to a certain distance. The effects or ambiguity of aliasing may be further reduced in some embodiments if confidence levels are determined for distance measurements based on factors or processes other than ToF. According to an embodiment of the present invention, a reliability threshold is applied. This confidence threshold may be based, for example, on the distance-dependent characteristics of objects in the scene that can be detected. Such distance-related factors can be passive, such as the amount of IR light reflected from the object, the size of the speckle in the returning image, or projected by a divergence pattern and seen by the sensor. It may be active, such as measuring the size of the pattern. A threshold may be applied on a pixel-by-pixel basis. In alternative embodiments, the confidence threshold is based on a portion of the complete image, for example, based on object (s) that can be detected in the image by image processing techniques, or for the entire image. May be applied. For example, the reflectance of objects detected in the scene may be used to confirm that the measured distance may be correct. By using an intelligent threshold process where the confidence threshold is distance dependent, the maximum distance that can actually be supported by the TOF camera system of the present invention can be increased or reliably detected. It is possible to reduce the minimum reflectivity so that the maximum distance can be extended.

  Preferably, the third means is adapted to determine the presence of aliasing from the sign of the in-phase and / or quadrature component of the reflected light to be detected.

  Preferably, the first means is adapted to make a phase measurement from the correlation at various phase offsets for the first frequency. Preferably, the second means is adapted to make a phase measurement from the correlation at various phase offsets for the second frequency.

Optionally, the second frequency is 2- ⁿ of the first frequency, where n = 1, 2,. . . Etc.

  Preferably, the third means for determining is adapted to determine the I and Q values at the first frequency and / or the second frequency.

  In an embodiment of the present invention, the third means for determining comprises comparing the I value and / or Q value at the first frequency and / or the second frequency, for example by comparing the I value and / or Q value with zero. It is adapted to determine the presence of aliasing by determining the sign of the value.

  Any of the above sensors may be part of a TOF device.

  For a better understanding of the present invention, reference will now be made by way of example only to the accompanying drawings in which:

A phase circle with four phase quadrants or sectors is shown. Similar to FIG. 1, but with 8 phase sectors. 2 shows a graph illustrating the effects of de-aliasing and the reflectance of objects in the scene. Fig. 4 shows a graph showing the modulation signals for light, their reflections, and the signals required to drive the TOF multiplier mix input for one of the techniques. Fig. 4 shows a graph illustrating the modulation signals for light, their reflections, and the signals required to drive the TOF multiplier mix input for another one of the techniques. 1 illustrates a TOF camera according to an embodiment of the present invention. It is the schematic of a TOF system.

  The present invention will be described with respect to particular embodiments and with reference to certain drawings but the invention is not limited thereto. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated for illustrative purposes and not drawn to scale.

  In the remainder of this specification, the principle of de-aliasing will be described based on phase measurements. However, it should be noted that the present invention can be used for other periodic signals (eg, pseudo-random bitstreams or other digital signals).

  Referring to FIG. 7, in the TOF measurement, the irradiation light from the TOF camera system having the irradiation unit travels to points in the scene, and the reflected light from those points returns to the sensor unit of the TOF camera system. The round trip time is measured. In conventional TOF camera systems, the illumination light includes periodically modulated light that is generated by driving the illumination unit using a signal from a signal generator. One option for determining time of flight is to determine the phase delay between the light emitted from the TOF camera system for illuminating the scene and the light reflected from the received scene. This is done in a correlation and evaluation unit that receives the modulated signal from the signal generator.

This phase delay may then be converted to the measured distance, i.e. the distance of at least one point of the object from the TOF camera system, using the period length and the speed of light. This may be done on a pixel-by-pixel basis. Due to the periodic nature of the modulated light, the phase delay measurement is repeated (ie, aliased) at a particular distance, which occurs when the phase delay exceeds a multiple of the period. This repetition of phase delay measurements is referred to as “aliasing” of the signal, as described above. In essence, the TOF distance measurement D is

Where c is the speed of light, f is the frequency of the modulated light, phase_deg is the phase shift in degrees, and k is related to the number of “aliased” periods. The first factor in this equation is the aliasing distance for systems that do not have provisions for dealiasing, which corresponds to the following obvious distance.

This distance represents the maximum distance that may be included within one period. Larger distances are k iterations, so they are aliased if included in the image, and knowledge about k is required to be clearly measured.

  A distance measurement based on this principle is theoretically never uniquely defined and corresponds to an unlimited number of possible distance measurements. In the current system, it is assumed that the value of k is equal to 0 for all measurements. Thus, in practice, if k> 0 for some pixels, this assumption is not correct and the measurements are inaccurate.

In accordance with an aspect of the present invention, a novel de-aliasing method is described that identifies k to be 0 or within a specific range and / or to identify the exact value of k. May be used. Embodiments of the present invention describe a method for determining the presence of aliasing or avoiding aliasing within certain limits in correlation measurements in a time-of-flight system for use with an illumination unit having an imaging sensor. The illumination unit is adapted to illuminate the scene with the modulated light, and the system is adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from the scene. The method steps may be performed in any suitable order. This method
a) detecting reflected light from at least one object in the scene;
b) determining a first correlation between the first correlation signal and the reflected light received at the imaging sensor by means of one or more first correlation measurements; Having a periodic waveform of one frequency, the first correlation measurement provides a value of a parameter relating to the distance from the imaging sensor to at least one object;
c) determining a second correlation between the reflected light received at the imaging sensor and the second correlation signal by means of one or more second correlation measurements; A periodic waveform of a second frequency lower than the frequency of 1,
d) For the value of the parameter, it may include determining the presence of aliasing from the second correlation measurement.

  The parameter may be a phase difference. In this method, the first phase measurement may be determined from the first correlation measurement at various phase offsets for the first frequency. Preferably, the signal integration time of the detected reflected signal in the first correlation measurement is longer than the signal integration time of the second correlation measurement. This improves the accuracy of the distance measurement and at the same time still resolves aliasing.

  The second correlation signal may have a duty cycle that is different from the duty cycle of the detected reflected modulated light to which it is correlated. The duty cycle of the second correlation signal may be 50%.

  The first and / or second correlation measurements are combined as an aliasing indicator. Such an aliasing indicator may be an I value and a Q value. Another indicator may be based on distance related factors, such as reflectance from objects in the scene.

This method
Determining a second correlation measurement using various phase offsets for the second frequency;
Determining from the one or more second correlation measurements an in-phase and / or quadrature component of the detected reflected light and using the in-phase and / or quadrature component to determine the presence of aliasing; May further be included.

The presence of aliasing may be determined by using a confidence level determined from measuring distance related factors. The distance related factor may be the amount of reflected light, the size of the speckle pattern, the size of the pattern to be imposed.

  The presence of aliasing may be determined from indicators such as the sign of the in-phase and / or quadrature component of the reflected light that is detected.

  A second phase measurement may be determined from the second correlation measurement at various phase offsets for the second frequency. The first frequency is different from the second frequency, and is, for example, higher than the second frequency or a multiple of the second frequency.

  Determining in-phase and / or quadrature components may include determining an I value and a Q value at the first frequency and / or the second frequency.

  The presence of aliasing may be determined from indicators such as the sign of I and / or Q and / or the difference between the absolute values of I and Q.

Embodiments describe a time-of-flight or range detection sensor unit having means for determining the presence of aliasing within certain limits or avoiding aliasing, the sensor being for use with an illumination unit; The illumination unit is adapted to illuminate the scene with modulated light, and the system is adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from the scene,
The sensor unit
a) first means for detecting reflected light from at least one object in the scene;
b) second means for determining a first correlation between the first signal and the reflected light received at the imaging sensor by means of the first correlation measurement (first correlation measurement; Provides parameter values for the distance from the imaging sensor to the scene)
d) third means for determining a second correlation between the reflected light received at the imaging sensor and the second signal by means of the second correlation measurement; Having a periodic waveform of a second frequency lower than the frequency of 1),
e) fourth means for determining the presence of aliasing from the second correlation measurement for the value.

  The parameter may be a phase difference. The second means may be adapted to determine the first phase measurement from the first correlation measurement at various phase offsets for the first frequency.

  Preferably, the second and third means are adapted such that the signal integration time of the first correlation measurement is longer than the integration time of the second correlation measurement.

  The second correlation signal may have a duty cycle that is different from the duty cycle of the detected reflected modulated light to which it is correlated. The duty cycle of the second signal may be 50%.

The third means for determining the second correlation may be adapted to perform the second correlation measurement at various phase offsets for the second frequency.
The sensor unit may be adapted to determine an in-phase and / or quadrature component of the detected reflected light from the second correlation measurement, and the fourth means is in-phase and / or quadrature Ingredients are used to determine the presence of aliasing.

The presence of aliasing may be determined by using a confidence level determined from distance related factors. The distance related factor may be the amount of reflected light modulated at the first frequency and / or the second frequency, the size of the speckle pattern, the size of the pattern to be imposed.

  The means for determining the in-phase and / or quadrature component may be adapted to determine the presence of aliasing from the sign of the detected in-phase and / or quadrature component of the reflected light.

  The second means for determining may be adapted to make a phase measurement from the first correlation at various phase offsets for the first frequency.

  The third means for determining may be adapted to make a phase measurement from the second correlation at various phase offsets for the second frequency.

  The first frequency is different from the second frequency, and is, for example, higher than the second frequency or a multiple of the second frequency.

  The fourth means for determining in-phase and / or quadrature components may be adapted to determine the I and Q values at the second frequency.

  The fourth means for determining the in-phase and / or quadrature component may be adapted to determine the presence of aliasing by determining the sign of the I value and / or Q value.

  The first and / or second signal is preferably calibrated for phase offset to avoid offset errors.

  The sensor of the sensor unit may be for use with a correlation and evaluation unit that may include a processor such as a microprocessor or FPGA.

  Although the first and second signals are described for correlation with reflected light, the present invention includes the use of more signals that may be cascaded or applied repeatedly, These signals may be at a different frequency than the first and second signals.

  The first advantage is that when determining whether k is 0, the distance measurement is now a uniquely defined distance measurement, so that the system as described above when k> 0 Output of inaccurate distance information is avoided. A second advantage is that if the value of k is recognized, it is further possible for the system to convert this identified information into a longer distance range. A third advantage is that higher modulation frequencies can be used and may have an advantageous effect on depth accuracy.

In some distance measurement techniques such as TOF, additional values for the intensity of the reflected active infrared light used in the measurement are also obtained, which we refer to as confidence . Reliability decreases with the square of the distance.

The first approach in resolving aliasing ambiguity is to look at the confidence to determine k. This is not a good solution because reliability is not only a function of distance, but also a function of the reflectivity of the surface in the scene. Thus, a very low confidence may correspond to an object that is separated by more than one period (k> 0) or to a nearby object with very low reflectivity (eg a black leather jacket). Therefore, to rely on this principle, the reflectivity in the scene must be known for every pixel in the scene. In such cases, the reliability may be used to determine k. In practice, however, this is a rare case.

The approach used in the present invention relies in part on reliability . The first approach is to suppress all pixels that may have k> 0 based on confidence . As indicated above, the confidence does not allow a pixel with k> 0 to be distinguished from a nearby lower reflectance pixel, both are suppressed. Applying this principle therefore ensures that measurements with k> 0 are suppressed, but also good measurements below the aliasing distance below a certain reflectance limit are suppressed. In the first order, this minimum reflectivity limit is proportional to the square of the maximum distance required for the application and inversely proportional to the square of the apparent distance. This reflectivity limitation is a significant limitation of the above technology, as it is the purpose in many use cases of this technology to meet high reflectivity and very low reflectivity levels, such as low reflectivity clothing. is there. The object of the present invention is therefore to minimize this limitation. This is obtained by increasing the apparent distance or reducing the maximum distance required for the application. However, the latter provides little or no outlook because in most cases this is a fixed parameter as defined by the application and is therefore not considered in the rest of this document. The apparent distance can be easily increased by reducing the modulation frequency. However, this adversely affects the uncertainty associated with distance measurements. Therefore, other means of increasing the apparent distance described herein are important.

Using more complex reliability threshold mechanisms such as adaptive thresholds that are distance dependent, it is possible to increase the maximum supported distance at a specific reflectance limit and / or decrease the minimum reflectance limit. is there.

  For ease of explanation, a general TOF camera system is assumed throughout this specification. In order to measure phase in the TOF camera system, the in-phase or I value and the quadrature or Q value are determined. The term quadrature means a signal having phase information. In general, such signals are 90 ° out of phase, but the invention is not limited to implying 90 ° quadrature. Other angles located within 360 ° may be used, such as 60, 30 °, etc. The aliasing determination may be made by examining such a decision region in the constellation diagram.

According to the embodiment of the present invention, the I value and the Q value are determined. This is typically done by measuring the correlation between the emitted and reflected optical signals at multiple phase offsets, such as two or more, or three, four different phase offsets. For example, the four offsets are generally 0 °, 90 °, 180 °, and 270 °. When using these measurements, the I and Q values are
I = M_0-M_180
Q = M_90-M_270
Where M_0, M_90, M_180 and M_270 are measured values at 0 °, 90 °, 180 ° and 270 °, respectively.

I and Q are then standard trigonometric relations φ = atan 2 (Q, I)
May be used to calculate phase φ.

  The modulation frequency used as illumination light for TOF measurement is referred to as the first frequency or base frequency (BF) throughout the following description. In addition, the second frequency or derived frequency is referred to as HALF and QUATER, for example, according to their relationship to BF.

  It is also an object of the present invention to provide a technique that makes it possible to distinguish aliasing and determine the aliasing state by adding an additional shorter measurement following the TOF distance measurement. The purpose of the present invention is to keep the integration time required to make this measurement as short as possible, to save energy in the required irradiation, to concentrate as much as possible, and to provide a long integration time for the actual distance measurement. Is to make it possible. The longer the integration time, the less uncertainty. However, the available integration time is limited by the required frame rate, for example 20 ms for 50 fps. Therefore, it is important and very advantageous to have a technique for identifying aliasing within a very short time frame.

According to the first embodiment of the invention, one additional measurement is made at the lower second frequency using the standard TOF technique described above. This measurement is made, for example, at the second frequency HALF, ie at the first frequency or half of the BF. In that case, the system can use the following relationship to determine whether k is an even number and k is an odd number.
When QHALF> 0 → TOF measurement when k is an even number When QHALF <0 → TOF measurement when k is an odd number

  It is therefore possible to reduce the possible value for k to either odd or even by simply examining the sign of a single measurement. Thus, the apparent distance is increased by a factor of 2, which reduces the reflectance limit by a factor of four.

  However, due to noise in this measurement, for example, if QHALF is very close to 0, the measured value may be misinterpreted.

  Alternatively, IHALF measurements may also be used with the same problem of noise in measurements using the same relationship as above.

  To improve this, the temporal coherency of the measurement may be taken into account and the distances that completely or nearly completely change the aliasing distance may be rejected because they may be due to the noise described above Because. The aliasing distance is generally quite large, for example 7.5 m for a modulation frequency of 20 MHz. Surrounding pixels may also be taken into account after or in conjunction with temporal coherency.

  Another approach to improving immunity to noise is to add additional redundancy, for example, by examining both not only IHALF or QHALF but then also using QHALF, IHALF codes, and QBF and IBF codes. To add in the measurement. In this way, four possible results and conclusions can be obtained, as will be described in more detail below.

  QBF and IBF are generally readily available because they must be calculated to complete the TOF measurement itself, and therefore are not considered additional required measurements.

  FIG. 1 shows a constellation diagram 10 divided into four quadrants 1, 2, 3, and 4. Each quadrant 1, 2, 3, and 4 is between 0 ° and 90 °, between 90 ° and 180 °, between 180 ° and 270 °, and 270 ° and 0 ° or 360 °, respectively. Corresponds to the phase range between. For the first frequency or base frequency and the second frequency, which is half of the base frequency, Table 1 below shows that each row corresponds to a HALF or low frequency quadrant in circle 10 and each column is BF or high. Indicates that it corresponds to the frequency quadrant. For each measurement, the sign of QHALF, IHALF, QBF, and IBF may then be used to determine the quadrant for BF or first frequency, and HALF or lower frequency. Using these quadrants, the correct quadrant values for HALF or lower frequencies are shown in Table 1. This correct value is then used to determine the odd and even signs of k, as indicated above. In this way, noise up to 90 ° can be corrected due to the redundancy of the measured values.

Therefore, the confidence threshold must first be set to remove all possible measurements with k> 1, then the HALF quadrant is recognized and the value of “HALF — 3” or “HALF — 4” is simply It means that one aliasing distance needs to be added, and the quadrant value of “HALF_1” or “HALF_2” means that no aliasing exists. Thus, by adding HALFQ and HALFI measurements and only looking at the signs of HALFQ, HALFI, QBF, IBF, it is possible to achieve doubling of the apparent distance.

  Since only the sign needs to be distinguished for HALFQ and HALFI measurements, it is desirable to limit the amount of active light used to obtain these measurements. As indicated above, approximately 90 ° of noise can be corrected using this technique. Therefore, system operation should be designed to avoid noise above 90 °, as can be done by those skilled in the art of 3D time-of-flight cameras.

  Furthermore, a second important constraint is the offset between measurements made at both frequencies. This offset is derived for the above reason from an acceptable noise level of 90 °. Therefore, it is important to reduce this offset as much as possible. This may be done, for example, by 1) phase shifting the modulated signal, or 2) later computationally rotating one of the received correlations.

  Reducing the offset may be done by measuring the phase for each frequency at a fixed known distance and correcting for the measured offset during the TOF camera calibration phase during manufacture. This offset value is stored in the digital memory and is then used to apply a delay to one of the emitted or reflected light to compensate for the offset.

  In a typical implementation, the first frequency or high modulation frequency is achieved using the minimum number of digital steps allowed by the base clock in the TOF camera system, and the second frequency or minimum frequency is Made with that base clock tick. Therefore, it is necessary to adjust only the second frequency or the lowest frequency (in this case HALF) in order to match the delay to the first frequency or the highest frequency (BF).

  In addition, the remaining inconsistencies may be removed using a PLL or delay line.

  Alternatively, synchronization may be achieved by rotating the I and Q coordinates using a (2 × 2) × (2 × 1) matrix operation, where the (2 × 2) matrix is determined during calibration. Or may be defined as a fixed matrix for a particular TOF camera system production line.

  However, the first embodiment of the present invention is up to twice the unambiguous or unambiguous distance obtainable at the first frequency or the original modulation frequency or BF signal. Note that there is a clear limitation in that it can only be detected.

  In the second embodiment of the present invention, the second frequency may be a QUATER frequency, for example, for a first frequency BF of 20 MHz, the second or lowest frequency that may be used is 5 MHz. It is. This provides an extension of the range where identification can result, up to four times the aliasing distance range.

  FIG. 2 shows a circle 20 divided into eight sectors 1, 2, 3, 4, 5, 6, 7, and 8 as shown. In this case, there are eight possible cases shown in Table 2 below that need to be examined to avoid jumps from one case to another due to the presence of noise in the measurement.

  As with the HALF frequency above, an implementation without redundancy but with a tendency to noise is possible. In this case, this is based on splitting the QUATER frequency into only four quadrants. As before, compensation for the effects of noise can be provided using temporal coherency and / or coherency with neighboring pixels.

  In Table 2, each row corresponds to a second frequency QUATER or low frequency sector in circle 20, and each column corresponds to a first frequency BF or high frequency quadrant. The values in Table 2 correspond to the correct QUATER. In this way, BF or high frequency noise up to 90 ° can be corrected due to the redundancy of the measured values.

In Table 2, Cases 1-8 can be distinguished on the assumption that the following steps are performed.
1) Check the sign of I 2) Check the sign of Q 3) Check if I> Q

  As before, all these operations are simple comparison operations, but the constraints remain the same, ie, the first frequency BF or the high frequency signal 90 ° is the second frequency QUATER or low Equal to 22.5 ° of the frequency signal. This is also the reason why each of Tables 1 and 2 has four columns.

Distinguishing the case of QUATER_ 1 to QUATER_ 8 for lower frequencies will then help to identify:
QUATER_1 or QUATER_2: k is [0; 4; 8; . . ] QUATER_3 or QUATER_4: k belonging to [1; 5; 9; . . ] QUATER_5 or QUATER_6: k belonging to [2; 6; 10; . . QUATER_7 or QUATER_8: k belonging to [3; 7; 11; . . ] Belonging to

The apparent distance is thereby extended by a factor of four. Thus, the reflectivity limit that needs to be imposed on the confidence threshold is greatly reduced as desired.

  It will be readily appreciated that the present invention is not limited to the second frequency being the HALF or QUATER frequency, and may be extended to other frequencies such as EIGHTH and lower fractional frequencies. However, with the use of EIGHTH or lower fractional frequencies, there is a related increase in computational requirements, because in this case the technique has at least 8 and 16 preferred cases so that all aliasing patterns can be identified. It is because it uses.

  Extending the aliasing range up to four times the unambiguous or obvious spacing of BF or high frequencies is often already sufficient to achieve fully de-aliased distance measurements or related images. It is. However, it will be understood that further expansion of the unambiguous or apparent range of the TOP camera system is within the scope of the present invention.

  The trade-off between reflectivity and distance is shown in FIG. 3, where a graph of relative signal strength versus distance is plotted. Trace 30 relates to the unaliased signal when the reflectivity is 4%, and trace 35 relates to the aliased signal when the reflectivity is 100%.

  However, it has been found that increasing the modulation frequency is beneficial for the performance of the TOF camera system, but it may not be appropriate to extend the maximum range of the system by a factor of four. If a first frequency BF of 100 MHz with a natural unambiguous or obvious distance of 1.5 m is used, a second frequency that is the QUATER frequency described with respect to the second embodiment of the invention above. Is used, the unambiguous or obvious distance is now 6 m. However, due to the size of the area in which the TOF camera system operates, the distance measurement is limited to 3 m and the reflectivity limit is 25% in the first order, which makes it very difficult to determine the distance for lower reflectivity objects. To. Moreover, for systems where de-aliasing at distances greater than 6 m is required, the second frequency, which is the QUATER frequency, is not sufficient.

  As mentioned above, a second frequency that is an EIGHTH frequency may be used for these situations, but with more complex comparisons. As an alternative to more complex comparisons, iterative techniques may be used to extend the unambiguous or apparent distance.

  An iterative method may be implemented using the second frequency, which is the HALF frequency described above in the first embodiment of the present invention. This extends the unambiguous or apparent distance by 100%, ie, doubles the apparent distance obtained using the first frequency BF. In the first iteration, the distance is expanded 100% (doubled). In the second iteration, the distance is further expanded by 100% (substantially four times), and in the third iteration, the distance is further expanded by 100% (multiplied by eight). At each iteration, the signs of I and Q are determined so that the presence of aliasing in the measurement can be determined.

  It will be appreciated that iterations can be performed indefinitely, but acceptable noise limits must be carefully observed. The maximum allowable noise for one step is 90 °, but this must then be observed at all stages of this iterative process.

  Alternatively, instead of using a second HALF frequency iteration, an iteration may be performed for the second QUATER frequency, which allows a greater extension of the distance over which an object can be measured using the same number of measurements. To. In addition, the iterations may be combined using both the second HALF and QUATER frequencies. It may also be possible to include the second HALF frequency in the iteration, thereby further extending the unambiguous or obvious distance in other ways.

  TOF measurements are generally performed by making a plurality of different measurements. These measurements differ in the phase difference or time delay difference between the modulated signal emitted by the illumination source and that received at the imaging sensor of the TOF camera system. These measurements may be referred to as “phase frames”. In general, 0 °, 90 °, 180 °, and 270 ° are used for the phase frame. Each of these measurements may be performed in parallel or sequentially. Since the present invention requires that one or more additional measurements be made, various approaches are possible to perform these additional measurements.

  One way to perform additional measurements is to reduce the length of measurement time for the phase frame and add additional measurements as needed. In one implementation, four additional phase frames are required, resulting in measurements at 0_HF, 90_HF, 180_HF, 270_HF, 0_LF, 90_LF, 180_LF, 270_LF, where HF means high frequency and LF is Related to low frequencies. In general, these measurements are performed sequentially, and the measurement time required for sufficient accuracy of de-aliasing is much shorter for LF compared to HF.

  Another method is to add these additional measurements after each of the individual phase frames, resulting in measurements at 0_HF, 0_LF, 90_HF, 90_LF, 180_LF, 180_HF, 270_LF, 270_HF.

  These measurements may be performed in parallel if the system allows. The method of the present invention may be implemented using low level electronic circuitry. This method can be easily implemented in the driver circuit on the host or even behind it, but generally in firmware running on a companion chip or field programmable gate array (FPGA). Implemented. This method may also be incorporated into an integrated circuit (IC) of the imaging sensor. In addition, the method of the present invention is particularly useful because it is based on very simple comparison logic rather than on difficult algebraic functions such as arctangent (tan-1 or arctan), square root (sqrt), etc.

The system of the present invention can be optimized by reducing the number of measurements required. According to the TOF theory, at least three measurements are preferred. It is therefore possible to measure only the 0 ° and 90 ° values together with the reference value. Here, the required I value and Q value may be determined as follows.
I = M_0 ° -reference Q = M_90 ° -reference where E_M_0 ° and M_90 ° are measurements taken at 0 ° and 90 °, respectively.

  Although the present invention has been described using measurements taken at 90 ° intervals, such as 0 °, 90 °, 180 °, and 270 °, for example, other measurements may be used. Will be understood. The measurement interval may be 0 °, 120 °, and 240 °, or any other suitable measurement interval. When in-phase and / or quadrature signals are evaluated, intervals of 0 °, 90 °, 180 °, and 270 ° or 90 °, 180 °, 270 °, and 360 ° tend to be used.

  In another implementation of the invention, different duty cycles are used for the modulation signal sent to the light source and the mix signal sent to the mixer 29 in the sensor (see FIG. 6) for additional lower frequency measurements. May be. An exemplary frame with signals used for such an approach is shown in FIG. The frame has a length 110 and, as is common in these types of measurements, the illustrated frame is repeated multiple times for a single measurement to improve the signal-to-noise ratio. A common example of a number is 100 ns for frame length, 1000 iterations, and integration over an effective time of 100 μs.

For example, a signal such as a pulse 106 having a length of 100, for example half of the BF period, is used as the modulation signal for illumination, and the sensor mixer has a duty longer than the pulse length 100, for example corresponding to the HALF frequency pulse length. It is driven using a pulse 107 having a cycle length 103. Using such a signal, it can be checked whether k is odd or even as follows.
1) First measured value (M1) for the MIX signal shown in FIG.
2) Second measured value (M2) for the inverted MIX signal shown in FIG.
As indicated by RETURN_LIGHT_A in FIG. 4, M1> M2 indicates k = even. As indicated by RETURN_LIGHT_B in FIG. 4, M2> M1 indicates k = odd number.

  The waveform in FIG. 4 is preferably configured such that the distance where M1 = M2 matches the aliasing distance for BF. For this reason, the start of pulse 106 shown in FIG. 4 is phase shifted from the start of pulse 107. The pulse 106 spans the frame boundary and thus has a portion at the beginning of the frame and a portion at the end of the frame. If a smaller pulse width 100 is selected, more contrast is obtained with respect to the value of k, but as the pulse width 100 is selected shorter, less energy is released by the irradiation and thus a longer integration time. Note also that it is required to obtain a specific noise level.

  The advantage of this technique is that only two measurements have to be made. No additional redundancy is required. Therefore, more time is available to perform the actual TOF distance measurement.

An exemplary frame with signals used for the more general approach of the above duty cycle change technique is shown in FIG. Again, the frame has a length 210 and is considered to be continuously repeated during additional measurements for dealiasing. The pulse length 203 may be arbitrarily selected to examine with respect to k, for example, the pulse length 203 may be selected to match the aliasing distance, and the pulse length 204 may be selected to match twice the aliasing distance. Makes it possible to check for k as follows.
1) Measure M1 for the MIX signal shown in FIG. 5 2) Measure M2 for the inverted MIX signal shown in FIG. 5 M1> M2, where k is [0; 3; 4; 7; 8; 11 . . . M2> M1 indicating that the k belongs to [1; 2; 5; 6; 9; 10. . . ] Belonging to

  Thus, with only two additional measurements, the apparent distance extension can be up to k = 2 (k = 3 is similar to k = 0).

  Any confirmation about k, including non-integer numbers, can be confirmed in a similar manner by selecting different pulse lengths 203 and 204, as the above is an example, and can be configured by one skilled in the art. Please note that. Also, like the previous technique, this technique may be repeated to further extend the apparent distance.

  If a smaller pulse width 200 is selected, more contrast is obtained with respect to the value of k, but as the pulse width 200 is selected shorter, less energy is released by the irradiation and thus a longer integration time. Note again that it is required to obtain a specific noise level.

  It is important that the MIX signal waveform still has an average 50% balanced duty cycle. This is primarily to ensure that the background light is balanced and canceled out during the various required TOF measurements. As an example, as shown in FIG. 5, extending the pulse so that the MIX signal is high at the beginning and end of the period is achieved. The starting pulse length 205 is actually preceded by the pulse length 203 (at the end of the previous frame) and balanced, so both sums equal 204.

FIG. 6 shows another embodiment of a TOF camera or range detection system according to the present invention. The range detection system includes a light source 49 for emitting periodic light 51 onto the scene 55, preferably focused on a region of interest that reflects light. The periodic light 51 includes two detectably different frequencies—a first frequency or base frequency and a second lower frequency. The range detection system further includes at least one pixel 31 for receiving the reflected light. The reflected light also includes signal components, such as pulses, at the first and second frequencies. In order for the light source 49 to emit modulated light, a signal generator 43 is provided. The signal generator 43 generates on the node 48 a signal or modulated signal that is preferably permanently oscillating at a predetermined first frequency or base frequency, for example at about 10 MHz. This signal generator 43 also generates similar second to fifth clock signals, which are 0 °, 180 °, 90 °, and 270 ° phase with the first clock signal on node 48, respectively. Delivered on nodes 44, 45, 46, 47 in a relationship. One skilled in the art can also consider using other or more clock phases within the operating scheme, with more clock phases at the expense of better measurement accuracy and longer measurement times. Bring. The signal generator also generates a signal at the second or lower frequency. Accordingly, the signal generator generates first to fifth clock signals at the second frequency or lower frequency. This generates, for example, a signal at the first frequency or base frequency, but changes the amplitude of the base frequency signal at regular intervals, for example at a fraction of the base frequency (eg this Can be achieved). For example, the second frequency may be 2- ⁿ of the first frequency, where n = 1, 2,. . . Etc. By filtering, it is possible to extract either the signal at the first or second frequency from the modulated signal and, for example, from the reflected signal from the object under observation.

  The signal generator 43 is a control signal 41 that determines the modulation signal change by the modulation signal changing means, for example, between the second to fifth clock signals by the selector 58, that is, between various phases of the clock signal. A control signal 41 that determines the selection of these may also be generated. The selector 58 sequentially switches between these four phases to connect the input node 42 of the mixer 29 of the detector and mixer stage 200 to the second to fifth clock signals on the nodes 44, 45, 46 and 47. Connect sequentially. In each of these positions, the selector 58 may remain connected for a relaxation period of about 1 ms, for example.

  The buffer 50 drives a light source 49 that emits light 51 onto the scene 55, preferably focused on the region of interest. A portion of this light is reflected, thus producing reflected light 52. This reflected light 52 then reaches an optical focusing system, such as a lens 56, through which it is imaged or focused on the detector 28 in the pixel 31, where a portion of the incidence is This is called reflected modulated light (ML) 27.

  Both indirect light 53 and direct light 54 originating from the secondary light source 30 not intended for TOF measurements are also present in the scene and hit the optical focusing system 56 and are therefore focused on the detector 28. A portion of this light entering the detector 28 is referred to as background light (BL) 26. The light source 30 that generates BL includes incandescent, TL, sunlight, daylight, or any other light that is present on the scene and does not originate from the light source 49 for TOF measurement.

  ML27 and BL26 impinge on the photodetector 28 and generate an ML current and a BL current, respectively, which are photoinduced current responses by the BL26 and ML27 that are in contact therewith. The detector 28 outputs these currents to subsequent mixing means, such as a mixer 29, for mixing the current response due to the hitting BL 26 and ML 27 with the phase shifted clock signal on the input node 42. This BL 26 may induce a BL current up to 6 orders of magnitude greater than the ML current induced by the ML 27 received for TOF measurements.

  The detector 28 and the mixer 29 forming the detector and mixer stage 200 are mixed with photogenerated charges as described in, for example, European Patent Application Publication No. 1513202 (A1). May be implemented as one single device that generates The detector and mixer stage 200 generates a mixing product of the current response from the impinging BL 26 and ML 27 and the phase-shifted clock signal, which are implemented on node 38 using, for example, capacitor 25. Integrated by the integrator, the capacitor 25 is preferably kept at a small capacitance, for example the parasitic capacitance of the surrounding transistors. During integration, an automatic reset of the mixer output signal on integrator node 38 is performed. This is performed for both the first and second frequencies.

  This may be done, for example, by the comparator 33 triggering a reset switch, such as a reset transistor 32, so that the mixer output signal on node 38 is automatically generated when it reaches the reference value Vref. Reset, thus avoiding saturation.

  In alternative embodiments not shown in the drawings, the automatic reset of the mixer output signal on integrator node 38 may be implemented in several other ways. One of them is to trigger a charge pump in place of the reset switch 32 to add a certain amount of charge to the capacitor 25, which at the expense of some increase in complexity. Good noise performance is provided.

  The mixing product forming the mixer output signal is obtained at the integrator node 38 in sequential form synchronized with the modulation signal modification means (selector 58 in the example shown). The output driver 24, eg, a buffer, provides a substantially unity voltage gain and current amplification, thereby providing a stronger output signal at the output node 23.

  By filtering, either the signal at the first or second frequency can be extracted from the reflected signal from the object under observation.

  While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; The invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, upon review of the drawings, the disclosure, and the appended claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Appendix 1
A method of determining or avoiding aliasing within specified limits in correlation measurements in a time-of-flight system for use with an illumination unit having an imaging sensor, wherein the illumination unit is modulated Adapted to illuminate a scene with light, wherein the system is adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from the scene;
The method
a) detecting reflected light from at least one object in the scene;
b) determining a first correlation between a first correlation signal and the reflected light received at the imaging sensor according to one or more first correlation measurements; The signal has a periodic waveform of a first frequency, and the first correlation measurement provides a value of a parameter related to a distance from the imaging sensor to the at least one object;
c) determining, by one or more second correlation measurements, a second correlation between the reflected light received at the imaging sensor and a second correlation signal, the second correlation; The signal has a periodic waveform of a second frequency lower than the first frequency;
d) determining the presence of aliasing from the second correlation measurement for the value of the parameter.
Appendix 2
A first phase measurement is determined from the first correlation measurement at various phase offsets for the first frequency, and if necessary, a second phase measurement is obtained for the second frequency. The method of claim 1, wherein the method is determined from the second correlation measurement at various phase offsets.
Appendix 3
The method of claim 1 or 2, wherein the signal integration time of the detected reflected signal in the first correlation measurement is longer than the signal integration time of the second correlation measurement.
Appendix 4
The second correlation signal has a duty cycle different from the duty cycle of the detected reflected modulated light to which it is correlated, and the duty cycle of the second correlation signal is 50% as required The method according to any one of appendices 1 to 3.
Appendix 5
Determining the second correlation measurement using various phase offsets for the second frequency;
Determining an in-phase and / or quadrature component of the detected reflected light from one or more second correlation measurements and using the in-phase and / or quadrature component to determine the presence of aliasing. The method according to any one of appendices 1 to 3, further comprising:
Appendix 6
The presence of aliasing is determined by using a confidence level determined from measuring distance-related factors, where the distance-related factors include, as necessary, the amount of reflected light, the size of the speckle pattern, The method according to any one of appendices 1 to 5, which is a size of a pattern to be imposed.
Appendix 7
The first correlation measurement and / or the second correlation measurement are combined as an aliasing indicator, and the presence of aliasing is determined from an indicator that is a sign of the in-phase and / or quadrature component of the detected reflected light. The method according to any one of appendices 1 to 6, wherein:
Appendix 8
The step of determining an in-phase and / or quadrature component includes determining an I value and a Q value and / or a difference between absolute values of I and Q at the first frequency and / or the second frequency. The method according to any one of appendices 5 to 7, comprising:
Appendix 9
A time of flight or range detection sensor unit,
Means for determining the presence of aliasing within certain limits or avoiding aliasing, the sensor being for use with an illumination unit, said illumination unit illuminating the scene with modulated light And the system is adapted to obtain a correlation measurement by applying a correlation signal to the modulated light reflected from the scene,
The sensor unit is
a) first means for detecting reflected light from at least one object in the scene;
b) second means for determining, by a first correlation measurement, a first correlation between a first signal and the reflected light received at the imaging sensor;
Wherein the first correlation measurement provides a parameter value relating to a distance from the imaging sensor to the scene;
d) a third means for determining, by a second correlation measurement, a second correlation between the reflected light received at the imaging sensor and a second signal;
Here, the second signal has a periodic waveform having a second frequency lower than the first frequency,
e) a sensor unit comprising: fourth means for determining the presence of aliasing from the second correlation measurement value for the value.
Appendix 10
The second means is adapted or determined to determine a first phase measurement from the first correlation measurement at various phase offsets for the first frequency. Is adapted to make a phase measurement from the second correlation at various phase offsets for the second frequency, or the second and third means are signals of the first correlation measurement The sensor unit of claim 9, wherein the sensor unit is adapted such that an integration time is longer than an integration time of the second correlation measurement value.
Appendix 11
The second correlated signal has a duty cycle that is different from the duty cycle of the detected reflected modulated light to which it is correlated, and optionally the duty cycle of the second signal is 50%; The sensor unit according to appendix 9 or 10.
Appendix 12
The third means for determining the second correlation is adapted to perform a second correlation measurement at various phase offsets for the second frequency;
The sensor unit is adapted to determine an in-phase and / or quadrature component of the detected reflected light from the second correlation measurement, and the fourth means is the in-phase and / or quadrature A phase component is used to determine the presence of aliasing, where the presence of aliasing is determined, if necessary, by using a confidence level determined from a distance related factor, the distance related factor being required Depending on the amount of reflected light modulated at the first frequency and / or the second frequency, the size of the speckle pattern, the size of the pattern to be imposed,
The means for determining in-phase and / or quadrature components is adapted to determine the presence of aliasing from the sign of the in-phase and / or quadrature components of the detected reflected light, if necessary.
The sensor unit according to any one of appendices 9 to 11.
Appendix 13
The fourth means for determining an in-phase and / or quadrature component is adapted to determine an I value and a Q value at the second frequency, or the second means for determining an in-phase and / or quadrature component. The sensor unit according to any one of appendices 9 to 12, wherein the means of 4 is adapted to determine the presence of aliasing by determining the sign of the I value and / or Q value.
Appendix 14
The sensor unit according to any one of appendices 9 to 13, wherein the first signal and / or the second signal are calibrated for a phase offset.
Appendix 15
A TOF camera having the sensor unit according to any one of appendices 9 to 14.

Claims (13)

It has an imaging sensor, in the correlation measurement of time-of the flight system for use with irradiation unit, a how you determine the presence of d-aliasing, the irradiation unit has a first frequency and a second Adapted to illuminate the scene with modulated light modulated at a frequency, the system adapted to obtain a correlation measurement between a correlation signal and the modulated light reflected from the scene;
The method
a) detecting reflected light from at least one object in the scene;
b) determining a first correlation between a first correlation signal and the reflected light received at the imaging sensor according to one or more first correlation measurements; signal equal to the modulated light to have a periodic waveform of said first frequency, said first correlation measurements provides the value of a parameter related to the distance from the imaging sensor to said at least one object,
c) determining, by one or more second correlation measurements, a second correlation between the reflected light received at the imaging sensor and a second correlation signal, the second correlation; signal equal to the modulated light to have a periodic waveform of lower second frequency from the first frequency,
d) determining an in-phase and / or quadrature component of the reflected light to be detected from a plurality of the second correlation measurements at a plurality of different phase offsets for the second frequency;
e) using the signs of the in-phase and / or quadrature components of the light to determine possible values for parameters associated with the presence of aliasing;
Including a method.   A first phase measurement is determined from the first correlation measurement at various phase offsets for the first frequency, and a second phase measurement is determined for various phase offsets for the second frequency. The method of claim 1, determined from the second correlation measurement at.   The method according to claim 1 or 2, wherein a signal integration time of the detected reflected signal in the first correlation measurement is longer than a signal integration time of the second correlation measurement.   2. The second correlation signal has a duty cycle that is different from the duty cycle of the detected reflected modulated light to which it is correlated, and the duty cycle of the second correlation signal is 50%. The method as described in any one of -3. The method according to claim 1, wherein the parameter related to the presence of aliasing is related to the number of aliased periods.   The presence of aliasing is further determined by using a distance measurement reliability determined from measuring a distance related factor to the at least one object, wherein the distance related factor is the amount of reflected light, The method according to claim 1, wherein the size of the speckle pattern is the size of the pattern to be imposed.   The step of determining an in-phase and / or quadrature component comprises determining a difference between an I value and a Q value and / or an absolute value of I and Q at the second frequency. The method according to any one of the above.   The method of claim 1, wherein the second correlation signal is a pulse-shaped signal that is high at the beginning and end of a period and has a duty cycle of 50%.   A time of flight or range detection sensor unit,
  Means for determining the presence of aliasing, wherein the sensor is for use with the illumination unit, the illumination unit using the modulated light modulated at the first frequency and the second frequency to Adapted to illuminate and the system is adapted to obtain a correlation measurement between the correlation signal and the modulated light reflected from the scene;
The sensor unit is
a) first means for detecting reflected light from at least one object in the scene;
b) a second means for determining, by a first correlation measurement, a first correlation between a first correlation signal and the reflected light received at the imaging sensor;
    Here, the first correlation signal is equal to the modulated light having a periodic waveform of the first frequency, and the first correlation measurement value is a parameter value related to a distance from the imaging sensor to the scene. Offer to,
c) third means for determining, by a second correlation measurement, a second correlation between the reflected light received at the imaging sensor and a second correlation signal;
    Here, the second correlation signal is equal to the modulated light having a periodic waveform of the second frequency lower than the first frequency, and the third means includes various components for the second frequency. Adapted to perform a second correlation measurement at the phase offset;
d) determining the in-phase and / or quadrature components of the reflected light detected from a plurality of the second correlation measurements at a plurality of different phase offsets for the second frequency, and for the aliased period A fourth means for determining the presence of said aliasing by determining possible values of a number related parameter using the sign of the in-phase and / or quadrature component of the light;
With
  The presence of aliasing is further determined by using a distance measurement reliability determined from a distance-related factor to the at least one object, the distance-related factor being the first frequency and / or the second. The amount of reflected light modulated at the frequency of, the size of the speckle pattern, the size of the pattern to be imposed,
Sensor unit.   The second means is adapted or determined to determine a first phase measurement from the first correlation measurement at various phase offsets for the first frequency. Is adapted to make a phase measurement from the second correlation at various phase offsets for the second frequency, or the second and third means are signals of the first correlation measurement The sensor unit according to claim 9, wherein the sensor unit is adapted such that an integration time is longer than an integration time of the second correlation measurement.   10. The second correlation signal has a duty cycle that is different from the duty cycle of the detected reflected modulated light to which it is correlated, and the duty cycle of the second correlation signal is 50%. The sensor unit according to 10.   The sensor unit according to any one of claims 9 to 11, wherein the first correlation signal and / or the second correlation signal are calibrated for phase offset.   A TOF camera having the sensor unit according to any one of claims 9 to 12.