CN116299363A - Two-phase sampling calibration and correction method and device of iToF module - Google Patents
Two-phase sampling calibration and correction method and device of iToF module Download PDFInfo
- Publication number
- CN116299363A CN116299363A CN202310272161.5A CN202310272161A CN116299363A CN 116299363 A CN116299363 A CN 116299363A CN 202310272161 A CN202310272161 A CN 202310272161A CN 116299363 A CN116299363 A CN 116299363A
- Authority
- CN
- China
- Prior art keywords
- phase
- sampling
- module
- value
- itof
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005070 sampling Methods 0.000 title claims abstract description 229
- 238000012937 correction Methods 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000008859 change Effects 0.000 claims abstract description 45
- 238000012360 testing method Methods 0.000 claims abstract description 18
- 238000005259 measurement Methods 0.000 claims description 40
- 230000003287 optical effect Effects 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 238000004441 surface measurement Methods 0.000 claims description 8
- 230000002596 correlated effect Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 239000004973 liquid crystal related substance Substances 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 17
- 230000008569 process Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000010363 phase shift Effects 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002542 deteriorative effect Effects 0.000 description 1
- 238000013101 initial test Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Abstract
The invention relates to a two-phase sampling calibration and correction method and device of an iToF module. According to the invention, the sampling time sequence of the iToF module with two taps of a single pixel is configured, the change curve of the measured phase difference value of the two-phase sample and the four-phase sample along with the change of the measured phase of the two-phase sample is obtained and stored as a lookup table, the proportional coefficient of the measured phase difference value of the two-phase sample and the four-phase sample of each pixel relative to the change curve in the lookup table is obtained and stored as a proportional coefficient table, the test result of the two-phase sample is obtained according to the sampling time sequence in a two-phase sampling mode, the correction value of the test result is obtained by searching the lookup table, and the proportional coefficient of the current pixel is obtained by searching the proportional coefficient table to further correct the correction value so as to correct each pixel under the two-phase sample. The two-phase sampling calibration method can ensure the ranging accuracy and the motion blur resistance, and reduce the hardware cost.
Description
Technical Field
The invention relates to the technical field of TOF ranging, in particular to a two-phase sampling calibration and correction method and device of an iToF module.
Background
Binocular ranging, structured light and Time of Flight (ToF) are three major mainstream 3D imaging technologies today, wherein ToF has been gradually applied to the fields of gesture recognition, 3D modeling, unmanned and machine vision due to the advantages of simple principle, simple and stable structure, long measurement distance and the like. The ToF technology is a method for precisely measuring the distance of an object, and includes a direct-time-of-flight (dtofr) ranging technology, i.e., directly measuring the time of flight of light to calculate the distance of the object, and an indirect-time-of-flight (iToF) ranging technology, i.e., periodically modulating and demodulating the light intensity, and then calculating the distance of the object using phase information.
Please refer to fig. 1 and fig. 2, wherein fig. 1 is a schematic diagram of an iToF imaging principle, and fig. 2 is a schematic diagram of continuous wave modulation ranging. Specifically, the iToF module controls the light emitting module 12 to actively emit continuously modulated light pulses as the emitted light 13 through the Modulation module (Modulation) 11; the emitted light 13 is emitted to the surface of the target object 19, and the reflected light 14 formed after being reflected by the target object 19 is captured by a photosensitive pixel Array (Pxiel Array) 15 of an image Sensor (Sensor); by calculating the Phase shift (Phase shift) of the emitted light 13 and the reflected light 14
To obtain the depth of the target 19. The light emitting module 12 may employ a Vertical Cavity Surface Emitting Laser (VCSEL), an infrared emitter (IR emitter), a Light Emitting Diode (LED), or the like. Since the speed of light c, the modulation frequency f of the emitted light is a known quantity, a phase shift +.>
On the basis of (a), the depth d of the target 19 can be obtained by the following formula:
for iToF technology, in order to implement accurate measurement of phase due to non-ideal factors such as ambient light and errors in manufacturing process, a multiphase sampling technology (commonly known as a four-phase sampling technology, and other single-pixel multi-tap sampling technologies) is generally adopted to eliminate these non-ideal factors, so as to ensure accuracy and robustness of phase information, i.e., distance information. However, since the time points of the multi-phase sampling are not the same, the actual object may be moved, and the depth calculated by the information of the multi-phase sampling is affected by the motion blur, which greatly affects the accuracy of the ranging. In order to ensure the ranging accuracy and the motion blur resistance at the same time, more taps are placed in a single pixel and a correction method is matched with the taps as a common solution, so that higher requirements are put on the design of the iToF module, the correction method and the storage of correction information, and the overall cost of the iToF module is increased due to the complicated correction method and data storage requirements.
Therefore, how to effectively ensure the ranging accuracy and the motion blur resistance without increasing the overall cost of the iToF module is a technical problem to be solved currently.
Disclosure of Invention
The invention aims to provide a two-phase sampling calibration and correction method and device for an iToF module, which effectively ensure the ranging accuracy and the motion blur resistance through the two-phase sampling calibration and reduce the hardware cost.
In order to achieve the above purpose, the present invention provides a two-phase sampling calibration and correction method of an iToF module, comprising the following steps: configuring sampling time sequences under two-phase correlated sampling of an iToF module with two taps of a single pixel, wherein the sampling time sequences are used for sampling a first phase and a second phase in an active light source on state of the iToF module and sampling for the third time in an active light source off state so as to detect the intensity of ambient light; acquiring a change curve of a measured phase difference value of two-phase sampling and four-phase sampling along with the change of the measured phase of the two-phase sampling, and storing the change curve as a lookup table; acquiring a proportional coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to a change curve in the lookup table, and storing the proportional coefficient as a proportional coefficient table; and under the two-phase sampling mode, obtaining a test result of the two-phase sampling according to the sampling time sequence, obtaining a correction value of the test result by searching the lookup table, and obtaining a proportion coefficient of a current pixel by searching the proportion coefficient table so as to further correct the correction value, thereby correcting each pixel under the two-phase sampling.
In order to achieve the above objective, the present invention further provides a two-phase sampling calibration and correction device of an iToF module, including: the system comprises a sampling time sequence configuration module, a sampling time sequence detection module and a sampling time sequence detection module, wherein the sampling time sequence configuration module is used for configuring sampling time sequences under two-phase associated sampling of an iToF module with two taps of a single pixel, the sampling time sequence is used for sampling a first phase and a second phase in an active light source on state of the iToF module and sampling for the third time in an active light source off state so as to detect the intensity of ambient light; the lookup table acquisition module is used for acquiring a change curve of a measured phase difference value of two-phase sampling and four-phase sampling along with the measured phase change of the two-phase sampling, and storing the change curve as a lookup table; the proportional coefficient table acquisition module is used for acquiring the proportional coefficient of the measured phase difference value of the two-phase sampling and the four-phase sampling of each pixel relative to the change curve in the lookup table and storing the proportional coefficient as a proportional coefficient table; and the two-phase correction module is used for obtaining a test result of two-phase sampling according to the sampling time sequence in a two-phase sampling mode, obtaining a correction value of the test result by searching the lookup table, and obtaining a proportion coefficient of a current pixel by searching the proportion coefficient table so as to further correct the correction value, so that each pixel in the two-phase sampling is corrected.
According to the invention, the distance measurement accuracy of the two-phase sampling is improved by calibrating and correcting and compensating the extra error generated by the two-phase sampling compared with the four-phase sampling, and the capability of resisting motion blur, ambient light interference and process error under the two-phase sampling of the iToF module is realized with lower storage cost and lower algorithm cost; the invention does not need extra hardware to store the image of the difference of the signal offset values of the pixel taps and the correction data of the difference of the charge-voltage conversion gains of the pixel taps, thereby reducing the storage cost of the system.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the present invention and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a schematic diagram of the principle of iToF imaging;
FIG. 2 is a schematic diagram of continuous wave modulation ranging;
FIG. 3 is a diagram of four-phase sampling phases of an iToF module;
FIG. 4 is a diagram of four-phase sampling timing of the iToF module;
FIG. 5 is a schematic diagram showing steps of a two-phase sampling calibration and correction method of the iToF module according to the present invention;
FIG. 6 is a schematic diagram of two phase sampling timing according to an embodiment of the present invention;
FIG. 7 is a flowchart of a calculation LUT according to an embodiment of the invention;
FIG. 8 is a schematic diagram of a measured phase difference between two-phase sampling and four-phase sampling according to a square wave algorithm according to an embodiment of the present invention;
FIG. 9 is a flow chart for obtaining scaling parameters according to an embodiment of the present invention;
fig. 10 is a block diagram of a two-phase sampling calibration and correction device of the iToF module according to the present invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
Please refer to fig. 3 and 4, wherein fig. 3 is a schematic diagram of four-phase sampling phases of the iToF moduleFig. 4 is a schematic diagram of a four-phase sampling timing diagram of the iToF module. In the figure, mod is a modulated (Modulation) optical signal waveform of emitted light, and Demod is a demodulated (Demodulation) optical signal waveform of reflected light captured by the iToF module photosensitive pixel array. To eliminate the effects of ambient light and non-idealities in the process, a four-phase sampling method is typically used for an iToF module with two taps A, B for a single pixel, with two taps A, B for each sample to obtain a set of measurements. The demodulation waveform in the four samplings is delayed by 1/4 of the modulation period respectively to obtain 4 groups of tap A, B values, namely A 0 、B 0 ，A 90 、B 90 ，A 180 、B 180 ，A 270 、B 270 This removes the effects of ambient light and process non-idealities.
For the case where the modulated signal waveform is a sine wave (sin), the value of tap A, B at the time of sampling of the different phases can be expressed by the following equation:
wherein, the subscript phi represents 0 degrees, 90 degrees, 180 degrees, 270 degrees and G A 、G B Charge-to-voltage conversion gain, amp, representative of the pixel tap A, B (Environment+DC) Adding the Direct Current (DC) component of the intensity of the ambient light (ambient) received by the pixel to the intensity of the light emitted by the iToF module, A offset 、B offset Representing the signal offset value of the pixel tap A, B, respectively, and V represents the ac component amplitude of the light intensity of the light emitted from the iToF module. Because the two taps A, B are in the same pixel, the direct current component of the light intensity of the ambient light received by the tap A plus the light intensity emitted by the iToF module is the same as the direct current component of the light intensity of the ambient light received by the tap B plus the light intensity emitted by the iToF module, and the tap A and the tap B are the same (Environment+DC) And (3) representing.
The measured value of the tap A, B in the sampling of different phases is obtained by the above formula, and I, Q is obtained by the following formula:
after I, Q is obtained, the measured phase value can be calculated directly by a sin algorithm or a square wave algorithm:
but because of the time difference between adjacent phase sampling points, the difference is delta t 1 +Δt 2 As shown in fig. 4. An object moving at high speed may result in motion blur for the same pixel, where the result of the actual four-phase sampling is not obtained by photographing the same object. To improve the motion blur resistance, only the first two phase samples in FIG. 3 can be used to obtain 2 sets of tap A, B values, A 0 、B 0 ，A 90 、B 90 And calculates I, Q by the above formula
In this way, although the motion blur resistance is improved, the influence of environment light and non-ideal factors in the process greatly influences the ranging accuracy. If the A is obtained by pre-calibration offset 、B offset And subtracted from the initial test signal, and pre-calibrated G for each pixel A 、G B As a result, a compensation correction algorithm under gain mismatch is then performedTo store A offset 、B offset Images of differences G A 、G B The correction data of the gain difference increases the cost of correcting the stored data.
In order to achieve the higher capability of resisting motion blur, resisting ambient light interference and resisting process errors under two-phase sampling of the iToF module, the calibration and correction compensation of each pixel layer are performed on the two-phase sampling compared with the extra error generated by four-phase sampling so as to improve the ranging accuracy of the two-phase sampling, and the two-phase sampling calibration and correction method of the iToF module is achieved with lower storage cost and lower algorithm cost.
Referring to fig. 5 to fig. 9, fig. 5 is a schematic diagram of steps of a two-phase sampling calibration and correction method of an iToF module according to an embodiment of the present invention, fig. 6 is a two-phase sampling timing diagram of an embodiment of the present invention, fig. 7 is a flowchart of calculating an LUT according to an embodiment of the present invention, fig. 8 is a schematic diagram of a measured phase difference between two-phase sampling and four-phase sampling in a square wave algorithm according to an embodiment of the present invention, and fig. 9 is a flowchart of obtaining a scaling factor according to an embodiment of the present invention.
As shown in fig. 5, the two-phase sampling calibration and correction method of the iToF module according to the present embodiment includes the following steps: s1, configuring sampling time sequences under two-phase associated sampling of an iToF module with two taps of a single pixel, wherein the sampling time sequences are used for sampling a first phase and a second phase in an active light source on state of the iToF module and sampling for the third time in an active light source off state so as to detect the intensity of ambient light; s2, acquiring a change curve of a measured phase difference value of two-phase sampling and four-phase sampling along with the change of the measured phase of the two-phase sampling, and storing the change curve as a lookup table; s3, acquiring a proportionality coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to a change curve in the lookup table, and storing the proportionality coefficient as a proportionality coefficient table; and S4, under the two-phase sampling mode, obtaining a test result of the two-phase sampling according to the sampling time sequence, obtaining a correction value of the test result by searching the lookup table, and obtaining a proportion coefficient of the current pixel by searching the proportion coefficient table so as to further correct the correction value, thereby correcting each pixel under the two-phase sampling.
Regarding step S1, a sampling timing sequence under two-phase correlated sampling of the iToF module with two taps of a single pixel is configured, where the sampling timing sequence is that a first phase and a second phase are sampled in an active light source on state of the iToF module, and a third sampling is performed in an active light source off state to detect an ambient light intensity.
As shown in fig. 6, unlike the four-phase sampling timing (see fig. 4), the two-phase sampling timing configuration of the present invention discards the fourth-phase sampling, and simultaneously turns off the active light source of the iToF module for the third-phase sampling to detect the ambient light intensity, and the other configuration is consistent with the four-phase sampling.
In some embodiments, the first phase and the second phase samples are denoted as A φ 、B φ ：
Wherein, the footmark phi represents 0 DEG, 90 DEG and G A 、G B Charge-to-voltage conversion gain, amp, representing tap A, B (Environment+DC) Adding the direct current component of the light intensity of the ambient light received by the pixel to the light intensity of the light emitted by the iToF module, a offset 、B offset Representing the signal offset value of tap A, B, respectively, and V represents the ac component amplitude of the light intensity of the light emitted by the iToF module. Because the two taps A, B are in the same pixel, the direct current component of the light intensity of the ambient light received by the tap A plus the light intensity emitted by the iToF module is the same as the direct current component of the light intensity of the ambient light received by the tap B plus the light intensity emitted by the iToF module, and the tap A and the tap B are the same (Environment+DC) And (3) representing.
The result of the third sampling is marked as A Environment (environment) 、B Environment (environment) ：
A Environment (environment) ＝G A ×Amp Environment (environment) +A offset ，B Environment (environment) ＝G B ×Amp Environment (environment) +B offset 。
Wherein, amp Environment (environment) Since the two taps A, B are in the same pixel, the intensity of the ambient light received by tap A is the same as that of the ambient light received by tap B, and Amp is used Environment (environment) And (3) representing.
I, Q is obtained by the following formula:
wherein, (G) A -G B )×Amp DC Is a residual signal due to the difference in charge-voltage conversion gain of different taps of the same pixel. I. This part of the residual signal in Q will affect the measured phase value measured by this pixel, eventually deteriorating the accuracy performance of the two-phase ranging. In order to improve the accuracy performance of two-phase sampling ranging, the invention continuously performs respective calibration and correction on each pixel so as to compensate the ranging error caused by the residual signal.
Regarding step S2, a change curve of the measured phase difference value of the two-phase sample and the four-phase sample along with the measured phase change of the two-phase sample is obtained, and stored as a lookup table.
In some embodiments, step S2 further comprises: 1) When the difference value of the charge-voltage conversion gains of the two taps is a preset percentage, acquiring a change curve of the difference value between the two-phase sampling measurement phase and the four-phase sampling measurement phase along with the change of the two-phase sampling measurement phase by using an actual optical waveform or an ideal square waveform or an ideal sine waveform; 2) And according to the piecewise linear characteristic of the curve, obtaining and storing the inflection point of each piecewise of the change curve to form the lookup table. The preset percentage may be 3%, 3.7%, 7% or other differential percentage.
As shown in FIG. 7, at G A 、G B For example, the difference percentage of the table is 3%, and the lookup table acquisition mode of the present invention is further illustrated. The method comprises the following specific steps: s71, taking into account the charge-voltage conversion gain G of the pixel tap A, B by an actual optical waveform (actually measured) or an ideal square waveform or an ideal sine waveform, S72 A 、G B In the case of a difference percentage of 3%, S73, theoretically calculates the measured phase of the two-phase sample
And the measured phase of the four-phase sample +.>
S74, obtaining a difference value (& lt/EN) between the two>
) S75, obtaining a difference value (>
) And S76, taking the change curve as a Look-Up Table (LUT) and storing the Look-Up Table. According to the piecewise linear characteristic of the curve, only the inflection point of each piecewise of the change curve is actually needed to be stored, and the intermediate value can be obtained by using an interpolation method.
Regarding step S3, a scaling factor of the measured phase difference value of the two-phase sample and the four-phase sample of each pixel with respect to the change curve in the lookup table is obtained and stored as a scaling factor table.
The calibration and calibration principle under two-phase sampling of the invention is continuously described. Although non-ideal factors in the process may cause G A ≠G B And the different pixels are also different, but in practice these non-ideal factors are aligned with the range accuracy, i.e. wobbleThe effect of the Error (Wiggling Error) is regularly reproducible. As shown in fig. 8, a square wave phase algorithm is used at different G' s A 、G B And under the condition of the difference percentage, respectively carrying out theoretical calculation of the two-phase sampling measurement phase and the four-phase sampling measurement phase to obtain a difference value of the measurement phases calculated by the two methods. In fig. 8, the abscissa represents the coordinate value converted under the condition of pi taking 3.14, and the ordinate represents the difference value obtained by subtracting the four-phase measurement phase from the two-phase measurement phase. From FIG. 8 we find that either G A >G B Whether or not G A <G B The difference value has the same characteristic with the change of the measured phase of the two-phase sampling, i.e. there are two phases with constant difference value, respectively located at [ pi/2, pi ] of the measured phase]，[(3π)/2，2π]Is within the interval of (2). The difference between the two constant stages is linear connection, and the difference is zero at the positions where the measurement phase is equal to pi/4 and (5 pi)/4, G A 、G B The percentage difference only affects the magnitude of this difference curve. In practical applications, in order to make the difference curve more stable, a suitable phase calculation formula, such as sin algorithm or square wave algorithm, may be selected according to the actual light source waveform of the iToF module when the I, Q value is used to calculate the phase. When the calculation formula is matched with the light source waveform of the actual iToF module, the swing error curve of the four-phase sampling is maintained at a small level.
Since the variation curve of the difference value with the measured phase is different G A 、G B The difference percentages have similarity, so the invention obtains the proportionality coefficient of the measured phase difference value curve of the two-phase sampling and the four-phase sampling of each pixel relative to the change curve in the lookup table by storing a group of lookup tables (LUTs) representing the characteristics and calibrating each pixel. Through the proportionality coefficient and the lookup table, each pixel under the two-phase sampling can be independently corrected, and finally, the measured phase of the two-phase sampling is corrected to be equal to the measured phase value under the condition of four phases. The accuracy of the four-phase sampling measurement phase is not affected by the environment light and non-ideal factors in the process, so that the corrected two-phase samplingThe influence of most of environment light and non-ideal factors in the process is eliminated, so that the accuracy of the measurement phase result of the two-phase sampling reaches the accuracy performance under the four-phase sampling.
In some embodiments, step S3 further comprises: 1) Placing the iToF module with a first distance opposite to the calibration white board, and adjusting time delay between a modulated optical signal and a demodulated optical signal of the iToF module to enable a two-phase sampling measurement phase of an image sensor center point of the iToF module to be a first preset value, and acquiring and storing a first four-phase average value of an entire surface measurement phase of four-phase sampling and a first two-phase average value of an entire surface measurement phase of two-phase sampling of a preset number of sheets under the condition; 2) Adjusting the time delay between the modulated optical signal and the demodulated optical signal of the iToF module or adjusting the distance between the iToF module and the calibration white board, so that the two-phase sampling measurement phase of the center point of the image sensor of the iToF module is a second preset value, and acquiring and storing a second four-phase average value of the whole-surface measurement phase of four-phase sampling and a second two-phase average value of the whole-surface measurement phase of two-phase sampling of the preset number of sheets under the condition; 3) The method comprises the steps of obtaining a first difference value of a first two-phase average value and a first four-phase average value of each pixel, searching a lookup table according to a first preset value and a second preset value to obtain a first lookup value and a second lookup value, obtaining a proportionality coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to a change curve in the lookup table according to the first difference value, the second difference value, the first lookup value and the second lookup value, and storing the proportionality coefficient as a proportionality coefficient table. For example, a ratio of a difference value between the first difference value and the second difference value relative to a difference value between the first search value and the second search value is obtained as a scaling factor of the pixel.
In some embodiments, the first distance is a distance that enables a difference in two phase sampling measurement phases of the whole area pixel to be less than or equal to pi/2. For example, the first distance may be 50cm. Placing the iToF module over a distance of 50cm against the flat calibration white board so as to ensure that the difference of two-phase sampling measurement phases of the whole pixel is not more than pi/2, otherwise, reducing the distance between the calibration white board and the iToF module.
In some embodiments, the first preset value is a first constant phase capable of making a difference value between a two-phase sampling measurement phase and a four-phase sampling measurement phase of the whole pixel constant. Specifically, the first constant phase is a phase in which the measurement phase falls within [ pi/2, pi ] interval, and the first preset value may be about (3 pi)/4. The second preset value is a second constant phase capable of making the difference value between the two-phase sampling measurement phase and the four-phase sampling measurement phase of the whole pixel constant. Specifically, the second constant phase is a phase in which the measured phase falls within the [ (3pi)/2, 2pi ] interval, and the second preset value may be about (7pi)/4. The preset number of sheets can be 100 or can be changed according to the calibration time limit.
In some embodiments, the scaling factor is further expressed using the following formula:
wherein ratio is the scaling factor,
for the first two-phase average, +.>
For said first four-phase average, < > x->
For the second two-phase average, +.>
For the second four-phase average value,
for the first preset value of the first value,LUT(/>
) For searching the first search value obtained by the lookup table according to the first preset value, < >>
For said second preset value, LUT (-/-)>
) And searching a second search value acquired by the lookup table according to the second preset value.
As shown in fig. 9, taking the first preset value being equal to (3 pi)/4 and the second preset value being (7 pi)/4 as an example, the mode of obtaining the scaling factor according to the present invention is further illustrated. The method comprises the following specific steps: s901, placing the iToF module at a first distance over against the calibration whiteboard; s902, judging whether the difference of two-phase sampling measurement phases of the whole pixel is less than or equal to pi/2, if so, executing the next step, otherwise, returning to the step S901 to adjust the distance between the iToF module and the calibration whiteboard; s903, adjusting the time delay between the modulation optical signal and the demodulation optical signal of the iToF module; s904, judging whether the two-phase sampling measurement phase of the center point of the image sensor of the iToF module is approximately equal to (3pi)/4, if so, executing the next step, and if not, returning to the step S903 to adjust the time delay between the modulated optical signal and the demodulation optical signal; s905, obtaining and storing the first four-phase average value of the whole measured phase of 100 four-phase samples under the condition
And switching to the two-phase sample acquisition and storing the first two-phase average value of the whole measured phase of 100 two-phase samples under this condition +.>
S906, obtaining the first two-phase level of each pixelDifference value of mean value from first four-phase mean value +.>
And storing; returning to step S903 to adjust the time delay between the modulated optical signal and the demodulated optical signal (the distance between the iToF module and the calibration whiteboard can also be adjusted), and executing step S907 to determine whether the two-phase sampling measurement phase of the center point of the image sensor of the iToF module is about equal to (7pi)/4, if yes, executing the next step, otherwise returning to step S903 to adjust the time delay between the modulated optical signal and the demodulated optical signal; s908, obtaining and storing a second four-phase average value of the whole measured phase of 100 four-phase samples under the condition +.>
And switching to a second two-phase average value +.A second two-phase average value of the whole measured phase of 100 two-phase samples under the condition of two-phase sample acquisition and storage>
S909, obtaining the difference value between the second two-phase average value and the second four-phase average value of each pixel +.>
And storing; s910, obtaining a difference value 1 between a difference value 1 and a difference value 2 of each pixel (difference value 1=difference value 1-difference value 2); s911, searching a search value 1 with the measurement phase equal to (3pi)/4 and a search value 2 with the measurement phase equal to (7pi)/4 in the lookup table; s912, obtaining a difference 2 between the search value 1 and the search value 2 (difference 2=search value 1-search value 2); s913, obtaining the ratio of the difference value 1 and the difference value 2 of each pixel to the difference value 1 and the difference value 2 of the search value as the proportionality coefficient (ratio=difference value 1/difference value 2) of the pixel, namely the proportionality coefficient of the measured phase difference value of the two-phase sampling and the four-phase sampling of the pixel relative to the change curve in the search table; s914, storing the proportionality coefficient of each pixel to form a proportionality coefficient table.
That is, the invention only needs to store the lookup table and the proportion coefficient table, the storage cost is lower, and the algorithm cost is also lower.
Regarding step S4, in the two-phase sampling mode, a test result of the two-phase sampling is obtained according to the sampling timing sequence, a correction value of the test result is obtained by searching the lookup table, and a scaling factor of the current pixel is obtained by searching the scaling factor table to further correct the correction value, so as to correct each pixel in the two-phase sampling.
In some embodiments, the correction for each pixel under two phase samples further uses the following formula:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to correct the result->
For the test result, < >>
For the correction value, ratio is the proportionality coefficient, wiggling Four phase sampling Wobble error calibration data of four-phase samples pre-stored for iToF module, FPPN Four phase sampling FPPN calibration data of four-phase samples pre-stored for the iToF module.
Swing error calibration and FPPN calibration of four-phase sampling are performed in advance before the iToF module leaves the factory, and corresponding calibration data are obtained; the iToF module of the present invention may have two modes of use: a four-phase sampling mode and a two-phase sampling mode; the four-phase sampling mode can be adopted in an application scene with insensitive time delay and motion blur influence, and the specific working principle can refer to the existing four-phase sampling mode; under the application scene with relatively sensitive inter-delay and motion blur influence, for example, the application scene of shooting a high-speed moving object, the method can be switched to a two-phase sampling mode, and the two-phase sampling calibration and correction method disclosed by the invention is adopted to reduce the sampling interval and reduce the influence of motion blur.
According to the above, the invention can improve the ranging accuracy of the two-phase sampling by calibrating and correcting and compensating each pixel layer compared with the extra error generated by four-phase sampling, and realize the higher capability of resisting motion blur, ambient light interference and process error under the two-phase sampling of the iToF module with lower storage cost and lower algorithm cost.
Based on the same inventive concept, the invention also provides a two-phase sampling calibration and correction device of the iToF module. The provided two-phase sampling calibration and correction device of the iToF module can complete the two-phase sampling calibration and correction of the iToF module by adopting the two-phase sampling calibration and correction method of the iToF module as shown in fig. 5-9.
Please refer to fig. 10, which is a block diagram of a two-phase sampling calibration and correction device of the iToF module according to the present invention. As shown in fig. 10, the two-phase sampling calibration and correction device of the iToF module includes: a sampling timing configuration module 101, a lookup table acquisition module 102, a scale factor table acquisition module 103, and a two-phase correction module 104.
Specifically, the sampling timing configuration module 101 is configured to configure sampling timing under two-phase correlated sampling of the iToF module with two taps for a single pixel, where the sampling timing is that a first phase and a second phase are sampled in an active light source on state of the iToF module, and a third sampling is performed in an active light source off state to detect an ambient light intensity; the lookup table acquisition module 102 is configured to acquire a change curve of a measured phase difference value of two-phase samples and four-phase samples along with a measured phase change of the two-phase samples, and store the change curve as a lookup table; the proportional coefficient table obtaining module 103 is configured to obtain a proportional coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel with respect to a change curve in the lookup table, and store the proportional coefficient as a proportional coefficient table; the two-phase correction module 104 is configured to obtain a test result of two-phase sampling according to the sampling timing sequence in the two-phase sampling mode, obtain a correction value of the test result by searching the lookup table, and obtain a scaling factor of a current pixel by searching the scaling factor table to further correct the correction value, so as to correct each pixel in the two-phase sampling.
The operation modes of the modules can refer to the description of the corresponding steps in the two-phase sampling calibration and correction method of the iToF module shown in fig. 5 to 9, and are not repeated here.
Based on the same inventive concept, the invention also provides an electronic device, comprising a memory, a processor and a computer executable program stored on the memory and capable of running on the processor; the steps of the two-phase sampling calibration and correction method of the iToF module shown in fig. 5 to 9 are realized when the processor executes the computer executable program.
Within the scope of the inventive concept, embodiments may be described and illustrated in terms of modules that perform one or more of the functions described. These modules may be physically implemented by analog and/or digital circuits, for example logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic elements, active electronic elements, optical components, hardwired circuits, etc., and may optionally be driven by firmware and/or software. The circuitry may be implemented, for example, in one or more semiconductor chips. The circuitry comprising a module may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware that performs some of the functions of the module and a processor that performs other functions of the module. Each module of the embodiments may be physically separated into two or more interacting and discrete modules without departing from the scope of the inventive concept. Likewise, the modules of the embodiments may be physically combined into more complex modules without departing from the scope of the inventive concept.
Generally, the terms may be understood, at least in part, from the usage in the context. For example, the term "one or more" as used herein, depending at least in part on the context, may be used to describe a feature, structure, or characteristic in a singular sense, or may be used to describe a feature, structure, or combination of features in a plural sense. In addition, the term "based on" may be understood as not necessarily intended to express a set of exclusive factors, but may instead, depending at least in part on the context, allow for other factors that are not necessarily explicitly described.
It should be noted that the terms "comprising" and "having" and their variants are referred to in the document of the present invention and are intended to cover non-exclusive inclusion. The terms "first," "second," and the like are used to distinguish similar objects and not necessarily to describe a particular order or sequence unless otherwise indicated by context, it should be understood that the data so used may be interchanged where appropriate. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without collision. In addition, in the above description, descriptions of well-known components and techniques are omitted so as to not unnecessarily obscure the present invention. In the foregoing embodiments, each embodiment is mainly described for differences from other embodiments, and the same/similar parts between the embodiments are referred to each other.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (10)
1. The two-phase sampling calibration and correction method of the iToF module is characterized by comprising the following steps of:
configuring sampling time sequences under two-phase correlated sampling of an iToF module with two taps of a single pixel, wherein the sampling time sequences are used for sampling a first phase and a second phase in an active light source on state of the iToF module and sampling for the third time in an active light source off state so as to detect the intensity of ambient light;
acquiring a change curve of a measured phase difference value of two-phase sampling and four-phase sampling along with the change of the measured phase of the two-phase sampling, and storing the change curve as a lookup table;
acquiring a proportional coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to a change curve in the lookup table, and storing the proportional coefficient as a proportional coefficient table; and
and under the two-phase sampling mode, obtaining a test result of the two-phase sampling according to the sampling time sequence, obtaining a correction value of the test result by searching the lookup table, and obtaining a proportion coefficient of a current pixel by searching the proportion coefficient table so as to further correct the correction value, thereby correcting each pixel under the two-phase sampling.
2. The method of claim 1, wherein the steps of sampling the first phase and the second phase with the active light source of the iToF module on and sampling the third phase with the active light source off to detect the intensity of ambient light further comprise:
the sampling result of the first phase and the second phase is marked as A φ 、B φ ：
Wherein, the footmark phi represents 0 DEG, 90 DEG and G A 、G B Charge-to-voltage conversion gain, amp, representing tap A, B (Environment+DC) Adding the direct current component of the light intensity of the ambient light received by the pixel to the light intensity of the light emitted by the iToF module, a offset 、B offset Representing the signal offset value of the tap A, B, respectively, and V represents the amplitude of the ac component of the light intensity of the light emitted by the iToF module;
the result of the third sampling is marked as A Environment (environment) 、B Environment (environment) ：
A Environment (environment) ＝G A ×Amp Environment (environment) +A offset ，B Environment (environment) ＝G B ×Amp Environment (environment) +B offset ，
Wherein, amp Environment (environment) The intensity of ambient light received for the pixel;
i, Q is obtained by the following formula:
wherein, (G) A -G B )×Amp DC Is a residual signal due to the difference in charge-voltage conversion gain of different taps of the same pixel.
3. The method of claim 1, wherein the step of obtaining a change curve of the measured phase difference value of the two-phase samples and the four-phase samples as a function of the measured phase of the two-phase samples, and storing the change curve as a look-up table further comprises:
when the difference value of the charge-voltage conversion gains of the two taps is a preset percentage, acquiring a change curve of the difference value between the two-phase sampling measurement phase and the four-phase sampling measurement phase along with the change of the two-phase sampling measurement phase by using an actual optical waveform or an ideal square waveform or an ideal sine waveform;
and according to the piecewise linear characteristic of the curve, obtaining and storing the inflection point of each piecewise of the change curve to form the lookup table.
4. The method of claim 1, wherein the step of obtaining the scaling factor of the measured phase difference value curve of the two-phase samples and the four-phase samples of each pixel with respect to the variation curve in the lookup table and storing the scaling factor as a scaling factor table further comprises:
placing the iToF module with a first distance opposite to the calibration white board, and adjusting time delay between a modulated optical signal and a demodulated optical signal of the iToF module to enable a two-phase sampling measurement phase of an image sensor center point of the iToF module to be a first preset value, and acquiring and storing a first four-phase average value of an entire surface measurement phase of four-phase sampling and a first two-phase average value of an entire surface measurement phase of two-phase sampling of a preset number of sheets under the condition;
adjusting the time delay between the modulated optical signal and the demodulated optical signal of the iToF module or adjusting the distance between the iToF module and the calibration white board, so that the two-phase sampling measurement phase of the center point of the image sensor of the iToF module is a second preset value, and acquiring and storing a second four-phase average value of the whole-surface measurement phase of four-phase sampling and a second two-phase average value of the whole-surface measurement phase of two-phase sampling of the preset number of sheets under the condition;
the method comprises the steps of obtaining a first difference value of a first two-phase average value and a first four-phase average value of each pixel, searching a lookup table according to a first preset value and a second preset value to obtain a first lookup value and a second lookup value, obtaining a proportionality coefficient of a measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to a change curve in the lookup table according to the first difference value, the second difference value, the first lookup value and the second lookup value, and storing the proportionality coefficient as a proportionality coefficient table.
5. The method of claim 4, wherein the first distance is a distance such that a difference in measured phases of two phase samples of an entire pixel is less than or equal to pi/2.
6. The method of claim 4, wherein the first preset value is a first constant phase capable of making a difference value between a two-phase sampling measurement phase and a four-phase sampling measurement phase of the whole pixel constant, and the second preset value is a second constant phase capable of making a difference value between a two-phase sampling measurement phase and a four-phase sampling measurement phase of the whole pixel constant.
7. The method of claim 6, wherein the first constant phase is a phase in which the measurement phase falls within [ pi/2, pi ] interval, and the second constant phase is a phase in which the measurement phase falls within [ (3 pi)/2, 2 pi ] interval.
8. The method of claim 4, wherein the scaling factor is further represented by the following formula:
wherein ratio is the scaling factor,
for the first two-phase average, +.>
For said first four-phase average, < > x->
For the second two-phase average, +.>
For said second four-phase average, +.>
For said first preset value, +.>
For searching the first search value obtained by the lookup table according to the first preset value, < >>
For said second preset value, +.>
And searching a second search value acquired by the lookup table according to the second preset value.
9. The method of claim 1, wherein the correcting each pixel under two phase samples further uses the formula:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to correct the result->
For the test result, < >>
For the correction value, ratio is the proportionality coefficient, wiggling Four phase sampling Wobble error calibration data of four-phase samples pre-stored for iToF module, FPPN Four phase sampling FPPN calibration data of four-phase samples pre-stored for the iToF module.
10. Two-phase sampling calibration and correction device of iToF module, its characterized in that includes: the system comprises a sampling time sequence configuration module, a sampling time sequence detection module and a sampling time sequence detection module, wherein the sampling time sequence configuration module is used for configuring sampling time sequences under two-phase associated sampling of an iToF module with two taps of a single pixel, the sampling time sequence is used for sampling a first phase and a second phase in an active light source on state of the iToF module and sampling for the third time in an active light source off state so as to detect the intensity of ambient light;
the lookup table acquisition module is used for acquiring a change curve of a measured phase difference value of two-phase sampling and four-phase sampling along with the measured phase change of the two-phase sampling, and storing the change curve as a lookup table;
the proportional coefficient table acquisition module is used for acquiring the proportional coefficient of the measured phase difference value of the two-phase sampling and the four-phase sampling of each pixel relative to the change curve in the lookup table and storing the proportional coefficient as a proportional coefficient table; and
and the two-phase correction module is used for obtaining a test result of two-phase sampling according to the sampling time sequence in a two-phase sampling mode, obtaining a correction value of the test result by searching the lookup table, and obtaining a proportion coefficient of a current pixel by searching the proportion coefficient table so as to further correct the correction value, so that each pixel in the two-phase sampling is corrected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310272161.5A CN116299363A (en) | 2023-03-20 | 2023-03-20 | Two-phase sampling calibration and correction method and device of iToF module |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310272161.5A CN116299363A (en) | 2023-03-20 | 2023-03-20 | Two-phase sampling calibration and correction method and device of iToF module |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116299363A true CN116299363A (en) | 2023-06-23 |
Family
ID=86801006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310272161.5A Pending CN116299363A (en) | 2023-03-20 | 2023-03-20 | Two-phase sampling calibration and correction method and device of iToF module |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116299363A (en) |
-
2023
- 2023-03-20 CN CN202310272161.5A patent/CN116299363A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8587771B2 (en) | Method and system for multi-phase dynamic calibration of three-dimensional (3D) sensors in a time-of-flight system | |
| US10061029B2 (en) | Correction of depth images from T-O-F 3D camera with electronic-rolling-shutter for light modulation changes taking place during light integration | |
| US10416296B2 (en) | 3DI sensor depth calibration concept using difference frequency approach | |
| CN111045029B (en) | Fused depth measuring device and measuring method | |
| CN112946671A (en) | Optical distance calculating device and distance calculating method | |
| WO2017138032A1 (en) | Time-of-flight distance measuring device and method for detecting multipath error | |
| US20210263137A1 (en) | Phase noise and methods of correction in multi-frequency mode lidar | |
| US20190227169A1 (en) | Time-of-flight image sensor with distance determination | |
| CN112034485A (en) | Reflectivity sensing with time-of-flight camera | |
| CN114200466A (en) | Distortion determination apparatus and method of determining distortion | |
| CN116299363A (en) | Two-phase sampling calibration and correction method and device of iToF module | |
| US11412201B2 (en) | Calibrated photo-detecting apparatus and calibration method thereof | |
| CN113030999A (en) | Time-of-flight sensing system and image sensor used therein | |
| TW202139688A (en) | Pixel circuit, sensing device, and method thereof | |
| CN109756662B (en) | Depth information correction method and depth information correction device based on DLL (delay locked loop) | |
| CN112419427A (en) | Method for improving time-of-flight camera accuracy | |
| Li et al. | Measurement linearity and accuracy optimization for time-of-flight range imaging cameras | |
| CN113676260A (en) | Detection device and method | |
| US20220075063A1 (en) | Method and Apparatus for Time-of-Flight Sensing | |
| US20220326360A1 (en) | Multipath and noise reduction for time-of-flight cameras | |
| US11675079B2 (en) | Image sensing device and operating method thereof | |
| US20210286080A1 (en) | Time-of-Flight Sensing Using Continuous Wave and Coded Modulation Measurements | |
| US20220390575A1 (en) | Method and apparatus for determining an intensity value representing an intensity of light reflected from an object in a scene | |
| CN114930189A (en) | Elimination of voltage-dependent phase errors in time-of-flight imaging devices | |
| CN115267798A (en) | Flight time detection method and detection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231115 Address after: Room 308 and Room 309, No. 268 Xiangke Road, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai, 201210 Applicant after: Shanghai Jieming Technology Co.,Ltd. Address before: 351100 705-706, building a, Jiulong community, No. 999, lihuadong Avenue, Xialin street, Chengxiang District, Putian City, Fujian Province Applicant before: Fujian Jiemu Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right |