WO2020248334A1 - Caméra de profondeur de temps de vol, et procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit - Google Patents

Caméra de profondeur de temps de vol, et procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit Download PDF

Info

Publication number
WO2020248334A1
WO2020248334A1 PCT/CN2019/097098 CN2019097098W WO2020248334A1 WO 2020248334 A1 WO2020248334 A1 WO 2020248334A1 CN 2019097098 W CN2019097098 W CN 2019097098W WO 2020248334 A1 WO2020248334 A1 WO 2020248334A1
Authority
WO
WIPO (PCT)
Prior art keywords
taps
time
flight
charge signal
pulse
Prior art date
Application number
PCT/CN2019/097098
Other languages
English (en)
Chinese (zh)
Inventor
许星
Original Assignee
深圳奥比中光科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳奥比中光科技有限公司 filed Critical 深圳奥比中光科技有限公司
Publication of WO2020248334A1 publication Critical patent/WO2020248334A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/484—Transmitters
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers

Definitions

  • the present invention relates to the technical field of optical measurement, in particular to a time-of-flight depth camera and a noise-reducing distance measurement method for single-frequency modulation.
  • ToF ranging method is a technology that achieves precise ranging by measuring the round-trip flight time of light pulses between the transmitting/receiving device and the target object.
  • dToF direct-TOF
  • the emitted light signal is periodically modulated, and the phase delay of the reflected light signal relative to the emitted light signal is measured.
  • the measurement technique that calculates the time of flight phase delay is called iToF (Indirect-TOF) technique.
  • CW continuous wave
  • PM Pulse Modulated
  • CW-iToF technology is mainly used in measurement systems based on two-tap sensors.
  • the core measurement algorithm is a four-phase modulation and demodulation method, which requires at least two exposures (in order to ensure measurement accuracy, four exposures are usually required. ) Can complete the collection of four phase data and output one frame of depth image, so it is difficult to obtain a higher frame rate.
  • PM-iToF modulation technology is mainly used in four-tap pixel sensors (three taps are used for signal acquisition and output, and one tap is used to release invalid electrons).
  • the measurement distance of this measurement method is currently limited by the modulation and demodulation signal
  • the pulse width of the modulation and demodulation signal needs to be extended, and the extension of the pulse width of the modulation and demodulation signal will lead to an increase in power consumption and a decrease in measurement accuracy.
  • the present invention provides a time-of-flight depth camera and a noise-reducing distance measurement method for single-frequency modulation and demodulation.
  • a time-of-flight depth camera including: a transmitting module, including a light source, for emitting pulsed beams to an object to be measured; and an acquisition module, including an image sensor composed of at least one pixel, each of which includes at least 3 taps , The tap is used to collect the charge signal and/or the charge signal of the background light generated by the reflected pulse beam reflected by the test object; the processing circuit is used to control the at least three taps in at least the macro period Collect the charge signal in rounds between 3 frame periods, and receive the charge signal data to calculate the flight time of the pulse beam and/or the distance of the object to be measured.
  • the processing circuit calculates the flight time of the pulse beam according to the following formula:
  • Q 11 , Q 21 , Q 31 , Q 12 , Q 22 , Q 32 , Q 13 , Q 23 , and Q 33 respectively represent the signals collected by 3 taps in 3 consecutive frame periods of the macrocycle .
  • the processing circuit controls the acquisition timing of the at least 3 taps to continuously change or controls the time delay for the light source to emit the pulse beam to realize that the at least 3 taps collect charge signals in rotation.
  • the time delay between consecutive frame periods is regularly increasing, regularly decreasing or irregularly changing; the time delay difference between consecutive frame periods is an integer multiple of the pulse width.
  • the processing circuit is also used for judging the data of the charge signal to determine whether the data of the charge signal includes the charge signal of the reflected pulse beam, and then calculating the flight time of the pulse beam according to the judgment result And/or the distance of the object to be measured.
  • the present invention also provides a noise-reducing distance measurement method for single-frequency modulation and demodulation, including: S1: using a light source to emit a pulsed beam to the object to be measured; S2: using an image sensor composed of at least one pixel to collect The charge signal of the reflected pulse beam reflected by the measured object, each pixel includes at least 3 taps, and the taps are used to collect the charge signal and/or the charge signal of the background light; S3: control the at least 3 The tap alternately collects the charge signal between at least 3 frame periods of the macro period, and receives the data of the charge signal to calculate the flight time of the pulse beam and/or the distance of the object to be measured.
  • the flight time of the pulsed beam is calculated according to the following formula:
  • Q 11 , Q 21 , Q 31 , Q 12 , Q 22 , Q 32 , Q 13 , Q 23 , and Q 33 respectively represent the signals collected by the three taps in three consecutive frame periods.
  • the processing circuit controls the acquisition timing of the at least 3 taps to continuously change or controls the time delay for the light source to emit the pulse beam to realize that the at least 3 taps collect charge signals in rotation.
  • the time delay between consecutive frame periods is regularly increasing, regularly decreasing or irregularly changing; the time delay difference between consecutive frame periods is an integer multiple of the pulse width.
  • the method of the present invention also includes judging the data of the charge signal to determine whether the charge signal of the reflected pulse beam is included in the data of the charge signal, and then calculating the flight time of the pulse beam and the pulse beam according to the judgment result. / Or the distance of the object to be measured.
  • the beneficial effect of the present invention is to provide a time-of-flight depth camera and a noise-reducing distance measurement method for single-frequency modulation and demodulation, which get rid of the current PM-iToF measurement scheme where the pulse width is proportional to the measurement distance and power consumption, and is The contradiction of the negative correlation of measurement accuracy; the expansion of the measurement distance is no longer limited by the pulse width, so that the lower measurement power consumption and higher measurement accuracy can be maintained with a longer measurement distance, and the tap rotation Acquisition method to reduce or eliminate fixed-pattern noise (FPN) caused by mismatch between taps or readout circuits due to process manufacturing errors and other reasons.
  • FPN fixed-pattern noise
  • Fig. 1 is a schematic diagram of the principle of a time-of-flight camera according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a method for transmitting and collecting optical signals of a time-of-flight camera according to an embodiment of the present invention.
  • Fig. 3 is a schematic diagram of a noise-reduced time-of-flight camera optical signal emission and collection method according to an embodiment of the present invention.
  • Fig. 4 is a schematic diagram of yet another method of transmitting and collecting optical signals of a time-of-flight camera with reduced noise according to an embodiment of the present invention.
  • Fig. 5 is a schematic diagram of a noise-reducing distance measurement method for single-frequency modulation and demodulation according to an embodiment of the present invention.
  • Fig. 6 is a schematic diagram of optical signal emission and collection of yet another time-of-flight camera according to an embodiment of the present invention.
  • Fig. 7 is a method for forward and backward frame acquisition according to an embodiment of the present invention.
  • Fig. 8(a) is another method for forward and backward frame acquisition according to an embodiment of the present invention.
  • Fig. 8(b) is yet another method for forward and backward frame acquisition according to an embodiment of the present invention.
  • Fig. 9 is a schematic diagram of a noise-reducing distance measurement method for multi-frequency modulation and demodulation according to an embodiment of the present invention.
  • connection can be used for fixing or circuit connection.
  • first and second are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” may explicitly or implicitly include one or more of these features.
  • a plurality of means two or more than two, unless otherwise specifically defined.
  • Fig. 1 is a schematic diagram of a time-of-flight camera according to an embodiment of the present invention.
  • the time-of-flight camera 10 includes a transmitting module 11, a collecting module 12, and a processing circuit 13.
  • the transmitting module 11 provides a transmitted light beam 30 to the target space to illuminate an object 20 in the space, and at least part of the transmitted light beam 30 is reflected by the object 20 Afterwards, a reflected light beam 40 is formed. At least part of the reflected light beam 40 is collected by the collection module 12.
  • the processing circuit 13 is respectively connected with the transmission module 11 and the collection module 12 to synchronize the trigger signals of the transmission module 11 and the collection module 12 to calculate
  • the time required for the light beam to be emitted by the transmitter module 11 and received by the collection module 12, that is, the flight time t of the transmitted light beam 30 and the reflected light beam 40, further, the total light flight distance D of the corresponding point on the object can be calculated by the following formula :
  • c is the speed of light.
  • the transmitting module 11 includes a light source 111, a beam modulator 112, a light source driver (not shown in the figure), and the like.
  • the light source 111 can be a light source such as a light emitting diode (LED), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), or a light source array composed of multiple light sources.
  • the light beam emitted by the light source can be visible light or infrared light. , UV light, etc.
  • the light source 111 emits light beams outward under the control of the light source driver (which can be further controlled by the processing circuit 13). For example, in one embodiment, the light source 111 emits a pulsed light beam at a certain frequency under control, which can be used in the direct time flight method.
  • the frequency is set according to the measurement distance, for example, it can be set to 1MHz ⁇ 100MHz, and the measurement distance is from several meters to several hundred meters; in one embodiment, the light source 111 is controlled to modulate the beam amplitude. It can be used in indirect time-of-flight (Indirect TOF) measurement by emitting pulsed beam, square wave beam, sine wave beam and other beams. It is understandable that a part of the processing circuit 13 or a sub-circuit independent of the processing circuit 13 can be used to control the light source 111 to emit related light beams, such as a pulse signal generator.
  • the beam modulator 112 receives the light beam from the light source 111 and emits a spatially modulated beam, such as a flood beam with uniform intensity distribution or a patterned beam with uneven intensity distribution. It is understandable that the uniform distribution here is a relative concept, not absolute uniformity. Generally, a slightly lower beam intensity at the edge of the field of view is allowed, and the intensity of the imaging area in the middle can also be at a certain threshold. Internal changes, for example, can allow for intensity changes not exceeding 15% or 10%. In some embodiments, the beam modulator 112 is also used to expand the received beam to expand the angle of view.
  • the acquisition module 12 includes an image sensor 121, a lens unit 122, and may also include a filter (not shown in the figure).
  • the lens unit 122 receives and reflects at least part of the spatially modulated light beams reflected by the object and images at least part of the image.
  • the filter needs to select a narrow-band filter that matches the wavelength of the light source to suppress the background light noise in the other wavelength bands.
  • the image sensor 121 may be an image sensor composed of charge coupled device (CCD), complementary metal oxide semiconductor (CMOS), avalanche diode (AD), single photon avalanche diode (SPAD), etc.
  • the size of the array represents the resolution of the depth camera , Such as 320x240, etc.
  • connected to the image sensor 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ).
  • a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure).
  • the image sensor 121 includes at least one pixel, and each pixel includes multiple taps (tap, used to store and read or discharge the charge signal generated by incident photons under the control of the corresponding electrode), for example, including 3 taps , To read the charge signal data.
  • tap used to store and read or discharge the charge signal generated by incident photons under the control of the corresponding electrode
  • the time-of-flight depth camera 10 may also include a drive circuit, a power supply, a color camera, an infrared camera, an IMU and other devices, which are not shown in the figure.
  • the combination with these devices can achieve richer functions, such as 3D texture modeling, infrared face recognition, SLAM and other functions.
  • the time-of-flight depth camera 10 can be embedded in electronic products such as mobile phones, tablet computers, and computers.
  • the processing circuit 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc. composed of CPU, memory, bus, etc., or a general processing circuit, such as when the depth camera is integrated into a mobile phone, In smart terminals such as televisions and computers, the processing circuit in the terminal can be used as at least a part of the processing circuit 13.
  • the processing circuit 13 is used to provide a modulation signal (transmission signal) required when the light source 111 emits laser light, and the light source emits a pulsed beam to the object under the control of the modulation signal; in addition, the processing circuit 13 also provides an image sensor 121 The demodulated signal (collection signal) of the tap in each pixel.
  • the tap collects the charge signal generated by the beam containing the reflected pulse beam reflected by the object under the control of the demodulated signal.
  • the processing circuit 13 can also provide auxiliary monitoring signals, such as temperature sensing, overcurrent, overvoltage protection, fall protection, etc.; the processing circuit 13 can also It is used to save the raw data collected by each tap in the image sensor 121 and perform corresponding processing to obtain the specific location information of the object to be measured.
  • the modulation and demodulation method, control, processing and other functions performed by the processing circuit 13 will be described in detail in the embodiments of FIG. 2 to FIG. 8. For ease of explanation, the PM-iTOF modulation and demodulation method is used as an example for description.
  • FIG. 2 is a schematic diagram of a method for transmitting and collecting optical signals of a time-of-flight camera according to an embodiment of the present invention.
  • Figure 2 exemplarily shows the timing diagram of the laser emission signal (modulation signal), received signal, and acquisition signal (demodulation signal) within two frame periods T.
  • the meaning of each signal is: Sp represents the pulse emission of the light source Signal, each pulse emission signal represents a pulse beam; Sr represents the reflected light signal of the pulsed light reflected back by the object, and each reflected light signal represents the corresponding pulse beam reflected back by the object to be measured, which is in the time line ( Figure The horizontal axis) has a certain delay relative to the pulse emission signal.
  • the delay time t is the flight time of the pulse beam to be calculated;
  • S1 represents the pulse collection signal of the first tap of the pixel, and
  • S2 represents the pulse collection signal of the second tap of the pixel ,
  • S3 represents the pulse collection signal of the third tap of the pixel, each pulse collection signal represents the charge signal (electron) generated by the pixel in the time period corresponding to the signal that the tap has collected;
  • Tp N ⁇ Th, where N is the electron of the participating pixel
  • the entire frame period T is divided into two time periods Ta and Tb, where Ta represents the time period during which each tap of the pixel performs charge collection and storage, and Tb represents the time period during which the charge signal is read out.
  • the collected signal pulse of the n-th tap has a phase delay time of (n-1) ⁇ Th relative to the laser emission signal pulse.
  • each The tap collects the electrons generated on the pixel during its pulse period.
  • the collection signal of the first tap is triggered synchronously with the laser emission signal.
  • the first tap, the second tap, and the third tap respectively perform charge collection and storage in sequence.
  • the pulse period Tp or the number of laser pulse signal transmissions can be K times, and K is not less than 1, but can be as high as tens of thousands, or even higher. The number is determined according to actual requirements. In addition, the number of pulses in different frame periods can also vary.
  • the total amount of charge collected and read out by each tap in the Tb period is the sum of the corresponding charges of the optical signal collected by each tap multiple times in the entire frame period T.
  • the total charge amount of each tap in a single frame period can be expressed as follows:
  • the total charge in the single frame period of the first tap, second tap, and third tap is Q1, Q2, and Q3.
  • the measurement range is limited to a single pulse width time Th, that is, it is assumed that the reflected light signal is collected by the first tap and the second tap (the first tap and the second tap will also collect ambient light at the same time).
  • the third tap is used to collect the ambient light signal, so based on the total charge collected by each tap, the processing unit can calculate the total light flight distance of the pulsed light signal from emission to reflection to the pixel according to the following formula :
  • the spatial coordinates of the target can be calculated according to the optical and structural parameters of the camera.
  • the advantage of the traditional modulation and demodulation method is that the calculation is simple, but the disadvantage is that the measurement range is limited, the measurement flight time is limited within Th, and the corresponding maximum flight distance measurement range is limited within c ⁇ Th.
  • Fig. 2 is a schematic diagram of optical signal transmission and collection according to an embodiment of the present invention.
  • the reflected optical signal can not only fall on the first tap and the second tap, but can also fall on the second tap and the third tap. , It is even allowed to fall on the third tap and the first tap within the next pulse period Tp (for at least two pulse periods Tp or more). "Falling on the tap” mentioned here means that it can be tapped. Since the total charges read in the time period Tb are Q1, Q2, and Q3, unlike the traditional modulation and demodulation method, the present invention does not limit the tap or even the period of receiving the reflected light signal.
  • the processing circuit will judge the three total charges Q1, Q2, and Q3 acquired to determine that the capture contains reflection
  • the light signal excites the electronic taps and/or acquires the taps that only contain background signals.
  • the processing circuit can calculate the flight time of the optical signal according to the following formula:
  • n refers to the sequence number of the tap corresponding to the QA
  • the phase delay time of the tap with the sequence number n relative to the transmitted optical pulse signal is (n-1) ⁇ Th
  • Th is the pulse width of the pulse collection signal of each tap
  • Tp Is the pulse period, Tp N ⁇ Th, where N is the number of taps participating in pixel electronic collection.
  • a front background that is, the semaphore of one tap before the A tap is taken as the background semaphore.
  • Average background that is, the average value of all the taps except the A and B taps is taken as the background signal.
  • Subtract one average background that is, take the average value of the signal quantities of all the taps except one tap after the A and B taps as the background signal quantity.
  • the foregoing embodiment introduces the modulation and demodulation method based on 3-tap pixels. It can be understood that this modulation and demodulation method is also applicable to pixels with more taps, that is, N>3. For example, for a 4-tap pixel, the maximum 4Th measurement distance, for 5-tap pixels, a maximum measurement distance of 5Th can be achieved. Compared with the traditional PM-iTOF measurement scheme, this measurement method extends the farthest measurement flight time from the pulse width time Th to the entire pulse period Tp, which is called a single-frequency full-period measurement scheme here.
  • the charge amount collected by each tap and the calculation formula of the flight time are all for the ideal situation.
  • the mismatch of the pixels due to manufacturing errors or due to the ADC (analog to digital conversion) of each tap Mismatch between the converters) will cause FPN (Fixed-pattern Noise), which is manifested as the deviation between the gains of each tap or the offset of the ADC and other circuits. Measurement error.
  • Fig. 3 is a schematic diagram of a noise-reduced time-of-flight camera optical signal emission and collection method according to an embodiment of the present invention.
  • Figure 3 schematically shows a schematic diagram of the modulation and demodulation signal in three consecutive frame periods T1, T2, T3. These three consecutive frame periods are used as a macro-period unit of the scheme, that is, the modulation and demodulation signal will be Continuously cycle with the macro cycle of T1, T2, T3, T1, T2, T3, T1...
  • the processing circuit controls the acquisition timing (acquisition phase) of each tap to continuously change so that the three taps can alternately collect charge signals.
  • the three taps collect 0 ⁇ 1/3Tp (0 ⁇ 120°), 1/3Tp in the order of S1-S2-S3 in each pulse period Tp.
  • ⁇ 2/3Tp (120° ⁇ 240°), 2/3Tp ⁇ Tp (240° ⁇ 360°) time period of the charge signal in the T2 period, three taps in each pulse period Tp, with S3-S1 -S2 sequence to collect the charges in the time periods of 0 ⁇ 1/3Tp(0 ⁇ 120°), 1/3Tp ⁇ 2/3Tp(120° ⁇ 240°), 2/3Tp ⁇ Tp(240° ⁇ 360°) Signal; in the T3 period, the three taps collect 0 ⁇ 1/3Tp(0 ⁇ 120°), 1/3Tp ⁇ 2/3Tp(120° ⁇ ) in the order of S2-S3-S1 in each pulse cycle Tp. 240°), 2/3Tp ⁇ Tp (240° ⁇ 360°) charge signal.
  • the changing manner of the tap acquisition timing in each frame period is not limited to the sequential rotation method in the above example, and any change manner as long as the acquisition timing of each tap can achieve alternate acquisition.
  • a single macroperiod unit will contain at least N frame periods, so that it can be guaranteed that each tap can achieve complete rotation acquisition.
  • a single macroperiod unit contains 3 frame periods. It is understandable that a single macroperiod unit may also contain more frame periods. For example, in an embodiment It contains 3n frame periods, that is, an integer multiple of the tap data. Of course, it can also contain any number of other frame periods according to actual needs.
  • the N frame periods in the macroperiod unit are not necessarily continuous in timing. For example, in one embodiment, two macroperiods or multiple frame periods included in multiple macroperiods may cross each other.
  • the charge signals collected by the three taps along the time sequence are Q O , Q 120 , and Q 240 in an ideal case, in fact, due to the existence of FPN, the signals collected by each tap in three consecutive frame periods are respectively Q O , Q 120 , and Q 240 .
  • Q GQ+O, where G and O respectively represent the gain and offset of the corresponding tap.
  • T1 period in Figure 3 there are:
  • this solution uses the charge signals collected in three consecutive frames to calculate the flight time value (or depth value) of a single frame.
  • the flight time value or depth value
  • Fig. 4 is a schematic diagram of a method for transmitting and collecting optical signals of a time-of-flight camera with reduced noise according to another embodiment of the present invention.
  • the embodiment shown in Figure 3 adopts the method of changing the acquisition timing of the taps in each frame period in the macroperiod unit to achieve alternate acquisition.
  • the method of controlling the pulse emission time will be adopted in this embodiment.
  • the processing circuit controls the pulse beam to be emitted with a certain timing delay to realize the charge signal of each tap.
  • the reflected pulse signal enters the second pulse period Tp, resulting in only a single tap collecting the charge signal in the first pulse period, but because there are actually thousands to tens of thousands of pulse periods , So the error can be ignored.
  • the time delay of the pulsed beam may not be in the form of regular increase in the embodiment shown in FIG. 4, for example, it may be in a regular decreasing form or an irregular form, and the minimum The time delay may not be 0, and the difference between each time delay may not be a single pulse width, but may be an integer multiple of the pulse width, for example, 2 pulse widths.
  • FIG. 3 and 4 introduce a modulation and demodulation method based on 3-tap pixels to reduce noise. It is understandable that this modulation and demodulation method is also applicable to pixels with more taps, that is, N>3, for example
  • a 4-tap pixel a single macrocycle unit contains 4 consecutive frame periods. In each period, the processing circuit controls the acquisition timing of each tap to continuously change or controls the pulse beam to be emitted with a certain timing delay so that each tap can be The charge signal is collected alternately, thereby reducing noise.
  • the single-frequency full-period measurement scheme proposed in the embodiment shown in FIG. 2 is also applicable to the noise reduction measurement scheme shown in FIGS. 3 and 4, that is, the charge signal measured by each tap is judged to determine the collected charge Whether the signal data contains the charge signal of the reflected pulse beam to confirm the value of each charge Q in formula (9), and then calculate the flight time based on formula (9).
  • FIG. 5 a schematic diagram of a distance measurement method for noise reduction of single-frequency modulation and demodulation, which specifically includes the following steps:
  • S2 Use an image sensor composed of at least one pixel to collect the charge signal of the reflected pulse beam reflected by the object under test, and each pixel includes at least 3 taps, and the taps are used to collect the charge signal And/or the charge signal of the background light;
  • S3 Control the at least 3 taps to alternately collect charge signals between at least 3 frame periods of the macro cycle, and receive data of the charge signals to calculate the flight time of the pulse beam and/or the to-be-measured The distance between objects.
  • the single-frequency full-period measurement scheme increases the measurement distance to a certain extent, but it still cannot satisfy the measurement of longer distances. For example, based on the modulation and demodulation method of 3-tap pixels, when the flight time corresponding to the object distance exceeds 3Th, the reflected light signal in a certain pulse period Tp will first fall on the tap in the subsequent pulse period. Neither formula (3) nor formula (4) can accurately measure flight time or distance. For example, when the reflected light signal in a certain pulse period Tp first falls on the n-th tap in the subsequent j-th pulse period, the flight time of the light signal corresponding to the real object is shown in the following equation:
  • n is the sequence number of the tap corresponding to QA. Since the total charge of each tap integrates the charge accumulated during the pulse period involved, the specific value of j cannot be distinguished only from the total charge of each tap output, which causes confusion in distance measurement.
  • Fig. 6 is a schematic diagram of light signal emission and collection of a time-of-flight camera according to another embodiment of the present invention, which can be used to solve the above-mentioned confusion problem.
  • this embodiment adopts a multi-frequency modulation and demodulation method, that is, adjacent frames are controlled by a processing circuit to use different modulation and demodulation frequencies.
  • two adjacent frame periods are taken as an example for description.
  • the number of pixel taps N 3, the pulse period TPi is Tp1, Tp2, the pulse width Thi is Th1, Th2, the pulse frequency or modulation and demodulation frequency is f1, f2, and the accumulated charge for each pulse of the three taps is q11. , Q12, q21, q22, q31, q32, according to formula (2), the total charge can be Q11, Q12, Q21, Q22, Q31, Q32.
  • the processing circuit After the processing circuit receives the total charge of each tap, it uses the modulation and demodulation method shown in Figure 2 to measure the distance d (or t) in each frame period, and calculates the distance d (or t) in each frame period through the above judgment method.
  • the reflected light signal on a certain pixel in the i-th frame period first falls on the jith one after the pulse period where the light pulse is emitted.
  • the corresponding flight time can be expressed as follows according to equation (11):
  • the processing circuit can find a set of ji combinations with the smallest ti variance at each modulation and demodulation frequency according to the remainder theorem or by traversing various ji combinations within the maximum measurement distance as the solution value, and complete the ji Solve; and then obtain the final flight time or measured distance by weighted average of the flight time or measured distance solved under each group of frequencies.
  • the maximum measurement flight time is extended to:
  • the maximum measurement flight distance is expanded to:
  • LCM Local Common Multiple
  • T p 15 ns
  • the maximum measurement flying distance is 4.5 m
  • T p 20 ns
  • the maximum measurement flying distance is 6 m.
  • T p1 15ns
  • T p2 20ns
  • the least common multiple of 15ns and 20ns is 60ns
  • the maximum measurement distance corresponding to 60ns is 18m, which corresponds to the farthest measurement target The distance can reach 9m.
  • the forward and backward frames can be extended so as not to reduce the number of collected frames.
  • the forward and backward frame acquisition method according to an embodiment of the present invention, that is, for the case of obtaining a single flight time measurement through the forward and backward frames in the dual-frequency modulation and demodulation method, the first one is calculated from frames 1 and 2
  • the second flight time is calculated from frames 2 and 3, and so on, the frame rate of the flight time is only 1 frame less than the frame period, so that the measurement frame rate will not be reduced.
  • the multi-frequency modulation and demodulation method is also applicable to the time-of-flight measurement scheme with reduced noise shown in Figures 3 and 4.
  • Figures 8(a) and 8(b) show schematic diagrams of a multi-frequency modem time-of-flight measurement method for reducing noise according to an embodiment of the present invention.
  • a single macrocycle contains 3 frame periods.
  • the processing circuit controls the acquisition timing of each tap to continuously change or controls the pulse beam to be emitted with a certain timing delay to make each Taps can alternately collect charge signals, which can reduce noise.
  • the modulation and demodulation method shown in Figure 2 can be used to achieve high frame rate measurement
  • the modulation and demodulation method shown in Figure 3 or Figure 4 can also be used to achieve high-precision measurement.
  • the two correspond to the high frame rate measurement mode.
  • high-precision measurement mode On the basis of the two modes, a farther measurement range can be achieved through multi-frequency modulation, that is, a large-range measurement mode. It is understandable that frequency modulation needs to be realized by a specific modulation drive circuit.
  • the multi-frequency modulation method shown in Figure 7 corresponds to different modulation drive circuits as the multi-frequency modulation method shown in Figure 8(a). This means that when you want the depth camera to meet this modulation scheme, you need to set at least two independent modulation drive circuits for control, which undoubtedly increases the design difficulty and cost. For this reason, as shown in Figure 8(b), the frequency modulation method shown in Figure 7 can also be used to achieve high-precision measurement.
  • the macrocycle can be regarded as the composition of the nth, (n+2), and (n+4) frames.
  • the first 1, 3, and 5 form a macro cycle
  • the second, 4, and 5 form In another adjacent macro period, combined with the charge signal data collected in the two macro periods of different modulation and demodulation frequencies, the flight time of the pulse beam and/or the distance of the object to be measured can be calculated.
  • the forward and backward frames can also be postponed.
  • the first flight time is calculated from the collected signal data of the first to sixth frames
  • the second flight time is calculated from the second to the sixth frame.
  • the 7-frame acquisition signal data is calculated and obtained, and so on, the frame rate of the flight time is only 5 frames less than the frame period, which will not reduce the measurement frame rate.
  • the processing circuit will adaptively adjust the number of modulation and demodulation frequencies and specific frequency combinations through result feedback to meet the requirements in different measurement scenarios as much as possible. .
  • the processing circuit calculates the target distance after calculating the current distance (or flight time) of the object, and when most of the measured target distances are relatively close, a smaller frequency can be used to measure In order to ensure a higher frame rate and reduce the impact of target motion on the measurement result, when there are more distant targets in the measurement target, the number of measurement frequencies can be appropriately increased or the combination of measurement frequencies can be adjusted to ensure measurement accuracy.
  • a schematic diagram of a noise-reducing distance measurement method for multi-frequency modulation and demodulation includes the following steps:
  • T1 Use a light source to emit pulsed beams to the object under test
  • T2 Use an image sensor consisting of at least one pixel to collect the charge signal of the reflected pulse beam reflected by the object under test, each of the pixels includes at least 3 taps, and the taps are used to collect the charge signal And/or the charge signal of the background light;
  • T3 Control the at least 3 taps to alternately collect charge signals between at least 3 frame periods of the macro period, use different modulation and demodulation frequencies in the two adjacent macro periods, and receive the adjacent The charge signal data received in two macro periods is used to calculate the flight time of the pulse beam and/or the distance of the object to be measured.
  • any single-frequency full-period measurement scheme, noise reduction measurement scheme, and multi-frequency long-distance measurement scheme based on sensors with more than three taps Regardless of whether the waveform of the modulation and demodulation signal is continuous or discontinuous within the exposure time range, or the measurement sequence of the modulation and demodulation signal of different frequencies and the fine-tuning of the modulation frequency within the same exposure time, etc., all circumstances should be protected by this patent Within the scope, the example description or analysis algorithm to explain the principle of this patent is only an example description of this patent, and should not be regarded as a limitation to the content of this patent. For those skilled in the art to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious modifications can be made, and the same performance or use should be regarded as belonging to the protection scope of the present invention.
  • the beneficial effect achieved by the present invention is to get rid of the contradiction that the pulse width is directly proportional to the measurement distance and power consumption in the current PM-iToF measurement scheme, but is negatively related to the measurement accuracy; the expansion of the measurement distance is no longer limited to the pulse width, Therefore, it can maintain low measurement power consumption and high measurement accuracy even with a long measurement distance.
  • the tap rotation acquisition method can reduce or eliminate the tapping or readout circuit caused by process manufacturing errors. Fixed-Pattern Noise (FPN) caused by the mismatch between them.
  • FPN Fixed-Pattern Noise
  • the single set of modulation and demodulation frequency in this scheme only needs one exposure to output three-tap signal to obtain one frame of depth information, thus significantly reducing the overall measurement power consumption and increasing Measure the frame rate. Therefore, this solution has obvious advantages over the existing iToF technical solutions.
  • the present invention implements all or part of the processes in the above-mentioned embodiment methods, and can also be completed by instructing relevant hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium. When executed, the steps of the foregoing method embodiments can be realized.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media.
  • the content contained in the computer-readable medium can be appropriately added or deleted in accordance with the requirements of the legislation and patent practice in the jurisdiction.
  • the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

La présente invention concerne une caméra de profondeur de temps de vol (10) et un procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit. La caméra de profondeur de temps de vol comprend : un module d'émission comprenant une source de lumière et utilisé pour émettre un faisceau pulsé vers une cible de mesure ; un module d'acquisition comprenant un capteur d'image constitué d'au moins un pixel, chaque pixel comprenant au moins trois dérivations pour acquérir un signal de charge généré par le faisceau pulsé réfléchi par la cible de mesure, et/ou un signal de charge d'un fond lumineux ; et un circuit de traitement pour commander lesdites au moins trois dérivations pour acquérir alternativement des signaux de charge à partir d'au moins trois cycles de trame dans un cycle macro, et pour recevoir des données des signaux de charge de manière à calculer le temps de vol du faisceau pulsé et/ou la distance de la cible de mesure. L'étendue de la distance de mesure n'est plus limitée par la largeur d'impulsion. De plus, la procédure d'acquisition alternée par les dérivations réduit ou élimine le bruit cyclique fixe résultant du décalage entre les dérivations ou entre des circuits de lecture provoqués par des erreurs de processus de fabrication.
PCT/CN2019/097098 2019-06-14 2019-07-22 Caméra de profondeur de temps de vol, et procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit WO2020248334A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910518104.4 2019-06-14
CN201910518104.4A CN110361751B (zh) 2019-06-14 2019-06-14 时间飞行深度相机及单频调制解调的降低噪声的距离测量方法

Publications (1)

Publication Number Publication Date
WO2020248334A1 true WO2020248334A1 (fr) 2020-12-17

Family

ID=68217233

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097098 WO2020248334A1 (fr) 2019-06-14 2019-07-22 Caméra de profondeur de temps de vol, et procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit

Country Status (2)

Country Link
CN (1) CN110361751B (fr)
WO (1) WO2020248334A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2791240C1 (ru) * 2021-11-26 2023-03-06 Самсунг Электроникс Ко., Лтд. ВЫЧИСЛЕНИЕ РАССТОЯНИЯ НА ОСНОВЕ ПРОСТРАНСТВЕННОЙ ФАЗОВОЙ МОДУЛЯЦИИ В ЧЕТЫРЕХТАКТНОМ i-ToF ДАТЧИКЕ

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673152A (zh) * 2019-10-29 2020-01-10 炬佑智能科技(苏州)有限公司 飞行时间传感器及其测距方法
CN110988840B (zh) * 2019-11-01 2022-03-18 青岛小鸟看看科技有限公司 飞行时间的获取方法、装置及电子设备
WO2021177045A1 (fr) * 2020-03-04 2021-09-10 ソニーグループ株式会社 Dispositif de traitement de signal, procédé de traitement de signal et module de télémétrie
CN111580125B (zh) * 2020-05-28 2022-09-09 Oppo广东移动通信有限公司 飞行时间模组及其控制方法、电子设备
CN111580119B (zh) * 2020-05-29 2022-09-02 Oppo广东移动通信有限公司 深度相机、电子设备及控制方法
CN111880193B (zh) * 2020-07-16 2022-10-14 常州纵慧芯光半导体科技有限公司 一种激光驱动系统及方法、三维传感系统
CN114881894B (zh) * 2022-07-07 2022-09-30 武汉市聚芯微电子有限责任公司 一种像素修复方法、装置、设备以及计算机存储介质

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011057244A1 (fr) * 2009-11-09 2011-05-12 Mesa Imaging Ag Pixel de démodulation à étages multiples et procédé
US20110202310A1 (en) * 2010-02-12 2011-08-18 Samsung Electronics Co., Ltd. Depth sensor, depth estimation method using the same, and depth estimation device including the same
CN102622745A (zh) * 2011-01-12 2012-08-01 索尼公司 3d飞行时间相机和方法
CN103748479A (zh) * 2011-07-15 2014-04-23 软动力学传感器公司 用于提供距离信息的飞行时间摄像机及方法
CN105518485A (zh) * 2014-01-13 2016-04-20 软动力学传感器公司 用于驱动飞行时间系统的方法
CN109343070A (zh) * 2018-11-21 2019-02-15 深圳奥比中光科技有限公司 时间飞行深度相机
CN109991583A (zh) * 2019-03-14 2019-07-09 深圳奥比中光科技有限公司 一种抗干扰的距离测量方法及深度相机
CN110221273A (zh) * 2019-05-09 2019-09-10 深圳奥比中光科技有限公司 时间飞行深度相机及单频调制解调的距离测量方法
CN110320528A (zh) * 2019-06-14 2019-10-11 深圳奥比中光科技有限公司 时间深度相机及多频调制解调的降低噪声的距离测量方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1659418A1 (fr) * 2004-11-23 2006-05-24 IEE INTERNATIONAL ELECTRONICS & ENGINEERING S.A. Procédé de compensation d'erreur d'une caméra 3D

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011057244A1 (fr) * 2009-11-09 2011-05-12 Mesa Imaging Ag Pixel de démodulation à étages multiples et procédé
US20110202310A1 (en) * 2010-02-12 2011-08-18 Samsung Electronics Co., Ltd. Depth sensor, depth estimation method using the same, and depth estimation device including the same
CN102622745A (zh) * 2011-01-12 2012-08-01 索尼公司 3d飞行时间相机和方法
CN103748479A (zh) * 2011-07-15 2014-04-23 软动力学传感器公司 用于提供距离信息的飞行时间摄像机及方法
CN105518485A (zh) * 2014-01-13 2016-04-20 软动力学传感器公司 用于驱动飞行时间系统的方法
CN109343070A (zh) * 2018-11-21 2019-02-15 深圳奥比中光科技有限公司 时间飞行深度相机
CN109991583A (zh) * 2019-03-14 2019-07-09 深圳奥比中光科技有限公司 一种抗干扰的距离测量方法及深度相机
CN110221273A (zh) * 2019-05-09 2019-09-10 深圳奥比中光科技有限公司 时间飞行深度相机及单频调制解调的距离测量方法
CN110320528A (zh) * 2019-06-14 2019-10-11 深圳奥比中光科技有限公司 时间深度相机及多频调制解调的降低噪声的距离测量方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2791240C1 (ru) * 2021-11-26 2023-03-06 Самсунг Электроникс Ко., Лтд. ВЫЧИСЛЕНИЕ РАССТОЯНИЯ НА ОСНОВЕ ПРОСТРАНСТВЕННОЙ ФАЗОВОЙ МОДУЛЯЦИИ В ЧЕТЫРЕХТАКТНОМ i-ToF ДАТЧИКЕ

Also Published As

Publication number Publication date
CN110361751B (zh) 2021-04-30
CN110361751A (zh) 2019-10-22

Similar Documents

Publication Publication Date Title
WO2020248335A1 (fr) Caméra de profondeur à temps de vol et procédé de mesure de distance avec réduction du bruit basée sur la modulation et la démodulation multifréquences
WO2020248334A1 (fr) Caméra de profondeur de temps de vol, et procédé de mesure de distance utilisant une modulation/démodulation à fréquence unique et facilitant la réduction du bruit
CN110221274B (zh) 时间飞行深度相机及多频调制解调的距离测量方法
CN110221273B (zh) 时间飞行深度相机及单频调制解调的距离测量方法
WO2021008209A1 (fr) Appareil de mesure de profondeur et procédé de mesure de distance
CN110221272B (zh) 时间飞行深度相机及抗干扰的距离测量方法
WO2020223981A1 (fr) Caméra de profondeur de temps de vol et procédé de mesure de distance de modulation et de démodulation multifréquence
WO2021051477A1 (fr) Système et procédé de mesure de distance par temps de vol comportant un histogramme réglable
WO2021051478A1 (fr) Système et procédé de mesure de distance basé sur le temps de vol pour circuit tdc à double partage
CN110546530B (zh) 一种像素结构
WO2021051479A1 (fr) Procédé et système de mesure de temps de vol fondée sur une interpolation
WO2021128587A1 (fr) Dispositif de mesure de profondeur réglable et procédé de mesure associé
WO2021051480A1 (fr) Procédé de mesure de distance de temps de vol basé sur un dessin d'histogramme dynamique et système de mesure
WO2021051481A1 (fr) Procédé de mesure de distance par temps de vol par traçage d'un histogramme dynamique et système de mesure associé
WO2021120402A1 (fr) Appareil de mesure de la profondeur de fusion et procédé de mesure
WO2021120403A1 (fr) Dispositif et procédé de mesure de profondeur
TWI780462B (zh) 距離影像攝像裝置及距離影像攝像方法
WO2016075945A1 (fr) Dispositif de télémétrie optique par mesure du temps de vol
CN110187355B (zh) 一种距离测量方法及深度相机
US20200182983A1 (en) Hybrid center of mass method (cmm) pixel
WO2021103428A1 (fr) Système et procédé de mesure de profondeur
CN111885316B (zh) 一种图像传感器像素电路、图像传感器及深度相机
JP2003510561A (ja) Ｃｍｏｓ互換３次元画像センサｉｃ
US20210181316A1 (en) Time-of-flight-based distance measurement system and method
JP2004294420A (ja) 距離画像センサ

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19932421

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19932421

Country of ref document: EP

Kind code of ref document: A1