WO2022016448A1 - Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same - Google Patents

Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same Download PDF

Info

Publication number
WO2022016448A1
WO2022016448A1 PCT/CN2020/103779 CN2020103779W WO2022016448A1 WO 2022016448 A1 WO2022016448 A1 WO 2022016448A1 CN 2020103779 W CN2020103779 W CN 2020103779W WO 2022016448 A1 WO2022016448 A1 WO 2022016448A1
Authority
WO
WIPO (PCT)
Prior art keywords
photodetector
light
taps
photodetectors
tap
Prior art date
Application number
PCT/CN2020/103779
Other languages
French (fr)
Inventor
Kudoh YOSHIHARU
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Priority to PCT/CN2020/103779 priority Critical patent/WO2022016448A1/en
Priority to CN202080105137.7A priority patent/CN116113844A/en
Publication of WO2022016448A1 publication Critical patent/WO2022016448A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/491—Details of non-pulse systems
    • G01S7/4912—Receivers
    • G01S7/4913—Circuits for detection, sampling, integration or read-out
    • G01S7/4914—Circuits for detection, sampling, integration or read-out of detector arrays, e.g. charge-transfer gates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G01S17/36—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/491—Details of non-pulse systems
    • G01S7/493—Extracting wanted echo signals

Definitions

  • the present invention relates to an indirect ToF sensor, a stacked sensor chip, and a method for measuring a distance to an object using the same.
  • a multi-camera parallax system is known as a method of implementing three-dimensional sensing and depth imaging.
  • a multi-camera parallax system is generally large and needs a complicated operation. An accuracy of depth measurement by such a system is not sufficient.
  • depth sensors using infrared light are recently used.
  • a ToF (Time-of-Flight) system is known as a depth sensor using infrared light.
  • Commercially-available products mainly employ an indirect ToF system.
  • the indirect ToF method measures the flight time of light by using a signal ratio between multi-tap detection nodes.
  • Figure 12 shows a method for measuring a distance to an object by using a conventional ToF sensor.
  • the indirect ToF sensor emits pulsed light 10 at a predetermined cycle T from a light source.
  • the light 10 is reflected on the object and comes back to the indirect ToF sensor as reflected light 12, and impinges a photodetector.
  • the photodetector comprises a plurality of taps for storing signal charge.
  • the photodetector stores a signal charge generated by absorbing the light 12 reflected on the object.
  • the photodetector After repeating emitting of light 10 and storing the signal charge based on the reflected light 12 by the taps for a predetermined number of times, the photodetector outputs the stored signal charge from the taps to a detection node as a light-receiving signal.
  • the photodetector comprises taps and The taps and are sequentially activated at a predetermined cycle and store signal charge. The timing of activating the tap is synchronized with the pulse of the light 10. The reflected light 12 impinges the photodetector over the period when the tap is activated and the period when the tap is activated.
  • the ratio of the light-receiving signal stored in and output from the tap and the light-receiving signal stored in and output from the tap is 8: 2. Since depending on the timing when the reflected light 12 impinges the photodetector, the ratio of the light-receiving signals outputted from the taps and varies, the time when the reflected light 12 came back can be determined from the ratio of the light-receiving signals and therefore the distance to the object can be measured.
  • the photodetectors are arranged in an array, the distances to the object at various locations on the array are mapped and a depth image can be obtained.
  • the intensity of the light reflected on the object is proportional to the inverse of the square of the distance to the target object.
  • the light reflected on an object near the photodetector has a large intensity, the light can easily saturate the storing node.
  • light reflected on an object far from the photodetector has a small intensity and the signal charge generated by the reflected light is small. Therefore, the detection of the reflected light is not easy.
  • the frequency and pulse width of emitted light are important for the accuracy of determining the distance.
  • the accuracy of determining the distance increases.
  • an operation at a higher frequency reduces the range of measurable distance. Therefore, a wide range distance measurement is generally implemented by using data obtained in at least two frames in which pulsed light having different pulse frequencies is emitted.
  • a frame obtaining data for a nearby object and a frame obtaining data for a faraway object if the object moves between these two frames, motion blur may occur due to the long read-out time.
  • a plurality of kinds of light sources may be necessary for emissions for these frames.
  • an indirect ToF sensor implementing a distance measurement with a wide range from a short distance to a long distance with a single frame or a single light source; a stacked sensor chip of the indirect ToF sensor and an electronic circuit; and a method for measuring a distance to an object by using them.
  • the first aspect according to the present invention provides a method for measuring a distance to an object, comprising:
  • step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are simultaneously carried out
  • the method further comprising:
  • the at least one light source may comprise only one light source.
  • the number of taps of the second photodetector may be more than the number of taps of the first photodetector.
  • the first and second photodetectors may be arranged in an array
  • the method may further comprise obtaining a depth image representing the distance to the object in a two-dimensional manner based on the distance from the first photodetectors to the object determined from the first photodetectors and based on the distance from the second photodetectors to the object determined from the second photodetectors.
  • the method may further comprise determining the distance from the second photodetector to the object by interpolating the distances from the plurality of first photodetectors to the object.
  • the method may further comprise determining the distance from the first photodetector to the object by interpolating the distances from the plurality of second photodetectors to the object.
  • a cycle of sequentially storing signal charges by all of the taps of the first photodetector may be repeated a plurality of times during a cycle of sequentially storing signal charges by all of the taps of the second photodetector
  • the distance from the first photodetector to the object may be determined based on the light-receiving signals output from the taps of the second photodetector by the reflected light and based on the light-receiving signals output from the taps of the first photodetector by the reflected light.
  • the first photodetectors may include at least a first group of first photodetectors and a second group of first photodetectors, and
  • sequentially storing a signal charge during a predetermined period by the taps of the first photodetectors may comprise:
  • the second aspect according to the present invention provides an indirect ToF sensor, comprising:
  • a photodetector array comprising at least a first photodetector and a second photodetector
  • an area of the second photodetector is larger than an area of the first photodetector.
  • the at least one light source may comprise only one light source.
  • the first photodetector may comprise a plurality of taps
  • the second photodetector comprises more taps than the number of taps of the first photodetector.
  • the third aspect according to the present invention provides a stacked sensor chip comprising:
  • the electronic circuit may control the indirect ToF sensor
  • the electronic circuit may process light-receiving signals from the indirect ToF sensor.
  • the indirect ToF sensor implementing a distance measurement with a wide range from a short distance to a long distance with a single frame or a single light source; the stacked sensor chip of the indirect ToF sensor and an electronic circuit; and the method for measuring a distance to an object by using them may be provided.
  • Figure 1 shows a plan view of an indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 2 shows a schematic view of a method for measuring a distance to an object by the indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 3 shows a schematic view of a method for measuring a distance to a nearby object by the indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 4 shows a schematic view of a method for measuring a distance to a faraway object by the indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 5 shows a schematic view of a method for generating a depth image by the indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 6 shows a plan view of an indirect ToF sensor according to the second embodiment of the present invention.
  • Figure 7 shows a schematic view of a method for measuring a distance to an object by the indirect ToF sensor according to the second embodiment of the present invention.
  • Figure 8 shows a schematic view of a method for generating a depth image by the indirect ToF sensor according to the second embodiment of the present invention.
  • Figure 9 shows a plan view of an indirect ToF sensor according to the third embodiment of the present invention.
  • Figure 10 shows a schematic view of a method for measuring a distance to an object by an indirect ToF sensor according to the third embodiment of the present invention.
  • Figure 11 shows a cross-sectional view of a stacked sensor chip according to the present invention.
  • Figure 12 shows a schematic view of a method for measuring a distance to an object by a conventional indirect ToF sensor.
  • Figure 1 shows a plan view of an indirect ToF (Time-of-Flight) sensor according to a first embodiment of the present invention.
  • the indirect ToF sensor shown in Figure 1 comprises at least a first photodetector 2 and a second photodetector 4.
  • the first photodetector 2 and the second photodetector 4 are arranged in an array to form a photodetector array 1.
  • the area of the second photodetector 4 is larger than the area of the first photodetector 2.
  • the indirect ToF sensor comprises at least one light source not shown.
  • the photodetector array 1 may comprise a third photodetector, a fourth photodetector, ... having different areas from the areas of the first photodetector 2 and the second photodetector 4.
  • the at least one light source may comprise a single light source.
  • the first photodetector 2 comprises a plurality of taps not shown.
  • the first photodetector 2 comprises a tap and a tap
  • the second photodetector 4 comprises a plurality of taps not shown.
  • the second photodetector 4 comprises a tap and a tap which are the same number as that of the first photodetector 2.
  • a tap herein comprises at least a port for outputting a signal charge and a gate for controlling the output of the signal charge to the port.
  • the tap may further comprise a capacitor for storing a signal charge and a gate for controlling the storage of the signal charge in the capacitor and the output of the signal charge from the capacitor.
  • Figure 2 shows a schematic diagram of a method for measuring a distance D to an object 8 by the indirect ToF sensor according to the first embodiment.
  • Light 10 emitted from the light source of the indirect ToF sensor is reflected on the object 8 and returns to the indirect ToF sensor as reflected light 12.
  • the distance D from the indirect ToF sensor to the object 8 can be measured based on a time from the emission of the light 10 to the return of the reflected light 12 to the indirect ToF sensor.
  • Figure 3 shows a method for measuring a distance D to a nearby object 8 by the indirect ToF sensor according to the first embodiment of the present invention.
  • Figure 4 shows a method for measuring a distance D to an object 8 farther than the case of Figure 3 by the indirect ToF sensor according to the first embodiment of the present invention.
  • the light source of the indirect ToF sensor emits pulsed light 10 having a predetermined cycle time and a predetermined width.
  • the light 10 may have any wavelength. Infrared light is preferable because infrared light is invisible to humans.
  • the light reflected on the object 8 impinges the indirect ToF sensor as reflected light 12 with a delay relative to the emission of the light 10. The delay is proportional to the distance to the object 8.
  • the first photodetector 2 comprises, for example, a tap and a tap
  • the tap is activated by synchronizing the emission of the light 10 from the light source, and the tap stores a signal charge corresponding to light impinging the first photodetector 2 during the period of activating the tap in, for example, a storage capacitor not shown.
  • the tap is activated after the activation of the tap is finished.
  • the tap stores a signal charge corresponding to light impinging the first photodetector 2 during the period of activating the tap in, for example, a storage capacitor not shown.
  • the tap and the tap are sequentially activated again after the activation of the tap is finished.
  • the durations of activating the taps are preferably equal to each other.
  • the second photodetector 4 comprises, for example, a tap and a tap
  • the tap is activated by synchronizing with the emission of the light from the light source and stores a signal charge corresponding to light impinging the second photodetector 4 during the period of activating the tap in, for example, a storage capacitor not shown.
  • the tap is activated after the activation of the tap is finished.
  • the tap stores a signal charge corresponding to light impinging the second photodetector 4 during the period of activating the tap in, for example, a storage capacitor not shown.
  • the tap and the tap are sequentially activated again after the activation of the tap is finished.
  • the durations of activating the taps of the second photodetector 4 are preferably equal to each other, and equal to the duration and the cycle time of the taps of the first photodetector 2.
  • the signal charge stored in the each storage capacitor is output as light-receiving signals.
  • the cycle is repeated by outputting the pulsed light 10 at a frequency of 100 MHz in a duration of one millisecond. After the duration is elapsed, the stored signal charge is output as a light-receiving signal.
  • the time from the emission of the light 10 to the return of the reflected light 12 can be obtained from the ratio between the amount of the light-receiving signal output from the tap and the amount of the light-receiving signal output from the tap Since the time from the emission of the light 10 to the return of the reflected light 12 is proportional to the distance D from the indirect ToF sensor to the object 8, the distance D from the indirect ToF sensor to the object 8 can be determined based on the ratio between the amount of the light-receiving signal output from the tap and the amount of the light-receiving signal output from the tap
  • the intensity of the reflected light 12 reduces proportional to the inverse of the square of the distance D to the object 8.
  • the area of the first photodetector 2 is suitable for the intensity of the reflected light 12 from the nearby object 8
  • the signal charge stored by the first photodetector 2 by the reflected light 12 from the faraway object 8 decreases, and the signal to noise ratio may be degraded. Therefore, the measurement of the distance D to the faraway object 8 by the first photodetector 2 may not be preferable.
  • the second photodetector 4 since the second photodetector 4 has an area larger than the area of the first photodetector 2, the second photodetector 4 has a sensitivity larger than the sensitivity of the first photodetector 2, and is suitable for smaller light intensity.
  • the large area of the second photodetector 4 is suitable for the intensity of the reflected light 12 from the faraway object 8.
  • the second photodetector 4 may become saturated when the reflected light 12 having a large intensity impinges the second photodetector 4 from the nearby object 8. Therefore, the measurement of the distance D to the faraway object 8 by the second photodetector 4 may not be preferable.
  • the first photodetector 2 may be preferable for measuring the distance to the nearby object
  • the second photodetector 4 may be preferable for measuring the distance to the faraway object.
  • Figure 5 shows a method for generating a depth image representing a distance to an object 8 in a two-dimensional way regarding such an indirect ToF sensor according to the first embodiment.
  • the intensity of the reflected light 12 reduces proportional to the inverse of the square of the distance D to the object 8. If the distance D to the object 8 is small, the reflected light 12 has a large intensity, and the second photodetector 4 having a large area may become saturated. Therefore, the ratio of the light-receiving signals output from the taps of the second photodetector 4 may be inconsistent with the ratio of the reflected light impinging during the activation periods of the taps of the second photodetector 4. In this case, a depth image may be generated based on only the light-receiving signals from the first photodetector 2 without using the output from the second photodetector 4.
  • a depth image may be generated based on only the light-receiving signals from the second photodetector 4 without using the output from the first photodetector 2.
  • the reflected light 12 impinges the photodetector array 1 from the faraway object 8 as shown in Figure 5 (a) and the light-receiving signals from the first photodetectors 2 are less than or equal to a predetermined threshold, the light-receiving signals from the first photodetectors 2 are ignored as shown in Figure 5 (d) .
  • the data of the distances D at the locations of the first photodetectors 2 are derived by interpolating the light-receiving signals from the second photodetectors 4 as shown in Figure 5 (e) in order to obtain a depth image.
  • both of the depth image shown in Figure 5 (c) based on the light-receiving signals from the first photodetectors 2 and the depth image shown in Figure 5 (e) based on the light-receiving signals from the second photodetectors 4 may be generated.
  • the two depth images may be integrated as shown in Figure 5 (f) to generate the final depth image shown in Figure 5 (g) .
  • the shorter pulse width of the light 10 improves the accuracy of the distance D in a short range and is therefore preferable.
  • the next emission of the light 10 and measurement of the distance start before the reflected light 12 comes back to the indirect ToF sensor.
  • the frequency of the pulsed light should be low. However, in this case, the low frequency of the pulsed light degrades the measurement accuracy of the distance D to the nearby object 8.
  • Figure 6 shows a plan view of an indirect ToF sensor according to the second embodiment of the present invention.
  • Figure 7 shows a schematic view of a method for measuring a distance to an object by the indirect ToF sensor according to the second embodiment of the present invention.
  • the indirect ToF sensor according to the second embodiment of the present invention may adapt the measurement of a faraway object and increase the measurement accuracy of a nearby object.
  • the photodetector array 21 of the indirect ToF sensor according to the second embodiment of the present invention comprises at least a first photodetector 22 and a second photodetector 24 having an area larger than an area of the first photodetector 22 similarly to the photodetector array 1 of the indirect ToF sensor according to the first embodiment of the present invention.
  • the first photodetector 22 comprises a plurality of taps not shown.
  • the first photodetector 22 comprises a tap and a tap
  • the second photodetector 24 comprises taps not shown and the number of taps thereof is more than the number of taps of the first photodetector 22.
  • the second photodetector 24 comprises taps to
  • Figure 7 (a) shows a method for measuring the distance D to a nearby object 8 by the indirect ToF sensor.
  • the light 12 reflected on the object 8 impinges the first photodetector 22 over both of the period of activating the tap of the first photodetector 22 and the period of activating the tap of the first photodetector 22.
  • the time from the emission of the light 10 to the return of the reflected light 12, i.e., the distance D to the object 8 can be determined based on the ratio between the light- receiving signals output from the taps and
  • Figure 7 (b) shows a method for measuring a distance D to a faraway object 8 by the indirect ToF sensor.
  • a cycle in which all of the taps to of the second photodetector 24 sequentially store a signal charge
  • a cycle, in which all of the taps and of the first photodetector 22 sequentially store a signal charge is repeated a plurality of times, twice in the example shown in Figure 7 (b) .
  • the reflected light 12 impinges the first photodetector 22 during the second periods of activating the taps and of the first photodetector 22, and the reflected light 12 impinges the second photodetector 24 during the periods of activating the taps and of the second photodetector 24.
  • the signal charge stored by the taps and of the first photodetector 22 should be distinguished from the signal charge caused by the reflected light 12 from the nearby object 8 shown in Figure 7 (a) .
  • the light-receiving signal output from the first photodetector 22 can be determined to be the signal stored during the second periods of activating the taps and based on the signal charge stored by the taps and of the second photodetector 24. Then a higher resolution of a depth image regarding the faraway object 8 obtained by the second photodetectors 24 can be implemented by using the light-receiving signals output from the taps and of the first photodetectors 22.
  • Figure 8 shows a method for generating a depth image having an improved resolution from a distance to the object 8 measured by the method of Figure 7 by the indirect ToF sensor according to the second embodiment of the present invention.
  • the intensity of the reflected light 12 decreases proportional to the inverse of the square of the distance D to the object 8. Therefore, when the distance D to the object 8 is small, the reflected light 12 may have a large intensity and the second photodetector 24 having a large area may become saturated.
  • the ratio of the light-receiving signals output from the taps of the second photodetector 24 may be inconsistent with the ratio of the reflected light impinging the second photodetector 24 during the periods of activating the taps of the second photodetector 24.
  • the depth image may be generated from only the light-receiving signals from the first photodetector 22 without using the output from the second photodetector 24.
  • the reflected light 12 from the nearby object 8 impinges the photodetector array 21 in Figure 8 (a) .
  • the second photodetectors 24 become saturated, the light-receiving signals from the second photodetectors 24 may be ignored as shown in Figure 8 (b) .
  • the data of the distances D at the locations of the second photodetectors 24 may be determined by interpolating the light-receiving signals from the first photodetectors 22 as shown to generate the depth image in Figure 8 (c) .
  • the reflected light 12 may have a small intensity, and the signal charge stored by the photodetector 22 having a small area becomes smaller. Then, the signal to noise ratio may be degraded and the accurate detection of the distance D may be difficult. In such a case, a depth image may be generated from only the light-receiving signals from the second photodetectors 24 without using the light-receiving signals from the first photodetectors 22.
  • both of the depth image based on the light-receiving signals from the first photodetectors 22 shown in Figure 8 (c) and the depth image based on the light-receiving signals from the second photodetectors 24 shown in Figure 8 (e) may be generated and integrated to generate the final depth image shown in Figure 8 (f) .
  • the indirect ToF sensor may increase the accuracy of the measurement of a nearby object by shortening the pulse width of the light 10 and may be adapted to the measurement of a faraway object by increasing the number of taps of the second photodetectors 24.
  • Figure 9 shows a plan view of an indirect ToF sensor according to the third embodiment of the present invention.
  • Figure 10 shows a method for measuring the distance to an object by the indirect ToF sensor according to the third embodiment of the present invention.
  • the photodetector array 31 of the indirect ToF sensors comprises a first group of first photodetectors 32-1 and a second group of first photodetectors 32-2, and second photodetectors 34 as shown in Figure 9.
  • the first photodetectors 32-1 of the first group and the first photodetectors 32-2 of the second group have the same area and the same number of taps.
  • the second photodetectors 34 have areas larger than the areas of the first and second groups of the first photodetectors 32-1, 32-2, and the number of taps thereof is more than the number of taps of the first and second groups of the first photodetectors 32-1, 32-2.
  • the first photodetectors 32-1, 32-2 comprise two taps respectively.
  • the second photodetector 34 may comprise four taps to
  • the timing of activating the taps and of the second group of the first photodetectors 32-2 is shifted from the timing of activating the taps and of the first group of the first photodetectors 32-1 by a phase of 90 degrees. Therefore, the signal charge stored by the taps and of the first photodetectors 32-1 of the first group constitutes a first frame, and the signal charge stored by the taps and of the first photodetectors 32-2 of the second group constitutes a second frame. Comparing the two frames, the influence of the environment light can be removed.
  • the reflected light 12 impinges the second photodetector 34 during two periods activating two taps among four taps.
  • the reflected light 12 impinges the second photodetector 34 during the periods activating the taps and Therefore, there are always two continuous periods activating two taps during which the reflected light 12 does not impinge the second photodetector 34.
  • the light-receiving signals from the two taps during the activation periods when the reflected light 12 impinges are considered as a first frame, and the light-receiving signals from the two taps during the activation periods when the reflected light 12 does not impinge are considered as a second frame. Comparing these two frames, the influence of the environment light can be removed.
  • FIG 11 shows a cross-sectional view of a stacked sensor chip according to the embodiments of the present invention.
  • the stacked sensor chip 26 comprises a stacked structure in which a chip 28 of a photodetector array 1 and an electronic circuit chip 18 are stacked.
  • the electronic circuit chip 18 may comprise a control circuit for controlling a light source not shown and taps of photodetectors of the photodetector array 1.
  • the electronic circuit chip 18 may also comprise a process circuit for processing light-receiving signals output from the photodetectors.
  • Wirings 14 for controlling the taps of the photodetectors of the photodetector array 1 and transmitting the light-receiving signals output from the taps to the electronic circuit chip 18 may be integrated into several groups in a predetermined way.
  • the integrated wirings 14 may be electrically coupled with the electronic circuit chip 18 at coupling portions 16.
  • each photodetector of the photodetector array 1 comprises a plurality of taps
  • the photodetector array 1 may comprise a huge number of wirings for controlling the taps and transmitting the light-receiving signals, and therefore the coupling with the electronic circuit chip 18 may be complicated.
  • the stacked sensor chip 26 shown in Figure 11 may integrate the wirings coupled with the photodetector array 1 to simplify the communication between the photodetector array 1 and the electronic circuit chip 18.
  • the photodetector array 1 and the electronic chip 18 may be fabricated in separate processes and may be coupled with each other in the final step, the fabrication process may be simplified.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Measurement Of Optical Distance (AREA)

Abstract

A method for measuring a distance to an object, an indirect ToF sensor, and a stacked sensor chip are provided. The method includes: emitting pulsed light from at least one light source; sequentially storing a signal charge generated by received light during a predetermined period by each tap of at least one first photodetector, the first photodetector comprising a plurality of taps; sequentially storing a signal charge generated by received light during a predetermined period by each tap of at least one second photodetector, the second photodetector comprising a plurality of taps and an area larger than an area of the first photodetector, wherein the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are simultaneously carried out; outputting the signal charges stored by the taps of the at least one first photodetector and the taps of the at least one second photodetector as light-receiving signals after the steps of sequentially storing the signal charges are repeated a predetermined number of times; and determining a distance from the first and second photodetectors based on the taps of the at least one first photodetector outputting light-receiving signals by reflected light which is the emitted pulsed light reflected on the object, and based on the taps of the at least one second photodetector outputting light-receiving signals by the reflected light.

Description

Indirect ToF Sensor, Stacked Sensor Chip, and Method for Measuring Distance to Object Using the Same Technical Field
The present invention relates to an indirect ToF sensor, a stacked sensor chip, and a method for measuring a distance to an object using the same.
Background
Recently, the demand for three-dimensional sensing has been increasing. Mobile devices comprising a depth imaging camera are also proposed. Conventionally a multi-camera parallax system is known as a method of implementing three-dimensional sensing and depth imaging. However, a multi-camera parallax system is generally large and needs a complicated operation. An accuracy of depth measurement by such a system is not sufficient. Thus, instead of the multi-camera parallax system, depth sensors using infrared light are recently used.
A ToF (Time-of-Flight) system is known as a depth sensor using infrared light. Commercially-available products mainly employ an indirect ToF system.
The indirect ToF method measures the flight time of light by using a signal ratio between multi-tap detection nodes. Figure 12 shows a method for measuring a distance to an object by using a conventional ToF sensor. The indirect ToF sensor emits pulsed light 10 at a predetermined cycle T from a light source. The light 10 is reflected on the object and comes back to the indirect ToF sensor as reflected light 12, and impinges a photodetector. The photodetector comprises a plurality of taps for storing signal charge. The photodetector stores a signal charge generated by absorbing the light 12 reflected on the object. After repeating emitting of light 10 and storing the signal charge based on the reflected light 12 by the taps for a predetermined number of times, the photodetector outputs the stored signal charge from the taps to a detection node as a light-receiving signal. In the example shown in Figure 12, the photodetector comprises taps
Figure PCTCN2020103779-appb-000001
and
Figure PCTCN2020103779-appb-000002
The taps
Figure PCTCN2020103779-appb-000003
and
Figure PCTCN2020103779-appb-000004
are  sequentially activated at a predetermined cycle and store signal charge. The timing of activating the tap 
Figure PCTCN2020103779-appb-000005
is synchronized with the pulse of the light 10. The reflected light 12 impinges the photodetector over the period when the tap 
Figure PCTCN2020103779-appb-000006
is activated and the period when the tap 
Figure PCTCN2020103779-appb-000007
is activated. For example, in Figure 12, 80%of the reflected light 12 impinges the photodetector in the period during which the tap 
Figure PCTCN2020103779-appb-000008
is activated and 20%of the reflected light 12 impinges the photodetector in the period during which the tap 
Figure PCTCN2020103779-appb-000009
is activated. Therefore, the ratio of the light-receiving signal stored in and output from the tap 
Figure PCTCN2020103779-appb-000010
and the light-receiving signal stored in and output from the tap 
Figure PCTCN2020103779-appb-000011
is 8: 2. Since depending on the timing when the reflected light 12 impinges the photodetector, the ratio of the light-receiving signals outputted from the taps 
Figure PCTCN2020103779-appb-000012
and 
Figure PCTCN2020103779-appb-000013
varies, the time when the reflected light 12 came back can be determined from the ratio of the light-receiving signals and therefore the distance to the object can be measured.
If the photodetectors are arranged in an array, the distances to the object at various locations on the array are mapped and a depth image can be obtained.
In the indirect ToF method, the intensity of the light reflected on the object is proportional to the inverse of the square of the distance to the target object. In other words, since light reflected on an object near the photodetector has a large intensity, the light can easily saturate the storing node. On the other hand, light reflected on an object far from the photodetector has a small intensity and the signal charge generated by the reflected light is small. Therefore, the detection of the reflected light is not easy.
The frequency and pulse width of emitted light are important for the accuracy of determining the distance. In general, when the light is emitted at a higher frequency or with a narrower pulse width, the accuracy of determining the distance increases. However, since the frequency when activating the taps must be shorter than the flight time of the light, an operation at a higher frequency reduces the range of measurable distance. Therefore, a wide range distance measurement is generally implemented by using data obtained in at least two frames in which pulsed light having different pulse frequencies is emitted. However, when using a frame obtaining data for a nearby object and a frame obtaining data for a faraway object, if the object moves between these two frames, motion blur may occur due to the long  read-out time. In addition, a plurality of kinds of light sources may be necessary for emissions for these frames.
Summary of Invention
Problem to be Solved by the Invention
There is a demand for: an indirect ToF sensor implementing a distance measurement with a wide range from a short distance to a long distance with a single frame or a single light source; a stacked sensor chip of the indirect ToF sensor and an electronic circuit; and a method for measuring a distance to an object by using them.
Means for Solving the Problem
The first aspect according to the present invention provides a method for measuring a distance to an object, comprising:
emitting pulsed light from at least one light source;
sequentially storing a signal charge generated by received light during a predetermined period by each tap of at least one first photodetector, the first photodetector comprising a plurality of taps; and
sequentially storing a signal charge generated by receiving light during a predetermined period by each tap of at least one second photodetector, the second photodetector comprising a plurality of taps and an area larger than an area of the first photodetector,
wherein the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are simultaneously carried out,
the method further comprising:
outputting the signal charges stored by the taps of the at least one first photodetector and the taps of the at least one second photodetector as light-receiving signals after the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first  photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are repeated a predetermined number of times; and
determining a distance from the first and second photodetectors based on the taps of the at least one first photodetector outputting light-receiving signals by reflected light which is the emitted pulsed light reflected on the object, and based on the taps of the at least one second photodetector outputting light-receiving signals by the reflected light.
Regarding the first aspect according to the present invention, the at least one light source may comprise only one light source.
Regarding the first aspect according to the present invention, the number of taps of the second photodetector may be more than the number of taps of the first photodetector.
Regarding the first aspect according to the present invention, the first and second photodetectors may be arranged in an array, and
the method may further comprise obtaining a depth image representing the distance to the object in a two-dimensional manner based on the distance from the first photodetectors to the object determined from the first photodetectors and based on the distance from the second photodetectors to the object determined from the second photodetectors.
Regarding the first aspect according to the present invention, when the light-receiving signal output from the tap of the second photodetector by the reflected light is saturated, the method may further comprise determining the distance from the second photodetector to the object by interpolating the distances from the plurality of first photodetectors to the object.
Regarding the first aspect according to the present invention, when the light-receiving signal output from the tap of the first photodetector by the reflected light is lower than or equal to a predetermined value, the method may further comprise determining the distance from the first photodetector to the object by interpolating the distances from the plurality of second photodetectors to the object.
Regarding the first aspect according to the present invention, a cycle of sequentially storing signal charges by all of the taps of the first photodetector may be repeated a plurality of times during a cycle of sequentially storing signal charges  by all of the taps of the second photodetector, and
the distance from the first photodetector to the object may be determined based on the light-receiving signals output from the taps of the second photodetector by the reflected light and based on the light-receiving signals output from the taps of the first photodetector by the reflected light.
Regarding the first aspect according to the present invention, the first photodetectors may include at least a first group of first photodetectors and a second group of first photodetectors, and
sequentially storing a signal charge during a predetermined period by the taps of the first photodetectors may comprise:
sequentially storing a signal charge by each tap of the first photodetectors of the first group; and
sequentially storing a signal charge by each tap of the first photodetectors of the second group at a timing phase-shifted from a timing of sequentially storing a signal charge by each tap of the first photodetectors of the first group.
The second aspect according to the present invention provides an indirect ToF sensor, comprising:
at least one light source; and
a photodetector array comprising at least a first photodetector and a second photodetector,
wherein an area of the second photodetector is larger than an area of the first photodetector.
Regarding the second aspect according to the present invention, the at least one light source may comprise only one light source.
Regarding the second aspect according to the present invention, the first photodetector may comprise a plurality of taps,
wherein the second photodetector comprises more taps than the number of taps of the first photodetector.
The third aspect according to the present invention provides a stacked sensor chip comprising:
an indirect ToF sensor of the second aspect of the present invention; and
a chip comprising an electronic circuit,
wherein the indirect ToF sensor and the chip are stacked and electrically  coupled with each other.
Regarding the third aspect according to the present invention, the electronic circuit may control the indirect ToF sensor, and
the electronic circuit may process light-receiving signals from the indirect ToF sensor.
Effect of the Invention
According to the aspects of the present invention, the indirect ToF sensor implementing a distance measurement with a wide range from a short distance to a long distance with a single frame or a single light source; the stacked sensor chip of the indirect ToF sensor and an electronic circuit; and the method for measuring a distance to an object by using them may be provided.
Brief Explanation of Figures
Figure 1 shows a plan view of an indirect ToF sensor according to the first embodiment of the present invention.
Figure 2 shows a schematic view of a method for measuring a distance to an object by the indirect ToF sensor according to the first embodiment of the present invention.
Figure 3 shows a schematic view of a method for measuring a distance to a nearby object by the indirect ToF sensor according to the first embodiment of the present invention.
Figure 4 shows a schematic view of a method for measuring a distance to a faraway object by the indirect ToF sensor according to the first embodiment of the present invention.
Figure 5 shows a schematic view of a method for generating a depth image by the indirect ToF sensor according to the first embodiment of the present invention.
Figure 6 shows a plan view of an indirect ToF sensor according to the second embodiment of the present invention.
Figure 7 shows a schematic view of a method for measuring a distance  to an object by the indirect ToF sensor according to the second embodiment of the present invention.
Figure 8 shows a schematic view of a method for generating a depth image by the indirect ToF sensor according to the second embodiment of the present invention.
Figure 9 shows a plan view of an indirect ToF sensor according to the third embodiment of the present invention.
Figure 10 shows a schematic view of a method for measuring a distance to an object by an indirect ToF sensor according to the third embodiment of the present invention.
Figure 11 shows a cross-sectional view of a stacked sensor chip according to the present invention.
Figure 12 shows a schematic view of a method for measuring a distance to an object by a conventional indirect ToF sensor.
Embodiments
Figure 1 shows a plan view of an indirect ToF (Time-of-Flight) sensor according to a first embodiment of the present invention. The indirect ToF sensor shown in Figure 1 comprises at least a first photodetector 2 and a second photodetector 4. The first photodetector 2 and the second photodetector 4 are arranged in an array to form a photodetector array 1. The area of the second photodetector 4 is larger than the area of the first photodetector 2. The indirect ToF sensor comprises at least one light source not shown. The photodetector array 1 may comprise a third photodetector, a fourth photodetector, ... having different areas from the areas of the first photodetector 2 and the second photodetector 4. The at least one light source may comprise a single light source.
The first photodetector 2 comprises a plurality of taps not shown. For example, the first photodetector 2 comprises a tap 
Figure PCTCN2020103779-appb-000014
and a tap 
Figure PCTCN2020103779-appb-000015
The second photodetector 4 comprises a plurality of taps not shown. For example, the second photodetector 4 comprises a tap 
Figure PCTCN2020103779-appb-000016
and a tap 
Figure PCTCN2020103779-appb-000017
which are the same number as that of the first photodetector 2. A tap herein comprises at least a port for outputting a signal charge and a gate for controlling the output of the signal charge to the port.  The tap may further comprise a capacitor for storing a signal charge and a gate for controlling the storage of the signal charge in the capacitor and the output of the signal charge from the capacitor.
Figure 2 shows a schematic diagram of a method for measuring a distance D to an object 8 by the indirect ToF sensor according to the first embodiment. Light 10 emitted from the light source of the indirect ToF sensor is reflected on the object 8 and returns to the indirect ToF sensor as reflected light 12. The distance D from the indirect ToF sensor to the object 8 can be measured based on a time from the emission of the light 10 to the return of the reflected light 12 to the indirect ToF sensor.
Figure 3 shows a method for measuring a distance D to a nearby object 8 by the indirect ToF sensor according to the first embodiment of the present invention. Figure 4 shows a method for measuring a distance D to an object 8 farther than the case of Figure 3 by the indirect ToF sensor according to the first embodiment of the present invention.
The light source of the indirect ToF sensor emits pulsed light 10 having a predetermined cycle time and a predetermined width. The light 10 may have any wavelength. Infrared light is preferable because infrared light is invisible to humans. The light reflected on the object 8 impinges the indirect ToF sensor as reflected light 12 with a delay relative to the emission of the light 10. The delay is proportional to the distance to the object 8.
In the present embodiment, the first photodetector 2 comprises, for example, a tap 
Figure PCTCN2020103779-appb-000018
and a tap 
Figure PCTCN2020103779-appb-000019
The tap 
Figure PCTCN2020103779-appb-000020
is activated by synchronizing the emission of the light 10 from the light source, and the tap 
Figure PCTCN2020103779-appb-000021
stores a signal charge corresponding to light impinging the first photodetector 2 during the period of activating the tap 
Figure PCTCN2020103779-appb-000022
in, for example, a storage capacitor not shown. The tap 
Figure PCTCN2020103779-appb-000023
is activated after the activation of the tap 
Figure PCTCN2020103779-appb-000024
is finished. The tap 
Figure PCTCN2020103779-appb-000025
stores a signal charge corresponding to light impinging the first photodetector 2 during the period of activating the tap 
Figure PCTCN2020103779-appb-000026
in, for example, a storage capacitor not shown. The tap 
Figure PCTCN2020103779-appb-000027
and the tap 
Figure PCTCN2020103779-appb-000028
are sequentially activated again after the activation of the tap 
Figure PCTCN2020103779-appb-000029
is finished. The durations of activating the taps are preferably equal to each other.
In the example shown in Figure 3, the second photodetector 4 comprises, for example, a tap 
Figure PCTCN2020103779-appb-000030
and a tap 
Figure PCTCN2020103779-appb-000031
The tap 
Figure PCTCN2020103779-appb-000032
is activated by  synchronizing with the emission of the light from the light source and stores a signal charge corresponding to light impinging the second photodetector 4 during the period of activating the tap 
Figure PCTCN2020103779-appb-000033
in, for example, a storage capacitor not shown. The tap 
Figure PCTCN2020103779-appb-000034
is activated after the activation of the tap 
Figure PCTCN2020103779-appb-000035
is finished. The tap 
Figure PCTCN2020103779-appb-000036
stores a signal charge corresponding to light impinging the second photodetector 4 during the period of activating the tap 
Figure PCTCN2020103779-appb-000037
in, for example, a storage capacitor not shown. The tap
Figure PCTCN2020103779-appb-000038
and the tap 
Figure PCTCN2020103779-appb-000039
are sequentially activated again after the activation of the tap 
Figure PCTCN2020103779-appb-000040
is finished. The durations of activating the taps of the second photodetector 4 are preferably equal to each other, and equal to the duration and the cycle time of the taps of the first photodetector 2.
After the above cycle, i.e., the emission of the pulsed light 10 and the activations of the taps 
Figure PCTCN2020103779-appb-000041
and 
Figure PCTCN2020103779-appb-000042
of the first and  second photodetectors  2, 4 synchronizing with the emission of the light 10 are repeated a predetermined number of times, the signal charge stored in the each storage capacitor is output as light-receiving signals. For example, the cycle is repeated by outputting the pulsed light 10 at a frequency of 100 MHz in a duration of one millisecond. After the duration is elapsed, the stored signal charge is output as a light-receiving signal.
In Figure 3, since the reflected light 12 impinges the indirect ToF sensor with a delay proportional to the distance to the object 8, the reflected light 12 impinges the indirect ToF sensor over both of the period of activating the taps 
Figure PCTCN2020103779-appb-000043
and taps 
Figure PCTCN2020103779-appb-000044
of the first photodetector 2 and the period of activating the taps 
Figure PCTCN2020103779-appb-000045
and 
Figure PCTCN2020103779-appb-000046
of the second photodetector 4. For example, in Figure 3, 80%of the reflected light 12 impinges the first photodetector 2 and the second photodetector 4 during the period of activating the taps 
Figure PCTCN2020103779-appb-000047
and 20%of the reflected light 12 impinges the first photodetector 2 and the second photodetector 4 during the period of activating the taps 
Figure PCTCN2020103779-appb-000048
On the other hand, in Figure 4, since the return of the reflected light 12 is delayed compared to the case shown in Figure 3, the rate of the reflected light 12 impinging the first photodetector 2 and the second photodetector 4 during the period of activating the taps 
Figure PCTCN2020103779-appb-000049
decreases, and the rate of the reflected light 12 impinging the first photodetector 2 and the second photodetector 4 during the period of activating the taps 
Figure PCTCN2020103779-appb-000050
increases. For example, in Figure 4, 20%of the reflected light 12 impinges the first photodetector 2 and the second photodetector 4 during the period of  activating the tap 
Figure PCTCN2020103779-appb-000051
and 80%of the reflected light 12 impinges the first photodetector 2 and the second photodetector 4 during the period of activating the tap 
Figure PCTCN2020103779-appb-000052
Therefore, the time from the emission of the light 10 to the return of the reflected light 12 can be obtained from the ratio between the amount of the light-receiving signal output from the tap 
Figure PCTCN2020103779-appb-000053
and the amount of the light-receiving signal output from the tap 
Figure PCTCN2020103779-appb-000054
Since the time from the emission of the light 10 to the return of the reflected light 12 is proportional to the distance D from the indirect ToF sensor to the object 8, the distance D from the indirect ToF sensor to the object 8 can be determined based on the ratio between the amount of the light-receiving signal output from the tap 
Figure PCTCN2020103779-appb-000055
and the amount of the light-receiving signal output from the tap 
Figure PCTCN2020103779-appb-000056
The intensity of the reflected light 12 reduces proportional to the inverse of the square of the distance D to the object 8. In the case that the area of the first photodetector 2 is suitable for the intensity of the reflected light 12 from the nearby object 8, the signal charge stored by the first photodetector 2 by the reflected light 12 from the faraway object 8 decreases, and the signal to noise ratio may be degraded. Therefore, the measurement of the distance D to the faraway object 8 by the first photodetector 2 may not be preferable. On the other hand, since the second photodetector 4 has an area larger than the area of the first photodetector 2, the second photodetector 4 has a sensitivity larger than the sensitivity of the first photodetector 2, and is suitable for smaller light intensity. Therefore, the large area of the second photodetector 4 is suitable for the intensity of the reflected light 12 from the faraway object 8. However, in this case, the second photodetector 4 may become saturated when the reflected light 12 having a large intensity impinges the second photodetector 4 from the nearby object 8. Therefore, the measurement of the distance D to the faraway object 8 by the second photodetector 4 may not be preferable. Thus, in the indirect ToF sensor according to the first embodiment, the first photodetector 2 may be preferable for measuring the distance to the nearby object, and the second photodetector 4 may be preferable for measuring the distance to the faraway object.
Figure 5 shows a method for generating a depth image representing a distance to an object 8 in a two-dimensional way regarding such an indirect ToF sensor according to the first embodiment.
As discussed above, the intensity of the reflected light 12 reduces  proportional to the inverse of the square of the distance D to the object 8. If the distance D to the object 8 is small, the reflected light 12 has a large intensity, and the second photodetector 4 having a large area may become saturated. Therefore, the ratio of the light-receiving signals output from the taps of the second photodetector 4 may be inconsistent with the ratio of the reflected light impinging during the activation periods of the taps of the second photodetector 4. In this case, a depth image may be generated based on only the light-receiving signals from the first photodetector 2 without using the output from the second photodetector 4. Specifically, when the reflected light 12 from the nearby object 8 impinges the photodetector array 1 and the second photodetectors 4 are saturated as shown in Figure 5 (a) , the light-receiving signals from the second photodetectors 4 are ignored as shown in Figure 5 (b) . Then, the data of the distances D at the locations of the second photodetectors 4 are derived by interpolating the light-receiving signals from the first photodetectors 2 as shown in Figure 5 (c) in order to obtain a depth image.
On the other hand, when the distance D to the object 8 is large, the reflected light 12 has a small intensity, and the light-receiving signal output from the first photodetector 2 having a small area may become small. Therefore, the signal to noise ratio may be degraded and the accurate detection of the distance D may be difficult. In this case, a depth image may be generated based on only the light-receiving signals from the second photodetector 4 without using the output from the first photodetector 2. Specifically, when the reflected light 12 impinges the photodetector array 1 from the faraway object 8 as shown in Figure 5 (a) and the light-receiving signals from the first photodetectors 2 are less than or equal to a predetermined threshold, the light-receiving signals from the first photodetectors 2 are ignored as shown in Figure 5 (d) . Then, the data of the distances D at the locations of the first photodetectors 2 are derived by interpolating the light-receiving signals from the second photodetectors 4 as shown in Figure 5 (e) in order to obtain a depth image.
If the light-receiving signals output from the first photodetectors 2 are larger than the threshold and the second photodetectors 4 are not saturated, both of the depth image shown in Figure 5 (c) based on the light-receiving signals from the first photodetectors 2 and the depth image shown in Figure 5 (e) based on the light-receiving signals from the second photodetectors 4 may be generated. The two depth  images may be integrated as shown in Figure 5 (f) to generate the final depth image shown in Figure 5 (g) .
When an object 8 is located near the indirect ToF sensor, the shorter pulse width of the light 10 improves the accuracy of the distance D in a short range and is therefore preferable. However, when the object 8 is located far from the indirect ToF sensor, the next emission of the light 10 and measurement of the distance start before the reflected light 12 comes back to the indirect ToF sensor. If the measurement of a faraway object 8 is also desired, the frequency of the pulsed light should be low. However, in this case, the low frequency of the pulsed light degrades the measurement accuracy of the distance D to the nearby object 8.
Figure 6 shows a plan view of an indirect ToF sensor according to the second embodiment of the present invention. Figure 7 shows a schematic view of a method for measuring a distance to an object by the indirect ToF sensor according to the second embodiment of the present invention. The indirect ToF sensor according to the second embodiment of the present invention may adapt the measurement of a faraway object and increase the measurement accuracy of a nearby object.
As shown in Figure 6, the photodetector array 21 of the indirect ToF sensor according to the second embodiment of the present invention comprises at least a first photodetector 22 and a second photodetector 24 having an area larger than an area of the first photodetector 22 similarly to the photodetector array 1 of the indirect ToF sensor according to the first embodiment of the present invention.
The first photodetector 22 comprises a plurality of taps not shown. For example, the first photodetector 22 comprises a tap 
Figure PCTCN2020103779-appb-000057
and a tap 
Figure PCTCN2020103779-appb-000058
The second photodetector 24 comprises taps not shown and the number of taps thereof is more than the number of taps of the first photodetector 22. For example, the second photodetector 24 comprises taps 
Figure PCTCN2020103779-appb-000059
to
Figure PCTCN2020103779-appb-000060
Figure 7 (a) shows a method for measuring the distance D to a nearby object 8 by the indirect ToF sensor. In this case, as discussed for the first embodiment with reference to Figure 3, the light 12 reflected on the object 8 impinges the first photodetector 22 over both of the period of activating the tap 
Figure PCTCN2020103779-appb-000061
of the first photodetector 22 and the period of activating the tap 
Figure PCTCN2020103779-appb-000062
of the first photodetector 22. The time from the emission of the light 10 to the return of the reflected light 12, i.e., the distance D to the object 8, can be determined based on the ratio between the light- receiving signals output from the taps 
Figure PCTCN2020103779-appb-000063
and
Figure PCTCN2020103779-appb-000064
Figure 7 (b) shows a method for measuring a distance D to a faraway object 8 by the indirect ToF sensor. In a cycle in which all of the taps 
Figure PCTCN2020103779-appb-000065
to
Figure PCTCN2020103779-appb-000066
of the second photodetector 24 sequentially store a signal charge, a cycle, in which all of the taps 
Figure PCTCN2020103779-appb-000067
and 
Figure PCTCN2020103779-appb-000068
of the first photodetector 22 sequentially store a signal charge, is repeated a plurality of times, twice in the example shown in Figure 7 (b) . The reflected light 12 impinges the first photodetector 22 during the second periods of activating the taps 
Figure PCTCN2020103779-appb-000069
and 
Figure PCTCN2020103779-appb-000070
of the first photodetector 22, and the reflected light 12 impinges the second photodetector 24 during the periods of activating the taps 
Figure PCTCN2020103779-appb-000071
and 
Figure PCTCN2020103779-appb-000072
of the second photodetector 24. The signal charge stored by the taps 
Figure PCTCN2020103779-appb-000073
and
Figure PCTCN2020103779-appb-000074
of the first photodetector 22 should be distinguished from the signal charge caused by the reflected light 12 from the nearby object 8 shown in Figure 7 (a) . In this case, the light-receiving signal output from the first photodetector 22 can be determined to be the signal stored during the second periods of activating the taps 
Figure PCTCN2020103779-appb-000075
and 
Figure PCTCN2020103779-appb-000076
based on the signal charge stored by the taps 
Figure PCTCN2020103779-appb-000077
and 
Figure PCTCN2020103779-appb-000078
of the second photodetector 24. Then a higher resolution of a depth image regarding the faraway object 8 obtained by the second photodetectors 24 can be implemented by using the light-receiving signals output from the taps 
Figure PCTCN2020103779-appb-000079
and 
Figure PCTCN2020103779-appb-000080
of the first photodetectors 22.
Figure 8 shows a method for generating a depth image having an improved resolution from a distance to the object 8 measured by the method of Figure 7 by the indirect ToF sensor according to the second embodiment of the present invention. As discussed above, the intensity of the reflected light 12 decreases proportional to the inverse of the square of the distance D to the object 8. Therefore, when the distance D to the object 8 is small, the reflected light 12 may have a large intensity and the second photodetector 24 having a large area may become saturated. Thus, the ratio of the light-receiving signals output from the taps of the second photodetector 24 may be inconsistent with the ratio of the reflected light impinging the second photodetector 24 during the periods of activating the taps of the second photodetector 24. In this case, the depth image may be generated from only the light-receiving signals from the first photodetector 22 without using the output from the second photodetector 24. Specifically, the reflected light 12 from the nearby object 8 impinges the photodetector array 21 in Figure 8 (a) . If the second photodetectors 24 become saturated, the light-receiving signals from the second  photodetectors 24 may be ignored as shown in Figure 8 (b) . Then, the data of the distances D at the locations of the second photodetectors 24 may be determined by interpolating the light-receiving signals from the first photodetectors 22 as shown to generate the depth image in Figure 8 (c) .
On the other hand, if the distance D to the object 8 is large, the reflected light 12 may have a small intensity, and the signal charge stored by the photodetector 22 having a small area becomes smaller. Then, the signal to noise ratio may be degraded and the accurate detection of the distance D may be difficult. In such a case, a depth image may be generated from only the light-receiving signals from the second photodetectors 24 without using the light-receiving signals from the first photodetectors 22. Specifically, when the reflected light 12 from the faraway object 8 impinges the photodetector array 21 and the light-receiving signals from the first photodetectors 22 are smaller than or equal to a predetermined threshold in Figure 8 (a) , the light-receiving signals from the first photodetectors 22 are ignored as shown in Figure 8 (d) . Then, the data of the distances D at the locations of the first photodetectors 22 are derived by interpolating the light-receiving signals from the second photodetectors 24 to generate the depth image as shown in Figure 8 (e) .
When the light-receiving signals output from the first photodetectors 22 are larger than the threshold and the second photodetectors 24 do not become saturated, both of the depth image based on the light-receiving signals from the first photodetectors 22 shown in Figure 8 (c) and the depth image based on the light-receiving signals from the second photodetectors 24 shown in Figure 8 (e) may be generated and integrated to generate the final depth image shown in Figure 8 (f) .
In this way, the indirect ToF sensor may increase the accuracy of the measurement of a nearby object by shortening the pulse width of the light 10 and may be adapted to the measurement of a faraway object by increasing the number of taps of the second photodetectors 24.
Figure 9 shows a plan view of an indirect ToF sensor according to the third embodiment of the present invention. Figure 10 shows a method for measuring the distance to an object by the indirect ToF sensor according to the third embodiment of the present invention.
If one tries to remove the influence of environment light in a measurement of the distance to an object in the indirect ToF method, two images are  obtained by shifting the phase of activating taps by 90 degrees. However, since such an operation captures two images, two frames are necessary to obtain one depth image and therefore the temporal resolution decreases.
The photodetector array 31 of the indirect ToF sensors according to the third embodiment comprises a first group of first photodetectors 32-1 and a second group of first photodetectors 32-2, and second photodetectors 34 as shown in Figure 9. The first photodetectors 32-1 of the first group and the first photodetectors 32-2 of the second group have the same area and the same number of taps. The second photodetectors 34 have areas larger than the areas of the first and second groups of the first photodetectors 32-1, 32-2, and the number of taps thereof is more than the number of taps of the first and second groups of the first photodetectors 32-1, 32-2. For example, in the example shown in Figure 9, the first photodetectors 32-1, 32-2 comprise two taps 
Figure PCTCN2020103779-appb-000081
respectively. The second photodetector 34 may comprise four taps 
Figure PCTCN2020103779-appb-000082
to
Figure PCTCN2020103779-appb-000083
As shown in Figure 10, the timing of activating the taps 
Figure PCTCN2020103779-appb-000084
and 
Figure PCTCN2020103779-appb-000085
of the second group of the first photodetectors 32-2 is shifted from the timing of activating the taps 
Figure PCTCN2020103779-appb-000086
and 
Figure PCTCN2020103779-appb-000087
of the first group of the first photodetectors 32-1 by a phase of 90 degrees. Therefore, the signal charge stored by the taps 
Figure PCTCN2020103779-appb-000088
and 
Figure PCTCN2020103779-appb-000089
of the first photodetectors 32-1 of the first group constitutes a first frame, and the signal charge stored by the taps 
Figure PCTCN2020103779-appb-000090
and 
Figure PCTCN2020103779-appb-000091
of the first photodetectors 32-2 of the second group constitutes a second frame. Comparing the two frames, the influence of the environment light can be removed.
The reflected light 12 impinges the second photodetector 34 during two periods activating two taps among four taps. For example, in the example of Figure 10, the reflected light 12 impinges the second photodetector 34 during the periods activating the taps 
Figure PCTCN2020103779-appb-000092
and 
Figure PCTCN2020103779-appb-000093
Therefore, there are always two continuous periods activating two taps during which the reflected light 12 does not impinge the second photodetector 34. The light-receiving signals from the two taps during the activation periods when the reflected light 12 impinges are considered as a first frame, and the light-receiving signals from the two taps during the activation periods when the reflected light 12 does not impinge are considered as a second frame. Comparing these two frames, the influence of the environment light can be removed.
Figure 11 shows a cross-sectional view of a stacked sensor chip  according to the embodiments of the present invention. The stacked sensor chip 26 comprises a stacked structure in which a chip 28 of a photodetector array 1 and an electronic circuit chip 18 are stacked. The electronic circuit chip 18 may comprise a control circuit for controlling a light source not shown and taps of photodetectors of the photodetector array 1. The electronic circuit chip 18 may also comprise a process circuit for processing light-receiving signals output from the photodetectors. Wirings 14 for controlling the taps of the photodetectors of the photodetector array 1 and transmitting the light-receiving signals output from the taps to the electronic circuit chip 18 may be integrated into several groups in a predetermined way. The integrated wirings 14 may be electrically coupled with the electronic circuit chip 18 at coupling portions 16.
Since each photodetector of the photodetector array 1 according to the embodiments of the present invention comprises a plurality of taps, the photodetector array 1 may comprise a huge number of wirings for controlling the taps and transmitting the light-receiving signals, and therefore the coupling with the electronic circuit chip 18 may be complicated. However, the stacked sensor chip 26 shown in Figure 11 may integrate the wirings coupled with the photodetector array 1 to simplify the communication between the photodetector array 1 and the electronic circuit chip 18. Furthermore, since the photodetector array 1 and the electronic chip 18 may be fabricated in separate processes and may be coupled with each other in the final step, the fabrication process may be simplified. In addition, since a wafer, on which a lot of the photodetector arrays 1 are provided, and a wafer, on which a lot of the electronic circuit chips 18 are provided, may be bonded and may be cut into chips, the fabrication process may be more simplified.
Although the embodiments of the present invention have been illustrated as examples, those skilled in the art will readily understand that various modifications and variations may be embodied without deviating from the spirit and the scope of the present invention.
Designations
1, 21, and 31: Photodetector array
2 and 22: First photodetector
4 and 24: Second photodetector
8: Object
10: Light
12: Reflected light
14: Wiring
16: Coupling portion
18: Electronic circuit chip
26: Stacked sensor chip
28: Chip of photodetector array
32-1: First group of first photodetectors
32-2: Second group of first photodetectors
34: Second photodetector

Claims (13)

  1. A method for measuring a distance to an object, comprising:
    emitting pulsed light from at least one light source;
    sequentially storing a signal charge generated by received light during a predetermined period by each tap of at least one first photodetector, the first photodetector comprising a plurality of taps; and
    sequentially storing a signal charge generated by received light during a predetermined period by each tap of at least one second photodetector, the second photodetector comprising a plurality of taps and an area larger than an area of the first photodetector,
    wherein the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are simultaneously carried out,
    the method further comprising:
    outputting the signal charge stored by the taps of the at least one first photodetector and the taps of the at least one second photodetector as a light-receiving signal after the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one first photodetector and the step of sequentially storing a signal charge generated by received light during the predetermined period by each tap of the at least one second photodetector are repeated a predetermined number of times; and
    determining a distance from the first and second photodetectors based on the taps of the at least one first photodetector outputting light-receiving signals by reflected light which is the emitted pulsed light reflected on the object, and based on the taps of the at least one second photodetector outputting light-receiving signals by the reflected light.
  2. The method of Claim 1, wherein the at least one light source comprises only one light source.
  3. The method of Claim 1, wherein the number of taps of the second photodetector is more than the number of taps of the first photodetector.
  4. The method of Claim 1, wherein the first and second photodetectors are arranged in an array, and
    wherein the method further comprises obtaining a depth image representing the distance to the object in a two-dimensional manner based on the distance from the first photodetectors to the object determined from the first photodetectors and based on the distance from the second photodetectors to the object determined from the second photodetectors.
  5. The method of Claim 1, wherein when the light-receiving signal output from the tap of the second photodetector by the reflected light is saturated, the method further comprises determining the distance from the second photodetector to the object by interpolating the distances from the plurality of first photodetectors to the object.
  6. The method of Claim 1, wherein when the light-receiving signal output from the tap of the first photodetector by the reflected light is lower than or equal to a predetermined value, the method further comprises determining the distance from the first photodetector to the object by interpolating the distances from the plurality of second photodetectors to the object.
  7. The method of Claim 3, wherein a cycle of sequentially storing signal charges by all of the taps of the first photodetector is repeated a plurality of times during a cycle of sequentially storing signal charges by all of the taps of the second photodetector, and
    wherein the distance from the first photodetector to the object is determined based on the light-receiving signals output from the taps of the second photodetector by the reflected light and based on the light-receiving signals output from the taps of the first photodetector by the reflected light.
  8. The method of Claim 1, wherein the first photodetectors include at least a first  group of first photodetectors and a second group of first photodetectors, and
    wherein sequentially storing a signal charge during a predetermined period by the taps of the first photodetectors comprises:
    sequentially storing a signal charge by each tap of the first photodetectors of the first group; and
    sequentially storing a signal charge by each tap of the first photodetectors of the second group at a timing phase-shifted from a timing of the sequentially storing a signal charge by each tap of the first photodetectors of the first group.
  9. An indirect ToF sensor, comprising:
    at least one light source; and
    a photodetector array comprising at least a first photodetector and a second photodetector,
    wherein an area of the second photodetector is larger than an area of the first photodetector.
  10. The indirect ToF sensor of Claim 9, wherein the at least one light source comprises only one light source.
  11. The indirect ToF sensor of Claim 9, wherein the first photodetector comprises a plurality of taps, and
    wherein the second photodetector comprises more taps than the number of taps of the first photodetector.
  12. A stacked sensor chip comprising:
    an indirect ToF sensor of any one of Claims 9 to 11; and
    a chip comprising an electronic circuit,
    wherein the indirect ToF sensor and the chip are stacked and electrically coupled with each other.
  13. The stacked sensor chip of Claim 12, wherein the electronic circuit controls the indirect ToF sensor, and
    wherein the electronic circuit processes the light-receiving signals from the indirect ToF sensor.
PCT/CN2020/103779 2020-07-23 2020-07-23 Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same WO2022016448A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/103779 WO2022016448A1 (en) 2020-07-23 2020-07-23 Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same
CN202080105137.7A CN116113844A (en) 2020-07-23 2020-07-23 Indirect TOF sensor, stacked sensor chip, and method for measuring distance to object using the sensor and chip

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/103779 WO2022016448A1 (en) 2020-07-23 2020-07-23 Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same

Publications (1)

Publication Number Publication Date
WO2022016448A1 true WO2022016448A1 (en) 2022-01-27

Family

ID=79729928

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/103779 WO2022016448A1 (en) 2020-07-23 2020-07-23 Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same

Country Status (2)

Country Link
CN (1) CN116113844A (en)
WO (1) WO2022016448A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050236561A1 (en) * 2004-04-22 2005-10-27 Hin Chee C Photodetector array arrangement for optical encoders
US7283213B2 (en) * 2005-02-08 2007-10-16 Canesta, Inc. Method and system to correct motion blur and reduce signal transients in time-of-flight sensor systems
US20170102461A1 (en) * 2015-10-09 2017-04-13 Fujitsu Limited Distance measuring apparatus, distance measuring method, and table creating method
US20190170855A1 (en) * 2016-08-12 2019-06-06 Fastree3D Sa Method and device for measuring a distance to a target in a multi-user environment by means of at least one detector
US20190250257A1 (en) * 2018-02-13 2019-08-15 Sense Photonics, Inc. Methods and systems for high-resolution long-range flash lidar
CN110850426A (en) * 2019-11-20 2020-02-28 杭州光珀智能科技有限公司 TOF depth camera
WO2020058089A1 (en) * 2018-09-19 2020-03-26 Ams Ag Sensor device, sensor module, imaging system and method to operate a sensor device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050236561A1 (en) * 2004-04-22 2005-10-27 Hin Chee C Photodetector array arrangement for optical encoders
US7283213B2 (en) * 2005-02-08 2007-10-16 Canesta, Inc. Method and system to correct motion blur and reduce signal transients in time-of-flight sensor systems
US20170102461A1 (en) * 2015-10-09 2017-04-13 Fujitsu Limited Distance measuring apparatus, distance measuring method, and table creating method
US20190170855A1 (en) * 2016-08-12 2019-06-06 Fastree3D Sa Method and device for measuring a distance to a target in a multi-user environment by means of at least one detector
US20190250257A1 (en) * 2018-02-13 2019-08-15 Sense Photonics, Inc. Methods and systems for high-resolution long-range flash lidar
WO2020058089A1 (en) * 2018-09-19 2020-03-26 Ams Ag Sensor device, sensor module, imaging system and method to operate a sensor device
CN110850426A (en) * 2019-11-20 2020-02-28 杭州光珀智能科技有限公司 TOF depth camera

Also Published As

Publication number Publication date
CN116113844A (en) 2023-05-12

Similar Documents

Publication Publication Date Title
CN110596722B (en) System and method for measuring flight time distance with adjustable histogram
US11448757B2 (en) Distance measuring device
KR102403544B1 (en) Time-of-flight sensing using an addressable array of emitters
CN110596721B (en) Flight time distance measuring system and method of double-shared TDC circuit
CN110596725B (en) Time-of-flight measurement method and system based on interpolation
US7834985B2 (en) Surface profile measurement
CN101449181B (en) Distance measuring method and distance measuring instrument for detecting the spatial dimension of a target
CN110596724B (en) Method and system for measuring flight time distance during dynamic histogram drawing
US6535275B2 (en) High resolution 3-D imaging range finder
CN110596723B (en) Dynamic histogram drawing flight time distance measuring method and measuring system
US11119196B2 (en) First photon correlated time-of-flight sensor
CN110869804A (en) Early-late pulse counting for light emission depth sensors
CN113661407A (en) Method for measuring optical crosstalk in a time-of-flight sensor and corresponding time-of-flight sensor
US11209310B2 (en) Depth map sensor based on dToF and iToF
US11366225B2 (en) Active-pixel sensor array
US20220171038A1 (en) Multichannel time-of-flight measurement device with time-to-digital converters in a programmable integrated circuit
EP3987305B1 (en) Direct time-of-flight depth sensor architecture and method for operating of such a sensor
US20220099814A1 (en) Power-efficient direct time of flight lidar
WO2022016448A1 (en) Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same
US7907258B2 (en) Method of detecting a light pulse reflected on an object to determine the distance from the object, sensor and device for implementing same
US20230019246A1 (en) Time-of-flight imaging circuitry, time-of-flight imaging system, and time-of-flight imaging method
JP7003949B2 (en) Optical range measuring device
CN112835058A (en) Time-of-flight distance measurement three-dimensional sensor and control method thereof
WO2022056743A1 (en) Method for measuring distance using time-of-flight method and system for measuring distance
US11681028B2 (en) Close-range measurement of time of flight using parallax shift

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20946463

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20946463

Country of ref document: EP

Kind code of ref document: A1