WO2022113637A1 - Dispositif de traitement de signal, dispositif de mesure de distance, procédé de mesure de distance et capteur d'image - Google Patents

Dispositif de traitement de signal, dispositif de mesure de distance, procédé de mesure de distance et capteur d'image Download PDF

Info

Publication number
WO2022113637A1
WO2022113637A1 PCT/JP2021/039866 JP2021039866W WO2022113637A1 WO 2022113637 A1 WO2022113637 A1 WO 2022113637A1 JP 2021039866 W JP2021039866 W JP 2021039866W WO 2022113637 A1 WO2022113637 A1 WO 2022113637A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
light
depth information
modulated light
modulated
Prior art date
Application number
PCT/JP2021/039866
Other languages
English (en)
Japanese (ja)
Inventor
憲明 杉本
Original Assignee
ソニーセミコンダクタソリューションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーセミコンダクタソリューションズ株式会社 filed Critical ソニーセミコンダクタソリューションズ株式会社
Priority to JP2022565154A priority Critical patent/JPWO2022113637A1/ja
Publication of WO2022113637A1 publication Critical patent/WO2022113637A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G01S17/36—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal

Definitions

  • This technique relates to a signal processing device that performs signal processing for distance measurement, a distance measurement device, a distance measurement method, and an image sensor.
  • ToF Time of Flight
  • ToF it is desired to achieve both a range-finding range and a range-finding accuracy.
  • Patent Document 1 a technique is proposed in which two modulation frequencies are superimposed in an analog manner to measure a distance, and a De-alias process is performed.
  • This technique was made in view of the above circumstances, and the purpose is to perform De-alias processing after giving flexibility in the setting of the modulation frequency.
  • the signal processing device includes a first depth information calculation unit that calculates first depth information based on a received signal of the first modulated light modulated by the first frequency, and a first depth information calculation unit that is less than the first frequency.
  • a second depth information calculation unit that calculates second depth information based on the received signal of the second modulated light modulated by two frequencies, and distance information to the subject based on the first depth information and the second depth information.
  • the distance information calculation unit that calculates the above, the folding back determination unit that determines whether or not the phase change of the second modulated light from irradiation to light reception is less than 2 ⁇ , and the phase change of the second modulated light is less than 2 ⁇ . It is provided with a distance information output unit that outputs the distance information when it is determined to be present. When the phase change of the second modulated light is less than 2 ⁇ , the second depth information and the actual distance to the subject are substantially equal.
  • the distance information output unit in the above-mentioned signal processing device may output an invalid value when it is determined that the phase change of the second modulated light is 2 ⁇ or more.
  • the second depth information is shorter than the actual distance to the subject.
  • the distance information calculation unit in the signal processing device described above adds the first folding distance, which is the distance of the subject when the phase change of the first modulated light is 2 ⁇ , and the first depth information. Distance information may be calculated.
  • the first folding distance is a distance to a subject whose phase change from irradiation of the first modulated light to light reception is exactly 2 ⁇ , and is also a distance that can be measured by the first modulated light.
  • the first depth information obtained by the irradiation of the first modulated light is more accurate than the second depth obtained by the irradiation of the second modulated light.
  • the signal processing device described above may be configured to satisfy the following conditional expression (A).
  • the above-mentioned signal processing device may be configured to satisfy the following conditional expression (B).
  • (B) fH (n + 0.5) fL however, fH: 1st frequency fL: 2nd frequency n: Natural number.
  • the distance information calculated when the actual distance to the subject is the same as the distance that can be measured in the second modulated light is a numerical value that is approximately half of the distance that can be measured in the first modulated light.
  • the folding back determination unit in the signal processing device may determine whether or not the phase change of the second modulated light is less than 2 ⁇ by comparing the second depth information and the distance information.
  • the distance information calculated when the actual distance to the subject is 1 times or more and less than 2 times the distance measurable distance in the second modulated light deviates from the calculated second depth information.
  • the folding determination unit in the signal processing device may determine that the phase change of the second modulated light is 2 ⁇ or more when the difference between the second depth information and the distance information is equal to or greater than the threshold value. As a result, when the difference between the second depth information and the distance information is large to some extent, it is determined that the phase change of the second modulated light is 2 ⁇ or more.
  • a distance image may be generated by calculating the distance information for each pixel arranged two-dimensionally.
  • the distance measuring device is a light irradiation unit capable of irradiating a first modulated light modulated by a first frequency and a second modulated light modulated by a second frequency set to be less than the first frequency.
  • a light receiving unit that receives the reflected light reflected by the first modulated light and the second modulated light on the subject and outputs a light receiving signal according to the light intensity of the reflected light, and a light receiving signal output from the light receiving unit.
  • the signal processing unit includes a first depth information calculation unit that calculates first depth information based on the received signal of the first modulated light, and the second modulated light.
  • a second depth information calculation unit that calculates the second depth information based on the received light signal, a distance information calculation unit that calculates the distance information to the subject based on the first depth information and the second depth information, and irradiation.
  • the folding determination unit for determining whether or not the phase change of the second modulated light from to the light receiving light is less than 2 ⁇ , and the distance information when it is determined that the phase change of the second modulated light is less than 2 ⁇ . It has a distance information output unit for outputting.
  • the distance measuring method executed by the distance measuring device is an irradiation process of the first modulated light modulated by the first frequency and a process of calculating the first depth information based on the received signal of the first modulated light. And the irradiation process of the second modulated light modulated by the second frequency set to be less than the first frequency, the process of calculating the second depth information based on the received signal of the second modulated light, and the first.
  • the process of calculating the distance information to the subject based on the depth information and the second depth information, the process of determining whether or not the phase change of the second modulated light from irradiation to light reception is less than 2 ⁇ , and the above. It includes a process of outputting the distance information when it is determined that the phase change of the second modulated light is less than 2 ⁇ .
  • the image sensor according to the present technology can receive the first modulated light modulated by the first frequency and the second modulated light modulated by the second frequency set to be less than the first frequency, and is capable of receiving light in the photoelectric conversion unit.
  • a pixel array unit having a first tap and a second tap for detecting the charge generated by photoelectric conversion is provided, and it is for determining whether or not the phase change from the irradiation of the second modulated light to the light reception is 2 ⁇ or more.
  • the pixel array unit has the detection signals of the first tap and the second tap based on the light receiving of the first modulated light, and the first tap and the second tap based on the light receiving of the second modulated light. It outputs the tap detection signal.
  • the distance measuring device 1 includes a light irradiation unit 2, a light receiving unit 3, a control unit 4, and a signal processing unit 5.
  • the light irradiation unit 2 is configured to include, for example, a light source such as an infrared (IR: Infrared) LED (Light Emitting Diode), and irradiates light based on a control signal input from the control unit 4.
  • the light irradiation unit 2 is capable of irradiating modulated light whose light intensity is modulated based on a specific frequency. Further, the light irradiation unit 2 is capable of irradiating a plurality of types of modulated light having different specific frequencies. In the example shown below, it is possible to irradiate two types of modulated light modulated based on two types of specific frequencies. In the following description, the specific frequency will be referred to as the modulation frequency.
  • the light irradiation unit 2 can irradiate modulated light based on two types of modulation frequencies. Further, as for the two types of modulation frequencies, assuming that the modulation frequency having a relatively high frequency is the first frequency fH and the modulation frequency having a relatively low frequency is the second frequency fL, the first frequency fH is the second frequency. It is determined not to be an integral multiple of the frequency fL. Specifically, it is set so as to satisfy the following equation (1).
  • the first frequency fH and the second frequency fL are set to satisfy the following equation (2).
  • the modulated light modulated based on the first frequency fH is referred to as a first modulated light MLH
  • the modulated light modulated based on the second frequency fL is referred to as a second modulated light MLL.
  • the light irradiation unit 2 may be capable of irradiating modulated light based on three or more types of modulation frequencies.
  • the light receiving unit 3 is an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type, and is configured as a sensor capable of distance measurement using an iToF (Indirect ToF) method.
  • the light receiving unit 3 includes a pixel array unit in which the pixels 31 are two-dimensionally arranged. The light receiving unit 3 performs light receiving control based on the control signal input from the control unit 4.
  • FIG. 2 shows a configuration example of the pixel 31 of the light receiving unit 3.
  • the pixel 31 includes a PD (Photodiode) 32 as a light receiving element, and a first tap 33 and a second tap 34 for detecting charges generated by photoelectric conversion in the PD 32.
  • the PD32 in this example is supposed to have sensitivity to light in the infrared region, for example.
  • the first tap 33 includes a first FD (Floating Diffusion) 35, a first transfer transistor 36 for transferring charges from the PD 32 to the first FD 35, and a selection transistor and a reset transistor (not shown).
  • a first FD Floating Diffusion
  • a first transfer transistor 36 for transferring charges from the PD 32 to the first FD 35
  • a selection transistor and a reset transistor not shown.
  • the second tap 34 includes a second FD 37, a second transfer transistor 38 for transferring charges from the PD 32 to the second FD 37, and a selection transistor and a reset transistor (not shown).
  • both the first transfer transistor 36 and the second transfer transistor 38 are controlled so as not to be turned on at the same time.
  • the signal for controlling the ON / OFF state of the first transfer transistor 36 is synchronized with the light emission cycle of the light source included in the light irradiation unit 2. Further, the control signals applied to the first transfer transistor 36 and the second transfer transistor 38 have a phase difference of 180 degrees.
  • the electric charge transferred to the first FD 35 is output to the signal processing unit 5 as a first signal S1 which is a detection signal output from the first tap 33 in response to the read signal.
  • the electric charge transferred to the second FD 37 is output to the signal processing unit 5 as a second signal S2 which is a detection signal output from the second tap 34 in response to the read signal.
  • the pixel 31 includes a read transistor and a reset transistor (not shown) other than each part shown in FIG.
  • the control unit 4 supplies the light irradiation unit 2 with a drive signal (20 MHz or 50 MHz) corresponding to the modulation frequency for irradiating the modulated light.
  • the light irradiation unit 2 can irradiate the intensity-modulated light based on the drive signal of the supplied rectangular wave.
  • the light irradiation unit 2 may irradiate light whose intensity is modulated based on a sine wave drive signal.
  • control unit 4 controls ON / OFF of the first transfer transistor 36 and the second transfer transistor 38 at the timing synchronized with the modulation frequency.
  • the control unit 4 performs ON / OFF control a plurality of times (for example, hundreds to tens of thousands of times). Specifically, it will be described with reference to FIG. FIG. 3 shows an exposure control signal Se used for exposure control, an irradiation light signal SL provided to the light irradiation unit 2 for irradiating light intensity-modulated at a predetermined modulation frequency, and a first transfer transistor 36. It is a figure which showed the relationship between the 1st control signal St1 which is performed, and the 2nd control signal St2 supplied to the 2nd transfer transistor 38. The first transfer transistor 36 is ON / OFF controlled based on the first control signal St1. Further, the second transfer transistor 38 is ON / OFF controlled based on the second control signal St2.
  • the irradiation of light over the irradiation period TL is performed a plurality of times.
  • the irradiation of light a plurality of times is performed by alternately repeating the irradiation period TL of light and the non-irradiation period.
  • the non-irradiation period of light is the same as the irradiation period TL. That is, the duty ratio is set to 50%.
  • the control unit 4 controls switching of the first transfer transistor 36 so that the electric charge is accumulated in the first FD 35 over the first period T1 synchronized with the irradiation light signal SL. Further, the control unit 4 controls switching of the second transfer transistor 38 so that the electric charge is accumulated in the second FD 37 over the second period T2 synchronized with the irradiation light signal SL.
  • the first period T1 and the second period T2 have the same time length as the irradiation period TL.
  • the switch may be switched once so that the first FD35 and the second FD37 each accumulate an electric charge once in one exposure time. That is, the first transfer transistor 36 is controlled to be in the ON state so that the first FD 35 accumulates charges in the first half of the exposure time Te, and the first transfer transistor 36 is in the OFF state and the second transfer transistor is in the OFF state at the start of the latter half of the exposure time Te. By performing the switching control for switching the 38 to the ON state, the electric charge may be accumulated in the second FD 37 in the latter half of the exposure time.
  • the signal processing unit 5 performs various processing based on the first signal S1 output from the first FD35 of the light receiving unit 3 and the second signal S2 output from the second FD37, and outputs distance information. Therefore, the signal processing unit 5 includes a first depth information calculation unit 51, a second depth information calculation unit 52, a distance information calculation unit 53, a folding back determination unit 54, and a distance information output unit 55. ..
  • the first signal S1 which is the output of the first FD35 and the second signal S2 which is the output of the second FD37 are output.
  • the signals output based on the light received in response to the irradiation of the first modulated light MLH are the first signal S1H and the second signal S1H.
  • the signal S2H is described, and the signals output based on the light received in response to the irradiation of the second modulated light MLL are described as the first signal S1L and the second signal S2L.
  • the first signal S1H and the second signal S2H are input to the first depth information calculation unit 51.
  • An example of the first signal S1H and the second signal S2H is shown in FIG.
  • the first modulated light MLH irradiated based on the irradiation light signal SL is received by the light receiving unit 3 as the reflected light RL reflected by the subject.
  • the electric charge generated by the photoelectric conversion is accumulated in the first FD35 and the second FD37 according to the first control signal St1 and the second control signal St2.
  • the first region AR1 shown in FIG. 5 shows the amount of electric charge accumulated in the first FD35 while the first control signal St1 is turned on. Further, the second region AR2 indicates the amount of electric charge accumulated in the second FD 37 while the second control signal St2 is turned on.
  • the first depth information calculation unit 51 calculates depth information with the subject based on the first region AR1 and the second region AR2.
  • the reflected light reflected by the subject by the first modulated light MLH of 50 MHz is received by the light receiving unit 3 with a delay of the phase ⁇ .
  • the first depth information calculation unit 51 calculates the phase ⁇ , and calculates the depth information according to the phase ⁇ .
  • the first depth information calculation unit 51 calculates the phase ⁇ based on the charges accumulated in the first FD 35 and the second FD 37 in this way.
  • the phase ⁇ calculated by the first depth information calculation unit 51 is calculated between 0 and 2 ⁇ . Therefore, even if the delay of the reflected light with respect to the first modulated light MLH is 2 ⁇ + ⁇ or 4 ⁇ + ⁇ , the phase delay calculated by the first depth information calculation unit 51 is calculated as ⁇ .
  • the depth information calculated by the first depth information calculation unit 51 is the depth information calculated when the first modulated light MLH intensity-modulated at 50 MHz is irradiated, and this is referred to as the first depth information Dp1.
  • the first depth information Dp1 calculated by the first depth information calculation unit 51 has high distance measurement accuracy at the cost of a narrow distance measurement range.
  • a second depth information calculation unit 52 is provided to compensate for the narrow range of the distance measurement range in the first depth information calculation unit 51.
  • the second depth information calculation unit 52 receives the reflected light of the second modulated light MLL whose intensity is modulated at 20 MHz to the first signal S1L output from the first FD35 and the second signal S2L output from the second FD37.
  • the second depth information Dp2 is calculated based on this. Specifically, similarly to the first depth information Dp1, the depth information is calculated by calculating the phase delay ⁇ of the reflected light with respect to the second modulated light MLL.
  • the range-finding range is wider than that of the first depth information Dp1.
  • the first depth information Dp1 output from the first depth information calculation unit 51 and the second depth information Dp2 output from the second depth information calculation unit 52 are output to the distance information calculation unit 53.
  • the distance information calculation unit 53 calculates the distance information Da after De-alias processing using the first depth information Dp1 and the second depth information Dp2.
  • the distance to the subject whose phase ⁇ is 2 ⁇ when irradiated with the first modulated light MLH is defined as the first distance-measurable distance MD1.
  • the distance to the subject whose phase ⁇ is 2 ⁇ when irradiated with the second modulated light MLL is defined as the second distance-measurable distance MD2.
  • the first distance-measurable distance MD1 is defined as the first turn-back distance
  • the second distance-measurable distance MD2 is defined as the second turn-back distance.
  • the distance information Da calculated by the distance information calculation unit 53 is set to a value of 0 or more and less than the second distance-measurable distance MD2. Therefore, when the actual distance Dr to the subject is the second distance-measurable distance MD2 or more, inappropriate distance information Da is calculated.
  • the return determination unit 54 determines whether or not the distance information Da calculated by the distance information calculation unit 53 is appropriate. Specifically, as will be described later, the distance information Da is calculated every time the actual distance Dr to the subject reaches an integral multiple of the second distance-measurable distance MD2.
  • the folding determination unit 54 determines whether the distance information Da is before or after the folding. This determination process is synonymous with the process of determining whether the actual distance Dr to the subject is less than the second distance-measurable distance MD2 or the second distance-measurable distance MD2 or more.
  • the folding back determination unit 54 outputs the determination result of whether or not the second distance-measurable distance MD2 or more and the distance information Da are to the distance information output unit 55.
  • the distance information output unit 55 outputs the distance information Da based on the determination result output from the return determination unit 54.
  • the distance information output unit 55 outputs the distance information Da calculated by the distance information calculation unit 53 when a determination result indicating that the actual distance Dr to the subject is less than the second distance measurable distance MD2 is input. If a determination result indicating that the actual distance Dr to the subject is equal to or greater than the second distance measurable distance MD2 is input, the distance information Da calculated by the distance information calculation unit 53 is not output and is an invalid value. Is output.
  • the invalid value may be a value larger than the second distance measurable distance MD2 as a value larger than the value that the distance information Da can take, or may be a negative value that the distance information Da cannot take. It may be flag information other than a numerical value.
  • the distance measuring device 1 includes optical members such as various lenses and an aperture mechanism for appropriately condensing the reflected light reflected on the subject to the light receiving unit 3, and their optical members. It may be provided with a driver or the like for driving the.
  • FIG. 6 shows the relationship between the first depth information Dp1, the second depth information Dp2, and the distance information Da calculated by the distance information calculation unit 53.
  • both the first depth information Dp1 and the second depth information Dp2 are appropriate according to the actual distance Dr with the subject. Depth information is output.
  • the second depth information Dp2 outputs appropriate depth information according to the actual distance Dr with the subject.
  • depth information less than the first distance-measurable distance MD1 is output as depth information different from the actual distance Dr with the subject.
  • both the first depth information Dp1 and the second depth information Dp2 output depth information different from the actual distance Dr with the subject. .. Specifically, the depth information less than the first distance-measurable distance MD1 is output as the first depth information Dp1, and the depth information less than the second distance-measurable distance MD2 is output as the second depth information Dp2.
  • the distance information calculation unit 53 calculates the distance information Da from the first depth information Dp1 and the second depth information Dp2. Specifically, first, a natural number N satisfying the following equation (3) is obtained.
  • the distance information Da is calculated using the formula (4) shown below.
  • the distance measurement accuracy can be ensured.
  • the distance information Da when the actual distance Dr to the subject is the second distance-measurable distance MD2 will be described.
  • the folding back determination unit 54 determines whether the distance information Da calculated by the distance information calculation unit 53 is before or after being folded back. This determination process is synonymous with the process of determining whether or not the phase change until the second modulated light MLL emitted from the light irradiation unit 2 is reflected by the subject and received by the light receiving unit 3 is less than 2 ⁇ . be.
  • phase change is less than 2 ⁇ by calculating the difference Dt between the distance information Da and the second depth information Dp2 and determining whether or not the difference Dt is larger than the threshold value Th. do.
  • the difference Dt can be calculated by, for example, the following equation (5).
  • the abs function is a function that acquires an absolute value.
  • the phase change is less than 2 ⁇ , that is, if the actual distance Dr to the subject is less than the second distance-measurable distance MD2, the difference Dt between the distance information Da and the second depth information Dp2 is. It is almost zero value.
  • the phase change is 2 ⁇ or more, that is, if the actual distance Dr to the subject is the second distance-measurable distance MD2 or more, the difference Dt is approximately half of the first distance-measurable distance MD1.
  • the threshold value Th which is the comparison target of the difference Dt, is set by using, for example, the following equation (6) in consideration of noise and the like generated in the actual environment.
  • Th (MD1) / 3 ... Equation (6)
  • threshold value Th is a small value close to the zero value, a phase change of less than 2 ⁇ may be erroneously determined to be 2 ⁇ or more, and if it is a large value close to half of the first distance-measurable distance MD1, it is 2 ⁇ or more. There is a risk that the phase change will be erroneously determined to be less than 2 ⁇ .
  • the threshold value Th is appropriately determined by the magnitude of noise in the measurement environment.
  • the noise is small, it is permissible to set the threshold value Th to a small value close to the zero value or a large value close to half of the first distance-measurable distance MD1.
  • the threshold value Th may be configured to change according to the intensity of the received light signal so as not to make an erroneous determination due to noise.
  • the determination result of whether or not the phase change is less than 2 ⁇ (the determination result of whether or not the actual distance Dr to the subject is less than the second distance-measurable distance MD2) is determined from the folding determination unit 54 to the distance information output unit 55. Is output to.
  • the difference Dt is set to a substantially zero value.
  • the actual distance Dr to the subject is more than twice the distance of the second distance-measurable distance MD2, which is the distance to be measured, the reflected light is remarkably weakened. Conceivable. Therefore, when the amount of received light is lower than the predetermined value, the calculated first depth information Dp1 and second depth information Dp2 are ignored so that inappropriate distance information is not output in the subsequent distance information output processing. Can be.
  • the actual distance Dr by adjusting the emission intensity so that the amount of light received by the subject at a distance more than twice the distance to be measured is lower than the predetermined value, the actual distance Dr.
  • the distance is 2 times or more and less than 3 times the second distance measurable distance MD2
  • the distance information output unit 55 receives the determination result and the distance information Da from the return determination unit 54, and outputs the distance information Da as necessary. Specifically, when the determination result indicating that the phase change is less than 2 ⁇ is received, the distance information Da is the information indicating the actual distance Dr with the subject, so the distance information Da is output.
  • the distance information Da is the information deviating from the actual distance Dr with the subject, so the distance information Da received from the folding determination unit 54 is used. It is not output as it is.
  • information indicating an invalid value may be stored in the distance information Da and output as the distance information Da.
  • a value larger than the value that the distance information Da can take may be set to a value larger than the second distance-measurable distance MD2, or a negative value that the distance information Da cannot take. It may be a value of, or it may be a flag information other than a numerical value.
  • step S101 the control unit 4 of the distance measuring device 1 performs light emission processing and exposure processing by cooperating with the light irradiation unit 2 and the light receiving unit 3.
  • step S101 irradiation and exposure of the first modulated light MLH and irradiation and exposure of the second modulated light MLL are performed. Therefore, in step S101, the first signal S1 and the second signal S2 based on the light reception of the first modulated light MLH and the first signal S1 and the second signal S2 based on the light reception of the second modulated light MLL are output.
  • the signal processing unit 5 performs a process of resetting the counter ct to a zero value in step S102.
  • the counter ct is a counter that is added each time the processing for each pixel is completed, and is used to determine whether or not the processing required for generating one distance image has been completed.
  • the first depth information calculation unit 51 and the second depth information calculation unit 52 of the signal processing unit 5 perform a process of calculating the first depth information Dp1 and the second depth information Dp2 in step S103.
  • the distance information calculation unit 53 of the signal processing unit 5 calculates the distance information Da using the first depth information Dp1 and the second depth information Dp2 in step S104.
  • the distance information Da calculated here is a value equal to or greater than a zero value and less than or equal to the second distance-measurable distance MD2.
  • the wrapping determination unit 54 of the signal processing unit 5 makes a wrapping determination in step S105.
  • the distance information output unit 55 of the signal processing unit 5 is the distance calculated by the distance information calculation unit 53 in step S106. Information Da is output.
  • the distance information output unit 55 of the signal processing unit 5 outputs an invalid value in step S107.
  • the signal processing unit 5 determines in step S108 whether or not the counter ct is less than the value obtained by multiplying the number of horizontal pixels Pw and the number of vertical pixels Ph.
  • the number of horizontal pixels Pw indicates the number of pixels in the horizontal direction (horizontal direction) in the captured image.
  • the number of vertical pixels Ph indicates the number of pixels in the vertical direction (vertical direction) in the captured image.
  • the signal processing unit 5 adds 1 to the counter ct in step S109 and returns to the processing of step S103.
  • the signal processing unit 5 returns to the processing of step S101 and performs light emission processing and exposure processing for obtaining a distance image of the next frame.
  • the distance measuring device 1 can generate a plurality of distance images by executing a series of processes shown in FIG. 7.
  • the ranging device 1 as a signal processing device calculates the first depth information Dp1 based on the received signal of the first modulated optical MLH modulated by the first frequency fH (for example, 50 MHz).
  • the second depth information Dp2 is calculated based on the received signal of the first depth information calculation unit 51 and the second modulated light MLL modulated by the second frequency fL (for example, 20 MHz) set to be less than the first frequency fH.
  • Phase change of the depth information calculation unit 52, the distance information calculation unit 53 that calculates the distance information Da to the subject based on the first depth information Dp1 and the second depth information Dp2, and the second modulated light MLL from irradiation to light reception.
  • the second depth information Dp2 and the actual distance to the subject are made substantially equal. Therefore, it is possible to output appropriate distance information Da according to the actual distance to the subject.
  • the distance information output unit 55 in the distance measuring device 1 has an invalid value as the distance information Da when it is determined that the phase change of the second modulated light MLL is 2 ⁇ or more. May be output.
  • the second depth information Dp2 is shorter than the actual distance to the subject (actual distance Dr from the subject). Therefore, it is possible to prevent the distance information Da that deviates from the actual distance to the subject from being output.
  • the distance information calculation unit 53 in the distance measuring device 1 determines the distance of the subject when the phase change of the first modulated light MLH is 2 ⁇ in the second depth information Dp2.
  • the distance information Da may be calculated by adding the first turn-back distance and the first depth information Dp1.
  • the first folding distance is the distance to the subject where the phase change from the irradiation of the first modulated light MLH to the light reception is exactly 2 ⁇ , and is also the distance that can be measured by the first modulated light MLH. ..
  • the first depth information Dp1 obtained by irradiation with the first modulated light MLH has higher accuracy than the second depth information Dp2 obtained by irradiation with the second modulated light MLL.
  • the distance information Da calculated by the distance information calculation unit 53 can be made highly accurate.
  • the distance measuring device 1 may be configured to satisfy the following conditional expression (A).
  • A fH ⁇ n ⁇ fL however, fH: 1st frequency fL: 2nd frequency n: Natural number.
  • the distance information Da calculated when the actual distance to the subject is 1 times or more and less than 2 times the second distance measurable distance MD2 in the second modulated light MLL is obtained. It is no longer a zero value. Therefore, it is possible to determine whether or not the distance to the subject deviates from the actual distance.
  • the distance measuring device 1 may be configured to satisfy the following conditional expression (B).
  • (B) fH (n + 0.5) fL however, fH: 1st frequency fL: 2nd frequency n: Natural number.
  • the distance information Da calculated when the actual distance to the subject (actual distance Dr from the subject) is the same as the second distance-measurable distance MD2 in the second modulated light MLL is the first in the first modulated light MLH. It is set to be approximately half the value of one distance-measurable distance MD1. Therefore, the second distance-measurable distance MD2 in the second modulated light MLL is a value at the exact center of the distance that is a constant multiple of the first distance-measurable distance MD1 in the first modulated light MLH, so that the actual distance to the subject is actually reached. Not only is it possible to determine whether or not there is a deviation from the distance, but it is also possible to increase the resistance to erroneous determination due to noise, calibration deviation, color mixing, and the like.
  • the folding determination unit 54 in the distance measuring device 1 compares the second depth information Dp2 with the distance information Da to determine whether the phase change of the second modulated light MLL is less than 2 ⁇ . It may be determined whether or not.
  • the distance information Da calculated when the actual distance to the subject (actual distance Dr with the subject) is 1 times or more and less than 2 times the second distance measurable distance MD2 in the second modulated light MLL is calculated. It is different from the second depth information Dp2 to be performed. Therefore, by calculating the difference Dt between the second depth information Dp2 and the distance information Da, it can be determined whether or not the phase change of the second modulated light MLL is less than 2 ⁇ . This makes it possible to output appropriate distance information according to the actual distance to the subject.
  • the folding determination unit 54 in the distance measuring device 1 determines the phase change of the second modulated light MLL when the difference Dt between the second depth information Dp2 and the distance information Da is equal to or greater than the threshold value Th. May be determined to be 2 ⁇ or more. As a result, it is determined that the phase change of the second modulated light MLL is 2 ⁇ or more when the difference Dt between the second depth information Dp2 and the distance information Da is large to some extent. Therefore, it is possible to prevent erroneous determination due to noise or the like.
  • a distance image may be generated by calculating the distance information Da for each of the two-dimensionally arranged pixels 31.
  • the distance information Da for each pixel 31 in the distance image when the phase change of the second modulated light MLL is less than 2 ⁇ , the second depth information Dp2 and the actual distance to the subject (actual distance Dr from the subject). ) are approximately equal. Therefore, it is possible to output a distance image based on appropriate distance information Da according to the actual distance to the subject.
  • This technology> (1) A first depth information calculation unit that calculates first depth information based on the received signal of the first modulated light modulated by the first frequency, and a first depth information calculation unit. A second depth information calculation unit that calculates the second depth information based on the received signal of the second modulated light modulated by the second frequency set to be less than the first frequency, and the second depth information calculation unit.
  • a distance information calculation unit that calculates distance information to the subject based on the first depth information and the second depth information, and A folding determination unit for determining whether or not the phase change of the second modulated light from irradiation to light reception is less than 2 ⁇
  • a signal processing device including a distance information output unit that outputs the distance information when it is determined that the phase change of the second modulated light is less than 2 ⁇ .
  • the distance information calculation unit performs the first folding distance and the above.
  • the signal processing device according to any one of (1) to (2) above, which calculates the distance information by adding the first depth information.
  • the signal processing apparatus according to (3) above which satisfies the following conditional expression (A).
  • the folding determination unit determines whether or not the phase change of the second modulated light is less than 2 ⁇ by comparing the second depth information and the distance information. Any of the above (4) to (5).
  • a light irradiation unit capable of irradiating a first modulated light modulated by a first frequency and a second modulated light modulated by a second frequency set to be less than the first frequency, and a light irradiation unit.
  • a light receiving unit that receives the reflected light reflected by the first modulated light and the second modulated light on the subject and outputs a light receiving signal corresponding to the light intensity of the reflected light.
  • a signal processing unit that performs signal processing on the light receiving signal output from the light receiving unit is provided.
  • the signal processing unit A first depth information calculation unit that calculates first depth information based on the received signal of the first modulated light, and a first depth information calculation unit.
  • a second depth information calculation unit that calculates second depth information based on the received signal of the second modulated light, and a second depth information calculation unit.
  • a distance information calculation unit that calculates distance information to the subject based on the first depth information and the second depth information, and A folding determination unit for determining whether or not the phase change of the second modulated light from irradiation to light reception is less than 2 ⁇ , and
  • a distance measuring device including a distance information output unit that outputs the distance information when it is determined that the phase change of the second modulated light is less than 2 ⁇ .
  • a pixel array unit having a first tap and a second tap is provided.
  • the pixel array unit has the first tap and the first tap based on the light receiving of the first modulated light.
  • An image sensor that outputs the detection signal of the second tap and the detection signals of the first tap and the second tap based on the light reception of the second modulated light.

Landscapes

  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Measurement Of Optical Distance (AREA)

Abstract

L'invention concerne un dispositif de traitement de signal comprenant : une unité de calcul de premières informations de profondeur pour calculer des premières informations de profondeur sur la base d'un signal de réception de lumière pour une première lumière modulée qui a été modulée à l'aide d'une première fréquence ; une unité de calcul de secondes informations de profondeur pour calculer des secondes informations de profondeur sur la base d'un signal de réception de lumière pour une seconde lumière modulée qui a été modulée à l'aide d'une seconde fréquence inférieure à la première fréquence ; une unité de calcul d'informations de distance pour calculer des informations concernant la distance à un sujet sur la base des premières informations de profondeur et des secondes informations de profondeur ; une unité de détermination de pliage pour déterminer si le changement de phase de la seconde lumière modulée de l'émission à la réception de lumière est inférieur à 2π ; et une unité de sortie d'informations de distance pour délivrer les informations de distance si le changement de phase de la seconde lumière modulée a été déterminé comme étant inférieur à 2π.
PCT/JP2021/039866 2020-11-26 2021-10-28 Dispositif de traitement de signal, dispositif de mesure de distance, procédé de mesure de distance et capteur d'image WO2022113637A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022565154A JPWO2022113637A1 (fr) 2020-11-26 2021-10-28

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-196095 2020-11-26
JP2020196095 2020-11-26

Publications (1)

Publication Number Publication Date
WO2022113637A1 true WO2022113637A1 (fr) 2022-06-02

Family

ID=81755753

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/039866 WO2022113637A1 (fr) 2020-11-26 2021-10-28 Dispositif de traitement de signal, dispositif de mesure de distance, procédé de mesure de distance et capteur d'image

Country Status (2)

Country Link
JP (1) JPWO2022113637A1 (fr)
WO (1) WO2022113637A1 (fr)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS4928238B1 (fr) * 1969-02-26 1974-07-24
JP2013538342A (ja) * 2010-07-21 2013-10-10 マイクロソフト コーポレーション 階層的な飛行時間（ｔｏｆ）システムのディエリアシング方法およびシステム
DE102013207653A1 (de) * 2012-05-21 2013-11-21 Ifm Electronic Gmbh Lichtlaufzeitkamerasystem
US20160116594A1 (en) * 2014-10-27 2016-04-28 Microsoft Corporation Time of flight camera
US20160216376A1 (en) * 2015-01-28 2016-07-28 Texas Instruments Incorporated Efficient Implementation of Distance De-Aliasing for Ranging Systems using Phase Domain Computation
JP2017201760A (ja) * 2016-05-06 2017-11-09 株式会社ニコン 撮像装置および測距装置
US20180011195A1 (en) * 2016-07-07 2018-01-11 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
JP2018036145A (ja) * 2016-08-31 2018-03-08 キヤノン株式会社 距離計測装置、および距離計測方法
WO2020037167A1 (fr) * 2018-08-17 2020-02-20 Sense Photonics, Inc. Procédés et systèmes pour accroître la plage de systèmes de temps de vol par basculement de plage non ambiguë

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS4928238B1 (fr) * 1969-02-26 1974-07-24
JP2013538342A (ja) * 2010-07-21 2013-10-10 マイクロソフト コーポレーション 階層的な飛行時間（ｔｏｆ）システムのディエリアシング方法およびシステム
DE102013207653A1 (de) * 2012-05-21 2013-11-21 Ifm Electronic Gmbh Lichtlaufzeitkamerasystem
US20160116594A1 (en) * 2014-10-27 2016-04-28 Microsoft Corporation Time of flight camera
US20160216376A1 (en) * 2015-01-28 2016-07-28 Texas Instruments Incorporated Efficient Implementation of Distance De-Aliasing for Ranging Systems using Phase Domain Computation
JP2017201760A (ja) * 2016-05-06 2017-11-09 株式会社ニコン 撮像装置および測距装置
US20180011195A1 (en) * 2016-07-07 2018-01-11 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
JP2018036145A (ja) * 2016-08-31 2018-03-08 キヤノン株式会社 距離計測装置、および距離計測方法
WO2020037167A1 (fr) * 2018-08-17 2020-02-20 Sense Photonics, Inc. Procédés et systèmes pour accroître la plage de systèmes de temps de vol par basculement de plage non ambiguë

Also Published As

Publication number Publication date
JPWO2022113637A1 (fr) 2022-06-02

Similar Documents

Publication Publication Date Title
US10686994B2 (en) Imaging device, and solid-state imaging element used for same
US10545239B2 (en) Distance-measuring imaging device and solid-state imaging device
JP6631718B2 (ja) マルチパス誤差を検出するための飛行時間型距離測定装置及び方法
JP5593479B2 (ja) バックグランド放射光の抑制に有利なｔｏｆ領域
US10237534B2 (en) Imaging device and a method for producing a three-dimensional image of an object
US10073164B2 (en) Distance-measuring/imaging apparatus, distance measuring method of the same, and solid imaging element
US20190179017A1 (en) Time-of-flight distance measurement device
JP2010504509A (ja) シーン（情景）の３ｄ画像を捕捉する方法及びシステム
WO2019200833A1 (fr) Procédé et appareil de mesure de distance, capteur de mesure de distance et réseau de détection de mesure de distance
JP5180501B2 (ja) 測距装置及び測距方法
JP2020046247A (ja) 距離計測装置、及び距離計測方法
JP2017201760A (ja) 撮像装置および測距装置
KR20120043843A (ko) 3차원 영상화 펄스 레이저 레이더 시스템 및 이 시스템에서의 자동 촛점 방법
WO2022113637A1 (fr) Dispositif de traitement de signal, dispositif de mesure de distance, procédé de mesure de distance et capteur d'image
EP4136402A1 (fr) Détection 3d à grand angle
US20220075070A1 (en) Distance image measurement device, distance image measurement system, and distance image measurement method
JP2001264014A (ja) 光センサおよび三次元形状計測装置
JP2008241258A (ja) 測距装置及び測距方法
JP4673674B2 (ja) 形状測定装置
JP2023001122A (ja) 撮像装置
US20220276361A1 (en) Time-of-flight sensing circuitry, time-of-flight imaging portion, signal processing method
CN113740866A (zh) 探测单元、探测装置及方法
US20230204727A1 (en) Distance measurement device and distance measurement method
WO2022224580A1 (fr) Dispositif de mesure de distance et système de mesure de distance
US20230082977A1 (en) Apparatus and method for time-of-flight sensing of a scene

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21897613

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022565154

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21897613

Country of ref document: EP

Kind code of ref document: A1