WO2024014393A1 - Dispositif de télémétrie, dispositif de génération de quantité de correction, procédé de télémétrie et procédé de génération de quantité de correction - Google Patents

Dispositif de télémétrie, dispositif de génération de quantité de correction, procédé de télémétrie et procédé de génération de quantité de correction Download PDF

Info

Publication number
WO2024014393A1
WO2024014393A1 PCT/JP2023/025129 JP2023025129W WO2024014393A1 WO 2024014393 A1 WO2024014393 A1 WO 2024014393A1 JP 2023025129 W JP2023025129 W JP 2023025129W WO 2024014393 A1 WO2024014393 A1 WO 2024014393A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
correction amount
pixel
solid
image sensor
Prior art date
Application number
PCT/JP2023/025129
Other languages
English (en)
Japanese (ja)
Inventor
真由 小川
清治 中村
Original Assignee
ヌヴォトンテクノロジージャパン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ヌヴォトンテクノロジージャパン株式会社 filed Critical ヌヴォトンテクノロジージャパン株式会社
Publication of WO2024014393A1 publication Critical patent/WO2024014393A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • G01S7/4861—Circuits for detection, sampling, integration or read-out
    • G01S7/4863—Detector arrays, e.g. charge-transfer gates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/497—Means for monitoring or calibrating

Definitions

  • the present disclosure relates to a distance measurement device, a correction amount generation device, a distance measurement method, and a correction amount generation method.
  • a distance measuring device that employs the indirect TOF method includes, for example, a light source and a solid-state image sensor. Such a distance measuring device uses a solid-state imaging device to receive light emitted by a light source that is reflected by an object, and generates a distance image using a signal based on the reflected light.
  • the present disclosure provides a distance measurement device, a correction amount generation device, a distance measurement method, and a correction amount generation method that can improve the quality of distance images.
  • a distance measuring device includes a light source that emits irradiation light, and a photoelectric conversion element that receives reflected light reflected by an object of the irradiation light irradiated from the light source and converts it into an electric charge.
  • a solid-state image sensor that includes a plurality of pixels arranged in a two-dimensional manner and outputs a signal based on the charge for each pixel; a distance calculation unit that calculates the distance to the object for each pixel by performing a predetermined calculation based on the output signal; and correction of the distance for each pixel based on fixed pattern noise of the solid-state image sensor. and a distance correction unit that corrects the distance calculated by the distance calculation unit for each pixel using the correction amount stored in the storage unit.
  • a correction amount generation device is a correction amount generation device that generates a correction amount for correcting a distance calculated by a distance measuring device including a solid-state image sensor, the solid-state image sensor comprising:
  • the correction amount generating device includes a plurality of pixels each including a photoelectric conversion element that receives light and converts it into an electric charge, and is arranged in a two-dimensional manner, and the correction amount generating device is configured to fix the solid-state image sensor.
  • a distance correction amount for each pixel is generated based on pattern noise, and the generated correction amount is output.
  • a distance measuring method is a distance measuring method using a distance measuring device, wherein the distance measuring device includes a light source that irradiates irradiation light and an object of the irradiation light irradiated from the light source.
  • a plurality of pixels each of which includes a photoelectric conversion element that receives reflected reflected light and converts it into an electric charge, are arranged in a two-dimensional manner, and each pixel generates a signal based on the electric charge. and a solid-state image sensor that outputs, and the distance measuring method calculates the distance to the target object by performing a predetermined calculation based on the signal output from the solid-state image sensor for each pixel.
  • a distance calculation step, and a distance correction step of correcting the distance calculated in the distance calculation step for each pixel using a distance correction amount for each pixel based on fixed pattern noise of the solid-state image sensor. include.
  • a correction amount generation method is a correction amount generation method for generating a correction amount for correcting a distance calculated by a distance measuring device including a solid-state image sensor, the solid-state image sensor comprising:
  • the correction amount generation method includes a plurality of pixels each including a photoelectric conversion element that receives light and converts it into an electric charge, and is arranged in a two-dimensional manner, and the correction amount generation method is performed by fixing the solid-state image sensor.
  • the method includes a generation step of generating a distance correction amount for each pixel based on pattern noise, and an output step of outputting the generated correction amount.
  • the quality of distance images can be improved.
  • FIG. 1 is a functional block diagram showing an example of the configuration of a distance measuring device according to an embodiment.
  • FIG. 2 is a schematic diagram of a solid-state image sensor included in the distance measuring device according to the embodiment.
  • FIG. 3 is a flowchart showing an overview of the operation of the distance measuring device according to the embodiment.
  • FIG. 4 is a diagram illustrating an example of timing of light emission and exposure of the distance measuring device according to the embodiment.
  • FIG. 5 is a flowchart showing an overview of the operation of the correction amount generation section of the distance measuring device according to the embodiment.
  • FIG. 6 is a functional block diagram showing an example of the configuration of a distance measuring device according to a modification of the embodiment.
  • FIG. 7 is a diagram showing an example of the correspondence between temperature ranges and correction tables.
  • FIG. 8 is a flowchart showing an overview of the operation of the distance measuring device according to a modification of the embodiment.
  • FIG. 9 is a functional block diagram showing an example of the configuration of another distance measuring device according
  • Patent Document 1 and Patent Document 2 disclose distance measuring devices that perform correction processing in order to improve distance measurement accuracy.
  • the techniques of Patent Document 1 and Patent Document 2 are insufficient to suppress variations in distance measurement accuracy between pixels and improve the quality of distance images.
  • noise reduction processing may be performed using a two-dimensional spatial filter such as a Gaussian filter and an averaging filter.
  • a two-dimensional spatial filter such as a Gaussian filter and an averaging filter.
  • two-dimensional spatial filters can remove high-frequency component noise and improve distance measurement accuracy, low-frequency component noise may remain, resulting in wave-like patterns in distance images. be.
  • applying a large spatial filter significantly reduces the resolution of the range image.
  • adaptive processing is required, leading to a significant increase in the number of processing circuits.
  • the present inventors focused on the fact that the main noises that occur in a solid-state image sensor and affect the distance include shot noise and fixed pattern noise.
  • Shot noise is noise that occurs as random thermal noise. For example, shot noise tends to increase outdoors where sunlight is incident because the average size of the noise varies depending on the amount of light incident on the solid-state image sensor. Furthermore, the influence of shot noise also changes depending on the distance from the distance measuring device to the target object and the reflectance of the target object.
  • fixed pattern noise is fixed noise whose magnitude does not change depending on the environment, distance to the object to be measured, time, etc.
  • Fixed pattern noise is caused by, for example, variations in manufacturing of solid-state image sensors.
  • the magnitude of noise differs from pixel to pixel.
  • the influence of shot noise can be removed by increasing the number of distance measurements.
  • fixed pattern noise occurs as a fixed amount of error for each pixel, the influence cannot be removed even if the number of distance measurements is increased.
  • fixed pattern noise varies from pixel to pixel and causes quality deterioration such as unevenness in a distance image.
  • the present inventors have found that it is important to reduce the influence of fixed pattern noise on distance in order to improve distance image quality.
  • the inventors also discovered that the effect of fixed pattern noise appears as an offset in the distance for each pixel.
  • the magnitude of fixed pattern noise is constant regardless of the environment, etc., and fixed pattern noise affects distance as an offset, so the effect is particularly large when measuring the distance to a relatively close object. .
  • the present disclosure provides a distance measuring device, a correction amount generation device, a distance measurement method, and a correction amount generation method that can improve the quality of distance images by reducing the influence of fixed pattern noise on distance measurement accuracy.
  • FIG. 1 is a functional block diagram showing an example of the configuration of a distance measuring device 100 according to an embodiment.
  • FIG. 2 is a schematic diagram of the solid-state image sensor 20 included in the distance measuring device 100 according to the embodiment.
  • the distance measuring device 100 is a distance measuring device using a TOF method.
  • the distance measuring device 100 generates a distance image indicating the distance to the object OBJ.
  • the quality of the generated distance image can be improved, so the accuracy in a recognition system such as face recognition can be improved, for example. Therefore, the distance measuring device 100 may be used in a recognition system such as a face recognition system.
  • the distance measuring device 100 includes a light source 10, a solid-state image sensor 20, a drive control section 30, a distance calculation section 50, a distance correction section 60, a correction amount generation section 70, and a storage section 80.
  • the light source 10 irradiates the object OBJ with irradiation light at a predetermined pulse width according to the timing indicated by the input light emission control pulse.
  • the light source 10 is, for example, a light emitting diode or a laser element that emits infrared light.
  • the solid-state image sensor 20 is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.
  • the solid-state image sensor 20 has a plurality of pixels 21 arranged two-dimensionally. In FIG. 2, for the sake of explanation, the configuration is 4 pixels horizontally and 4 pixels vertically, which is a total of 16 pixels, but the number of pixels 21 that the solid-state image sensor 20 has is not particularly limited.
  • the number of pixels 21 that the solid-state image sensor 20 has may be, for example, 20,000 pixels or more and 5 million pixels or less.
  • Each of the plurality of pixels 21 includes at least one photoelectric conversion element such as a photodiode that receives reflected light from the object OBJ of the irradiation light emitted from the light source 10 and converts it into an electric charge.
  • a photoelectric conversion element such as a photodiode that receives reflected light from the object OBJ of the irradiation light emitted from the light source 10 and converts it into an electric charge.
  • the solid-state image sensor 20 outputs, for each pixel 21, a signal based on the charge obtained in each exposure according to a plurality of exposure control pulses.
  • the solid-state image sensor 20 transfers charges obtained in each exposure to different vertical transfer paths (VCCDs) or charge storage units (FDs) in accordance with a plurality of exposure control pulses for each pixel 21, and performs multiple exposures.
  • a signal based on the charge obtained in each exposure is read out according to the control pulse.
  • the solid-state image sensor 20 performs, for example, AD (Analog to Digital) conversion when reading a signal, and outputs an AD-converted digital signal.
  • AD Analog to Digital
  • the drive control section 30 outputs control signals for controlling the operations of the light source 10 and the solid-state image sensor 20.
  • the drive control unit 30 outputs, as a control signal for controlling the light source 10, a light emission control pulse that instructs the light source 10 to emit irradiation light with a predetermined pulse width. Further, the drive control unit 30 outputs an exposure control pulse that instructs the solid-state image sensor 20 to perform exposure as a control signal for controlling the solid-state image sensor 20 .
  • the distance calculation unit 50 calculates the distance to the object OBJ for each pixel 21 by performing a predetermined calculation based on the signal output from the solid-state image sensor 20 for each pixel 21.
  • the distance calculation unit 50 outputs the calculated distance to the distance correction unit 60. Further, the distance calculation unit 50 outputs the calculated distance to the correction amount generation unit 70 when a process for generating a correction amount, which will be described later, is performed.
  • the distance correction unit 60 corrects the distance calculated by the distance calculation unit 50 for each pixel 21 based on the distance correction amount for each pixel 21 stored in the storage unit 80.
  • the distance correction unit 60 outputs the corrected distance for each pixel 21.
  • the correction amount used by the distance correction unit 60 is a fixed distance amount for each pixel 21, and does not change every time the distance is corrected.
  • the correction amount generation unit 70 generates a distance correction amount for each pixel 21 for all pixels 21 included in the solid-state image sensor 20 based on the fixed pattern noise of the solid-state image sensor 20.
  • the correction amount generation unit 70 outputs the generated correction amount and stores it in the storage unit 80.
  • the drive control unit 30, distance calculation unit 50, distance correction unit 60, and correction amount generation unit 70 are realized by, for example, a memory that stores a program, a processor that executes the program, and the like. Although they are shown as separate block diagrams, all or some of the drive control section 30, distance calculation section 50, distance correction section 60, and correction amount generation section 70 are configured with the same memory and processor. You can.
  • the storage unit 80 stores the distance correction amount for each pixel 21 based on the fixed pattern noise of the solid-state image sensor 20.
  • the storage unit 80 stores, for example, the correction amount generated by the correction amount generation unit 70.
  • the storage unit 80 stores, for example, a correction table 81 in which the position of the pixel 21 and the distance correction amount are associated for each pixel 21 with respect to all the pixels 21 included in the solid-state image sensor 20.
  • the storage unit 80 is realized by, for example, a memory such as a semiconductor memory.
  • the correction amount generation section 70 does not need to be included in the distance measuring device 100, and may be realized as an independent correction amount generation device having the function of the correction amount generation section 70, for example. Further, in this case, the correction amount generation device may further include a distance calculation section 50.
  • FIG. 3 is a flowchart showing an overview of the operation (distance measurement method) of the distance measurement device 100 according to the embodiment.
  • FIG. 4 is a diagram showing an example of timing of light emission and exposure of the distance measuring device 100 according to the embodiment.
  • the drive control unit 30 outputs a light emission control pulse to the light source 10 and outputs an exposure control pulse to the solid-state image sensor 20 (step S11).
  • the solid-state image sensor 20 receives the reflected light from the object OBJ of the irradiation light from the light source 10, and outputs a signal based on the charge obtained by exposure according to the exposure control pulse for each pixel 21. do.
  • a digital signal that has been AD converted by the solid-state image sensor 20 is output to the distance calculation section 50.
  • the drive control section 30 outputs a light emission control pulse with a pulse width Tp, for example.
  • the light emission control pulse causes the light source 10 to emit illumination light when it is at a high level. Thereby, the light source 10 emits pulsed light having a pulse width Tp.
  • the drive control unit 30 outputs a plurality of exposure control pulses A0, A1, and A2 for each pixel 21, for example.
  • the exposure control pulse exposes the pixel 21 of the solid-state image sensor 20 when it is at a low level.
  • the plurality of exposure control pulses A0, A1, and A2 have different start timings based on the light emission control pulse.
  • the exposure widths (exposure periods) of the plurality of exposure control pulses A0, A1, and A2 are each the same as the pulse width Tp of the irradiation light. Specifically, the exposure control pulse A0 starts at the same time t1 as the start timing of the light emission control pulse.
  • the exposure control pulse A1 starts at time t3, which is the same time as time t2, which is the end timing of the light emission control pulse and the exposure control pulse A0.
  • the exposure control pulse A2 starts at time t5, which is the same time as time t4, which is the end timing of the exposure control pulse A1, and ends at time t6.
  • the phase difference between the exposure control pulse A0 and the exposure control pulse A1 and the phase difference between the exposure control pulse A1 and the exposure control pulse A2 are each the pulse width Tp of the irradiation light.
  • the solid-state image sensor 20 transmits, for each pixel 21, a signal S0 based on the charge obtained by exposure according to the exposure control pulse A0, a signal S1 based on the charge obtained by the exposure according to the exposure control pulse A1, and a signal S1 based on the charge obtained by exposure according to the exposure control pulse A1.
  • a signal S2 based on the charge obtained by exposure according to the control pulse A2 is output.
  • the exposure control pulse A0 is an example of the first exposure control pulse
  • the exposure control pulse A1 is an example of the second exposure control pulse.
  • the signal S0 is an example of the first signal
  • the signal S1 is an example of the second signal.
  • any one of the exposure control pulse A0, the exposure control pulse A1, and the exposure control pulse A2 may be output for one emission control pulse.
  • the signals S0, S1, and S2 are output from each pixel 21 by outputting the light emission control pulses corresponding to the plurality of exposure control pulses A0, A1, and A2, and the plurality of exposure control pulses A0, A1, and A2. May be output.
  • the distance calculation unit 50 calculates the distance to the object OBJ for each pixel 21 by performing a predetermined calculation based on the signal output from the solid-state image sensor 20 for each pixel 21 (step S12). .
  • the distance calculation unit 50 outputs the calculated distance to the distance correction unit 60.
  • step S12 is an example of a distance calculation step.
  • the calculation of the distance by the distance calculation unit 50 is performed, for example, as follows.
  • the reflected light of the irradiation light from the light source 10 reflected by the object OBJ returns to the distance measuring device 100 with a delay of time ⁇ t from the irradiation of the irradiation light, and enters the solid-state image sensor 20.
  • the time ⁇ t is shorter than the pulse width Tp.
  • signals based on charges generated by exposure according to each exposure control pulse are schematically shown as patterned rectangles. The area of the rectangle corresponds to the size of the signal. A part of the reflected light is received by the pixel 21 during exposure according to the exposure control pulse A0, and is converted into an electric charge.
  • the signal S0 is a signal based on the background light and the reflected light having a time width obtained by subtracting the time ⁇ t from the pulse width Tp.
  • the signal S1 is a signal based on background light and reflected light having a time width of time ⁇ t.
  • Signal S2 is a signal based on background light. Therefore, by subtracting the signal S2 from each of the signal S0 and the signal S1, the influence of background light can be removed. Further, the sum of the signal S0 obtained by subtracting the signal S2 and the signal S1 obtained by subtracting the signal S2 becomes a signal based on the reflected light having the time width of the pulse width Tp. Therefore, ⁇ t is calculated by the following formula.
  • the signals S0, S1, and S2 may each be output multiple times from the solid-state image sensor 20 in step S11, and the distance calculation unit 50 may calculate the distance using the signals S0, S1, and S2 output multiple times. .
  • the distance calculation unit 50 may calculate the distance using the average value of each of the signals S0, S1, and S2 output multiple times.
  • the distance calculated by the distance calculation unit 50 does not have to be an absolute distance, and may be a normalized distance value that takes a value from 0 to 1, for example.
  • the timing of the light emission control pulse and the exposure control pulse is not limited to the example shown in FIG. You may go.
  • the distance calculation unit 50 may calculate the distance without subtracting the signal S2 from the signal S1 and the signal S0 when the influence of background light is slight. In this case, the exposure control pulse A2 may not be output.
  • step S13 is an example of a distance correction step.
  • the distance correction unit 60 outputs the corrected distance for each pixel 21, that is, a distance image, to the outside.
  • step S13 the distance correction unit 60 corrects the distance calculated by the distance calculation unit 50 for each pixel 21, for example, for all pixels 21 included in the solid-state image sensor 20. Note that the distance correction unit 60 may correct the distance calculated by the distance calculation unit 50 for each pixel 21 for some of the pixels 21 included in the solid-state image sensor 20.
  • the distance correction unit 60 calculates the distance for each pixel 21 only for the pixels 21 corresponding to the area necessary for recognition processing.
  • the distance calculated by the calculation unit 50 may be corrected.
  • each pixel 21 of the solid-state image sensor 20 is ideally performed at the same timing as the exposure control pulse as shown in FIG. Due to variations, etc., a deviation from the timing of the exposure control pulse may occur for each pixel 21.
  • the timing of the switching operation varies among the pixels 21 due to the characteristics of the transistor for starting the exposure operation. Therefore, a shift in the times at which the transistors should be turned on and off according to each exposure control pulse, such as from time t1 to t6 shown in FIG. 4, may occur for each pixel 21. In other words, a shift in the start and end times of exposure according to each exposure control pulse may occur for each pixel 21.
  • the transfer time when transferring charges converted by a photoelectric conversion element to convert them into a signal may vary from pixel to pixel 21.
  • the charge transfer time from the photoelectric conversion element to the vertical transfer path varies.
  • the time for charge transfer from the photoelectric conversion element to the charge storage section varies. Therefore, even if each pixel 21 receives the reflected light for the same amount of time, the timing at which the charges are transferred is shifted, and the time for receiving the reflected light (the amount of light received) during exposure substantially varies.
  • the solid-state image sensor 20 unique fixed pattern noise occurs in each pixel 21 due to variations in switching operation and transfer. Further, since this variation is caused by the characteristics of the pixel 21, it is constant for each pixel 21 regardless of the environment, etc., and occurs in the same way for each exposure. Therefore, the magnitudes of the signals S0, S1, and S2 vary by a certain amount for each pixel 21. For example, when the end of exposure by the exposure control pulse A0, which should end at time t2, in a certain pixel 21 is a certain amount earlier than time t2, the signal S0 becomes smaller by a certain amount.
  • the distance calculated by the distance calculation unit 50 based on the above equation will shift by a certain amount.
  • the influence of fixed pattern noise of the solid-state image sensor 20 occurs as a distance offset. Further, this distance offset occurs by the same amount for each ranging operation.
  • the distance correction unit 60 performs correction using the distance correction amount for each pixel 21 based on the fixed pattern noise of the solid-state image sensor 20. Specifically, the distance correction unit 60 adds or subtracts the correction amount stored in the storage unit 80 to the distance calculated by the distance calculation unit 50 for each pixel 21. Thereby, the influence of fixed pattern noise on distance measurement accuracy can be reduced, and the quality of the output distance image can be improved. In particular, the influence of fixed pattern noise appears as a distance offset, so the shorter the distance to be measured, the greater the influence on distance measurement accuracy, and the effect of improving the quality of a distance image is greater for short distance measurement. Therefore, by applying the distance measuring device 100 to applications such as face recognition that require relatively short distances (for example, the measurement distance is 1 m or less), the accuracy of the applications can be improved.
  • the distance correction unit 60 uses the distance correction amount, the correction process can be simplified and the correction It is possible to reduce the number of correction tables, etc. for
  • FIG. 5 is a flowchart outlining the operation (correction amount generation method) of the correction amount generation section 70 of the distance measuring device 100 according to the embodiment.
  • the correction amount generation unit 70 acquires the distance for each pixel 21 calculated by the distance calculation unit 50 for a plurality of frames (step S21). That is, the correction amount generation unit 70 obtains distance images for a plurality of frames that have been calculated by the distance calculation unit 50 and have not been corrected.
  • the distance calculating section 50 calculates the distance for each pixel 21 from the distance measuring device 100 to an object placed at a predetermined distance, and outputs the distance to the correction amount generating section 70.
  • the predetermined distance is an arbitrary distance within the distance measurement range of the distance measurement device 100, and is, for example, the minimum value of the distance measurement range.
  • a series of light emission control pulses and exposure control pulses necessary for distance calculation by the distance calculation unit 50 are output.
  • step S11 to step S12 described above are repeated for the number of frames, and the distance calculating section 50 outputs the distance for each pixel 21 for the plurality of frames.
  • the correction amount generation unit 70 obtains the distance for each pixel 21 for a plurality of frames output from the distance calculation unit 50.
  • the number of frames for acquiring the distance for each pixel 21 is, for example, 100 frames or more and 2000 frames or less.
  • the correction amount generation unit 70 calculates a first representative value of distance for multiple frames for each pixel 21 (step S22).
  • the correction amount generation unit 70 calculates, for example, an average value of distances for a plurality of frames for each pixel 21. That is, the first representative value is, for example, an average value of distances for a plurality of frames, and can also be said to be a time average value of distances for each pixel 21.
  • shot noise that randomly occurs for each frame is averaged and canceled, and a distance image from which the influence of shot noise has been removed is obtained.
  • the first representative value may be another representative value such as the median or mode of distances for multiple frames.
  • the first representative value in order to remove extremely large or small distances, distances outside the predetermined range are removed from the calculation target of the representative value, and then the representative value is calculated. Good too.
  • the correction amount generation unit 70 calculates, for each pixel 21, a second representative value that is a representative value of the first representative values of the own pixel 21 and the surrounding pixels 21 (step S23).
  • the correction amount generation unit 70 calculates, for each pixel 21, a representative value of the first representative value of the pixel 21 in a range of 5 ⁇ 5 pixels or more and 25 ⁇ 25 pixels or less centered on the own pixel 21 as the second representative value. do.
  • the correction amount generation unit 70 calculates a second representative value for each pixel 21 by applying averaging filter processing to the first representative values of the own pixel 21 and the surrounding pixels 21. .
  • the second representative value is, for example, the average value of the first representative values of the own pixel 21 and the surrounding pixels 21, and can also be said to be the spatial average value of the first representative values for each pixel 21.
  • variations in distance for each pixel 21 due to fixed pattern noise are averaged, and a distance image is obtained in which the influence of fixed pattern noise in addition to shot noise is removed.
  • the second representative value may be another representative value such as the median or mode of the first representative values of the own pixel 21 and the surrounding pixels 21.
  • the second representative value in order to remove extremely large or small first representative values, first representative values outside a predetermined range are removed from the representative value calculation target, and then, A representative value may be calculated. Further, the second representative value may be a representative value of the first representative values of all pixels 21.
  • the correction amount generation unit 70 generates a distance correction amount for each pixel 21 based on the difference between the first representative value and the second representative value calculated for each pixel 21 (step S24).
  • the first representative value calculated as described above is a distance where the influence of shot noise is removed and the influence of fixed pattern noise remains.
  • the second representative value calculated as described above is a distance from which the influence of shot noise and fixed pattern noise has been removed. Therefore, by subtracting the second representative value from the first representative value for each pixel 21, the influence of fixed pattern noise can be extracted, and a distance correction amount for each pixel 21 based on the fixed pattern noise is generated.
  • the distance is calculated by the distance calculating section 50 to be longer than the actual distance. It is corrected by subtracting the value (absolute value).
  • the distance is calculated by the distance calculation unit 50 to be shorter than the actual distance, so the distance calculated by the distance calculation unit 50 is It is corrected by adding the value (absolute value) to
  • the correction amount generation unit 70 generates a distance correction amount for each pixel 21 based on the fixed pattern noise of the solid-state image sensor 20.
  • the correction amount generation method according to the present embodiment includes a generation step of generating a distance correction amount for each pixel 21 based on the fixed pattern noise of the solid-state image sensor 20.
  • step S25 is an example of an output step.
  • the correction amount generation unit 70 stores the position of the pixel 21 and the distance correction amount in association with each other for each pixel 21 in the storage unit 80 .
  • a correction table 81 is generated and stored in the storage section 80.
  • the processing of the above correction amount generation method may be performed in conjunction with the calibration processing of the distance measuring device 100.
  • parameters for calculation by the distance calculation unit 50 are adjusted based on a predetermined distance from the distance measuring device 100 to the target object and the distance calculated by the distance calculation unit 50.
  • correction amounts are generated for all pixels 21 included in the solid-state image sensor 20, but correction amounts are generated for some pixels 21 included in the solid-state image sensor 20. You can.
  • the distance measuring device 100 receives the light source 10 that emits irradiation light and the reflected light that is reflected by the target object of the irradiation light irradiated from the light source 10, and converts it into an electric charge.
  • a solid-state image sensor 20 that has a plurality of pixels 21 each including a photoelectric conversion element and is arranged two-dimensionally, and outputs a signal based on electric charge for each pixel 21;
  • a distance calculation unit 50 calculates the distance to the object for each pixel 21 by performing a predetermined calculation based on the signal output from the solid-state image sensor 20 at each time, and a distance calculation unit 50 calculates the distance to the target object for each pixel 21.
  • a storage unit 80 that stores the distance correction amount for each pixel 21, and a distance correction unit 60 that corrects the distance calculated by the distance calculation unit 50 for each pixel 21 using the correction amount stored in the storage unit 80. and.
  • the distance for each pixel 21 calculated by the distance calculation unit 50 is corrected by the distance correction amount based on the fixed pattern noise of the solid-state image sensor 20, so it is possible to reduce the variation in distance errors between pixels 21. . Therefore, unevenness and the like in the distance image are reduced, and the quality of the distance image can be improved. Furthermore, even when noise reduction processing is performed at a later stage to further improve the quality of the distance image, the weight of the noise reduction processing can be reduced because distance errors due to fixed pattern noise are reduced by correction. For example, since the size of a two-dimensional spatial filter used for noise reduction processing can be reduced, the scale of the processing circuit can be reduced, and the resolution of the processed distance image can be improved.
  • the distance measuring device 100 may include a correction amount generation unit 70 that generates a correction amount based on fixed pattern noise and stores the generated correction amount in the storage unit 80.
  • the distance measuring device 100 can generate the correction amount based on the fixed pattern noise by itself.
  • the correction amount generation unit 70 can generate the correction amount in accordance with the calibration process of the distance measuring device 100.
  • the correction amount generation unit 70 acquires the distance for each pixel 21 calculated by the distance calculation unit 50, and the distance from the distance measuring device 100 to the target object placed at a predetermined distance for a plurality of frames. Then, for each pixel 21, a first representative value of the distance for a plurality of frames is calculated, and for each pixel 21, a second representative value that is a representative value of the first representative values of itself and pixels around itself is calculated. , the correction amount may be generated based on the difference between the first representative value and the second representative value calculated for each pixel 21.
  • the distance that removes the influence of shot noise is calculated, and for each pixel 21, the distance that is the representative value of the first representative value of the spatial distance is calculated.
  • the distance that is the representative value of the first representative value of the spatial distance is calculated.
  • the first representative value may be an average value of distances for multiple frames.
  • the first representative value can be calculated with simple processing.
  • the second representative value may be the average value of the first representative values of the pixels 21 of itself and its surroundings.
  • the second representative value can be calculated with simple processing.
  • the second representative value may be the median value of the first representative values of the pixels 21 of itself and its surroundings.
  • the distance measuring device 100 may generate a light emission control pulse that instructs the light source 10 to emit the irradiation light with a predetermined pulse width, instructs the solid-state image sensor 20 to perform exposure, and starts using the light emission control pulse as a reference.
  • the solid-state image sensor 20 includes a drive control unit 30 that outputs a first exposure control pulse and a second exposure control pulse whose timings are different from each other, and the solid-state image sensor 20 can obtain exposure for each pixel 21 according to the first exposure control pulse.
  • the distance calculation unit 50 outputs a first signal based on the electric charge obtained by exposure according to the second exposure control pulse, and a second signal based on the electric charge obtained by exposure according to the second exposure control pulse. The distance may be calculated based on the two signals.
  • the distance correction unit 60 performs correction using the distance correction amount, so the first signal and the second signal are individually calculated. Distance can be corrected without any correction.
  • the distance measuring method is a distance measuring method using the distance measuring device 100, in which a predetermined calculation is performed for each pixel 21 based on the signal output from the solid-state image sensor 20, and a target A distance calculation step that calculates the distance to , and a distance that corrects the distance calculated in the distance calculation step for each pixel 21 using a distance correction amount for each pixel 21 based on the fixed pattern noise of the solid-state image sensor 20. a correction step.
  • the distance for each pixel 21 calculated in the distance calculation step is corrected by the distance correction amount based on the fixed pattern noise of the solid-state image sensor 20, variations in distance errors between pixels 21 can be reduced. Therefore, unevenness and the like in the distance image are reduced, and the quality of the distance image can be improved.
  • the correction amount generation method is a correction amount generation method for generating a correction amount for correcting the distance calculated by the distance measuring device 100 including the solid-state image sensor 20.
  • the method includes a generation step of generating a distance correction amount for each pixel 21 based on the 20 fixed pattern noises, and an output step of outputting the generated correction amount.
  • the distance measuring device 100 can reduce variations in distance errors between pixels 21 and improve the quality of distance images.
  • FIG. 6 is a functional block diagram showing an example of the configuration of a distance measuring device 100A according to this modification.
  • the distance measuring device 100A is different from the distance measuring device 100 according to the embodiment in that the storage unit 80 stores a plurality of correction tables 81, and further includes a temperature sensor 90. They differ in some respects.
  • the distance correction section 60 corrects the distance calculated by the distance calculation section 50 using the temperature measured by the temperature sensor 90 in addition to the correction amount based on fixed pattern noise.
  • the storage unit 80 stores a plurality of correction tables 81.
  • the plurality of correction tables 81 are, for example, tables in which the position of the pixel 21 and the distance correction amount are associated for each pixel 21, which are generated by the correction amount generation unit 70 under mutually different temperature conditions of the solid-state image sensor 20. It is.
  • the plurality of correction tables 81 are associated with, for example, a temperature range based on the temperature of the solid-state image sensor 20 when the correction table 81 is generated.
  • the correction table 81 associated with the temperature range to which the temperature of the solid-state image sensor 20 measured by the temperature sensor 90 corresponds is used by the distance correction unit 60 to correct the distance.
  • FIG. 7 is a diagram showing an example of the correspondence between temperature ranges and the correction table 81.
  • the distance measuring device 100A operates in a temperature range of -20° C. to 120° C.
  • the storage unit 80 stores four correction tables 81: A, B, C, and D. remembered.
  • Correction tables A, B, C, and D are each associated with temperature ranges that do not overlap with each other.
  • the temperature ranges associated with the correction tables A, B, C, and D cover the temperature range in which the distance measuring device 100A operates.
  • FIG. 7 is a diagram showing an example of the correspondence between temperature ranges and the correction table 81.
  • the correction tables A, B, C, and D are respectively -20°C or higher and lower than 20°C, 20°C or higher and lower than 60°C, 60°C or higher and lower than 100°C, and 100°C or higher and lower than 120°C. mapped to a temperature range.
  • the temperature range associated with the correction table 81 and the number of correction tables 81 are not particularly limited, and can be arbitrarily set according to the characteristics of the solid-state image sensor 20.
  • Each of the plurality of correction tables 81 is generated, for example, by performing the correction amount generation method described using FIG. 5 above in an environment where the temperature of the solid-state image sensor 20 is adjusted to a predetermined temperature. be done.
  • the predetermined temperature is, for example, the upper limit, lower limit, or median value of the temperature range associated with the correction table 81.
  • the temperature sensor 90 measures the temperature of the solid-state image sensor 20.
  • the temperature sensor 90 outputs the measured temperature of the solid-state image sensor 20 to the distance correction section 60.
  • the temperature sensor 90 may directly measure the temperature of the solid-state image sensor 20, or may measure the temperature of the solid-state image sensor 20 indirectly, such as by measuring the temperature around the solid-state image sensor 20.
  • FIG. 8 is a flowchart showing an overview of the operation (distance measurement method) of the distance measurement device 100A according to this modification.
  • step S11 and step S12 are performed in the same way as the operation of the ranging device 100 described using FIG. 3 above.
  • the distance correction unit 60 acquires the temperature of the solid-state image sensor 20 measured by the temperature sensor 90 (step S31). Then, the distance correction unit 60 uses the temperature of the solid-state image sensor 20 acquired in step S31 and the correction amount based on the fixed pattern noise of the solid-state image sensor 20 stored in the storage unit 80 to calculate the following for each pixel 21. The distance calculated by the distance calculating section 50 in step S12 is corrected (step S32).
  • the distance correction unit 60 first selects one correction table 81 used for distance correction from among the plurality of correction tables 81 stored in the storage unit 80 based on the obtained temperature of the solid-state image sensor 20. select. For example, the distance correction unit 60 selects one correction table 81 that is associated with a temperature range to which the obtained temperature of the solid-state image sensor 20 corresponds. For example, when the temperature of the solid-state image sensor 20 is 40°C and selection is made from correction tables A, B, C, and D shown in FIG. Select the attached correction table B. The distance correction unit 60 then refers to the selected one correction table 81 and corrects the distance calculated by the distance calculation unit 50 for each pixel 21.
  • fixed pattern noise is noise unique to each pixel 21 due to variations in switching operation and transfer.
  • the switching operation and transfer characteristics in such a pixel 21 can be affected by temperature.
  • the fixed pattern noise of the solid-state image sensor 20 can be affected by temperature.
  • the distance correction unit 60 corrects the distance calculated by the distance calculation unit 50 based on the temperature of the solid-state image sensor 20 and using a correction amount based on fixed pattern noise of the solid-state image sensor 20. Therefore, even if the temperature of the solid-state image sensor 20 changes, the distance correction unit 60 can correct the distance for each pixel 21 by taking into account the influence of the temperature of the solid-state image sensor 20. Therefore, variations in distance errors between pixels 21 can be further reduced.
  • the influence of temperature on the switching operation and transfer characteristics in the pixel 21 can vary from pixel to pixel 21. That is, the effect of temperature on fixed pattern noise can vary from pixel to pixel 21.
  • the distance correction unit 60 selects one correction table 81 to be used for distance correction from the plurality of correction tables 81 as described above, so that each pixel 21 corresponds to the temperature of the solid-state image sensor 20. The distance for each pixel 21 can be corrected using the correction amount. Furthermore, since the amount of correction can be determined by referring to the correction table 81, the processing by the distance correction section 60 can also be reduced.
  • FIG. 9 is a functional block diagram showing an example of the configuration of another distance measuring device 100B according to this modification. As shown in FIG.
  • the storage unit 80 stores one correction table 81 generated in an environment where the temperature of the solid-state image sensor 20 is adjusted to a predetermined temperature, and Change amount information 82 indicating the amount of change in the correction amount with respect to the temperature of the image sensor 20 is stored.
  • the change amount information 82 is, for example, a change amount table in which temperature and change amount are associated with each other, but it may also be a function indicating the relationship between temperature and change amount.
  • the change amount information 82 is calculated using, for example, a correction amount obtained by performing the correction amount generation method described using FIG. 5 above under several temperature conditions of the solid-state image sensor 20.
  • the distance correction unit 60 calculates the amount of change based on the obtained temperature of the solid-state image sensor 20 using the amount of change information 82. Then, the distance correction unit 60 adds or subtracts the change amount to the correction amount for each pixel 21, and corrects the distance calculated by the distance calculation unit 50 using the correction amount after the addition or subtraction. Thereby, the number of correction tables 81 stored in the storage section 80 can be reduced, so the capacity of the storage section 80 can be reduced. Furthermore, by using the change amount information 82, the correction amount can be changed in response to small changes in the temperature of the solid-state image sensor 20, so that sudden changes in the correction amount can be avoided.
  • the change amount information 82 is, for example, commonly used by all pixels 21, but a plurality of change amount information 82 that is commonly used by pixels 21 having similar relationships between temperature and change amount among all pixels 21 is stored.
  • the information may be stored in the section 80.
  • the distance measuring device does not need to include all the components described in the above embodiments, and may be configured only with components for performing the desired operation.
  • each component may be realized by executing a software program suitable for each component.
  • Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
  • each component may be realized by hardware.
  • Each component may be a circuit (or integrated circuit). These circuits may constitute one circuit as a whole, or may be separate circuits. Further, each of these circuits may be a general-purpose circuit or a dedicated circuit.
  • general or specific aspects of the present disclosure may be implemented in a system, apparatus, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM. Further, the present invention may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.
  • the present disclosure may be realized as the distance measuring device of the above embodiment, or may be realized as a control device that controls the distance measuring device, or the steps ( It may be realized as a distance measurement method including processing), or it may be realized as a program for causing a computer to execute such a distance measurement method. It may also be realized as a temporary recording medium. Further, the present disclosure may be realized as a correction amount generation device including a correction amount generation unit, or as a correction amount generation method including steps (processing) performed by components constituting the correction amount generation device. Alternatively, it may be realized as a program for causing a computer to execute such a correction amount generation method, or it may be realized as a computer-readable non-temporary recording medium in which such a program is recorded. .
  • the distance measuring device and the like according to the present disclosure can be applied to various uses such as a distance measuring system and a sensing system and recognition system using distance images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Measurement Of Optical Distance (AREA)

Abstract

Le présent dispositif de télémétrie (100) comprend : une source de lumière (10) qui émet une lumière d'émission ; un élément d'imagerie à semi-conducteurs (20) ayant une pluralité de pixels qui sont agencés de manière bidimensionnelle et qui comprennent respectivement un élément de conversion photoélectrique pour recevoir la lumière réfléchie, produite par la lumière d'émission émise par la source de lumière (10) réfléchie par un objet, et convertir la lumière réfléchie reçue en charge électrique, l'élément d'imagerie à semi-conducteurs (20) délivrant, pour chacun des pixels, un signal sur la base de la charge ; une unité de calcul de distance (50) pour effectuer un calcul prescrit sur la base du signal émis par l'élément d'imagerie à semi-conducteurs (20) pour chacun des pixels et ainsi calculer la distance à l'objet pour chacun des pixels ; une unité de stockage (80) pour stocker une quantité corrective pour la distance pour chacun des pixels, sur la base d'un bruit de motif fixe de l'élément d'imagerie à semi-conducteurs (20) ; et une unité de correction de distance (60) pour corriger, pour chacun des pixels, la distance calculée par l'unité de calcul de distance (50), à l'aide des quantités correctives stockées dans l'unité de stockage (80).
PCT/JP2023/025129 2022-07-15 2023-07-06 Dispositif de télémétrie, dispositif de génération de quantité de correction, procédé de télémétrie et procédé de génération de quantité de correction WO2024014393A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-113912 2022-07-15
JP2022113912 2022-07-15

Publications (1)

Publication Number Publication Date
WO2024014393A1 true WO2024014393A1 (fr) 2024-01-18

Family

ID=89536635

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/025129 WO2024014393A1 (fr) 2022-07-15 2023-07-06 Dispositif de télémétrie, dispositif de génération de quantité de correction, procédé de télémétrie et procédé de génération de quantité de correction

Country Status (1)

Country Link
WO (1) WO2024014393A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015177528A (ja) * 2014-03-18 2015-10-05 キヤノン株式会社 画像処理装置、画像処理方法、及びプログラム
US20200074608A1 (en) * 2018-09-05 2020-03-05 Infineon Technologies Ag Time of Flight Camera and Method for Calibrating a Time of Flight Camera
CN111624580A (zh) * 2020-05-07 2020-09-04 Oppo广东移动通信有限公司 飞行时间模组的校正方法、校正装置和校正系统
WO2021251152A1 (fr) * 2020-06-10 2021-12-16 ソニーセミコンダクタソリューションズ株式会社 Dispositif de réception de lumière, son procédé de fabrication et dispositif de télémétrie

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015177528A (ja) * 2014-03-18 2015-10-05 キヤノン株式会社 画像処理装置、画像処理方法、及びプログラム
US20200074608A1 (en) * 2018-09-05 2020-03-05 Infineon Technologies Ag Time of Flight Camera and Method for Calibrating a Time of Flight Camera
CN111624580A (zh) * 2020-05-07 2020-09-04 Oppo广东移动通信有限公司 飞行时间模组的校正方法、校正装置和校正系统
WO2021251152A1 (fr) * 2020-06-10 2021-12-16 ソニーセミコンダクタソリューションズ株式会社 Dispositif de réception de lumière, son procédé de fabrication et dispositif de télémétrie

Similar Documents

Publication Publication Date Title
KR100946536B1 (ko) 광학적으로 검은 픽셀 및 결합된 픽셀을 모두 사용하여이미저에서 블랙 레벨을 설정하는 방법 및 장치
WO2017006546A1 (fr) Dispositif de mesure de distance et procédé de combinaison d'image de distance
JP5934128B2 (ja) 放射線撮像装置及び放射線撮像システム
JP2019091969A (ja) 放射線撮像装置
JP6738334B2 (ja) 固体撮像装置
JP2011257402A (ja) 飛行時間撮像装置
JPWO2017061104A1 (ja) 撮像装置、およびそれに用いられる固体撮像素子
US9247163B2 (en) Radiation image pickup system, computer having reset operation to reset electric charge of pixels caused by dark current, and computer-readable medium storing program therefor
WO2014207992A1 (fr) Dispositif de mesure de distance et d'imagerie et procédé de mesure associé
US7869007B2 (en) Ranging apparatus and ranging method
JP6773724B2 (ja) 精度情報を出力する測距装置
US9894300B2 (en) Image sensing device for measuring temperature without temperature sensor and method for driving the same
WO2013099348A1 (fr) Dispositif de mesure de distance
WO2016133053A1 (fr) Appareil de mesure d'image télémétrique
JP2012151664A (ja) 固体撮像装置
JP2012225807A (ja) 距離画像カメラおよび距離画像合成方法
JP6362511B2 (ja) 撮像装置及びその制御方法
JP2017158062A5 (fr)
JP6190690B2 (ja) 距離測定システム及び補正用データの取得方法
WO2024014393A1 (fr) Dispositif de télémétrie, dispositif de génération de quantité de correction, procédé de télémétrie et procédé de génération de quantité de correction
WO2021059748A1 (fr) Dispositif de traitement d'informations, procédé de correction, et programme
US20210074743A1 (en) Light detection apparatus and electronic device
WO2023139916A1 (fr) Dispositif de mesure
JP2008241258A (ja) 測距装置及び測距方法
KR20110005776A (ko) 별 추적기용 능동 픽셀 센서 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23839565

Country of ref document: EP

Kind code of ref document: A1