KR20230036978A - FLASH LiDAR SENSOR USING ZOOM HISTOGRAMMING TIME-TO-DIGITAL CONVERTER - Google Patents

Abstract

The present invention relates to a flash LiDAR sensor using a zoom histogramming time-to-digital converter (TDC). An aspect of the present invention relates to a LiDAR sensor including a plurality of pixels, wherein each of the plurality of pixels includes a TDC using a histogram method, the TDC performs a histogramming operation using M/2 number of up-down counters (UDC), the TDC divides a section to be measured into M number of time bins, and each of the M number of time bins is allocated to correspond to either an up count or a down count of the M/2 number of UDCs. Therefore, provided is a LiDAR sensor capable of improving the accuracy of distance (depth) information by reducing the influence of background light.

Description

Flash lidar sensor using zoom histogramming TDC {FLASH LiDAR SENSOR USING ZOOM HISTOGRAMMING TIME-TO-DIGITAL CONVERTER}

The present invention relates to a LiDAR sensor. Specifically, the present invention relates to a flash LiDAR sensor using a zoom histogramming time-to-digital converter (TDC).

Recently, as metaverse applications or unmanned vehicles running on mobile devices are attracting attention, the demand for depth image acquisition is increasing. A light detection and ranging sensor (LiDAR sensor) based on time of flight (ToF) technology is one of the promising candidates for real-time distance measurement. The ToF technique can be mainly classified into direct ToF (dToF) and indirect ToF (iToF) depending on whether depth map extraction information is a time difference of laser pulses or a modulated phase difference between irradiation light and reflected light. iToF sensors are useful for providing high spatial resolution and low depth error, but their maximum distance is usually limited by their low optical gain and the period of the modulating waveform. In addition, the iToF sensor is vulnerable to background light, which has disadvantages in outdoor environments. In contrast, dToF sensors using highly sensitive single photon avalanche diodes (SPADs) can extend their detectable range to hundreds of meters. In addition, a Time-to-Digital Converter (TDC) using a Time-Correlated Single Photon Counting (TCSPC) method with a Coincidence Detection Circuit (CDC) Effective attenuation of background light makes the dToF sensor suitable for outdoor use.

In view of eye safety considerations and limited light power density, a single point or 1-D line type emitter that increases the maximum distance by concentrating light of limited power into a small area may be desirable. In this case, a rotating mirror for a 1-D lidar module with both emitter and detector is required to scan surrounding objects using a small field of view (FoV). However, a scanning structure for supporting a delicate optical system that requires high durability even in a harsh environment has a problem in that it is not only large in size but also difficult to implement at low cost.

In mobile applications, even if the detection range is less than several tens of meters, small-sized and cost-effective lidar sensors are suitable, so full-flash type lidar sensors that do not use rotating equipment are currently being intensively developed. They are attempting to reduce pixel size and increase spatial resolution by adopting a 3D stacking process and sharing a time-to-digital converter (TDC), but there are still many challenges to overcome.

One of the most important issues hindering pixel shrinkage is the large memory capacity that stores tens of GS/s of ToF data proportional to the size of the array. The TCSPC method needs to draw a histogram of the collected ToF values, so it needs to record all the data generated by the pixel array for every laser shot. Existing lidar sensors used a method of integrating a histogram function into a TDC called histogram TDC (hTDC) and designating a counter for every time bin. Since the number of counters increases correspondingly, there is a problem that the counter occupies a wide area.

Another problem with lidar sensors is noise caused by background light. The lidar sensor needs to separate and detect the light reflected from the subject from the background light. However, when the background light is strong, it is not easy to separate the reflected light from the background light and detect it.

One object of the present invention is to provide a lidar sensor capable of increasing the accuracy of distance (depth) information by reducing the influence of background light.

One object of the present invention is to provide a lidar sensor capable of reducing memory requirements.

One object of the present invention is to provide a lidar sensor capable of effectively increasing the frame rate.

One object of the present invention is to provide a lidar sensor capable of processing a high-speed photodetection signal without using a counter having a high bandwidth.

One aspect of the present invention for achieving the above object is a lidar sensor including a plurality of pixels, each of the plurality of pixels including a histogram-type time-to-digital converter (TDC), The time-to-digital converter (TDC) performs a histogram operation using M/2 up-down counters (UDC), and the time-to-digital converter (TDC) divides the measurement target section into M time bins. Divided by , and each of the M time bins is a lidar sensor, characterized in that allocated to correspond to one of the up count and down count of the M / 2 up-down counters (UDC).

In the lidar sensor, the M time bins include a time bin corresponding to an up count and a time bin corresponding to a down count of substantially the same length for each of the M/2 up-down counters (UDC) Therefore, when the noise pulses caused by the background light are uniformly distributed over the M time bins, each of the M/2 up-down counters UDC has substantially the same number of up counts and down counts due to the noise pulses. This can be done to cancel out the effect of background light.

In the lidar sensor, the time-to-digital converter (TDC) includes a first mode (coarse mode) for performing a plurality of steps, and each time the plurality of steps are performed, the measurement target section moves to the previous step. Compared to , it can be reduced to 1/M.

In the lidar sensor, among the plurality of steps, the M time bins of the n-th step are generated by dividing the time bins having the highest pulse intensity in the n-1th step into M again, and the n-th step time bins. may be substantially reduced in length to 1/M compared to the time bin of the n-1th step.

In the lidar sensor, a code corresponding to a time bin having the highest pulse intensity at each step of a plurality of steps of the first mode may be determined as a partial code of the first mode time-of-flight (ToF_coarse).

In the lidar sensor, the first mode (coarse mode) includes the first to third steps, the M is 4, four time bins are generated for each step, and the four time bins are sequentially Codes of 00(2), 01(2), 10(2) and 11(2) are allocated, and the code of the time bin having the strongest pulse intensity in the first step (step C1) is the first mode flight time It is determined by the upper two bits of (ToF_coarse), and the code of the time bin having the strongest pulse intensity in the second step (step C2) is determined by the middle two bits of the first mode time-of-flight (ToF_coarse), and the third In step C3, the code of the time bin having the strongest pulse intensity is determined as the lower two bits of the first mode flight time (ToF_coarse), and the first mode flight of 6 bits through the first to third steps. Time (ToF_coarse) may be determined.

In the lidar sensor, the time-to-digital converter (TDC) further includes a second mode (fine mode), and the second mode (fine mode) is generated at the last stage of the first mode (coarse mode). M time bins having a length smaller than the length of the specified time bin are generated, and a predetermined phase may be assigned to each of the M time bins of the second mode (fine mode).

In the lidar sensor, M = 4, the up-down counters (UDC) are two, and the count values of the two up-down counters (UDC) determine the ToF (ToF_fine) value of the second mode can be used to

In the lidar sensor, in the first mode, the up-down counter (UDC) counts a signal generated through a simultaneous occurrence detection circuit (CDC), and in the second mode, the up-down counter (UDC) can count signals that have not passed through the simultaneous detection circuit (CDC).

In the lidar sensor, the up-down counter UDC may be an asynchronous/synchronous mixed counter.

In the lidar sensor, each of the up-down counters (UDC) includes a plurality of flip-flops, and at least some of the plurality of flip-flops receive signals different from those of the other flip-flops as a clock input terminal. can be entered as

In the lidar sensor, a signal (SiPM) generated from a photodetection signal is applied to a clock input terminal of one flip-flop among the plurality of flip-flops, and a SiPM signal is applied to a clock input terminal of other flip-flops. A signal generated from an output signal of the flip-flop may be applied.

In the lidar sensor, the flip-flop to which the SiPM signal is applied to the clock input terminal may be a flip-flop that processes a least significant bit (LSB).

According to the lidar sensor of the present invention, it is possible to increase the accuracy of distance (depth) information by reducing the influence of background light.

According to the lidar sensor of the present invention, memory requirements can be reduced.

According to the lidar sensor of the present invention, the frame rate can be effectively increased.

According to the lidar sensor of the present invention, a high-speed photodetection signal can be processed without using a counter having a high bandwidth.

Figure 1 illustrates the conceptual operating principle of a DIQS hTDC using two up-down counters (UDC) as an embodiment of the present invention.
2 exemplarily illustrates a principle of suppressing noise caused by background light using an up-down counter.
3 illustrates an In-Pixel Time-to-Digital Converter (TDC) according to one embodiment.
4 exemplarily illustrates the operation of the coincidence detection circuit CDC.
FIG. 5 is an enlarged view of a counting operation part by an up-down counter in FIG. 3 .
6 illustrates an up-down counter according to one embodiment of the present invention.
7 and 8 illustrate operating waveforms of the up-down counter illustrated in FIG. 6 .

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present invention. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is directly connected or connectable to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

1 shows the conceptual working principle of a delta-intensity quaternary search (DIQS) histogramming time-to-digital converter (hTDC) using two up-down counters (UDCs). Here, DIQS hTDC means a histogram TDC method according to an embodiment of the present invention.

In the first step (STEP_C1) of the first mode (coarse mode), the DIQS hTDC substantially equally divides the entire period corresponding to the maximum detectable range into a plurality of time bins (UP_A, UP_B, DN_A, DN_B), and pulses occurs, it is possible to determine the time bin in which the light reflected by the target object is located by accumulating pulse counts in each time bin. 1 illustrates a case in which time bins are divided into four (UP_A, UP_B, DN_A, and DN_B), but the number of time bins is not limited to four. Hereinafter, a 'time bin' is sometimes simply referred to as a 'bin'.

For example, dividing a bin into 4 can be identified as: Among the four bins, the first pair consisting of UP_A and UP_B and the second pair consisting of DN_A and DN_B can be classified according to whether the counting operation of the up-down counter (UDC) increases or decreases. That is, the first pair of UP_A bins and UP_B bins may correspond to an up count, and the second pair of DN_A bins and DN_B bins may correspond to a down count. Odd-numbered bins UP_A and DN_A and even-numbered bins UP_B and DN_B may belong to different up-down counters UDC. For example, odd-numbered bins UP_A and DN_A may correspond to the first up-down counter UDC_A, and even-numbered bins UP_B and DN_B may correspond to the second up-down counter UDC_B. . In this way, four bins composed of UP_A, UP_B, DN_A, and DN_B are generated, and each can correspond to 2-bit digital ToF codes 00(2), 01(2), 10(2), and 11(2) in order. (In 00(2), etc., the number 2 in parentheses means a binary number). The last letters A and B of the names of the bins (UP_A, UP_B, DN_A, DN_B) indicate the corresponding up-down counters (UDC).

When a pulse is generated by detecting incident light by a photodiode (eg, SPAD), the value of the up-down counter (UDC) may increase or decrease according to a time bin to which the pulse belongs. For example, when a pulse is generated in the section of the UP_A bin, the first up-down counter UDC_A is incremented, and if a pulse is generated in the section of the UP_B bin, the second up-down counter UDC_B is incremented, and When a pulse is generated in the section of the DN_B bin, the first up-down counter UDC_A is decremented, and when the pulse is generated in the section of the DN_B bin, the second up-down counter UDC_B is decremented.

After repeating the pulse count, the absolute values of the two up-down counters (UDCs) are compared, and a ToF code can be assigned depending on which value is greater than the other value and what its sign is. A and B, which are the IDs of up-down counters (UDC) with large absolute values, can determine the least significant bit (LSB) of the 2-bit ToF_coarse code as 0 and 1, respectively (the final ToF_coarse is generated by 2 bits at each step, Steps (STEP_C1, C2, C3) can be 6 bits). The most significant bit (MSB) of the 2-bit ToF_coarse code is set to the sign of an up-down counter (UDC) value having a large absolute value, and positive and negative values can be mapped to 0 and 1, respectively. For example, if the value of UDC_A is -5 and the value of UDC_B is -3, LSB = 0 because the absolute value of UDC_A is large and MSB = 1 because the value of UDC_A is negative, so the ToF code is 10(2 ) can be set.

Referring to the example of FIG. 1 , since a pulse generated by light incident on a photodiode (SPAD) (SiPM_E; SiPM_E will be described in detail below) is located in the DN_A bin in STEP_C1, UDC_A counts down to negative It has a value of , and UDC_B has a value of 0 because there is no increase or decrease. Therefore, ToF_coarse<5:4> (the first mode (coarse mode) generates a 2-bit ToF_coarse code at each step and has a 6-bit value through three steps (STEP_C1, C2, C3). STEP_C1 is the first step Since the upper 2 bits are determined among the 6-bit ToF_coarse codes, ToF_coarse<5:4>) can be set to 10(2). This means that the detected light belongs to the DN_A bin.

In the second step (STEP_C2), the section of the bin (DN_A in the example of FIG. 1) judged to belong to the pulse in the first step (STEP_C1) (or to have the strongest pulse intensity when there are multiple pulses) is divided into four bins again. It can be divided and allocated. That is, the section of the DN_A bin of the first step (STEP_C1) is divided again using four bins with a section length of 1/4 compared to that of the first step (STEP_C1), and UP_A, UP_B, DN_A, and DN_B are sequentially allocated. can

In the second step (STEP_C2), the ToF_coarse<3:2> value may be determined according to a procedure similar to that of the first step (STEP_C1). Referring to the example of FIG. 1, since the SiPM_E pulse belongs to the UP_A bin in the second step (STEP_C2) (it is assumed that the position of the pulse is the same as that of the first step), 00(2) is assigned to ToF_coarse<3:2>. can In FIG. 1, one SiPM_E pulse is exemplarily shown for convenience of explanation, but in general, the position of the pulse is repeatedly determined several times. In this case, as described above, the first up-down counter (UDC_A) and The code can be determined through comparison of the magnitudes of the absolute values of the two up-down counters (UDC_B) and their signs, which is substantially the same as the principle of determining the code when there is only one SiPM_E pulse.

In the third step (STEP_C3), the section of the bin (the UP_A bin in the example of FIG. 1) determined to belong to the pulse in the second step (STEP_C2) may be divided into four bins and allocated. That is, the section of the UP_A bin of the second step (STEP_C2) is divided again using four bins with a section length of 1/4 compared to that of the second step (STEP_C2), and UP_A, UP_B, DN_A, and DN_B are sequentially allocated. can

In the third step (STEP_C3), the ToF_coarse<1:0> value may be determined according to a procedure similar to the previous step. Referring to the example of FIG. 1 , since the SiPM_E pulse belongs to the UP_B bin in the third step (STEP_C3), 01(2) may be assigned to ToF_coarse<1:0>.

After going through the first, second, and third steps (STEP_C1, C2, and C3), all 6-bit ToF_coarse<5:0> values can be determined. In the example of FIG. 1, ToF_coarse<5:0> has a value of 100001(2).

After going through the first, second, and third steps (STEP_C1, C2, and C3) of the first mode (coarse mode), the pulse is located in a time range reduced by (1/4) ³ in the entire period corresponding to the maximum detectable range. The range may be narrowed. For example, in a sensor with a maximum detection distance of 48 m, when the first, second, and third steps (STEP_C1, C2, and C3) of the first mode (coarse mode) are performed, the time length of the bin in the third step (STEP_C3) is about several ns. It may be reduced to a degree similar to the width of the SiPM_E pulse, and may proceed to the second mode (fine mode). In this way, according to the present embodiment, each time a step is performed in the first mode (coarse mode), the measurement target section is reduced by a predetermined multiple compared to the previous step, which can be understood as a zoom-type histogram TDC.

The second mode (fine mode) may be used to achieve higher resolution after extracting depth information based on the dToF method in the first mode (coarse mode). For example, after extracting the ToF_coarse<5:0> value through the first mode (coarse mode), more precise depth information can be extracted using the phase detection technique based on the iTOF method in the second mode (fine mode). .

Utilization of four bins in the second mode (fine mode) may be the same as in the first mode (coarse mode). In the second mode (fine mode), depth information may be extracted from the phase difference by acquiring the intensity of each bin in four bins. The four bins used in the second mode (fine mode) may be configured in a similar manner as in the first mode (coarse mode). The four bins (UP_A, UP_B, DN_A, and DN_B) may indicate phase delays of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively.

Illustratively, referring to FIG. 1, SiPM_L, which is a pulse counted in the fourth step (STEP_F) of the second mode (fine mode), is a simultaneous detection circuit compared to the pulse SiPM_E used in the first mode (coarse mode). It may be a signal that has not passed through (CDC) (the SiPM_L pulse will be described in detail below). As illustrated in FIG. 1 , SiPM_L pulses may occur across multiple bins. The first and second up-down counters UDC_A and UDC_B may perform an up or down count according to a corresponding bin whenever the SiPM_L pulse signal is generated. In the example of FIG. 1, the first up-down counter (UDC_A) will have a positive count value because the intensity (pulse generation frequency) of the UP_A bin is stronger than that of the DN_A bin, and the intensity of the UP_B bin is strong, whereas the intensity of the DN_B Since there is no pulse generation in the bin, the second up-down counter UDC_B will also have a positive count value. Also, since the intensity of the UP_B bin is stronger than that of the UP_A bin, the second up-down counter UDC_B will have a greater positive value than the first up-down counter UDC_A. In this way, when the count for SiPM_L, which is a pulse generated by light incident on the SPAD, is finished, the value of the first up-down counter (UDC_A) is output as 9-bit A<8:0>, and the second up-down counter (UDC_A) is output as A<8:0>. The value of the down counter (UDC_B) can be output as 9-bit B<8:0> (Figure 1 illustrates the case of using a 9-bit up-down counter, but the number of bits of the up-down counter is different values may be used). ToF_Fine can be calculated using the count value of the first up-down counter UDC_A and the count value of the second up-down counter UDC_B obtained through the second mode (fine mode) as shown in Equation 1 below. .

[Equation 1]

Here, c is the speed of light (km/s), and T_LSB is the length of one time bin at the last step (STEP_C3) of the first mode (coarse mode).

In the second mode (fine mode), high-resolution depth information can be obtained by using four phase data acquired through four bins (UP_A, UP_B, DN_A, and DN_B).

In this embodiment, noise caused by background light can be suppressed by using up-down counters in the first mode (coarse mode) and the second mode (fine mode). The principle will be described with reference to FIG. 2 .

FIG. 2 illustrates a case in which a signal to background ratio (SBR) is a background light (BGL) intensity of about 1/100. Here, it is assumed that the background light is uniformly distributed over the entire section. In the case of using the existing up counter, the reflected light signal (signal) is generated in the UP_B bin. There is a problem in that it is not easy to accurately detect a bin where a reflected light signal is located because the values of the two counters reach saturation in such a short period of time that the count values of the B counter are difficult to distinguish.

In this embodiment, since the difference value is counted using an up-down counter, the influence of the background light (BGL) can be reduced compared to the conventional method using an up-counter. In the example of FIG. 2 , it is assumed that the reflected light signal is generated in the DN_A bin. When the background light (BGL) is uniformly distributed, the intensity of the background light (BGL) in the UP_A bin and the DN_A bin is almost the same, so the up count in the UP_A bin and the down count in the DN_A bin cancel each other, resulting in a background light ( BGL) does not saturate UDC_A. Therefore, the reflected light signal can be accumulated to a sufficient size in UDC_A by repeated execution over several times. In the case of UDC_B, since the intensity of the background light (BGL) in the UP_B bin and the DN_B bin is almost the same, the up count in the UP_B bin and the down count in the DN_B bin cancel each other, so the count value of UDC_B remains substantially zero. Therefore, it is possible to detect repeated signals in a non-saturated state.

As described with reference to FIGS. 1 and 2 , the DIQS hTDC method of the present embodiment can reduce the effect of background light and reduce memory requirements by using a histogram method through an up-down counter. In addition, the DIQS hTDC method of this embodiment uses a zoom method that reduces the search period by 1/4 for each step of the first mode (coarse mode), thereby reducing the number of subframes to increase the frame rate and reduce memory requirements. can reduce When the search period is reduced by 1/4 at each step, the background light intensity is reduced by 1/4 while maintaining the signal strength, so the signal-to-background ratio (SBR) also increases by 4 times at each step, so the effect of the background light is rapidly reduced. It can be.

In FIGS. 1 and 2, it is illustrated that there are two up-down counters (UDC), four bins are generated for each step, and the first mode (coarse) is performed in three steps. It is only an example for convenience, and the present embodiment is not limited to these numerical values.

Expanding by applying the principle described with reference to FIGS. 1 and 2, the time-to-digital converter (TDC) performs a histogram operation using M/2 up-down counters (UDC), but the time-to-digital converter ( TDC) divides the measurement target section into M time bins, and each of the M time bins is allocated to correspond to one of the up count and down count of M/2 up-down counters (UDC). can

Here, the M time bins include time bins corresponding to the up count and time bins corresponding to the down count with substantially the same length for each of the M/2 up-down counters (UDCs), so that noise caused by background light is reduced. When the pulses are uniformly distributed over M time bins, each of the M/2 up-down counters (UDCs) performs substantially the same number of up-counts and down-counts by the noise pulses, canceling the effect of background light. It can be.

In addition, the time-to-digital converter (TDC) includes a first mode (coarse mode) that performs a plurality of steps, and each time a plurality of steps are performed, the measurement target section is reduced to 1/M compared to the previous step. It can be. Among the plurality of steps, M time bins of the n-th step are generated by dividing the time bins having the highest pulse intensity in the n-1th step into M again, and the time bins of the n-th step are generated by dividing the time bins having the highest pulse intensity in the n-1th step. Compared to bins, the length can be substantially reduced to 1/M. For each step of the plurality of steps of the first mode, a code corresponding to a time bin having the highest pulse intensity may be determined as a partial code of the first mode time-of-flight (ToF_coarse).

In addition, the time-to-digital converter (TDC) further includes a second mode (fine mode), and in the second mode (fine mode), a length smaller than the length of the time bin generated in the last step of the first mode (coarse mode) M time bins having n are generated, and a predetermined phase may be assigned to each of the M time bins of the second mode (fine mode). In the second mode (fine mode), four time bins are preferably generated for phase difference calculation.

3 shows a lidar sensor 100 according to one embodiment. The lidar sensor 100 includes a clock tree 110, a row decoder 120, a column decoder/multiplexer 130, a bias circuit 140, a phase locked loop (PLL) circuit 150, and a number of may include pixels 200 .

The row decoder 120 may sequentially drive data of pixels of each row to a column data bus (not shown) in order to read out data.

The column decoder/multiplexer 130 may sequentially connect data driven to the column data bus to output pads (not shown).

The bias circuit 140 may generate a bias voltage to be used by an in-pixel circuit inside the photodiode SPAD.

The PLL circuit 150 may generate a clock to be used by an in-pixel circuit. For example, the PLL circuit 150 may receive a 50 MHz reference clock from the outside, generate a 400 MHz master clock, and provide the generated master clock to the clock tree 110 .

The clock tree 110 may generate multiple clocks for use by in-pixel circuits. For example, the clock tree 110 divides the master clock of 400 MHz received from the PLL circuit 150 7 times by 1/2 to generate 8 clocks consisting of 400 MHz, 200 MHz, 100 MHz, ... 3.125 MHz, clock repeater 260.

Here, a detailed description of the clock tree 110, row decoder 120, column decoder/multiplexer 130, bias circuit 140, and phase locked loop (PLL) circuit 150 may obscure the subject matter of the present invention. Therefore, a detailed description will be omitted.

A plurality of pixels 200 may be arranged in a matrix form, for example, within the lidar sensor 100 . Illustratively, some of the plurality of pixels 200 may share the clock repeater 260 . 3 illustrates that four pixels 200 share one clock repeater 260 .

The clock repeater 260 may provide clocks of various frequencies provided from the clock tree 110 to corresponding pixels 200 . For example, the clock repeater 260 may transfer 8 clocks provided from the clock tree 110 to the pixel 200 . The TDC implemented in the pixel 200 may perform the DIQS hTDC process described with reference to FIG. 1 using a plurality of clocks provided from the clock repeater 260 .

Each pixel 200 may include a plurality of photodiodes 211 and a TDC. In FIG. 3, a SPAD is illustrated as a photodiode. The SPAD is the most desirable device to be used in this embodiment because it can detect light with high sensitivity, but the photodiode of this embodiment is not limited to the SPAD and other types of photodiodes may be used. In addition, FIG. 3 illustrates a case where each pixel 200 includes 6 SPADs 211, but is not limited thereto.

In the embodiment of FIG. 3 , the TDC may be an in-pixel method implemented in each pixel 200 . Below, the operation of the TDC implemented in each pixel will be described as an example. Here, the function of the TDC can be schematically understood as extracting time information from a pulse signal generated by light detection of the SPAD and converting it into a digital value. Time information generated by the TDC may be used to generate a depth map.

An output signal of the SPAD 211 may be output through a single stabilization circuit 212, an XOR gate 215 and a mask circuit 214, and the like. A single stabilization circuit 212, an XOR gate 215, and a mask circuit 214 may be provided to correspond to each SPAD 211.

The single stabilization circuit 212 may convert the pulse output from the SPAD 211 into a signal having a predetermined pulse width and output the converted signal. Since the single stabilization circuit 212 is provided for each SPAD 211, the output pulses of the plurality of SPADs 211 can be changed to have the same pulse width.

The XOR gate 215 and the mask circuit 214 may determine when to output a signal received from the single stabilization circuit 212 . For example, the outputs of the six SPADs 211 may be output to the OR gate 221 at a point in time designated by the corresponding mask circuit 214 . Signals output from the plurality of SPADs 211 may be converted into serial signals continuously arranged through the OR gate 221 . That is, the output of the OR gate 221 may be a signal in which pulses having substantially the same pulse width generated corresponding to the output pulses of the six SPADs 211 are continuously arranged.

The single stabilization circuit 212, the XOR gate 215, the mask circuit 214, and the OR gate 221 illustrated in FIG. 3 convert the output signals of the plurality of SPADs 211 into serial signals and transmit them to the next stage. It is only an example of, and the present embodiment is not limited to this structure.

The output of the OR gate 221 may itself become the SiPM_L signal, and the signal generated by passing the SiPM_L signal through the CDC 222 may become the SiPM_E signal. A method of generating the SiPM_E signal from the SiPM_L signal using the simultaneous occurrence detection circuit (CDC) 222 will be described with reference to FIG. 4 .

In the example of FIG. 4 , when a falling edge occurs in the SiPM_L signal (401), the WIN_CDC signal may be switched from a 'low' state to a 'high' state. The time for which the WIN_CDC signal maintains a 'high' state (t_CDC) may be a preset time (eg, 5 ns). If an additional falling edge does not occur in the SiPM_L signal within the time (t_CDC) during which the WIN_CDC signal maintains the 'high' state, the SiPM_E signal is not generated and the WIN_CDC signal may return to the 'low' state. When another falling edge occurs on the SiPM_L signal (402), the WIN_CDC signal switches from the 'low' state to the 'high' state again, and within the time the WIN_CDC signal maintains the 'high' state, an additional falling edge on the SiPM_L signal occurs. When it occurs (403), the SiPM_E signal is switched from the 'high' state to the 'low' state, and the SiPM_E signal 404 may be generated. Although the present embodiment illustrates that the SiPM_E signal is generated when the SiPM_L signal is detected twice or more while the WIN_CDC signal is 'high', the number of SiPM_L signals for generating the SiPM_E signal may be set differently.

In this way, the simultaneous detection circuit CDC 222 may be understood as a circuit that determines whether pulses are simultaneously generated (sensing light) in a plurality of SPADs within a predetermined period (t_CDC). Considering that the SPAD is a high-sensitivity photodetection device, a simultaneous detection circuit (CDC) 222 may be used to increase the reliability of the photodetection signal.

The operation of the coincidence detection circuit (CDC) 222 illustrated in FIG. 4 is only an example for explaining its function, and the operation of the simultaneous occurrence detection circuit (CDC) 222 is not limited to this form. For example, as long as the principle described above is maintained, the 'low' and 'high' states of the SiPM_L, SiPM_E, and WIN_CDC signals may be set in reverse. In addition, the simultaneous detection circuit (CDC) 222 may be selectively used according to circumstances.

Referring back to FIG. 3 , the SiPM_L signal output from the OR gate 221 and the SiPM_E signal generated through the simultaneous detection circuit (CDC) 222 are input to the first multiplexer 223 and selected from among the SiPM_L and SiPM_E signals. One can be used as a SiPM signal. In the exemplary description with reference to FIG. 1 , the SiPM_E signal is used in the first mode (coarse mode) and the SiPM_L signal is used in the second mode (fine mode), but the present embodiment is not limited thereto. Which one of the SiPM_L signal and the SiPM_E signal is used as the SiPM signal may be appropriately determined according to circumstances.

Next, the pulses of the SiPM signal can be counted by two up-down counters. In FIG. 3 , a part explaining the counting operation by the up-down counter is enlarged and displayed in FIG. 5 .

Referring to FIG. 5, four time bins (UP_A, UP_B, DN_A, and DN_B) are generated, and two up-down counters (UDC_A, UDC_B) can perform a count operation corresponding to the time bins where pulses are located. there is.

In the example of FIG. 5 , a background noise pulse (BGL) and a reflected light pulse (IR; hatched) are mixed in the SiPM pulse. It is exemplified that two background noise pulses (BGL) are uniformly generated every two pulses in four time bins, and two reflected light pulses (IR) are generated in a DN_A bin. UDC_A is counted up twice by two background noise pulses (BGL) belonging to the UP_A bin, and counted down four times by two background noise pulses (BGL) and two reflected light pulses (IR) belonging to the DN_A bin. is performed, resulting in two count down values (hatched). UDC_B is up-counted twice by two background noise pulses (BGL) belonging to the UP_B bin and down-counted twice by two background noise pulses (BGL) belonging to the DN_B bin. It can have a count value of 0. The count values of the two up-down counters (UDC_A, UDC_B) may be output to the next stage (A[8:0], B[8:0]).

Details of the counting operation by the two up-down counters UDC_A and UDC_B illustrated in FIG. 5 may be applied as described with reference to FIG. 1 .

Referring back to FIG. 3 , the second multiplexer 231 may receive as input A[8:0], which is an output of ToF and UDC_A stored in the memory 250, and selectively output one of the two values. The ToF stored in the memory 250 may be, for example, ToF_coarse obtained through the first mode (coarse mode) described with reference to FIG. 1 .

The third multiplexer 232 may receive TCNT received from the clock repeater 260 and B[8:0], which is an output of UDC_B, as inputs, and may selectively output one of the two values.

The comparator 233 may compare a value input from the second multiplexer 231 and a value input from the third multiplexer 232 and output the result. For example, the second multiplexer 231 may output ToF, the third multiplexer 232 may output TCNT, and the comparator 233 may output a result of comparing ToF and TCNT. Alternatively, the second multiplexer 231 outputs A[8:0], the third multiplexer 232 outputs B[8:0], and the comparator 233 outputs A[8:0] and B[8: 0] can be output.

An output of the comparator 233 may be input to the ToF_coarse determiner 241 and the time bin window determiner 242 in the first mode (coarse mode).

The ToF_coarse determiner 241 may receive a result of the comparator 233 comparing A[8:0] and B[8:0]. As described above with reference to FIG. 1, the ToF_coarse determiner 241 determines the absolute values of the count values A[8:0] and B[8:0] of the first and second up-down counters UDC_A and UDC_B. The ToF_coarse value can be determined according to the result of comparing values and the sign of a value having a large absolute value. To this end, the ToF_coarse determiner 241 receives the comparison result of A[8:0] and B[8:0] from the comparator 233, determines a ToF_coarse value based on the result, and stores the result in the memory 250. ) can be passed on. The memory 250 may store the ToF_coarse value determined by the ToF_coarse determiner 241 step by step, and store the 6-bit ToF_coarse value when all three steps are completed.

The bin window determiner 242 may receive a result of comparing ToF and TCNT from the comparator 233 and generate a bin having an appropriate length at an appropriate time by using the result. TCNT can be understood as a value starting at 000000 (2) and counting up one by one. The bin window determiner 242 may create bins from a time point when TCNT becomes equal to or greater than ToF. For example, when the first step (Step_C1) of the first mode (coarse mode) ends and ToF_coarse = 10(2) is determined, the ToF at this point becomes 10 0000(2). To find the execution point of the second step (Step_C2), TCNT starts counting up at 000000(2), and when TCNT reaches 10 0000(2) or more, which is the ToF value at this point, it can start creating bins. . A preset value may be used for the bin length (T_win) depending on which stage of the first mode. For example, in the first step (Step_C1) of the first mode, the bin length (T_win) is set to 80 ns (when the search range is 320 ns), and in the second step (Step_C2), the bin length (T_win) is set to 1 It is set to 20ns reduced by /4, and in the third step (Step_C3), the bin length (T_win) may be set to 5ns reduced by 1/4 again. That is, it may be understood that the bin window determiner 242 determines timing for newly generating four bins having a length of 1/4 of that of the previous step within the bin in which the pulse is detected in the previous step.

In the example of FIG. 3 , each pixel 200 includes a single stabilization circuit 212, an XOR gate 215, a mask circuit 214, an OR gate 221, a simultaneous detection circuit (CDC; 222), a second 1 multiplexer 223, 2 up-down counters (UDC_A, UDC_B), 2nd multiplexer 231, 3rd multiplexer 232, comparator 233, ToF_coarse determiner 241, empty window determiner 242 ), memory 250, etc. can only be understood as an example for explaining the principle of the DIQS hTDC of this embodiment, but the DIQS hTDC of this embodiment is not limited to this structure. The DIQS hTDC of this embodiment can be implemented in various modified structures while the principle described with reference to the drawings is equally applied.

6 to 8 illustrate an up-down counter (UDC) and its operating waveforms according to an embodiment of the present invention. The up-down counter 600 illustrated in FIG. 6 is an asynchronous/synchronous mixed method and can process a SiPM signal having a very small pulse width (eg, 500 ps).

Referring to FIG. 6 , the up-down counter 600 may include a plurality of flip-flops and a plurality of AND, OR, and XOR gates. Depending on embodiments, the plurality of flip-flops may include two D flip-flops 601 and 607 and ten T-flops 602 to 606 and 608 to 612.

The first to sixth flip-flops 601 to 606 are included in the first up-down counter UDC_A, and the seventh to twelfth flip-flops 607 to 612 are included in the second up-down counter UDC_B. can

Among the second to sixth flip-flops 602 to 606 included in the first up-down counter UDC_A, clock input terminals of the second and third flip-flops 602 and 603 have fourth to sixth flip-flops 604 to 606 . A signal different from that of the clock input terminal of 606) may be applied. In this way, some of the flip-flops included in the first up-down counter UDC_A of the present embodiment operate in synchronization with each other, but some operate out of synchronization, so that the first up-down counter UDC_A is asynchronous / It can be understood as a synchronous mixing method. That is, at least some of the plurality of flip-flops included in the first up-down counter UDC_A may receive signals different from those of the other flip-flops through the clock input terminal.

Specifically, the SiPM signal, which is a signal generated from the photodetection signal, is applied to the clock input terminal of the second flip-flop 602 and operates in synchronization with the edge of the SiPM signal. A signal generated from an output signal of the second flip-flop 602 may be applied to a clock input terminal of the third flip-flop 603 . Also, a signal generated from an output signal of the third flip-flop 603 may be applied to a clock input terminal of the fourth to sixth flip-flops 604 to 606 . Since the same signal is applied to clock input terminals of the fourth to sixth flip-flops 604 to 606, the fourth to sixth flip-flops 604 to 606 can operate in synchronization with each other. That is, the signal (SiPM) generated from the photodetection signal is applied to the clock input terminal of one flip-flop among the plurality of flip-flops, and the output signal of the flip-flop to which the SiPM signal is applied to the clock input terminal of the other flip-flops. A signal generated from may be applied. At this time, the flip-flop 602 to which the SiPM signal is applied to the clock input terminal may be a flip-flop that processes a least significant bit (LSB).

By applying a high frequency SiPM to the clock input terminal of the second flip-flop 602 in the first up-down counter UDC_A, the second flip-flop 602 responsible for the least significant bit (LSB) part is a high-speed signal It can operate in response to SiPM. Therefore, the first up-down counter UDC_A can process a high-speed SiPM signal without having a high bandwidth.

Typically, the pulse width of a SiPM signal is on the order of hundreds of ps (eg, 500 ps). Assuming that the first up-down counter UDC_A uses a synchronous method, the first up-down counter UDC_A must have a bandwidth of 2 GHz or more to operate normally. On the other hand, the first up-down counter UDC_A of this embodiment can count the SiPM signal, which is a high-speed signal, without having a high bandwidth by using an asynchronous/synchronous mixed method.

The second up-down counter UDC_B may also be implemented in the same way as the first up-down counter UDC_A.

FIG. 7 illustrates an operation in the first mode (coarse mode) described with reference to FIG. 1 .

In FIG. 7 , UD_A may be a signal for determining whether to up count or down count UDC_A. For example, when the SiPM signal is generated when UD_A is in a 'high' state, UDC_A may be up-counted, and if the SiPM signal is generated when UD_A is in a 'low' state, UDC_A may be counted down. Signal UD_B can also be used to determine whether to up count or down count UDC_B in the same way.

EN_A may be a signal that determines whether to count UDC_A. For example, if an SiPM signal is generated when EN_A is in a 'high' state, UDC_A may be up/down counted, and if EN_A is in a 'low' state, even if a SiPM signal is generated, UDC_A may not be counted. EN_B can also be used to determine whether to count UDC_B in the same way.

UD_A, UD_B, EN_A, and EN_B may be understood as signals for controlling the up-down counters UDC_A and UDC_B. CLK_win may be a clock signal used to generate signals UD_A, UD_B, EN_A, and EN_B for controlling the up-down counters UDC_A and UDC_B.

As described above, the SiPM is a signal generated based on a signal detected by the photodiode SPAD, and may be understood as a target signal to be counted by the up-down counters UDC_A and UDC_B. A[8:0] and B[8:0] may be count values of up-down counters UDC_A and UDC_B, respectively.

In FIG. 7 , four sections in which T_b has values of 32, 33, 34, and 35 may be understood as four bins. For example, assuming a state in which ToF_coarse = 10(2) is determined in the first stage of the first mode and ToF_coarse = 00(2) is determined in the second stage, ToF_coarse = 1000XX(2) up to the second stage ('X' means the value has not been determined. The lower two bits have not yet been determined as step 3 has not been performed). In this case, since four bins will be created starting from 100000(2) to 100011(2) in the third step, it can be understood that the four bins are assigned to decimal numbers 32, 33, 34, and 35, respectively.

In interval 32, since UD_A is in a 'high' state and EN_A is in a 'high' state, if an edge of the SiPM pulse occurs in this interval, UDC_A may perform an up count. Since UD_B is in a 'high' state and EN_B is in a 'high' state in interval 33, if an edge of the SiPM pulse occurs in this interval, UDC_B may perform an up count. In interval 34, since UD_A is in a 'low' state and EN_A is in a 'high' state, if an edge of the SiPM pulse occurs in this interval, UDC_A may perform a down count. In interval 35, since UD_B is in a 'low' state and EN_B is in a 'high' state, if an edge of the SiPM pulse occurs in this interval, UDC_B may perform a down count. In the example of FIG. 7, since the rising edge of SiPM occurs in interval 33, UDC_B performs an up count and the value of B[8:0] increases by 1 (in FIG. 7, the rising edge of the SiPM signal is used. Depending on the circuit implementation, a falling edge may be used).

FIG. 8 illustrates an operation in the second mode (fine mode) described with reference to FIG. 1 . As described with reference to FIGS. 3 and 4 , in the second mode (fine mode), the SiPM_L signal that has not passed through the CDC circuit can be used as the SiPM signal, and in this case, the SiPM signal can include more pulses.

FIG. 8 illustrates a state in which the frequency of CLK_win is doubled compared to FIG. 7 and the frequencies of all control signals UD_A, EN_A, UD_B, and EN_B are doubled. Accordingly, the length of the bin in the second mode (fine mode) in FIG. 8 is half that of the bin in the first mode (coarse mode) in FIG. 7 .

In FIG. 8, the SiPM signal includes 5 pulses, among which the first and second rising edges occur in the UP_B section, the third and fourth rising edges occur in the DN_A section, and the last rising edge A case that occurs in the DN_B section is exemplified. As a result, the first up-down counter UDC_A has a value of -2 through two down counts, and the second up-down counter UDC_B has a value of +1 through two up counts and one down count. will have the value of

As such, the first and second up-down counters UDC_A and UDC_B illustrated in FIG. 6 correspond to each of the four bins by the control signals UD_A, EN_A, UD_B, and EN_B for up/down of the high-speed SiPM signal. A count operation can be performed, and the length of the bin can be adjusted. In particular, the first and second up-down counters UDC_A and UDC_B have an advantage of processing a high-speed SiPM signal without having a high bandwidth by using an asynchronous/synchronous mixed method.

Terms such as "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless otherwise stated, and therefore do not exclude other components. It should be construed that it may further include other components. All terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, they are not interpreted in an ideal or excessively formal meaning.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (13)

In the lidar sensor including a plurality of pixels,
Each of the plurality of pixels includes a histogram-type time-to-digital converter (TDC),
The time-to-digital converter (TDC) performs a histogram operation using M/2 up-down counters (UDC),
The time-to-digital converter (TDC) divides the measurement target section into M time bins, and each of the M time bins is an up count and a down count of the M/2 up-down counters (UDC) Lidar sensor, characterized in that allocated to correspond to any one of.
The method of claim 1,
The M time bins include time bins corresponding to up counts and time bins corresponding to down counts of substantially the same length for each of the M/2 up-down counters (UDCs), so that noise caused by background light is reduced. When the pulses are uniformly distributed over the M time bins, each of the M/2 up-down counters (UDC) performs an up-count and a down-count by the noise pulse substantially the same number of times, thereby reducing the effect of the background light. A lidar sensor characterized in that this is offset.
The method of claim 1,
The time-to-digital converter (TDC) includes a first mode (coarse mode) performing a plurality of steps (step),
LiDAR sensor, characterized in that each time the plurality of steps proceeds, the measurement target section is reduced to 1 / M compared to the previous step.
The method of claim 3,
Among the plurality of steps, the M time bins of the n-th step are generated by dividing the time bins having the highest pulse intensity in the n-1th step into M again, and the n-th step time bins are generated by dividing the time bins having the highest pulse intensity in the n-1th step. A lidar sensor, characterized in that its length is substantially reduced to 1/M compared to the time bin.
The method of claim 4,
LiDAR sensor, characterized in that for each step of the plurality of steps of the first mode, the code corresponding to the time bin having the highest pulse intensity is determined as a partial code of the first mode flight time (ToF_coarse).
The method of claim 5,
The first mode (coarse mode) includes first to third steps,
The M is 4, and four time bins are generated for each step, and codes of 00(2), 01(2), 10(2), and 11(2) are sequentially assigned to the four time bins,
In the first step (step C1), the code of the time bin having the strongest pulse intensity is determined as the upper two bits of the first mode time-of-flight (ToF_coarse), and in the second step (step C2), the code of the time bin having the strongest pulse intensity is determined. The code of the time bin is determined by the middle 2 bits of the first mode time of flight (ToF_coarse), and the code of the time bin having the strongest pulse intensity in the third step (step C3) is the first mode time of flight (ToF_coarse). It is determined by the lower two bits of , and the first mode flight time (ToF_coarse) of 6 bits is determined through the first to third steps.
The method of claim 3,
The time-to-digital converter (TDC) further includes a second mode (fine mode),
In the second mode (fine mode), M time bins having a length smaller than the length of the time bins generated in the last step of the first mode (coarse mode) are generated,
A lidar sensor, characterized in that a predetermined phase is assigned to each of the M time bins of the second mode (fine mode).
The method of claim 7,
M = 4, the up-down counter (UDC) is two,
A lidar sensor, characterized in that the count value of the two up-down counters (UDC) is used to determine the ToF (ToF_fine) value of the second mode.
The method of claim 7,
In the first mode, the up-down counter (UDC) counts a signal generated through a simultaneous occurrence detection circuit (CDC),
In the second mode, the up-down counter (UDC) is a LiDAR sensor, characterized in that for counting signals that have not passed through the simultaneous occurrence detection circuit (CDC).
The method of claim 1,
The up-down counter (UDC) is a lidar sensor, characterized in that the counter of the asynchronous / synchronous mixed method.
The method of claim 10,
Each of the up-down counters (UDC) includes a plurality of flip-flops,
LiDAR sensor, characterized in that at least some of the plurality of flip-flops receive a signal different from the rest of the flip-flops as a clock input terminal.
The method of claim 11,
The signal (SiPM) generated from the photodetection signal is applied to the clock input terminal of one flip-flop among the plurality of flip-flops, and the output signal of the flip-flop to which the SiPM signal is applied to the clock input terminal of the other flip-flops. A lidar sensor, characterized in that the signal generated from is applied.
The method of claim 12,
The flip-flop to which the SiPM signal is applied to the clock input terminal is a lidar sensor, characterized in that the flip-flop processes the least significant bit (LSB).

Images