KR20200084699A - Apparatus and Method for measuring distance using SPAD(Single Photon Avalanche Diode) using TOF(Time of Flight) - Google Patents

Abstract

According to one aspect, a distance measuring device is presented. At this time, the device includes: a light source for emitting an optical signal at an object; at least one light-receiving unit for receiving the optical signal; a light-receiving time determination unit which determines a light-receiving time, which is the difference between the time when the optical signal is emitted by the light source and the time when the optical signal is received, as a light-receiving event in which an optical signal having threshold energy or higher is received is detected by the light-receiving unit; a counter which counts the difference between the number of times of light-receiving events having occurred before a reference time within a time range of interest and the number of times of light-receiving events having occurred after the reference time; and a processor. Based on the count of the counter, the processor can determine a time of flight (ToF) required until the optical signal emitted by the light source is reflected from the object and detected by the light-receiving unit. Therefore, the distance measuring device can acquire a distance image in real time by simultaneously operating pixels by using a distance sensor.

Description

Apparatus and Method for measuring distance using Single Photon Avalanche Diode (SPAD) using Time of Flight (TOF)}

The present disclosure relates to an apparatus and method for measuring a distance using a time of flight (TOF).

TOF (Time of Flight) technology is based on the law of the speed of light after irradiating pulsed light from a light source placed in or near the sensor and receiving the reflected light to measure the time between them. To extract the distance. In order to accurately measure TOF, a device having a very high sensitivity is required because a reaction must occur as soon as light reaches the light-receiving device.

For this, a single photon avalanche diode (SPAD), which can be fabricated by CMOS process technology, is mainly used. In addition, considering a speed of light fast enough to convert a distance of 15 m into a TOF of 100 ns, a precise time-to-digital conveter (TDC) having a resolution of several hundred ps to several ns is required.

TDCs with high precision are usually large, making it difficult to integrate them in pixels. In addition, in order to remove the noise component caused by heat or ambient light, the signal from the SPAD must be repeated several times to calculate the histogram of the TOF to obtain an accurate value. TOF values are generally measured at the 10-b level, but even if only once every 1 μs, 10,000 TOF values are generated from one pixel for 10 ms. Therefore, it is difficult to manufacture an array-type TOF sensor because a high-capacity memory is mounted on the pixel and the TOF value must be stored or read out quickly.

The present disclosure aims to realize a TOF pixel having a TAD that calculates a histogram based on SPAD that can be implemented by a CMOS process. Unlike the prior art, the size of the memory can be minimized by changing the structure of the TDC and extracting the TOF value only one bit at a time. In addition, when extracting each single bit of the multi-bit data for the TOF or the distance of the object, the SPAD signal is cumulatively calculated, so that the histogram can be calculated without additional computational processing burden. At this time, the system cost can be reduced by acquiring a 3D distance image by calculating a histogram of TOF at all pixels simultaneously without a high-speed data output terminal that consumes relatively high power.

According to one aspect, a distance measuring device is proposed. At this time, the device is a light source that irradiates an optical signal to the object, at least one light receiving unit that receives the optical signal, and when a light receiving event in which a light signal having a threshold energy or higher is received is detected by the light receiving unit, when the light signal is irradiated from the light source A light receiving time determining unit that determines a light receiving time, which is a difference between time points when an optical signal is received, counting the difference between the number of light receiving events occurring before the reference time and the number of light receiving events occurring after the reference time in the time range of interest It includes a counter, and a processor, the processor, based on the counting value of the counter, the light signal irradiated from the light source is reflected from the object to determine the flight time (Time of flight: ToF) until it is detected by the light receiving unit Can.

According to another aspect, a distance measuring method performed by a distance measuring device is proposed. At this time, the method is a predetermined irradiation cycle, the step of irradiating the optical signal to the object, the step of detecting a light-receiving event that receives a light signal of a threshold energy or more, as the light-receiving event is detected, the light signal from the time when the light signal is irradiated Determining a light reception time that is a difference between received time points, determining a difference between the number of light reception events occurring before the first reference time in the time range of interest and the number of light reception events occurring after the first reference time, and The method may include determining a time of flight (ToF) taken until the light signal irradiated from the light source is reflected from the object and sensed by the light receiving unit based on the difference in the determined number of times.

Using the distance sensor, as the pixels operate simultaneously, real-time acquisition of the distance image is possible. In addition, it is possible to acquire high-resolution, high-precision distance images and use it as a vehicle sensor for autonomous driving and ADAS, as well as mobile devices for Augmented Reality (AR) and Virtual Reality (VR).

1 is a block diagram of a distance measuring device according to an embodiment.
2 is a view for explaining a matching method between TOF and multi-bit data according to an embodiment.
3 and 4 are diagrams for explaining a binary search method for determining a TOF of an object.
5 and 6 are diagrams for explaining a method of determining a counting increment value used for binary search by using a plurality of SPADs.
7 is a flowchart illustrating a distance measurement method performed by a distance measurement device according to an embodiment.

The phrases “in some embodiments” or “in one embodiment” appearing in various places in this specification are not necessarily all referring to the same embodiment.

Some embodiments of the present disclosure can be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented with various numbers of hardware and/or software configurations that perform particular functions. For example, the functional blocks of the present disclosure can be implemented by one or more microprocessors, or by circuit configurations for a given function. Also, for example, functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks can be implemented with algorithms running on one or more processors. In addition, the present disclosure may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” can be widely used, and are not limited to mechanical and physical configurations.

In addition, the connection lines or connection members between the components shown in the drawings are merely illustrative of functional connections and/or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a distance measuring device according to an embodiment.

The device may include a light source 410, a light receiving unit 120, a water tube time determining unit 130, and a processor 140.

In the apparatus 100 illustrated in FIG. 1, only components related to the present embodiment are illustrated. Accordingly, it is apparent to those skilled in the art that the apparatus 100 may further include other general-purpose components in addition to the components shown in FIG. 4. For example, the device may further include a communication module (not shown), a memory (not shown), and the like.

The light source 110 may irradiate the object with light used to measure the distance of the object. For example, the light source 110 may irradiate a pulsed optical signal. At this time, light in the infrared region may be irradiated from the light source 110. Also, the light source 110 may be disposed adjacent to the light receiving unit 120 that receives light.

In general, a distance measurement method using a time of flight (TOF) may be used. That is, a method of measuring the distance of the object is used by converting the time taken before the light irradiated from the light source 110 and reflected on the object is received by the light receiving unit 120 using a speed of light. At this time, the flight time in units of time includes distance information in that it can be converted into a distance multiplied by the speed of light.

The light receiving unit 120 may be implemented as a device that converts light energy into electrical energy, for example, a photo detector. As an optical signal having a predetermined energy or higher is received, the light receiving unit 120 may output an electrical signal. Through the output electrical signal, external components may determine that light has been received.

For example, the light receiving unit 120 may include a single-photon avalanche diode (SPAD). According to an implementation method, one pixel may include a plurality of SPADs. Single-photon avalanche diode (SPAD) can detect single photons and has high resolution. SPAD can be manufactured with CMOS technology. The SPAD can be triggered by the received light to output a pulse signal. At this time, information regarding a point in time when light has reached may be obtained through a pulse signal.

SPAD is a photodetector using a phenomenon in which a pair of electrons and holes is generated by the energy of a photon as a single photon enters a diode, the generated pair moves, and energy is chained to generate a new pair of electrons and holes. . Since avalanche occurs by a single photon, SPAD has a very high resolution, but at the same time, it is very sensitive and can be relatively affected by weak noise.

When light having a predetermined energy or higher is received by the SPAD, it may be determined that a light reception event has occurred. The fact that light having a predetermined energy or more is received may indicate, for example, a case where photons are detected by a predetermined number or more of SPADs among a plurality of SPADs. That is, it may be determined that the light-receiving event has occurred when the photon is detected by the SPAD of a predetermined number or more in the same minimum unit time for every minimum unit time distinguishable according to resolution in the device 100. This is for distinguishing a noise component having a relatively low energy since light reflected from an object will have a relatively high energy compared to noise.

The light reception time determination unit 130 may determine a light reception time at which light is received by the light reception unit 120. At this time, the light-receiving time represents a time from the time when the light is irradiated from the light source 110 to the time when the light is received. When light is continuously irradiated from the light source 110 at a predetermined period, the light reception time may be determined based on the irradiation time of the nearest light from the time when the light is received. At this time, the period in which light is irradiated may be very short, and thus, the light irradiated over several times at regular periods may be used as an image to measure the distance of the object in the frame at the same time. As described above, since light of a noise component such as ambient light as well as reflected light of an object may be received, when a light reception event occurs several times at different light reception times, a light reception event in which the reflected light is received among the light reception events is determined. , By determining the light reception time of the determined light reception event as the TOF of the object, the distance of the object may be determined.

For example, when SPAD is used, as the photon is detected by the SPAD, the light reception time determination unit 130 may determine the light reception time corresponding to the time at which the photon was received. The light-receiving time can be determined within a certain region of time of interest. Since the speed of light is very high, a significant light reception time can also be limited. That is, the light reception time required for the light irradiated at one time point to be reflected and received by the real object may be determined within a relatively small range of time regions as a round trip time of light.

In the above-described embodiment, the TOF used when measuring the distance of an object in a frame of the same time is determined within a corresponding time region of interest.

For example, to measure the distance of the object, light may be irradiated from the light source 110 at regular intervals. At this time, the size of the time region of interest may be the same as or less than the irradiation period of light. Meanwhile, light irradiated several times during a plurality of cycles may be used to measure a distance of an object corresponding to one frame. Accordingly, it can be assumed that among the plurality of light irradiation periods corresponding to the same frame, the TOF of light irradiated at a time point of one period is the same as the TOF of light irradiated at a time point of another period. As a result, the period of light irradiation may be shorter than the time interval between frames, and the size of the time region of interest may be smaller than that of light.

The counter may count the difference between the number of light reception events in which the light reception time is before the reference time in the time region of interest and the number of light reception events in which the light reception time is after the reference time. For example, the counter can be controlled by a control signal transmitted from the processor 140. The counter can be, for example, an up-down counter. That is, as the counter receives the up counting control signal, it can output the counting value increased in constant increments. Conversely, as the counter receives the down counting control signal, it can output a counting value reduced in constant increments. At this time, the increment value can be variously determined and can be changed by a predetermined factor. This will be described in detail below. For example, the light reception time determination unit 130 may convert the light reception time to a digital data format and output the converted data. The light receiving time determining unit 130 may include a time digital converter (TDC), but is not limited thereto.

The processor 140 may be implemented by one or a plurality of processors 140. For example, the processor 140 may be implemented as an array of multiple logic gates, or may be implemented as a combination of a general-purpose microprocessor 140 and a memory in which programs that can be executed in the microprocessor 140 are stored. For example, the processor 140 may be a central processing unit (CPU), a graphic processing unit (GPU), or the like.

The processor 140 may play an overall role for controlling the device 100. For example, the processor 140 may control the device 100 as a whole by executing programs stored in a memory (not shown) in the device 100. Also, the at least one processor 140 may execute functions of the device 100 in parallel by executing programs stored in a memory (not shown).

The processor 140 determines a time of flight (ToF) until the light signal irradiated from the light source 110 is reflected from the object and sensed by the light receiving unit 120 based on the counting value of the counter. Can.

For example, when the bits allocated to the TOF are N bits, the processor 140 may determine from bit to bit, from the most significant bit to the Nth bit.

For example, in the first step, the processor 140 may determine a first reference time in the time domain of interest. For example, the first reference time may be a light reception time corresponding to an intermediate point in time region of interest, but is not limited thereto.

The processor 140 may acquire the light reception time determined by the light reception time determination unit 130 as the light reception event occurs. In addition, the processor 140 may transmit a down counting control signal to the counter when the acquired light reception time is ahead of the first reference time. Conversely, when the acquired light reception time is after the first reference time, the processor 140 may transmit an up-counting control signal to the counter. At this time, a signal including incremental information of down counting or up counting may be applied to the counter together. The method in which the processor 140 determines the increment will be described with reference to FIGS. 5 and 6 below.

The processor 140 may determine a difference between the number of light reception events occurring before the first reference time and the number of light reception events occurring after the first reference time for a predetermined period corresponding to the first step from the counting value of the counter. At this time, by applying an exponential weight to a light receiving event that receives relatively high energy light without applying the incremental value equally to all light receiving events, the processor 140 considers the importance of each light receiving event The counter can be controlled to determine the counting value.

Hereinafter, a large number of light reception events means that more light reception events have occurred in consideration of importance.

For example, when the sign of the counter's counting value is negative, the processor 140 may determine that the number of light-receiving events occurring before the first reference time is large. In this case, the processor 140 may determine that the TOF of the reflected light is ahead of the first reference time. Conversely, when the sign of the counting value is positive, it can be determined that the number of light reception events occurring after the first reference time is large. In this case, the processor 140 may determine that the first reference time is ahead of the TOF of the reflected light.

For example, the most significant bit value of the TOF may have a value of 0 and 1, respectively, depending on whether the TOF is before or after the first reference time. The processor 140 may determine the TOF according to the binary search method described above.

In the second step, the processor 140 may determine the second upper bit of the TOF according to the same binary search method as the first step.

When the TOF determines that the TOF is before the first reference time in the first step, the processor 140 may perform a binary search for the TOF targeting a sub-region corresponding to the first reference time region among all time regions of interest. Conversely, if it is determined that the TOF is before the first reference time, a binary search for the TOF may be performed on a sub-region corresponding to an area after the first reference time among all time regions of interest.

The processor 140 may determine a second reference time in the sub-region that is the target of binary search in the first step. For example, the second reference time may be a light reception time corresponding to an intermediate viewpoint of the sub-region.

Subsequently, the binary search is performed based on the second reference time in the second step in the same manner as the binary search based on the first reference time in the first step.

The processor 140 may determine, in the second step, if the sign of the counting value of the counter is negative, the second upper bit of the TOF may be determined as 0, and when it is positive, it may be determined as 1.

As N steps are performed as performed in the first step and the second step, N bits of TOF data may be determined by determining a single bit in each step.

The above-described light-receiving unit 120, the light-receiving time determining unit 130, and the processor 140 related to the determination of the TOF may be implemented by being integrated for each single pixel. As described above, since the TOF is determined in a manner of determining one bit for each step according to the counting value, only a relatively small amount of memory is required, and the size and integration of pixels can be increased.

2 is a view for explaining a matching method between TOF and multi-bit data according to an embodiment.

In FIG. 2, the horizontal axis represents TOF, which includes distance information to the object.

The time domain from 0 to T on the horizontal axis represents the entire time domain of interest.

At this time, the time region of interest is divided into 8 regions, and TOFs belonging to each region are matched with multi-bit data of the region. Looking at the allocation method of the multi-bit data, the most significant bit value is 0 when the TOF is 4T/8 before the center of the entire time domain of interest and 1 when it is thereafter.

In the sub-region where the most significant bit value is 0, that is, from 0 to 4T/8, the second high-order bit value of TOF is 0 when TOF is before the center point 2T/8 of the sub-region, and 1 when thereafter. Likewise, in the sub-region where the most significant bit value is 1, that is, in the region from 4T/8 to T, the second high-order bit value of TOF is 0 when TOF is before the center point 6T/8 of the sub-region, and 1 when thereafter to be. In this way, the least significant bit is determined, and each TOF is matched with 8 multi-bit data.

For example, when the origin is a point at which light is irradiated, the point at which the pulse signal is received by being reflected on the object (250, 255) may represent the TOF of the object. That is, an object having a TOF of 001 is detected as the pulse signal 250 is received, and an object having a TOF of 101 is detected as the pulse signal 255 is received. In general, each pixel corresponds to one object, so one TOF value is sensed, so the above example shows the TOF value detected in multiple pixels.

3 and 4 are diagrams for explaining a binary search method for determining a TOF of an object.

In FIG. 3, a process of determining a TOF of 3 bits is illustrated by performing a binary search through 3 steps, and FIG. 4 shows a memory in which the TOF is determined and stored from the most significant bit in each step. .

When the four graphs are shown in FIG. 3, the graph shown at the top is shown on the horizontal axis where the TOF of the object 320 is a time axis based on the internal clock (CLK) of the device.

The second graph is a process for determining the most significant bit of the object 302. In step 1, the entire time domain is divided into two domains, and down counting is performed as a light-receiving event is detected in the region indicated by the down arrow (0). Conversely, when a light-receiving event is detected in the region where the up arrow 1 is shown, up counting is performed. In the following, since the increment of up counting and down counting is determined by the light reflected from the object 302, that is, a relatively dominant light receiving event occurs in the up counting area, which is the area where the TOF of the object 302 belongs, so counting of the counter The value is a positive number, and the highest bit value of the TOF of the object 302 is determined to be 1, and a value of 1 is stored in the corresponding region 310 of the memory.

The third graph is a process for determining the second upper bit of the object 302. In step 2, in step 1, as the most significant bit is determined as 1, a binary search is performed based on the corresponding sub-region. The sub-region is divided into two regions, and down-counting is performed as a light-receiving event is detected in the region indicated by the down arrow (0). Conversely, when a light-receiving event is detected in the region where the up arrow 1 is shown, up counting is performed. Similarly, since a definitive light-receiving event will occur in an area to which the TOF of the object 302 belongs, the value of the second high-order bit of the TOF of the object 302 is determined to be 0, and a value of 0 is stored in the corresponding area 320 of the memory.

Similarly, in step 3, the lowest bit value of the TOF of the object 302 is determined to be 1, and a value of 1 is stored in the corresponding area 330 of the memory.

Since the TOF can be determined only by the memory for storing the current counting value and the memory storing the value determined by each bit value of the TOF, the distance measuring device can be implemented with less memory than the method for storing the histogram in all time domains. have.

5 and 6 are diagrams for explaining a method of determining a counting increment value used for binary search using a plurality of SPADs.

Numbers in square brackets are used to distinguish different SPADs. Referring to FIGS. 5 and 6, a total of 8 SPADs are used. Eight SPADs may correspond to the same pixel and may be used to measure the distance of the same object.

Theoretically, an SPAD response event (indicated by a circle) may be generated by sunlight as well as light reflected from an object. Since sunlight does not change rapidly in intensity and is incident on all SPADs equally, the number of events that respond to each SPAD is the same, but since the reaction time is very short within tens of ns, all SPADs are highly unlikely to react simultaneously. On the other hand, SPAD reaction events caused by light reflected from an object return are highly likely to react at the same time because all SPADs are looking at the same object. Therefore, it is possible to reduce the influence of sunlight by considering the number of SPAD reaction events inside the pixel. The effects of heat events or other noise inside the sensor can be regarded as in the case of sunlight, so the influence can be reduced similarly.

In the graphs at the bottom of FIGS. 5 and 6, the horizontal axis represents the time axis to which the TOF belongs, and the vertical axis represents the increment of counting.

In FIG. 5 and FIG. 6, when a photon is detected only in a single SPAD, the counting increment may be determined as 0 by considering it as a solar component or a noise component. That is, it may be considered that the light reception event has not occurred. For example, the photon is detected in SPAD[4] at 410, but the photon is not detected in all other SPADs, so that no light receiving event is considered and the counting increment may be determined to be 0. On the other hand, the photon was detected at SPAD[0] at time 420, but this is different from the photon detection time 430 at a different time as the time 430 and time delay at which the photons were detected at four SPADs.

Since the photons are detected in the four SPADs at the photon detection time 430, the counting increment is determined to be proportional to the number. That is, since only one SPAD is considered as noise when photons are detected, the number of SPADs that detect photons minus 1 is determined as the counting increment. Depending on the intensity of sunlight, the number of occurrences of the light-receiving event can be increased to 2 or 3, so that only the number of events after that can be considered.

In FIG. 6, the counting increment is determined exponentially to give a greater weight to simultaneous photon sensing. For example, the photon detection time 530 corresponds to the photon detection time 430 in FIG. 5, but the power of 2, not 3, or 8 may be determined as the counting increment. In addition to this, counting increments can be determined in a number of ways.

7 is a flowchart illustrating a distance measurement method performed by a distance measurement device according to an embodiment.

The method illustrated in FIG. 7 may be performed by the distance measuring device 100 of FIG. 1.

In operation 710, the apparatus 100 may irradiate an optical signal to the object at a predetermined irradiation cycle.

In operation 720, the device 100 may detect a light reception event in which an optical signal having a threshold energy or higher is received. For example, the apparatus 100 may determine that a light-receiving event has occurred when a photon corresponding to the same TOF is detected in a certain number of SPADs among a plurality of SPADs allocated to each pixel.

In operation 730, as the light reception event is detected, the device 100 may determine a light reception time that is a difference between a time point when the light signal is received and a time point when the light signal is received.

In operation 740, the apparatus 100 may determine a difference between the number of light reception events occurring before the first reference time in the time range of interest and the number of light reception events occurring after the first reference time. It may be an intermediate point in time region of interest of the first reference time.

In operation 750, it is possible to determine a time of flight (ToF) taken before the light signal emitted from the light source is reflected from the object and sensed by the light receiving unit based on the determined number of times difference.

The embodiments can also be implemented in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The scope of the present embodiment is indicated by the claims, which will be described later, rather than by the detailed description, and should be interpreted to include all modified or modified forms derived from the meaning and scope of the claims and their equivalent concepts.

Claims (9)

A light source irradiating an optical signal to the object at a predetermined irradiation cycle;
At least one light receiving unit that receives an optical signal;
A light reception time determination unit configured to determine a light reception time that is a difference between a time at which the light signal is received from a time when the light signal is irradiated from the light source, as a light reception event in which a light signal having a threshold energy or more is received is detected by the light reception unit;
A counter for counting the difference between the number of light reception events occurring before the reference time in the time range of interest and the number of light reception events occurring after the reference time; And
Including a processor,
The processor, based on the counting value of the counter, determines a flight time (Time of flight: ToF) taken before the light signal emitted from the light source is reflected from the object and sensed by the light receiving unit Device. According to claim 1,
The processor,
Determine a first reference time in the time-of-interest region,
As the light reception event is detected, when the light reception time corresponding to the detected light reception event is before the first reference time, an up counting control signal is transmitted to the counter,
When it is after the first reference time, the distance measuring device transmits a down counting control signal to the counter. According to claim 1,
The processor,
Based on the counting value of the counter, obtain a first single bit value indicating whether the flight time is greater than the first reference time,
And determining the obtained first single bit value as the most significant bit value of multi-bit data representing the flight time. The method of claim 3,
The processor,
Based on the first single bit value, a distance for determining one of the sub-regions corresponding to the first reference time and the smaller sub-regions corresponding to the first reference time of the interest time region. Measuring device. The method of claim 4,
The processor,
Determine a second reference time in the determined sub-region,
As the light reception event is detected, when the light reception time determined in response to the detected light reception event is before the second reference time, an up counting control signal is transmitted to the counter,
When it is after the second reference time, the distance measuring device transmits a down counting control signal to the counter. The method of claim 5,
The processor,
Based on the counting value of the counter, obtain a second single bit value indicating whether the flight time is greater than the first reference time,
And determining the acquired second single bit value as a second upper bit value of multi-bit data indicating the flight time. According to claim 1,
The light receiving unit includes a single photon Avalanche Diode (SPAD), a distance measuring device. The method of claim 7,
The light receiving unit includes a plurality of single photon detection elements,
When an optical signal is received at the same light receiving time by a single photon detection element having a predetermined number or more among the plurality of single square detection elements, the processor determines that the light reception event has occurred. In the distance measuring method performed by the distance measuring device,
Irradiating an optical signal to the object at a predetermined irradiation cycle;
Detecting a light reception event in which an optical signal having a threshold energy or higher is received;
Determining a light reception time, which is a difference between a time point when the light signal is received, from a time point when the light signal is irradiated, as the light reception event is detected;
Determining a difference between the number of light reception events occurring before the first reference time in the time range of interest and the number of light reception events occurring after the first reference time;
And determining a time of flight (ToF) taken before the light signal irradiated from the light source is reflected from the object and sensed by the light receiving unit based on the difference in the determined number of times.

Images