WO2021051479A1

WO2021051479A1 - Interpolation-based time of flight measurement method and system

Info

Publication number: WO2021051479A1
Application number: PCT/CN2019/113710
Authority: WO
Inventors: 何燃; 朱亮; 王瑞; 闫敏
Original assignee: 深圳奥锐达科技有限公司
Priority date: 2019-09-19
Filing date: 2019-10-28
Publication date: 2021-03-25
Also published as: CN110596725B; CN110596725A

Abstract

An interpolation-based time of flight measurement method. The method comprises the following steps: S1, acquiring a first time of flight of a first combined pixel corresponding to a first light source (801); S2, obtaining, through interpolation calculation, a second time of flight of a second superpixel corresponding to a second light source (802); S3, locating, according to the second time of flight, a second combined pixel corresponding to the second light source, and drawing a histogram (803); and S4, calculating a third time of flight by means of the histogram (804). Rough time of flight values are directly supplied to most pixels by means of an interpolation mode, so that the drawing of a fine histogram can be directly executed by means of these pixels on the basis of the rough time of flight values, so as to calculate a high-precision fine time of flight value; and since the step of drawing a rough histograms is omitted, the calculation time can be greatly reduced, thereby increasing the frame rate.

Description

Method and system for measuring flight time based on interpolation

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 19, 2019, the application number is 201910889455.6, and the invention title is "Interpolation-based time-of-flight measurement method and measurement system", the entire content of which is incorporated by reference In this application.

Technical field

This application relates to the field of computer technology, and in particular to a method and system for measuring flight time based on interpolation.

Background technique

The Time of Flight (TOF) method calculates the distance of an object by measuring the flight time of the beam in space. Due to its high accuracy and large measurement range, it is widely used in consumer electronics, unmanned aerial vehicles, AR/ VR and other fields.

Distance measurement systems based on the time-of-flight principle, such as time-of-flight depth cameras, lidars, and other systems often include a light source emitting end and a receiving end. The light source emits a light beam to the target space to provide illumination, and the receiving end receives the light beam reflected by the target. Calculate the distance of the object by calculating the time required for the beam to be reflected and received.

At present, the lidar based on the time-of-flight method is mainly mechanical and non-mechanical. The mechanical type uses a rotating base to achieve a 360-degree distance measurement with a large field of view. The advantage is that the measurement range is large, but it has high power consumption and high resolution. And the problem of low frame rate. Non-mechanical mid-area array lidar can solve the problem of mechanical lidar to a certain extent. It transmits a surface beam of a certain field of view into space at a time and receives it through the area array receiver, so its resolution and frame The rate has been improved, and because there is no need to rotate parts, it is easier to install. Nevertheless, area array lidar still faces some challenges.

The higher the resolution of the area array lidar, the more comprehensive the effective information. In addition, the dynamic measurement also has higher requirements on the frame rate and measurement accuracy. However, the improvement of resolution, frame rate, and accuracy often depends on the circuit scale of the receiving end and the improvement of the modulation and demodulation method. However, increasing the circuit scale will increase power consumption, signal-to-noise ratio and cost; in addition, it will also increase the amount of on-chip storage. This has brought serious challenges to mass production; current modem methods are also difficult to achieve high-precision, low-power consumption and other requirements.

Summary of the invention

The purpose of the present application is to provide a time-of-flight measurement method and measurement system based on interpolation, so as to solve at least one of the above-mentioned background art problems.

To achieve the foregoing objective, an embodiment of the present application provides an interpolation-based time-of-flight measurement method, which includes the following steps:

S1. Acquire the first flight time of the first combined pixel corresponding to the first light source;

S2. Obtain the second flight time of the second superpixel corresponding to the second light source through interpolation calculation;

S3. According to the second flight time, locate a second composite pixel corresponding to the second light source and draw a histogram;

S4. Calculate the third flight time by using the histogram.

In some embodiments, the first light source and the second light source are arranged on the same light source array, and the first light source and the second light source can be independently controlled in groups.

In some embodiments, the interpolation includes one-dimensional interpolation or two-dimensional interpolation, and the interpolation method includes at least one of linear interpolation, spline interpolation, and polynomial interpolation.

In some embodiments, when the histogram is drawn, only the pixels in the second combined pixel are activated.

In some embodiments, the step S2 further includes performing a difference calculation on the flight time values of multiple spots and pixels to be interpolated, and the interpolation calculation is performed only when the difference is less than a certain threshold.

The embodiment of the present application also provides a time-of-flight measurement system based on interpolation, including:

An emitter configured to emit a pulsed light beam, which includes a first light source and a second light source;

A collector configured to collect photons in the pulsed beam reflected by an object and form a photon signal, which includes a plurality of pixels;

The processing circuit is connected to the transmitter and the collector, and is used to perform the following steps to calculate the flight time: S1, obtain the first flight time of the first combined pixel corresponding to the first light source; S2, obtain by interpolation calculation The second time of flight of the second superpixel corresponding to the second light source; S3. According to the second time of flight, locate the second composite pixel corresponding to the second light source and draw a histogram; S4. Use the histogram The graph calculates the third flight time.

In some embodiments, step S2 further includes performing a difference calculation on the flight time values of the multiple spots and pixels to be interpolated, and the interpolation calculation is performed only when the difference is less than a certain threshold.

The embodiment of the present application provides a method for measuring time of flight based on interpolation, including the following steps: S1, obtaining the first time of flight of the first combined pixel corresponding to the first light source; S2, obtaining the time of flight corresponding to the second light source through interpolation calculation The second flight time of the second superpixel; S3. According to the second flight time, locate the second composite pixel corresponding to the second light source and draw a histogram; S4, use the histogram to calculate the third flight time ; The coarse flight time value is directly provided for most of the pixels through interpolation, so that these pixels can directly perform fine histogram drawing based on the coarse flight time value to calculate the high-precision fine flight time value, because there is no coarse histogram The drawing steps can greatly reduce the calculation time, thereby increasing the frame rate.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Fig. 1 is a schematic diagram of a time-of-flight distance measurement system according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a light source according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a pixel unit in a collector according to an embodiment of the present application.

Fig. 4 is a schematic diagram of a readout circuit according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a histogram according to an embodiment of the present application.

Fig. 6 is a dynamic histogram drawing time-of-flight measurement method according to an embodiment of the present application.

Fig. 7 is a time-of-flight measurement method according to an embodiment of the present application.

Fig. 8 is an interpolation-based time-of-flight measurement method according to an embodiment of the present application.

detailed description

In order to make the technical problems, technical solutions, and beneficial effects to be solved by the embodiments of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be used for fixing or circuit connection.

It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top" The orientation or positional relationship indicated by "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the application and simplifying the description, rather than indicating or implying what is meant. The device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, "multiple" means two or more than two, unless otherwise specifically defined.

The present application provides an interpolation-based time-of-flight measurement method and measurement system. For ease of understanding, the following describes an embodiment of the distance measurement system first.

As an embodiment of the present application, a distance measurement system is provided, which has stronger resistance to ambient light and higher resolution.

Fig. 1 is a schematic diagram of a time-of-flight distance measurement system according to an embodiment of the present application. The distance measurement system 10 includes a transmitter 11, a collector 12, and a processing circuit 13. The transmitter 11 provides a emitted light beam 30 to the target space to illuminate an object 20 in the space. At least part of the emitted light beam 30 is reflected by the object 20 to form a reflected light beam. 40. At least part of the light signals (photons) of the reflected light beam 40 are collected by the collector 12. The processing circuit 13 is connected to the transmitter 11 and the collector 12 respectively, and the trigger signals of the transmitter 11 and the collector 12 are synchronized to calculate the light beam from the transmitter The time required for 11 to be emitted and received by the collector 12, that is, the flight time t between the emitted light beam 30 and the reflected light beam 40, further, the distance D of the corresponding point on the object can be calculated by the following formula:

D=c·t/2 (1)

Among them, c is the speed of light.

The transmitter 11 includes a light source 111 and an optical element 112. The light source 111 can be a light source such as a light emitting diode (LED), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), etc., or an array light source composed of multiple light sources. Optionally, the array light source 111 is in a single block A plurality of VCSEL light sources are generated on a semiconductor substrate to form a VCSEL array light source chip. The light beam emitted by the light source 111 may be visible light, infrared light, ultraviolet light, or the like. The light source 111 emits a light beam outward under the control of the processing circuit 13. For example, in some embodiments, the light source 111 emits a pulsed light beam at a certain frequency (pulse period) under the control of the processing circuit 13, which can be used in the direct time-of-flight method ( In Direct TOF measurement, the frequency is set according to the measurement distance, for example, it can be set to 1MHz-100MHz, and the measurement distance is several meters to several hundred meters. It is understandable that it may be a part of the processing circuit 13 or a sub-circuit independent of the processing circuit 13 to control the light source 111 to emit related light beams, such as a pulse signal generator.

The optical element 112 receives the pulsed beam from the light source 111, and optically modulates the pulsed beam, such as diffraction, refraction, reflection, etc., and then emits the modulated beam into space, such as a focused beam, a flood beam, and a structured light beam. Wait. The optical element 112 may be one or more combinations of lenses, diffractive optical elements, masks, mirrors, MEMS galvanometers, and the like.

The processing circuit 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc., or a general-purpose processor. For example, when the depth camera is integrated into smart terminals such as mobile phones, TVs, and computers, The processor in the terminal can be used as at least a part of the processing circuit 13.

The collector 12 includes a pixel unit 121 and an imaging lens unit 122. The imaging lens unit 122 receives and guides at least part of the modulated light beam reflected by the object to the pixel unit 121. In some embodiments, the pixel unit 121 is composed of a single photon avalanche photodiode (SPAD), or an array pixel unit composed of multiple SPAD pixels. The array size of the array pixel unit represents the resolution of the depth camera, such as 320 ×240 etc. SPAD can respond to the incident single photon to realize the detection of single photon. Because of its high sensitivity and fast response speed, it can realize long-distance and high-precision measurement. Compared with CCD/CMOS and other image sensors based on the principle of light integration, SPAD can count single photons, such as the use of time-correlated single photon counting (TCSPC) to realize the collection of weak light signals and the calculation of flight time . Generally, connected to the pixel unit 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ). These circuits can be integrated with the pixels, and they can also be part of the processing circuit 13. For ease of description, the processing circuit 13 will be collectively regarded.

In some embodiments, the distance measurement system 10 may also include a color camera, an infrared camera, an IMU, and other devices. The combination of these devices can achieve richer functions, such as 3D texture modeling, infrared face recognition, SLAM and other functions.

In some embodiments, the transmitter 11 and the collector 12 can also be arranged in a coaxial form, that is, the two are realized by optical devices with reflection and transmission functions, such as a half mirror.

In the direct time-of-flight distance measurement system using SPAD, a single photon incident on the SPAD pixel will cause an avalanche, the SPAD will output an avalanche signal to the TDC circuit, and the TDC circuit will detect the time interval between the photon emission from the emitter 11 and the avalanche. After multiple measurements, the time interval is counted through the time-correlated single photon counting (TCSPC) circuit for histogram statistics to recover the waveform of the entire pulse signal, and the time corresponding to the waveform can be further determined, and the flight time can be determined based on this time , So as to achieve accurate flight time detection, and finally calculate the distance information of the object based on the flight time. Assuming that the pulse period emitted by the pulse beam is Δt, the maximum measurement range of the distance measurement system is Dmax, and the corresponding maximum flight time is

Generally, Δt≥t ₁ is required to avoid signal aliasing, where c is the speed of light. If the number of multiple measurements required by TCSPC is n, then the time (frame period) to achieve a single frame measurement will not be less than n*t ₁ , that is, the period of each frame measurement includes n photon counting measurements. For example, the maximum measurement range is 150m, the corresponding pulse period Δt=1us, n=100000, then the frame period will not be less than 100ms, and the frame rate will be less than 10fps. It can be seen that the maximum measurement range in the TCSPC method limits the pulse period, which further affects the frame rate of distance measurement.

Fig. 2 is a schematic diagram of a light source according to an embodiment of the present application. The light source 111 is composed of a plurality of sub-light sources arranged on a single substrate (or multiple substrates), and the sub-light sources are arranged on the substrate in a certain pattern. The substrate may be a semiconductor substrate, a metal substrate, etc., and the sub-light source may be a light emitting diode, an edge-emitting laser emitter, a vertical cavity surface laser emitter (VCSEL), etc., optionally, the light source 111 is composed of a plurality of VCSELs arranged on the semiconductor substrate. An array of VCSEL chips composed of sub-light sources. The sub-light source is used to emit light beams of any desired wavelength, such as visible light, infrared light, and ultraviolet light. The light source 111 emits light under the modulation drive of the driving circuit (which may be part of the processing circuit 13), such as continuous wave modulation, pulse modulation, and the like. The light source 111 can also emit light in groups or as a whole under the control of the driving circuit. For example, the light source 111 includes a first sub-light source array 201, a second sub-light source array 202, etc., and the first sub-light source array 201 emits light under the control of the first driving circuit. , The second sub-light source array 202 emits light under the control of the second driving circuit. The arrangement of the sub-light sources can be a one-dimensional arrangement or a two-dimensional arrangement, and can be a regular arrangement or an irregular arrangement. For ease of analysis, only one example is schematically given in Figure 2. In this example, the light source 111 is a regular array of 8×9 sub-light sources, and the sub-light sources are divided into 4×3=12 groups. Each light source is shown in the figure. It is distinguished by different symbols, that is, the light source 111 is composed of 12 3×2 regularly arranged sub-light source arrays.

Fig. 3 is a schematic diagram of a pixel unit in a collector according to an embodiment of the present application. The pixel unit includes a pixel array 31 and a readout circuit 32. The pixel array 31 includes a two-dimensional array composed of a plurality of pixels 310 and a readout circuit 32 composed of a TDC circuit 321, a histogram circuit 322, etc., where the pixel array is used for Collect at least part of the light beam reflected by the object and generate the corresponding photon signal. The readout circuit 32 is used to process the photon signal to draw a histogram reflecting the pulse waveform emitted by the light source in the transmitter. Further, it can also be based on the histogram. Calculate the flight time, and finally output the result. The readout circuit 32 may be composed of a single TDC circuit and histogram circuit, or may be an array readout circuit composed of multiple TDC circuit units and histogram circuit units.

In some embodiments, when the emitter 11 emits a spot beam to the object to be measured, the optical element 112 in the collector 12 will guide the spot beam to the corresponding pixel. Generally, in order to receive as much of the reflected beam as possible For the optical signal, the size of a single spot is set to correspond to multiple pixels (the correspondence here can be understood as imaging, and the optical element 112 generally includes an imaging lens). For example, a single spot in Figure 3 corresponds to 2×2=4 pixels, that is, the The photons reflected by the spot beam will be accepted by the corresponding 4 pixels with a certain probability. For the convenience of description, the pixel area composed of the corresponding multiple pixels is called "combined pixel" in this application. The size of the combined pixel can be based on actual conditions. Need to be set, including at least one pixel, for example, it can be 3×3, 4×4 size. Generally, the light spot is round, oval, etc., and the combined pixel size should be set to be the same as the light spot size, or slightly smaller than the light spot size, but considering the different magnifications caused by the distance of the measured object, the combined pixel The size needs to be considered comprehensively when setting.

In the embodiment shown in FIG. 3, the pixel unit 31 includes an array composed of 14×18 pixels as an example for description. Generally, the measurement system 10 between the transmitter 11 and the collector 12 can be divided into coaxial and off-axis according to the different setting modes. For the coaxial situation, the light beam emitted by the transmitter 11 will be reflected by the collector 12 after being reflected by the object to be measured. For the corresponding combined pixel collection in, the position of the combined pixel will not be affected by the distance of the measured object; but for off-axis situations, due to the existence of parallax, when the measured object is at different distances, the position of the light spot on the pixel unit is also Will change, usually along the baseline (the line between the transmitter 11 and the collector 12, the horizontal direction is used to represent the baseline direction in this application) direction, so when the distance of the measured object is unknown, it The position of the pixel is uncertain. In order to solve this problem, this application will set a pixel area (here called "super pixel") composed of multiple pixels exceeding the number of pixels in the combined pixel to receive the reflected spot light beam. When setting the size of the super pixel, it is necessary to consider the measurement range of the system 10 and the length of the baseline at the same time, so that the combined pixels corresponding to the spots reflected by objects at different distances within the measurement range will all fall into the super pixel area, namely The size of a super pixel should exceed at least one super pixel. Generally, the size of the super pixel is the same as the sum pixel along the vertical direction of the baseline, and is larger than the sum pixel along the baseline direction. The number of super pixels is generally the same as the number of spot beams collected by the collector 12 in a single measurement, which is 4×3 in FIG. 3.

In some embodiments, the super pixel is set to: when at the lower limit of the measurement range, that is, the spot falls on one side of the super pixel (left or right, depending on the relative position of the emitter 11 and the collector 12) at close range. Position); when at the upper limit of the measurement range, that is, the spot falls on the other side of the super pixel at a distance. In the embodiment shown in FIG. 3, the super pixels are set to a size of 2x6. For example, for

spots

363, 373, and 383, the corresponding super pixels are 361, 371, and 381, respectively. The

spots

363, 373, and 383 are respectively far and middle. , The spot beam reflected by a close object, the corresponding combined pixels fall on the left, middle and right side of the super pixel respectively.

In some embodiments, the combined pixels share one TDC circuit unit, that is, one TDC circuit unit is connected to each pixel in the combined pixel. When any pixel in the combined pixel receives photons and generates a photon signal, the TDC circuit unit Both can calculate the flight time corresponding to the photon signal. This situation is more suitable for the coaxial situation, for the off-axis situation, because the combined pixel position will change with the distance of the measured object. In the embodiment shown in FIG. 3, a TDC circuit array composed of 4×3 TDC circuit units will be included.

In some embodiments, the super pixels share a TDC circuit unit, that is, a TDC circuit unit is connected to each pixel in the super pixel. When any pixel in the super pixel receives a photon and generates a photon signal, the TDC circuit unit Both can calculate the flight time corresponding to the photon signal. Since the super pixel can include the pixel shift caused by the off-axis parallax, the super pixel sharing TDC can be applied to the off-axis situation. In the embodiment shown in FIG. 3, a TDC circuit array composed of 4×3 TDC circuit units will be included. Sharing the TDC circuit can effectively reduce the number of TDC circuits, thereby reducing the size and power consumption of the readout circuit.

For off-axis situations, more pixels need to be set to form super pixels. In a single measurement (or single exposure) time, the number of spots that can be collected will be much smaller than the number of pixels, in other words, the effective depth of the collection The resolution of the data (time-of-flight value) is much smaller than the pixel resolution. For example, the pixel resolution in Figure 3 is 14×18, and the spot distribution is 4×3, that is, the effective depth data resolution of a single frame measurement is 4× 3.

In order to improve the resolution of the measured depth data, multi-frame measurement can be used. The spots emitted by the transmitter 11 during the multi-frame measurement are "off", resulting in a scanning effect. The spots received by the collector 12 are in multiple frames. Deviations also occur in the measurement. For example, the spots corresponding to two adjacent frames of measurement in Figure 3 are 343 and 353 respectively, which can improve the resolution. In some embodiments, the “deviation” of the spots can be achieved by grouping control of the sub-light sources on the light source 111, that is, in the measurement of two or more adjacent frames, the adjacent sub-light sources are sequentially turned on, for example, in the first frame. During the measurement, turn on the first sub-light source array 201, and in the second frame of measurement, turn on the second sub-light source array 202, and so on. Not only can the horizontal grouping control but also the vertical grouping control to improve the effectiveness in the two-dimensional direction The resolution of the depth data.

For the "deviation" of the spot measured in multiple frames, the super pixels corresponding to the spots in different positions also need to be deviated when setting. As shown in Figure 3, the super pixel corresponding to the spot 343 is 341, the super pixel corresponding to the spot 353 is 351, and the super pixel is 351. 351 is laterally shifted relative to super pixel 341, and there is a partial pixel overlap between super pixel 341 and super pixel 351. For super pixels measured in multiple frames, they overlap each other. In order to ensure that the TDC circuit can be used every time The frame can accurately measure the photon counting flight time of the corresponding super pixel. This application provides a solution for dual sharing of the TDC circuit.

In some embodiments, the pixel area connected by a single TDC circuit unit will include the area composed of all super pixels that deviate in the multi-frame measurement, and there is overlap between the pixel areas corresponding to two adjacent TDC circuit units. Specifically, in the embodiment shown in FIG. 3, the pixel area 391 shares a TDC circuit unit, and the pixel area 391 includes 6 superpixels corresponding to 6 frames of measurement when the 6 groups of sub-light sources are turned on in sequence. Similarly, adjacent pixel regions 392 share a TDC circuit unit, and there is a partial overlap between the two

pixel regions

391 and 392, which results in some pixels connected to two TDC circuit units. In a single frame measurement, according to For the projected spots, the processing circuit 13 will gate the corresponding pixels so that the acquired photon signals can be measured by a single TDC circuit unit, so as to avoid crosstalk and errors. In some embodiments, the number of TDC circuits is the same as the number of spots collected by the collector 12 during a single frame measurement. In Figure 3, the number of spots is 4×3. Each shared TDC circuit is connected to 4×10 pixels, and adjacent TDCs are connected to each other. There are 4×4 pixels overlap between the pixel regions connected by the circuit unit.

The following describes the adjustable histogram circuit scheme. In a single frame measurement period, the TDC circuit will receive the photon signal from the pixel in the super pixel area connected to it, and calculate the time interval between the signal and the starting clock signal (ie, flight Time), and convert the time interval into a temperature code or binary code and save it in the histogram circuit. After multiple measurements, the histogram circuit can draw a histogram that reflects the pulse waveform. Based on the histogram, the pulse can be accurately obtained Flight time. Generally, the larger the measurement range, the wider the measurable time interval of the TDC circuit is required. If the accuracy requirement is higher, the higher the time resolution of the TDC circuit is required. Both the wider the time interval and the higher the time resolution are required. The TDC circuit uses a larger scale to output a binary code with a larger number of digits. Due to the increase in the number of binary code digits, the storage capacity of the memory of the histogram circuit is higher. The larger the memory capacity, the higher the cost, and the greater the difficulty in mass production of monolithic integration. For this reason, the present application provides a readout circuit solution with adjustable histogram circuit.

Fig. 4 is a schematic diagram of a readout circuit according to an embodiment of the present application. The readout circuit includes a TDC circuit 41 and a histogram circuit 42. The TDC circuit 41 collects the time interval of the photon signal and converts the time interval into a time code (binary code, temperature code, etc.), and then the histogram circuit 42 will be based on this The time code is counted on the corresponding internal time unit (that is, the storage unit used to save time information), such as adding 1, after multiple measurements, the photon counts in all time units can be counted and the time histogram can be drawn Figure. The drawn histogram is shown in Figure 5, where ΔT refers to the width of the time unit, T ₁ , T ₂ respectively refer to the start and end time of the histogram drawing, [T ₁ , T ₂ ] is the histogram Time interval, T=T ₂ -T ₁ refers to the total time width. The ordinate of the time unit ΔT is the photon count value stored in the corresponding storage unit. Based on the histogram, the highest peak method can be used to determine the pulse The position of the waveform, and get the corresponding flight time t.

In some embodiments, the histogram circuit 42 includes an address decoder 421, a memory matrix 422, a read/write circuit 424, and a histogram drawing circuit 425. The TDC circuit inputs the acquired time code (binary code, temperature code, etc.) that reflects the time interval to the address decoder 421, and the address decoder 421 converts it into address information, which will be stored in the storage matrix 422 in. Specifically, the storage matrix 422 includes a plurality of storage units 423, that is, time units. Each storage unit 423 is pre-configured with a certain address (or address interval). When the time code address received by the address decoder 421 is equal to a certain When the address of the storage unit is the same or within the address range of the storage unit, the read/write circuit 424 will perform the +1 operation on the corresponding storage unit, that is, complete one photon count, and after multiple measurements, the value of each storage unit The data reflects the number of photons received during the time interval. After a single frame measurement (multiple measurements), the data of all memory cells in the memory matrix 422 are read out to the histogram drawing circuit 425 for histogram drawing.

In order to reduce the storage capacity of the storage matrix as much as possible, in fact, it is necessary to reduce the number of storage units 423. For this reason, in this application, a control signal is applied to the histogram circuit 42 through the processing circuit to dynamically set the address (address interval) of each storage unit 423. , Thereby further realizing the dynamic control of the histogram time resolution ΔT and/or the time interval width T. For example, under the premise that the number of storage units 423 remains unchanged, by setting the address interval corresponding to the storage unit 423 to a larger time interval, that is, increasing the width ΔT of the time unit, the time interval interval that the total storage matrix can store will change. Larger, the total time interval of the histogram will become larger. For ease of description, the histogram with a larger time interval is called a thick histogram; for example, the address interval corresponding to the storage unit 423 can be set to a smaller time interval. , The time interval interval that the total storage matrix can store is reduced, but the time resolution of storage will increase, and the time resolution of the histogram will increase. Compared with the coarse histogram, the histogram with a smaller time interval is called Is a fine histogram.

In this application, a large-scale and high-precision time-of-flight measurement will be realized by dynamically coarse-fine adjustment of the histogram during the time-of-flight measurement process.

Fig. 6 is a dynamic histogram drawing time-of-flight measurement method according to an embodiment of the present application. It includes the following steps:

Step 1: Draw a coarse histogram in coarse precision time units. That is, the address or address interval corresponding to each time unit in the storage matrix 422 is configured by applying a control signal, that is, T and ΔT are set, and ΔT is configured as a larger time interval ΔT ₁ in this step. Generally, the setting of the histogram time interval T needs to consider the measurement range. The time interval ΔT ₁ should be set in consideration of the measurement range and the number of histogram storage units, that is, the flight time corresponding to the measurement range is allocated to all the histogram storage units. , Such as equal distribution or non-equal distribution, etc., so that all storage units can cover the measurement range. After a number of measurements, the flight time value obtained from each measurement is matched to perform an operation of adding 1 to the corresponding time unit, and finally the drawing of the rough histogram is completed.

Step 2. Use the rough histogram to calculate the rough flight time value t _1. Based on the rough histogram, the maximum peak value method can be used to find the pulse waveform position, and the corresponding flight time can be read as the rough flight time value t ₁ . The accuracy or minimum resolution of the time value is the time interval ΔT ₁ of the time unit.

When the measurement range is relatively large and the number of storage units is limited, ΔT ₁ will be relatively large. When the number of photons is large, the pulsed photons will be submerged in the background light, making it impossible to detect the pulse waveform. Therefore, in some embodiments, the measurement range may be divided into several intervals, and each interval corresponds to a respective flight time interval, and the time interval ΔT of each time interval T may be the same or different. When drawing a rough histogram, you can draw each time interval one by one. Since the distance of the measured object is unknown, the time interval in which the corresponding flight time will fall is also unknown, so it may be in a certain time interval. When drawing the rough histogram, the pulse waveform cannot be detected, that is, the rough flight time value cannot be calculated. In this case, for example, when the waveform position cannot be found based on the rough histogram in step 2, then go back to step 1 for the next step The rough histogram is drawn until the pulse waveform is found in the rough histogram. Of course, it may also be impossible to find the pulse waveform due to errors or the object distance is too far. In order to avoid the problem of continuous cycle detection, you can set the number of cycles, for example, when the number of rough histogram drawing times exceeds a certain threshold (such as 3 times) , It is considered that the target is not detected this time, or that the target is located at infinity this time, thus ending the measurement.

Step 3: According to the obtained rough flight time value, draw a fine histogram in fine time units. At this time, since the rough value of the flight time value is already known, one more round of measurement can be performed and the corresponding histogram is drawn. At this time, the histogram circuit is controlled by the control signal and corresponds to each time unit in the storage matrix 422. The address or address interval is configured as a smaller time interval ΔT ₂ . Generally, the time interval ΔT ₂ only needs to correspond to a smaller measurement range interval that can contain the true flight time value and the number of histogram storage units when setting. The measurement range interval can be set as a rough flight time value In the middle, add a certain margin on both sides, for example, it can be set to [t ₁ -T',t ₁ -T'], where the smaller T'is set, the smaller the time interval ΔT ₂ , and the higher the resolution. For example, in some In the embodiment, it can be set as T'=5%T, so the sum of the time intervals of all time units is only 10% of the time interval corresponding to the rough histogram. In other embodiments, the ratio of the margin to the time interval of the coarse histogram may be set in the range of 1%-25%. Then a new round of multiple measurements is performed, and the flight time value obtained each time is matched and the corresponding time unit is increased by 1 to complete the drawing of the fine histogram.

Step 4. Use the fine histogram to calculate the fine flight time t _2. Based on the fine histogram, the maximum peak method can be used to find the waveform position, and the corresponding flight time can be read as the fine flight time value t ₂ . The accuracy or minimum resolution is the time interval ΔT ₂ of the time unit. If the setting of T'=5%T in the third step is used to illustrate, the accuracy of the fine flight time is increased by 10 times compared with the rough flight time (the minimum resolution is increased by 10 times).

The measurement method of the above-mentioned histogram dynamic coarse and fine adjustment is essentially a process of first performing coarse positioning in a larger measurement range, and then performing fine measurement based on the positioning result. It is understandable that the above-mentioned coarse and fine adjustment method can also be extended to three or more steps of measurement. For example, in some embodiments, the first time resolution is first measured to obtain the first flight time, and then based on The first flight time is measured with the second time resolution to obtain the second flight time, and finally based on the second flight time with the third time resolution, the third flight time is obtained. The accuracy of the three times is increased successively, and finally a higher-precision measurement can be achieved.

In some embodiments, since the histogram only counts the flight time value within the time interval T when the histogram is drawn, each pixel in the measurement system collector 12 can be activated within a specified time interval (enable ), thereby reducing power consumption, the specified time interval generally includes the time interval T drawn by the histogram. For example, when the time interval of the histogram is [3ns, 10ns], the time interval during which the pixel is activated can be set to [2.5ns, 10.5ns].

It is understandable that the above-mentioned measurement method is not only applicable to the coaxial distance measurement system, but also applicable to the off-axis measurement system. In particular, it should be noted that for the off-axis measurement system including the collector shown in FIG. 3, the histogram dynamic adjustment scheme can be further used for super pixel positioning not only to improve accuracy but also to reduce power consumption. Fig. 7 is a time-of-flight measurement method according to another embodiment of the present application. The following will be described with reference to Figure 3. The method includes the following steps:

Step 1: Receive the super pixel TDC output signal, and draw a coarse histogram in coarse-precision time units. Since the distance of the object is not clear before the measurement, the position of the spot cannot be determined, that is, the position of the combined pixel cannot be determined, and the combined pixel may fall into different positions of the super pixel according to the distance of the object. Therefore, in this step, firstly, each pixel in the super pixel is enabled to be in an activated state to receive photons, and the photon signal output by the shared TDC of the super pixel is received, and then the histogram is drawn. The histogram adopts the dynamic adjustment histogram scheme shown in Fig. 6, in this step, the coarse histogram will be drawn using the coarse precision time unit.

Step 2. Use the rough histogram to calculate the rough flight time value t _1. Based on the rough histogram, use the maximum peak method to find the waveform position, and read the corresponding flight time as the rough flight time value t ₁ , the flight time The accuracy or minimum resolution of the value is the time interval ΔT ₁ of the time unit.

When the measurement range is relatively large and the number of storage units is limited, ΔT ₁ will be relatively large. When the number of photons is large, the pulsed photons will be submerged in the background light, making it impossible to detect the pulse waveform. Therefore, in some embodiments, the measurement range may be divided into several intervals, and each interval corresponds to a respective flight time interval, and the time interval ΔT of each time interval T may be the same or different. When drawing a rough histogram, you can draw each time interval one by one. Since the distance of the measured object is unknown, it is also unknown which time interval the corresponding flight time will fall into, so it may be in a certain time interval. The pulse waveform cannot be detected when drawing the rough histogram. For this case, for example, when the waveform position cannot be found based on the rough histogram in step 2, then go back to step 1 to draw the next rough histogram until the rough histogram Until the pulse waveform is found in the figure. Of course, it may also be impossible to find the pulse waveform due to errors or the object distance is too far. In order to avoid the problem of continuous cycle detection, you can set the number of cycles, for example, when the number of rough histogram drawing times exceeds a certain threshold (such as 3 times) , It is considered that the target is not detected this time, or that the target is located at infinity this time, thus ending the measurement.

Step 3: According to the obtained rough time-of-flight value, locate the combined pixels and draw a fine histogram in fine time units. Since the rough time-of-flight value has been clarified, it can be based on the rough time-of-flight value and the parallax to locate the position of the combined pixel. It is usually necessary to save the relationship between the position of the combined pixel and the rough time-of-flight value in the system in advance After obtaining the rough time-of-flight value, the position of the combined pixel can be directly located according to the relationship; then based on the position of the combined pixel, only the combined pixel is activated, and a fine histogram is drawn in fine time units. Since the rough value of the flight time value is already known, you can perform multiple measurements and draw the corresponding histogram. At this time, the histogram circuit is controlled by the control signal and stores the address or address corresponding to each time unit in the matrix 422. The address interval is configured as a small time interval ΔT ₂ . Generally, the time interval ΔT ₂ only needs to correspond to a smaller measurement range interval that can contain the true flight time value and the number of histogram storage units when setting it. The measurement range interval can be set as a rough flight time value In the middle, a certain margin is added on both sides, for example, it can be set to [t ₁ -T',t ₁ -T'], where the smaller T'is set, the smaller the time interval ΔT ₂ , and the higher the resolution. For example, in some In the embodiment, it can be set as T'=5%T, so the sum of the time intervals of all time units is only 10% of the time interval corresponding to the rough histogram. In other embodiments, the ratio of the margin to the time interval of the coarse histogram may be set in the range of 1% to 25%. Then a new round of multiple measurements is performed, and the flight time value obtained each time is matched and the corresponding time unit is increased by 1 to complete the drawing of the fine histogram.

The following describes the time-of-flight measurement method based on interpolation. The embodiments in Figures 2 and 3 introduce examples of improving resolution through multi-frame measurement. It can be understood that when multi-frame measurement is performed, the depth data of each frame is measured Both can use the histogram dynamic adjustment scheme shown in Fig. 6 or Fig. 7. For example, when the first sub-light source array 201 is turned on, the dynamic coarse-fine histogram is drawn to obtain the first frame of depth image; when the second sub-light source array 202 is turned on, the dynamic coarse-fine histogram is drawn to obtain the second frame of depth image; The first and second frames of depth images get higher resolution depth images. In some embodiments, depth images of more than 3 frames can also be collected and merged into a higher resolution depth image.

However, if each frame of depth image acquisition requires dynamic adjustment of the thickness, the acquisition time of each high-resolution fusion depth image will be relatively long, and the overall frame rate will not be high. In order to increase the frame rate as much as possible, this application provides an interpolation-based time-of-flight measurement method according to an embodiment of the application as shown in FIG. 8. The method includes the following steps:

Step 1: Obtain the first flight time of the first combined pixel corresponding to the first light source. In this step, the first light source in the emitter 11 is turned on to emit the spot beam corresponding to the first light source, and the spot beam will fall on the closed pixel on the pixel unit 31 in the collector 12, as shown in 4x3 in Figure 3 Take the spot represented by the solid circle as an example. The processing circuit can further obtain the first flight time of the combined pixel. For example, the coarse-fine dynamic adjustment scheme in the embodiment shown in FIG. 6 or FIG. 7 or any other scheme can be used. Obtain the fine flight time (first flight time) of the combined pixel.

Step 2: Obtain the second flight time of the second superpixel corresponding to the second light source through interpolation calculation. When the second light source is turned on, a spot beam adjacent to the spot beam corresponding to the first light source will be emitted, and the spot beam will also fall on the combined pixel of the collector 12. For ease of illustration, only used in Figure 3 The dotted circle draws a spot 353. The spot 353 and the spot 343 are spatially staggered because the positions of the first light source and the second light source are staggered, and therefore the respective corresponding pixels are also staggered. Generally, when the space points are relatively close, the distance between the two points will not differ too much. Therefore, in some embodiments, the flight time value of the corresponding pixel corresponding to the spot 343 obtained in step 1 can be used as the second flight time value (rough flight time) of the super pixel 351 corresponding to the spot 353, and then the fine flight time is performed. Calculation. In some embodiments, the second time-of-flight value of the superpixel of the spot 353 can be estimated by using the combined pixels corresponding to multiple first light sources around the spot 353, for example, using the time-of-flight values of the left and right combined pixels for interpolation. The interpolation may be one-dimensional interpolation or two-dimensional interpolation, and the interpolation method may be at least one of interpolation methods such as linear interpolation, spline interpolation, and polynomial interpolation.

Step 3: According to the second flight time, locate the second composite pixel corresponding to the second light source and draw a histogram. After the second flight time is obtained by interpolation, based on the flight time and the parallax, the position of the spot in the superpixel, that is, the position of the combined pixel, can be located, and then based on the position of the combined pixel, only the combined pixel is activated, and at the same time Plot histograms in fine time units.

Step 4. Use the histogram to calculate the third flight time. Based on the histogram, the maximum peak value method can be used to find the waveform position, and the corresponding flight time can be read as the third (fine) flight time value t ₂ , the flight time value The accuracy or minimum resolution of is the time interval ΔT ₂ of the time unit.

Compared with the method described in Figure 6 or Figure 7, the flight time measurement method in the above steps only needs to use the coarse-fine histogram drawing method for the flight time calculation of only a few spots, and at least 2 frames of flight time measurement are required to obtain high accuracy. The flight time value of most spots can be calculated by using the flight time value of known spots as the rough flight time value of the coarse histogram, and based on the rough flight time value, only a single fine histogram drawing is required. However, this can greatly improve efficiency. For example, if the light sources are divided into 6 groups, only the first group of light sources need to perform coarse-fine measurement when the light source is turned on, and the subsequent 5 groups only need to perform a single fine measurement when the flight time measurement is performed after the light source is turned on.

In some embodiments, considering that the surface of the object to be measured often has jumps, that is, the distance difference is large, it is difficult to obtain an accurate time-of-flight value by interpolation. Therefore, performing a fine measurement based on the result of the interpolation may cause errors. Therefore, a judgment can be made before the interpolation in step 2. For example, when the flight time values of the combined pixels corresponding to the multiple spots to be interpolated (such as the left and right spots) are greater than a certain threshold, the difference between the two spots on the surface There is a jump in the surface depth value of the object between the two spots, the spots between the two spots will still maintain the measurement scheme of the thick-fine histogram drawing, as long as the interpolation calculation is performed when it is less than the subtraction value.

In some embodiments, the first time of flight of the first combined pixel may also be the rough time of flight, that is, when the first time of flight of the first combined pixel is demodulated and calculated, only a single rough histogram drawing is required, and then the Interpolation is performed based on the rough flight time obtained by the rough histogram drawing.

It is understandable that when the distance ranging system of this application is embedded in a device or hardware, corresponding structural or component changes will be made to adapt to requirements. The essence of this application is still the distance ranging system of this application, so it should be regarded as this application. The scope of protection. The above content is a further detailed description of the application in combination with specific/preferred implementations, and it cannot be determined that the specific implementation of the application is limited to these descriptions. For those of ordinary skill in the technical field to which this application belongs, without departing from the concept of this application, they can also make several substitutions or modifications to the described implementations, and these substitutions or modifications should be regarded as It belongs to the protection scope of this application.

In the description of this specification, reference to the description of the terms "one embodiment", "some embodiments", "preferred embodiment", "examples", "specific examples", or "some examples" etc. means to incorporate the implementation The specific features, structures, materials, or characteristics described by the examples or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner.

In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification without contradicting each other. Although the embodiments of the present application and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope defined by the appended claims. In addition, the scope of the present application is not intended to be limited to the specific embodiments of the processes, machines, manufacturing, material composition, means, methods, and steps described in the specification. A person of ordinary skill in the art will readily understand that the above-mentioned disclosures, processes, machines, processes, machines, and machines that currently exist or will be developed later that perform substantially the same functions as the corresponding embodiments described herein or obtain substantially the same results as the embodiments described herein can be used. Manufacturing, material composition, means, method, or step. Therefore, the appended claims intend to include these processes, machines, manufacturing, material compositions, means, methods, or steps within their scope.

Claims

A time-of-flight measurement method based on interpolation, characterized in that it comprises the following steps:

S1. Acquire the first flight time of the first combined pixel corresponding to the first light source;

S2. Obtain the second flight time of the second superpixel corresponding to the second light source through interpolation calculation;

S3. According to the second flight time, locate a second composite pixel corresponding to the second light source and draw a histogram;

S4. Calculate the third flight time by using the histogram.
The time-of-flight distance measurement method according to claim 1, wherein the first light source and the second light source are arranged on the same light source array, and the first light source and the second light source can be Group independent control.
The time-of-flight distance measurement method according to claim 1, wherein the interpolation includes one-dimensional interpolation or two-dimensional interpolation, and the interpolation method includes at least one of linear interpolation, spline interpolation, and polynomial interpolation.
3. The time-of-flight distance measurement method according to claim 1, wherein when the histogram is drawn, only the pixels in the second sum pixel are activated.
5. The time-of-flight distance measurement method according to claim 1, characterized in that: the step S2 further comprises calculating the difference of the time-of-flight values of the multiple spots and pixels to be interpolated, and when the difference is less than a certain threshold, The interpolation calculation is performed.
A time-of-flight measurement system based on interpolation, which is characterized in that it comprises:

An emitter configured to emit a pulsed light beam, which includes a first light source and a second light source;

A collector configured to collect photons in the pulsed beam reflected by an object and form a photon signal, which includes a plurality of pixels;

The processing circuit is connected to the transmitter and the collector, and is used to perform the following steps to calculate the flight time:

S1. Acquire the first flight time of the first combined pixel corresponding to the first light source;

S2. Obtain the second flight time of the second superpixel corresponding to the second light source through interpolation calculation;

S3. According to the second flight time, locate a second composite pixel corresponding to the second light source and draw a histogram;

S4. Calculate the third flight time by using the histogram.
The time-of-flight distance measurement system according to claim 6, wherein the first light source and the second light source are arranged on the same light source array, and the first light source and the second light source can be Group independent control.
The time-of-flight distance measurement system according to claim 6, wherein the interpolation includes one-dimensional interpolation or two-dimensional interpolation, and the interpolation method includes at least one of linear interpolation, spline interpolation, and polynomial interpolation.
7. The time-of-flight distance measurement system according to claim 6, wherein when the histogram is drawn, only the pixels in the second sum pixel are activated.
The time-of-flight distance measurement system according to claim 6, characterized in that: step S2 further comprises calculating the difference of the time-of-flight values of the multiple spots and pixels to be interpolated, and only when the difference is less than a certain threshold Perform the interpolation calculation.