WO2021051480A1

WO2021051480A1 - Dynamic histogram drawing-based time of flight distance measurement method and measurement system

Info

Publication number: WO2021051480A1
Application number: PCT/CN2019/113712
Authority: WO
Inventors: 何燃; 朱亮; 王瑞; 闫敏
Original assignee: 深圳奥锐达科技有限公司
Priority date: 2019-09-19
Filing date: 2019-10-28
Publication date: 2021-03-25
Also published as: CN110596724A; CN110596724B

Abstract

Provided is a dynamic histogram drawing-based time of flight distance measurement method, comprising the following steps: S1, receiving a superpixel TDC output signal, and drawing a first histogram on the basis of time units of first precision (701); S2, calculating a first time of flight by using the first histogram (702); S3, positioning a combined pixel according to the first time of flight and drawing a second histogram on the basis of time units of second precision, the combined pixel being composed of at least one pixel, and a superpixel comprising at least one combined pixel (703); and S4, calculating a second time of flight by using the second histogram (704). Also provided is a dynamic histogram drawing-based time of flight distance measurement system. Wide-range and high-precision time of flight measurement is achieved by performing dynamic coarse-fine adjustment on histograms in the time-of-flight distance measurement system.

Description

A dynamic histogram drawing flight time and distance measurement method and measurement system

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 19, 2019. The application number is 201910889452.2, and the invention title is "Dynamic Histogram Drawing Flight Time Distance Measurement Method and Measurement System". The entire content is by reference. Incorporated in this application.

Technical field

This application relates to the field of computer technology, and in particular to a method and system for measuring flight time and distance drawn by dynamic histogram.

Background technique

The Time of Flight (TOF) method calculates the distance of an object by measuring the flight time of the beam in space. Due to its high accuracy and large measurement range, it is widely used in consumer electronics, unmanned aerial vehicles, AR/ VR and other fields.

Distance measurement systems based on the time-of-flight principle, such as time-of-flight depth cameras, lidars, and other systems often include a light source emitting end and a receiving end. The light source emits a light beam to the target space to provide illumination, and the receiving end receives the light beam reflected by the target. Calculate the distance of the object by calculating the time required for the beam to be reflected and received.

At present, the lidar based on the time-of-flight method is mainly mechanical and non-mechanical. The mechanical type uses a rotating base to achieve a 360-degree distance measurement with a large field of view. The advantage is that the measurement range is large, but it has high power consumption and high resolution. And the problem of low frame rate. Non-mechanical mid-area array lidar can solve the problem of mechanical lidar to a certain extent. It transmits a surface beam of a certain field of view into space at a time and receives it through the area array receiver, so its resolution and frame The rate has been improved, and because there is no need to rotate parts, it is easier to install. Nevertheless, area array lidar still faces some challenges.

The higher the resolution of the area array lidar, the more comprehensive the effective information. In addition, the dynamic measurement also has higher requirements on the frame rate and measurement accuracy. However, the improvement of resolution, frame rate, and accuracy often depends on the circuit scale of the receiving end and the improvement of the modulation and demodulation method. However, increasing the circuit scale will increase power consumption, signal-to-noise ratio and cost; in addition, it will also increase the amount of on-chip storage. This has brought serious challenges to mass production; current modem methods are also difficult to achieve high-precision, low-power consumption and other requirements.

Summary of the invention

The purpose of this application is to provide a dynamic histogram drawing time-of-flight distance measurement method and measurement system, so as to solve at least one of the above-mentioned background art problems.

To achieve the foregoing objective, an embodiment of the present application provides a dynamic histogram drawing time-of-flight distance measurement method, which includes the following steps:

S1. Receive the super pixel TDC output signal, and draw the first histogram with the first precision time unit;

S2. Calculate the first flight time by using the first histogram;

S3. According to the first flight time, locate the combined pixel and draw a second histogram with a second precision time unit; the combined pixel is composed of at least one pixel, and the super pixel includes at least one of the combined pixel;

S4. Calculate the second flight time by using the second histogram.

In some embodiments, the addresses of the first precision time unit and the second precision time unit are pre-configured such that the time interval of the first histogram is greater than the time interval of the second histogram; or , So that the time resolution of the first histogram is smaller than the time resolution of the second histogram.

In some embodiments, the pulse waveform position is located using the maximum peak method based on the first or second histogram and the first or second flight time is calculated.

In some embodiments, when the second histogram is drawn, only the pixels in the combined pixels are activated.

In some embodiments, when the first flight time cannot be calculated using the first histogram in step S2, return to step S1 to continue drawing the first histogram until the first flight time is calculated.

In some embodiments, the time interval of the first histogram is determined according to the measurement range and the number of time units of the histogram; the time interval of the second histogram is determined according to the first flight time. The time interval of the second histogram is determined by taking the first flight time as the middle and adding a certain margin on both sides. The margin is set to 1%-25% of the time interval of the first histogram.

In some embodiments, the relationship between the position of the combined pixel and the first flight time is stored in advance, so that the combined pixel can be located according to the relationship after the first flight time is acquired.

In some embodiments, when the first histogram or the second histogram is drawn, the corresponding pixel only collects photons at a time interval that includes the first histogram or the second histogram. The range is activated.

The embodiment of the present application also provides a dynamic histogram drawing time-of-flight distance measurement system, including: a transmitter configured to emit a pulsed beam; a collector configured to collect photons in the pulsed beam reflected back by an object and A photon signal is formed; a processing circuit, connected to the transmitter and the collector, is used to perform the following steps: S1, receive the superpixel TDC output signal, and draw the first histogram with the first precision time unit; S2, use The first histogram calculates the first flight time; S3. According to the first flight time, locate the combined pixel and draw a second histogram with the second precision time unit; the combined pixel is composed of at least one pixel, so The super pixel includes at least one of the composite pixels; S4, using the second histogram to calculate a second flight time.

The embodiment of the application provides a dynamic histogram drawing time-of-flight distance measurement method, which includes the steps of: S1, receiving the superpixel TDC output signal, and drawing a first histogram with a first precision time unit; S2, using the first line The square graph calculates the first flight time; S3. According to the first flight time, locate the combined pixel and draw a second histogram with the second precision time unit; the combined pixel is composed of at least one pixel, and the super pixel includes at least One said combined pixel; S4. Calculate the second flight time using the second histogram; realize large-scale and high-precision flight time measurement by dynamically coarse-fine adjustment of the histogram in the flight time distance measurement system This avoids the problems of large memory capacity of the histogram circuit in the prior art, which leads to high cost and difficulty in mass production of monolithic integration.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Fig. 1 is a schematic diagram of a time-of-flight distance measurement system according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a light source according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a pixel unit in a collector according to an embodiment of the present application.

Fig. 4 is a schematic diagram of a readout circuit according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a histogram according to an embodiment of the present application.

Fig. 6 is a dynamic histogram drawing time-of-flight measurement method according to an embodiment of the present application.

Fig. 7 is a time-of-flight measurement method according to an embodiment of the present application.

Fig. 8 is an interpolation-based time-of-flight measurement method according to an embodiment of the present application.

detailed description

In order to make the technical problems, technical solutions, and beneficial effects to be solved by the embodiments of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be used for fixing or circuit connection.

It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top" The orientation or positional relationship indicated by "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the application and simplifying the description, rather than indicating or implying what is meant. The device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, "multiple" means two or more than two, unless otherwise specifically defined.

This application provides a dynamic histogram drawing a flight time distance measurement method and a measurement system. For ease of understanding, an embodiment of the measurement system will be described first.

As an embodiment of the present application, a distance measurement system is provided, which has stronger resistance to ambient light and higher resolution.

Fig. 1 is a schematic diagram of a time-of-flight distance measurement system according to an embodiment of the present application. The distance measurement system 10 includes a transmitter 11, a collector 12, and a processing circuit 13. The transmitter 11 provides a emitted light beam 30 to the target space to illuminate an object 20 in the space. At least part of the emitted light beam 30 is reflected by the object 20 to form a reflected light beam. 40. At least part of the light signals (photons) of the reflected light beam 40 are collected by the collector 12. The processing circuit 13 is connected to the transmitter 11 and the collector 12 respectively, and the trigger signals of the transmitter 11 and the collector 12 are synchronized to calculate the light beam from the transmitter The time required for 11 to be emitted and received by the collector 12, that is, the flight time t between the emitted light beam 30 and the reflected light beam 40, further, the distance D of the corresponding point on the object can be calculated by the following formula:

D=c·t/2 (1)

Among them, c is the speed of light.

The transmitter 11 includes a light source 111 and an optical element 112. The light source 111 can be a light source such as a light emitting diode (LED), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), etc., or an array light source composed of multiple light sources. Optionally, the array light source 111 is in a single block A plurality of VCSEL light sources are generated on a semiconductor substrate to form a VCSEL array light source chip. The light beam emitted by the light source 111 may be visible light, infrared light, ultraviolet light, or the like. The light source 111 emits a light beam outward under the control of the processing circuit 13. For example, in some embodiments, the light source 111 emits a pulsed light beam at a certain frequency (pulse period) under the control of the processing circuit 13, which can be used in the direct time-of-flight method ( In Direct TOF measurement, the frequency is set according to the measurement distance, for example, it can be set to 1MHz-100MHz, and the measurement distance is from several meters to several hundred meters. It is understandable that a part of the processing circuit 13 or a sub-circuit independent of the processing circuit 13 may be used to control the light source 111 to emit related light beams, such as a pulse signal generator.

The optical element 112 receives the pulsed beam from the light source 111, and optically modulates the pulsed beam, such as diffraction, refraction, reflection, etc., and then emits the modulated beam into space, such as a focused beam, a flood beam, and a structured light beam. Wait. The optical element 112 may be one or more combinations of lenses, diffractive optical elements, masks, mirrors, MEMS galvanometers, and the like.

The processing circuit 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc., or a general-purpose processor. For example, when the depth camera is integrated into smart terminals such as mobile phones, TVs, and computers, The processor in the terminal can be used as at least a part of the processing circuit 13.

The collector 12 includes a pixel unit 121 and an imaging lens unit 122. The imaging lens unit 122 receives and guides at least part of the modulated light beam reflected by the object to the pixel unit 121. In some embodiments, the pixel unit 121 is composed of a single photon avalanche photodiode (SPAD), or an array pixel unit composed of multiple SPAD pixels. The array size of the array pixel unit represents the resolution of the depth camera, such as 320 ×240 etc. SPAD can respond to the incident single photon to realize the detection of single photon. Because of its high sensitivity and fast response speed, it can realize long-distance and high-precision measurement. Compared with CCD/CMOS and other image sensors based on the principle of light integration, SPAD can count single photons, such as the use of time-correlated single photon counting (TCSPC) to realize the collection of weak light signals and the calculation of flight time . Generally, connected to the pixel unit 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ). These circuits can be integrated with the pixels, and they can also be part of the processing circuit 13. For ease of description, the processing circuit 13 will be collectively regarded.

In some embodiments, the distance measurement system 10 may also include a color camera, an infrared camera, an IMU, and other devices. The combination of these devices can achieve richer functions, such as 3D texture modeling, infrared face recognition, SLAM and other functions.

In some embodiments, the transmitter 11 and the collector 12 can also be arranged in a coaxial form, that is, the two are realized by optical devices with reflection and transmission functions, such as a half mirror.

In the direct time-of-flight distance measurement system using SPAD, a single photon incident on the SPAD pixel will cause an avalanche, the SPAD will output an avalanche signal to the TDC circuit, and the TDC circuit will detect the time interval between the photon emission from the emitter 11 and the avalanche. After multiple measurements, the time interval is counted through the time-correlated single photon counting (TCSPC) circuit for histogram statistics to recover the waveform of the entire pulse signal, and the time corresponding to the waveform can be further determined, and the flight time can be determined based on this time , So as to achieve accurate flight time detection, and finally calculate the distance information of the object based on the flight time. Assuming that the pulse period emitted by the pulse beam is Δt, the maximum measurement range of the distance measurement system is Dmax, and the corresponding maximum flight time is

Generally, Δt≥t ₁ is required to avoid signal aliasing, where c is the speed of light. If the number of multiple measurements required by TCSPC is n, then the time (frame period) to achieve a single frame measurement will not be less than n*t ₁ , that is, the period of each frame measurement includes n photon counting measurements. For example, the maximum measurement range is 150m, the corresponding pulse period Δt=1us, n=100000, then the frame period will not be less than 100ms, and the frame rate will be less than 10fps. It can be seen that the maximum measurement range in the TCSPC method limits the pulse period, which further affects the frame rate of distance measurement.

Fig. 2 is a schematic diagram of a light source according to an embodiment of the present application. The light source 111 is composed of a plurality of sub-light sources arranged on a single substrate (or multiple substrates), and the sub-light sources are arranged on the substrate in a certain pattern. The substrate may be a semiconductor substrate, a metal substrate, etc., and the sub-light source may be a light emitting diode, an edge-emitting laser emitter, a vertical cavity surface laser emitter (VCSEL), etc., optionally, the light source 111 is composed of a plurality of VCSELs arranged on the semiconductor substrate. An array of VCSEL chips composed of sub-light sources. The sub-light source is used to emit light beams of any desired wavelength, such as visible light, infrared light, and ultraviolet light. The light source 111 emits light under the modulation drive of the driving circuit (which may be part of the processing circuit 13), such as continuous wave modulation, pulse modulation, and the like. The light source 111 can also emit light in groups or as a whole under the control of the driving circuit. For example, the light source 111 includes a first sub-light source array 201, a second sub-light source array 202, etc., and the first sub-light source array 201 emits light under the control of the first driving circuit. , The second sub-light source array 202 emits light under the control of the second driving circuit. The arrangement of the sub-light sources can be a one-dimensional arrangement or a two-dimensional arrangement, and can be a regular arrangement or an irregular arrangement. For ease of analysis, only one example is schematically given in Figure 2. In this example, the light source 111 is a regular array of 8×9 sub-light sources, and the sub-light sources are divided into 4×3=12 groups. Each light source is shown in the figure. It is distinguished by different symbols, that is, the light source 111 is composed of 12 3×2 regularly arranged sub-light source arrays.

Fig. 3 is a schematic diagram of a pixel unit in a collector according to an embodiment of the present application. The pixel unit includes a pixel array 31 and a readout circuit 32. The pixel array 31 includes a two-dimensional array composed of a plurality of pixels 310 and a readout circuit 32 composed of a TDC circuit 321, a histogram circuit 322, etc., where the pixel array is used for Collect at least part of the light beam reflected by the object and generate the corresponding photon signal. The readout circuit 32 is used to process the photon signal to draw a histogram reflecting the pulse waveform emitted by the light source in the transmitter. Further, it can also be based on the histogram. Calculate the flight time, and finally output the result. The readout circuit 32 may be composed of a single TDC circuit and histogram circuit, or may be an array readout circuit composed of multiple TDC circuit units and histogram circuit units.

In some embodiments, when the emitter 11 emits a spot beam to the object to be measured, the optical element 112 in the collector 12 will guide the spot beam to the corresponding pixel. Generally, in order to receive as much of the reflected beam as possible For the optical signal, the size of a single spot is set to correspond to multiple pixels (the correspondence here can be understood as imaging, and the optical element 112 generally includes an imaging lens). For example, a single spot in Figure 3 corresponds to 2×2=4 pixels, that is, the The photons reflected by the spot beam will be accepted by the corresponding 4 pixels with a certain probability. For the convenience of description, the pixel area composed of the corresponding multiple pixels is called "combined pixel" in this application. The size of the combined pixel can be based on actual conditions. Need to be set, including at least one pixel, for example, it can be 3×3, 4×4 size. Generally, the light spot is round, oval, etc., and the combined pixel size should be set to be the same as the light spot size, or slightly smaller than the light spot size, but considering the different magnifications caused by the distance of the measured object, the combined pixel The size needs to be considered comprehensively when setting.

In the embodiment shown in FIG. 3, the pixel unit 31 includes an array composed of 14×18 pixels as an example for description. Generally, the measurement system 10 between the transmitter 11 and the collector 12 can be divided into coaxial and off-axis according to the different setting modes. For the coaxial situation, the light beam emitted by the transmitter 11 will be reflected by the collector 12 after being reflected by the object to be measured. For the corresponding combined pixel collection in, the position of the combined pixel will not be affected by the distance of the measured object; but for off-axis situations, due to the existence of parallax, when the measured object is at different distances, the position of the light spot on the pixel unit is also Will change, usually along the baseline (the line between the transmitter 11 and the collector 12, the horizontal direction is used to represent the baseline direction in this application) direction, so when the distance of the measured object is unknown, it The position of the pixel is uncertain. In order to solve this problem, this application will set a pixel area (here called "super pixel") composed of multiple pixels exceeding the number of pixels in the combined pixel to receive the reflected spot light beam. When setting the size of the super pixel, it is necessary to consider the measurement range of the system 10 and the length of the baseline at the same time, so that the combined pixels corresponding to the spots reflected by objects at different distances within the measurement range will all fall into the super pixel area, namely The size of a super pixel should exceed at least one super pixel. Generally, the size of the super pixel is the same as the sum pixel along the vertical direction of the baseline, and is larger than the sum pixel along the baseline direction. The number of super pixels is generally the same as the number of spot beams collected by the collector 12 in a single measurement, which is 4×3 in FIG. 3.

In some embodiments, the super pixel is set to: when at the lower limit of the measurement range, that is, the spot falls on one side of the super pixel (left or right, depending on the relative position of the emitter 11 and the collector 12) at close range. Position); when at the upper limit of the measurement range, that is, the spot falls on the other side of the super pixel at a distance. In the embodiment shown in FIG. 3, the super pixels are set to a size of 2×6. For example, for

spots

363, 373, and 383, the corresponding super pixels are 361, 371, and 381, respectively, where

spots

363, 373, and 383 are far away. The corresponding combined pixels of the spot beams reflected by objects at, middle, and close distances fall on the left, middle, and right sides of the superpixel, respectively.

In some embodiments, the combined pixels share one TDC circuit unit, that is, one TDC circuit unit is connected to each pixel in the combined pixel. When any pixel in the combined pixel receives photons and generates a photon signal, the TDC circuit unit Both can calculate the flight time corresponding to the photon signal. This situation is more suitable for the coaxial situation, for the off-axis situation, because the combined pixel position will change with the distance of the measured object. In the embodiment shown in FIG. 3, a TDC circuit array composed of 4×3 TDC circuit units will be included.

In some embodiments, the super pixels share a TDC circuit unit, that is, a TDC circuit unit is connected to each pixel in the super pixel. When any pixel in the super pixel receives a photon and generates a photon signal, the TDC circuit unit Both can calculate the flight time corresponding to the photon signal. Since the super pixel can include the pixel shift caused by the off-axis parallax, the super pixel sharing TDC can be applied to the off-axis situation. In the embodiment shown in FIG. 3, a TDC circuit array composed of 4×3 TDC circuit units will be included.

Sharing the TDC circuit can effectively reduce the number of TDC circuits, thereby reducing the size and power consumption of the readout circuit.

For off-axis situations, more pixels need to be set to form super pixels. In a single measurement (or single exposure) time, the number of spots that can be collected will be much smaller than the number of pixels, in other words, the effective depth of the collection The resolution of the data (time-of-flight value) is much smaller than the pixel resolution. For example, the pixel resolution in Figure 3 is 14×18, and the spot distribution is 4×3, that is, the effective depth data resolution of a single frame measurement is 4× 3.

In order to improve the resolution of the measured depth data, multi-frame measurement can be used. The spots emitted by the transmitter 11 during the multi-frame measurement are "off", resulting in a scanning effect. The spots received by the collector 12 are in multiple frames. Deviations also occur in the measurement. For example, the spots corresponding to two adjacent frames of measurement in Figure 3 are 343 and 353 respectively, which can improve the resolution. In some embodiments, the “deviation” of the spots can be achieved by grouping control of the sub-light sources on the light source 111, that is, in the measurement of two or more adjacent frames, the adjacent sub-light sources are sequentially turned on, for example, in the first frame. During the measurement, turn on the first sub-light source array 201, and in the second frame of measurement, turn on the second sub-light source array 202, and so on. Not only can the horizontal grouping control but also the vertical grouping control to improve the effectiveness in the two-dimensional direction The resolution of the depth data.

For the "deviation" of the spot measured in multiple frames, the super pixels corresponding to the spots in different positions also need to be deviated when setting. As shown in Figure 3, the super pixel corresponding to the spot 343 is 341, the super pixel corresponding to the spot 353 is 351, and the super pixel is 351. 351 is laterally shifted relative to super pixel 341, and there is a partial pixel overlap between super pixel 341 and super pixel 351. For super pixels measured in multiple frames, they overlap each other. In order to ensure that the TDC circuit can be used every time The frame can accurately measure the photon counting flight time of the corresponding super pixel. This application provides a solution for dual sharing of the TDC circuit.

In some embodiments, the pixel area connected by a single TDC circuit unit will include the area composed of all super pixels that deviate in the multi-frame measurement, and there is overlap between the pixel areas corresponding to two adjacent TDC circuit units. Specifically, in the embodiment shown in FIG. 3, the pixel area 391 shares a TDC circuit unit, and the pixel area 391 includes 6 superpixels corresponding to 6 frames of measurement when the 6 groups of sub-light sources are turned on in sequence. Similarly, adjacent pixel regions 392 share a TDC circuit unit, and there is a partial overlap between the two

pixel regions

391 and 392, which results in some pixels connected to two TDC circuit units. In a single frame measurement, according to For the projected spots, the processing circuit 13 will gate the corresponding pixels so that the acquired photon signals can be measured by a single TDC circuit unit, so as to avoid crosstalk and errors. In some embodiments, the number of TDC circuits is the same as the number of spots collected by the collector 12 during a single frame measurement. In Figure 3, there are 4×3. Each shared TDC circuit is connected to 4×10 pixels, adjacent to each other. There are 4×4 pixels overlap between the pixel areas connected by the TDC circuit unit.

The following describes the adjustable histogram circuit scheme. In a single frame measurement period, the TDC circuit will receive the photon signal from the pixel in the super pixel area connected to it, and calculate the time interval between the signal and the starting clock signal (ie, flying Time), and convert the time interval into a temperature code or binary code and save it in the histogram circuit. After multiple measurements, the histogram circuit can draw a histogram that reflects the pulse waveform. Based on the histogram, the pulse can be accurately obtained Flight time. Generally, the larger the measurement range, the wider the measurable time interval of the TDC circuit is required. In addition, if the accuracy requirement is higher, the higher the time resolution of the TDC circuit is required, whether it is the wider the time interval or the higher the time resolution. The TDC circuit is required to use a larger scale to output a binary code with a larger number of digits. Due to the increase of the number of binary codes, the storage capacity of the memory of the histogram circuit is higher. The larger the memory capacity, the higher the cost, and the greater the difficulty in mass production of monolithic integration. For this reason, the present application provides a readout circuit solution with adjustable histogram circuit.

Fig. 4 is a schematic diagram of a readout circuit according to an embodiment of the present application. The readout circuit includes a TDC circuit 41 and a histogram circuit 42. The TDC circuit 41 collects the time interval of the photon signal and converts the time interval into a time code (binary code, temperature code, etc.), and then the histogram circuit 42 will be based on this The time code is counted on the corresponding internal time unit (that is, the storage unit used to save time information), such as adding 1, after multiple measurements, the photon counts in all time units can be counted and the time histogram can be drawn Figure. The histogram drawn is shown in Figure 5, where ΔT refers to the width of the time unit, T1 and T2 respectively refer to the start and end time of the histogram drawing, [T1, T2] is the time interval of the histogram, T =T2-T1 refers to the total time width. The ordinate of the time unit ΔT is the photon count value stored in the corresponding storage unit. Based on the histogram, the pulse waveform position can be determined by the highest peak method and other methods, and get The corresponding flight time t.

In some embodiments, the histogram circuit 42 includes an address decoder 421, a memory matrix 422, a read/write circuit 424, and a histogram drawing circuit 425. The TDC circuit inputs the acquired time code (binary code, temperature code, etc.) that reflects the time interval to the address decoder 421, and the address decoder 421 converts it into address information, which will be stored in the storage matrix 422 in. Specifically, the storage matrix 422 includes a plurality of storage units 423, that is, time units. Each storage unit 423 is pre-configured with a certain address (or address interval). When the time code address received by the address decoder 421 is equal to a certain When the address of the storage unit is the same or within the address range of the storage unit, the read/write circuit 424 will perform the +1 operation on the corresponding storage unit, that is, complete one photon count, and after multiple measurements, the value of each storage unit The data reflects the number of photons received during the time interval. After a single frame measurement (multiple measurements), the data of all memory cells in the memory matrix 422 are read out to the histogram drawing circuit 425 for histogram drawing.

In order to reduce the storage capacity of the storage matrix as much as possible, in fact, it is necessary to reduce the number of storage units 423. For this reason, in this application, a control signal is applied to the histogram circuit 42 through the processing circuit to dynamically set the address (address interval) of each storage unit 423. , Thereby further realizing the dynamic control of the histogram time resolution ΔT and/or the time interval width T. For example, under the premise that the number of storage units 423 remains unchanged, by setting the address interval corresponding to the storage unit 423 to a larger time interval, that is, increasing the width ΔT of the time unit, the time interval interval that the total storage matrix can store will change. Larger, the total time interval of the histogram will become larger. For ease of description, the histogram with a larger time interval is called a thick histogram; for example, the address interval corresponding to the storage unit 423 can be set to a smaller time interval. , The time interval interval that the total storage matrix can store is reduced, but the time resolution of storage will increase, and the time resolution of the histogram will increase. Compared with the coarse histogram, the histogram with a smaller time interval is called Is a fine histogram.

In this application, a large-scale and high-precision time-of-flight measurement will be realized by dynamically coarse-fine adjustment of the histogram during the time-of-flight measurement process.

Fig. 6 is a dynamic histogram drawing time-of-flight measurement method according to an embodiment of the present application. It includes the following steps:

Step 1: Draw a coarse histogram in coarse precision time units. That is, the address or address interval corresponding to each time unit in the storage matrix 422 is configured by applying a control signal, that is, T and ΔT are set, and ΔT is configured as a larger time interval ΔT ₁ in this step. Generally, the setting of the histogram time interval T needs to consider the measurement range. The time interval ΔT ₁ should be set in consideration of the measurement range and the number of histogram storage units, that is, the flight time corresponding to the measurement range is allocated to all the histogram storage units. , Such as equal distribution or non-equal distribution, etc., so that all storage units can cover the measurement range. After a number of measurements, the flight time value obtained from each measurement is matched to perform an operation of adding 1 to the corresponding time unit, and finally the drawing of the rough histogram is completed.

Step 2. Use the rough histogram to calculate the rough flight time value t _1. Based on the rough histogram, the maximum peak value method can be used to find the pulse waveform position, and the corresponding flight time can be read as the rough flight time value t ₁ . The accuracy or minimum resolution of the time value is the time interval ΔT ₁ of the time unit.

When the measurement range is relatively large and the number of storage units is limited, ΔT ₁ will be relatively large. When the number of photons is large, the pulsed photons will be submerged in the background light, making it impossible to detect the pulse waveform. Therefore, in some embodiments, the measurement range may be divided into several intervals, and each interval corresponds to a respective flight time interval, and the time interval ΔT of each time interval T may be the same or different. When drawing a rough histogram, you can draw each time interval one by one. Since the distance of the measured object is unknown, the time interval in which the corresponding flight time will fall is also unknown, so it may be in a certain time interval. When drawing the rough histogram, the pulse waveform cannot be detected, that is, the rough flight time value cannot be calculated. In this case, for example, when the waveform position cannot be found based on the rough histogram in step 2, then go back to step 1 for the next step The rough histogram is drawn until the pulse waveform is found in the rough histogram. Of course, it may also be impossible to find the pulse waveform due to errors or the object distance is too far. In order to avoid the problem of continuous cycle detection, you can set the number of cycles, for example, when the number of rough histogram drawing times exceeds a certain threshold (such as 3 times) , It is considered that the target is not detected this time, or that the target is located at infinity this time, thus ending the measurement.

Step 3: According to the obtained rough flight time value, draw a fine histogram in fine time units. At this time, since the rough value of the flight time value is already known, one more round of measurement can be performed and the corresponding histogram is drawn. At this time, the histogram circuit is controlled by the control signal and corresponds to each time unit in the storage matrix 422. The address or address interval is configured as a smaller time interval ΔT ₂ . Generally, the time interval ΔT ₂ only needs to correspond to a smaller measurement range interval that can contain the true flight time value and the number of histogram storage units when setting it. The measurement range interval can be set as a rough flight time value In the middle, a certain margin is added on both sides, for example, it can be set to [t ₁ -T',t ₁ -T'], where the smaller T'is set, the smaller the time interval ΔT ₂ , and the higher the resolution. For example, in some In the embodiment, it can be set as T'=5%T, so the sum of the time intervals of all time units is only 10% of the time interval corresponding to the rough histogram. In other embodiments, the ratio of the margin to the time interval of the coarse histogram may be set in the range of 1%-25%. Then a new round of multiple measurements is performed, and the flight time value obtained each time is matched and the corresponding time unit is increased by 1 to complete the drawing of the fine histogram.

Step 4. Use the fine histogram to calculate the fine flight time t _2. Based on the fine histogram, the maximum peak method can be used to find the waveform position, and the corresponding flight time can be read as the fine flight time value t ₂ . The accuracy or minimum resolution is the time interval ΔT ₂ of the time unit. If the setting of T'=5%T in the third step is used to illustrate, the accuracy of the fine flight time is increased by 10 times compared with the rough flight time (the minimum resolution is increased by 10 times).

The measurement method of the above-mentioned histogram dynamic coarse and fine adjustment is essentially a process of first performing coarse positioning in a larger measurement range, and then performing fine measurement based on the positioning result. It is understandable that the above-mentioned coarse and fine adjustment method can also be extended to three or more steps of measurement. For example, in some embodiments, the first time resolution is first measured to obtain the first flight time, and then based on The first flight time is measured with the second time resolution to obtain the second flight time, and finally based on the second flight time with the third time resolution, the third flight time is obtained. The accuracy of the three times is increased successively, and finally a higher-precision measurement can be achieved.

In some embodiments, since the histogram only counts the flight time value within the time interval T when the histogram is drawn, each pixel in the measurement system collector 12 can be activated within a specified time interval (enable ), thereby reducing power consumption, the specified time interval generally includes the time interval T drawn by the histogram. For example, when the time interval of the histogram is [3ns, 10ns], the time interval during which the pixel is activated can be set to [2.5ns, 10.5ns].

It is understandable that the above-mentioned measurement method is not only applicable to the coaxial distance measurement system, but also applicable to the off-axis measurement system. In particular, it should be noted that for the off-axis measurement system including the collector shown in FIG. 3, the histogram dynamic adjustment scheme can be further used for super pixel positioning not only to improve accuracy but also to reduce power consumption. Fig. 7 is a time-of-flight measurement method according to another embodiment of the present application. The following will be described with reference to Figure 3. The method includes the following steps:

Step 1: Receive the super pixel TDC output signal, and draw a coarse histogram in coarse-precision time units. Since the distance of the object is not clear before the measurement, the position of the spot cannot be determined, that is, the position of the combined pixel cannot be determined, and the combined pixel may fall into different positions of the super pixel according to the distance of the object. Therefore, in this step, firstly, each pixel in the super pixel is enabled to be in an activated state to receive photons, and the photon signal output by the shared TDC of the super pixel is received, and then the histogram is drawn. The histogram adopts the dynamic adjustment histogram scheme shown in Fig. 6, in this step, the coarse histogram will be drawn using the coarse precision time unit.

Step 2. Use the rough histogram to calculate the rough flight time value t _1. Based on the rough histogram, use the maximum peak method to find the waveform position, and read the corresponding flight time as the rough flight time value t ₁ , the flight time The accuracy or minimum resolution of the value is the time interval ΔT ₁ of the time unit.

When the measurement range is relatively large and the number of storage units is limited, ΔT ₁ will be relatively large. When the number of photons is large, the pulsed photons will be submerged in the background light, resulting in the impulse waveform cannot be detected. Therefore, in some embodiments, the measurement range may be divided into several intervals, and each interval corresponds to a respective flight time interval, and the time interval ΔT of each time interval T may be the same or different. When drawing a rough histogram, you can draw each time interval one by one. Since the distance of the measured object is unknown, it is also unknown which time interval the corresponding flight time will fall into, so it may be in a certain time interval. The pulse waveform cannot be detected when drawing the rough histogram. For this case, for example, when the waveform position cannot be found based on the rough histogram in step 2, then go back to step 1 to draw the next rough histogram until the rough histogram Until the pulse waveform is found in the figure. Of course, it may also be impossible to find the pulse waveform due to errors or the object distance is too far. In order to avoid the problem of continuous cycle detection, you can set the number of cycles, for example, when the number of rough histogram drawing times exceeds a certain threshold (such as 3 times) , It is considered that the target is not detected this time, or that the target is located at infinity this time, thus ending the measurement.

Step 3: According to the obtained rough time-of-flight value, locate the combined pixels and draw a fine histogram in fine time units. Since the rough time-of-flight value has been clarified, it can be based on the rough time-of-flight value and the parallax to locate the position of the combined pixel. It is usually necessary to save the relationship between the position of the combined pixel and the rough time-of-flight value in the system in advance After obtaining the rough time-of-flight value, the position of the combined pixel can be directly located according to the relationship; then based on the position of the combined pixel, only the combined pixel is activated, and a fine histogram is drawn in fine time units. Since the rough value of the flight time value is already known, you can perform multiple measurements and draw the corresponding histogram. At this time, the histogram circuit is controlled by the control signal and stores the address or address corresponding to each time unit in the matrix 422. The address interval is configured as a small time interval ΔT ₂ . Generally, the time interval ΔT ₂ only needs to correspond to a smaller measurement range interval that can contain the true flight time value and the number of histogram storage units when setting it. The measurement range interval can be set as a rough flight time value In the middle, a certain margin is added on both sides, for example, it can be set to [t ₁ -T',t ₁ -T'], where the smaller T'is set, the smaller the time interval ΔT ₂ , and the higher the resolution. For example, in some In the embodiment, it can be set as T'=5%T, so the sum of the time intervals of all time units is only 10% of the time interval corresponding to the rough histogram. In other embodiments, the ratio of the margin to the time interval of the coarse histogram may be set in the range of 1%-25%. Then a new round of multiple measurements is performed, and the flight time value obtained each time is matched and the corresponding time unit is increased by 1 to complete the drawing of the fine histogram.

The following describes the time-of-flight measurement method based on interpolation. The embodiments of Fig. 2 and Fig. 3 introduce examples of improving the resolution through multi-frame measurement. It is understandable that when performing multi-frame measurement, the depth data of each frame is measured Both can use the histogram dynamic adjustment scheme shown in Fig. 6 or Fig. 7. For example, when the first sub-light source array 201 is turned on, the dynamic coarse-fine histogram is drawn to obtain the first frame of depth image; when the second sub-light source array 202 is turned on, the dynamic coarse-fine histogram is drawn to obtain the second frame of depth image; The first and second frames of depth images get higher resolution depth images. In some embodiments, depth images of more than 3 frames can also be collected and merged into a higher resolution depth image.

However, if each frame of depth image acquisition requires dynamic adjustment of the thickness, the acquisition time of each high-resolution fusion depth image will be relatively long, and the overall frame rate will not be high. In order to increase the frame rate as much as possible, this application provides an interpolation-based time-of-flight measurement method according to an embodiment of the application as shown in FIG. 8. The method includes the following steps:

Step 1: Obtain the first flight time of the first combined pixel corresponding to the first light source. In this step, the first light source in the transmitter 11 is turned on to emit the spot beam corresponding to the first light source, and the spot beam will fall on the closed pixel on the pixel unit 31 in the collector 12, as shown in 4 in Figure 3 The spot represented by the solid circle ×3 is taken as an example. The processing circuit can further obtain the first flight time of the combined pixel. For example, the coarse-fine dynamic adjustment scheme in the embodiment shown in FIG. 6 or FIG. 7 or any other method can be used. The solution can obtain the fine flight time (first flight time) of the combined pixel.

Step 2: Obtain the second flight time of the second superpixel corresponding to the second light source through interpolation calculation. When the second light source is turned on, a spot beam adjacent to the spot beam corresponding to the first light source will be emitted, and the spot beam will also fall on the combined pixel of the collector 12. For ease of illustration, only used in Figure 3 The dotted circle draws a spot 353. The spot 353 and the spot 343 are spatially staggered because the positions of the first light source and the second light source are staggered, and therefore the respective corresponding pixels are also staggered. Generally, when the space points are relatively close, the distance between the two points will not differ too much. Therefore, in some embodiments, the flight time value of the corresponding pixel corresponding to the spot 343 obtained in step 1 can be used as the second flight time value (rough flight time) of the super pixel 351 corresponding to the spot 353, and then the fine flight time is performed. Calculation. In some embodiments, the second time-of-flight value of the superpixel of the spot 353 can be estimated by using the combined pixels corresponding to multiple first light sources around the spot 353, for example, using the time-of-flight values of the left and right combined pixels for interpolation. The interpolation may be one-dimensional interpolation or two-dimensional interpolation, and the interpolation method may be at least one of interpolation methods such as linear interpolation, spline interpolation, and polynomial interpolation.

Step 3: According to the second flight time, locate the second composite pixel corresponding to the second light source and draw a histogram. After the second flight time is obtained by interpolation, based on the flight time and the parallax, the position of the spot in the superpixel, that is, the position of the combined pixel, can be located, and then based on the position of the combined pixel, only the combined pixel is activated, and at the same time Plot histograms in fine time units.

Step 4. Use the histogram to calculate the third flight time. Based on the histogram, the maximum peak value method can be used to find the waveform position, and the corresponding flight time can be read as the third (fine) flight time value t ₂ , the flight time value The accuracy or minimum resolution of is the time interval ΔT ₂ of the time unit.

Compared with the method described in Figure 6 or Figure 7, the flight time measurement method in the above steps only needs to use the coarse-fine histogram drawing method for the flight time calculation of only a few spots, and at least 2 frames of flight time measurement are required to obtain high accuracy. The flight time value of most spots can be calculated by using the flight time value of known spots as the rough flight time value of the coarse histogram, and based on the rough flight time value, only a single fine histogram drawing is required. However, this can greatly improve efficiency. For example, if the light sources are divided into 6 groups, only the first group of light sources need to perform coarse-fine measurement when the light source is turned on, and the subsequent 5 groups only need to perform a single fine measurement when the flight time measurement is performed after the light source is turned on.

In some embodiments, considering that the surface of the object to be measured often has jumps, that is, the distance difference is large, it is difficult to obtain an accurate time-of-flight value by interpolation. Therefore, performing a fine measurement based on the result of the interpolation may cause errors. Therefore, a judgment can be made before the interpolation in step 2. For example, when the flight time values of the combined pixels corresponding to the multiple spots to be interpolated (such as the left and right spots) are greater than a certain threshold, the difference between the two spots on the surface There is a jump in the surface depth value of the object between the two spots, the spots between the two spots will still maintain the measurement scheme of the thick-fine histogram drawing, as long as the interpolation calculation is performed when it is less than the subtraction value.

In some embodiments, the first time of flight of the first combined pixel may also be the rough time of flight, that is, when the first time of flight of the first combined pixel is demodulated and calculated, only a single rough histogram drawing is required, and then the Interpolation is performed based on the rough flight time obtained by the rough histogram drawing.

It is understandable that when the distance ranging system of this application is embedded in a device or hardware, corresponding structural or component changes will be made to adapt to requirements. The essence of this application is still the distance ranging system of this application, so it should be regarded as this application. The scope of protection. The above content is a further detailed description of the application in combination with specific/preferred implementations, and it cannot be determined that the specific implementation of the application is limited to these descriptions. For those of ordinary skill in the technical field to which this application belongs, without departing from the concept of this application, they can also make several substitutions or modifications to the described implementations, and these substitutions or modifications should be regarded as It belongs to the protection scope of this application.

In the description of this specification, reference to the description of the terms "one embodiment", "some embodiments", "preferred embodiment", "examples", "specific examples", or "some examples" etc. means to incorporate the implementation The specific features, structures, materials, or characteristics described by the examples or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner.

In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification without contradicting each other. Although the embodiments of the present application and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope defined by the appended claims. In addition, the scope of this application is not intended to be limited to the specific embodiments of the processes, machines, manufacturing, material composition, means, methods, and steps described in the specification. A person of ordinary skill in the art will easily understand that the above-mentioned disclosures, processes, machines, processes, machines, processes that currently exist or will be developed later that perform substantially the same functions as the corresponding embodiments described herein or obtain substantially the same results as the embodiments described herein can be used. Manufacturing, material composition, means, method, or step. Therefore, the appended claims intend to include these processes, machines, manufacturing, material compositions, means, methods, or steps within their scope.

Claims

A dynamic histogram drawing time-of-flight distance measurement method, which is characterized in that it comprises the following steps:

S1. Receive the super pixel TDC output signal, and draw the first histogram with the first precision time unit;

S2. Calculate the first flight time by using the first histogram;

S3. According to the first flight time, locate the combined pixel and draw a second histogram with a second precision time unit; the combined pixel is composed of at least one pixel, and the super pixel includes at least one of the combined pixel;

S4. Calculate the second flight time by using the second histogram.
The method for measuring a time-of-flight distance according to claim 1, wherein the addresses of the first precision time unit and the second precision time unit are pre-configured so that the time interval of the first histogram is greater than The time interval of the second histogram; or, making the time resolution of the first histogram smaller than the time resolution of the second histogram.
5. The flight time distance measurement method according to claim 1, wherein the pulse waveform position is located by the maximum peak method based on the first or second histogram and the first or second flight time is calculated.
5. The time-of-flight distance measurement method according to claim 1, wherein when the second histogram is drawn, only the pixels in the corresponding pixels are activated.
The method for measuring flight time distance according to claim 1, characterized in that: when the first flight time cannot be calculated by using the first histogram in step S2, return to step S1 to continue drawing the first straight square. Figure until the first flight time is calculated.
5. The flight time distance measurement method of claim 1, wherein the time interval of the first histogram is determined according to the measurement range and the number of time units of the histogram; the time interval of the second histogram It is determined according to the first flight time.
7. The flight time distance measurement method according to claim 6, wherein the time interval of the second histogram is determined by taking the first flight time as the middle and adding a certain margin on both sides.
7. The flight time distance measurement method according to claim 7, wherein the margin is set to 1%-25% of the time interval of the first histogram.
The method for measuring flight time distance according to claim 1, wherein the relationship between the position of the combined pixel and the first flight time is pre-stored, so as to realize that after acquiring the first flight time according to The relationship locates the combined pixel.
The time-of-flight distance measurement method according to claim 1, wherein when the first histogram or the second histogram is drawn, the corresponding pixel only includes the first histogram when collecting photons. Or the second histogram is activated within the time range of the time interval.
A dynamic histogram drawing time-of-flight distance measurement system, which is characterized in that it comprises:

Emitter, configured to emit a pulsed beam;

A collector configured to collect photons in the pulsed beam reflected back by an object and form a photon signal;

The processing circuit is connected to the transmitter and the collector, and is used to perform the following steps:

S1. Receive the super pixel TDC output signal, and draw the first histogram with the first precision time unit;

S2. Calculate the first flight time by using the first histogram;

S3. According to the first flight time, locate the combined pixel and draw a second histogram with a second precision time unit; the combined pixel is composed of at least one pixel, and the super pixel includes at least one of the combined pixel;

S4. Calculate the second flight time by using the second histogram.
The dynamic histogram drawing time-of-flight distance measurement system of claim 11, wherein the addresses of the first precision time unit and the second precision time unit are pre-configured so that the first histogram The time interval of is greater than the time interval of the second histogram; or, the time resolution of the first histogram is made smaller than the time resolution of the second histogram.
The dynamic histogram drawing time-of-flight distance measurement system of claim 11, wherein the pulse waveform position is located by the maximum peak method based on the first or second histogram, and the first or second histogram is calculated. flight duration.
The dynamic histogram drawing time-of-flight distance measurement system according to claim 11, wherein when the first flight time cannot be calculated using the first histogram in step S2, return to step S1 to continue drawing The first straight graph until the first flight time is calculated.
The dynamic histogram drawing time-of-flight distance measurement system according to claim 11, wherein the time interval of the first histogram is determined according to the measurement range and the number of time units of the histogram; the second histogram The time interval of the graph is determined according to the first flight time.
15. The dynamic histogram drawing flight time distance measurement system according to claim 15, wherein the time interval of the second histogram is determined by taking the first flight time as the middle and adding a certain margin on both sides.
The dynamic histogram drawing time-of-flight distance measurement system according to claim 11, wherein the relationship between the position of the combined pixel and the first time-of-flight is pre-stored, so as to realize the acquisition of the first time After the flight time, the combined pixel is located according to the relationship.
The dynamic histogram drawing time-of-flight distance measurement system according to claim 11, wherein when the first histogram or the second histogram is drawn, the corresponding pixel only includes the first histogram when collecting photons. The histogram or the second histogram is activated within the time range of the time interval.