CN115616608A - Single-photon three-dimensional imaging distance super-resolution method and system - Google Patents

Single-photon three-dimensional imaging distance super-resolution method and system Download PDF

Info

Publication number
CN115616608A
CN115616608A CN202211631142.9A CN202211631142A CN115616608A CN 115616608 A CN115616608 A CN 115616608A CN 202211631142 A CN202211631142 A CN 202211631142A CN 115616608 A CN115616608 A CN 115616608A
Authority
CN
China
Prior art keywords
time
resolution
histogram
photon
dependent
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211631142.9A
Other languages
Chinese (zh)
Other versions
CN115616608B (en
Inventor
殷浩蒙
赵惠
杨明洋
刘永安
盛立志
夏思宇
樊学武
杨向辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
XiAn Institute of Optics and Precision Mechanics of CAS
Original Assignee
XiAn Institute of Optics and Precision Mechanics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by XiAn Institute of Optics and Precision Mechanics of CAS filed Critical XiAn Institute of Optics and Precision Mechanics of CAS
Priority to CN202211631142.9A priority Critical patent/CN115616608B/en
Publication of CN115616608A publication Critical patent/CN115616608A/en
Application granted granted Critical
Publication of CN115616608B publication Critical patent/CN115616608B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • G01S7/4865—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

The invention belongs to the field of distance measurement, and particularly relates to a distance super-resolution method and system for single-photon three-dimensional imaging. The method solves the problems of high implementation difficulty and high cost of the conventional single-photon three-dimensional imaging algorithm due to the fact that the distance measurement precision and the distance resolution are limited by a hardware timing module. According to the method, a series of single photon detector detection distance gating threshold starting times with sub-time resolution intervals are set, and a plurality of original low-resolution time-dependent photon counting histograms are obtained for each pixel; then, sampling, registering and adding a plurality of original low-resolution time-dependent photon counting histograms to obtain a high-resolution time-dependent photon counting histogram; and finally, estimating the flight time of the echo photons from the high-resolution time-dependent photon counting histogram, and further obtaining a target three-dimensional imaging distance map. Because the obtained histogram contains more fine change information, the distance resolution capability of the system breaks through the limitation of hardware, and the ranging precision is greatly improved.

Description

Single-photon three-dimensional imaging distance super-resolution method and system
Technical Field
The invention belongs to the field of distance measurement, and particularly relates to a distance super-resolution method and system for single-photon three-dimensional imaging.
Background
The single photon detector has ultrahigh sensitivity and can respond to a single photon, and can realize three-dimensional imaging with ultra-long distance and ultra-sensitivity by matching with a low-power high-repetition-frequency narrow-pulse laser and a medium-caliber optical receiver, so that the single photon detection technology becomes a research hotspot in recent years. However, the technology is proposed and developed so far, the imaging performance of the system is not enough to meet the requirements of practical application, and especially, the minimum resolvable distance and the ranging accuracy of the system are regarded as the most interesting performance, so that how to effectively improve the distance resolution and the ranging accuracy of the single photon detection system becomes a very interesting problem at present.
The existing Photon counting three-dimensional imaging algorithm mainly aims at denoising sparse reconstruction technology and spatial Super-resolution technology under low signal-to-noise ratio and weak light, although the schemes can also improve the reconstruction result and reduce the distance measurement error, such as 3D deconvolution (LI, z. P., et al (2020) 'Super-resolution single-Photon imaging at 8.2 kilometers.' Opt Express 28 (3): 4076-4087.) proposed by ZHENG-PING LI et al in 2020 can reconstruct three-dimensional target elevation distribution under low signal-to-back ratio and sparse echo and has smaller distance measurement error compared with the traditional schemes (D. Shin, et. 'Photon-effective imaging with a single-Photon camera.' nat. Com. 7 (1), 12046 (2016).) but the scheme is limited by hardware precision and distance measurement resolution, and also needs to be realized by a high-precision timing module, so that the distance measurement error is difficult to realize.
Disclosure of Invention
The invention aims to provide a distance super-resolution method and system for single-photon three-dimensional imaging, which solve the problems of high implementation difficulty and high cost of the conventional single-photon three-dimensional imaging algorithm due to the fact that the distance measurement precision and the distance resolution are limited by a hardware timing module. The invention obtains the photon time distribution histogram of sub-time resolution by setting a series of single photon detector detection distance gating threshold start time with sub-time (in the invention, the time length smaller than the minimum time resolution of the single photon detector is defined as the sub-time) resolution interval, further obtains the sub-time resolution scale change of the echo photon flight time, breaks through the limitation of a hardware timing module on the distance measurement precision and the distance resolution, and can obtain higher distance measurement precision and distance resolution breaking through the system limitation in the same imaging environment and the same exposure time.
The technical scheme of the invention is as follows:
a distance super-resolution method for single photon three-dimensional imaging is characterized by comprising the following steps:
step 1, setting the starting time of the detection range gating threshold of the single-photon detector as
Figure DEST_PATH_IMAGE001
In whichi=1，
Figure DEST_PATH_IMAGE002
For sub-time-resolved delay scales, values less thanT bin ，T bin The minimum time resolution capability of the single-photon detector; receiving echo photon point clouds to obtain an original low-resolution time-dependent photon counting histogram corresponding to each pixel;
step 2, judgmentiWhether or not equal toNIf yes, executing step 3, otherwise, returning to step 1 to orderi=i+1, up toiIs equal toN(ii) a WhereinNIn order to sub-time-resolve the number of translations,is an integer of 2 or more, and,
Figure 183006DEST_PATH_IMAGE003
obtaining a corresponding pixel at each pixelNA raw low-resolution time-dependent photon count histogram;
step 3, corresponding to each pixelNThe original low-resolution time-related photon counting histogram is respectively up-sampled and obtained on each pixelNA pseudo high resolution time dependent photon count histogram;
step 4, for each pixel obtained in step 3NA pseudo high resolution time-dependent photon count histogram toTS 1 Registering and adding the reference on a time axis to obtain a high-resolution time-dependent photon counting histogram of the corresponding pixel;
and 5, obtaining a target three-dimensional imaging distance map based on the high-resolution time-dependent photon counting histogram of each pixel.
Further, the specific steps of step 4 for each pixel are:
step 4.1, defineNNumber of each time slot is
Figure DEST_PATH_IMAGE004
All zero histogram of (H0) (H)i) As an initial high resolution histogram; whereiniIs 1 toNThe number of the integer (c) of (a),Ntcounting the number of time slots of the original low-resolution time-dependent photon counting histogram;
step 4.2, the detection range gating threshold starting time of the single photon detector isTS 1 Assigning corresponding pseudo high-resolution time-dependent photon count histogram to all-zero histogram H0 (1)
Figure 924566DEST_PATH_IMAGE005
Time slot, after which, the single photon detector detection range gating threshold starts at timeTS 2 Assigning corresponding pseudo high-resolution time-dependent photon count histogram to all-zero histogram H0 (2)
Figure DEST_PATH_IMAGE006
Time slot, and so on, the detection range gating threshold of the single photon detector is started at the timeTS N Assigning the corresponding pseudo high resolution time-dependent photon count histogram to the all-zero histogram H0: (N) Is
Figure 542629DEST_PATH_IMAGE007
Time slot, thus obtainingNA registered pseudo high-resolution time-dependent photon count histogram;
step 4.3, mixingNAnd adding the registered pseudo high-resolution time-dependent photon counting histograms to obtain a high-resolution time-dependent photon counting histogram of the pixel.
Further, the specific steps of step 3 for each pixel are:
step 3.1, define a time slot number as
Figure DEST_PATH_IMAGE008
The empty histogram Tsbin of;
step 3.2, the detection range gating threshold starting time of the single photon detector isTS 1 Corresponding primary low resolution time-dependent photon count histogramjPhoton number R corresponding to each time slot j Assigned to the empty histogram Tbin respectively
Figure 48959DEST_PATH_IMAGE009
To
Figure DEST_PATH_IMAGE010
In totalNPhotons corresponding to a time slot, whereinj=1；
Step 3.3, judgmentjWhether or not to be equal toNtIf yes, executing step 3.4, otherwise, returning to step 3.2 to orderj=j+1, up tojIs equal toNtObtaining the pixel atTS 1 A corresponding pseudo high resolution time dependent photon count histogram;
step 3.4, for the rest of the pixelsN-1The original low-resolution time-dependent photon count histograms are obtained at the pixel by performing the operations of steps 3.1 to 3.3NPseudo high resolution time dependent photon countingA histogram.
Further, step 5 specifically comprises:
step 5.1, extracting a maximum probability time-of-flight time slot position from the high-resolution time-dependent photon counting histogram obtained in the step 4 by adopting a three-dimensional reconstruction method, and obtaining the echo photon time-of-flight;
and 5.2, obtaining the target distance corresponding to each pixel by utilizing the relationship between the flight time and the distance of the echo photons, wherein the target distances corresponding to all the pixels form a target three-dimensional imaging distance map.
Further, in step 5, the three-dimensional reconstruction method is a maximum likelihood estimation method, TV sparse reconstruction, or 3D deconvolution reconstruction, or the like.
The invention also provides a single photon three-dimensional imaging distance super-resolution system, which comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that: and when the computer program runs in a processor, executing the steps of the single-photon three-dimensional imaging distance super-resolution method.
The invention has the beneficial effects that:
1. through the delay of the sub-time resolution scale, compared with the traditional reconstruction method, the method has better distance resolution capability under the same imaging environment and the same exposure time and exceeds the minimum distance resolution capability of the system; and secondly, because the method can improve the minimum distance resolution capability of the system, the measurement accuracy is greatly improved, and the distance measurement error is further reduced, the root mean square error of the reconstructed result distance measurement is also greatly reduced.
2. The method can be independently applied, can be combined with the existing denoising sparse reconstruction method and other methods for use, keeps the advantages of the original algorithm under the condition that the imaging condition and the system are not changed, and can increase the distance resolution capability of the system and reduce the distance measurement error.
Drawings
FIG. 1 is a block diagram of a prior art photon counting three-dimensional imaging system;
FIG. 2 is a flow chart of the distance super-resolution method of single photon three-dimensional imaging of the present invention;
FIG. 3 is a graph of an original low-resolution time-dependent photon count histogram and a reconstructed high-resolution time-dependent photon count histogram of the present invention for different pulse widths;
FIG. 4 shows different pulse widths P w The minimum time slot is 1ns, the sub-time resolution translation is 0.01ns, the reconstructed high-time resolution correlation photon counting histogram and the minimum time slot are 0.01ns, and a result graph of the change of the centroid offset of the histogram acquired without the sub-time resolution translation along with the sub-time resolution translation times is obtained;
FIG. 5 is a graph of the results of a sub-time-resolved translation simulation using Middlebury volume datasets in accordance with the present invention; wherein a is real three-dimensional distribution, b is a reconstruction result graph adopting the existing maximum likelihood estimation, and c is a reconstruction result graph adopting the method of the invention;
FIG. 6 is a graph of the range-resolved target sub-time-resolved translation simulation results constructed in accordance with the present invention; wherein a is an original three-dimensional graph, b is a simulation result by adopting the existing maximum likelihood estimation, and c is a simulation result graph by adopting the method of the invention;
FIG. 7 is a graph of a sub-time resolved translation simulation result of a target when the integration time constructed by the present invention is a variable; wherein a is an initial three-dimensional graph, and b is a simulation result graph under the condition of sparse echo; and c is a simulation result diagram under the condition of strong echo.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The single photon three-dimensional imaging distance super-resolution method is realized based on a conventional photon counting three-dimensional imaging system and generally comprises a laser emission module, a single photon high-sensitivity receiving module, a distance gating module, a time-dependent photon counting module and a three-dimensional reconstruction module. As shown in fig. 1, the laser emitting module includes a high-repetition-frequency laser, a laser emitting optical system (collimating and expanding), and a light splitting module, and is used for providing a reliable and stable high-repetition-frequency pulsed optical signal to the system. The single-photon high-sensitivity receiving module comprises a receiving optical system and a single-photon detector, wherein the receiving optical system is responsible for collecting laser reflected by a target and coupling the laser to the single-photon detector; the single photon detector detects whether each pixel in the timeAnd detecting photons, and recording the arrival time of the echo photons if the echo photons exist. The range gating module is used for controlling the starting working time of the single photon detectorTSAnd detection range threshold gating widthTGTo reduce the false alarm probability. And the time-dependent photon counting module is used for receiving the feedback of the single photon detector, accumulating a plurality of pulse periods and obtaining a time-dependent photon counting histogram of the echo photons in the measuring period. The three-dimensional reconstruction module is used for estimating the target distance and the reflectivity according to the time-dependent photon counting histogram obtained by the time-dependent photon counting module, and meanwhile, a better three-dimensional reconstruction effect can be obtained by properly adopting a denoising algorithm and a sparse reconstruction algorithm according to the actual situation.
The traditional reconstruction method starts from the distance gating threshold of the single-photon detector in each pulse period in the integration time of the single-photon detectorTSAnd detection range threshold gating widthTGAre all fixed, and the minimum time resolution capability can be obtained by the time-dependent photon counting moduleT bin And then a three-dimensional reconstruction algorithm is adopted to obtain a target three-dimensional imaging distance map. Different from the traditional reconstruction method, the invention proposes to divide the original single-frame imaging accumulated time T, namely the single-photon detector integrated time T intoNA cumulative time period (orNDefined as the number of sub-time-resolved shifts), whereinNIs an integer greater than or equal to 2, each accumulated period length being T-NSetting the starting time of the single photon detector distance gating threshold of the adjacent accumulated time periods to be a fixed sub-time resolution delay scale deltat. That is to say if it isiThe starting time of the single photon detector detection range gating threshold isTS i Then it is firstiThe starting time of the detection range gating threshold of the single-photon detector in +1 accumulated time period isTS i + Δ t, obtained for each pixel by the method described aboveNAll the accumulated time is T-NThe raw low-resolution time-dependent photon count histogram of (a). Unlike conventional accumulation methods, the present invention allows this to be achieved by delays of a sub-time resolution scaleNA low score of originalThe difference of sub-time resolution delay scales exists between the time-resolved photon counting histograms, and the variation information of the sub-time resolution scales of the flight time of the echo photons is included; then by applying thisNThe original low-resolution time-dependent photon counting histogram is sampled and registered and added to obtainNA sub-time-resolved time-dependent photon count histogram (i.e., a high-resolution time-dependent photon count histogram); and then estimating the flight time of the echo photon from the high-resolution time correlation photon counting histogram according to methods such as maximum likelihood estimation and the like, and further obtaining a target three-dimensional imaging distance map. Because the obtained high-resolution time-dependent photon counting histogram contains more precise change information, the distance resolution capability of the system breaks through the limitation of hardware, and the distance measurement precision is greatly improved.
In the specific reconstruction process, as shown in fig. 1, a high-repetition-frequency laser is adopted to emit a high-repetition-frequency pulse light signal, a target scene is illuminated through a laser emission optical system, and laser reflected by a target is collected by a receiving optical system and coupled to a single photon detector. The single photon detector and the high-repetition-frequency pulse light signal adopt the same synchronous clock, the accuracy of flight time recording is ensured, a light splitting module can be added, the high-repetition-frequency pulse light signal emitted by the high-repetition-frequency laser is divided into two paths, one path is received by the timing detector and used for recording laser emission time, the other path is used for illuminating a target scene, the target scene is coupled to the single photon detector through a receiving optical system after being reflected by the target, and finally, an echo photon point cloud is obtained by utilizing a time-related photon counting module.
As shown in fig. 2, the present invention, in the specific reconstruction, includes the following steps:
step 1, setting the starting time of the detection range gating threshold of the single-photon detector as
Figure 345948DEST_PATH_IMAGE001
Whereini=1，
Figure 235407DEST_PATH_IMAGE002
For sub-time-resolved delay scales, values less thanT bin ，T bin The minimum time resolution capability of the single-photon detector; the high-repetition-frequency laser continuously emits laser pulses within the integral time of the single-photon detector, the single-photon detector receives the echo photon point cloud, and a corresponding original low-resolution time-dependent photon counting histogram can be obtained on each pixelLi。
Step 2, judgmentiWhether or not equal toNIf yes, executing step 3, otherwise, returning to step 1 to orderi=i+1, up toiIs equal toN(ii) a WhereinNIs an integer greater than or equal to 2 for the number of sub-time-resolved translations,
Figure 301452DEST_PATH_IMAGE003
obtained at each pixelNA raw low-resolution time-dependent photon count histogram.
Step 3, obtainingNThe original low-resolution time-dependent photon counting histogram is up-sampled respectively to obtain at each pixelNA pseudo high resolution time dependent photon count histogram;
step 3.1, define a time slot number as
Figure 883743DEST_PATH_IMAGE008
The empty histogram of (Tsbin);Ntcounting the number of time slots of the original low-resolution time-dependent photon counting histogram;
step 3.2, the detection range gating threshold starting time of the single photon detector isTS 1 Corresponding primary low resolution time-dependent photon count histogramjPhoton number R corresponding to each time slot j Assigned to the empty histogram Tsbin respectively
Figure 402449DEST_PATH_IMAGE009
To
Figure 95598DEST_PATH_IMAGE010
In allNPhotons corresponding to a time slot, whereinj=1；
Step 3.3, judgmentjWhether or not to be equal toNtIf yes, then executeGo to step 3.4, otherwise, return to step 3.2, orderj=j+1, up tojIs equal toNtObtaining the pixel atTS 1 A corresponding pseudo high resolution time dependent photon count histogram;
step 3.4, for the rest of the pixelN-1The original low-resolution time-dependent photon count histograms are obtained at the pixel by performing the operations of steps 3.1 to 3.3NA pseudo high resolution time dependent photon count histogram.
Step 4, for each pixel obtained in step 3NA pseudo high resolution time-dependent photon count histogram toTS 1 And performing registration addition on a time axis for reference to obtain a high-resolution time-dependent photon counting histogram of the corresponding pixel, wherein the histogram contains the flight time change information of the echo photons with the sub-time resolution scale, and further the distance change for resolving more details is obtained.
Step 4.1, defineNNumber of each time slot is
Figure 252035DEST_PATH_IMAGE004
All zero histogram of (H0) (H)i) As an initial high resolution histogram; whereiniIs 1 toNThe number of the integer (c) of (d),Ntcounting the number of time slots of the original low-resolution time-dependent photon counting histogram;
step 4.2, because the minimum time slot of the pseudo high-resolution time-dependent photon counting histogram obtained in the step 3 is consistent with the sub-time resolution delay scale, each sub-time resolution delay corresponds to exactly one time slot, and the starting time of the detection distance gating threshold of the single photon detector is equal toTS 1 Assigning corresponding pseudo high-resolution time-dependent photon count histogram to all-zero histogram H0 (1)
Figure 5228DEST_PATH_IMAGE005
Time slot, after which, the single photon detector detection range gating threshold starts at timeTS 2 Assigning the corresponding pseudo high-resolution time-dependent photon count histogram to the all-zero histogram H0 (2)
Figure 11230DEST_PATH_IMAGE006
Time slot, and so on, the detection range gating threshold of the single photon detector is started at the timeTS N Assigning the corresponding pseudo high resolution time-dependent photon count histogram to the all-zero histogram H0: (N) Is/are as follows
Figure 508070DEST_PATH_IMAGE011
Time slot, thus obtainingNThe registered pseudo high-resolution time correlation photon counting histogram;
step 4.3, mixingNAnd adding the registered pseudo high-resolution time-dependent photon counting histograms to obtain a high-resolution time-dependent photon counting histogram of the pixel.
And 5, extracting the maximum probability time-of-flight time slot position from the high-resolution time-dependent photon counting histogram obtained in the step 4 by adopting maximum likelihood estimation. And obtaining the target distance corresponding to each pixel, namely a target three-dimensional imaging distance map, by using the relationship between the flight time and the distance.
For the case of sparse echo and containing noise, the maximum likelihood estimation method in step 5 may be replaced by related sparse reconstruction, denoising, and other methods, such as TV (total variation) sparse reconstruction and 3D deconvolution reconstruction scheme.
Taking Gaussian pulse as an example, the timing sequence of a single Gaussian pulse is analyzed, and the energy of a Q-switched laser pulse is generally usedP(t) Over timetThe variation of (d) can be expressed as:
Figure 17549DEST_PATH_IMAGE012
，ndetermining the shape of the laser pulse whennWhen the ratio is not less than 1,
Figure 941643DEST_PATH_IMAGE013
= laser pulse width/3.5, parameterAAs determined by the energy P0 of the single laser pulse,
Figure 169362DEST_PATH_IMAGE014
. Original low-resolution time-dependent photon counting histogram under different pulse widths according to the expression and high time resolution reconstructed by adopting the method of the inventionThe correlation photon counting histograms are subjected to simulation comparison, and the result is shown in fig. 3, and it can be seen from fig. 3 that under the condition of the same time resolution, the high-time-resolution correlation photon counting histogram reconstructed by the method comprises signal distribution of smaller time slots, and the comparison between the centroid position and the peak position of the histogram is prominent, so that the time slots can be accurate to sub-time resolution, and the accuracy is improved to a great extent.
FIG. 4 shows different pulse widths P w Next, the reconstructed high time resolution correlation photon counting histogram and the ideal histogram have the centroid deviation along with the change result graph of the sub time resolution translation times. As can be seen from FIG. 4, the method of the present invention can obtain more smaller error deviations at any pulse width, and even in the case of extremely narrow pulse width (the pulse width of the laser is much smaller than the time resolution of the single photon detector), the method of the present invention can make the centroid position of the echo signal closer to the ideal and the peak position prominent by continuously scanning and accumulating. And with the continuous reduction of the sub-time resolution delay scale (the number of times of sub-pixel translation is increased), the deviation between the centroid position of the reconstructed histogram and the centroid position of the ideal histogram is gradually reduced and approaches to 0. The results are in agreement with expectations, fully demonstrating the feasibility of the process of the invention in the time dimension.
The method of the invention is characterized in that a Middlebury stereo data set is used for verifying the spatial super-resolution capability and the time super-resolution capability:
with minimum time resolution of single photon detector
Figure 204314DEST_PATH_IMAGE015
Corresponding distance resolution is 15cm, the depth image is down-sampled in the depth dimension, and then sub-time resolution delay scale is carried out
Figure 600922DEST_PATH_IMAGE016
Respectively set as 0.5ns,0.2ns, 0.1ns and 0.05ns, and respectively collect continuous signals
Figure 430338DEST_PATH_IMAGE017
The image, processed and the result is shown in FIG. 5. As can be seen from FIG. 5, the invention is usedThe method can distinguish more distance detail changes, the root mean square error is greatly reduced, along with the unchanged reduction of the sub-time resolution delay scale, the more distance information which can be distinguished by a reconstruction result is, and the smaller the error is.
To further verify the performance of the method of the present invention, a range resolution target model was constructed, the entire model containing 64 × 64 pixels and 16 elevations, with the heights increasing from D1 to D16 by 0.1m, respectively. Minimum time resolution capability of single photon detector
Figure 410932DEST_PATH_IMAGE018
Is set as 1nsSub-time resolved delay scale
Figure 249575DEST_PATH_IMAGE016
Set to 0.5ns,0.2ns and 0.1ns, respectively; the results obtained by simulation using the Monte Carlo model are shown in FIG. 6, and it can be seen from FIG. 6 that the delay scale is resolved along with sub-time
Figure 733646DEST_PATH_IMAGE016
The reconstruction result can be resolved to more and more elevation changes, and the delay scale is resolved in sub-time
Figure 999543DEST_PATH_IMAGE016
For single-photon detectors with minimum time resolution
Figure 201854DEST_PATH_IMAGE018
0.5 times, the distance resolution capability is improved by about 2 times, and the delay scale is resolved in sub-time
Figure 640925DEST_PATH_IMAGE016
For single-photon detectors with minimum time resolution
Figure 120448DEST_PATH_IMAGE018
When the distance is 0.1 times, the distance resolution is improved by about 10 times, and the improvement capability of the method for distance resolution is fully proved.
Considering that the method of the present invention needs to acquire multiple different sets of histogram images, if compared with the same single frame image, it needs longer delay time, so in addition to the sub-time resolution delay scale, the integration time is also a factor that affects the imaging result, especially in the case of sparse echo, so fig. 7 sets the integration time as a variable, and sets the reconstruction results for comparison in the case of long accumulation time multiphoton and short time sparse echo photon, respectively. The result shows that when the echo is strong, the main factor limiting the imaging precision is hardware time resolution, the reconstruction error caused by singly increasing the number of echo photons is not obviously improved, and after the time resolution capability of the system is improved by adopting sub-time resolution delay sampling, the reconstruction error is obviously reduced, and the detail information is increased; under sparse echo, the main reasons causing errors at the moment are the number of echo photons is too small, uncertainty is caused, and local echo signals are absent, and reconstruction errors can be effectively reduced by increasing the number of echo photons. The results show that the method is applicable to all time-correlated single photon three-dimensional imaging systems under different imaging environments and echo intensities, and does not take time as a cost.

Claims (6)

1. A single photon three-dimensional imaging distance super-resolution method is characterized by comprising the following steps:
step 1, setting the starting time of a single-photon detector detection distance gate to be within the integral time of the single-photon detector
Figure 833779DEST_PATH_IMAGE001
Whereini=1，
Figure 683048DEST_PATH_IMAGE002
For sub-time-resolved delay scales, values less thanT bin ，T bin The minimum time resolution capability of the single-photon detector; receiving the echo photon point cloud to obtain the original low-resolution time-related light corresponding to each pixelA sub-count histogram;
step 2, judgmentiWhether or not equal toNIf yes, executing step 3, otherwise, returning to step 1 to orderi=i+1, up toiIs equal toN(ii) a WhereinNIs an integer greater than or equal to 2 for the number of sub-time-resolved translations,
Figure 888901DEST_PATH_IMAGE003
obtaining a corresponding pixel at each pixelNA raw low-resolution time-dependent photon count histogram;
step 3, corresponding to each pixelNThe original low-resolution time-dependent photon counting histogram is up-sampled respectively to obtain at each pixelNA pseudo high resolution time dependent photon count histogram;
step 4, for each pixel obtained in step 3NA pseudo high resolution time-dependent photon count histogram toTS 1 Registering and adding the reference on a time axis to obtain a high-resolution time-dependent photon counting histogram of the corresponding pixel;
and 5, obtaining a target three-dimensional imaging distance map based on the high-resolution time-dependent photon counting histogram of each pixel.
2. The single photon three-dimensional imaging distance super-resolution method according to claim 1, wherein the specific steps of step 4 for each pixel are as follows:
step 4.1, defineNNumber of each time slot is
Figure 209024DEST_PATH_IMAGE004
All zero histogram of (H0) (H)i) As an initial high resolution histogram; whereiniIs 1 toNThe number of the integer (c) of (d),Ntcounting the number of time slots of the original low-resolution time-dependent photon counting histogram;
step 4.2, the detection range gating threshold starting time of the single photon detector is
Figure 904448DEST_PATH_IMAGE005
Assigning a pseudo high-resolution time-dependent photon count histogram of (1) to an all-zero histogram H0
Figure 739549DEST_PATH_IMAGE006
Time slot, after which, the single photon detector detection range gating threshold starts at time
Figure 811410DEST_PATH_IMAGE007
Assigning the pseudo high-resolution time-dependent photon count histogram of (2) to the all-zero histogram H0
Figure 923722DEST_PATH_IMAGE008
Time slot, and so on, the detection range gating threshold of the single photon detector is started at the time
Figure 524468DEST_PATH_IMAGE009
The pseudo high-resolution time-dependent photon count histogram of (a) is assigned to the all-zero histogram H0: (b)N) Is/are as follows
Figure 348330DEST_PATH_IMAGE010
Time slot, thus obtainingNThe registered pseudo high-resolution time correlation photon counting histogram;
step 4.3, mixingNAnd adding the registered pseudo high-resolution time-dependent photon count histograms to obtain a high-resolution time-dependent photon count histogram of the pixel.
3. The single photon three-dimensional imaging distance super-resolution method according to claim 1 or 2, characterized in that the specific steps of step 3 for each pixel are:
step 3.1, define a time slot number as
Figure 895986DEST_PATH_IMAGE011
The empty histogram Tsbin of;
step 3.2, the detection range gating threshold starting time of the single photon detector isTS 1 Corresponding to the first in the original low-resolution time-dependent photon count histogramjPhoton number R corresponding to each time slot j Assigned to the empty histogram Tsbin respectively
Figure 987439DEST_PATH_IMAGE012
To
Figure 227927DEST_PATH_IMAGE013
In totalNPhotons corresponding to a time slot, whereinj=1；
Step 3.3, judgmentjWhether or not equal toNtIf yes, go to step 3.4, otherwise, go back to step 3.2, orderj=j+1, up tojIs equal toNtObtaining the pixel atTS 1 A corresponding pseudo high resolution time dependent photon count histogram;
step 3.4, for the rest of the pixelN-1The original low-resolution time-dependent photon count histograms are obtained at the pixel by performing the operations of steps 3.1 to 3.3NA pseudo high resolution time dependent photon count histogram.
4. The single photon three-dimensional imaging distance super-resolution method according to claim 3, wherein the step 5 is specifically as follows:
step 5.1, extracting a maximum probability time-of-flight time slot position from the high-resolution time-dependent photon counting histogram of each pixel obtained in the step 4 by adopting a three-dimensional reconstruction method, and obtaining the time-of-flight of the echo photon;
and 5.2, obtaining the target distance corresponding to each pixel by utilizing the relationship between the flight time and the distance of the echo photons, wherein the target distances corresponding to all the pixels form a target three-dimensional imaging distance map.
5. The single photon three-dimensional imaging distance super-resolution method according to claim 4, characterized in that: in step 5, the three-dimensional reconstruction method is a maximum likelihood estimation method or a TV sparse reconstruction or a 3D deconvolution reconstruction method.
6. A single photon three-dimensional imaging distance super-resolution system comprises a memory and a processor, wherein the memory stores a computer program, and the system is characterized in that: the computer program, when running on a processor, performs the steps of the method of any one of claims 1 to 5.
CN202211631142.9A 2022-12-19 2022-12-19 Single photon three-dimensional imaging distance super-resolution method and system Active CN115616608B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211631142.9A CN115616608B (en) 2022-12-19 2022-12-19 Single photon three-dimensional imaging distance super-resolution method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211631142.9A CN115616608B (en) 2022-12-19 2022-12-19 Single photon three-dimensional imaging distance super-resolution method and system

Publications (2)

Publication Number Publication Date
CN115616608A true CN115616608A (en) 2023-01-17
CN115616608B CN115616608B (en) 2023-08-11

Family

ID=84880470

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211631142.9A Active CN115616608B (en) 2022-12-19 2022-12-19 Single photon three-dimensional imaging distance super-resolution method and system

Country Status (1)

Country Link
CN (1) CN115616608B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117332585A (en) * 2023-09-27 2024-01-02 奕富通集成科技(珠海横琴)有限公司 Simulation model modeling method and system for time-of-flight ranging laser radar

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104914446A (en) * 2015-06-19 2015-09-16 南京理工大学 Three-dimensional distance image time domain real-time denoising method based on photon counting
US20160209498A1 (en) * 2012-02-16 2016-07-21 Reilly Daniel R System and method for measuring optical delay using a single photon detector with pulsed optical signals
CN109887018A (en) * 2019-02-28 2019-06-14 中国计量大学 A kind of photon 3D imaging system based on deep learning
CN111562593A (en) * 2020-04-23 2020-08-21 中国科学院西安光学精密机械研究所 Area array push-broom type photon counting three-dimensional imaging system based on pixel multiplexing
CN111694014A (en) * 2020-06-16 2020-09-22 中国科学院西安光学精密机械研究所 Laser non-visual field three-dimensional imaging scene modeling method based on point cloud model
CN112731443A (en) * 2021-02-08 2021-04-30 山东大学 Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image
CN113376653A (en) * 2021-08-12 2021-09-10 清华大学 Photon counting-based three-dimensional imaging method and device for computer-generated holographic radar
US20210302553A1 (en) * 2020-03-30 2021-09-30 Semiconductor Components Industries, Llc Lidar systems with improved time-to-digital conversion circuitry
WO2021212914A1 (en) * 2020-04-20 2021-10-28 深圳奥锐达科技有限公司 Time delay-based distance measurement system and method

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160209498A1 (en) * 2012-02-16 2016-07-21 Reilly Daniel R System and method for measuring optical delay using a single photon detector with pulsed optical signals
CN104914446A (en) * 2015-06-19 2015-09-16 南京理工大学 Three-dimensional distance image time domain real-time denoising method based on photon counting
CN109887018A (en) * 2019-02-28 2019-06-14 中国计量大学 A kind of photon 3D imaging system based on deep learning
US20210302553A1 (en) * 2020-03-30 2021-09-30 Semiconductor Components Industries, Llc Lidar systems with improved time-to-digital conversion circuitry
WO2021212914A1 (en) * 2020-04-20 2021-10-28 深圳奥锐达科技有限公司 Time delay-based distance measurement system and method
CN111562593A (en) * 2020-04-23 2020-08-21 中国科学院西安光学精密机械研究所 Area array push-broom type photon counting three-dimensional imaging system based on pixel multiplexing
CN111694014A (en) * 2020-06-16 2020-09-22 中国科学院西安光学精密机械研究所 Laser non-visual field three-dimensional imaging scene modeling method based on point cloud model
CN112731443A (en) * 2021-02-08 2021-04-30 山东大学 Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image
CN113376653A (en) * 2021-08-12 2021-09-10 清华大学 Photon counting-based three-dimensional imaging method and device for computer-generated holographic radar

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
LI ZHENGPING ET AL.: "Single-photon computational 3D imaging at 45 km", vol. 8, no. 8, pages 1532 - 1540 *
XUBIN FENG.ETAL.: "Single Space Object Image Super Resolution Reconstructing Using Convolutional Networks in Wavelet Transform Domain", 2020 IEEE 3RD INTERNATIONAL CONFERENCE ON ELECTRONICS TECHNOLOGY *
林杰;何伟基;叶凌;方剑;陈钱;顾国华;: "基于光子计数的自适应深度成像方法", vol. 35, no. 10, pages 94 - 101 *
赵惠等: "基于交叉延迟线位敏阳极微通道板探测器的光子计数激光三维超分辨成像", vol. 51, no. 7, pages 1 - 16 *
骆乐;吴长强;林杰;冯振超;何伟基;陈钱;: "基于光子计数激光雷达的时域去噪", 光学精密工程, no. 05 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117332585A (en) * 2023-09-27 2024-01-02 奕富通集成科技(珠海横琴)有限公司 Simulation model modeling method and system for time-of-flight ranging laser radar

Also Published As

Publication number Publication date
CN115616608B (en) 2023-08-11

Similar Documents

Publication Publication Date Title
WO2021051477A1 (en) Time of flight distance measurement system and method with adjustable histogram
CN111492265B (en) Multi-resolution, simultaneous localization and mapping based on 3D lidar measurements
WO2021051478A1 (en) Time-of-flight-based distance measurement system and method for dual-shared tdc circuit
WO2021051479A1 (en) Interpolation-based time of flight measurement method and system
EP1769266B1 (en) Method and apparatus for high resolution 3d imaging
WO2021051481A1 (en) Dynamic histogram drawing time-of-flight distance measurement method and measurement system
CA2650235C (en) Distance measuring method and distance measuring element for detecting the spatial dimension of a target
WO2021051480A1 (en) Dynamic histogram drawing-based time of flight distance measurement method and measurement system
US9602807B2 (en) Single frequency time of flight de-aliasing
CN115616608A (en) Single-photon three-dimensional imaging distance super-resolution method and system
CN105954733A (en) Time-domain filtering method based on photon flight time correlation
WO2021035694A1 (en) System and method for time-coding-based time-of-flight distance measurement
US11561291B2 (en) High pulse repetition frequency lidar
CN112424639A (en) Measuring distance to an object using time of flight and a pseudorandom bit sequence
CN210129035U (en) Laser radar echo data extraction device
CN110471084B (en) Range-gated laser radar imaging method based on adjacent frame difference
CN109407104B (en) Increasing depth resolution and depth accuracy of TOF sensors by avoiding histogram formation
Padmanabhan et al. Depth precision in dToF sensors for AR applications
RU2361237C2 (en) Method of photo-location measurement of height of cloud layers and device to this end
CN117492028A (en) Super-resolution three-dimensional imaging method based on sub-time resolution delay scanning
Zhou et al. Research of detecting the laser’s secondary reflected echo from target by using Geiger-mode avalanche photodiode
Zhou et al. A New Approach for Accuracy Improvement of Pulsed LiDAR Remote Sensing Data
CN116381709A (en) Method, storage medium and apparatus for measuring time of flight
WO2021035695A1 (en) System and method for measuring distance by time of flight based on time codes
US20210247179A1 (en) De-jitter of point cloud data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant