WO2024065094A1 - 超分辨显微成像方法、装置、计算机设备及存储介质 - Google Patents

超分辨显微成像方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2024065094A1
WO2024065094A1 PCT/CN2022/121400 CN2022121400W WO2024065094A1 WO 2024065094 A1 WO2024065094 A1 WO 2024065094A1 CN 2022121400 W CN2022121400 W CN 2022121400W WO 2024065094 A1 WO2024065094 A1 WO 2024065094A1
Authority
WO
WIPO (PCT)
Prior art keywords
iterations
deconvolution
resolution
super
image
Prior art date
Application number
PCT/CN2022/121400
Other languages
English (en)
French (fr)
Inventor
李浩宇
赵唯淞
赵士群
Original Assignee
广州超视计生物科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州超视计生物科技有限公司 filed Critical 广州超视计生物科技有限公司
Priority to CA3210853A priority Critical patent/CA3210853A1/en
Priority to PCT/CN2022/121400 priority patent/WO2024065094A1/zh
Priority to CN202280017314.5A priority patent/CN117157532A/zh
Priority to EP23199782.6A priority patent/EP4345735A1/en
Publication of WO2024065094A1 publication Critical patent/WO2024065094A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10064—Fluorescence image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the invention relates to a super-resolution microscopic imaging method, device, computer equipment and storage medium, and belongs to the field of computational imaging and super-resolution microscopic imaging.
  • Super-resolution optical fluctuation imaging based on the molecular intensity signal fluctuation model only uses the physical model of random fluctuations in the intensity of fluorescent molecules and does not rely on any hardware modulation. It is a flexible and cost-effective super-resolution method. Due to its characteristics of not being limited by the hardware system, it can be flexibly coupled to various imaging modalities. The disadvantage of this type of device is that the current fluorescence super-resolution method has a low temporal resolution and requires continuous acquisition of at least 500 to 1000 frames of images to achieve the expected high-quality super-resolution effect, which hinders its application in super-resolution imaging of living cells. Therefore, it is necessary to provide a method to maximize the use of the fluorescence fluctuation behavior that can be detected in each measurement to achieve the required high temporal resolution and high throughput.
  • the present invention provides a super-resolution microscopy method, apparatus, computer equipment and storage medium, which have the advantages of non-parameterization, high throughput and high resolution, and can overcome the shortcomings and deficiencies of the prior art.
  • the first object of the present invention is to provide a super-resolution microscopic imaging method
  • the second object of the present invention is to provide a super-resolution microscopic imaging device.
  • a third object of the present invention is to provide a computer device.
  • a fourth object of the present invention is to provide a storage medium.
  • a super-resolution microscopic imaging method comprising:
  • the output image is reconstructed twice;
  • a second deconvolution iteration is performed on the two reconstructed images, and output is performed when the iteration reaches the number of iterations or half of the number of iterations.
  • the method further includes:
  • Wavelet transform was used to estimate the background of fluorescence signal and remove background noise.
  • the method of estimating the background of the fluorescence signal by wavelet transform and removing the background noise specifically includes:
  • the background is estimated from the lowest frequency of the input image using wavelet estimation
  • the lowest frequency band is inversely wavelet transformed to the spatial domain and the result is compared with half the square root of the input image.
  • the two images are merged by keeping the minimum value of each pixel.
  • the estimated low-frequency band low-peak background data is used as a new input image, and wavelet estimation is performed cyclically until a preset number of cycles is reached.
  • determining the number of deconvolution iterations specifically includes:
  • the rolling Fourier ring correlation is used for judgment, and the number of iterations when the maximum rolling Fourier ring correlation resolution is reached is used as the number of deconvolution iterations.
  • the use of rolling Fourier ring correlation to make a judgment, using the number of iterations when the maximum rolling Fourier ring correlation resolution is reached as the number of deconvolution iterations specifically includes:
  • An averaging filter with an average window half-width is used to smooth the noisy rolling Fourier ring correlation curve
  • the frequency is defined as the effective cutoff frequency
  • the resolution is the inverse of the effective cutoff frequency
  • the given threshold represents the maximum spatial frequency of meaningful information outside the random noise
  • the number of iterations when the maximum resolution is reached is used as the number of deconvolution iterations.
  • reconstructing the output image twice specifically includes:
  • the output image is reconstructed for the first time using the principle of fluorescence signal fluctuation
  • the sparsity-continuity joint constraint is applied to perform a second reconstruction on the image after the first reconstruction.
  • the first reconstruction of the output image using the fluorescence signal fluctuation principle specifically includes:
  • the expression of the fluorescence signal is obtained according to the point spread function of the microscope, the brightness constant of the fluorescent molecule and the function of the fluctuation of the brightness of the fluorescent molecule over time;
  • the expression of the time cumulative amount with zero time delay is obtained according to the expression of the fluorescence signal
  • time accumulant is expanded, and when the preset conditions are met, the cross-correlation terms of the time accumulant expansion are regarded as zero, so that the time accumulant is expressed as the sum of squares of the corresponding brightness constant weighted point spread function.
  • applying the sparse-continuity joint constraint to perform a second reconstruction on the image after the first reconstruction specifically includes:
  • the reconstruction model includes a first item, a second item and a third item, the first item is a fidelity item, which represents the distance between the first reconstructed image and the acquired initial image, the second item represents the continuity constraint of the first reconstructed image, and the third item represents the sparsity constraint of the first reconstructed image;
  • the reconstruction model is used to perform a second reconstruction on the image after the first reconstruction.
  • the iteration of deconvolution specifically includes:
  • the iterative formula is obtained by using the iterative method of solving the maximum likelihood in the spatial domain
  • an acceleration method based on vector extrapolation is adopted to realize the iterative calculation of deconvolution.
  • a super-resolution microscopic imaging device comprising:
  • An acquisition module used for acquiring a set of fluorescence signal sequences of samples to be observed
  • a judgment module used to judge the number of deconvolution iterations
  • a pre-deconvolution module performing pre-deconvolution iterations on each initial image frame in the fluorescence signal sequence, and outputting the result when the iterations reach half of the number of iterations;
  • a reconstruction module used to reconstruct the output image twice
  • the deconvolution module is used to perform a second deconvolution iteration on the twice reconstructed images, and output when the iteration reaches the number of iterations or half of the number of iterations.
  • a computer device comprises a processor and a memory for storing a program executable by the processor, wherein when the processor executes the program stored in the memory, the above-mentioned super-resolution microscopic imaging method is implemented.
  • a storage medium stores a program, and when the program is executed by a processor, the above-mentioned super-resolution microscopic imaging method is implemented.
  • the present invention has the advantage of high flexibility and can be widely coupled to various imaging modalities, such as acoustic microscopy, that is, photoacoustic and ultrasonic microscopy imaging technology; it has the advantage of high throughput, and further processes the obtained single image using post-deconvolution with multiple iterations, which can achieve a 2-fold improvement in three-dimensional spatial resolution; while maintaining super-resolution, only 20 frames or less are needed to achieve a high-quality super-resolution effect, achieving a 50-100-fold improvement in time resolution; it has the advantage of no parameterization, and proposes an automatic parameter estimation method based on rolling Fourier ring correlation, so as to achieve no parameterization, thereby realizing automatic super-resolution high-throughput imaging.
  • imaging modalities such as acoustic microscopy, that is, photoacoustic and ultrasonic microscopy imaging technology
  • it has the advantage of high throughput, and further processes the obtained single image using post-deconvolution with multiple iterations, which can achieve a 2-
  • FIG1 is a flow chart of a super-resolution microscopic imaging method according to Embodiment 1 of the present invention.
  • FIG. 2 is an exemplary result diagram of the improvement of the three-dimensional spatial resolution of Example 1 of the present invention.
  • FIG. 3 is an exemplary result diagram of the time resolution improvement of Example 1 of the present invention.
  • FIG. 4 is an exemplary result diagram of the non-parameterized reconstruction capability of Example 1 of the present invention.
  • FIG. 5 is an exemplary result diagram of the automatic high-throughput reconstruction of Example 1 of the present invention.
  • FIG. 6 is a structural block diagram of a super-resolution microscopic imaging device according to Embodiment 2 of the present invention.
  • FIG. 7 is a block diagram of the structure of a computer device according to Embodiment 3 of the present invention.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • this embodiment provides a super-resolution microscopic imaging method, which is implemented based on deconvolution enhancement and includes the following steps:
  • the number of frames of the fluorescence signal sequence is 20. Those skilled in the art will appreciate that the number of frames of the fluorescence signal sequence may also be 10, 50, or the like.
  • the background estimation operation can be removed, that is, the background parameter value b is set to zero to avoid information removal; in particular, considering that images under low-dose illumination generally only show low and stable background fluorescence noise distribution, the value exceeding the image mean is directly set to zero, and the obtained residual image is used for subsequent background estimation;
  • step S101 under other conditions besides the above, after step S101, the following steps may also be included:
  • step S102 specifically includes:
  • the signal in order to extract the lowest band in the frequency domain, is multi-level decomposed into 7 levels using a two-dimensional Daubechies-6 wavelet filter.
  • S1022 perform inverse wavelet transform on the lowest frequency band to the spatial domain, and compare the result with half the square root of the input image, and merge the two images by keeping the minimum value of each pixel.
  • the lowest frequency band is inversely wavelet transformed to the spatial domain and the result is compared to half the square root of the input image (smoother information), and the two images are merged by keeping the minimum value of each pixel. This operation removes high-intensity pixels in the background due to inaccurate background estimation.
  • the preset number of cycles is set to 3 times to estimate the actual fluorescence background with a minimum distribution.
  • determining the number of deconvolution iterations specifically includes: using rolling Fourier ring correlation to make a determination, and using the number of iterations when the maximum rolling Fourier ring correlation resolution is reached as the number of deconvolution iterations.
  • a rolling Fourier ring correlation (FRC) is used for judgment, and the number of iterations when the maximum rolling Fourier ring correlation resolution is reached is used as the number of deconvolution iterations, specifically including:
  • the rolling Fourier ring correlation measures the statistical correlation between two two-dimensional signals on a series of concentric rings in the Fourier domain.
  • the rolling Fourier ring correlation can be considered as a function of the spatial frequency q i :
  • the rolling Fourier ring correlation curve consists of N/2 values, and the discretization step size ⁇ f of the spatial frequency is:
  • an averaging filter with an averaging window half-width (typically equal to 3) is used to smooth the noisy rolling Fourier ring correlation curve.
  • the frequency is defined as an effective cutoff frequency
  • the resolution is the inverse of the effective cutoff frequency.
  • the given threshold represents the maximum spatial frequency of meaningful information outside random noise.
  • the frequency is defined as the effective cutoff frequency
  • the resolution is the inverse of the effective cutoff frequency
  • the given threshold represents the maximum spatial frequency of meaningful information outside of random noise.
  • the common choice of the threshold is a fixed value threshold or a ⁇ factor curve.
  • the fixed value is usually a 1/7 hard threshold, and the criterion of the ⁇ factor curve can be written as:
  • Ni represents the number of pixels in a ring with radius qi
  • ⁇ factor is 3. If the two measurements contain only noise, the rolling Fourier ring correlation curve can be expressed as Therefore, the 3 ⁇ factor curve is actually a relatively effective information for judging the frequency components whose correlation is three times greater than that of pure noise.
  • the number of iterations when the maximum resolution is reached is recorded as k, where k is the number of deconvolution iterations.
  • the iteration of the pre-deconvolution is completed when the iteration reaches half of the number of iterations determined in step S102.
  • the iteration of the deconvolution specifically includes:
  • the deconvolution adopts the Richardson-Lucy (RL) algorithm, and the deconvolution model is based on the Poisson noise model.
  • the maximum likelihood is solved iteratively in the spatial domain to obtain the following iterative formula:
  • x and y are spatial coordinates
  • h represents the point spread function (PSF) of the microscope
  • f represents the real fluorescence signal in the physical world
  • g represents the signal finally collected by the microscope.
  • an acceleration method based on vector extrapolation is used to implement the iterative calculation of deconvolution:
  • g is the image reconstructed by the prior knowledge constraint
  • h represents the point spread function of the microscope
  • xj +1 is the image after j+1 iterations
  • ⁇ is the adaptive acceleration factor
  • step S104 specifically includes:
  • the imaging sample is generally considered to be composed of N individual fluorescent molecules located at rk. Assuming that the fluorescent molecules have their own independent molecular brightness that varies with time, the fluorescence signal at r and time t is expressed as:
  • h, c, and ⁇ represent the point spread function of the corresponding microscope, the molecular brightness constant, and the function of the molecular brightness fluctuation over time, respectively.
  • the time accumulative amount is a second-order time accumulative amount.
  • the time accumulative amount can also be a third-order time accumulative amount or a fourth-order time accumulative amount.
  • the individual fluctuation characteristics of each fluorescent molecule are used to calculate the relevant cumulative amount of each pixel along the t axis to improve the resolution, and the second-order time cumulative amount G2 with zero delay is calculated to obtain the following formula:
  • ⁇ > t is the time-averaged function.
  • the expression of the second-order time accumulant G2 is expanded to obtain the following formula:
  • the cross-correlation term in the expansion is regarded as zero
  • the second-order time accumulant G2 is expressed as the square sum of the weighted point spread function corresponding to the brightness constant ⁇ , as follows:
  • S10421 Construct a reconstruction model, wherein the reconstruction model includes a first item, a second item, and a third item, wherein the first item is a fidelity item, which indicates the distance between the image after the first reconstruction and the acquired initial image, the second item indicates the continuity constraint of the image after the first reconstruction, and the third item indicates the sparsity constraint of the image after the first reconstruction;
  • the first item is a fidelity item, which indicates the distance between the image after the first reconstruction and the acquired initial image
  • the second item indicates the continuity constraint of the image after the first reconstruction
  • the third item indicates the sparsity constraint of the image after the first reconstruction
  • the reconstruction model is constructed as follows:
  • the first term on the left is the fidelity term, which represents the distance between the restored image x and the acquired initial image f
  • A is the point spread function of the optical system
  • the second and third terms are the continuity and sparsity constraints, respectively
  • 2 are the l 1 and l 2 norms, respectively
  • ⁇ and ⁇ L1 represent the weights of the fidelity and sparsity constraints, respectively;
  • the Hessian matrix continuity prior of the xy-t(z) axis is defined as:
  • the second deconvolution iteration is completed when the iteration reaches half of the iteration number determined by step S102 (i.e., k/2 times), and the iterations of the pre-deconvolution and the second deconvolution are both k/2 times, which just reaches the iteration number determined by step S102.
  • the specific content of the second deconvolution iteration can be found in step S103, which will not be described one by one here.
  • the second deconvolution iteration may also be completed when the iteration reaches the number of iterations determined in step S102.
  • Figure 2 is an exemplary result showing the improvement of the three-dimensional spatial resolution of the embodiment of the present invention.
  • the three-dimensional spatial resolution can be improved by 2 times.
  • the three-dimensional PSF of the quantum dot (QD525) sample was measured using the method of the embodiment of the present invention.
  • the image sequence was obtained using a standard spinning disk confocal microscope, and the lateral/axial half-height width of such a molecular point source was extracted respectively.
  • the lateral/axial half-height width of the traditional confocal mode is 325/655nm
  • the traditional fluorescence fluctuation mode is 295/510nm
  • the embodiment of the present invention is 135/334nm.
  • Figure 3 is an exemplary result showing the improvement in time resolution of an embodiment of the present invention (microtubules labeled with quantum dots in COS-7 cells).
  • the embodiment of the present invention only needs 20 frames to achieve high-quality super-resolution while maintaining super-resolution, achieving a 50-100 times improvement in time resolution compared to traditional fluorescence fluctuation technology (SOFI).
  • SOFI fluorescence fluctuation technology
  • the traditional fluorescence fluctuation technology requires 1000 frames of image reconstruction to maintain relatively stable performance, while the embodiment of the present invention can reconstruct the two-point structure with high fidelity using only 20 frames of images under all conditions.
  • FIG4 and FIG5 are exemplary results showing the non-parameterized reconstruction capability and automatic high-throughput reconstruction of the embodiment of the present invention, respectively.
  • the embodiment of the present invention has the advantage of non-parameterization, and proposes an automatic parameter estimation method based on rolling Fourier ring correlation to achieve non-parameterization, thereby achieving automatic super-resolution high-throughput imaging (microtubules labeled with quantum dots in COS-7 cells).
  • This embodiment images a large field of view of 2.0 mm ⁇ 1.4 mm, which contains more than 2000 cells and is composed of approximately 2,400,000,000 (2.4 billion) pixels (each pixel is 32.5 nm ⁇ 32.5 nm), spanning a regional spatial scale of almost five orders of magnitude.
  • This embodiment only takes about 10 minutes to reconstruct it, while the traditional fluorescence fluctuation technology (SOFI) takes about 17 hours to achieve similar imaging performance.
  • SOFI fluorescence fluctuation technology
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • this embodiment provides a super-resolution microscopic imaging device, which includes an acquisition module 601, a wavelet transform module 602, a judgment module 603, a pre-deconvolution module 604, a reconstruction module 605 and a deconvolution module 606.
  • acquisition module 601 a wavelet transform module 602
  • judgment module 603 a judgment module 603
  • pre-deconvolution module 604 a reconstruction module 605
  • the specific description of each module is as follows:
  • the acquisition module 601 is used to acquire a set of fluorescence signal sequences of samples to be observed.
  • the wavelet transform module 602 is used to estimate the background of the fluorescence signal by using wavelet transform and remove the background noise.
  • the judgment module 603 is used to judge the number of deconvolution iterations.
  • the pre-deconvolution module 604 performs pre-deconvolution iterations on each initial image frame in the fluorescence signal sequence, and outputs the result when the iterations reach half of the number of iterations.
  • the reconstruction module 605 is used to reconstruct the output image twice.
  • the deconvolution module 606 is used to perform a second deconvolution iteration on the twice reconstructed images, and output when the iteration reaches the number of iterations or half of the number of iterations.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • This embodiment provides a computer device, which may be a computer, as shown in FIG7 , comprising a processor 702, a memory, an input device 703, a display 704, and a network interface 705 connected via a system bus 701, wherein the processor is used to provide computing and control capabilities, the memory comprises a non-volatile storage medium 706 and an internal memory 707, the non-volatile storage medium 706 stores an operating system, a computer program, and a database, the internal memory 707 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium, and when the processor 702 executes the computer program stored in the memory, the super-resolution microscopic imaging method of the above-mentioned embodiment 1 is implemented as follows:
  • the output image is reconstructed twice;
  • a second deconvolution iteration is performed on the two reconstructed images, and output is performed when the iteration reaches the number of iterations or half of the number of iterations.
  • the method further includes:
  • Wavelet transform was used to estimate the background of fluorescence signal and remove background noise.
  • the method of estimating the background of the fluorescence signal by wavelet transform and removing the background noise specifically includes:
  • the background is estimated from the lowest frequency of the input image using wavelet estimation
  • the lowest frequency band is inversely wavelet transformed to the spatial domain and the result is compared with half the square root of the input image.
  • the two images are merged by keeping the minimum value of each pixel.
  • the estimated low-frequency band low-peak background data is used as a new input image, and wavelet estimation is performed cyclically until a preset number of cycles is reached.
  • determining the number of deconvolution iterations specifically includes:
  • the rolling Fourier ring correlation is used for judgment, and the number of iterations when the maximum rolling Fourier ring correlation resolution is reached is used as the number of deconvolution iterations.
  • the use of rolling Fourier ring correlation to make a judgment, using the number of iterations when the maximum rolling Fourier ring correlation resolution is reached as the number of deconvolution iterations specifically includes:
  • An averaging filter with an average window half-width is used to smooth the noisy rolling Fourier ring correlation curve
  • the frequency is defined as the effective cutoff frequency
  • the resolution is the inverse of the effective cutoff frequency
  • the given threshold represents the maximum spatial frequency of meaningful information outside the random noise
  • the number of iterations when the maximum resolution is reached is used as the number of deconvolution iterations.
  • reconstructing the output image twice specifically includes:
  • the output image is reconstructed for the first time using the principle of fluorescence signal fluctuation
  • the sparsity-continuity joint constraint is applied to perform a second reconstruction on the image after the first reconstruction.
  • the first reconstruction of the output image using the fluorescence signal fluctuation principle specifically includes:
  • the expression of the fluorescence signal is obtained according to the point spread function of the microscope, the brightness constant of the fluorescent molecule and the function of the fluctuation of the brightness of the fluorescent molecule over time;
  • the expression of the time cumulative amount with zero time delay is obtained according to the expression of the fluorescence signal
  • time accumulant is expanded, and when the preset conditions are met, the cross-correlation terms of the time accumulant expansion are regarded as zero, so that the time accumulant is expressed as the sum of squares of the corresponding brightness constant weighted point spread function.
  • applying the sparse-continuity joint constraint to perform a second reconstruction on the image after the first reconstruction specifically includes:
  • the reconstruction model includes a first item, a second item and a third item, the first item is a fidelity item, which represents the distance between the first reconstructed image and the acquired initial image, the second item represents the continuity constraint of the first reconstructed image, and the third item represents the sparsity constraint of the first reconstructed image;
  • the reconstruction model is used to perform a second reconstruction on the image after the first reconstruction.
  • the iteration of deconvolution specifically includes:
  • the iterative formula is obtained by using the iterative method of solving the maximum likelihood in the spatial domain
  • an acceleration method based on vector extrapolation is adopted to realize the iterative calculation of deconvolution.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program.
  • the computer program is executed by a processor, the super-resolution microscopic imaging method of the above embodiment 1 is implemented as follows:
  • the output image is reconstructed twice;
  • a second deconvolution iteration is performed on the two reconstructed images, and output is performed when the iteration reaches the number of iterations or half of the number of iterations.
  • the method further includes:
  • Wavelet transform was used to estimate the background of fluorescence signal and remove background noise.
  • the method of estimating the background of the fluorescence signal by wavelet transform and removing the background noise specifically includes:
  • the background is estimated from the lowest frequency of the input image using wavelet estimation
  • the lowest frequency band is inversely wavelet transformed to the spatial domain and the result is compared with half the square root of the input image.
  • the two images are merged by keeping the minimum value of each pixel.
  • the estimated low-frequency band low-peak background data is used as a new input image, and wavelet estimation is performed cyclically until a preset number of cycles is reached.
  • determining the number of deconvolution iterations specifically includes:
  • the rolling Fourier ring correlation is used for judgment, and the number of iterations when the maximum rolling Fourier ring correlation resolution is reached is used as the number of deconvolution iterations.
  • the use of rolling Fourier ring correlation to make a judgment, using the number of iterations when the maximum rolling Fourier ring correlation resolution is reached as the number of deconvolution iterations specifically includes:
  • An averaging filter with an average window half-width is used to smooth the noisy rolling Fourier ring correlation curve
  • the frequency is defined as the effective cutoff frequency
  • the resolution is the inverse of the effective cutoff frequency
  • the given threshold represents the maximum spatial frequency of meaningful information outside the random noise
  • the number of iterations when the maximum resolution is reached is used as the number of deconvolution iterations.
  • reconstructing the output image twice specifically includes:
  • the output image is reconstructed for the first time using the principle of fluorescence signal fluctuation
  • the sparsity-continuity joint constraint is applied to perform a second reconstruction on the image after the first reconstruction.
  • the first reconstruction of the output image using the fluorescence signal fluctuation principle specifically includes:
  • the expression of the fluorescence signal is obtained according to the point spread function of the microscope, the brightness constant of the fluorescent molecule and the function of the fluctuation of the brightness of the fluorescent molecule over time;
  • the expression of the time cumulative amount with zero time delay is obtained according to the expression of the fluorescence signal
  • time accumulant is expanded, and when the preset conditions are met, the cross-correlation terms of the time accumulant expansion are regarded as zero, so that the time accumulant is expressed as the sum of squares of the corresponding brightness constant weighted point spread function.
  • applying the sparse-continuity joint constraint to perform a second reconstruction on the image after the first reconstruction specifically includes:
  • the reconstruction model includes a first item, a second item and a third item, the first item is a fidelity item, which represents the distance between the first reconstructed image and the acquired initial image, the second item represents the continuity constraint of the first reconstructed image, and the third item represents the sparsity constraint of the first reconstructed image;
  • the reconstruction model is used to perform a second reconstruction on the image after the first reconstruction.
  • the iteration of deconvolution specifically includes:
  • the iterative formula is obtained by using the iterative method of solving the maximum likelihood in the spatial domain
  • an acceleration method based on vector extrapolation is adopted to realize the iterative calculation of deconvolution.
  • the present invention has the advantage of high flexibility and can be widely coupled to various imaging modalities, such as acoustic microscopy, that is, photoacoustic and ultrasonic microscopy imaging technology; it has the advantage of high throughput, and further processes the obtained single image using post-deconvolution with multiple iterations, which can achieve a 2-fold improvement in three-dimensional spatial resolution; while maintaining super-resolution, only 20 frames are needed to achieve high-quality super-resolution effects, achieving a 50-100-fold improvement in time resolution; it has the advantage of non-parameterization, and an automatic parameter estimation method based on rolling Fourier ring correlation is proposed to achieve non-parameterization, thereby realizing automatic super-resolution high-throughput imaging.
  • imaging modalities such as acoustic microscopy, that is, photoacoustic and ultrasonic microscopy imaging technology
  • it has the advantage of high throughput, and further processes the obtained single image using post-deconvolution with multiple iterations, which can achieve a 2-fold improvement in
  • the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.
  • the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device.
  • the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • the computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device.
  • the computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the computer readable storage medium can be written in one or more programming languages or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and a conventional procedural programming language, such as C or a similar programming language.
  • the program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).
  • LAN local area network
  • WAN wide area network
  • the computer program code required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., such as C programming languages, VisualBasic, Fortran2103, Perl, COBOL2102, PHP, ABAP, such as Python, Ruby and Groovy dynamic programming languages or other programming languages.
  • the program code can be executed completely on the user's computer, can be executed partially on the user's computer as an independent software package, can be executed partially on the user's computer and partially on a remote computer, or can be executed completely on a remote computer or server.
  • the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or as a service such as software as a service (SaaS).
  • LAN local area network
  • WAN wide area network
  • SaaS software as a service

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Image Processing (AREA)

Abstract

一种超分辨显微成像方法、装置、计算机设备及存储介质，该方法包括:采集一组待观测样本的荧光信号序列；判断解卷积的迭代次数;对该荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到该迭代次数的一半时进行输出;对输出的图像进行两次重建;对两次重建后的图像执行第二次解卷积的迭代，在迭代达到该迭代次数或该迭代次数的一半时进行输出。具有无参化、高通量、分辨率高的优点，能够克服现有技术的缺点与不足之处，能够广泛的耦合于各类成像模态，例如声学显微，即光声与超声显微成像技术中。

Description

超分辨显微成像方法、装置、计算机设备及存储介质 技术领域
本发明涉及一种超分辨显微成像方法、装置、计算机设备及存储介质，属于计算成像和超分辨显微成像领域。
背景技术
基于分子强度信号涨落模型的荧光超分辨方法(Super-resolution optical fluctuation imaging，SOFI)，只利用荧光分子强度随机涨落的物理模型，不依靠任何硬件调制，是一种灵活且成本效益极高的超分辨手段。由于其不受硬件系统限制的特性，可以灵活的耦合于各类不同的成像模态。这类装置的缺点是目前荧光超分辨方法的时间分辨率较低，需要连续采集至少500～1000帧图像才能达到期待的高质量超分辨效果，这阻碍了其在活细胞超分辨成像中的应用。因此，需要提供最大化利用每次测量中可检测到的荧光波动行为，以达到所需的高时间分辨率和高通量的方法。
发明内容
有鉴于此，本发明提供了一种超分辨显微成像方法、装置、计算机设备及存储介质，其具有无参化、高通量、分辨率高的优点，能够克服现有技术的缺点与不足之处。
本发明的第一个目的在于提供一种超分辨显微成像方法
本发明的第二个目的在于提供一种超分辨显微成像装置。
本发明的第三个目的在于提供一种计算机设备。
本发明的第四个目的在于提供一种存储介质。
本发明的第一个目的可以通过采取如下技术方案达到：
一种超分辨显微成像方法，所述方法包括：
采集一组待观测样本的荧光信号序列；
判断解卷积的迭代次数；
对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
对输出的图像进行两次重建；
对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
在一个实施例中，所述采集一组待观测样本的序列之后，还包括：
利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
在一个实施例中，所述利用小波变换对荧光信号的背景进行估计，并去除背景噪声，具体包括：
利用小波估计从输入图像的最低频率估计得到背景；
将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半进行比较，通过保持每个像素的最小值来合并这两个图像；
将估计得到的低频带低峰值背景数据作为新的输入图像，循环进行小波估计，直至达到预设循环次数。
在一个实施例中，所述判断解卷积的迭代次数，具体包括：
使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数，具体包括：
将滚动傅里叶环相关作为空间频率的函数，定义滚动傅里叶环相关曲线空间频率的离散化计算对应空间频率的离散值；
采用平均窗半宽的平均滤波器来平滑有噪声的滚动傅里叶环相关曲线；
当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，分辨率为有效截止频率的倒数，所述给定阈值表示随机噪声外有意义信息的最大空间频率；
将达到最大分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述对输出的图像进行两次重建，具体包括：
利用荧光信号涨落原理对输出的图像进行第一次重建；
应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建。
在一个实施例中，所述利用荧光信号涨落原理对输出的图像进行第一次重建，具体包括：
根据显微镜的点扩散函数、荧光分子亮度常数以及荧光分子亮度随时间变化波动的函数，获得荧光信号的表达式；
利用每个荧光分子的个体波动特征，根据荧光信号的表达式，获得零时延的时间累积量表达式；
将时间累积量的表达式展开，当符合预设条件时，将时间累积量展开式的互相关项视为零，使时间累积量表示为对应亮度常量加权点扩散函数的平方和。
在一个实施例中，所述应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建，具体包括：
构造重建模型，所述重建模型包括第一项、第二项和第三项，所述第一项为保真度项，表示第一次重建后的图像与采集的初始图像之间的距离，所述第二项表示第一次重建后的图像的连续性约束，所述第三项表示第一次重建后的图像的稀疏性约束；
利用重建模型对第一次重建后的图像进行第二次重建。
在一个实施例中，解卷积的迭代，具体包括：
在空域利用迭代求解最大似然的方式，得到迭代式；
根据迭代式，采用基于矢量外推的加速方式实现对解卷积的迭代计算。
本发明的第二个目的可以通过采取如下技术方案达到：
一种超分辨显微成像装置，所述装置包括：
采集模块，用于采集一组待观测样本的荧光信号序列；
判断模块，用于判断解卷积的迭代次数；
预解卷积模块，对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
重建模块，用于对输出的图像进行两次重建；
解卷积模块，用于对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
本发明的第三个目的可以通过采取如下技术方案达到：
一种计算机设备，包括处理器以及用于存储处理器可执行程序的存储器，其特征在于，所述处理器执行存储器存储的程序时，实现上述的超分辨显微成像方法。
本发明的第四个目的可以通过采取如下技术方案达到：
一种存储介质，存储有程序，所述程序被处理器执行时，实现上述的超分辨显微成像方法。
本发明相对于现有技术具有如下的有益效果：
本发明具有灵活性高的优点，能够广泛的耦合于各类成像模态，例如声学显微，即光声与超声显微成像技术中；具有高通量的优点，使用多迭代次数的后解卷积对得到的单幅图像进行进一步处理，可实现2倍的三维空间分辨率提升；在保持超分辨率的同时只需要20帧以下就可以实现高质量的超分辨效果，实现了50～100倍的时间分辨率提升；具有无参化的优点，提出基于滚动傅里叶环相关的自动参数估计手段，使实现无参化，从而能实现自动超分辨高通量成像。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图示出的结构获得其他的附图。
图1为本发明实施例1的超分辨显微成像方法的流程图。
图2为本发明实施例1的三维空间分辨率提升的示例性结果图。
图3为本发明实施例1的时间分辨率提升的示例性结果图。
图4为本发明实施例1的无参化重建能力的示例性结果图。
图5为本发明实施例1的自动高通量重建的示例性结果图。
图6为本发明实施例2的超分辨显微成像装置的结构框图。
图7为本发明实施例3的计算机设备的结构框图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部的实施例，基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
实施例1：
如图1所示，本实施例提供了一种超分辨显微成像方法，该方法基于解卷积增强实现，包括以下步骤：
S101、采集一组待观测样本的荧光信号序列。
在一个实施例中，荧光信号序列的帧数为20，本领域技术人员可以理解，荧光信号序列的帧数还可以为10、50等。
在一个实施例中，对于有弱背景甚至没有背景的数据，则可以去掉背景估计操作，即将背景参数值b设为零，以避免信息的去除；特别地，考虑到低剂量照明下的图像一般只显示出低而稳定的背景荧光类噪声分布，直接将超过图像均值的值设为零，将得到的残差图像用于后续的背景估计；
在一个实施例中，在除了上述之外的其他条件下，步骤S101之后，还可包括：
S102、利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
在一个实施例中，步骤S102具体包括：
S1021、利用小波估计从输入图像的最低频率估计得到背景。
在一个实施例中，为了提取频域最低带，使用二维Daubechies-6小波滤波器将信号多级分解到7级。
S1022、将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半进行比较，通过保持每个像素的最小值来合并这两个图像。
在一个实施例中，为了防止意外去除微小的有用信号，将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半(更平滑的信息)进行比较，通过保持每个像素的最小值来合并这两个图像，该操作去除了由于不准确的背景估计而产生的在背景中的高强度像素。
S1023、将估计得到的低频带低峰值背景数据作为新的输入图像，循环进行小波估计，即重复执行步骤S1021和S1022，直至达到预设循环次数。
在一个实施例中，将预设循环次数设置为3次，以估计具有最小分布的真实荧光背景。
S102、判断解卷积的迭代次数。
在一个实施例中，判断解卷积的迭代次数，具体包括：使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，使用滚动傅里叶环相关(Fourier Ring Correlation，简称FRC)进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数，具体包括：
S1021、将滚动傅里叶环相关作为空间频率的函数，定义滚动傅里叶环相关曲线空间频率的离散化计算对应空间频率的离散值。
在一个实施例中，滚动傅里叶环相关测量傅立叶域中一系列同心环上的两个二维信号之间的统计相关性，可以将滚动傅里叶环相关视为空间频率q i的函数：
Figure PCTCN2022121400-appb-000001
其中，
Figure PCTCN2022121400-appb-000002
和
Figure PCTCN2022121400-appb-000003
表示的是两个信号的离散傅里叶变换，累加号表示对应空间频率qi的圆周界上像素的总和。
定义滚动傅里叶环相关曲线空间频率的离散化来计算对应空间频率的离散值，其中最大频率fmax对应的是像素尺寸p s倒数的一半，即fmax＝1/(2p s)，滚动傅里叶环相关曲线由N/2个值组成，空间频率的离散化步长Δf为：
Figure PCTCN2022121400-appb-000004
S1022、采用平均窗半宽的平均滤波器来平滑有噪声的滚动傅里叶环相关曲线。
在一个实施例中，采用平均窗半宽的平均滤波器(通常等于3)来平滑有噪声的滚动傅里叶环相关曲线。
S1023、当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，分辨率为有效截止频率的倒数，所述给定阈值表示随机噪声外有意义信息的最大空间频率。
在一个实施例中，当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，而分辨率则是有效截止频率的倒数，给定阈值表示随机噪声外有意义信息的最大空间频率，具体来说，阈值的常见选择是固定值阈值或σ因子曲线。固定值通常为1/7硬阈值，σ因子曲线的判据可写为：
Figure PCTCN2022121400-appb-000005
其中，N i表示半径为q i的环中的像素数，最常用的σ因子为3，如果两个测量值只含有噪声，则滚动傅里叶环相关曲线可以表示为
Figure PCTCN2022121400-appb-000006
故3σ因子曲线实际上是大于纯噪声三倍相关度的频率成分判定是相对有效信息。
S1024、将达到最大分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，将达到最大分辨率时的迭代次数记为k，k即为解卷积的迭代次数。
S103、对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出。
在一个实施例中，在迭代达到步骤S102判断的迭代次数的一半时完成预解卷积的迭代，通过执行预解卷积，可以提高信号的有效开/关对比度和信噪比，解卷积的迭代，具体包括：
S1031、在空域利用迭代求解最大似然的方式，得到迭代式。
在一个实施例中，解卷积采用Richardson-Lucy(RL)算法，解卷积模型基于泊松噪声模型，在空域利用迭代求解最大似然的方式，得到如下迭代式：
Figure PCTCN2022121400-appb-000007
其中，x和y为空间坐标，h分别表示显微镜的点扩散函数(Point spread function，简称PSF)，f表示物理世界中的真实荧光信号，g表示最终显微镜采集得到的信号。
S1032、根据迭代式，采用基于矢量外推的加速方式实现对解卷积的迭代计算。
在一个实施例中，为了加速迭代收敛速度，采用基于矢量外推的加速方式实现对解卷积的迭代计算：
Figure PCTCN2022121400-appb-000008
v j＝x j+1-y j
Figure PCTCN2022121400-appb-000009
x j+1＝y j+1+α j+1·(y j+1-y j)
其中，g为前面先验知识约束重建后的图像，h表示显微镜的点扩散函数，x j+1为j+1次迭代后的图像，α为自适应加速因子。
S104、对输出的图像进行两次重建
在一个实施例中，步骤S104具体包括：
S1041、利用荧光信号涨落原理对输出的图像进行第一次重建。
S10411、根据显微镜的点扩散函数、荧光分子亮度常数以及荧光分子亮度随时间变化波动的函数，获得荧光信号的表达式；
在一个实施例中，成像样本通常被认为是由N个单独的、位于rk的荧光分子组成的，假设荧光分子具有各自独立的随时间变化的分子亮度，位于r处和时间t的荧光信号表示为：
F(r,t)＝h(r-r k)·c k·ω k(t)
其中，h、c、ω分别表示对应显微镜的点扩散函数、分子亮度常数以及分子亮度随时间变化波动的函数。
S10412、利用每个荧光分子的个体波动特征，根据荧光信号的表达式，获得零时延的时间累积量表达式；
在一个实施例中，时间累积量为二阶时间累积量，本领域技术人员可以理解，时间累积量还可以为三阶时间累积量、四阶时间累积量。
在一个实施例中，根据荧光涨落超分辨成像技术，利用每个荧光分子的这种个体波动特征，计算每个像素沿着t轴的相关累积量来提高分辨率，计算零时延的二阶时间累积量G 2得到下式：
G 2(r)＝<δF(r,t)·δF(r,t)> t
δF(r,t)＝F(r,t)-<F(r,t)> t
其中，<·> t为时间平均函数。
S10413、将时间累积量的表达式展开，当符合预设条件时，将时间累积量展开式的互相关项视为零，使时间累积量表示为对应亮度常量加权点扩散函数的平方和。
在一个实施例中，将二阶时间累积量G 2的表达式展开，得到下式：
Figure PCTCN2022121400-appb-000010
假设每个荧光分子的发光强度都是不相关联的单独涨落，当i≠k(预设条件)时，将展开式中的互相关项视为零，二阶时间累积量G 2表示为对应亮度常量γ加权点扩散函数的平方和，如下式：
Figure PCTCN2022121400-appb-000011
S1042、应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建。
S10421、构造重建模型，所述重建模型包括第一项、第二项和第三项，所述第一项为保真度项，表示第一次重建后的图像与采集的初始图像之间的距离，所述第二项表示第一次重建后的图像的连续性约束， 所述第三项表示第一次重建后的图像的稀疏性约束；
在一个实施例中，构造重建模型，如下式：
Figure PCTCN2022121400-appb-000012
其中，左侧第一项为保真度项，表示恢复后的图像x与采集的初始图像f之间的距离，A为光学系统的点扩散函数，第二项和第三项分别是连续性和稀疏性约束，||·|| 1与||·|| 2分别为l 1和l 2范数，而λ和λ L1分别表示保真度和稀疏性约束的权重；将xy-t(z)轴的Hessian矩阵连续性先验定义为：
Figure PCTCN2022121400-appb-000013
S10422、利用重建模型对第一次重建后的图像进行第二次重建。
S106、对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
在一个实施例中，在迭代达到步骤S102判断的迭代次数的一半(即k/2次)时完成第二次解卷积的迭代，预解卷积和第二次解卷积的迭代均为k/2次，刚好达到步骤S102判断的迭代次数，第二次解卷积的迭代的具体内容可以参见步骤S103，在此不再一一赘述，通过执行两次解卷积迭代(预解卷积迭代和第二次解卷积迭代)，在保持图像质量和最小化伪影的同时进一步提高分辨率。
本领域技术人员可以理解，还可以在迭代达到步骤S102判断的迭代次数时完成第二次解卷积的迭代。
图2是表明本发明实施例的三维空间分辨率提升的示例性结果，对比传统的共焦模式可实现2倍的三维空间分辨率提升，使用本发明实施例的方法测量了量子点(QD525)样品的三维PSF，用标准的转盘共焦显微镜获取图像序列，分别提取了这样的分子点源的横向/轴向半高宽。传统共焦模式的横向/轴向半高宽为325/655nm，传统荧光涨落模式为295/510nm，而本发明实施例为135/334nm，。
图3是表明本发明实施例的时间分辨率提升的示例性结果(COS-7细胞中利用量子点标记的微管)，本发明实施例在保持超分辨率同时只需要20帧就可以实现高质量的超分辨效果，实现了对比传统荧光涨落技术(SOFI)50～100倍的时间分辨率提升，在信噪比逐渐降低的情况下，传统荧光涨落技术需要1000帧图像重建才能保持性能相对稳定，而本发明实施例在所有条件下都能只使用20帧图像以高保真度重建两点结构。
图4和图5是分别表明本发明实施例的无参化重建能力和自动高通量重建的示例性结果，本发明实施例具有无参化的优点，提出基于滚动傅里叶环相关的自动参数估计手段，使实现无参化，从而能实现自动超分辨高通量成像(COS-7细胞中利用量子点标记的微管)。本实施例对2.0mm×1.4mm的大视场进行成像，其包含超过2000个细胞，由约2,400,000,000个(24亿)像素(每个像素32.5nm×32.5nm)构成，跨越了几乎五个数量级的区域空间尺度，本实施例对其重建只需要约10分钟，而传统荧光涨落技术(SOFI)需要约17小时才能实现相似的成像性能。
应当注意，尽管以特定顺序描述了上述实施例的方法操作，但是这并非要求或者暗示必须按照该特定顺序来执行这些操作，或是必须执行全部所示的操作才能实现期望的结果。相反，描绘的步骤可以改变执行顺序。附加地或备选地，可以省略某些步骤，将多个步骤合并为一个步骤执行，和/或将一个步骤分解为多个步骤执行。
实施例2：
如图6所示，本实施例提供了一种超分辨显微成像装置，该装置包括采集模块601、小波变换模块602、 判断模块603、预解卷积模块604、重建模块605和解卷积模块606，各个模块的具体说明如下：
采集模块601，用于采集一组待观测样本的荧光信号序列。
小波变换模块602，用于利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
判断模块603，用于判断解卷积的迭代次数。
预解卷积模块604，对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出。
重建模块605，用于对输出的图像进行两次重建。
解卷积模块606，用于对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
需要说明的是，本实施例提供的系统仅以上述各功能模块的划分进行举例说明，在实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
实施例3：
本实施例提供了一种计算机设备，该计算机设备可以是计算机，如图7所示，其包括通过系统总线701连接的处理器702、存储器、输入装置703、显示器704和网络接口705，该处理器用于提供计算和控制能力，该存储器包括非易失性存储介质706和内存储器707，该非易失性存储介质706存储有操作系统、计算机程序和数据库，该内存储器707为非易失性存储介质中的操作系统和计算机程序的运行提供环境，处理器702执行存储器存储的计算机程序时，实现上述实施例1的超分辨显微成像方法，如下：
采集一组待观测样本的荧光信号序列；
判断解卷积的迭代次数；
对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
对输出的图像进行两次重建；
对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
在一个实施例中，所述采集一组待观测样本的序列之后，还包括：
利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
在一个实施例中，所述利用小波变换对荧光信号的背景进行估计，并去除背景噪声，具体包括：
利用小波估计从输入图像的最低频率估计得到背景；
将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半进行比较，通过保持每个像素的最小值来合并这两个图像；
将估计得到的低频带低峰值背景数据作为新的输入图像，循环进行小波估计，直至达到预设循环次数。
在一个实施例中，所述判断解卷积的迭代次数，具体包括：
使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数，具体包括：
将滚动傅里叶环相关作为空间频率的函数，定义滚动傅里叶环相关曲线空间频率的离散化计算对应空间频率的离散值；
采用平均窗半宽的平均滤波器来平滑有噪声的滚动傅里叶环相关曲线；
当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，分辨率为有效截止频率的倒数，所述给定阈值表示随机噪声外有意义信息的最大空间频率；
将达到最大分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述对输出的图像进行两次重建，具体包括：
利用荧光信号涨落原理对输出的图像进行第一次重建；
应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建。
在一个实施例中，所述利用荧光信号涨落原理对输出的图像进行第一次重建，具体包括：
根据显微镜的点扩散函数、荧光分子亮度常数以及荧光分子亮度随时间变化波动的函数，获得荧光信号的表达式；
利用每个荧光分子的个体波动特征，根据荧光信号的表达式，获得零时延的时间累积量表达式；
将时间累积量的表达式展开，当符合预设条件时，将时间累积量展开式的互相关项视为零，使时间累积量表示为对应亮度常量加权点扩散函数的平方和。
在一个实施例中，所述应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建，具体包括：
构造重建模型，所述重建模型包括第一项、第二项和第三项，所述第一项为保真度项，表示第一次重建后的图像与采集的初始图像之间的距离，所述第二项表示第一次重建后的图像的连续性约束，所述第三项表示第一次重建后的图像的稀疏性约束；
利用重建模型对第一次重建后的图像进行第二次重建。
在一个实施例中，解卷积的迭代，具体包括：
在空域利用迭代求解最大似然的方式，得到迭代式；
根据迭代式，采用基于矢量外推的加速方式实现对解卷积的迭代计算。
实施例4：
本实施例提供了一种存储介质，该存储介质为计算机可读存储介质，其存储有计算机程序，计算机程序被处理器执行时，实现上述实施例1的超分辨显微成像方法，如下：
采集一组待观测样本的荧光信号序列；
判断解卷积的迭代次数；
对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
对输出的图像进行两次重建；
对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
在一个实施例中，所述采集一组待观测样本的序列之后，还包括：
利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
在一个实施例中，所述利用小波变换对荧光信号的背景进行估计，并去除背景噪声，具体包括：
利用小波估计从输入图像的最低频率估计得到背景；
将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半进行比较，通过保持每个像素的最小值来合并这两个图像；
将估计得到的低频带低峰值背景数据作为新的输入图像，循环进行小波估计，直至达到预设循环次数。
在一个实施例中，所述判断解卷积的迭代次数，具体包括：
使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数，具体包括：
将滚动傅里叶环相关作为空间频率的函数，定义滚动傅里叶环相关曲线空间频率的离散化计算对应空间频率的离散值；
采用平均窗半宽的平均滤波器来平滑有噪声的滚动傅里叶环相关曲线；
当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，分辨率为有效截止频率的倒数，所述给定阈值表示随机噪声外有意义信息的最大空间频率；
将达到最大分辨率时的迭代次数作为解卷积的迭代次数。
在一个实施例中，所述对输出的图像进行两次重建，具体包括：
利用荧光信号涨落原理对输出的图像进行第一次重建；
应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建。
在一个实施例中，所述利用荧光信号涨落原理对输出的图像进行第一次重建，具体包括：
根据显微镜的点扩散函数、荧光分子亮度常数以及荧光分子亮度随时间变化波动的函数，获得荧光信号的表达式；
利用每个荧光分子的个体波动特征，根据荧光信号的表达式，获得零时延的时间累积量表达式；
将时间累积量的表达式展开，当符合预设条件时，将时间累积量展开式的互相关项视为零，使时间累积量表示为对应亮度常量加权点扩散函数的平方和。
在一个实施例中，所述应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建，具体包括：
构造重建模型，所述重建模型包括第一项、第二项和第三项，所述第一项为保真度项，表示第一次重建后的图像与采集的初始图像之间的距离，所述第二项表示第一次重建后的图像的连续性约束，所述第三项表示第一次重建后的图像的稀疏性约束；
利用重建模型对第一次重建后的图像进行第二次重建。
在一个实施例中，解卷积的迭代，具体包括：
在空域利用迭代求解最大似然的方式，得到迭代式；
根据迭代式，采用基于矢量外推的加速方式实现对解卷积的迭代计算。
综上所述，本发明具有灵活性高的优点，能够广泛的耦合于各类成像模态，例如声学显微，即光声与超声显微成像技术中；具有高通量的优点，使用多迭代次数的后解卷积对得到的单幅图像进行进一步处理，可实现2倍的三维空间分辨率提升；在保持超分辨率的同时只需要20帧就可以实现高质量的超分辨效果，实现了50～100倍的时间分辨率提升；具有无参化的优点，提出基于滚动傅里叶环相关的自动参数估计手段，使实现无参化，从而能实现自动超分辨高通量成像。
需要说明的是，本实施例的计算机可读存储介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
在本实施例中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本实施例中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读存储介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读存储介质上包含的计算机程序可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读存储介质可以以一种或多种程序设计语言或其组合来编写用于执行本实施例的计算机程序，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Python、C++，还包括常规的过程式程序设计语言—诸如C语言或类似的程序设计语言。程序可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
本申请各部分操作所需的计算机程序代码可以用任意一种或以上程序设计语言编写，包括如Java、Scala、Smalltalk、Eiffel、JADE、Emerald、C++、C#、VB.NET、Python等的面向对象程序设计语言、如C程序 设计语言、VisualBasic、Fortran2103、Perl、COBOL2102、PHP、ABAP的常规程序化程序设计语言、如Python、Ruby和Groovy的动态程序设计语言或其它程序设计语言等。程序代码可以完全在用户计算机上执行，可以部分在用户计算机上作为独立软件包执行，可以部分在用户计算机上并部分在远程计算机上执行，或者完全在远程计算机或服务器上执行。在后种情况下，远程计算机可以通过任何网络形式与用户计算机连接，比如局域网(LAN)或广域网(WAN)，或连接至外部计算机(例如通过因特网)，或在云计算环境中，或作为服务使用如软件即服务(SaaS)。
此外，除非权利要求中明确说明，本申请所述处理元素和序列的顺序、数字字母的使用、或其它名称的使用，并非用于限定本申请流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本申请实施例实质和范围的修正和等价组合。例如，尽管上述各种组件的实现可以体现在硬件设备中，但也可以实现为纯软件解决方案，例如，在现有服务器或移动设备上的安装。
最后，应当理解的是，本申请中所述实施例仅用以说明本申请实施例的原则。其他的变形也可能属于本申请的范围。因此，作为示例而非限制，本申请实施例的替代配置可视为与本申请的教导一致。相应地，本申请的实施例不仅限于本申请明确介绍和描述的实施例。

Claims (12)

  1. 一种超分辨显微成像方法，其特征在于，所述方法包括：
    采集一组待观测样本的荧光信号序列；
    判断解卷积的迭代次数；
    对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
    对输出的图像进行两次重建；
    对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
  2. 根据权利要求1所述的超分辨显微成像方法，其特征在于，所述采集一组待观测样本的序列之后，还包括：
    利用小波变换对荧光信号的背景进行估计，并去除背景噪声。
  3. 根据权利要求2所述的超分辨显微成像方法，其特征在于，所述利用小波变换对荧光信号的背景进行估计，并去除背景噪声，具体包括：
    利用小波估计从输入图像的最低频率估计得到背景；
    将最低频带进行小波逆变换到空间域，并将结果与输入图像平方根的一半进行比较，通过保持每个像素的最小值来合并这两个图像；
    将估计得到的低频带低峰值背景数据作为新的输入图像，循环进行小波估计，直至达到预设循环次数。
  4. 根据权利要求1所述的超分辨显微成像方法，其特征在于，所述判断解卷积的迭代次数，具体包括：
    使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数。
  5. 根据权利要求4所述的超分辨显微成像方法，其特征在于，所述使用滚动傅里叶环相关进行判断，将达到最大滚动傅里叶环相关分辨率时的迭代次数作为解卷积的迭代次数，具体包括：
    将滚动傅里叶环相关作为空间频率的函数，定义滚动傅里叶环相关曲线空间频率的离散化计算对应空间频率的离散值；
    采用平均窗半宽的平均滤波器来平滑有噪声的滚动傅里叶环相关曲线；
    当滚动傅里叶环相关曲线低于给定阈值时，将频率定义为有效截止频率，分辨率为有效截止频率的倒数，所述给定阈值表示随机噪声外有意义信息的最大空间频率；
    将达到最大分辨率时的迭代次数作为解卷积的迭代次数。
  6. 根据权利要求1所述的超分辨显微成像方法，其特征在于，所述对输出的图像进行两次重建，具体包括：利用荧光信号涨落原理对输出的图像进行第一次重建；
    应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建。
  7. 根据权利要求6所述的超分辨显微成像方法，其特征在于，所述利用荧光信号涨落原理对输出的图像进行第一次重建，具体包括：
    根据显微镜的点扩散函数、荧光分子亮度常数以及荧光分子亮度随时间变化波动的函数，获得荧光信号的表达式；
    利用每个荧光分子的个体波动特征，根据荧光信号的表达式，获得零时延的时间累积量表达式；
    将时间累积量的表达式展开，当符合预设条件时，将时间累积量展开式的互相关项视为零，使时间累积量表示为对应亮度常量加权点扩散函数的平方和。
  8. 根据权利要求6所述的超分辨显微成像方法，其特征在于，所述应用稀疏-连续性联合约束对第一次重建后的图像进行第二次重建，具体包括：
    构造重建模型，所述重建模型包括第一项、第二项和第三项，所述第一项为保真度项，表示第一次重建后的图像与采集的初始图像之间的距离，所述第二项表示第一次重建后的图像的连续性约束，所述第三项表示第一次重建后的图像的稀疏性约束；
    利用重建模型对第一次重建后的图像进行第二次重建。
  9. 根据权利要求1-8任一项所述的超分辨显微成像方法，其特征在于，解卷积的迭代，具体包括：
    在空域利用迭代求解最大似然的方式，得到迭代式；
    根据迭代式，采用基于矢量外推的加速方式实现对解卷积的迭代计算。
  10. 一种超分辨显微成像装置，其特征在于，所述装置包括：
    采集模块，用于采集一组待观测样本的荧光信号序列；
    判断模块，用于判断解卷积的迭代次数；
    预解卷积模块，对所述荧光信号序列中的每一帧初始图像执行预解卷积的迭代，在迭代达到所述迭代次数的一半时进行输出；
    重建模块，用于对输出的图像进行两次重建；
    解卷积模块，用于对两次重建后的图像执行第二次解卷积的迭代，在迭代达到所述迭代次数或所述迭代次数的一半时进行输出。
  11. 一种计算机设备，包括处理器以及用于存储处理器可执行程序的存储器，其特征在于，所述处理器执行存储器存储的程序时，实现权利要求1-9任一项所述的超分辨显微成像方法。
  12. 一种存储介质，存储有程序，其特征在于，所述程序被处理器执行时，实现权利要求1-9任一项所述的超分辨显微成像方法。
PCT/CN2022/121400 2022-09-26 2022-09-26 超分辨显微成像方法、装置、计算机设备及存储介质 WO2024065094A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CA3210853A CA3210853A1 (en) 2022-09-26 2022-09-26 Super-resolution microscopic imaging method and apparatus, computer device, and storage medium
PCT/CN2022/121400 WO2024065094A1 (zh) 2022-09-26 2022-09-26 超分辨显微成像方法、装置、计算机设备及存储介质
CN202280017314.5A CN117157532A (zh) 2022-09-26 2022-09-26 超分辨显微成像方法、装置、计算机设备及存储介质
EP23199782.6A EP4345735A1 (en) 2022-09-26 2023-09-26 Super-resolution microscopic imaging method and apparatus, computer device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/121400 WO2024065094A1 (zh) 2022-09-26 2022-09-26 超分辨显微成像方法、装置、计算机设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024065094A1 true WO2024065094A1 (zh) 2024-04-04

Family

ID=88287522

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/121400 WO2024065094A1 (zh) 2022-09-26 2022-09-26 超分辨显微成像方法、装置、计算机设备及存储介质

Country Status (4)

Country Link
EP (1) EP4345735A1 (zh)
CN (1) CN117157532A (zh)
CA (1) CA3210853A1 (zh)
WO (1) WO2024065094A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009095296A1 (en) * 2008-01-29 2009-08-06 Bergen Teknologioverforing As Constrained iterative blind deconvolution
CN102708543A (zh) * 2012-04-19 2012-10-03 北京工商大学 基于盲反卷积和稀疏表示的荧光显微图像复原方法和装置
CN106770131A (zh) * 2017-01-17 2017-05-31 清华大学 三维超光谱显微成像系统及成像方法
CN106952233A (zh) * 2017-03-24 2017-07-14 深圳大学 荧光多分子定位方法、装置以及超分辨成像方法、系统
CN108318464A (zh) * 2018-01-23 2018-07-24 深圳大学 一种超分辨荧光波动显微成像方法、装置及存储介质
CN113112405A (zh) * 2021-04-12 2021-07-13 广州超视计生物科技有限公司 超分辨率显微镜图像的自适应校正方法及sim-odt双模态系统
US11189016B1 (en) * 2020-06-08 2021-11-30 Guangzhou Computational Super-Resolution Biotech Co., Ltd. Systems and methods for image processing
CN113971722A (zh) * 2021-12-23 2022-01-25 清华大学 一种傅里叶域的光场解卷积方法及装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009095296A1 (en) * 2008-01-29 2009-08-06 Bergen Teknologioverforing As Constrained iterative blind deconvolution
CN102708543A (zh) * 2012-04-19 2012-10-03 北京工商大学 基于盲反卷积和稀疏表示的荧光显微图像复原方法和装置
CN106770131A (zh) * 2017-01-17 2017-05-31 清华大学 三维超光谱显微成像系统及成像方法
CN106952233A (zh) * 2017-03-24 2017-07-14 深圳大学 荧光多分子定位方法、装置以及超分辨成像方法、系统
CN108318464A (zh) * 2018-01-23 2018-07-24 深圳大学 一种超分辨荧光波动显微成像方法、装置及存储介质
US11189016B1 (en) * 2020-06-08 2021-11-30 Guangzhou Computational Super-Resolution Biotech Co., Ltd. Systems and methods for image processing
CN114998100A (zh) * 2020-06-08 2022-09-02 广州超视计生物科技有限公司 一种图像处理系统和方法
CN113112405A (zh) * 2021-04-12 2021-07-13 广州超视计生物科技有限公司 超分辨率显微镜图像的自适应校正方法及sim-odt双模态系统
CN113971722A (zh) * 2021-12-23 2022-01-25 清华大学 一种傅里叶域的光场解卷积方法及装置

Also Published As

Publication number Publication date
CN117157532A (zh) 2023-12-01
CA3210853A1 (en) 2024-03-26
EP4345735A1 (en) 2024-04-03

Similar Documents

Publication Publication Date Title
Zhang et al. Deep unfolding network for image super-resolution
Li et al. A reweighted l2 method for image restoration with poisson and mixed poisson-gaussian noise
CN107481192B (zh) 图像处理方法、装置、存储介质、计算机程序和电子设备
JP2016058081A (ja) ぼやけた画像に基づいて鮮明な画像を生成する方法および装置
EP3105736A1 (en) Method for performing super-resolution on single images and apparatus for performing super-resolution on single images
KR20150034656A (ko) 다수의 웨이블렛 도메인 희소성을 이용한 단일­이미지 초고해상도 및 잡음제거
JP2014199664A (ja) グレースケール画像の処理方法及び装置
CN111178499B (zh) 一种基于生成对抗网络改进的医学图像超分辨率方法
Lv et al. An efficient nonconvex regularization for wavelet frame and total variation based image restoration
Hao et al. Fast L1 regularized iterative forward backward splitting with adaptive parameter selection for image restoration
Lee et al. Blind inverse gamma correction with maximized differential entropy
Witwit et al. Global motion based video super-resolution reconstruction using discrete wavelet transform
WO2024065094A1 (zh) 超分辨显微成像方法、装置、计算机设备及存储介质
Ponti et al. Image restoration using gradient iteration and constraints for band extrapolation
Ray et al. Minimax theory for a class of nonlinear statistical inverse problems
CN111714124A (zh) 磁共振电影成像方法、装置、成像设备及存储介质
Klann et al. Two-step regularization methods for linear inverse problems.
KR101303647B1 (ko) 영상 복원을 이용한 영상의 해상도 향상 시스템 및 방법
KR20130101829A (ko) 강건하게 비균일 모션 블러를 추정하는 방법 및 장치
CN112801899B (zh) 基于互补结构感知的内外循环驱动图像盲去模糊方法和装置
CN109087257B (zh) 一种基于参数估计框架的空域增量图像滤波方法
Solanki et al. An efficient satellite image super resolution technique for shift-variant images using improved new edge directed interpolation
JP2022142757A (ja) リモートフォトプレチスモグラフィモデルを訓練するためのシステムと方法
Rodriguez Total variation regularization for Poisson vector-valued image restoration with a spatially adaptive regularization parameter selection
Hao et al. Iterative total variation image deblurring with varying regularized parameter

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2023563283

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22959746

Country of ref document: EP

Kind code of ref document: A1