WO2019076192A1 - 图像重建方法、装置及显微成像装置 - Google Patents

图像重建方法、装置及显微成像装置 Download PDF

Info

Publication number
WO2019076192A1
WO2019076192A1 PCT/CN2018/108865 CN2018108865W WO2019076192A1 WO 2019076192 A1 WO2019076192 A1 WO 2019076192A1 CN 2018108865 W CN2018108865 W CN 2018108865W WO 2019076192 A1 WO2019076192 A1 WO 2019076192A1
Authority
WO
WIPO (PCT)
Prior art keywords
fiber
image
gray value
center position
center
Prior art date
Application number
PCT/CN2018/108865
Other languages
English (en)
French (fr)
Inventor
邵金华
孙锦
段后利
王强
Original Assignee
苏州微景医学科技有限公司
南京亘瑞医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州微景医学科技有限公司, 南京亘瑞医疗科技有限公司 filed Critical 苏州微景医学科技有限公司
Priority to MX2020003985A priority Critical patent/MX2020003985A/es
Priority to RU2020115467A priority patent/RU2747129C1/ru
Priority to EP18868288.4A priority patent/EP3699576A4/en
Priority to JP2020541842A priority patent/JP7064796B2/ja
Priority to CA3079243A priority patent/CA3079243C/en
Priority to BR112020007609-0A priority patent/BR112020007609B1/pt
Priority to KR1020207013484A priority patent/KR102358848B1/ko
Priority to AU2018352821A priority patent/AU2018352821B2/en
Publication of WO2019076192A1 publication Critical patent/WO2019076192A1/zh
Priority to US16/850,077 priority patent/US11449964B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6408—Fluorescence; Phosphorescence with measurement of decay time, time resolved fluorescence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458—Fluorescence microscopy
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6486—Measuring fluorescence of biological material, e.g. DNA, RNA, cells
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0036—Scanning details, e.g. scanning stages
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052—Optical details of the image generation
    • G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/008—Details of detection or image processing, including general computer control
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/02—Objectives
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/06—Means for illuminating specimens
    • G02B21/08—Condensers
    • G02B21/082—Condensers for incident illumination only
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/16—Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N2021/6463—Optics
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N2021/6484—Optical fibres
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2201/00—Features of devices classified in G01N21/00
    • G01N2201/06—Illumination; Optics
    • G01N2201/063—Illuminating optical parts
    • G01N2201/0635—Structured illumination, e.g. with grating
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2201/00—Features of devices classified in G01N21/00
    • G01N2201/08—Optical fibres; light guides
    • G01N2201/0826—Fibre array at source, distributing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N2201/00—Features of devices classified in G01N21/00
    • G01N2201/08—Optical fibres; light guides
    • G01N2201/0833—Fibre array at detector, resolving
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10064—Fluorescence image

Definitions

  • the present invention relates to image processing technologies, and in particular, to an image reconstruction method, apparatus, and microscopic imaging apparatus.
  • a structured illumination based microscope has a sectioning imaging function that suppresses out of focus noise, and has the advantages of simple structure and fast imaging speed compared with a confocal microscope.
  • it is often used as a conventional endoscope to scan the internal organs of the human digestive tract and observe changes in cell shape to predict tumor occurrence and evolution in advance, which has important guiding significance for cancer screening.
  • the structured light illumination-based microscope specifically emits fluorescence for exciting the fluorescence of the living body through the exciter, and the fluorescence then passes through the grating to form a sinusoidal light source of black and white stripes; and then the scanning body is collected by moving the grating pitch of 1/3 of the grating each time.
  • Multiple images returned after the cell for example, image I 1 , image I 2 , image I 3 ; then according to the root mean square formula
  • the image I 1 , the image I 2 , and the image I 3 are reconstructed to obtain a reconstructed image I.
  • the reconstruction method needs to calculate the gray level of all pixels in the image I 1 , the image I 2 , and the image I 3 by using the root mean square formula. Therefore, a large amount of calculation time is consumed, and the residual of the grating and the fiber bundle honeycomb grid in the reconstructed image is conspicuous, and the image quality is not high.
  • the present invention provides an image reconstruction method, a device and a microscopic imaging device. In order to speed up the image reconstruction rate, the residual of the grating in the reconstructed image is removed, and the quality of the reconstructed image is improved.
  • the invention provides an image reconstruction method, comprising:
  • Spatial interpolation is performed by using the gray value of the center of the fiber to obtain gray values of other pixels in the fiber bundle in the reconstructed image to form the reconstructed image.
  • it also includes:
  • a target pixel point having a pixel value higher than a peripheral pixel value is confirmed in the original image, and the target pixel point is determined as a center position of each of the optical fibers in the bundle.
  • the original image of the fiber bundle for obtaining uniform fluorescence includes:
  • An average image of the plurality of fiber bundle images is obtained to form an original image of the uniformly fluorescent fiber bundle.
  • the method before performing the spatial interpolation, the method further includes:
  • the interpolation weight between each pixel point in the bundle and the center position of each fiber is determined based on the center position of each fiber.
  • the method further includes determining, by using the following method, the interpolation weight:
  • an interpolation weight between a pixel point within each triangular structure and a center position of each fiber is determined.
  • the method further includes acquiring a plurality of sample images by using the following method:
  • N-1 times are moved within a grating separation distance, and N sample images including the initial phase are obtained, and the preset phase interval is moved each time from the initial phase.
  • the method further includes:
  • the gray value of each fiber center in the fiber bundle in the reconstructed image is calculated according to the gray value of each fiber center position determined in the plurality of sample images, including:
  • the gray values of each fiber center position in the plurality of sample images are made to be different from each other, and the obtained difference is squared and re-squared to obtain a gray value of each fiber center in the fiber bundle in the reconstructed image.
  • the invention also provides an image reconstruction device, comprising:
  • a calculation module configured to calculate a gray value of each fiber center in the fiber bundle in the reconstructed image according to the gray value of each fiber center position determined in one or more sample images
  • it also includes:
  • a first acquisition module configured to acquire an original image of the uniformly fluorescent fiber bundle
  • a first determining module configured to identify, in the original image, a target pixel point whose pixel value is higher than a peripheral pixel value, and determine the target pixel point as a center position of each fiber in the fiber bundle.
  • the first acquiring module includes:
  • a collection sub-module for collecting a plurality of fiber bundle images with a preset step size within a range of a grating separation distance
  • the device further includes:
  • a second determining module configured to determine an interpolation weight between each pixel point in the fiber bundle and the center position of each fiber according to a center position of each fiber.
  • the device further includes:
  • a third determining module configured to form a plurality of triangular structures with a central position of each optical fiber and a central position of an adjacent optical fiber as a vertex; and determining, according to the triangular structure, a pixel point and each optical fiber in each triangular structure The interpolation weight between the center positions.
  • the device further includes:
  • a second acquiring module configured to move N-1 times within a grating separation distance according to a preset phase interval, to obtain an N sample image including an initial phase and moving the preset phase interval each time from the initial phase .
  • the preset phase interval is 120 degrees;
  • the device further includes:
  • a judging module configured to perform saturation judgment on a gray value of each fiber center position
  • a first processing module configured to: when there is an optical fiber whose center position gray value exceeds a preset saturation threshold in the sample image, determine that the optical fiber exceeding the preset saturation threshold is a fiber to be corrected; Correcting a gray value of a center position of the fiber to be corrected to the preset saturation threshold, and performing a calculation and reconstruction image according to the gray value of each fiber center position determined in the corrected sample image a step of grayscale values at the center of each fiber in the bundle;
  • a second processing module configured to: when there is no optical fiber whose center position gray value exceeds a preset saturation threshold in the sample image, according to the gray level of each fiber center position determined in the sample image Value, the step of calculating the gray value of each fiber center in the bundle in the reconstructed image.
  • the calculating module is specifically configured to compare gray values of each fiber center position in the plurality of sample images with each other, and obtain a difference between squares and squares to obtain an optical fiber in the reconstructed image.
  • the gray value of the center of each fiber in the bundle is specifically configured to compare gray values of each fiber center position in the plurality of sample images with each other, and obtain a difference between squares and squares to obtain an optical fiber in the reconstructed image. The gray value of the center of each fiber in the bundle.
  • the invention also provides a microscopic imaging device comprising:
  • a light emitting unit a phase adjusting unit, a steering unit, a fiber bundle including a plurality of optical fibers, a detecting unit, and a processing unit, wherein:
  • the light emitting unit is configured to emit excitation light
  • the phase adjustment unit is disposed at an optical path exit of the excitation light, and is connected to the processing unit, configured to adjust a phase of the excitation light according to a phase adjustment amount sent by the processing unit, to obtain excitation of different phases Light;
  • the steering unit is configured to steer excitation light of different phases to focus the steered excitation light along the fiber bundle to the tissue to be detected and to transmit fluorescence of different phases returned by the tissue to be detected;
  • the detecting unit is configured to collect fluorescence of different phases to form a plurality of sample images
  • the processing unit is connected to the detecting unit, configured to receive the plurality of sample images, and determine gray values of a center position of each fiber in the fiber bundle in the plurality of sample images, and calculate the reconstructed image
  • the gray value of each fiber center in the fiber bundle; the spatial interpolation of the gray value of the fiber center is used to obtain the gray value of other pixels in the fiber bundle in the reconstructed image to form the reconstructed image.
  • the phase adjustment unit includes: a motor and a grating
  • the motor is respectively connected to the processing unit and the grating, and is configured to drag the grating to move according to a phase adjustment amount sent by the processing unit, so that the excitation light is transmitted through the grating to obtain the The excitation light corresponding to the phase adjustment amount.
  • the motor comprises: a DC motor
  • the processing unit determines an equal interval phase adjustment amount according to a preset phase interval; the DC motor receives the equal interval phase adjustment amount, and drags the grating to move at an equally spaced distance within a grating pitch range So that the processing unit acquires a plurality of sample images corresponding to the preset phase interval.
  • the preset phase interval is 120 degrees; and the phase adjustment amount is three.
  • the light emitting unit includes: a laser for emitting excitation light; and a beam expander focus concentrator disposed at an exit of the excitation light of the laser for expanding the excitation light and One-dimensional focusing is a line beam.
  • the steering unit is a bisection mirror.
  • the method further includes: a filter; the filter is disposed between the phase adjustment unit and the steering unit for filtering out stray light.
  • the detecting unit comprises: a charge coupled device CCD.
  • the method further includes: an objective lens composed of a plurality of lenses; the objective lens is disposed between the steering unit and the fiber bundle for performing focusing processing on the excitation light after the steering unit is turned.
  • the image reconstruction method, apparatus and microscopic imaging apparatus of the present invention calculate the center of each fiber in the bundle in the reconstructed image by calculating the gray value of the center position of each fiber in one or more sample images. Gray value; spatial interpolation is performed by using the gray value of the center of the fiber to obtain the gray value of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the image reconstruction method reduces the calculation value of the gray value of each pixel by calculating only the gray value of the pixel point at the center position of the fiber, and then obtaining the gray value of the pixel of the entire image based on the spatial interpolation.
  • the rate of image reconstruction is accelerated, and the method helps to remove the residual of the grating and the fiber bundle honeycomb grid in the reconstructed image, and improves the image quality of the reconstructed image.
  • FIG. 1 is a flow chart of an image reconstruction method of the present invention shown in an exemplary embodiment
  • FIG. 2 is a schematic view of a structured light microscopic endoscope device of the embodiment shown in FIG. 1;
  • FIG. 3 is a flowchart of an image reconstruction method of the present invention shown in another exemplary embodiment
  • FIG. 4 is a schematic diagram of a triangular structure of a fiber optic pixel of the embodiment shown in FIG. 3;
  • FIG. 5 is a schematic structural diagram of an image reconstruction apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an image reconstruction apparatus according to another embodiment of the present invention.
  • FIG. 7 is a schematic structural view of a microscopic imaging apparatus of the present invention, which is shown in an exemplary embodiment
  • FIG. 8 is a schematic structural view of a microscopic imaging apparatus of the present invention shown in another exemplary embodiment.
  • FIG. 1 is a flow chart of an image reconstruction method of the present invention shown in an exemplary embodiment.
  • the image reconstruction method of the present invention is suitable for reconstruction of all optically imaged images, particularly for structured light based images.
  • the present embodiment takes a structured light based endoscope as an example to briefly explain the principle of structured light imaging:
  • CCD Charge-coupled Device
  • the modulated sinusoidal light source is focused on a focal plane of the tissue, and by exciting the fluorescence imaging for multiple phases (for example, three phases), the Neil formula is used to filter out the background fluorescence outside the focal plane, thereby realizing the layer.
  • Analysis of imaging Tomography is a kind of geophysical exploration that uses medical CT to invert the calculated information according to the ray scan and reconstruct the elastic wave and electromagnetic wave parameters of the rock mass in the measured range. Inversion interpretation method.
  • the grating-modulated structured light source can be expressed as
  • m is a modulation contrast
  • ⁇ is the magnification between the specimen plane and the grid plane
  • ⁇ is the wavelength
  • v is the actual spatial frequency.
  • NA is a numerical aperture.
  • Step 101 Calculate a gray value of each fiber center in the fiber bundle in the reconstructed image according to the gray value of each fiber center position determined in one or more sample images.
  • the structured light microscopic endoscope device shown in FIG. 2 drives the DC motor to drag the grating to move to obtain one or more sample images.
  • the sample image contains the pixel information transmitted by each fiber in the fiber bundle.
  • a fiber bundle usually consists of nearly 30,000 fibers (the number difference can reach several thousand). Pixel information is transmitted in each fiber, so the fiber bundle can be referred to as a multi-sensor.
  • the imaging of the fiber generally presents a hexagonal honeycomb shape in the image, and each fiber diameter is preferably 5 to 6 pixels. In a plurality of sample images, the center position of each fiber is determined, and the gray value of each central position pixel is obtained.
  • the method for determining the gray value of the center position can be obtained by using the root mean square formula described above, that is, the mean value of the gray value of the same center position in the plurality of sample images is obtained.
  • the calculated gray value mean value is used as the gray value of the center of the fiber in the reconstructed image, thereby obtaining the gray value of each fiber center in the fiber bundle in the reconstructed image.
  • Step 102 Perform spatial interpolation on the gray value of the fiber center to obtain gray values of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • a linear relationship between other pixel points in each fiber and the central location pixel is found by using the center position of each fiber as a reference, thereby determining that all pixels in each fiber are relative to the center pixel.
  • the interpolation weight of the point that is, the weight value of other pixels in each fiber relative to the central location pixel. Therefore, based on the interpolation weight between each pixel and the center of the fiber, the gray value of the center of the fiber is spatially interpolated, and the gray values of other pixels in the fiber bundle in the reconstructed image are obtained to form a reconstructed image.
  • the image reconstruction method of the embodiment calculates the gray value of each fiber center in the fiber bundle in the reconstructed image by using the gray value of each fiber center position determined in one or more sample images;
  • the gray value of the center is spatially interpolated to obtain the gray value of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the image reconstruction method reduces the calculation value of the gray value of each pixel by calculating only the gray value of the pixel point at the center position of the fiber, and then obtaining the gray value of the pixel of the entire image based on the spatial interpolation.
  • the rate of image reconstruction is accelerated, and the method helps to remove the residual of the grating and the fiber bundle honeycomb grid in the reconstructed image, and improves the image quality of the reconstructed image.
  • FIG. 3 is a flowchart of an image reconstruction method according to another embodiment of the present invention. As shown in FIG. 3, the image reconstruction method of the embodiment includes:
  • Step 301 Acquire an original image of the uniformly fluorescent fiber bundle.
  • Step 302 Confirm a target pixel point whose pixel value is higher than a peripheral pixel value in the original image, and determine the target pixel point as a center position of each optical fiber in the fiber bundle.
  • a uniform fluorescent image can be taken, and the uniform fluorescent image is used for accurately positioning the optical fiber.
  • a bundle of fibers typically consists of nearly 30,000 fibers (a few thousand in number). Pixel information is transmitted in each fiber, so the fiber bundle can be referred to as a multi-sensor.
  • the imaging of the fiber presents a hexagonal honeycomb shape in the image, and each fiber diameter is preferably 5 to 6 pixels.
  • the fibers are arranged irregularly in space instead of presenting rows or columns.
  • the central position of the optical fiber in this embodiment refers to the brightest point of the optical fiber center as the center of the optical fiber.
  • the so-called brightest point means that the target pixel point whose pixel value is higher than the peripheral pixel value is confirmed in the original image, and the target pixel point is determined as the optical fiber.
  • the center position of each fiber in the bundle with the coordinates of the brightest point of the center as the fiber coordinates, to locate other pixels in each fiber.
  • the captured image that is, the grating will exist in the original image.
  • the grating can be removed for imaging to obtain the original image of the fiber bundle with uniform fluorescence; alternatively, it can be separated by a grating.
  • multiple fiber bundle images with preset steps are collected; the average image of the plurality of fiber bundle images is obtained to form an original image of the uniformly fluorescent fiber bundle. That is to say, the DC motor in Fig. 2 is uniformly moved by a plurality of identical displacements within a grating pitch range, and then the acquired mean image is taken.
  • a method for obtaining an original image of a bundle of uniformly fluorescent fibers can be determined by a person skilled in the art, which is not specifically limited in this embodiment.
  • Step 303 Calculate a gray value of each fiber center in the fiber bundle in the reconstructed image according to the gray value of each fiber center position determined in the plurality of sample images.
  • the obtaining of the sample image may be performed by moving N-1 times within a grating separation distance according to a preset phase interval, and obtaining N frames including the initial phase and moving the preset phase interval from the initial phase.
  • Sample image For example, the grating is mounted and the grating is moved by the motor to obtain a sample image of the N fiber bundles. For example, before starting to acquire a sample image, take a sample image at the initial position of the motor; then move the motor to another position, and then take a sample image; the motor moves again, and then shoots, thereby obtaining N sample images.
  • the motor can be rotated clockwise to obtain the above N sample images, and after waiting for a period of time, the motor is moved counterclockwise in the opposite direction, and then N samples are acquired.
  • the image so that the image of the two structured lights can be reconstructed, and the accuracy of the reconstructed image is ensured by comparison.
  • the three sample images may be a 0 degree phase sample image I 1 (initial phase), a 120 degree phase sample image I 2 (moving a preset phase interval threshold), and a 240 degree phase sample image I 3 (moving two preset phases) Interval threshold), in the three sample images, according to the fiber center position, the gray level of the fiber center of the three phase images is retrieved, that is, the fiber center gray value G 1 , 120 degree phase sample of the 0 degree phase sample image I 1 is obtained. center of the fiber image gray value I G 2, 240 degree phase sample image fiber center gradation value I 2 3 G 3.
  • the calculation of the gray value of each fiber center in the fiber bundle in the reconstructed image may be performed, and the gray values of each fiber center position in the plurality of sample images are mutually different, and the obtained difference is squared. And re-opening, the gray value of each fiber center in the fiber bundle in the reconstructed image is obtained.
  • the three central gray values in the three sample images are made to be different from each other, and then the squared difference is added, and the squared differences are added and then the root number is added to calculate the gray level of the fiber center in the reconstructed image. value.
  • the disadvantage is that when the sample image is supersaturated, the central gray value is subtracted by two or two, which causes the gray value calculated by the center point to be a black point with a small gray scale. This causes the reconstructed image to appear in a black area, making it impossible to clearly image the cells. In order to avoid the problem that the image saturation causes the image to be unclear, the saturation of the gray level of the fiber center point can be taken. The reconstructed image will have a good chromatographic effect.
  • the step of determining the saturation of the gray value of each fiber center position may be increased, that is, If there is an optical fiber whose center position gray value exceeds a preset saturation threshold in the sample image, it is determined that the fiber exceeding the preset saturation threshold is the fiber to be corrected; the gray level of the center position of the fiber to be corrected is reconstructed in the image.
  • the value is corrected to a preset saturation threshold, and the step of calculating the gray value of each fiber center in the fiber bundle in the reconstructed image is performed according to the gray value of each fiber center position determined in the corrected sample image;
  • the calculation is performed to obtain the reconstructed image in the fiber bundle according to the gray value of each fiber center position determined in the sample image.
  • the preset saturation threshold may be determined according to the performance of the CCD, e.g., center of the fiber is determined gradation 0 degree phase sample image I 1 is the value of G 1, 120 degree phase sample fiber center gradation image I 2 value G 2, The fiber center gray value G 3 of the 240-degree phase sample image I 3 , whether the three gray values are greater than 4095, (4095 corresponds to the maximum value of the 12-bit image, indicating CCD saturation), and then does not adopt the above Neil formula The center point gray value of the reconstructed image is calculated, and the preset saturation threshold of 4095 is directly used as the center point gray value. This processing avoids the phenomenon that the sample image is visually opposite to the reconstructed structured light image.
  • the problem of image saturation may occur when the sample image is collected.
  • the exposure time to avoid camera parameters may be too long, the gain is too large; the sample fluorescent dyeing substance is prevented from being too rich; and the light intensity of the laser light emitted by the laser is not excessively strong.
  • the fiber exceeding the preset saturation threshold is the fiber to be corrected; the center of the fiber to be corrected is reconstructed in the image.
  • the gray value of the position is corrected to the preset saturation threshold. That is to say if calculated If the value exceeds the preset saturation threshold, the fiber is determined to be the fiber to be corrected, and the preset saturation threshold is also used as the gray value of the center position of the fiber, thereby achieving saturation correction of the sample image.
  • Step 304 Determine, according to a center position of each fiber, an interpolation weight between each pixel point in the fiber bundle and a center position of each fiber.
  • the corresponding image in the sample image can be found according to the center position of each fiber determined in the original image.
  • the center position of the fiber and the gray value of the center point is read.
  • Each fiber in the N sample images is positioned and its gray value is obtained. Therefore, for each fiber, it corresponds to the gray value of N central positions, based on the preset algorithm (the Neil formula of the root mean square as described above), the gray value of N central positions is obtained.
  • the mean value of the gray value is taken, and the calculated gray value value is used as the gray value of the center of the fiber in the reconstructed image.
  • a plurality of triangular structures can be formed by using the center position of each fiber and the center position of the adjacent fiber as a vertex; according to the triangular structure , determining the interpolation weight between the pixel points within each triangular structure and the center position of each fiber.
  • the center coordinates of the fiber can be obtained according to the region maximum value method, that is, the center position of the fiber A shown in FIG. 4 is taken as a vertex, and the three center positions of the fiber A and the adjacent fiber B and the fiber C form a triangle. So that the entire fiber bundle is divided into a plurality of triangles. The interpolation relationship between the pixel and the fiber is established by these triangles. Since the beam of light is roughly hexagonal, the distribution is irregular. Adjacent fibers do not have a horizontal or vertical coordinate alignment relationship, so it is not possible to interpolate the middle pixels by four regular vertices like conventional bilinear interpolation. However, with this triangular structure, the interpolation weight between the pixel points in each triangular structure and the center position of each fiber can also be determined.
  • the region maximum value method that is, the center position of the fiber A shown in FIG. 4 is taken as a vertex, and the three center positions of the fiber A and the adjacent fiber B and the fiber C form a triangle. So that the entire
  • Step 305 Perform spatial interpolation on the gray value of the fiber center to obtain gray values of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the center position of all the fibers included in the fiber bundle is determined in the original image.
  • the center position of each fiber is determined in the original image.
  • the weight that is, the weight value of other pixels in each fiber relative to the central location pixel.
  • Subsequent reconstruction of the sample image obtained after the structured light is irradiated to the tissue may be based on the calculated linear weight value in advance, and multiplied by the gray value of the optical fiber during reconstruction to obtain the gray value of the pixel to be interpolated to form a reconstructed image.
  • the image reconstruction method of the present embodiment reconstruction of the structured light image is obtained by using fiber positioning based on the pixel space of the triangle, and only the pixel of the center point of the fiber is calculated by using a Neil formula, and then the entire structure is interpolated and reconstructed. Light image.
  • the N sample images for example, the three sample images, are exactly 120 degrees out of phase, the traces of the raster are also absent. Therefore, the image reconstruction method of the present invention can greatly reduce the calculation amount of calculating the gray value of each pixel point, greatly speeding up the image reconstruction rate, and the method also helps to remove the raster and the fiber bundle honeycomb network in the reconstructed image. The residual of the grid improves the image quality of the reconstructed image.
  • FIG. 5 is a schematic structural diagram of an image reconstruction apparatus according to an embodiment of the present invention. As shown in FIG. 5, the image reconstruction apparatus of this embodiment includes:
  • the calculation module 1 is configured to calculate a gray value of each fiber center in the fiber bundle in the reconstructed image according to the gray value of each fiber center position determined in one or more sample images;
  • the module 2 is configured to spatially interpolate with the gray value of the fiber center to obtain gray values of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • This embodiment can be used to implement the embodiment shown in FIG. 1 , and its implementation principle is similar, and details are not described herein again.
  • the image reconstruction apparatus of this embodiment calculates the gradation value of each fiber center in the fiber bundle in the reconstructed image by using the gradation value of each fiber center position determined in the plurality of sample images;
  • the degree values are spatially interpolated to obtain gray values of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the image reconstruction method reduces the calculation value of the gray value of each pixel by calculating only the gray value of the pixel point at the center position of the fiber, and then obtaining the gray value of the pixel of the entire image based on the spatial interpolation.
  • the rate of image reconstruction is accelerated, and the method helps to remove the residual of the grating and the fiber bundle honeycomb grid in the reconstructed image, and improves the image quality of the reconstructed image.
  • FIG. 6 is a schematic structural diagram of an image reconstruction apparatus according to another embodiment of the present invention. As shown in FIG. 6, the image reconstruction apparatus of the embodiment further includes:
  • a first obtaining module 3 configured to acquire an original image of the uniformly fluorescent fiber bundle
  • the first determining module 4 is configured to confirm, in the original image, a target pixel point whose pixel value is higher than a peripheral pixel value, and determine the target pixel point as a center position of each fiber in the fiber bundle.
  • the first obtaining module 3 includes:
  • the collecting sub-module 31 is configured to collect a plurality of fiber bundle images with a preset step size within a range of a grating separation distance;
  • a sub-module 32 is formed for determining an average image of the plurality of fiber bundle images to form an original image of the uniformly fluorescent fiber bundle.
  • the device further includes:
  • the second determining module 5 is configured to determine an interpolation weight between each pixel point in the fiber bundle and a center position of each fiber according to a center position of each fiber.
  • the device further includes:
  • a third determining module 6 configured to form a plurality of triangular structures with a center position of each optical fiber and a center position of an adjacent optical fiber as a vertex; and determine a pixel point and each optical fiber in each triangular structure according to the triangular structure The interpolation weight between the center positions.
  • the device further includes:
  • the second obtaining module 7 is configured to move N-1 times within a grating separation distance according to a preset phase interval, and obtain N sample images including an initial phase and a preset phase interval each time the initial phase is moved.
  • the device further includes:
  • the determining module 8 is configured to perform saturation determination on the gray value of each fiber center position
  • the first processing module 9 is configured to: when there is an optical fiber whose center position gray value exceeds a preset saturation threshold in the sample image, determine that the fiber exceeding the preset saturation threshold is the fiber to be corrected; Correcting the gray value of the center position of the optical fiber to a preset saturation threshold, and performing calculation on the center of each fiber in the fiber bundle in the reconstructed image according to the gray value of each fiber center position determined in the corrected sample image a step of gray value;
  • the second processing module 10 is configured to perform, according to the gray value of each fiber center position determined in the sample image, when there is no optical fiber whose center position gray value exceeds the preset saturation threshold in the sample image A step of calculating the gray value of each fiber center in the bundle in the reconstructed image.
  • the calculating module 1 is specifically configured to compare the gray values of each fiber center position in the plurality of sample images with each other, and obtain a difference between the squared sum and the square root to obtain a reconstructed image in each of the fiber bundles.
  • the gray value of the fiber center is specifically configured to compare the gray values of each fiber center position in the plurality of sample images with each other, and obtain a difference between the squared sum and the square root to obtain a reconstructed image in each of the fiber bundles.
  • the gray value of the fiber center is specifically configured to compare the gray values of each fiber center position in the plurality of sample images with each other, and obtain a difference between the squared sum and the square root to obtain a reconstructed image in each of the fiber bundles. The gray value of the fiber center.
  • This embodiment can be used to implement the embodiment shown in FIG. 3, and its implementation principle is similar, and details are not described herein again.
  • FIG. 7 is a schematic structural view of a microscopic imaging device of the present invention, which is shown in FIG. 7.
  • the present embodiment provides a microscopic imaging device including: a light emitting unit 01, a phase adjusting unit 02, and a steering a unit 03, a fiber bundle 04 comprising a plurality of optical fibers, a detecting unit 05, and a processing unit 06, wherein:
  • the light emitting unit 01 is configured to emit excitation light
  • the phase adjustment unit 02 is disposed at the optical path exit of the excitation light, and is connected to the processing unit 06 for adjusting the phase of the excitation light according to the phase adjustment amount sent by the processing unit 06 to obtain excitation light of different phases;
  • the steering unit 03 is configured to steer excitation light of different phases, so that the steered excitation light is focused along the fiber bundle 04 to the tissue to be detected, and the fluorescence of different phases returned by the tissue to be detected is transmitted;
  • the detecting unit 05 is configured to collect fluorescence of different phases to form a plurality of sample images
  • the processing unit 06 is connected to the detecting unit 05 for receiving a plurality of sample images, and determining the gray value of each fiber center position in the fiber bundle in the plurality of sample images, and calculating each fiber in the fiber bundle in the reconstructed image.
  • the gray value of the center; the spatial interpolation of the gray value of the fiber center is used to obtain the gray value of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the excitation light emitted by the light emitting unit 01 passes through the steering unit 03 (that is, transmits light of a specific frequency to reflect a non-specific frequency), and excites the dyed tissue along the fiber bundle 04 (for example)
  • the cell structure in the human body, the excited fluorescence is image-collected along the fiber bundle, the steering unit 03, and the detecting unit 05.
  • the detecting unit 05 may be a charge-coupled device ("CCD"). Called an image sensor or image controller, it is a semiconductor device that converts optical images into electrical signals.
  • the excitation light emitted by the light emitting unit 01 is focused on a certain focal plane of the tissue, and the phase adjustment unit 02 adjusts the phase of the excitation light according to the phase adjustment amount sent by the processing unit 06 to obtain excitation light of different phases; the processing unit 06 Excitation fluorescence imaging is performed on multiple phases (for example, three phases), and the Neil formula is used to cause the background fluorescence outside the focal plane to be filtered out, thereby performing tomography.
  • Tomography is a kind of geophysical exploration that uses medical CT to invert the calculated information according to the ray scan and reconstruct the elastic wave and electromagnetic wave parameters of the rock mass in the measured range. Inversion interpretation method.
  • the processing unit 06 calculates the gray value of each fiber center in the fiber bundle in the reconstructed image by using the gray value of each fiber center position in the fiber bundle determined in the plurality of sample images of the plurality of phases;
  • the gray value of the fiber center is spatially interpolated to obtain the gray value of other pixels in the fiber bundle in the reconstructed image to form a reconstructed image.
  • the microscopic imaging device of the embodiment includes: a light emitting unit, a phase adjusting unit, a steering unit, a fiber bundle including a plurality of optical fibers, a detecting unit, and a processing unit, wherein: the light emitting unit is configured to emit excitation light; and the phase adjusting unit Provided at an optical path exit of the excitation light, and connected to the processing unit for adjusting the phase of the excitation light according to the phase adjustment amount sent by the processing unit to obtain excitation light of different phases; and the steering unit is configured to perform excitation light of different phases Steering, so that the deflected excitation light is focused along the fiber bundle to the tissue to be detected, and transmits fluorescence of different phases returned by the tissue to be detected; the detecting unit is configured to collect fluorescence of different phases to form a plurality of sample images; The processing unit is connected to the detecting unit, configured to receive the plurality of sample images, and determine the gray value of each fiber center position in the fiber bundle in the plurality of sample images, and calculate
  • the phase adjustment unit adjusts the phase of the excitation light according to the phase adjustment amount sent by the processing unit, and enables the processing unit to acquire a plurality of sample images of the desired phase thereof, thereby improving the imaging quality of the reconstructed image obtained by processing the plurality of sample images.
  • the device can also reduce the calculation amount of the pixel gray value in the reconstructed image and speed up the image reconstruction.
  • FIG. 8 is a schematic structural diagram of a microscopic imaging device of the present invention, which is shown in FIG. 8.
  • the phase adjustment unit 02 includes: a motor 021 and a grating 022;
  • the motor 021 is connected to the processing unit 06 and the grating 022, respectively, for moving the grating 022 according to the phase adjustment amount sent by the processing unit 06, so that the excitation light is transmitted through the grating 022 to obtain excitation light corresponding to the phase adjustment amount.
  • the motor 021 includes: a DC motor; the processing unit 06 determines an equal interval phase adjustment amount according to the preset phase interval; the DC motor receives the equal phase adjustment amount, and the drag grating 022 moves within a grating pitch range.
  • the separation distance is such that the processing unit 06 acquires a plurality of sample images corresponding to the preset phase interval.
  • the processing unit 06 drives the DC motor to move the grating 022 to acquire a plurality of sample images.
  • the sample image contains the pixel information transmitted by each fiber in the fiber bundle 04.
  • a fiber bundle 04 usually consists of nearly 30,000 fibers (the number difference can reach several thousand).
  • the pixel information is transmitted in each of the optical fibers, and therefore, the optical fiber bundle 04 can be referred to as a multi-sensor.
  • a schematic diagram of fiber imaging is shown in Figure 4.
  • the imaging of the fiber presents a hexagonal honeycomb shape in the image, and each fiber diameter is preferably 5 to 6 pixels.
  • the center position of each fiber is determined, and the gray value of each central position pixel is obtained.
  • the method for determining the gray value of the center position can be obtained by using the root mean square formula, that is, the mean value of the gray value of the same center position in the plurality of sample images is obtained, and the calculated gray value is obtained.
  • the mean value is used as the gray value of the center of the fiber in the reconstructed image, and then the gray value of each fiber center in the fiber bundle 04 in the reconstructed image is obtained.
  • the preset phase interval is 120 degrees; the phase adjustment amount is 3.
  • the grating 022 is mounted and moved by the motor 021 by dragging the grating 022 to acquire a sample image of the N fiber bundles.
  • a sample image is taken at the initial position of the motor 021; then the motor 021 is moved to another position, and then a sample image is taken; the motor 021 is moved again, and then photographed, thereby obtaining N samples. image.
  • the motor 021 can be rotated clockwise to obtain the above N sample images, and after waiting for a period of time, the motor 021 is moved counterclockwise in the opposite direction, and then N is obtained.
  • the sample image can be reconstructed to reconstruct the image of the two structured lights, and the accuracy of the reconstructed image is ensured by comparison.
  • the motor 021 drags the grating to move horizontally, and each time the preset phase interval threshold is 1/3 of the grating 022 pitch.
  • the camera is taken at the initial position of the motor 021, the motor 021 moves, the camera is taken, the motor 021 moves again, the camera is taken, and the sample images of the three phases are obtained, and the image is reconstructed; then, for the same period, wait for a while; Shooting, moving in the opposite direction...
  • the three sample images may be a 0 degree phase sample image I 1 (initial phase), a 120 degree phase sample image I 2 (moving a preset phase interval threshold), and a 240 degree phase sample image I 3 (moving two preset phases) Interval threshold), in the three sample images, according to the fiber center position, the gray level of the fiber center of the three phase images is retrieved, that is, the fiber center gray value G 1 , 120 degree phase sample of the 0 degree phase sample image I 1 is obtained. center of the fiber image gray value I G 2, 240 degree phase sample image fiber center gradation value I 2 3 G 3.
  • the light emitting unit 01 includes: a laser 011 for emitting excitation light; and a beam expander focus 012 disposed at an exit of the excitation light of the laser 011 for expanding the excitation light and one-dimensionally Focus is the line beam.
  • the laser 011 is used to emit excitation light. It can be a laser that emits a collimated laser of a specific wavelength. The specific wavelength range may be from 20 nm to 2000 nm. Lasers in this wavelength range can excite a wide range of phosphors.
  • the laser 011 can be a quantum well laser, a solid state laser, a gas laser (such as an argon ion laser), or a laser diode.
  • a beam expander focus 012 is disposed at the exit of the excitation light of the laser 011 for expanding the excitation light and focusing it into a line beam in one dimension. It may include a beam expander lens and a cylindrical lens. The beam expander lens cooperates to expand the collimated beam emitted by the laser 011 to change the diameter of the collimated beam, and the cylindrical lens focuses the expanded beam into a line beam and conducts it to the steering unit 03.
  • the steering unit 03 is a binary mirror, or a dichroic mirror. Its wavelength range can be in the wavelength range of 40nm-2200nm, which can transmit light of a specific frequency and reflect it at a specific frequency.
  • the method further includes: a filter 07; the filter 07 is disposed between the phase adjustment unit 02 and the steering unit 03, and is configured to filter out stray light to improve imaging quality of the sample image, thereby improving imaging of the reconstructed image. quality.
  • the detecting unit 05 comprises: a charge coupled device CCD.
  • the detecting unit 05 can be a line array detecting unit or an area array detecting unit.
  • the imaging speed of the line array detecting unit is in the range of several tens of frames to tens of millions of frames.
  • the objective lens 08 is composed of a plurality of lenses; the objective lens 08 is disposed between the steering unit 03 and the fiber bundle 04 for performing focusing processing on the excitation light after the steering unit 03 is turned.
  • the microscopic imaging device can be used to implement the image reconstruction method in any of the method embodiments of FIG. 1 and FIG. 3, and the implementation principle is similar, and details are not described herein again.
  • the aforementioned program can be stored in a computer readable storage medium.
  • the program when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Optics & Photonics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Microscoopes, Condenser (AREA)
  • Image Processing (AREA)

Abstract

一种图像重建方法、装置及显微成像装置，通过根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束(04)内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束(04)内其他像素点的灰度值，形成重建图像。图像重建方法通过仅计算光纤中心位置的像素点的灰度值，然后基于空间插值得到整幅图像的像素点的灰度值，从而减少了计算每个像素点灰度值的计算量，大大加快了图像重建的速率，且该方法有助于去除重建图像中光栅(022)以及光纤束(04)蜂窝状网格的残留，提高重建图像的成像质量。

Description

图像重建方法、装置及显微成像装置 技术领域
本发明涉及图像处理技术，尤其涉及一种图像重建方法、装置及显微成像装置。
背景技术
基于结构光照明(structured illumination)的显微镜，其具有抑制焦平面外(out of focus)噪声的层析(sectioning)成像功能，且其与共聚焦显微镜相比，有着结构简单，成像速度快的优点。现在技术中，其常常被作为常规内窥镜，通过扫描人体消化道等内部器官，观察细胞形状的变化，以提前预知肿瘤发生与演变，对于癌症的筛检有着重要的指导意义。
该基于结构光照明的显微镜，具体通过激发器发出激发生物体荧光的荧光，该荧光随后通过光栅，形成黑白条纹的正弦光源；再通过每次移动光栅1/3的光栅间距，采集得到扫描人体细胞后返回的多幅图像，例如，图像I 1，图像I 2，图像I 3；再根据均方根公式
Figure PCTCN2018108865-appb-000001
对图像I 1，图像I 2，图像I 3进行重建，得到重建后的图像I。然而，该重建方法需要将图像I 1，图像I 2，图像I 3中全部像素灰度采用该均方根公式计算得到。因此，耗费大量计算时间，且重建后的图像中光栅以及光纤束(fiber bundle)蜂窝状网格的残留明显，成像质量不高。
发明内容
为了解决现有技术中基于结构光照明的显微镜对图像的重建效率不高，图像中残留光栅明显，成像质量不高的技术问题，本发明提供一种图像重建方法、装置及显微成像装置，以加快图像重建的速率，去除重建图像中光栅的残留，提高重建后图像的质量。
本发明提供一种图像重建方法，包括：
根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；
用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
可选的，还包括：
获取均匀荧光的光纤束的原始图像；
在所述原始图像中确认像素值高于周边像素值的目标像素点，将所述目标像素点确定为光纤束中各光纤的中心位置。
可选的，所述获取均匀荧光的光纤束的原始图像包括：
在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；
对所述多个光纤束图像求取其均值图像，形成所述均匀荧光的光纤束的原始图像。
可选的，执行所述空间插值前还包括：
根据每个光纤的中心位置，确定光纤束内各个像素点与每个光纤的所述中心位置之间的插值权值。
可选的，还包括采用如下方法确定所述插值权值：
以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；
根据所述三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
可选的，还包括采用如下方法获取多幅样本图像：
根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离所述初始相位每次移动所述预设相位间隔的N幅样本图像。
可选的，所述预设相位间隔为120度；N＝3。
可选的，所述在所述一幅或者多幅样本图像中确定出每个光纤中心位置的灰度值之后，还包括：
对每个光纤中心位置的灰度值进行饱和度判断；
若在所述样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定所述超出预设饱和度阈值的光纤为待校正光纤；
在重建图像中将所述待校正光纤的中心位置的灰度值校正为所述预设饱和度阈值，根据校正后的所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
若在所述样本图像中不存在中心位置的灰度值超出预设饱和度阈值的光纤，则根据在所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤。
可选的，所述根据在多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值，包括：
将每个光纤中心位置在所述多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。
本发明还提供一种图像重建装置，包括：
计算模块，用于根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；
形成模块，用于用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
可选的，还包括：
第一获取模块，用于获取均匀荧光的光纤束的原始图像；
第一确定模块，用于在所述原始图像中确认像素值高于周边像素值的目标像素点，将所述目标像素点确定为光纤束中各光纤的中心位置。
可选的，所述第一获取模块，包括：
采集子模块，用于在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；
形成子模块，用于对所述多个光纤束图像求取其均值图像，形成所述均匀荧光的光纤束的原始图像。
可选的，所述装置还包括：
第二确定模块，用于根据每个光纤的中心位置，确定光纤束内各个 像素点与每个光纤的所述中心位置之间的插值权值。
可选的，所述装置还包括：
第三确定模块，用于以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；根据所述三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
可选的，所述装置还包括：
第二获取模块，用于根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离所述初始相位每次移动所述预设相位间隔的N幅样本图像。
可选的，所述预设相位间隔为120度；
N＝3。
可选的，所述装置还包括：
判断模块，用于对每个光纤中心位置的灰度值进行饱和度判断；
第一处理模块，用于当在所述样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定所述超出预设饱和度阈值的光纤为待校正光纤；在重建图像中将所述待校正光纤的中心位置的灰度值校正为所述预设饱和度阈值，根据校正后的所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
第二处理模块，用于当在所述样本图像中不存在中心位置的灰度值超出预设饱和度阈值的光纤，则根据在所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤。
可选的，所述计算模块，具体用于将每个光纤中心位置在所述多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。
本发明还提供一种显微成像装置，包括：
光发射单元、相位调节单元、转向单元、包含多个光纤的光纤束、探测单元、处理单元，其中：
所述光发射单元用于发射激发光；
所述相位调节单元设置在所述激发光的光路出口处，且与所述处理单元连接，用于根据所述处理单元发送的相位调节量，调节所述激发光的相位，得到不同相位的激发光；
所述转向单元用于对不同相位的激发光进行转向，以使转向后的激发光沿着所述光纤束聚焦到待检测组织，并透过所述待检测组织返回的不同相位的荧光；
所述探测单元用于对不同相位的荧光进行采集，形成多幅样本图像；
所述处理单元与所述探测单元连接，用于接收所述多幅样本图像，并在多幅样本图像中确定出的所述光纤束中每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
可选的，所述相位调节单元包括：电机、光栅；
所述电机与所述处理单元、所述光栅分别连接，用于根据所述处理单元发送的相位调节量，拖动所述光栅移动，以使所述激发光透射所述光栅后得到与所述相位调节量对应的激发光。
可选的，所述电机包括：直流电机；
相应的，所述处理单元根据预设相位间隔，确定等间隔的相位调节量；所述直流电机接收所述等间隔的相位调节量，拖动所述光栅在一个光栅间距范围内移动等间隔距离，以使所述处理单元获取到与所述预设相位间隔对应的多幅样本图像。
可选的，所述预设相位间隔为120度；所述相位调节量为3个。
可选的，所述光发射单元包括：激光器，用于发射激发光；还包括：扩束线聚焦器，设置在所述激光器的激发光的出口处，用于将所述激发光扩束并一维聚焦为线光束。
可选的，所述转向单元为二分镜。
可选的，还包括：滤光片；所述滤光片设置于所述相位调节单元和所述转向单元之间，用于滤除杂散光。
可选的，所述探测单元包括：电荷耦合元件CCD。
可选的，还包括：由多个透镜组成的物镜；所述物镜设置在所述转向单元和所述光纤束之间，用于对所述转向单元转向后的激发光进行聚焦处理。
本发明的图像重建方法、装置及显微成像装置，通过根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。该图像重建方法通过仅计算光纤中心位置的像素点的灰度值，然后基于空间插值得到整幅图像的像素点的灰度值，从而减少了计算每个像素点灰度值的计算量，大大加快了图像重建的速率，且该方法有助于去除重建图像中光栅以及光纤束蜂窝状网格的残留，提高重建图像的成像质量。
附图说明
图1为一示例性实施例示出的本发明图像重建方法的流程图；
图2为图1所示实施例的结构光显微内窥装置示意图；
图3为另一示例性实施例示出的本发明图像重建方法的流程图；
图4为图3所示实施例的光纤像素三角形结构示意图；
图5为一示例性实施例示出的本发明图像重建装置的结构示意图；
图6为另一示例性实施例示出的本发明图像重建装置的结构示意图；
图7为一示例性实施例示出的本发明显微成像装置的结构示意图；
图8为另一示例性实施例示出的本发明显微成像装置的结构示意图。
附图标记：光发射单元01、激光器011、扩束线聚焦器012、相位调节单元02、电机021、光栅022、转向单元03、光纤束04、探测单元05、处理单元06、滤光片07、物镜08。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造 性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
图1为一示例性实施例示出的本发明图像重建方法的流程图，如图1所示，本发明的图像重建方法适用于对所有光学成像的图像的重建，尤其适用于基于结构光的图像重建，首先本实施例以基于结构光的内窥镜为例，对结构光成像的原理进行简要说明：
如图2所示的基于结构光的显微内窥装置，其中，由激发器发出的光源经光栅调制出正弦光，正弦光经过二分镜(即对特定频率的光进行透射，对非特定频率的进行反射)、物镜、沿着光纤束激发染色后的组织(例如，人体内的细胞组织)，激发后的荧光沿着光纤束、物镜、二分镜到电荷耦合元件(Charge-coupled Device，简称“CCD”)进行图像采集，CCD也叫做图像传感器或图像控制器，是一种半导体器件，能够把光学影像转化为电信号。其中调制后的正弦光源聚焦到组织的某一焦平面上，通过对多个相位(例如，三个相位)激发荧光成像，运用Neil公式，使得焦平面外的背景荧光被滤掉，从而实现层析成像。层析成像技术是借鉴医学CT，根据射线扫描，对所得到的信息进行反演计算，重建被测范围内岩体弹性波和电磁波参数分布规律的图像，从而达到圈定地质异常体的一种物探反演解释方法。
其中，经过光栅调制的结构光光源可以表示为，
Figure PCTCN2018108865-appb-000002
上式中，m为调制对比(modulation contrast)；
Figure PCTCN2018108865-appb-000003
为规范化的空间频率，
Figure PCTCN2018108865-appb-000004
值改变后可以用于实现不同深度(axial depth)图像的层析；β为样本平面(specimen plane)与栅格平面(grid plane)之间的放大倍数，λ为波长，v为实际空间频率，NA为数值孔径(numerical aperture)。
本实施例就是需要对图2中光纤束中各个光纤中所传递出的像素信息进行确定，以准确获得结构光照射染色组织后返回的荧光信息，并对该信息形成清晰准确的图像。本实施例的图像重建方法的具体实现步骤为：
步骤101、根据在一幅或者多幅样本图像中确定出的每个光纤中心位 置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值。
具体的，如图2所示的结构光显微内窥装置，驱动直流电机拖动光栅移动，以获取到一幅或者多幅样本图像。样本图像中包含了光纤束中各个光纤所传递出的像素信息，对于光纤束来说，一根光纤束通常由近三万根光纤(数目差异可达到几千)组成。每根光纤中都传导着像素信息，因此，光纤束又可被称为多传感器。光纤的成像在图像中一般呈现六角蜂窝状，每个光纤直径以5到6个像素为宜。在多幅样本图像中，确定出每个光纤的中心位置，并获取到各个中心位置像素点的灰度值。对中心位置的灰度值的确定方法可以采用前面所述的均方根公式求得，也就是对多幅样本图像中的同一中心位置的灰度值求取其灰度值的均值，以所计算出的灰度值均值作为重建后图像中该光纤中心的灰度值，进而得到重建图像中光纤束内每个光纤中心的灰度值。
步骤102、用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。
具体的，以每个光纤的中心位置作为基准，找到每个光纤中的其他像素点与该中心位置像素点之间的线性关系，从而确定出每个光纤中所有像素点相对于该中心位置像素点的插值权值，也就是各个光纤中其他像素点相对于中心位置像素点的权重值。从而基于各个像素点与光纤中心之间的插值权值，用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。
本实施例的图像重建方法，通过根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。该图像重建方法通过仅计算光纤中心位置的像素点的灰度值，然后基于空间插值得到整幅图像的像素点的灰度值，从而减少了计算每个像素点灰度值的计算量，大大加快了图像重建的速率，且该方法有助于去除重建图像中光栅以及光纤束蜂窝状网格的残留，提高重建图像的成像质量。
图3为另一示例性实施例示出的本发明图像重建方法的流程图，如图3所示，本实施例的图像重建方法，包括：
步骤301、获取均匀荧光的光纤束的原始图像。
步骤302、在原始图像中确认像素值高于周边像素值的目标像素点，将目标像素点确定为光纤束中各光纤的中心位置。
具体的，可以在图像重建前，拍摄一幅均匀荧光的图像，该均匀荧光的图像用于对光纤进行准确定位。对于光纤束来说，一根光纤束通常由近三万根光纤(数目差异可达到几千)组成。每根光纤中都传导着像素信息，因此，光纤束又可被称为多传感器。光纤的成像在图像中呈现六角蜂窝状，每个光纤直径以5到6个像素为宜。为了减少光纤之间的相互干扰，光纤之间为空间不规则排列，而不是呈现出行或列的对其排列。本实施例中的光纤的中心位置指的就是光纤中心最亮点作为光纤中心，所谓最亮点也就是说在原始图像中确认像素值高于周边像素值的目标像素点，将目标像素点确定为光纤束中各光纤的中心位置，以该中心最亮点的坐标作为光纤坐标，以对每个光纤中的其他像素点进行定位。为了去除网格，也就是光纤的六角蜂窝，需要用光纤中心的灰度进行空间插值获得整个光纤束范围内其他像素的灰度。通常安装光栅后，所拍摄的定位图像，也就是原始图像中会存在光栅，因此，可以去除光栅进行拍摄以获取得到均匀荧光的光纤束的原始图像；可选的，还可以在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；对多个光纤束图像求取其均值图像，形成均匀荧光的光纤束的原始图像。也就说让图2中的直流电机在一个光栅间距范围内均匀移动若干个相同的位移，然后取采集到的均值图像。本领域技术人员可以自行确定获取到均匀荧光的光纤束的原始图像的方法，本实施例对此不作具体限定。
步骤303、根据在多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值。
其中，对样本图像的获取可以通过根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离所述初始相位每次移动所述预设相位间隔的N幅样本图像。举例来说，安装光栅，并通过电机拖动光栅移动，以获取到N幅光纤束的样本图像。例如，开始采集样本图像前，在电机的初始位置处拍摄一张样本图像；然后电机移动到另一位置，再拍摄一张样本图像；电机再移动，再拍摄，从而得到N幅 样本图像。为了保证所获取到的样本图像的准确性，可以使电机顺时针旋转多个位置以获取到上述的N幅样本图像，再等待一段时间后，使电机逆时针反方向移动，再获取N幅样本图像，这样可以重建出两幅结构光的图像，通过比对，保证重建图像的准确性。优选的，预设相位间隔为120度，N＝3；则相应的，电机拖动光栅做水平移动，每次移动预设相位间隔阈值为1/3的光栅间距。开始采集前，在电机移动初始位置相机拍摄一张，电机移动，拍摄，电机再移动，拍摄，得到了三个相位的样本图像后重建图像；然后为了周期一致，等待一段时间；再拍摄，反方向移动......这样电机来回往返移动一次，能重建两幅结构光的图像。三幅样本图像可以分别是0度相位样本图像I 1(初始相位)，120度相位样本图像I 2(移动一个预设相位间隔阈值)，240度相位样本图像I 3(移动两个预设相位间隔阈值)，在这三幅样本图像中根据光纤中心位置，检索三个相位图像的光纤中心的灰度，即得到0度相位样本图像I 1的光纤中心灰度值G 1，120度相位样本图像I 2的光纤中心灰度值G 2，240度相位样本图像I 3的光纤中心灰度值G 3。可选的，对于重建图像中光纤束内每个光纤中心的灰度值的计算可以采用，将每个光纤中心位置在多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。举例来说，基于Neil公式，
Figure PCTCN2018108865-appb-000005
对3幅样本图像中的三个中心灰度值两两彼此作差，再对差值平方，将平方后的各个差值相加再开根号，从而计算得到重建图像中光纤中心的灰度值。
但是，对于上述的Neil公式来说，其缺点是当样本图像过饱和时，中心灰度值两两相减，会使得中心点计算出来的灰度值反而是一个灰度很小的黑点，这就会导致重建出来的图像出现黑色区域，无法对细胞进行清晰的成像。为了避免图像饱和给重建图像造成成像不清晰的问题，可以对光纤中心点的灰度采取饱和校正。这样重建的图像才会有良好的层析效果。
可选的，在一幅或者多幅样本图像中确定出每个光纤中心位置的灰度值之后，可以增加对每个光纤中心位置的灰度值的饱和度进行判断的步骤，也就是说，若在样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定超出预设饱和度阈值的光纤为待校正光纤；在重 建图像中将待校正光纤的中心位置的灰度值校正为预设饱和度阈值，根据校正后的样本图像中确定出的每个光纤中心位置的灰度值，再执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
若在样本图像中不存在中心位置的灰度值超出预设饱和度阈值的光纤，则根据在样本图像中确定出的每个光纤中心位置的灰度值，执行计算得到重建图像中光纤束内每个光纤中心的灰度值的步骤。
其中，预设饱和度阈值可以根据CCD的性能进行确定，例如，判断0度相位样本图像I 1的光纤中心灰度值G 1，120度相位样本图像I 2的光纤中心灰度值G 2，240度相位样本图像I 3的光纤中心灰度值G 3，三个灰度值是否有大于4095，(4095对应12位图像最大值，表示CCD饱和)，然后对其不采用上述的Neil公式
Figure PCTCN2018108865-appb-000006
计算重建图像的中心点灰度值，而是直接以4095的预设饱和度阈值作为该中心点灰度值。这种处理避免了样本图像与重建结构光图像视觉上白黑相反的现象。但是这种处理是不得已采取的补救措施，对于本领域技术人员来说，还是应该尽量避免采集样本图像时，出现图像饱和的问题。例如，可以采用避免相机参数的曝光时间过长，增益过大；避免样本荧光染色物质太浓；避免激光器发出的激光的光强过强等措施。
同样的，若在重建图像中存在光纤中心位置的灰度值超出预设饱和度阈值的光纤，则确定超出预设饱和度阈值的光纤为待校正光纤；在重建图像中将待校正光纤的中心位置的灰度值校正为预设饱和度阈值。也就是说若计算得到的
Figure PCTCN2018108865-appb-000007
的值超出预设饱和度阈值，则确定该光纤为待校正光纤，也将预设饱和度阈值作为该光纤的中心位置的灰度值，从而实现对样本图像的饱和校正。
步骤304、根据每个光纤的中心位置，确定光纤束内各个像素点与每个光纤的中心位置之间的插值权值。
具体的，如上所述，无论是样本图像还是原始图像，其都是对同一结构的光纤束进行光学成像，因此，可以根据在原始图像中确定的各个光纤的中心位置，找到样本图像中对应的光纤的中心位置，并读取出该中心点的灰度值。对N幅样本图像中的每个光纤都进行定位并获取其灰度值。因此，对于每个光纤来说，其都对应了N个中心位置的灰度值， 基于预设算法(如前所述的均方根的Neil公式)，对N个中心位置的灰度值求取其灰度值的均值，以所计算出的灰度值均值作为重建后图像中该光纤中心的灰度值。
对于光纤束内各个像素点与每个光纤的中心位置之间的插值权值，可以通过以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；根据三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
具体的，可以根据区域极大值法得到光纤中心坐标，即由图4所示的光纤A的中心位置作为一顶点，光纤A与其相邻的光纤B和光纤C的三个中心位置构成一个三角形，使得整个光纤束范围内被剖分为多个三角形。通过这些三角形建立像素与光纤的插值关系。由于光线束大致为六边形，分布不规则。相邻光纤不具有横向或纵向的坐标对齐关系，所以不可以像常规的双线性插值，由四个规则的顶点插值中间的像素。但是，采用这种三角形的结构，同样可以确定出每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
步骤305、用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。
具体的，获取到均匀荧光的光纤束的原始图像后，在原始图像中确定出光纤束中所包含的所有光纤的中心位置，也就是每个光纤中最亮的那个像素点的位置坐标。以每个光纤的中心位置作为基准，找到每个光纤中的其他像素点与该中心位置像素点之间的线性关系，从而确定出每个光纤中所有像素点相对于该中心位置像素点的插值权值，也就是各个光纤中其他像素点相对于中心位置像素点的权重值。后续对于结构光照射组织后得到的样本图像的重建，可以基于提前计算好的线性权重值，重建时乘以光纤的灰度值，从而得到待插值像素的灰度值，形成重建图像。
本实施例的图像重建方法，通过基于三角形的像素空间利用光纤定位获得对结构光成像的重建，其只对光纤中心点的像素运用诸如Neil公式进行灰度值的计算，然后插值重建整幅结构光图像。大大节省了计算时间，并且能够去掉光纤的蜂窝结构。当N幅样本图像，例如三幅样本 图像相位准确相差120度时，则光栅的痕迹也是没有的。因此，本发明的图像重建方法可以极大地减少计算每个像素点灰度值的计算量，大大加快了图像重建的速率，且该方法还有助于去除重建图像中光栅以及光纤束蜂窝状网格的残留，提高重建图像的成像质量。
图5为一示例性实施例示出的本发明图像重建装置的结构示意图，如图5所示，本实施例的图像重建装置包括：
计算模块1，用于根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；
形成模块2，用于用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。
本实施例可用于实现图1所示实施例，其实现原理相似，在此不再赘述。
本实施例的图像重建装置，通过根据在多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。该图像重建方法通过仅计算光纤中心位置的像素点的灰度值，然后基于空间插值得到整幅图像的像素点的灰度值，从而减少了计算每个像素点灰度值的计算量，大大加快了图像重建的速率，且该方法有助于去除重建图像中光栅以及光纤束蜂窝状网格的残留，提高重建图像的成像质量。
图6为另一示例性实施例示出的本发明图像重建装置的结构示意图，如图6所示，基于上述实施例，本实施例的图像重建装置中，还包括：
第一获取模块3，用于获取均匀荧光的光纤束的原始图像；
第一确定模块4，用于在原始图像中确认像素值高于周边像素值的目标像素点，将目标像素点确定为光纤束中各光纤的中心位置。
可选的，第一获取模块3，包括：
采集子模块31，用于在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；
形成子模块32，用于对多个光纤束图像求取其均值图像，形成均匀荧光的光纤束的原始图像。
可选的，该装置还包括：
第二确定模块5，用于根据每个光纤的中心位置，确定光纤束内各个像素点与每个光纤的中心位置之间的插值权值。
可选的，该装置还包括：
第三确定模块6，用于以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；根据三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
可选的，该装置还包括：
第二获取模块7，用于根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离初始相位每次移动预设相位间隔的N幅样本图像。
可选的，预设相位间隔为120度；N＝3。
可选的，该装置还包括：
判断模块8，用于对每个光纤中心位置的灰度值进行饱和度判断；
第一处理模块9，用于当在样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定超出预设饱和度阈值的光纤为待校正光纤；在重建图像中将待校正光纤的中心位置的灰度值校正为预设饱和度阈值，根据校正后的样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
第二处理模块10，用于当在样本图像中不存在中心位置的灰度值超出预设饱和度阈值的光纤，则根据在样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤。
可选的，计算模块1，具体用于将每个光纤中心位置在多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。
本实施例可用于实现图3所示实施例，其实现原理相似，在此不再赘述。
图7为一示例性实施例示出的本发明显微成像装置的结构示意图，如图7所示，本实施例提供一种显微成像装置，包括：光发射单元01、相 位调节单元02、转向单元03、包含多个光纤的光纤束04、探测单元05、处理单元06，其中：
光发射单元01用于发射激发光；
相位调节单元02设置在激发光的光路出口处，且与处理单元06连接，用于根据处理单元06发送的相位调节量，调节激发光的相位，得到不同相位的激发光；
转向单元03用于对不同相位的激发光进行转向，以使转向后的激发光沿着光纤束04聚焦到待检测组织，并透过待检测组织返回的不同相位的荧光；
探测单元05用于对不同相位的荧光进行采集，形成多幅样本图像；
处理单元06与探测单元05连接，用于接收多幅样本图像，并在多幅样本图像中确定出的光纤束中每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。
具体的，由光发射单元01发出的激发光，激发光经过转向单元03(即对特定频率的光进行透射，对非特定频率的进行反射)、沿着光纤束04激发染色后的组织(例如，人体内的细胞组织)，激发后的荧光沿着光纤束、转向单元03到探测单元05进行图像采集，该探测单元05可以为电荷耦合元件(Charge-coupled Device，简称“CCD”)，也叫做图像传感器或图像控制器，是一种半导体器件，能够把光学影像转化为电信号。其中，光发射单元01发出的激发光聚焦到组织的某一焦平面上，通过相位调节单元02根据处理单元06发送的相位调节量，调节激发光的相位，得到不同相位的激发光；处理单元06对多个相位(例如，三个相位)激发荧光成像，运用Neil公式，使得焦平面外的背景荧光被滤掉，从而实现层析成像。层析成像技术是借鉴医学CT，根据射线扫描，对所得到的信息进行反演计算，重建被测范围内岩体弹性波和电磁波参数分布规律的图像，从而达到圈定地质异常体的一种物探反演解释方法。具体的，处理单元06通过对多个相位的多幅样本图像中确定出的光纤束中每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他 像素点的灰度值，形成重建图像。
本实施例的显微成像装置，包括：光发射单元、相位调节单元、转向单元、包含多个光纤的光纤束、探测单元、处理单元，其中：光发射单元用于发射激发光；相位调节单元设置在激发光的光路出口处，且与处理单元连接，用于根据处理单元发送的相位调节量，调节激发光的相位，得到不同相位的激发光；转向单元用于对不同相位的激发光进行转向，以使转向后的激发光沿着光纤束聚焦到待检测组织，并透过待检测组织返回的不同相位的荧光；探测单元用于对不同相位的荧光进行采集，形成多幅样本图像；处理单元与探测单元连接，用于接收多幅样本图像，并在多幅样本图像中确定出的光纤束中每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成重建图像。实现相位调节单元根据处理单元发送的相位调节量调节激发光的相位，并使得处理单元可以获取到其所需相位的多幅样本图像，从而提高多幅样本图像处理后得到的重建图像的成像质量，采用该装置还可以减少重建图像中像素点灰度值的计算量，加快图像重建的速率。
图8为另一示例性实施例示出的本发明显微成像装置的结构示意图，如图8所示，在上一实施例的基础上，相位调节单元02包括：电机021、光栅022；
电机021与处理单元06、光栅022分别连接，用于根据处理单元06发送的相位调节量，拖动光栅022移动，以使激发光透射光栅022后得到与相位调节量对应的激发光。
可选的，电机021包括：直流电机；处理单元06根据预设相位间隔，确定等间隔的相位调节量；直流电机接收等间隔的相位调节量，拖动光栅022在一个光栅间距范围内移动等间隔距离，以使处理单元06获取到与预设相位间隔对应的多幅样本图像。
具体的，处理单元06驱动直流电机拖动光栅022移动，以获取到多幅样本图像。样本图像中包含了光纤束04中各个光纤所传递出的像素信息，对于光纤束04来说，一根光纤束04通常由近三万根光纤(数目差异可达到几千)组成。每根光纤中都传导着像素信息，因此，光纤束04又 可被称为多传感器。光纤成像的示意图如图4所示，光纤的成像在图像中呈现六角蜂窝状，每个光纤直径以5到6个像素为宜。在多幅样本图像中，确定出每个光纤的中心位置，并获取到各个中心位置像素点的灰度值。对中心位置的灰度值的确定方法可以采用均方根公式求得，也就是对多幅样本图像中的同一中心位置的灰度值求取其灰度值的均值，以所计算出的灰度值均值作为重建后图像中该光纤中心的灰度值，进而得到重建图像中光纤束04内每个光纤中心的灰度值。
可选的，预设相位间隔为120度；相位调节量为3个。
举例来说，安装光栅022，并通过电机021拖动光栅022移动，以获取到N幅光纤束的样本图像。例如，开始采集样本图像前，在电机021的初始位置处拍摄一张样本图像；然后电机021移动到另一位置，再拍摄一张样本图像；电机021再移动，再拍摄，从而得到N幅样本图像。为了保证所获取到的样本图像的准确性，可以使电机021顺时针旋转多个位置以获取到上述的N幅样本图像，再等待一段时间后，使电机021逆时针反方向移动，再获取N幅样本图像，这样可以重建出两幅结构光的图像，通过比对，保证重建图像的准确性。对于预设相位间隔为120度，相位调节量为3个(即N＝3)的情况，电机021拖动光栅做水平移动，每次移动预设相位间隔阈值为1/3的光栅022间距。开始采集前，在电机021移动初始位置相机拍摄一张，电机021移动，拍摄，电机021再移动，拍摄，得到了三个相位的样本图像后重建图像；然后为了周期一致，等待一段时间；再拍摄，反方向移动......这样电机021来回往返移动一次，能重建两幅结构光的图像。三幅样本图像可以分别是0度相位样本图像I 1(初始相位)，120度相位样本图像I 2(移动一个预设相位间隔阈值)，240度相位样本图像I 3(移动两个预设相位间隔阈值)，在这三幅样本图像中根据光纤中心位置，检索三个相位图像的光纤中心的灰度，即得到0度相位样本图像I 1的光纤中心灰度值G 1，120度相位样本图像I 2的光纤中心灰度值G 2，240度相位样本图像I 3的光纤中心灰度值G 3。
可选的，光发射单元01包括：激光器011，用于发射激发光；还包括：扩束线聚焦器012，设置在激光器011的激发光的出口处，用于将激发光扩束并一维聚焦为线光束。
激光器011用于发射激发光。其可以为发射特定波长的准直激光的激光器。所述特定波长范围可以为20nm-2000nm。该波长范围内的激光可以激发大范围的荧光体。激光器011可以为量子阱激光器、固态激光器、气体激光器(例如氩离子激光器)或者激光二极管。扩束线聚焦器012设置在激光器011的激发光的出口处，用于将激发光扩束并一维聚焦为线光束。其可以包括扩束透镜和柱透镜。扩束透镜配合将激光器011发出的准直光束进行扩束，以改变准直光束的直径，柱透镜将扩束后的光束一维聚焦为线光束并传导至转向单元03。
可选的，转向单元03为二分镜，或叫做二向色镜。其波长范围可以在40nm-2200nm波长范围内，可以实现对特定频率的光进行透射，对非特定频率的进行反射的作用。
可选的，还包括：滤光片07；滤光片07设置于相位调节单元02和转向单元03之间，用于滤除杂散光，以提高样本图像的成像质量，进而提高重建图像的成像质量。
可选的，探测单元05包括：电荷耦合元件CCD。该探测单元05可以为线阵探测单元，也可以是面阵探测单元。例如，CCD(电荷耦合元件)线阵相机或CMOS(互补金属氧化物半导体)线阵相机等。线阵探测单元的成像速度在几十帧到几千万帧的范围内。
可选的，还包括：由多个透镜组成的物镜08；物镜08设置在转向单元03和光纤束04之间，用于对转向单元03转向后的激发光进行聚焦处理。
该显微成像装置可用于实现图1、图3任一方法实施例的图像重建方法，实现原理相似，在此不再赘述。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的 普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (27)

  1. 一种图像重建方法，其特征在于，包括：
    根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；
    用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
  2. 如权利要求1所述的方法，其特征在于，还包括：
    获取均匀荧光的光纤束的原始图像；
    在所述原始图像中确认像素值高于周边像素值的目标像素点，将所述目标像素点确定为光纤束中各光纤的中心位置。
  3. 如权利要求2所述的方法，其特征在于，所述获取均匀荧光的光纤束的原始图像包括：
    在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；
    对所述多个光纤束图像求取其均值图像，形成所述均匀荧光的光纤束的原始图像。
  4. 如权利要求1所述的方法，其特征在于，执行所述空间插值前还包括：
    根据每个光纤的中心位置，确定光纤束内各个像素点与每个光纤的所述中心位置之间的插值权值。
  5. 如权利要求4所述的方法，其特征在于，还包括采用如下方法确定所述插值权值：
    以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；
    根据所述三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
  6. 如权利要求1所述的方法，其特征在于，还包括采用如下方法获取多幅样本图像：
    根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离所述初始相位每次移动所述预设相位间隔的N幅样本 图像。
  7. 如权利要求6所述的方法，其特征在于：
    所述预设相位间隔为120度；
    N＝3。
  8. 根据权利要求1所述的方法，其特征在于，所述在所述一幅或者多幅样本图像中确定出每个光纤中心位置的灰度值之后，还包括：
    对每个光纤中心位置的灰度值进行饱和度判断；
    若在所述样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定所述超出预设饱和度阈值的光纤为待校正光纤；
    在重建图像中将所述待校正光纤的中心位置的灰度值校正为所述预设饱和度阈值，根据校正后的所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
    若在所述样本图像中不存在中心位置的灰度值超出预设饱和度阈值的光纤，则根据在所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤。
  9. 根据权利要求1所述的方法，其特征在于，所述根据在多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值，包括：
    将每个光纤中心位置在所述多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。
  10. 一种图像重建装置，其特征在于，包括：
    计算模块，用于根据在一幅或者多幅样本图像中确定出的每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；
    形成模块，用于用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
  11. 如权利要求10所述的装置，其特征在于，还包括：
    第一获取模块，用于获取均匀荧光的光纤束的原始图像；
    第一确定模块，用于在所述原始图像中确认像素值高于周边像素值 的目标像素点，将所述目标像素点确定为光纤束中各光纤的中心位置。
  12. 如权利要求11所述的装置，其特征在于，所述第一获取模块，包括：
    采集子模块，用于在一个光栅间隔距离范围内，采集间隔预设步长的多个光纤束图像；
    形成子模块，用于对所述多个光纤束图像求取其均值图像，形成所述均匀荧光的光纤束的原始图像。
  13. 如权利要求10所述的装置，其特征在于，所述装置还包括：
    第二确定模块，用于根据每个光纤的中心位置，确定光纤束内各个像素点与每个光纤的所述中心位置之间的插值权值。
  14. 如权利要求13所述的装置，其特征在于，所述装置还包括：
    第三确定模块，用于以每个光纤的中心位置、以及相邻光纤的中心位置作为顶点，形成多个三角形结构；根据所述三角形结构，确定每个三角形结构内的像素点与每个光纤的中心位置之间的插值权值。
  15. 如权利要求10所述的装置，其特征在于，所述装置还包括：
    第二获取模块，用于根据预设相位间隔，在一个光栅间隔距离内移动N-1次，获取得到包含初始相位，距离所述初始相位每次移动所述预设相位间隔的N幅样本图像。
  16. 如权利要求15所述的装置，其特征在于，
    所述预设相位间隔为120度；
    N＝3。
  17. 根据权利要求10所述的装置，其特征在于，所述装置还包括：
    判断模块，用于对每个光纤中心位置的灰度值进行饱和度判断；
    第一处理模块，用于当在所述样本图像中存在中心位置的灰度值超出预设饱和度阈值的光纤，则确定所述超出预设饱和度阈值的光纤为待校正光纤；在重建图像中将所述待校正光纤的中心位置的灰度值校正为所述预设饱和度阈值，根据校正后的所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤；
    第二处理模块，用于当在所述样本图像中不存在中心位置的灰度值 超出预设饱和度阈值的光纤，则根据在所述样本图像中确定出的每个光纤中心位置的灰度值，执行计算重建图像中光纤束内每个光纤中心的灰度值的步骤。
  18. 根据权利要求10所述的装置，其特征在于，
    所述计算模块，具体用于将每个光纤中心位置在所述多幅样本图像中的灰度值彼此作差，得到的差值取平方和再开方，得到重建图像中光纤束内每个光纤中心的灰度值。
  19. 一种显微成像装置，其特征在于，包括：光发射单元、相位调节单元、转向单元、包含多个光纤的光纤束、探测单元、处理单元，其中：
    所述光发射单元用于发射激发光；
    所述相位调节单元设置在所述激发光的光路出口处，且与所述处理单元连接，用于根据所述处理单元发送的相位调节量，调节所述激发光的相位，得到不同相位的激发光；
    所述转向单元用于对不同相位的激发光进行转向，以使转向后的激发光沿着所述光纤束聚焦到待检测组织，并透过所述待检测组织返回的不同相位的荧光；
    所述探测单元用于对不同相位的荧光进行采集，形成多幅样本图像；
    所述处理单元与所述探测单元连接，用于接收所述多幅样本图像，并在多幅样本图像中确定出的所述光纤束中每个光纤中心位置的灰度值，计算重建图像中光纤束内每个光纤中心的灰度值；用光纤中心的灰度值进行空间插值，得到重建图像中光纤束内其他像素点的灰度值，形成所述重建图像。
  20. 如权利要求19所述的装置，其特征在于，所述相位调节单元包括：电机、光栅；
    所述电机与所述处理单元、所述光栅分别连接，用于根据所述处理单元发送的相位调节量，拖动所述光栅移动，以使所述激发光透射所述光栅后得到与所述相位调节量对应的激发光。
  21. 如权利要求20所述的装置，其特征在于，所述电机包括：直流 电机；
    相应的，所述处理单元根据预设相位间隔，确定等间隔的相位调节量；所述直流电机接收所述等间隔的相位调节量，拖动所述光栅在一个光栅间距范围内移动等间隔距离，以使所述处理单元获取到与所述预设相位间隔对应的多幅样本图像。
  22. 如权利要求21所述的装置，其特征在于，
    所述预设相位间隔为120度；所述相位调节量为3个。
  23. 如权利要求19所述的装置，其特征在于，所述光发射单元包括：激光器，用于发射激发光；还包括：扩束线聚焦器，设置在所述激光器的激发光的出口处，用于将所述激发光扩束并一维聚焦为线光束。
  24. 如权利要求19所述的装置，其特征在于，所述转向单元为二分镜。
  25. 如权利要求19所述的装置，其特征在于，还包括：滤光片；所述滤光片设置于所述相位调节单元和所述转向单元之间，用于滤除杂散光。
  26. 如权利要求19所述的装置，其特征在于，所述探测单元包括：电荷耦合元件CCD。
  27. 如权利要求19所述的装置，其特征在于，还包括：由多个透镜组成的物镜；所述物镜设置在所述转向单元和所述光纤束之间，用于对所述转向单元转向后的激发光进行聚焦处理。
PCT/CN2018/108865 2017-10-16 2018-09-29 图像重建方法、装置及显微成像装置 WO2019076192A1 (zh)

Priority Applications (9)

Application Number Priority Date Filing Date Title
MX2020003985A MX2020003985A (es) 2017-10-16 2018-09-29 Metodo de reconstruccion de imagenes, dispositivo y dispositivo de imagen microscopica.
RU2020115467A RU2747129C1 (ru) 2017-10-16 2018-09-29 Способ и устройство для реконструкции изображения, а также устройство для формирования изображения в микроскопе
EP18868288.4A EP3699576A4 (en) 2017-10-16 2018-09-29 IMAGE RECONSTRUCTION METHOD AND DEVICE, AND MICROSCOPE IMAGING DEVICE
JP2020541842A JP7064796B2 (ja) 2017-10-16 2018-09-29 画像再構成方法、装置及び顕微結像装置
CA3079243A CA3079243C (en) 2017-10-16 2018-09-29 Image reconstruction method, device and microscopic imaging device
BR112020007609-0A BR112020007609B1 (pt) 2017-10-16 2018-09-29 Método e dispositivo de reconstrução de imagem e dispositivo de imagiologia microscópica
KR1020207013484A KR102358848B1 (ko) 2017-10-16 2018-09-29 이미지 재건 방법, 장치 및 현미 이미징 장치
AU2018352821A AU2018352821B2 (en) 2017-10-16 2018-09-29 Image reconstruction method, device and microscopic imaging device
US16/850,077 US11449964B2 (en) 2017-10-16 2020-04-16 Image reconstruction method, device and microscopic imaging device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710959434.8 2017-10-16
CN201710959434.8A CN107621463B (zh) 2017-10-16 2017-10-16 图像重建方法、装置及显微成像装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/850,077 Continuation US11449964B2 (en) 2017-10-16 2020-04-16 Image reconstruction method, device and microscopic imaging device

Publications (1)

Publication Number Publication Date
WO2019076192A1 true WO2019076192A1 (zh) 2019-04-25

Family

ID=61092506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/108865 WO2019076192A1 (zh) 2017-10-16 2018-09-29 图像重建方法、装置及显微成像装置

Country Status (11)

Country Link
US (1) US11449964B2 (zh)
EP (1) EP3699576A4 (zh)
JP (1) JP7064796B2 (zh)
KR (1) KR102358848B1 (zh)
CN (1) CN107621463B (zh)
AU (1) AU2018352821B2 (zh)
BR (1) BR112020007609B1 (zh)
CA (1) CA3079243C (zh)
MX (1) MX2020003985A (zh)
RU (1) RU2747129C1 (zh)
WO (1) WO2019076192A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11449964B2 (en) 2017-10-16 2022-09-20 Suzhou Microview Medical Technologies Co., Ltd. Image reconstruction method, device and microscopic imaging device

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107622491B (zh) * 2017-10-16 2022-03-11 苏州微景医学科技有限公司 光纤束图像分析方法和装置
CN111024659B (zh) * 2019-11-28 2020-12-11 浙江大学 一种基于并行探测的多图像重建显微成像方法和装置
CN111461975B (zh) * 2020-03-17 2021-04-23 中国科学院苏州生物医学工程技术研究所 光纤束超分辨成像方法、系统、计算机设备和存储介质
CN113962918A (zh) * 2020-06-08 2022-01-21 飞巽传感技术(上海)有限公司 一种光纤纤芯的聚焦方法及系统
CN113570520B (zh) * 2021-07-28 2024-09-20 苏州微景医学科技有限公司 光纤图像校正方法、装置和计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1758754A (zh) * 2005-10-27 2006-04-12 中国科学院上海技术物理研究所 基于光纤耦合的焦平面阵列图像时空变换的方法
US20090092363A1 (en) * 2006-03-14 2009-04-09 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Method and device for generating a structure-free fiberscopic picture
CN104794475A (zh) * 2015-04-03 2015-07-22 燕山大学 一种光子晶体光纤的端面结构特征提取方法
CN106447717A (zh) * 2016-09-30 2017-02-22 中国科学院自动化研究所 一种基于多角度的选择性光片照明显微成像的重建方法
CN106981090A (zh) * 2017-02-16 2017-07-25 南京邮电大学 一种管内步进单向光束扫描断层图像的三维重建方法
CN107621463A (zh) * 2017-10-16 2018-01-23 南京亘瑞医疗科技有限公司 图像重建方法、装置及显微成像装置

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5436980A (en) * 1988-05-10 1995-07-25 E. I. Du Pont De Nemours And Company Method for determining quality of dispersion of glass fibers in a thermoplastic resin preform layer and preform layer characterized thereby
JPH04138127A (ja) * 1990-09-28 1992-05-12 Olympus Optical Co Ltd 内視鏡網目画像軽減装置
AUPQ685900A0 (en) * 2000-04-12 2000-05-11 Goyen Controls Co Pty Limited Method and apparatus for detecting particles in a gas flow
JP4440764B2 (ja) * 2002-12-05 2010-03-24 テレフオンアクチーボラゲット エル エム エリクソン（パブル） 融着温度の校正方法および校正装置
CN100481937C (zh) * 2006-05-12 2009-04-22 北京理工大学 一种高动态、超分辨率图像重建装置
DE102007018048A1 (de) * 2007-04-13 2008-10-16 Michael Schwertner Verfahren und Anordnung zur optischen Abbildung mit Tiefendiskriminierung
DE102007026595A1 (de) * 2007-06-08 2008-12-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur Kompensation von Farbverschiebungen in faseroptischen Abbildungssystemen
EP2171516A1 (en) * 2007-07-06 2010-04-07 National University of Signapore Fluorescence focal modulation microscopy system and method
TWI426296B (zh) * 2009-06-19 2014-02-11 Ind Tech Res Inst 利用光學偏極特性之三維顯微共焦量測系統與方法
CN101692971B (zh) * 2009-10-13 2011-04-06 西安电子科技大学 非接触式光学断层成像方法
JP5343065B2 (ja) * 2010-12-07 2013-11-13 富士フイルム株式会社 放射線撮影システム
KR101260051B1 (ko) * 2012-01-30 2013-05-06 한국화학연구원 살아있는 세포에 대한 명시야 이미징 및 형광 이미징의 동시 수행이 가능한 세포 이미징 장치 및 방법
JP5951293B2 (ja) * 2012-03-14 2016-07-13 日立Ｇｅニュークリア・エナジー株式会社 画像処理方法および装置
US9885859B2 (en) * 2012-07-05 2018-02-06 Martin Russell Harris Structured illumination microscopy apparatus and method
US10096098B2 (en) * 2013-12-30 2018-10-09 Carestream Health, Inc. Phase retrieval from differential phase contrast imaging
CN103592722B (zh) * 2013-08-22 2015-08-05 浙江大学 一种熊猫型保偏光纤侧视对轴装置及方法
CN103438802B (zh) * 2013-09-17 2016-04-20 上海理工大学 光纤涂覆层几何参数测量方法
CN106031143A (zh) * 2014-02-21 2016-10-12 皇家飞利浦有限公司 颜色空间和用于视频的解码器
JP2016109579A (ja) * 2014-12-08 2016-06-20 ソニー株式会社 情報処理装置、画像取得システム、情報処理方法、画像情報取得方法及びプログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1758754A (zh) * 2005-10-27 2006-04-12 中国科学院上海技术物理研究所 基于光纤耦合的焦平面阵列图像时空变换的方法
US20090092363A1 (en) * 2006-03-14 2009-04-09 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Method and device for generating a structure-free fiberscopic picture
CN104794475A (zh) * 2015-04-03 2015-07-22 燕山大学 一种光子晶体光纤的端面结构特征提取方法
CN106447717A (zh) * 2016-09-30 2017-02-22 中国科学院自动化研究所 一种基于多角度的选择性光片照明显微成像的重建方法
CN106981090A (zh) * 2017-02-16 2017-07-25 南京邮电大学 一种管内步进单向光束扫描断层图像的三维重建方法
CN107621463A (zh) * 2017-10-16 2018-01-23 南京亘瑞医疗科技有限公司 图像重建方法、装置及显微成像装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3699576A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11449964B2 (en) 2017-10-16 2022-09-20 Suzhou Microview Medical Technologies Co., Ltd. Image reconstruction method, device and microscopic imaging device

Also Published As

Publication number Publication date
US20200242732A1 (en) 2020-07-30
MX2020003985A (es) 2020-08-13
CN107621463B (zh) 2024-03-22
CA3079243A1 (en) 2019-04-25
US11449964B2 (en) 2022-09-20
KR20200070313A (ko) 2020-06-17
AU2018352821B2 (en) 2021-09-16
KR102358848B1 (ko) 2022-02-08
RU2747129C1 (ru) 2021-04-28
CN107621463A (zh) 2018-01-23
AU2018352821A1 (en) 2020-05-14
EP3699576A1 (en) 2020-08-26
EP3699576A4 (en) 2020-11-25
BR112020007609A2 (pt) 2020-11-10
CA3079243C (en) 2023-05-16
JP2020537162A (ja) 2020-12-17
JP7064796B2 (ja) 2022-05-11
BR112020007609B1 (pt) 2023-11-28

Similar Documents

Publication Publication Date Title
US11449964B2 (en) Image reconstruction method, device and microscopic imaging device
JP5784435B2 (ja) 画像処理装置、蛍光顕微鏡装置および画像処理プログラム
JP7252190B2 (ja) 生物試料の向上された被写界深度の合成２ｄ画像を生成するシステム
US20230085827A1 (en) Single-shot autofocusing of microscopy images using deep learning
JP2015092253A (ja) 適応光学系を有する顕微鏡検査法
JP2011530708A (ja) 多重スポット走査装置におけるレンズ歪みの測定及び補正
US10416427B2 (en) Scan-based imaging with variable scan speed using predictions of region-of-interest positions
JP2006221190A (ja) 共焦点走査型顕微鏡システム
JP2013113696A (ja) 変位測定方法および変位測定装置
JP2017203822A (ja) 照明設定方法、シート照明顕微鏡装置、及びプログラム
US10776955B2 (en) Method for the analysis of spatial and temporal information of samples by means of optical microscopy
JP2003255231A (ja) 光イメージングシステム及び光イメージのデータ処理方法
JP2006317261A (ja) 走査型サイトメータの画像処理方法及び装置
JP5471715B2 (ja) 合焦装置、合焦方法、合焦プログラム及び顕微鏡
EP4332878A1 (en) Optical image processing method, machine learning method, trained model, machine learning preprocessing method, optical image processing module, optical image processing program, and optical image processing system
US20170108532A1 (en) Image processing method of two-photon structured illumination point scanning microscopy
JP2010164635A (ja) 共焦点顕微鏡
US20170248778A1 (en) Method, Device and Laser Scanning Microscope for Generating Rasterized Images
EP4343681A1 (en) Optical image processing method, machine learning method, trained model, machine learning pre-processing method, optical image processing module, optical image processing program, and optical image processing system
CN116539576A (zh) 寻址扫描超分辨率显微成像方法及相关设备
CN117631249A (zh) 线扫共聚焦扫描光场显微成像装置及方法
CN115689959A (zh) 一种正交线扫描成像处理方法及系统
Kharfi et al. Temporal and Spatial Resolution Limit Study of Radiation Imaging Systems: Notions and Elements of Super Resolution

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18868288

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020541842

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 3079243

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20207013484

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2018352821

Country of ref document: AU

Date of ref document: 20180929

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2018868288

Country of ref document: EP

Effective date: 20200518

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112020007609

Country of ref document: BR

REG Reference to national code

Ref country code: BR

Ref legal event code: B01E

Ref document number: 112020007609

Country of ref document: BR

Free format text: APRESENTE DOCUMENTO DE CESSAO DA PRIORIDADE CN 201710959434.8 ASSINADO E DATADO POR PHILIPS NANJING GENRUI MEDICAL TECHNOLOGIES CO., LTD CONTENDO, PELO MENOS, NUMERO E DATA DE DEPOSITO DO DOCUMENTO DE PATENTE QUE ESTA SENDO CEDIDO, UMA VEZ QUE O DOCUMENTO DISPONIBILIZADO NA PETICAO 870200048036 EVIDENCIA QUE ESTE E DEPOSITANTE DA PRIORIDADE E QUE ESTE DEPOSITANTE E DISTINTO DAQUELE QUE DEPOSITOU A PETICAO DE REQUERIMENTO DO PEDIDO NA FASE NACIONAL. A NAO APRESENTACAO DESSE DOCUMENTO ACARRETARA NA PERDA DA PRIORIDADE.

REG Reference to national code

Ref country code: BR

Ref legal event code: B01Y

Ref document number: 112020007609

Country of ref document: BR

Kind code of ref document: A2

Free format text: ANULADA A PUBLICACAO CODIGO 1.5 NA RPI NO 2594 DE 24/09/2020 POR TER SIDO INDEVIDA.

ENP Entry into the national phase

Ref document number: 112020007609

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20200416