WO2020088013A1 - 一种高通量光学层析成像方法及成像系统 - Google Patents

一种高通量光学层析成像方法及成像系统 Download PDF

Info

Publication number
WO2020088013A1
WO2020088013A1 PCT/CN2019/098364 CN2019098364W WO2020088013A1 WO 2020088013 A1 WO2020088013 A1 WO 2020088013A1 CN 2019098364 W CN2019098364 W CN 2019098364W WO 2020088013 A1 WO2020088013 A1 WO 2020088013A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixels
sample
row
stripe
Prior art date
Application number
PCT/CN2019/098364
Other languages
English (en)
French (fr)
Inventor
骆清铭
袁菁
钟秋园
金锐
龚辉
Original Assignee
华中科技大学苏州脑空间信息研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华中科技大学苏州脑空间信息研究院 filed Critical 华中科技大学苏州脑空间信息研究院
Priority to AU2019373533A priority Critical patent/AU2019373533B8/en
Priority to KR1020217016622A priority patent/KR102593252B1/ko
Priority to JP2021523977A priority patent/JP7235861B2/ja
Priority to CA3118389A priority patent/CA3118389C/en
Priority to EP19880631.7A priority patent/EP3875945A4/en
Publication of WO2020088013A1 publication Critical patent/WO2020088013A1/zh
Priority to US17/302,329 priority patent/US11852794B2/en

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/008Details of detection or image processing, including general computer control
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N21/6456Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458Fluorescence microscopy
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01Arrangements or apparatus for facilitating the optical investigation
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/6428Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0032Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0036Scanning details, e.g. scanning stages
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • G02B21/0076Optical details of the image generation arrangements using fluorescence or luminescence
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/361Optical details, e.g. image relay to the camera or image sensor
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/09Beam shaping, e.g. changing the cross-sectional area, not otherwise provided for
    • G02B27/0938Using specific optical elements
    • G02B27/095Refractive optical elements
    • G02B27/0955Lenses
    • G02B27/0966Cylindrical lenses
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/003Reconstruction from projections, e.g. tomography
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01Arrangements or apparatus for facilitating the optical investigation
    • G01N2021/0106General arrangement of respective parts
    • G01N2021/0112Apparatus in one mechanical, optical or electronic block
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N2021/1765Method using an image detector and processing of image signal
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N2021/178Methods for obtaining spatial resolution of the property being measured
    • G01N2021/1785Three dimensional
    • G01N2021/1787Tomographic, i.e. computerised reconstruction from projective measurements

Definitions

  • the present disclosure relates to optical imaging technology, in particular to a high-throughput optical tomography method and imaging system.
  • the out-of-focus background interference of the traditional wide-field microscope makes it impossible to obtain a clear image of the focal plane.
  • the tissue is cut into thin slices to avoid background interference, and optical tomography uses optical imaging methods to achieve similar tissue slices.
  • the imaging effect is also called optical sectioning.
  • the confocal microscopic imaging technology blocks the defocused background interference by placing a pinhole in front of the camera, and only passes the effective signal of the focal plane to achieve the optical tomography effect.
  • Multi-photon excitation microscopic imaging technology utilizes nonlinear effects and only has sufficient energy at the focal point to excite the fluorescence signal of the sample to achieve the ideal optical tomography effect.
  • these two optical tomography techniques use point-by-point scanning imaging, which is obviously insufficient in imaging flux compared with wide-field imaging.
  • Structured light illumination microimaging technology uses a high-frequency periodic pattern modulation to superimpose wide-field illumination to realize the modulation of the focal plane signal, while the defocus signal is suppressed due to the rapid attenuation of this high-frequency modulation, thereby achieving optical Chromatography.
  • a structured light illumination microscopic imaging reconstruction algorithm to obtain an optical tomographic image.
  • This method also needs to scan the sample imaging area back and forth three times to obtain the original data required for the reconstruction of the structured light illumination microscopic imaging algorithm, which sacrifices the imaging speed, and this imaging method needs to be used in the strip imaging system
  • the beam modulation device realizes the modulation of the illumination light field, which increases the complexity of the system.
  • the imaging quality depends greatly on the contrast of the modulation pattern. Therefore, it is necessary to develop a simple and efficient high-throughput optical tomography method and system.
  • the purpose of the present disclosure is to overcome the above technical deficiencies, and propose a high-throughput optical tomography method and imaging system to solve the problem that the structured light illumination micro-imaging technology in the prior art has a slow imaging speed for large-size samples and requires the use of additional modulation devices 1.
  • the technical solution of the present disclosure provides a high-throughput optical tomography method, including the following steps:
  • S1 Modulate the light beam into a modulated light beam that can be focused on the focal plane of the objective lens and can diverge on the defocused surface of the objective lens.
  • the modulated light beam has different modulation intensity on the focal plane of the objective lens;
  • the camera is used to image the same sample illuminated by the modulated beam under different pixels.
  • the calculation formula of the formed sample image is:
  • I (i) is the sample image formed under i pixels
  • f (i) is the modulation intensity corresponding to the sample image I (i)
  • I in is the focal plane image of the sample image
  • I out is the defocus of the sample image Face image
  • the focal plane image is an optical tomographic image; where the demodulation formula of the demodulation algorithm is:
  • I 1 is the cumulative sum of sample images acquired under ⁇ pixels
  • I 2 is the cumulative sum of sample images acquired under ⁇ pixels
  • the accumulated value of the modulation intensity corresponding to the sample image of is different from the accumulated value of the modulation intensity corresponding to the sample image under ⁇ pixels.
  • the present disclosure also provides a high-throughput optical tomography system, including:
  • a light beam modulation module which is used to modulate the light beam into a modulated light beam that can be focused on the focal plane of the objective lens and can diverge on the defocused surface of the objective lens, and the modulated light beam has not exactly the same modulation intensity on the focal plane of the objective lens;
  • a demodulation module which is used to demodulate the sample images under different pixels to obtain a focal plane image of the sample image by a demodulation algorithm, and the focal plane image is an optical tomographic image;
  • the demodulation formula of the demodulation algorithm Is I in c ⁇
  • , ⁇ and ⁇ are positive integers, c is a constant greater than 0,
  • I 1 is the cumulative sum of sample images acquired under ⁇ pixels, and I 2 is under ⁇ pixels The accumulated sum of the acquired sample images; the accumulated value of the modulation intensity corresponding to the sample image under ⁇ pixels and the accumulated value of the modulation intensity corresponding to the sample image under ⁇ pixels are different.
  • the present disclosure illuminates with light beams with not exactly the same modulation intensity, and after imaging on different pixels based on the same sample, a simpler demodulation algorithm is used to obtain the focal plane image; it simplifies The structured light reconstruction algorithm improves the reconstruction efficiency and the imaging speed of large-size samples.
  • FIG. 1 is a flowchart of a high-throughput optical tomography method of the present disclosure
  • FIG. 2 is a sub-flow diagram of the high-throughput optical tomography method of the present disclosure
  • FIG. 4 is a schematic diagram of reconstruction of an optical tomographic image according to Embodiment 1 of the present disclosure
  • FIG. 5 is a schematic diagram of reconstruction of an optical tomographic image according to Embodiment 2 of the present disclosure.
  • FIG. 6 is a schematic diagram of the optical structure of the high-throughput optical tomography system of the present disclosure.
  • FIG. 7 is a connection block diagram of the high-throughput optical tomography system of the present disclosure.
  • the present disclosure provides a high-throughput optical tomography method, including the following steps:
  • S1 Modulate the light beam into a modulated light beam that can be focused on the focal plane of the objective lens and can diverge on the defocused surface of the objective lens.
  • the modulated light beam has different modulation intensity on the focal plane of the objective lens;
  • the beam is first shaped into a linear linear beam, and then the linear beam is modulated into a linear illumination modulated beam.
  • the focus can be focused on the focal plane of the objective lens and can be diverged on the defocused surface of the objective lens.
  • the light beam illuminates the sample linearly, which can facilitate the sample to excite fluorescence, thereby facilitating subsequent imaging.
  • the above-mentioned modulated light beam is specifically modulated by waveforms with not exactly the same modulation intensity at the focal plane of the objective lens, such as Gaussian modulation, sinusoidal modulation, and triangular modulation. Since the illumination beam of this embodiment uses a Gaussian beam, the illumination modulation beam formed in this embodiment is Gaussian modulated. In this embodiment, other waveform modulations with different modulation strengths may also be used as needed.
  • I (i) is the sample image formed under i pixels
  • f (i) is the modulation intensity corresponding to the sample image I (i)
  • I in is the focal plane image of the sample image
  • I out is the defocus of the sample image Face image
  • the specific imaging includes the following steps:
  • the camera sequentially sequentially images the sample along its relative motion direction
  • the modulated beam can be perpendicular to the direction of sample movement, and the direction of continuous imaging of the sample is the same as the direction in which multiple rows of pixels are arranged. That is, when the sample moves relative to the modulated beam, the continuously illuminated portion of the sample is continuously imaged.
  • this embodiment can drive the sample to move continuously and at a constant speed in a direction perpendicular to the line illumination modulation beam, or can drive the modulation beam to move continuously and at a constant speed in a direction parallel to the sample, as long as the modulation beam and the sample can produce a relatively continuous, Just move at a constant speed.
  • the imaging area of this embodiment is N rows of pixels, N ⁇ 2; two directions perpendicular to X and Y are formed on a plane parallel to the imaging plane of the sample, and the modulated beam is at
  • the X and Y directions have the following characteristics: the modulated light beam has different modulation intensity along the X direction on the N rows of pixels, and the modulated light beam along the Y direction on each row of pixels of the N rows of pixels Has the same modulation intensity.
  • the distribution direction and width of the pixels in the N rows are the same as the distribution direction and width of the linear illumination modulated light beam, respectively, and are in a conjugate relationship with each other, thereby facilitating the imaging area to correspond to the linear illumination modulated light beam.
  • the movement direction of the sample relative to the modulated beam is also along the X direction, so as to ensure that the movement direction of the sample relative to the modulated beam is the same as the arrangement direction of the N rows of pixels.
  • Set to static that is, the movement direction of the sample can be set in the same direction as the arrangement of N rows of pixels, and the exposure time of a single frame of imaging is the same as the time that the sample moves by one row of pixels.
  • any row of pixels to correspond to the image in a frame of image If it is a stripe image block, the corresponding multiple stripe image blocks of any row of pixels in the multi-frame image are successively imaged for each part of the sample, the continuous imaging can be stitched into a stripe image, and the N-line pixels are N strip images can be formed.
  • imaging can be judged. After continuous imaging is completed, subsequent steps can be performed. If continuous imaging is not completed, the sample is continuously driven to move. Since this embodiment realizes continuous imaging of the sample through continuous and uniform movement of the sample, it is equivalent to continuous scanning imaging of the sample. After imaging, it needs to judge whether the entire sample has completed continuous scanning imaging, which is conducive to ensuring the integrity and continuity of imaging .
  • I t (i) is the strip image block corresponding to the pixel of the i-th row in the t-th image
  • ie Is the focal plane image of the m-th strip image block in the complete strip image
  • f (i) is the modulation intensity corresponding to the pixel in the i-th row
  • the sample moves along the imaging pixel arrangement direction. Since the exposure time of a single frame of imaging is the same as the time that the sample moves by one row of pixels, each row of pixels forms multiple The strip image block, the multiple strip image blocks are continuous imaging of the sample.
  • stripe image blocks of the i-th row of pixels in each frame image are sequentially stitched to obtain a stripe image of the i-th row of pixels.
  • the calculation formula of the stripe image is:
  • the stripe image is formed by stitching M stripe image blocks, where, Is the focal plane image corresponding to the m-th strip image block in the strip image, m ⁇ M.
  • the above-mentioned stripe image is formed by shift stitching of multiple stripe image blocks corresponding to a row of pixels, that is, N rows of pixels can be stitched to form N sample images, respectively.
  • ⁇ and ⁇ are positive integers, c is a constant greater than 0, I 1 is the cumulative sum of the stripe images acquired under ⁇ pixels, and I 2 is the cumulative sum of the sample images acquired under ⁇ pixels; ⁇ pixels The accumulated value of the modulation intensity corresponding to the sample image below is not the same as the accumulated value of the modulation intensity corresponding to the sample image under ⁇ pixels.
  • the optical tomographic image obtained by the above demodulation algorithm is zero, so in this embodiment, the cumulative value of the modulation intensity corresponding to the stripe image under ⁇ pixels and the modulation intensity corresponding to the stripe image under ⁇ pixels can be set The accumulated value of is different.
  • Embodiment 1 As shown in (a) of FIG. 4, when the sample moves along the pixel arrangement direction of N rows, it can obtain an N + M-1 frame image (M in the time t 1 to t N + M-1 Is the number of stripe image blocks corresponding to the complete stripe image.
  • N is 8 and M is 9
  • each row of pixels in the N + M-1 frame image corresponds to a stripe image block
  • a first frame bar may acquire a first row of pixels of the image with the image block I 1 (1), the pixels of row 1 of the second frame image block with the image I 2 (1), the first and the frame image N
  • Strip image block I 2 (1) to strip image block I N + M-1 (1) can be spliced in sequence to form a strip image, and the corresponding pixels from the second row to the Nth row can be spliced to form the corresponding Stripe image.
  • Embodiment 2 As shown in FIG. 5, the stripe image formed by stitching under the pixels in the fourth row among them Strip image formed by stitching under the first row of pixels among them Strip image formed by stitching under the second row of pixels among them Stripe image formed by stitching under the third row of pixels among them
  • I 1 is the cumulative sum of the sample images acquired under the pixels in rows 1, 2, and 3, that is I 2 is the cumulative sum of the sample images obtained under the pixels in the fourth row, ie Then it is equivalent to selecting the value of ⁇ to be 3 and the value of ⁇ to be 1, so it can be known from the demodulation formula: therefore,
  • this embodiment also provides a high-throughput optical tomography system 10, which includes a beam modulation module 11, an imaging module 12, and a demodulation module 13.
  • the light beam modulation module 11 is used to modulate the light beam into a modulated light beam that can be focused on the focal plane of the objective lens and can diverge on the defocused surface of the objective lens, and the modulated light beam has different modulation intensity on the focal plane of the objective lens.
  • the beam modulation module 11 described in this embodiment includes a shaping optical path for shaping the illumination light into a linear beam and a modulation optical path for modulating the linear beam into a linear illumination modulation beam.
  • the shaping optical path includes a laser light source 111, a first lens 112, a second lens 113, and a cylindrical lens 114 arranged in sequence along the transmission direction of the illumination light;
  • the modulation optical path includes a method for modulating the divergent light of the line beam into parallel
  • the laser light source 111 emits illumination light, which is processed by the first lens 112 and the second lens 113 in order to expand the beam.
  • the expanded beam is shaped by the cylindrical lens 114 to form a linear beam 11a.
  • the linear beam 11a is divergent light, so it passes through the third lens 115 to form a linear beam 11a of parallel rays, and after passing through the dichroic mirror 116 to change its incident direction, it enters the objective lens 117 to form a focus on the focal plane of the objective lens 117
  • the linearly modulated light beam 11b can be diverged on the defocused surface of the objective lens 117.
  • the optical axis of the linear modulated beam 11b is perpendicular to the optical axis of the illumination light and the linear beam 11a that has not undergone reflection, that is, the first lens 112, the second lens 113, the cylindrical lens 114, and the third lens 115 is coaxially arranged, and the central axes of the first lens 112, the second lens 113, the cylindrical lens 114, and the third lens 115 are perpendicular to the central axis of the objective lens 117.
  • the angle between the dichroic mirror 116 and the optical axis of the linear illumination modulation beam 11b is 45 °, which can ensure that the width of the linear beam 11a reflected by the dichroic mirror 116 does not change.
  • the imaging module 12 is used to image the same sample illuminated by the modulated beam under different pixels using a camera, and includes a driving unit 121, an imaging unit 122, an image block acquiring unit 123, and a stitching unit 124.
  • the modulation intensity corresponding to the image I (i) I in is the focal plane image of the sample image, and I out is the defocus plane image of the sample image.
  • the driving unit 121 is used to drive the modulated light beam 11b and the sample 20 to move relatively continuously and at a constant speed in the X direction.
  • the exposure time of a single frame of the camera is the same as the time of moving a row of pixels relatively; to facilitate driving, the driving unit of this embodiment 121 can use a translation stage, which can drive the sample 20 to move continuously and at a constant speed in the direction perpendicular to the modulated beam 11b; the translation stage 12 can use an electric translation stage 12, which is located directly under the objective lens 117, and the sample 20 is placed on the translation stage 12 and can move with the translation stage 12.
  • the upper surface of the translation stage 12 is perpendicular to the optical axis of the linear modulated beam 11b.
  • the sample 20 is placed on the translation stage 12 and passes through the modulation area of the linear modulated beam 11b during the movement, and the linear modulated beam 11b Under the action, the sample 20 is excited to emit fluorescence.
  • the translation stage 12 of this embodiment is in a horizontal state, and the linear modulated beam 11b is parallel to the translation stage and perpendicular to the direction in which the sample 20 moves.
  • the imaging unit 122 is used to sequentially image along the relative movement direction of the sample 20; it specifically performs sequential and continuous imaging along with the continuous movement of the sample 20, which can be realized by an imaging optical path, which is routed by the emission filter located directly above the objective lens 117 122a, a barrel lens 122b and a camera 122c, the fluorescence emitted by the sample 20 after being excited passes through the objective lens 117, the dichroic mirror 116, the emission filter 122a and the barrel lens 122b in sequence, and is detected and imaged by the camera 122c.
  • the camera 122c in this embodiment is an area array CCD (Charge-coupled device, charge coupled element) or area array CMOS (Complementary Metal) with Sub-array or ROI (Region of Interest)
  • Oxide Semiconductor (complementary metal oxide semiconductor) cameras can also use linear CCD or linear CMOS cameras with surface mode function.
  • the imaging area of the camera 122c in this embodiment is N rows of pixels, N ⁇ 2, and the imaging direction and the width of the imaging area of the camera 122c and the direction of the linear illumination modulation beam 11b are respectively With the same width, the single-frame exposure time of the camera 122c is the same as the time for the translation stage to drive the sample 20 to move a row of pixels, which has been described above, so it will not be repeated here.
  • the image block obtaining unit 123 is used to obtain a stripe image block of the i-th row of pixels in each frame image obtained in time sequence, and the calculation formula of the stripe image block is I t (i) is the strip image block corresponding to the pixel of the i-th row in the t-th frame image, Is the focal plane image of the strip image block opposite to I t (i), ie Is the focal plane image of the m-th strip image block in the complete strip image, Is the defocused image of the strip image block opposite to I t (i), and f (i) is the modulation intensity corresponding to the pixel in the i-th row.
  • the stitching unit 124 is used to stitch the stripe image blocks of the i-th row of pixels in each frame image in sequence to obtain the stripe image of the i-th row of pixels.
  • M is the number of stripe image blocks corresponding to the complete stripe image, specifically: the stripe image is formed by stitching M stripe image blocks, where, Is the focal plane image corresponding to the m-th strip image block in the strip image, m ⁇ M.
  • the demodulation module 13 is used to demodulate a plurality of sample images through a demodulation algorithm to obtain focal plane images of the plurality of sample images, which includes an image accumulation unit 131 and a demodulation unit 132.
  • the sample images in this embodiment are strip images.
  • the image accumulation unit 131 is used to accumulate the stripe images of at least one row of pixels to form a first stripe image, and accumulate the stripe images of at least one row of pixels to form a second stripe image; and the demodulation unit 132 is used to integrate the first The strip image and the second strip image are demodulated into the optical tomographic image of the strip image according to the demodulation formula.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Health & Medical Sciences (AREA)
  • Optics & Photonics (AREA)
  • Immunology (AREA)
  • Engineering & Computer Science (AREA)
  • Pathology (AREA)
  • Biochemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Chemical Kinetics & Catalysis (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • Microscoopes, Condenser (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Studio Devices (AREA)

Abstract

本公开公开了一种高通量光学层析成像方法及成像系统,方法包括将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,该调制光束在物镜的焦平面具有不完全相同的调制强度;对调制光束照明下的样本在不同像素下进行成像,获取不同像素下的样本图像;将不同像素下的样本图像通过解调算法解调获取样本图像的焦平面图像;系统包括光束调制模块、成像模块及解调模块。本公开通过具有不完全相同的调制强度的光束进行照明,并基于同一样本在不同的像素下进行成像后,采用较简便的解调算法得到焦面图像;其简化了结构光重建算法、提高了重建效率,并提高了大尺寸样本的成像速度。

Description

一种高通量光学层析成像方法及成像系统 技术领域
本公开涉及光学成像技术,尤其是涉及一种高通量光学层析成像方法及成像系统。
背景技术
在光学成像技术领域,传统宽场显微镜的焦外背景干扰使得无法获得焦面清晰的图像,一般通过将组织切成薄片来避免背景干扰,而光学层析通过光学的成像方法达到类似组织切片的成像效果,也称为光学切片。共聚焦显微成像技术通过在相机前放置小孔(pinhole)阻挡离焦的背景干扰,而只让焦面的有效信号通过,达到光学层析效果。多光子激发显微成像技术利用非线性效应,只在焦点处具有足够的能量以激发样本的荧光信号,实现理想的光学层析效果。然而,这两种光学层析技术都采用逐点扫描的成像方式,在成像通量上较宽场成像方式有明显不足。
结构光照明显微成像技术利用对宽场照明叠加一种高频的周期性图案调制实现对焦面信号的调制,而离焦信号则因这种高频调制的快速衰减而被抑制,从而实现光学层析。这一过程的实现,需要至少三张不同调制相位的原始图像,通过结构光照明显微成像重建算法解调出焦面信号,得到光学层析图像。相比于同样具有光学层析功能的共聚焦和多光子激发显微成像技术,结构光照明显微成像因采用宽场成像方式而有着成像通量高、速度快的优势。当需要对大尺寸样本进行成像时,结构光照明显微成像技术通常需采用马赛克拼接方式来扩大成像视场。这使得大尺寸样本成像时所耗时间大部分都用于了马赛克与马赛克间的样本移动,限制了整体成像速度。为避免过多的马赛克拼接,申请号为201310131718.X的中国专利申请公开了一种结构光快速扫描的成像方法,采用线扫描条带式成像提高成像速度,采用结构光照明抑制背景干扰,实现快速获取大尺寸样本的光学层析图像。但这一方法同样需要对样本成像区域来回扫描三次,以获得结构光照明显微成像算法重建所需的原始数据,其牺牲了成像速度,而且该种成像方法需要在条带式成像系统中使用光束调制器件以实现对照明光场的调制,增加了系统的复杂性;同时,其因采用传统结构光照明显微成像方法,成像质量对调制图案对比度依赖较大。因此,发展一种简单、高效的高通量光学层析成像方法和系统是很有必要的。
发明内容
本公开的目的在于克服上述技术不足,提出一种高通量光学层析成像方法及成像系统,解决现有技术中结构光照明显微成像技术对大尺寸样本成像速度慢、需使用额外调制器件、对调制图案对比度依赖大及重建光学层析图像时解调算法复杂的技术问题。
为达到上述技术目的,本公开的技术方案提供一种高通量光学层析成像方法,包括如下步骤:
S1、将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,该调制光束在物镜的焦平面具有不完全相同的调制强度;
S2、采用相机对调制光束照明下的同一样本在不同像素下进行成像,形成的样本图像的计算公式为:
I(i)=I inf(i)+I out
其中,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像;
S3、将不同像素下的样本图像通过解调算法解调获取样本图像的焦平面图像,该焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为:
I in=c×|βI 1–αI 2|
其中,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的样本图像的累加和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
同时,本公开还提供一种高通量光学层析成像系统,包括:
光束调制模块,其用于将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,且该调制光束在物镜的焦平面具有不完全相同的调制强度;
成像模块,其用于采用相机对调制光束照明下的同一样本在不同像素下进行成像,其形成的样本图像的计算公式为I(i)=I inf(i)+I out,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像;
解调模块,其用于将不同像素下的样本图像通过解调算法解调获取样本图像的焦平面图像,该焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为I in=c×|βI 1–αI 2|,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的样本图像的累加 和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
与现有技术相比,本公开通过具有不完全相同的调制强度的光束进行照明,并基于同一样本在不同的像素下进行成像后,采用较简便的解调算法得到焦面图像;其简化了结构光重建算法、提高了重建效率,并提高了大尺寸样本的成像速度。
附图说明
图1是本公开的高通量光学层析成像方法的流程图;
图2是本公开的高通量光学层析成像方法的其中一个子流程图;
图3是本公开的高通量光学层析成像方法的另一个子流程图;
图4是本公开的实施例1的光学层析图像的重建原理图;
图5是本公开的实施例2的光学层析图像的重建原理图;
图6是本公开的高通量光学层析成像系统的光学结构示意图;
图7是本公开的高通量光学层析成像系统的连接框图。
具体实施方式
为了使本公开的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本公开进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本公开,并不用于限定本公开。
如图1~3所示,本公开提供了一种高通量光学层析成像方法,包括如下步骤:
S1、将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,该调制光束在物镜的焦平面具有不完全相同的调制强度;
具体调制时,首先将光束整形成呈线状的线光束,然后再将线光束调制成线照明调制光束,本实施例通过能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束对样本进行线照明,其可利于样本激发出荧光,从而利于后续成像。
其中,上述调制光束在物镜焦平面具体由不完全相同的调制强度,例如高斯调制、正弦调制、三角调制等具有不完全相同调制强度的波形调制。由于本实施例的照明光束采用高斯光束,故本实施例形成的照明调制光束为高斯调制。其中,本实施例也可根据需要采用其他具有不完全相同的调制强度的波形调制。
S2、对调制光束照明下的样本在不同行像素下进行成像,形成的样本图像的计算公 式为:
I(i)=I inf(i)+I out
其中,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像;
具体成像时,其包括如下步骤:
S21、驱动所述调制光束与样本在所述X方向上相对作连续、匀速移动;
S22、所述相机对所述样本沿其相对运动方向依次连续成像;
本实施例的调制光束可垂直于样本移动方向,且对样本成像连续成像的方向与多行像素排列的方向相同,即当样本相对调制光束移动过程中,样本被连续照明的部分被连续成像。其中,本实施例可驱动样本沿垂直于线照明调制光束的方向作连续、匀速移动,也可驱动调制光束沿平行于样本的方向作连续、匀速移动,只要调制光束与样本能够产生相对连续、匀速运动即可。
如图4中a所示,本实施例的成像区域为N行像素,N≥2;在与所述样本成像平面相平行的平面上形成相垂直X与Y两个方向,所述调制光束在X与Y方向分别具有如下特性:所述调制光束在所述N行像素上沿X方向上具有不完全相同的调制强度,所述调制光束在所述N行像素的每一行像素上沿Y方向上具有相同的调制强度。而且,N行像素的分布方向和宽度分别与线照明调制光束的分布方向和宽度相同,并互为物像共轭关系,从而便于成像区域与线照明调制光束相对应。
对应的,样本相对调制光束移动的方向也沿X方向,从而保证样本相对调制光束移动方向与N行像素的排列方向相同,为了便于操作,本实施例优选采用驱动样本运动,而调制光束则可设置为静态,即样本的运动方向可设置与N行像素的排列方向相同,且成像的单帧曝光时间与样本移动一行像素的时间相同,若设定任意一行像素在一帧图像中对应的图像为一个条带图像块,则任意一行像素在多帧图像中对应的多个条带图像块为该样本的各个部分的依次连续成像,该连续成像可拼接成一条带图像,而N行像素则可形成N个条带图像。
其中,本实施例可对成像进行判断,当连续成像完成后,则可执行后续的步骤,若未完成连续成像,则持续驱动样本移动。由于本实施例通过样本的连续、匀速移动实现样本的连续成像,其相当于对样本的连续扫描成像,成像后其需要判断整个样本是否完成连续扫描成像,其利于保证成像的完整性和连续性。
S23、获取按时间顺序所得每帧图像中的第i行像素的条带图像块I t(i),所述条带 图像块的计算公式为:
Figure PCTCN2019098364-appb-000001
其中,I t(i)为第t帧图像中第i行像素对应的条带图像块,
Figure PCTCN2019098364-appb-000002
为I t(i)相对的条带图像块的焦平面图像,即
Figure PCTCN2019098364-appb-000003
为完整的条带图像中的第m个条带图像块的焦平面图像,
Figure PCTCN2019098364-appb-000004
为I t(i)相对的条带图像块的离焦面图像,f(i)为第i行像素对应的调制强度;
如图4中(a)所示,在成像时,样本沿成像像素排列方向运动,由于成像的单帧曝光时间与样本移动一行像素的时间相同,故每行像素沿样本长度方向依次形成多个条带图像块,该多个条带图像块则是对样本的连续成像。
S24、将每帧图像中的第i行像素的条带图像块依次拼接,即得第i行像素的条带图像,所述条带图像的计算公式为:
Figure PCTCN2019098364-appb-000005
其中,M为完整的条带图像所对应的条带图像块数量,具体为:所述条带图像由M个条带图像块拼接形成,其中,
Figure PCTCN2019098364-appb-000006
为所述条带图像中第m个条带图像块所对应的焦平面图像,m≤M。
需要说明的是,上述条带图像为一行像素相对应的多个条带图像块移位拼接而成,即N行像素可分别拼接形成N个样本图像。
S3、将不同像素下的多个条带图像通过解调算法解调获取所述条带图像的焦平面图像,该焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为:
I in=c×|βI 1–αI 2|
其中,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的条带图像的累加和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
其具体步骤如下:
S31、将至少一行像素的条带图像累加形成第一条带图像,并将至少一行像素的条带图像累加形成第二条带图像;
当获取上述N个条带图像时,可任意选择其中的一个、两个或多个条带图像进行累加并形成第一条带图像,然后按同样的方式累加获得第二条带图像,为了避免上述解调算法获取的光学层析图像为零,故本实施例可设定α个像素下的条带图像相对应的调制强度的累加值和β个像素下的条带图像相对应的调制强度的累加值不相同。
S32、将第一条带图像和第二条带图像按解调公式解调成所述条带图像的光学层析 图像,则
Figure PCTCN2019098364-appb-000007
为了便于说明本实施例的条带图像的获取流程,按以下实施例进行说明。
实施例1:如图4中(a)所示,当样本沿N行像素排列方向运动时,其可在时间t 1至t N+M-1时间内获得N+M-1帧图像(M为完整的条带图像所对应的条带图像块数量,该实施例中N为8,M为9),而其N+M-1帧图像中的每一行像素均对应一条带图像块,例如:可获取第一帧图像的第一行像素的条带图像块I 1(1)、第2帧图像的第1行像素的条带图像块I 2(1),第N帧图像的第1行像素的条带图像块I N(1)以及第N+M-1帧图像的第1行像素的条带图像块I N+M-1(1),而上述条带图像块I 1(1)、条带图像块I 2(1)至条带图像块I N+M-1(1)可依次拼接形成一条带图像,对应的第二行像素至第N行像素均可拼接形成对应的条带图像。
如图4中(b)和(c)所示,为了便于说明如何获取更清晰的条带图像块和条带图像,先以第2行像素和第4行像素为例进行说明,由条带图像块和样本图像的计算公式分别可知:
Figure PCTCN2019098364-appb-000008
Figure PCTCN2019098364-appb-000009
则第4帧图像在第4行像素下的条带图像块
Figure PCTCN2019098364-appb-000010
(其中m=1,因为条带图像由9个条带图像块拼接形成,而第4帧图像在第4行的像素下的条带图像块为所述条带图像的第一个条带图像块,即
Figure PCTCN2019098364-appb-000011
为所述条带图像中第1个条带图像块所对应的焦平面图像);对应的,
Figure PCTCN2019098364-appb-000012
其中
Figure PCTCN2019098364-appb-000013
第2帧图像在第2行像素下的条带图像块
Figure PCTCN2019098364-appb-000014
I 1为第4行像素下获取的样本图像的累加和,即
Figure PCTCN2019098364-appb-000015
I 2为第2行像素下获取的样本图像的累加和,即
Figure PCTCN2019098364-appb-000016
选取α和β的值均为1,
Figure PCTCN2019098364-appb-000017
Figure PCTCN2019098364-appb-000018
因此,
Figure PCTCN2019098364-appb-000019
实施例2:如图5所示,第4行像素下拼接形成的条带图像
Figure PCTCN2019098364-appb-000020
其中
Figure PCTCN2019098364-appb-000021
第1行像素下拼接形成的条带图像
Figure PCTCN2019098364-appb-000022
其中
Figure PCTCN2019098364-appb-000023
第2行像素下拼接形成的条带图像
Figure PCTCN2019098364-appb-000024
其中
Figure PCTCN2019098364-appb-000025
第3行像素下拼接形成的条带图像
Figure PCTCN2019098364-appb-000026
其中
Figure PCTCN2019098364-appb-000027
若I 1为第1、2、3行像素下获取的样本图像的累加和,即
Figure PCTCN2019098364-appb-000028
Figure PCTCN2019098364-appb-000029
I 2为第4行像素下获取的样本图像的累加和,即
Figure PCTCN2019098364-appb-000030
Figure PCTCN2019098364-appb-000031
则相当于对应选取α的值为3、β的值为1,故由解调公式可知:
Figure PCTCN2019098364-appb-000032
Figure PCTCN2019098364-appb-000033
因此,
Figure PCTCN2019098364-appb-000034
为了便于说明本实施例,如图6、图7所示,本实施例还提供一种高通量光学层析成像系统10,其包括光束调制模块11、成像模块12和解调模块13。
光束调制模块11用于将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,且该调制光束在物镜的焦平面具有不完全相同的调制强度。
本实施例所述光束调制模块11包括一用于将照明光线整形成呈线状的线光束的整形光路及一用于将线光束调制成线照明调制光束的调制光路。其中,所述整形光路包括沿照明光线的传递方向依次设置的激光光源111、第一透镜112、第二透镜113、柱透镜114;所述调制光路包括用于将线光束的发散光线调制成平行光线的第三透镜115、用于调制线光束的入射方向的二向色镜116及一与调制入射方向后的线光束同轴设置的物镜117。
光线调制时,激光光源111出射形成照明光线,其由第一透镜112和第二透镜113依次处理后进行扩束,扩束后的光束由柱透镜114整形形成呈线状的线光束11a,该线状光束11a为发散的光线,故其经过第三透镜115形成平行光线的线状光束11a,并经过二向色镜116改变其入射方向后进入物镜117以形成能够在物镜117的焦平面聚焦并能够在物镜117的离焦面发散的线状调制光束11b。为了便于后续的成像,线状调制光束11b的光轴与照明光线和未经过反射的线状光束11a的光轴垂直设置,即第一透镜112、第二透镜113、柱透镜114、第三透镜115同轴设置,且第一透镜112、第二透镜113、柱透镜114、第三透镜115的中轴线与物镜117的中轴线垂直设置。而且,二向色镜116与线照明调制光束11b的光轴的夹角为45°,其可保证经过二向色镜116反射后的线状光束11a的宽度不会发生改变。
成像模块12用于采用相机对调制光束照明下的同一样本在不同像素下进行成像,其包括驱动单元121、成像单元122、图像块获取单元123及拼接单元124。其中,成像模块12成像形成的样本图像的计算公式为I(i)=I inf(i)+I out,其中,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像。
驱动单元121用于驱动所述调制光束11b与样本20在X方向上相对作连续、匀速移动,相机的单帧曝光时间与相对移动一行像素的时间相同;为了便于驱动,本实施例的驱动单元121可采用一平移台,其可驱动样本20在垂直于调制光束11b的方向作连续、匀速移动;平移台12可采用电动平移台12,其位于物镜117的正下方,样本20放置于平移台12上并能够随平移台12移动。为了控制成像的精度,平移台12上表面垂直于线状调制光束11b的光轴,样本20放置于平移台12并在移动过程中经过线状调制光束11b的调制区域内,在线状调制光束11b作用下,样本20被激发发出荧光。本实施例的平移台12处于水平状态,线状调制光束11b处于与所述平移台平行、且与样本20移动的方向相垂直。
成像单元122用于沿样本20相对运动方向依次成像;其具体随着样本20的连续移动进行依次连续成像,其可通过一成像光路实现,该成像光路由位于物镜117正上方的发射滤光片122a、筒镜122b和相机122c组成,样本20被激发发出的荧光依次经过物镜117、二向色镜116、发射滤光片122a和筒镜122b,并由相机122c探测成像。其中,本实施例的相机122c为具有Sub-array(子阵列)或ROI(Region of interest,感兴趣区)功能的面阵CCD(Charge-coupled device,电荷耦合元件)或面阵CMOS(Complementary Metal Oxide Semiconductor,互补金属氧化物半导体)相机,也可采用具有面模式功能的线阵CCD或线阵CMOS相机。为了便于后续的光学层析图像的重建,本实施例所述相机122c的成像区域为N行像素,N≥2,且相机122c的成像方向和成像区域的宽度分别与线照明调制光束11b的方向和宽度相同,所述相机122c的单帧曝光时间与所述平移台驱动样本20移动一行像素的时间相同,前文已对其进行说明,故不作赘述。
图像块获取单元123用于获取按时间顺序所得每帧图像中的第i行像素的条带图像块,条带图像块的计算公式为
Figure PCTCN2019098364-appb-000035
I t(i)为第t帧图像中第i行像素对应的条带图像块,
Figure PCTCN2019098364-appb-000036
为I t(i)相对的条带图像块的焦平面图像,即
Figure PCTCN2019098364-appb-000037
为完整的条带图像中的第m个条带图像块的焦平面图像,
Figure PCTCN2019098364-appb-000038
为I t(i)相对的条带图像块的离焦面图像,f(i)为第i行像素对应的调制强度。
拼接单元124用于将每帧图像中的第i行像素的条带图像块依次拼接获得第i行像素的条带图像,其计算公式为
Figure PCTCN2019098364-appb-000039
其中,M为完整的条带图像所对应的条带图像块数量,具体为:所述条带图像由M个条带图像块拼接形成,其中,
Figure PCTCN2019098364-appb-000040
为所述条带图像中第m个条带图像块所对应的焦平面图像,m≤M。
解调模块13用于将多个样本图像通过解调算法解调获取多个样本图像的焦平面图 像,其包括图像累加单元131和解调单元132,本实施例的样本图像为条带图像,则图像累加单元131用于将至少一行像素的条带图像累加形成第一条带图像,并将至少一行像素的条带图像累加形成第二条带图像;而解调单元132用于将第一条带图像和第二条带图像按所述解调公式解调成所述条带图像的光学层析图像。需要说明的是,本实施例所述的焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为I in=c×|βI 1–αI 2|,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的样本图像的累加和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
上述图像块获取单元123、拼接单元124、图像累加单元131和解调单元132的具体作用过程,前文已详细描述。
以上所述本公开的具体实施方式,并不构成对本公开保护范围的限定。任何根据本公开的技术构思所做出的各种其他相应的改变与变形,均应包含在本公开权利要求的保护范围内。

Claims (10)

  1. 一种高通量光学层析成像方法,其特征在于,包括如下步骤:
    S1、将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,该调制光束在物镜的焦平面具有不完全相同的调制强度;
    S2、采用相机对调制光束照明下的同一样本在不同像素下进行成像,形成的样本图像的计算公式为:
    I(i)=I inf(i)+I out
    其中,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像;
    S3、将不同像素下的样本图像通过解调算法解调获取样本图像的焦平面图像,该焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为:I in=c×|βI 1–αI 2|其中,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的样本图像的累加和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
  2. 根据权利要求1所述的高通量光学层析成像方法,其特征在于,所述相机的成像区域为N行像素,N≥2;在与所述样本成像平面相平行的平面上形成相垂直X与Y两个方向,所述调制光束在X与Y方向分别具有如下特性:所述调制光束在所述N行像素上沿X方向上具有不完全相同的调制强度,所述调制光束在所述N行像素的每一行像素上沿Y方向上具有相同的调制强度;所述像素为行像素,所述样本图像为条带图像。
  3. 根据权利要求2所述的高通量光学层析成像方法,其特征在于,所述步骤S2包括:
    S21、驱动所述调制光束与样本在所述X方向上相对作连续、匀速移动;
    S22、所述相机对所述样本沿其相对运动方向依次连续成像;
    S23、获取按时间顺序所得每帧图像中的第i行像素的条带图像块,条带图像块的计算公式为:
    Figure PCTCN2019098364-appb-100001
    其中,I t(i)为第t帧图像中第i行像素对应的条带图像块,
    Figure PCTCN2019098364-appb-100002
    为I t(i)相对的条带图像块的焦平面图像,即
    Figure PCTCN2019098364-appb-100003
    为完整的条带图像中的第m个条带图像块的焦平面图像,
    Figure PCTCN2019098364-appb-100004
    为I t(i)相对的条带图像块的离焦面图像,f(i)为第i行像素对应的调制强度;
    S24、将每帧图像中的第i行像素的条带图像块依次拼接,即得第i行像素的条带图像,其计算公式为:
    Figure PCTCN2019098364-appb-100005
    其中,M为完整的条带图像所对应的条带图像块数量,m≤M。
  4. 根据权利要求3所述的高通量光学层析成像方法,其特征在于,所述相机的单帧曝光时间与相对移动一行像素的时间相同。
  5. 根据权利要求4所述的高通量光学层析成像方法,其特征在于,所述N行像素的分布方向和宽度分别与调制光束的分布方向和宽度相同,并互为物像共轭关系。
  6. 根据权利要求5所述的高通量光学层析成像方法,其特征在于,所述步骤S3包括:
    S31、将至少一行像素的条带图像累加形成第一条带图像,并将至少一行像素的条带图像累加形成第二条带图像;
    S32、将第一条带图像和第二条带图像按所述解调公式解调成所述条带图像的光学层析图像,则
    Figure PCTCN2019098364-appb-100006
  7. 根据权利要求1所述的高通量光学层析成像方法,其特征在于,所述调制光束呈线状。
  8. 一种高通量光学层析成像系统,其特征在于,包括:
    光束调制模块,其用于将光束调制成能够在物镜的焦平面聚焦并能够在物镜的离焦面发散的调制光束,且该调制光束在物镜的焦平面具有不完全相同的调制强度;
    成像模块,其用于采用相机对调制光束照明下的同一样本在不同像素下进行成像,其形成的样本图像的计算公式为I(i)=I inf(i)+I out,I(i)为i像素下形成的样本图像,f(i)为样本图像I(i)所对应的调制强度,I in为样本图像的焦平面图像,I out为样本图像的离焦面图像;
    解调模块,其用于将不同像素下的样本图像通过解调算法解调获取样本图像的焦平面图像,该焦平面图像即为光学层析图像;其中,所述解调算法的解调公式为I in=c×|βI 1–αI 2|,α、β为正整数,c为大于0的常数,I 1为α个像素下获取的样本图像的累加和,I 2为β个像素下获取的样本图像的累加和;α个像素下的样本图像相对应的调制强度的累加值和β个像素下的样本图像相对应的调制强度的累加值不相同。
  9. 根据权利要求8所述的高通量光学层析成像系统,其特征在于,所述相机的成 像区域为N行像素,N≥2;在与所述样本成像平面相平行的平面上形成相垂直X与Y两个方向,所述调制光束在X与Y方向分别具有如下特性:所述调制光束在所述N行像素上沿X方向上具有不完全相同的调制强度,所述调制光束在所述N行像素的每一行像素上沿Y方向上具有相同的调制强度;所述像素为行像素,所述样本图像为条带图像;所述N行像素的分布方向和宽度分别与调制光束的分布方向和宽度相同,并互为物像共轭关系。
  10. 根据权利要求9所述的高通量光学层析成像系统,其特征在于,所述成像模块包括:
    驱动单元,其用于驱动所述调制光束与样本在X方向上相对作连续、匀速移动,所述相机的单帧曝光时间与相对移动一行像素的时间相同;
    成像单元,其用于采用相机对所述样本沿其相对运动方向依次成像;
    图像块获取单元,其用于获取按时间顺序所得每帧图像中的第i行像素的条带图像块,条带图像块的计算公式为
    Figure PCTCN2019098364-appb-100007
    I t(i)为第t帧图像中第i行像素对应的第m个条带图像块,
    Figure PCTCN2019098364-appb-100008
    为I t(i)相对的条带图像块的焦平面图像,
    Figure PCTCN2019098364-appb-100009
    为I t(i)相对的条带图像块的离焦面图像,f(i)为第i行像素对应的调制强度;
    拼接单元,其用于将每帧图像中的第i行像素的条带图像块依次拼接获得第i行像素的条带图像,所述条带图像的计算公式为
    Figure PCTCN2019098364-appb-100010
    M为完整的条带图像所对应的行像素数量;
    所述解调模块包括:
    图像累加单元,其用于将至少一行像素的条带图像累加形成第一条带图像,并将至少一行像素的条带图像累加形成第二条带图像;
    解调单元,其用于将第一条带图像和第二条带图像按所述解调公式解调成所述条带图像的光学层析图像,则
    Figure PCTCN2019098364-appb-100011
PCT/CN2019/098364 2018-11-01 2019-07-30 一种高通量光学层析成像方法及成像系统 WO2020088013A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
AU2019373533A AU2019373533B8 (en) 2018-11-01 2019-07-30 High-throughput optical sectioning imaging method and imaging system
KR1020217016622A KR102593252B1 (ko) 2018-11-01 2019-07-30 고플럭스 광학 단층촬영 이미징 방법 및 이미징 시스템
JP2021523977A JP7235861B2 (ja) 2018-11-01 2019-07-30 高スループット光トモグラフィイメージング方法及びイメージングシステム
CA3118389A CA3118389C (en) 2018-11-01 2019-07-30 High-throughput optical sectioning imaging method and imaging system
EP19880631.7A EP3875945A4 (en) 2018-11-01 2019-07-30 High-throughput optical tomography method and imaging system
US17/302,329 US11852794B2 (en) 2018-11-01 2021-04-30 High-throughput optical sectioning imaging method and imaging system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811297110.3 2018-11-01
CN201811297110.3A CN111122568B (zh) 2018-11-01 2018-11-01 一种高通量光学层析成像方法及成像系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/302,329 Continuation US11852794B2 (en) 2018-11-01 2021-04-30 High-throughput optical sectioning imaging method and imaging system

Publications (1)

Publication Number Publication Date
WO2020088013A1 true WO2020088013A1 (zh) 2020-05-07

Family

ID=70462911

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/098364 WO2020088013A1 (zh) 2018-11-01 2019-07-30 一种高通量光学层析成像方法及成像系统

Country Status (8)

Country Link
US (1) US11852794B2 (zh)
EP (1) EP3875945A4 (zh)
JP (1) JP7235861B2 (zh)
KR (1) KR102593252B1 (zh)
CN (1) CN111122568B (zh)
AU (1) AU2019373533B8 (zh)
CA (1) CA3118389C (zh)
WO (1) WO2020088013A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207449A (zh) * 2013-04-17 2013-07-17 华中科技大学 一种结构光快速扫描显微成像方法
KR20130115891A (ko) * 2012-04-13 2013-10-22 한국과학기술원 광량 변조와 스캐닝 시스템 기반의 구조 조명 현미경
CN104159094A (zh) * 2014-07-09 2014-11-19 四川大学 一种改善光学扫描全息层析成像效果的方法
CN105371757A (zh) * 2015-10-27 2016-03-02 西安交通大学 一种差动式并行光学层析显微测量装置及方法
CN207516243U (zh) * 2017-07-20 2018-06-19 苏州微景医学科技有限公司 层析内窥显微光谱成像装置

Family Cites Families (77)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE1298851B (de) 1963-12-02 1969-07-03 Steigerwald Verfahren zur Materialbearbeitung mittels Strahlungsenergie
US3622743A (en) 1969-04-28 1971-11-23 Hrand M Muncheryan Laser eraser and microwelder
US3699334A (en) 1969-06-16 1972-10-17 Kollsman Instr Corp Apparatus using a beam of positive ions for controlled erosion of surfaces
US4618262A (en) 1984-04-13 1986-10-21 Applied Materials, Inc. Laser interferometer system and method for monitoring and controlling IC processing
US4892098A (en) 1985-06-26 1990-01-09 Sauer Jude S Tubular tissue welding device without moving parts
US4733397A (en) 1985-08-12 1988-03-22 Electrical Power Research Institute, Inc. Resonant cavity optical modulator
US4859826A (en) 1987-07-17 1989-08-22 Laser Applications, Inc. Simultaneously cutting and welding sheet metal using laser energy
US6485413B1 (en) 1991-04-29 2002-11-26 The General Hospital Corporation Methods and apparatus for forward-directed optical scanning instruments
US5446547A (en) 1991-06-12 1995-08-29 Wyko Corporation Combination of motorized and piezoelectric translation for long-range vertical scanning interferometry
US5494829A (en) 1992-07-31 1996-02-27 Biostar, Inc. Devices and methods for detection of an analyte based upon light interference
US5339323A (en) 1993-04-30 1994-08-16 Lumonics Corporation Laser system for controlling emitted pulse energy
US5387969A (en) 1993-06-22 1995-02-07 Optima Industries, Inc. Machine tool position measurement employing multiple laser distance measurements
DE69533903T2 (de) 1994-08-18 2005-12-08 Carl Zeiss Meditec Ag Mit optischer Kohärenz-Tomographie gesteuerter chirurgischer Apparat
US6454761B1 (en) 1995-01-30 2002-09-24 Philip D. Freedman Laser surgery device and method
US5961861A (en) 1996-01-15 1999-10-05 The University Of Tennessee Research Corporation Apparatus for laser alloying induced improvement of surfaces
US6043870A (en) 1996-07-01 2000-03-28 Cybernet Systems Corporation Compact fiber optic electronic laser speckle pattern interferometer
US6476343B2 (en) 1996-07-08 2002-11-05 Sandia Corporation Energy-beam-driven rapid fabrication system
DE19930408A1 (de) 1999-07-02 2001-01-04 Zeiss Carl Fa OCT-gestütztes Chirurgiesystem
GB2352401B (en) 1999-07-20 2001-06-06 Ajoy Inder Singh Atheroma ablation
ATE398433T1 (de) 1999-09-10 2008-07-15 Haag Ag Streit Vorrichtung zur fotoablation der kornea mit einem laserstrahl
AU2001247240A1 (en) 2000-03-01 2001-09-12 Heraeus Amersil, Inc. Method, apparatus, and article of manufacture for determining an amount of energy needed to bring a quartz workpiece to a fusion weldable condition
DE10020559A1 (de) 2000-04-27 2001-10-31 Hannover Laser Zentrum Laser-Bearbeitung von Materialien
US6720567B2 (en) 2001-01-30 2004-04-13 Gsi Lumonics Corporation Apparatus and method for focal point control for laser machining
JP3603843B2 (ja) 2001-02-23 2004-12-22 日産自動車株式会社 レーザー溶接部の品質モニタリング方法およびその装置
US7274446B2 (en) * 2001-04-07 2007-09-25 Carl Zeiss Jena Gmbh Method and arrangement for the deep resolved optical recording of a sample
US8046057B2 (en) 2001-04-11 2011-10-25 Clarke Dana S Tissue structure identification in advance of instrument
DE10155203A1 (de) 2001-11-09 2003-06-18 Bosch Gmbh Robert Laserbearbeitungsvorrichtung
IL149016A0 (en) 2002-04-07 2004-03-28 Green Vision Systems Ltd Green Method and device for real time high speed high resolution spectral imaging
JP3941104B2 (ja) 2002-06-11 2007-07-04 ブラザー工業株式会社 インクジェット記録装置
DE10300091A1 (de) 2003-01-04 2004-07-29 Lubatschowski, Holger, Dr. Mikrotom
US7436520B1 (en) 2005-01-18 2008-10-14 Carl Zeiss Smt Ag Method of calibrating an interferometer optics and of processing an optical element having an optical surface
WO2007030741A2 (en) * 2005-09-09 2007-03-15 Trustees Of Boston University Imaging system using dynamic speckle illumination
WO2007038975A1 (en) 2005-10-05 2007-04-12 Alexandre Carpentier Method for cutting a biological tissue, and installation for cutting a biological tissue
JP4850495B2 (ja) 2005-10-12 2012-01-11 株式会社トプコン 眼底観察装置及び眼底観察プログラム
US7333214B2 (en) 2006-03-31 2008-02-19 Mitutoyo Corporation Detector for interferometric distance measurement
JP4958489B2 (ja) 2006-06-30 2012-06-20 株式会社キーエンス レーザ加工装置、レーザ加工条件設定装置、レーザ加工条件設定方法、レーザ加工条件設定プログラム
JP4328349B2 (ja) 2006-11-29 2009-09-09 株式会社日立製作所 残留応力測定方法及び装置
US7688453B2 (en) 2006-12-21 2010-03-30 Johnson & Johnson Vision Care, Inc. Interferometry testing of lenses, and systems and devices for same
KR101519932B1 (ko) 2006-12-22 2015-05-13 지고 코포레이션 표면 특징물의 특성을 측정하기 위한 장치 및 방법
JP2010520801A (ja) 2007-03-13 2010-06-17 オプティメディカ・コーポレイション 眼球手術及び減張切開部を作成するための装置
DE102007016444A1 (de) 2007-04-05 2008-10-16 Precitec Optronik Gmbh Bearbeitungseinrichtung
DE102007032743A1 (de) 2007-07-13 2009-01-15 Robert Bosch Gmbh Messvorrichtung, Messverfahren, Laserstrahlbearbeitungsvorrichtung, Laserstrahlbearbeitungsverfahren
US7619746B2 (en) 2007-07-19 2009-11-17 Zygo Corporation Generating model signals for interferometry
US20100324542A1 (en) 2007-11-02 2010-12-23 Kurtz Ronald M Method to Guide a Cataract Procedure by Corneal Imaging
US9229209B2 (en) * 2008-01-21 2016-01-05 Carl Zeiss Smt Gmbh Autofocus device and autofocusing method for an imaging device
EP2113332B1 (de) 2008-05-02 2010-08-18 Leister Process Technologies Verfahren und Laservorrichtung zum Bearbeiten und/oder Verbinden von Werkstücken mittels Laserstrahlung mit Leistungswirk- und Pilotlaser und mindestens einem diffraktiven optischen Element
WO2009157570A1 (ja) 2008-06-23 2009-12-30 Jfeスチール株式会社 レーザ溶接鋼管の製造方法
US8723078B2 (en) 2008-11-21 2014-05-13 The Regents Of The University Of Michigan Monitoring of a welding process
US8264694B2 (en) 2009-03-16 2012-09-11 Ut-Battelle, Llc Quantitative phase-contrast and excitation-emission systems
US20110222024A1 (en) 2010-03-15 2011-09-15 Walsin Lihwa Corporation Illumination system for projection display
DE102010016862B3 (de) 2010-05-10 2011-09-22 Precitec Optronik Gmbh Materialbearbeitungsvorrichtung mit in-situ Messen des Bearbeitungsabstands
US20110284508A1 (en) 2010-05-21 2011-11-24 Kabushiki Kaisha Toshiba Welding system and welding method
US10124410B2 (en) 2010-09-25 2018-11-13 Ipg Photonics Corporation Methods and systems for coherent imaging and feedback control for modification of materials
WO2012037694A2 (en) 2010-09-25 2012-03-29 Queen's University At Kingston Methods and systems for coherent imaging and feedback control for modification of materials
CA3190196A1 (en) 2010-09-25 2012-03-25 Ipg Photonics Corporation Methods and systems for coherent imaging and feedback control for modification of materials
JP5252026B2 (ja) 2011-05-10 2013-07-31 パナソニック株式会社 レーザ溶接装置及びレーザ溶接方法
DE102012207835A1 (de) 2011-05-11 2012-11-15 Arges Gmbh Verfahren und Vorrichtung zur Überwachung eines Laserbearbeitungsprozesses
DE102011104550B4 (de) 2011-06-17 2014-04-30 Precitec Kg Optische Messvorrichtung zur Überwachung einer Fügenaht, Fügekopf und Laserschweißkopf mit der selben
CN102288117B (zh) * 2011-06-23 2013-11-06 中国科学院西安光学精密机械研究所 基于光学联合变换相关器的像移探测成像系统
DE102011114500B4 (de) * 2011-09-29 2022-05-05 Fei Company Mikroskopvorrichtung
JP2015502187A (ja) 2011-10-17 2015-01-22 アイ シン アジョイ 動脈樹の領域からアテローム性動脈硬化を除去するための方法および装置
US9360660B2 (en) * 2012-05-24 2016-06-07 Northwestern University Methods and apparatus for laser scanning structured illumination microscopy and tomography
PL2972479T3 (pl) 2013-03-13 2021-04-19 Ipg Photonics (Canada) Inc. Sposoby i układy do charakteryzacji właściwości obróbki laserem poprzez pomiar dynamiki kapilary z zastosowaniem interferometrii
US9364167B2 (en) 2013-03-15 2016-06-14 Lx Medical Corporation Tissue imaging and image guidance in luminal anatomic structures and body cavities
US9690107B2 (en) 2013-03-15 2017-06-27 Trumpf Laser Gmbh Device for wavelength combining of laser beams
DE102013008269C5 (de) 2013-05-15 2019-01-24 Precitec Optronik Gmbh Bearbeitungskopf für eine Laserbearbeitungsvorrichtung
DE102013015656B4 (de) 2013-09-23 2016-02-18 Precitec Optronik Gmbh Verfahren zum Messen der Eindringtiefe eines Laserstrahls in ein Werkstück, Verfahren zum Bearbeiten eines Werkstücks sowie Laserbearbeitungsvorrichtung
CN103852878B (zh) * 2014-01-08 2016-05-25 麦克奥迪实业集团有限公司 一种具有实时聚焦的显微切片快速数字扫描装置及其方法
DE102014007887B4 (de) 2014-05-26 2015-12-10 Lessmüller Lasertechnik GmbH Laserbearbeitungsvorrichtung mit einer Messvorrichtung zum Erfassen von Oberflächendaten und/oder Grenzflächen eines durch eine Laserbearbeitungsvorrichtung zu bearbeitenden Werkstücks
US10602925B2 (en) * 2015-09-11 2020-03-31 Simon Fraser University Coherence-gated wavefront-sensorless adaptive-optics multi-photon microscopy, and associated systems and methods
EP3362778B1 (en) * 2015-10-13 2022-08-03 Omega Biosystems Incorporated Multi-modal fluorescence imaging flow cytometry system
WO2018083772A1 (ja) 2016-11-02 2018-05-11 株式会社ニコン 顕微鏡システム
CN106691394B (zh) * 2017-02-17 2023-04-18 浙江大学 一种基于光程编码的高分辨长焦深oct成像系统和方法
CN107607040B (zh) * 2017-08-11 2020-01-14 天津大学 一种适用于强反射表面的三维扫描测量装置及方法
CN108020503B (zh) * 2017-11-20 2020-09-08 苏州博芮恩光电科技有限公司 一种光片照明显微切片成像系统及成像结果处理方法
CN108319008B (zh) * 2018-02-02 2019-11-12 华中科技大学 一种光学显微成像方法及装置
CN108519329B (zh) * 2018-03-26 2021-01-15 华中科技大学 一种多路扫描与探测的线共聚焦成像装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130115891A (ko) * 2012-04-13 2013-10-22 한국과학기술원 광량 변조와 스캐닝 시스템 기반의 구조 조명 현미경
CN103207449A (zh) * 2013-04-17 2013-07-17 华中科技大学 一种结构光快速扫描显微成像方法
CN104159094A (zh) * 2014-07-09 2014-11-19 四川大学 一种改善光学扫描全息层析成像效果的方法
CN105371757A (zh) * 2015-10-27 2016-03-02 西安交通大学 一种差动式并行光学层析显微测量装置及方法
CN207516243U (zh) * 2017-07-20 2018-06-19 苏州微景医学科技有限公司 层析内窥显微光谱成像装置

Also Published As

Publication number Publication date
EP3875945A1 (en) 2021-09-08
US11852794B2 (en) 2023-12-26
AU2019373533B2 (en) 2023-01-19
KR102593252B1 (ko) 2023-10-23
AU2019373533A1 (en) 2021-06-03
JP7235861B2 (ja) 2023-03-08
AU2019373533B8 (en) 2023-02-09
KR20210084605A (ko) 2021-07-07
JP2022511677A (ja) 2022-02-01
CA3118389A1 (en) 2020-05-07
CN111122568B (zh) 2022-04-22
CN111122568A (zh) 2020-05-08
US20210311292A1 (en) 2021-10-07
EP3875945A4 (en) 2021-12-29
CA3118389C (en) 2023-06-27

Similar Documents

Publication Publication Date Title
US7339148B2 (en) Confocal microscope
JP6035018B2 (ja) 連続的な光シートを用いるspim顕微鏡
US11906723B2 (en) High-throughput optical sectioning three-dimensional imaging system
KR20200070313A (ko) 이미지 재건 방법, 장치 및 현미 이미징 장치
CN115248197A (zh) 一种三维成像装置和成像方法
US11356593B2 (en) Methods and systems for single frame autofocusing based on color- multiplexed illumination
WO2020088013A1 (zh) 一种高通量光学层析成像方法及成像系统
JP5471715B2 (ja) 合焦装置、合焦方法、合焦プログラム及び顕微鏡
CN107850766A (zh) 用于光学显微镜中的图像处理的系统和方法
CN111650739B (zh) 基于dmd的单帧曝光快速三维荧光成像系统及方法
CN112986195B (zh) 一种显微层析成像方法与装置
KR101391180B1 (ko) 레이저 스캔 구조조명 이미징 방법
CN117631249B (zh) 线扫共聚焦扫描光场显微成像装置及方法
CN111260747B (zh) 基于虚拟数字调制的高通量光学层析成像方法及系统
CN220913429U (zh) 一种光学成像装置
CN117631249A (zh) 线扫共聚焦扫描光场显微成像装置及方法
CN116539576A (zh) 寻址扫描超分辨率显微成像方法及相关设备
CN115689959A (zh) 一种正交线扫描成像处理方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19880631

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021523977

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 3118389

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20217016622

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019373533

Country of ref document: AU

Date of ref document: 20190730

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019880631

Country of ref document: EP

Effective date: 20210601