WO2021184169A1 - 一种用于显微镜的实时自动对焦系统 - Google Patents

一种用于显微镜的实时自动对焦系统 Download PDF

Info

Publication number
WO2021184169A1
WO2021184169A1 PCT/CN2020/079547 CN2020079547W WO2021184169A1 WO 2021184169 A1 WO2021184169 A1 WO 2021184169A1 CN 2020079547 W CN2020079547 W CN 2020079547W WO 2021184169 A1 WO2021184169 A1 WO 2021184169A1
Authority
WO
WIPO (PCT)
Prior art keywords
spot
microscope
objective lens
real
lens
Prior art date
Application number
PCT/CN2020/079547
Other languages
English (en)
French (fr)
Inventor
李思宏
黄术强
傅雄飞
于跃
温慧
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2020/079547 priority Critical patent/WO2021184169A1/zh
Publication of WO2021184169A1 publication Critical patent/WO2021184169A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals

Definitions

  • the present invention relates to the field of auto-focusing technology, and more specifically, to a real-time auto-focusing system for microscopes.
  • Autofocus technology is the key core technology for stable imaging of the microscopy system. It can compensate for the focus drift of the microscopy system due to environmental temperature changes, mechanical vibrations, motion vibrations and other reasons during the long-term image data acquisition process to ensure that the acquisition is clear Microscopic image.
  • Traditional microscope operations require users to frequently adjust the distance between the objective lens and the glass slide to obtain a proper focus, which is time-consuming and labor-intensive.
  • automated microscopic imaging technology requires stable and reliable autofocus technology to ensure the quality of collected image data.
  • the focus reference plane is not the actual sample imaging focus plane. Since the auxiliary light source spot can only be reflected on the interface between the sample and the glass slide, the focus reference plane is the reflective interface, which is different from the imaging focal plane, which results in the need to move the objective lens a fixed distance to reach the imaging focal plane after focusing, so that the focus accuracy can be achieved.
  • the second is that the one-time compensation for focus drift depends on the accurate calculation of the defocus and the accuracy of the moving machinery.
  • the software-based autofocus method takes a long time and cannot focus in real time, and when the sample has a three-dimensional structure, the software algorithm cannot find the clearest imaging plane position due to the deep imaging focal plane range, resulting in focus failure .
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art and provide a real-time auto-focusing system for microscopes, which can dynamically control the objective lens to adjust to the best focus point by detecting and analyzing the real-time focus state of the microscopy system.
  • the invention provides a real-time auto-focusing system for microscopes.
  • the system includes an infrared light source module, an offset lens light path module, a detection module, and a focus control module.
  • the infrared light source module is used to emit a parallel laser beam; the offset lens light path module parallels the output of the infrared light source module.
  • the laser beam is expanded into divergent light, and the sample interface is irradiated through the microscope objective lens, so that the laser diffraction spot irradiated at the sample reflection interface forms coma aberration;
  • the detection module modulates the coma-shaped diffraction spot formed by the reflection of the sample interface It is a linear light spot, and by analyzing the linear light spot, information related to the defocus amount of the microscope system in the current field of view is obtained;
  • the focus control module iteratively controls the axial movement of the microscope objective lens based on the obtained defocus amount related information, Until the defocus amount meets the set target.
  • the infrared light source module includes a laser and a beam splitting device.
  • the laser is used to generate a parallel laser beam.
  • Offset lens optical path module is used to generate a parallel laser beam.
  • the offset lens optical path module includes a first reflector, an offset lens fixed on a one-dimensional electric translation stage, and a second reflector, and the first reflector receives the infrared light source module to emit light.
  • the parallel laser beam is reflected to the offset lens, is incident on the second mirror through the offset lens, and is reflected into the objective lens through the internal filter of the microscope to illuminate the sample interface.
  • the detection module includes a focusing lens, a cylindrical mirror, a camera, and a control system.
  • the focusing lens After the laser diffraction spot reflected by the irradiated sample interface passes through the spectroscopic device of the infrared light source module, the focusing lens and the cylindrical surface The lens is focused on the photosensitive chip of the camera.
  • the cylindrical mirror transforms the circular spot space into an elliptical spot to form a linear spot.
  • the control system communicates with the camera and obtains the current view by analyzing the spatial position change of the linear spot. Information about the amount of defocus of the off-site microscopy system.
  • the defocus amount of the microscopy system is calculated according to the following steps:
  • the focus control module is respectively connected to the control system and the microscope objective lens, and is used to iteratively control the axial movement of the microscope objective lens according to the corresponding relationship between the obtained defocus amount related information and the adjustment feedback amount of the microscope objective lens , Until the defocus amount meets the set target.
  • the focus control module is a motor controller, and the adjustment feedback amount of the microscope objective lens is determined by a proportional-integral-derivative controller.
  • the adjustment feedback amount of the microscope objective lens is calculated by the following formula:
  • the infrared light source module uses a near-infrared or infrared wavelength laser with a wavelength range of 650 nm to 1100 nm and a power of less than 5 mW.
  • the beam splitter is a beam splitter, a D-shaped mirror or a mirror with a knife edge.
  • the advantage of the present invention is that the operation process of the proposed auto-focus system is independent of the selection of the imaging focal plane by the microscopic imaging system, and the instantaneous focus state of the microscopic system is detected with sub-pixel precision. Calculate the defocus amount of the focus shift, and dynamically feedback and control the objective lens to adjust to the best focus point.
  • the invention has the advantages of real-time, high stability, high precision and low cost.
  • Fig. 1 is a schematic diagram of a real-time auto-focusing system for a microscope according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the principle of real-time autofocus according to an embodiment of the present invention.
  • Fig. 3 is a flowchart of a method for detecting a defocus amount according to an embodiment of the present invention
  • Fig. 4 is a data diagram of an auto-focus effect test experiment according to an embodiment of the present invention.
  • the invention provides a real-time high-precision auto-focusing system for microscopes.
  • the system uses a near-infrared laser beam through the objective lens of the microscope to irradiate the interface of the biological sample slide to form a laser spot.
  • the laser spot is reflected by the interface and collected by the same objective lens, and the image is imaged on the photosensitive chip of the autofocus system.
  • the drift of the focus will bring about the corresponding movement of the position of the reflected light spot on the photosensitive chip.
  • the microscope objective lens is controlled to correct the drift and complete the real-time autofocus process.
  • the system involves multiple modular subsystems, including an infrared light source module, an offset lens optical path module, a detection module, and a focus control module.
  • the infrared light source module includes a laser 1 and a beam splitter 2.
  • laser 1 generates near-infrared or infrared wavelength laser
  • the output end of laser 1 is a single-mode fiber
  • the single-mode fiber is used to couple the laser output to the collimator to form a fundamental mode Gaussian beam and collimate the output divergence angle less than 0.5mrad Parallel laser beams.
  • the laser beam then passes through the attenuator to reduce the power, and enters the offset lens optical path module through the beam splitter 2.
  • a low-power (less than 5 mW) laser with near-infrared or infrared wavelengths (such as 650 nm-1100 nm) can be used.
  • the offset lens optical path module includes a first mirror 6, a offset lens 7, a one-dimensional electric translation stage (not shown) and a second mirror 8.
  • the laser beam emitted from the infrared light source module is incident on the first reflecting mirror 6 and reflected to the offset lens 7.
  • the offset lens 7 is installed and fixed on a one-dimensional electric translation stage, and its position can be adjusted along the optical axis.
  • the laser beam is incident on the second reflector 8 through the offset lens 7, is reflected into the objective lens 10 by the short-pass filter 9 inside the microscope, and is finally irradiated at the sample interface 11.
  • the laser beam after being spatially transformed by the offset lens is a divergent beam with a certain degree of divergence, and a spot (such as 3mm to 4mm) close to the size of the objective aperture is formed at the rear focal plane of the objective lens 10
  • the incident position of the objective lens 10 is adjusted by the first reflector 6 and the second reflector 8 to be close to the edge of the objective lens diaphragm, so that the laser beam condensed by the objective lens 10 is irradiated at the sample interface 11 at a certain inclination with the optical axis of the objective lens; the incident laser and the objective lens
  • the optical axis is adjusted to a certain inclination angle by the first reflector 6 and the second reflector 8, so that the convergent laser spot at the sample interface 11 forms a relatively obvious coma aberration.
  • the reflected laser spot returns via the same path, and then enters the detection module after being reflected by the beam splitter 2 in the infrared light source module.
  • the offset lens optical path module has the following characteristics: the offset lens is fixed on a one-dimensional electric translation stage, and its position can be adjusted along the optical axis, so that the transformation of the laser beam can satisfy the selection of different objective lenses;
  • the lens transforms the parallel laser beam into a divergent beam with a certain degree of divergence, so that the focal point of the beam passing through the objective lens is far away from the focal plane of the objective lens;
  • the divergent beam forms a laser beam with a diameter close to the size of the objective aperture at the focal plane of the objective lens.
  • the laser spot irradiated at the reflection interface of the sample forms a relatively obvious diffraction spot; the laser beam optical axis of the incident objective lens does not coincide with the optical axis of the objective lens, and is close to the edge of the objective lens diaphragm, so that the convergent beam of the objective lens forms a certain inclination angle with the objective lens optical axis. Obliquely irradiate at the sample reflection interface; the incident laser beam optical axis and the objective lens have a certain angle, so that the laser spot irradiated at the sample reflection interface forms a more obvious coma aberration.
  • the detection module includes a focusing lens 3, a cylindrical lens 4 and an imaging camera 5 from the angle of optical path configuration.
  • the reflected laser spot passes through the beam splitter 2 in the optical path system and is focused on the photosensitive chip of the imaging camera 5 by the focusing lens 3 and the cylindrical lens 4.
  • the cylindrical lens 4 transforms the circular spot space into an elliptical spot, and after appropriately selecting the focal length, an approximately linear spot can be formed.
  • the imaging spot is adjusted by the beam splitter 2 to the center of the photosensitive chip of the camera 5.
  • the detection module also includes a control system 12.
  • a computer control system is taken as an example for introduction.
  • the imaging camera 5 communicates with the computer, and the laser spot image collected by its photosensitive chip is algorithmically detected and analyzed by the computer control system 12, and the sub-pixel precision coordinates of the linear spot are calculated, and the defocus of the focus shift is obtained by coefficient transformation. quantity.
  • the focus control module (take the motor controller 13 as an example) is used to receive feedback from the control system 12 on the defocus amount analysis and control the movement of the objective lens.
  • a proportional-integral-derivative controller PID
  • PID proportional-integral-derivative controller
  • the auto-focusing process of the system provided by the present invention includes the following steps:
  • Step S210 the infrared light source module emits a parallel laser beam, which is expanded by the offset lens optical path module into divergent light, illuminates the sample interface through the microscope objective lens, and the coma-shaped diffraction spot formed by the interface reflection is further spatially modulated into approximately parallel lines by the detection module Type light spot, and perform image collection to obtain the light spot image;
  • Step S220 the computer control system runs an image detection algorithm on the linear light spot image, and calculates the defocus amount of the microscope system in the current field of view;
  • Step S230 the motor controller compensates the defocus amount
  • step S240 steps S210 to S230 are repeated until the defocus amount in the current field of view is less than the set threshold, and the auto focus is completed.
  • the present invention analyzes and judges the defocus state of the microscopy system by means of the reflection spot of the near-infrared or infrared laser at the interface of the glass slide.
  • the immersion medium between the objective lens and the glass slide is usually oil or water to improve the resolution.
  • the interface with strong laser reflection is the interface with the larger difference in refractive index between the glass slide and the biological sample.
  • the focal plane of the objective lens (that is, the object plane) is in the sample area far from the reflective interface, and the distance from the reflective interface is the focus offset distance. Adjusting this focus offset distance value can achieve different depths of biological samples with three-dimensional structures. Imaging observation. In the microscope imaging process, when a specific imaging focal plane is selected, the focus offset distance is determined.
  • a laser beam with a small divergence angle can be regarded as parallel light, and its convergence through the objective lens will focus on the focal plane of the objective lens.
  • the laser spot focused on the focal plane of the objective lens has a very high energy density due to its extremely small size, which will cause a local thermal effect on the biological tissue sample.
  • an offset lens is added to the optical path, so that the parallel laser beam passes through the offset lens to form a divergent beam, which is incident into the objective lens with a certain divergence.
  • the laser with a certain degree of divergence passes through the converging effect of the objective lens to form a laser focusing surface with a certain distance from the focal plane of the objective lens far away from the objective lens.
  • the distance between the laser focusing surface and the focal plane of the objective lens depends on the divergence of the incident laser. At this time, because the laser spot with high energy density on the laser focusing surface is far away from the biological sample, it will no longer affect the biological sample, and realize the protection of the biological sample in the long-term microbiological experiment.
  • the reflective interface where the glass slide and the sample are connected will form a larger laser spot.
  • the reflective interface has a strong reflection effect on the laser spot, so a laser with a lower energy (such as less than 1mW) can realize this technical solution.
  • the current spatial position of the unfocused laser spot at the reflective interface will be used as the basis for the focus drift of the microscopic imaging system, and does not depend on the change in the distance between the imaging plane and the reflective interface (ie, the focus offset distance).
  • the laser beam incident on the objective lens can reach a size similar to the aperture of the objective lens, forming a more significant diffraction effect on the focused laser spot.
  • the larger laser spot at the reflective interface has a circular hole diffraction structure, that is, a concentric circular ring-shaped spot.
  • the offset lens is fixed on a one-dimensional electric platform, and the distance can be adjusted according to the specifications of the objective lens (such as magnification, numerical aperture, etc.).
  • the optical axis of the laser incident on the objective lens deviates from the optical axis of the objective lens by a certain distance, and is incident on the reflective interface through the convergence of the objective lens to form an angle ⁇ with the optical axis of the objective lens.
  • the laser optical axis and the objective lens optical axis form a certain angle, and the laser beam forms an oblique incident state, so that the laser spot at the reflective interface is far away from the center of the imaging field of view, avoiding direct irradiation of biological samples.
  • the oblique incidence can avoid the ghosting of the spot caused by the multiple reflections between the optical elements in the optical path of the microscopy system in the case of vertical incidence, thereby improving the focusing effect;
  • the laser spot on the reflective interface forms a coma aberration ,
  • the eccentric circular ring-shaped diffracted spot with an asymmetric structure is formed, and the energy is concentrated on one side of the diffracted spot.
  • the two sides of the center of gravity of the light spot are respectively the ring structure and the background noise.
  • the invention uses the asymmetry of the diffraction spot to determine the defocusing direction of the microscopy system.
  • the size of the laser spot located on the reflective interface different from the objective lens will linearly decrease as the objective lens moves away from the reflective interface, and the position of the diffraction structure on the plane will also be along the line.
  • the distance between the objective lens and the reflective interface will change.
  • the center of the laser spot will shift in the horizontal direction by +dx; when the objective lens is close to the reflective interface, the distance change is - ⁇ , the center of the laser spot will be Pan in the horizontal direction by -dx amount.
  • the laser spot with the coma-shaped phase difference diffraction structure at the reflective interface passes through the objective lens and is focused on the photosensitive chip of the imaging camera.
  • the present invention uses a cylindrical mirror to stretch the light spot one-dimensionally. Specifically, the cylindrical mirror is stretched in a direction perpendicular to the moving direction of the light spot on the plane, so that the laser light spot of the diffractive structure forms a series of parallel linear light spots on the imaging photosensitive chip.
  • the comet-shaped phase difference effect makes the linear spot with the most concentrated energy on one side of the parallel lines.
  • the defocus amount of the focus shift reflects the movement of the parallel line along its orthogonal direction, and the shift direction reflects the movement direction of the linear light spot with the most concentrated energy.
  • the imaging camera communicates with the computer, transmits image data in real time, and applies a displacement detection algorithm with sub-pixel accuracy to the parallel linear laser spot.
  • the displacement-based defocus detection method includes the following steps:
  • Step S310 Collect an original image containing parallel-line light spots, for example, an image with a pixel size of MXN, denoted as I, contains parallel-line light spots in the vertical direction, where M and N can be based on the resolution and image size of the original image, etc. Choose the appropriate value;
  • Step S320 accumulate the original image along the line-shaped light spots perpendicular to the line to obtain a 1XN image intensity curve, which is expressed as Profile;
  • Step S330 Detect the peak position of the linear laser spot with the highest signal value from the profile profile, denoted as PeakLoc;
  • Step S340 taking [PeakLoc, M/2] as the center to intercept an image sub-window with a size of 64 ⁇ 64 pixels, denoted as ISub;
  • Step S350 using a threshold algorithm based on OTSU (Maximum Between-Class Variance Method) to calculate a binarized segmentation threshold T for the original image;
  • OTSU Maximum Between-Class Variance Method
  • Step S360 using the threshold T to binarize the image sub-window to obtain a binarized image, denoted as BSub;
  • Step S370 extract the linear light spot at the center of the BSub through morphological operations, and calculate the barycentric coordinates with sub-pixel accuracy in combination with ISub, which is expressed as LocX;
  • Step S380 comparing the spot coordinate LocX with the focal plane setting value coordinate ZeroX calculated by using steps S310 to S370 to obtain the movement amount of the spot, which is expressed as dx;
  • iterative feedback is performed before the defocus amount of the microscopy system is lower than the preset threshold.
  • the beam splitter can be replaced with a D-shaped mirror or a knife-edge Mirror, under this alternative, the power of the laser can be appropriately reduced;
  • the offset lens can be replaced with a variable focal length lens, such as a flexible zoom lens driven by electro-deformation;
  • the imaging camera can be replaced by a linear array camera;
  • the feedback PID algorithm can be Replace with other algorithms with fast adjustment and stability.
  • the objective lens is scanned along the optical axis of the objective lens with a fixed step length of 10nm precision, and the linear light spot collected by the imaging camera is detected by algorithm at each position point, and its spatial position (unit is pixel coordinates) is obtained.
  • the scanning range is ⁇ 4 ⁇ m far exceeding the depth of field of the objective lens, and the data of the relationship between the pixel coordinate and the axial coordinate of the objective lens is obtained, and linear fitting is performed.
  • the R-square factor reaches 99.5%
  • the transformation coefficient K of the auto-focus system is 240 nanometers/pixel. That is, for every detected displacement of 1 pixel, the defocus of the microscopic imaging system linearly corresponds to about 240 nm.
  • a 60X oil immersion objective lens is used, with a numerical aperture of 1.40 and a depth of field of about 542nm.
  • the sample slide device is in a constant temperature box, and the constant temperature box and the objective lens are continuously heated to 37°C and maintained for 4 hours to achieve normal biological observation experimental conditions.
  • the experimental test time is 30 minutes, and the focus drift of the microscopic imaging system is tested after the auto focus is turned off and the auto focus is turned on. When autofocus is not turned on, the imaging system loses focus within a few minutes and drifts out of the depth of field of the objective lens.
  • the standard deviation (STD) of the drift reached 578.2nm;
  • the imaging system continuously maintains the focus state
  • the standard deviation of the drift amount is 4.7nm
  • the distribution range is ⁇ 14.1nm
  • the focus repeat accuracy can reach 5.2% of the depth of field of the objective lens, realizing high-precision autofocus.
  • Figure 4 where the ordinate represents the axial position (Axial position), the abscissa represents the test time, Autofocus OFF represents the auto focus off (corresponding to the upper curve), and Autofocus ON represents the auto focus on (corresponding to the lower curve).
  • the laser focus plane is the sample reflection plane, which will cause local high-energy density irradiation of biological samples.
  • the present invention uses an offset lens to make the laser focus plane far away from the sample for imaging. Area, avoid local high-energy-density light irradiation; for the existing hardware-based auto-focus method, the laser focus plane is the objective focal plane, and there is a certain distance from the sample reflection interface (focus reference plane), which leads to the need to re-focus after focusing Move the objective lens to a fixed distance to reduce the problem of focusing accuracy.
  • the present invention uses the spatial position of the diffracted spot in the non-focused state as the basis for the focus shift of the microscopic imaging system, and does not rely on the change of the distance between the focal plane of the objective lens and the reflection interface, and improves the stability. And focusing accuracy; the present invention uses the diffracted spot with coma aberration, analyzes its real-time displacement under sub-pixel precision, and iteratively adjusts the real-time defocus of the microscopic imaging system through PID algorithm to achieve high-precision autofocus.
  • the defocus amount detection process in the foregoing embodiment of the present invention may be a system, a method, and/or a computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present invention.
  • the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon
  • the computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or in one or more programming languages.
  • Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user's computer through any kind of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect to the user's computer) connect).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions.
  • the computer-readable program instructions are executed to implement various aspects of the present invention.
  • These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function.
  • Executable instructions may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation through hardware, implementation through software, and implementation through a combination of software and hardware are all equivalent.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Microscoopes, Condenser (AREA)
  • Automatic Focus Adjustment (AREA)

Abstract

一种用于显微镜的实时自动对焦系统，包括红外光源模块、偏移透镜光路模块、检测模块和对焦控制模块。红外光源模块用于发射平行激光光束；偏移透镜光路模块将红外光源模块出射的平行激光光束扩束为发散光，并通过显微镜物镜（10）照射样品界面（11），以使照射在样品界面（11）处的激光衍射光斑形成彗星形；检测模块将样品界面（11）反射形成的彗星形衍射光斑调制为线型光斑并通过对线型光斑分析获得当前视场下显微系统的离焦量相关信息；对焦控制模块基于获得的离焦量相关信息迭代控制显微镜物镜（10）的轴向运动，直至离焦量满足设定目标。

Description

一种用于显微镜的实时自动对焦系统 技术领域
本发明涉及自动对焦技术领域，更具体地，涉及一种用于显微镜的实时自动对焦系统。
背景技术
近年来，显微成像技术广泛应用于高速发展的生物医学领域，其中分为研究级的全自动显微镜和应用于病理诊断的数字病理切片扫描仪。全自动显微镜通常需要长时间(持续约数小时至数十小时)的自动化观测生物样品，而数字病理切片扫描仪则是借助显微系统快速扫描采集病理切片得到全玻片数字化图像，提供会诊诊断依据。
自动对焦技术是显微系统进行稳定成像的关键核心技术，可以补偿显微系统在长时间的图像数据采集过程中由于环境温度变化、机械振动、运动震动等原因导致的焦点漂移，确保获取到清晰的显微图像。传统的显微镜操作需要使用者频繁地调节物镜和玻片之间的距离以获得合适的焦点，这种方式耗时费力。而自动化的显微成像技术需要稳定可靠的自动对焦技术，以保证采集图像数据的质量。
目前，应用于显微镜的自动对焦方法主要分为两种，一种是基于软件图像算法，即沿着物镜光轴方向扫描不同的成像平面得到多组样品图像，依靠图像质量评价函数确定离焦量；另外一种是基于硬件，借助额外的辅助光源(如激光、LED等)通过物镜照射到样品玻片表面，根据反射的光斑形态确定系统的离焦量。
在现有技术中，基于硬件的自动对焦方法通常借助辅助光源并计算反射光斑形态来确定显微系统的离焦量。这种方法的明显缺点在于：一是对焦参考平面并非实际的样品成像焦平面。由于辅助光源光斑只能于样品与玻片的界面上反射，对焦参考平面为此反射界面，不同于成像焦平面，从 而导致在对焦结束后需要移动物镜固定距离才能达到成像焦平面，使对焦精度变差；二是由于对焦点漂移的一次性补偿依赖于离焦量的准确计算和移动机械的精度，在高数值孔径的物镜下会带来不小的偏差，从而进一步降低对焦精度；三是由于引入复杂的光束方法而导致系统复杂和成本高。而基于软件的自动对焦方法，耗时较长且无法实时对焦，并且当样品具备三维结构时，由于清晰的成像焦平面范围较深，软件算法无法寻找到最清晰的成像平面位置，导致对焦失败。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种用于显微镜的实时自动对焦系统，通过对显微系统即时的焦点状态进行检测分析，动态地控制物镜调整至最佳对焦点。
本发明提供一种用于显微镜的实时自动对焦系统。该系统包括红外光源模块、偏移透镜光路模块、检测模块和对焦控制模块，其中，所述红外光源模块用于发射平行激光光束；所述偏移透镜光路模块将所述红外光源模块出射的平行激光光束扩束为发散光，并通过显微镜物镜照射样品界面，以使照射在样品反射界面处的激光衍射光斑形成彗星形像差；所述检测模块将样品界面反射形成的彗差形衍射光斑调制为线型光斑，并通过对该线型光斑分析获得当前视场下显微系统的离焦量相关信息；所述对焦控制模块基于获得的离焦量相关信息迭代控制显微镜物镜的轴向运动，直至离焦量满足设定目标。
在一个实施例中，所述红外光源模块包括激光器和分光器件，所述激光器用于产生平行激光光束，该平行激光光束依次经准直器和衰减片后，由所述分光器件入射至所述偏移透镜光路模块。
在一个实施例中，所述偏移透镜光路模块包括第一反射镜、固定在一维电动平移台上的偏移透镜和第二反射镜，所述第一反射镜接收所述红外光源模块出射的平行激光光束，并反射至所述偏移透镜，经所述偏移透镜入射到所述第二反射镜上，并经显微镜内部滤光片反射入物镜，进而照射样品界面。
在一个实施例中，所述检测模块包括聚焦透镜、柱面镜、相机和控制系统，由照射样品界面所反射的激光衍射光斑经所述红外光源模块的分光器件后，由聚焦透镜和柱面镜聚焦在相机的感光芯片上，柱面镜将圆形光斑空间变换为椭圆形光斑，以形成线型光斑，控制系统与相机进行通讯并通过分析该线型光斑的空间位置变化以获得当前视场下显微系统的离焦量相关信息。
在一个实施例中，根据以下步骤计算显微系统的离焦量：
采集包含线型光斑的原始图像；
对原始图像沿垂直于线型光斑方向像素累加，得到图像强度曲线；
从图像强度曲线中检测信号值最高的线型激光光斑的峰值位置；
基于峰值位置截取包含多个像素的图像子窗口；
对原始图像计算二值化分割阈值；
使用分割阈值对图像子窗口进行二值化，得到二值化的图像；
提取二值化图像中心的线型光斑，并结合图像子窗口计算得到亚像素级精度的光斑中心坐标；
利用获得的光斑中心坐标得到光斑移动量dx；
通过焦点漂移量与光斑移动距离的对应关系δ＝K·dx得到离焦量δ，其中K是变换系数。
在一个实施例中，所述对焦控制模块分别连接所述控制系统和显微镜物镜，用于根据获得的离焦量相关信息与显微镜物镜调整反馈量之间的对应关系迭代控制显微镜物镜的轴向运动，直至离焦量满足设定目标。
在一个实施例中，所述对焦控制模块是电机控制器，通过比例-积分-微分控制器确定显微镜物镜的调整反馈量。
在一个实施例中，通过以下公式计算显微镜物镜的调整反馈量：
F(t)＝K P·δ(t)+K I·∫δ(t)+K D·δ′(t)，其中K P、K I、K D表示对应项的系数，δ(t)是实时计算的当前视场下显微系统的离焦量。
在一个实施例中，所述红外光源模块使用波长范围650nm至1100nm、功率小于5mW的近红外或红外波长的激光。
在一个实施例中，所述分光器件是分光镜、D形反射镜或具有刀口的 反射镜。
与现有技术相比，本发明的优点在于，所提出的自动对焦系统运行过程与显微成像系统对成像焦平面的选择无关，通过对显微系统即时的焦点状态进行亚像素级精度检测，计算焦点漂移的离焦量，动态地反馈控制物镜调整至最佳对焦点。本发明具备实时性、高稳定性高、高精度以及低成本等优势。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的用于显微镜的实时自动对焦系统的示意图；
图2是根据本发明一个实施例的实时自动对焦原理示意图；
图3是根据本发明一个实施例的离焦量检测方法的流程图；
图4是根据本发明一个实施例的自动对焦效果测试实验的数据图。
附图中，1-激光器；2-分光镜；6-第一反射镜；7-偏移透镜；8-第二反射镜；3-聚焦透镜；4-柱面镜；5-成像相机；9-显微镜内短通滤光片；10-可电动移动控制的物镜；11-样品反射界面；12-控制系统；13-电机控制器。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨 论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本发明提供一种用于显微镜的实时高精度自动对焦系统。简言之，该系统采用近红外激光光束通过显微镜的物镜照射到装载生物样品玻片的界面上形成激光光斑，激光光斑经过界面的反射经由同一物镜收集，成像于自动对焦系统的感光芯片。焦点的漂移将带来感光芯片上反射光斑位置的相应移动，通过实时地分析光斑移动量并计算得到系统的焦点漂移量，进而控制显微镜物镜进行漂移量修正，完成实时自动对焦过程。
参见图1所示，该系统涉及多个模块化的子系统，具体包括红外光源模块、偏移透镜光路模块、检测模块和对焦控制模块。
红外光源模块包括激光器1和分光镜2。例如，激光器1产生近红外或红外波长的激光，激光器1的输出端是单模光纤，使用单模光纤耦合激光输出至准直器，以形成基模高斯光束并准直输出发散角小于0.5mrad的平行激光光束。激光光束随后通过衰减片降低功率，通过分光镜2入射至偏移透镜光路模块。在本发明实施例中，可使用近红外或红外波长(如650nm-1100nm)的低功率(小于5mW)激光。
偏移透镜光路模块包括第一反射镜6、偏移透镜7、一维电动平移台(未示出)和第二反射镜8。红外光源模块出射的激光光束入射到第一反射镜6上，反射至偏移透镜7。偏移透镜7安装固定在一维电动平移台上，可以沿光轴调整其位置。激光光束经过偏移透镜7入射到第二反射镜8上，经显微镜内部短通滤光片9反射入物镜10，最终照射在样品界面11处。在本发明的系统中，经过偏移透镜空间变换后的激光光束为具有一定的发散度的发散光束，在物镜10后焦平面处形成与物镜光阑尺寸接近的光斑(如3mm至4mm)进入物镜10，其入射位置由第一反射镜6和第二反射镜 8调节至近物镜光阑边缘，使物镜10会聚的激光光束与物镜光轴成一定倾角照射在样品界面11处；入射激光与物镜光轴由第一反射镜6和第二反射镜8调节成一定倾角，使样品界面11处会聚的激光光斑形成较为明显地彗星形像差。反射的激光光斑经由同一路径返回，后经红外光源模块中的分光镜2反射后进入检测模块。
总之，本发明提供的偏移透镜光路模块具有以下特点：偏移透镜固定于一维电动平移台上，可以沿光轴调整其位置，使激光光束的变换可以满足对不同物镜的选择；偏移透镜将平行的激光光束变换为具有一定发散度的发散光束，使经过物镜光束的焦点远离物镜成像的焦平面；发散光束在物镜后焦平面处形成直径与物镜光阑尺寸接近的激光光束，使照射在样品反射界面处的激光光斑形成较为明显的衍射光斑；入射物镜的激光光束光轴与物镜光轴不重合，靠近物镜光阑边缘，使经过物镜会聚的光束与物镜光轴形成一定倾角，倾斜地照射在样品反射界面处；入射物镜的激光光束光轴与物镜光轴具有一定夹角，使照射在样品反射界面处的激光光斑形成较为明显的彗星形像差。
检测模块从光路配置角度包括聚焦透镜3、柱面镜4和成像相机5。反射的激光光斑经过光路系统中的分光镜2后由聚焦透镜3和柱面镜4聚焦在成像相机5的感光芯片上。其中柱面镜4将圆形光斑空间变换为椭圆形光斑，经过适当的选取焦距，可以形成近似线型的光斑。成像光斑由分光镜2调节至相机5的感光芯片的中心。
检测模块还包括控制系统12，在本文中，以计算机控制系统为例进行介绍。具体地，成像相机5与计算机通讯，其感光芯片采集的激光光斑图像由计算机控制系统12进行算法检测和分析，计算得到线型光斑的亚像素级精度坐标，由系数变换得到焦点漂移的离焦量。
对焦控制模块(以电机控制器13为例)用于接收控制系统12对于离焦量分析的反馈并控制物镜运动，例如可通过比例-积分-微分控制器(PID)实现焦点漂移的校正。
结合图1所示，本发明所提供系统的自动对焦过程包括以下步骤：
步骤S210，红外光源模块发射平行激光光束，由偏移透镜光路模块扩 束为发散光，通过显微镜物镜照射样品界面，界面反射形成的彗差形衍射光斑由检测模块进一步地空间调制为近似平行线型光斑，并进行图像采集，获得光斑图像；
步骤S220，计算机控制系统对线型光斑图像运行图像检测算法，计算当前视场下显微系统的离焦量；
步骤S230，电机控制器补偿离焦量；
步骤S240，重复步骤S210至S230，直至当前视场下的离焦量小于设定阈值，自动对焦完成。
为清楚起见，下文将结合图1和图2阐述本发明提供的实时对焦系统的原理。
参见图2(A)所示，本发明借助近红外或红外激光在玻片界面处的反射光斑对显微系统的离焦状态进行分析判定。在高数值孔径物镜的应用中，物镜与玻片之间的浸渍介质通常为油或水，以提高分辨率。此时由于折射率的相近，对激光反射较强的界面为玻片与生物样品间的折射率差较大的交界面。物镜的焦平面(即物平面)在远离此反射界面的样品区域中，其与反射界面的距离为焦点偏移距离，调节此焦点偏移距离值可以实现对具有立体结构的生物样品不同深度的成像观察。在显微镜成像过程中，当选定了特定成像焦平面后，即确定了焦点偏移距离。
一束具有很小发散角的激光可视作平行光，其经过物镜的会聚作用将聚焦于物镜焦平面处。在物镜焦平面聚焦的激光光斑由于尺寸极小因而具有极高的能量密度，会对生物组织样品形成局部的热效应。本发明通过在光路中加入偏移透镜，使平行的激光经过偏移透镜形成发散光束，以一定的发散度入射到物镜中。具有一定发散度的激光经过物镜的会聚作用，会在远离物镜处形成与物镜焦平面具有一定距离的激光聚焦面，此激光聚焦面与物镜焦平面的距离取决于入射激光的发散度。此时在激光聚焦面上具有高能量密度的激光光斑由于远离生物样品，因而将不再对生物样品造成影响，在长期的显微生物实验中实现对生物样品的保护。
在本发明提供的实时对焦系统配置下，玻片与样品交接的反射界面将形成一个较大的激光光斑。反射界面对激光光斑具有较强的反射作用，因 此能量较低的激光(如小于1mW)即可实现该技术方案。在对焦过程中，当前反射界面处的非聚焦激光光斑的空间位置状态均将作为显微成像系统焦点漂移的依据，而不依赖于成像平面与反射界面距离(即焦点偏移距离)的变化。进一步地，通过调整偏移透镜在光路中相对物镜的距离，使入射到物镜的激光光束达到与物镜孔径近似的尺寸，对聚焦的激光光斑形成较为显著的衍射效应。在衍射效应的影响下，反射界面处较大的激光光斑具有圆孔衍射结构，即同心圆环状的光斑。随着物镜与反射界面距离的变化，此激光光斑大小会随之细微变化。偏移透镜固定在一维电动平台上，可以根据物镜的规格(如放大倍数、数值孔径等)进行相应的距离调整。
参见图2(B)所示，入射到物镜的激光光轴偏离物镜光轴一定距离，经过物镜会聚作用形成与物镜光轴夹角θ入射到反射界面上。同时激光光轴与物镜光轴形成一定的夹角，激光光束形成倾斜入射状态，使反射界面处的激光光斑远离成像视野中心，避开对生物样品的直接照射。倾斜入射一方面能够避免垂直入射情况下，由于显微系统光路中的光学元件间的多重反射导致的光斑鬼影，从而改善对焦效果；另一方面使反射界面上的激光光斑形成彗星形像差，形成非对称结构的偏心圆环状衍射光斑，能量集中于衍射光斑的一侧。光斑的重心两侧分别为圆环结构和本底噪声。本发明利用衍射光斑的非对称性判断显微系统的离焦方向。
对于本发明实施例提供的光路配置，在一定范围内，位于距离物镜不同的反射界面上的激光光斑的大小随着物镜远离反射界面将线性变小，其衍射结构在平面上的位置亦将沿一维方向单调移动。当显微系统由于多种原因导致焦点漂移时，物镜与反射界面之间的距离将发生变化。当物镜远离反射界面，距离变化量为+δ时，此时激光光斑的中心将在水平方向平移+dx量；当物镜靠近反射界面，距离变化量为-δ时，此时激光光斑的中心将在水平方向平移-dx量。
反射界面处的具有彗星形相差衍射结构的激光光斑经过物镜，聚焦于成像相机的感光芯片上。在检测模块中，本发明采用柱面镜对光斑进行一维拉伸。具体地，该柱面镜在与光斑在平面上移动方向的垂直方向上进行拉伸，使衍射结构的激光光斑在成像感光芯片上形成一系列平行的线型光 斑。彗星形相差效果则使能量最集中的线型光斑位于平行线的一侧。焦点漂移的离焦量反映在平行线沿其正交方向上的移动，漂移的方向反映是能量最集中的线型光斑的移动方向。离焦量与线型光斑移动距离的对应关系为δ＝K·dx，其中变换系数K通过对系统轴向扫描校正曲线的拟合得到。
进一步地，成像相机与计算机通讯，实时的传输图像数据并对平行的线型激光光斑应用亚像素级精度的位移检测算法。结合图3所示，在一个实施例中，基于位移的离焦量检测方法包括以下步骤：
步骤S310，采集包含平行线型光斑的原始图像，例如，MXN像素大小的图像，表示为I，包含垂直方向的平行线型光斑，其中，M和N可根据原始图像的分辨率和图像尺寸等选取适当值；
步骤S320，对原始图像沿垂直于线型光斑累加，得到1XN的图像强度曲线，表示为Profile；
步骤S330，从曲线Profile中检测信号值最高的线型激光光斑的峰值位置，表示为PeakLoc；
步骤S340，以[PeakLoc，M/2]为中心截取64X64像素大小的图像子窗口，表示为ISub；
步骤S350，使用基于OTSU(最大类间方差法)的阈值算法对原始图像计算二值化分割阈值T；
步骤S360，使用阈值T对图像子窗口进行二值化，得到二值化的图像，表示为BSub；
步骤S370，通过形态学运算提取BSub中心的线型光斑，并结合ISub计算得到亚像素级精度的重心坐标，表示为LocX；
步骤S380，将光斑坐标LocX与采用步骤S310至S370计算得到的焦平面设定值坐标ZeroX比较，得到光斑移动量，表示为dx；
步骤S390，通过焦点漂移量与光斑移动距离对应关系δ＝K·dx的系数变换，得到系统离焦量δ。
当显微系统开始成像时，首先确定(例如人工确定)物镜成像焦平面的位置，此时激光光斑在成像相机中的位置被控制系统自动设定为焦点位置(Z＝0)。控制系统应用检测算法对此时的光斑图像进行上述步骤S310 至S370的分析，得到显微系统焦点位置参考值ZeroX。当发生焦点漂移时，控制系统应用完整的检测算法检测到激光光斑的位移量dx，并计算得到离焦量δ，通过PID算法计算出物镜调整反馈量F(t)＝K P·δ(t)+K I·∫δ(t)+K D·δ′(t)(K P、K I、K D是对应项的系数)并反馈至物镜轴向运动控制电机，补偿焦点漂移。在显微系统的离焦量低于预设阈值前，迭代反馈。
需要说明的是，上述实施例仅是示意性，在不违背本发明精神的前提下，本领域技术人员可进行适当的改变或变型，例如，分光镜可替换为D形反射镜或具有刀口的反射镜，在该替代方案下，激光器的功率可以适当降低；偏移透镜可替换为可变焦距透镜，如电致变形驱动的柔性变焦透镜等；成像相机可由线性阵列相机替代；反馈PID算法可替换为其他具有快速调整和稳定性的算法。
为验证本发明的可行性和效果，进行了以下实验。
(1)离焦量与线型光斑移动距离的实验
实验测定离焦量与线型光斑移动距离对应关系δ＝K·dx中的变换系数K。实验中，沿物镜光轴方向以10nm精度的固定步长扫描物镜，在每个位置点对成像相机采集的线型光斑进行算法检测，得到其空间位置(单位为像素坐标)。扫描范围为远超物镜景深范围的±4μm，得到像素坐标-物镜轴向坐标关系的数据，并进行线性拟合。拟合结果中R平方因子达到99.5％，得到自动对焦系统的变换系数K为240纳米/像素，即每检测到1像素的位移，线性对应显微成像系统的离焦量约为240nm。
(2)自动对焦效果测试的实验
实验中，使用60X的油浸物镜，其数值孔径为1.40，景深约为542nm。样品玻片装置在恒温箱内，恒温箱和物镜均持续加热至37℃并维持4小时，实现正常生物活体观测实验条件。实验测试时间为30分钟，分别测试关闭自动对焦和开启自动对焦后，显微成像系统焦点的漂移状况。未开启自动对焦的情况下，成像系统在几分钟之内即失焦，漂移出物镜的景深范围，在整个实验过程中，其漂移量的标准差(STD)达到578.2nm；在开启自动对焦的情况下，成像系统持续性的维持对焦状态，其漂移量的标准差为4.7nm，分布范围为±14.1nm，对焦重复精度可达到物镜景深的5.2％，实 现高精度的自动对焦效果。实验结果和实验数据参见图4所示，其中纵坐标表示轴向位置(Axial position)，横坐标表示测试时间，Autofocus OFF表示自动对焦关闭(对应上方曲线)，Autofocus ON表示自动对焦开启(对应下方曲线)。
综上所述，针对现有基于硬件的自动对焦方法，激光聚焦平面为样品反射平面，会对生物样品形成局部高能量密度的照射的问题，本发明使用偏移透镜使激光聚焦平面远离样品成像区域，避开局部高能量密度的光照射；针对现有基于硬件的自动对焦方法，激光聚焦平面为物镜焦平面，与样品反射界面(对焦参考平面)有一定的距离，导致对焦结束后需再度移动物镜固定距离，降低对焦精度的问题，本发明借助非聚焦状态下衍射光斑的空间位置状态作为显微成像系统焦点漂移的依据，不依赖物镜焦平面与反射界面距离的变化，提高了稳定性和对焦精度；本发明利用具有彗星形像差的衍射光斑，在亚像素级精度下分析其实时位移量，并通过PID算法迭代调整显微成像系统的实时离焦量，实现高精度自动对焦。
本发明上述实施例的离焦量检测过程可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些 计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种用于显微镜的实时自动对焦系统，包括红外光源模块、偏移透镜光路模块、检测模块和对焦控制模块，其中：
    所述红外光源模块用于发射平行激光光束；
    所述偏移透镜光路模块将所述红外光源模块出射的平行激光光束扩束为发散光，并通过显微镜物镜照射样品界面，以使照射在样品反射界面处的激光衍射光斑形成彗星形像差；
    所述检测模块将样品界面反射形成的彗差形衍射光斑调制为线型光斑，并通过对该线型光斑分析获得当前视场下显微系统的离焦量相关信息；
    所述对焦控制模块基于获得的离焦量相关信息迭代控制显微镜物镜的轴向运动，直至离焦量满足设定目标。
  2. 根据权利要求1所述的用于显微镜的实时自动对焦系统，其中，所述红外光源模块包括激光器和分光器件，所述激光器用于产生平行激光光束，该平行激光光束依次经准直器和衰减片后，由所述分光器件入射至所述偏移透镜光路模块。
  3. 根据权利要求1所述的用于显微镜的实时自动对焦系统，其中，所述偏移透镜光路模块包括第一反射镜、固定在一维电动平移台上的偏移透镜和第二反射镜，所述第一反射镜接收所述红外光源模块出射的平行激光光束，并反射至所述偏移透镜，经所述偏移透镜入射到所述第二反射镜上，并经显微镜内部滤光片反射入物镜，进而照射样品界面。
  4. 根据权利要求2所述的用于显微镜的实时自动对焦系统，其中，所述检测模块包括聚焦透镜、柱面镜、相机和控制系统，由照射样品界面所反射的激光光斑经所述红外光源模块的分光器件后，由聚焦透镜和柱面镜聚焦在相机的感光芯片上，柱面镜将圆形光斑空间变换为椭圆形光斑，以形成线型光斑，控制系统与相机进行通讯并通过分析该线型光斑的空间位置变化以获得当前视场下显微系统的离焦量相关信息。
  5. 根据权利要求1所述的用于显微镜的实时自动对焦系统，其中，根据以下步骤计算显微系统的离焦量：
    采集包含线型光斑的原始图像；
    对原始图像沿垂直于线型光斑方向像素累加，得到图像强度曲线；
    从图像强度曲线中检测信号值最高的线型激光光斑的峰值位置；
    基于峰值位置截取包含多个像素的图像子窗口；
    对原始图像计算二值化分割阈值；
    使用分割阈值对图像子窗口进行二值化，得到二值化的图像；
    提取二值化图像中心的线型光斑，并结合图像子窗口计算得到亚像素级精度的光斑中心坐标；
    利用获得的光斑中心坐标得到光斑移动量dx；
    通过焦点漂移量与光斑移动距离的对应关系δ＝K·dx得到离焦量δ，其中K是变换系数。
  6. 根据权利要求1所述的用于显微镜的实时自动对焦系统，其中，所述对焦控制模块分别连接所述控制系统和显微镜物镜，用于根据获得的离焦量相关信息与显微镜物镜调整反馈量之间的对应关系迭代控制显微镜物镜的轴向运动，直至离焦量满足设定目标。
  7. 根据权利要求6所述的用于显微镜的实时自动对焦系统，其中，所述对焦控制模块是电机控制器，通过比例-积分-微分控制器确定显微镜物镜的调整反馈量。
  8. 根据权利要求7所述的用于显微镜的实时自动对焦系统，其中，通过以下公式计算显微镜物镜的调整反馈量：
    F(t)＝K P·δ(t)+K I·∫δ(t)+K D·δ′(t)，其中K P、K I、K D表示对应项的系数，δ(t)是实时计算的当前视场下显微系统的离焦量。
  9. 根据权利要求1所述的用于显微镜的实时自动对焦系统，其中，所述红外光源模块使用波长范围650nm至1100nm、功率小于5mW的近红外或红外波长的激光。
  10. 根据权利要求2所述的用于显微镜的实时自动对焦系统，其中，所述分光器件是分光镜、D形反射镜或具有刀口的反射镜。
PCT/CN2020/079547 2020-03-16 2020-03-16 一种用于显微镜的实时自动对焦系统 WO2021184169A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/079547 WO2021184169A1 (zh) 2020-03-16 2020-03-16 一种用于显微镜的实时自动对焦系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/079547 WO2021184169A1 (zh) 2020-03-16 2020-03-16 一种用于显微镜的实时自动对焦系统

Publications (1)

Publication Number Publication Date
WO2021184169A1 true WO2021184169A1 (zh) 2021-09-23

Family

ID=77767968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/079547 WO2021184169A1 (zh) 2020-03-16 2020-03-16 一种用于显微镜的实时自动对焦系统

Country Status (1)

Country Link
WO (1) WO2021184169A1 (zh)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114688963A (zh) * 2022-03-22 2022-07-01 西安交通大学 多波长点衍射干涉仪用光路合束质量检测校准方法及系统
CN114838916A (zh) * 2022-03-31 2022-08-02 歌尔股份有限公司 衍射光波导的测试系统、方法及装置
CN114858060A (zh) * 2022-03-22 2022-08-05 合肥工业大学 基于同轴激光三角法与显微成像的位移传感器及应用
CN114858762A (zh) * 2021-11-01 2022-08-05 郑州思昆生物工程有限公司 一种成像装置对焦方法及成像系统
CN115026414A (zh) * 2022-06-22 2022-09-09 合肥工业大学 一种激光双光子直写加工设备及其自动对焦方法
CN115420691A (zh) * 2022-11-03 2022-12-02 北京云端光科技术有限公司 遥测收发装置的校正系统、方法、装置、设备及存储介质
CN115639777A (zh) * 2022-11-03 2023-01-24 河北工业大学 基于Labview-FPGA的光束稳定系统
CN116540393A (zh) * 2023-07-07 2023-08-04 睿励科学仪器（上海）有限公司 自动对焦系统及方法、半导体缺陷检测系统及方法
CN116754565A (zh) * 2023-08-04 2023-09-15 哈尔滨工业大学 一种光学元件全口径表面微缺陷光致荧光检测用自动对焦检测方法
CN116995986A (zh) * 2023-09-26 2023-11-03 中国科学院空天信息创新研究院 双回路运动镜控制方法、装置及系统
WO2024051198A1 (zh) * 2022-09-05 2024-03-14 苏州精濑光电有限公司 一种基于线扫描的自动聚焦系统、方法及应用
CN118567084A (zh) * 2024-08-01 2024-08-30 浙江荷湖科技有限公司 一种基于视差的光场显微系统自动对焦锁焦方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003077008A2 (en) * 2002-03-13 2003-09-18 Yeda Research And Development Company Ltd. Auto-focusing method and device
JP2005062515A (ja) * 2003-08-13 2005-03-10 Nikon Corp 蛍光顕微鏡
CN102597841A (zh) * 2009-10-29 2012-07-18 应用精密公司 用于光学仪器的连续、异步自动聚焦的系统和方法
CN104932092A (zh) * 2015-06-15 2015-09-23 上海交通大学 基于偏心光束法的自动对焦显微镜及其对焦方法
CN105093479A (zh) * 2014-04-30 2015-11-25 西门子医疗保健诊断公司 用于显微镜的自动对焦方法和装置
CN107407791A (zh) * 2015-03-31 2017-11-28 通用电气公司 用于光学仪器的连续异步自动聚焦的系统和方法
CN108646396A (zh) * 2018-04-27 2018-10-12 合肥工业大学 自动对焦显微镜系统
CN110809113A (zh) * 2019-01-25 2020-02-18 云南师范大学 一种基于fpga的红外成像自动聚焦系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003077008A2 (en) * 2002-03-13 2003-09-18 Yeda Research And Development Company Ltd. Auto-focusing method and device
JP2005062515A (ja) * 2003-08-13 2005-03-10 Nikon Corp 蛍光顕微鏡
CN102597841A (zh) * 2009-10-29 2012-07-18 应用精密公司 用于光学仪器的连续、异步自动聚焦的系统和方法
CN105093479A (zh) * 2014-04-30 2015-11-25 西门子医疗保健诊断公司 用于显微镜的自动对焦方法和装置
CN107407791A (zh) * 2015-03-31 2017-11-28 通用电气公司 用于光学仪器的连续异步自动聚焦的系统和方法
CN104932092A (zh) * 2015-06-15 2015-09-23 上海交通大学 基于偏心光束法的自动对焦显微镜及其对焦方法
CN108646396A (zh) * 2018-04-27 2018-10-12 合肥工业大学 自动对焦显微镜系统
CN110809113A (zh) * 2019-01-25 2020-02-18 云南师范大学 一种基于fpga的红外成像自动聚焦系统

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114858762A (zh) * 2021-11-01 2022-08-05 郑州思昆生物工程有限公司 一种成像装置对焦方法及成像系统
CN114688963B (zh) * 2022-03-22 2023-02-21 西安交通大学 多波长点衍射干涉仪用光路合束质量检测校准方法及系统
CN114858060A (zh) * 2022-03-22 2022-08-05 合肥工业大学 基于同轴激光三角法与显微成像的位移传感器及应用
CN114688963A (zh) * 2022-03-22 2022-07-01 西安交通大学 多波长点衍射干涉仪用光路合束质量检测校准方法及系统
CN114858060B (zh) * 2022-03-22 2023-07-14 合肥工业大学 基于同轴激光三角法与显微成像的位移传感器及应用
CN114838916A (zh) * 2022-03-31 2022-08-02 歌尔股份有限公司 衍射光波导的测试系统、方法及装置
CN115026414A (zh) * 2022-06-22 2022-09-09 合肥工业大学 一种激光双光子直写加工设备及其自动对焦方法
WO2024051198A1 (zh) * 2022-09-05 2024-03-14 苏州精濑光电有限公司 一种基于线扫描的自动聚焦系统、方法及应用
CN115420691B (zh) * 2022-11-03 2023-01-31 北京云端光科技术有限公司 遥测收发装置的校正系统、方法、装置、设备及存储介质
CN115639777A (zh) * 2022-11-03 2023-01-24 河北工业大学 基于Labview-FPGA的光束稳定系统
CN115420691A (zh) * 2022-11-03 2022-12-02 北京云端光科技术有限公司 遥测收发装置的校正系统、方法、装置、设备及存储介质
CN116540393A (zh) * 2023-07-07 2023-08-04 睿励科学仪器（上海）有限公司 自动对焦系统及方法、半导体缺陷检测系统及方法
CN116540393B (zh) * 2023-07-07 2024-01-30 睿励科学仪器（上海）有限公司 自动对焦系统及方法、半导体缺陷检测系统及方法
CN116754565A (zh) * 2023-08-04 2023-09-15 哈尔滨工业大学 一种光学元件全口径表面微缺陷光致荧光检测用自动对焦检测方法
CN116754565B (zh) * 2023-08-04 2024-04-26 哈尔滨工业大学 一种光学元件全口径表面微缺陷光致荧光检测用自动对焦检测方法
CN116995986A (zh) * 2023-09-26 2023-11-03 中国科学院空天信息创新研究院 双回路运动镜控制方法、装置及系统
CN116995986B (zh) * 2023-09-26 2024-02-06 中国科学院空天信息创新研究院 双回路运动镜控制方法、装置及系统
CN118567084A (zh) * 2024-08-01 2024-08-30 浙江荷湖科技有限公司 一种基于视差的光场显微系统自动对焦锁焦方法

Similar Documents

Publication Publication Date Title
WO2021184169A1 (zh) 一种用于显微镜的实时自动对焦系统
CN112415735A (zh) 一种用于显微镜的实时自动对焦系统
US9832365B2 (en) Autofocus based on differential measurements
CN211627931U (zh) 一种用于显微镜的实时自动对焦系统
US10775598B2 (en) Scanning microscope
JP7234254B2 (ja) 顕微鏡において光路を操作する方法および装置、顕微鏡においてスタック画像を撮影する方法
JP2018151624A5 (zh)
US20140071438A1 (en) Method and apparatus for image scanning
US11215814B2 (en) Detection of optical surface of patient interface for ophthalmic laser applications using a non-confocal configuration
JP6158183B2 (ja) 複数の較正曲線を使用した光学機器の連続した非同期的オートフォーカスのためのシステム及び方法
JP2016533525A (ja) スキャニング顕微鏡、およびスキャニング顕微鏡の点像分布関数（ｐｓｆ）を決定する方法
JP2008146060A (ja) オートフォーカス装置及びその方法
JP2018516380A (ja) 光学機器の連続非同期オートフォーカスのためのシステムおよび方法
KR101505745B1 (ko) 이중 검출 반사 공초점 현미경 및 이를 사용하는 시편의 높이의 정보를 검출하는 방법
CN116540393B (zh) 自动对焦系统及方法、半导体缺陷检测系统及方法
JP4974060B2 (ja) 創薬スクリーニング方法
EP3129817B1 (en) Autofocus system
Wang et al. High-robustness autofocusing method in the microscope with laser-based arrayed spots
CN112748562B (zh) 显微镜和用于确定显微镜中的像差的方法
US20240151937A1 (en) Systems and methods for autofocus
US20240319487A1 (en) Microscope and method for autofocusing
KR20240051022A (ko) 디지털 홀로그래픽 현미경을 제어하는 방법 및 장치
JP2004226771A (ja) 全反射蛍光顕微測定装置
Chen et al. Investigation of an autofocusing method for visible aerial cameras based on image processing techniques
CN116819892A (zh) 一种光学量测装置及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20925050

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20925050

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20925050

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04/07/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20925050

Country of ref document: EP

Kind code of ref document: A1