CN113589506A - Biological microscopic vision pre-focusing device and method based on spectrum confocal principle - Google Patents
Biological microscopic vision pre-focusing device and method based on spectrum confocal principle Download PDFInfo
- Publication number
- CN113589506A CN113589506A CN202110896664.0A CN202110896664A CN113589506A CN 113589506 A CN113589506 A CN 113589506A CN 202110896664 A CN202110896664 A CN 202110896664A CN 113589506 A CN113589506 A CN 113589506A
- Authority
- CN
- China
- Prior art keywords
- focusing
- light
- spectrum
- biological
- lens
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/006—Optical details of the image generation focusing arrangements; selection of the plane to be imaged
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0036—Scanning details, e.g. scanning stages
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/24—Base structure
- G02B21/241—Devices for focusing
- G02B21/244—Devices for focusing using image analysis techniques
Abstract
The invention discloses a biological microscopic vision pre-focusing device and method based on a spectrum confocal principle. The device comprises a spectrum confocal sensing module, a micro-optical module, a slide clamping device, a focusing movement device and a scanning movement device. The spectrum confocal sensor measures the distance to the upper surface/lower surface of the cover glass and the thickness of a biological tissue layer to be observed at one time; the spatial relationship between the object focus of the microscope objective and the measured value of the spectral confocal sensor is determined through calibration, and then the measured value of the spectral confocal sensor directly drives the focusing movement device to realize the accurate automatic focusing of biological microscopic vision. The invention also provides a digital slice scanner based on the device and the method. The method is suitable for the construction of the focusing topographic map of the digital section scanner, can also be used for the automatic focusing of the biological digital microscope, and has the characteristics of high focusing speed, accurate layered focus prediction and the like.
Description
Technical Field
The invention belongs to the field of automatic focusing, and particularly relates to an automatic focusing device and method of a biological microscopic vision system based on a spectrum confocal principle. The invention can be used for biological microscopic camera shooting-based equipment and instruments, and is particularly suitable for quick pre-focusing of a digital section scanner.
Background
The digital slice scanner scans and collects the traditional pathological glass slices by using a digital camera system to obtain a high-resolution digital Image for displaying the cell tissue condition of a patient, automatically carries out high-precision multi-view seamless splicing and processing on the obtained Image, and obtains a high-quality full-slice digital Image (WSI). By sharing the WSI data, remote pathological diagnosis and interpretation communication can be realized, and the sharing and utilization of high-quality medical expert resources are facilitated.
The imaging quality of digital slice scanners relies on high precision focus control, and therefore the autofocus technique of the microscopic vision system is of paramount importance. In order to improve efficiency, the conventional digital slice scanner selects some feature points on the slide, focuses the feature points to obtain the focal position, and then fits the feature focal points to a focal plane, which is also called a "focal topographic map". This process is called "pre-focusing". An autofocus microscope based on a liftable automatic stage is disclosed in the ZL200820169109.8 patent, but rapid high frequency response focusing cannot be achieved due to the high weight of the stage. Focusing was achieved by moving the image splitter lens and the microscope objective lens in patent application 03136023.8 filed by olympus optics, japan. An automatic focusing device based on a mobile object stage and a slice scanning device based on the automatic focusing device are disclosed in a patent of 'a digital slice real-time scanning automatic focusing tracking method' (ZL 201310549338.8) filed by Miaodi group of industries and Co., Ltd and a patent application 201410008180.8. A tissue slice scanning technique with a moving microscope objective is disclosed in patent 201410767713.0 entitled "method for scanning pathological section tissue based on fast and accurate focusing of an image acquisition device" filed by ningbo jiang feng bioinformatics technologies, ltd, and patent application 201610706704.X "a tissue slice scanning device and a tissue slice scanning method".
In the above techniques, feature points are focused by an image pickup device, and a "focused topographic map" is fitted. These techniques suffer from two significant drawbacks: firstly, the characteristic points are not properly selected, the phenomenon of imaging blurring of a large area is easy to occur, and the possibility of influencing diagnosis exists; secondly, the process of searching the focus needs to drive the focusing movement device to move, synchronously acquire images and calculate the position of the focus, and occupies a large amount of time. The first disadvantage is complex in cause, and focusing errors can be caused by insufficient details of the under-lens view under the characteristic points or the existence of foreign matters in non-tissue areas, which is difficult to avoid; the second disadvantage is that the pre-focus takes almost one third of the total scan time. Therefore, developing a new pre-focusing technique is necessary for further development of digital slice scanners.
Disclosure of Invention
The invention aims to provide a faster and more accurate biological microscopic vision pre-focusing technology. The invention is based on the spectrum confocal principle, realizes the rapid determination of the focus position by measuring the thickness of the cover glass and the tissue layer, has the characteristics of high focusing speed, high focusing precision and the like, and is particularly suitable for the focusing application of a digital pathological section scanning system.
In order to achieve the purpose, the invention adopts the following technical scheme to solve the problem:
a biological microscopic vision pre-focusing device based on a spectral confocal principle comprises a spectral confocal sensing module, a microscopic optical module, a slide clamping device, a focusing movement device and a scanning movement device, wherein:
the spectrum confocal sensing module consists of a dispersion lens, a wide spectrum light source, an optical fiber light path, a spectrum measurement module and a control system; the microscopic optical module consists of a microscope objective, an imaging lens, a multiplying power conversion lens, a condenser lens, a light source and a digital camera device; the slide clamping device is used for fixing the standard biological slide and ensuring that the standard biological slide does not slide when moving horizontally; the focusing motion device is used for driving the slide clamping device and the microscope objective to move mutually; and the scanning motion device is used for driving the slide clamping device to move in an X-Y plane so as to complete the scanning of the standard biological slide.
The measuring range of the spectrum confocal sensing module is preferably between 0.1mm and 1.5mm, and most preferably 0.4 mm; the longitudinal resolution of the spectrum confocal sensing module is less than 0.2 μm, and is preferably 25 nm; the dispersion lens is used for respectively focusing light with different wavelengths to realize axial dispersion of polychromatic light; the wide-spectrum light source consists of a white light LED and a collimating lens and generates multi-wavelength composite light; the optical fiber optical path consists of a Y-shaped optical fiber, a light guide optical fiber and an optical fiber connector; the spectrum measurement module comprises a first lens, a grating light splitter, a second lens and a light detection sensor;
the Y-shaped optical fiber is provided with three ports, namely an optical inlet, an optical outlet, a beam merging port and the like to form a small-hole confocal structure, the optical inlet of the Y-shaped optical fiber is connected to the wide-spectrum light source, the optical outlet of the Y-shaped optical fiber is connected to the optical inlet of the spectrum measuring device, and the beam merging port of the Y-shaped optical fiber is connected to one end of the optical fiber connector; the other end of the optical fiber connector is connected with a light guide optical fiber, and the other end of the light guide optical fiber is connected with the dispersion lens;
white light emitted by the broad spectrum light source sequentially passes through the light inlet of the Y-shaped optical fiber, the optical fiber connector and the light guide optical fiber, enters the dispersion lens, is divided into lights with different wavelengths, and is respectively focused at different axial positions; light focused on the upper surface of the cover glass, the lower surface of the cover glass, the upper surface of the tissue and the lower surface of the tissue is reflected to enter the dispersion lens, sequentially passes through the light guide optical fiber, the optical fiber connector and the beam combining port of the Y-shaped optical fiber and then enters the light inlet of the spectrum measuring device from the light outlet of the Y-shaped optical fiber; the light entering the spectrum measuring device is collimated by the first lens and then enters the grating light splitter; the grating light splitter reflects incident light at different angles according to different wavelengths, and the incident light forms a dispersed light beam through the second lens and finally enters the optical detection sensor; the light detection sensor is a linear array detector, and can adopt a CMOS (complementary metal oxide semiconductor) or CCD (charge coupled device) image sensor;
the control system consists of a logic control circuit, a light intensity driving and controlling circuit, a digital signal processor and the like; the logic control circuit controls the light detection sensor to acquire spectral information; the light intensity driving and controlling circuit controls the wide-spectrum light source to work and adjusts the luminous intensity of the wide-spectrum light source; and the digital signal processor converts the spectral information into the distance between the upper surface and the lower surface of the cover glass and the thickness information of the biological tissue layer to be observed.
A biological microscopic vision pre-focusing method based on the spectral confocal principle, which is realized by the biological microscopic vision pre-focusing device based on the spectral confocal principle as claimed in claim 1, comprising the following steps:
step 2, obtaining an overall preview image of the standard biological slide, and generating a plurality of characteristic points on the preview image;
step 3, driving the biological slide to be detected to complete scanning movement under the dispersive lens by the scanning movement device, and recording the measured value of the spectrum confocal sensing module and the corresponding X-Y plane position coordinate;
step 4, converting the measured value of the spectrum confocal sensing module into a focusing position f (x) of the focusing movement device in the axial direction of the microscope objective by using the space conversion relation determined in the step 10,y0) The position coordinates P (X) of the X-Y plane recorded in the step 20,y0) Converting into X-Y plane position coordinate P (X, Y) in micro-optical shooting, and converting f (X)0,y0) Corresponding to P (x, y) one by one to obtain a focusing position set f (x, y);
step 5, fitting a complete focusing topographic Map Z-Map (x, y) of the biological slide to be detected by a Kriging interpolation method according to the focusing position set f (x, y);
and 6, synchronously controlling the positions of the scanning motion device and the focusing motion device according to the focusing topographic Map Z-Map (x, y), completing scanning, and shooting the under-lens field image of the microscope objective by the digital camera device.
In step 1 of the biological microscopic vision pre-focusing method based on the spectral confocal principle, the calibration method for establishing the spatial conversion relationship between the object focus of the microscope objective and the measured value of the spectral confocal sensing module comprises the following steps:
step 1-1, acquiring a whole preview image of a biological slice for calibration, and presetting N characteristic mark points;
step 1-2, adjusting the relative distance between the dispersion lens and the calibration biological slide to ensure that two spectral wave crests of return light of the upper surface and the lower surface of a cover glass of the calibration biological slide are within a measuring range at the positions of all characteristic mark points;
step 1-3, driving the biological slide for calibration by a scanning movement device to enable each characteristic mark point to be in a light spot area of the dispersion lens one by one; recording a first spectrum peak p corresponding to each characteristic mark point by a spectrum confocal sensing module1Second spectral peak p2The third spectral peak p3Fourth spectral peak p4Recording the nth feature mark point data as L corresponding to the wavelengthn(p1,p2,p3,p4)[x0,y0]；
Step 1-4, driving a calibration biological slide by a scanning movement device to enable each characteristic mark point to be positioned below the visual field of a microscope objective one by one; controlling a focusing movement device to perform stepping movement at each mark point, so that the distance between the microscope objective and the calibration biological slide is changed from large to small; the number of steps is M, the image under the microscope objective lens of each step position z is obtained through a digital camera device, the sharpness of the corresponding image is calculated, and the nth characteristic mark point data is recorded as Sn(z1,z2,z3,…zM)[x,y]；
1-5, setting the X-Y plane position coordinates [ X, Y ] of each characteristic mark point in the steps 3 and 4]And [ x ]0,y0]One-to-one correspondence, calculating [ x, y ] by a robust algorithm based on RANSAC]And [ x ]0,y0]Homography matrix (Homography matrix) H, H is a 3 x 3 matrix, then
Step 1-6, focusing image sharpness data S of each characteristic mark pointn(z1,z2,z3,…zM) Analyzing to find two peak P-S with maximum valuen(zm) And P-Sn(zk) And z ism<zk(ii) a Making the wavelength L of the first spectral peakn(p1) And zmOne-to-one correspondence, fitting to determine formula z using least squaresm＝a Ln(p1) The parameters a and b to be calibrated in the + b; the wavelengths corresponding to the second, third and fourth spectral peaks are made to be zkOne-to-one correspondence, fitting to determine a formula using least squares
zk＝c[Ln(p2)-Ln(p1)]+d[Ln(p4)-Ln(p3)]+e+zmC, d, e to be calibrated.
The parameters a, b, c, d and e to be calibrated have the following characteristics: a. b, the chromatic dispersion lens or the microscope objective is needed to be calibrated again when the installation of the chromatic dispersion lens or the microscope objective is changed; c. d, e are generally needed to be calibrated again when the material (especially the refractive index) of the biological slide is changed; when the measuring range of the spectrum confocal sensing module is more than 20 times of the thickness of the cover glass, d can be a constant of 0; from the use point of view, the calibration frequency of a, b is much higher than that of c, d, e.
The invention also provides a digital section scanner based on the device and the method, and the focusing motion device comprises a Z-direction piezoelectric motion platform, a spectrum confocal sensing module, a plane reflecting mirror and a piezoelectric drive controller. The planar reflector is arranged on the movable side of the Z-direction piezoelectric motion platform and is used for reflecting dispersed light emitted by the spectrum confocal sensing module; the stroke of the Z-direction piezoelectric motion platform is smaller than the measuring range of the spectrum confocal sensing module; and the piezoelectric driving controller utilizes the displacement information of the plane reflector acquired by the spectrum confocal sensing module to carry out closed-loop focusing motion control.
Compared with the prior art, the invention has the beneficial effects that:
first, using the principle of spectral confocal, "focal topography" can be acquired quickly by planar scanning. In the process, the visual field under the mirror is not required to be imaged, the focusing device is not required to be driven to move, and the frequency of the acquired data can reach more than 1 kHz. Therefore, the technology can greatly improve the pre-focusing efficiency.
Secondly, the focusing position covering the whole cover glass area can be collected in a short time, the reflection intensity can be reflected through the amplitude of the spectrum peak, and the impurity area can be obviously distinguished. Statistical analysis of a larger number of foci also makes it easier to eliminate erroneous foci. And the position accuracy obtained by the spectral confocal method is an order of magnitude higher than that of the conventional method. Therefore, the method can improve the accuracy of the focusing topographic map and ensure the definition of the image.
Drawings
FIG. 1 is a schematic diagram of the biological microscopic vision pre-focusing device based on the spectrum confocal principle according to the present invention;
FIG. 2 is a schematic diagram of an alternative of a biological microscopic vision pre-focusing device based on the spectrum confocal principle;
FIG. 3 is a flow chart of a method for implementing biological microscopic vision pre-focusing based on the spectral confocal principle according to the present invention;
FIG. 4 is a diagram illustrating the calibration of the spatial transformation relationship between the focus position and the measured value of the spectral confocal sensing module according to the present invention.
Detailed Description
The present invention will be described in detail with reference to the accompanying drawings and examples.
The device comprises a spectrum confocal sensing module, a micro-optical module, a slide clamping device, a focusing movement device and a scanning movement device. The spectrum confocal sensor measures the distance to the upper surface/lower surface of the cover glass and the thickness of a biological tissue layer to be observed at one time; the spatial relationship between the object focus of the microscope objective and the measured value of the spectral confocal sensor is determined through calibration, and then the measured value of the spectral confocal sensor directly drives the focusing movement device to realize the accurate automatic focusing of biological microscopic vision. The invention also provides a digital slice scanner based on the device and the method. The method is suitable for the construction of the focusing topographic map of the digital section scanner, can also be used for the automatic focusing of the biological digital microscope, and has the characteristics of high focusing speed, accurate layered focus prediction and the like.
The embodiment of the invention relates to a novel device and a novel method for realizing biological microscopic vision pre-focusing based on a spectrum confocal principle, which can be used for focusing application of a digital section scanner and also can be used for automatic focusing of a biological digital microscope.
Fig. 1 shows an embodiment of a biological microscopic vision pre-focusing device based on the spectral confocal principle, which includes a spectral confocal sensing module, a microscopic optical module, a slide holding device 114, a focusing movement device 115, and a scanning movement device 116, wherein:
in this embodiment, the spectrum confocal sensing module includes a dispersive lens 101, a wide spectrum light source 102, an optical fiber light path 103, a spectrum measurement module 107, and a control system 106; the microscopic optical module comprises a microscopic objective lens 108, an imaging lens 109, a multiplying power conversion mirror 110, a condenser lens 111, a light source 112 and a digital camera device 113; the slide clamping device 114 is used for fixing the standard biological slide and ensuring that the standard biological slide does not slide when moving horizontally; the focusing motion device 115 is used for driving the slide holding device 114 and the microscope objective 108 to move mutually; the scanning motion device 116 is used to drive the slide holding device 114 to move in the X-Y plane to complete the scanning of the standard biological slide.
In this embodiment, the slide holding device 114 is mounted on the focusing motion device 115, and the focusing motion device 115 is mounted on the scanning motion device 116.
In this embodiment, the range of the spectral confocal sensing module is preferably between 0.1mm and 1.5mm, and most preferably 0.4 mm; the longitudinal resolution of the spectrum confocal sensing module is less than 0.2 μm, and is preferably 25 nm; the dispersion lens 101 is used for focusing light with different wavelengths respectively to realize axial dispersion of polychromatic light; the broad spectrum light source 102 consists of a white light LED and a collimating lens and generates multi-wavelength composite light; the optical fiber light path 103 consists of a Y-shaped optical fiber 103-1, a light guide optical fiber 103-3 and an optical fiber connector 103-2; the spectrum measuring module comprises a first lens 107-1, a grating light splitter 107-2, a second lens 107-3 and a light detection sensor 107-1.
In this embodiment, the Y-shaped optical fiber 103-1 has three ports, i.e., an optical inlet, an optical outlet, and a beam combining port, and forms a small-aperture confocal structure, wherein the optical inlet is connected to the broad-spectrum light source 102, the optical outlet is connected to the optical inlet of the spectrum measuring device 107, and the beam combining port is connected to one end of the optical fiber connector 103-2; the other end of the optical fiber connector 103-2 is connected with the light guide fiber 103-3, and the other end of the light guide fiber 103-3 is connected with the interface terminal 101-1 of the dispersion lens 101.
In this embodiment, the white light emitted by the broad spectrum light source 102 sequentially passes through the light inlet of the Y-shaped optical fiber 103-1, the optical fiber connector 103-2, and the light guide fiber 103-3, enters the dispersion lens 101, is divided into lights with different wavelengths, and is focused at different axial positions; the light focused on the upper surface of the cover glass, the lower surface of the cover glass, the upper surface of the tissue and the lower surface of the tissue is reflected to enter the dispersion lens 101, sequentially passes through the light guide fiber 103-3, the fiber connector 103-2 and the beam merging port of the Y-shaped fiber 103-1, and then enters the light inlet of the spectrum measuring device 107 through the light outlet of the Y-shaped fiber 103-1; the light entering the spectrum measuring device 107 is collimated by the first lens 107-1 and then enters the grating beam splitter 107-2; the grating light splitter 107-2 reflects incident light at different angles according to different wavelengths, and forms a dispersed light beam through the second lens 107-3, and finally the dispersed light beam is incident on the light detection sensor 107-4; the light detecting sensor 107-4 is a line array type detector, and may employ a CMOS (complementary metal oxide semiconductor) or CCD (charge coupled device) image sensor.
The control system 106 consists of a logic control circuit, a light intensity driving and controlling circuit and a digital signal processor; the logic control circuit controls the light detection sensor 107-1 to acquire spectral information; the light intensity driving and controlling circuit controls the wide-spectrum light source 102 to work and adjusts the light emitting intensity of the wide-spectrum light source; and the digital signal processor converts the spectral information into the distance between the upper surface and the lower surface of the cover glass and the thickness information of the biological tissue layer to be observed.
Fig. 2 shows another embodiment of a biological microscopic vision pre-focusing device based on the spectral confocal principle, which includes a spectral confocal sensing module, a microscopic optical module, a slide holding device 114, a focusing moving device 201 and a scanning moving device 116. In this embodiment, the focus motion device 115 is separate from the scan motion device 116, and the slide holding device 114 is secured to the scan motion device 116 by a height adjustment adapter 202.
As shown in fig. 3, depending on the above-mentioned device, the invention discloses a biological microscopic vision pre-focusing method based on the spectrum confocal principle, which specifically comprises the following steps:
step 1: calibrating and determining a spatial conversion relation between the object focus of the microscope objective 108 and the measured value of the spectral confocal sensing module;
step 2: acquiring an overall preview image of a standard biological slide, and generating a plurality of feature points on the preview image;
and step 3: the scanning motion device 116 drives the biological slide to be detected to complete the scanning motion under the dispersion lens 101, and records the position coordinate P (X) of the X-Y plane corresponding to each feature point0,y0) And a spectral confocal sensing module measurement at the location;
and 4, step 4: converting the measured value of the spectral confocal sensing module into a focusing position f (x) of the focusing movement device 115 in the axial direction of the microscope objective 108 through the spatial conversion relation determined in the step 10,y0) The position coordinates P (X) of the X-Y plane recorded in the step 30,y0) Converting into X-Y plane position coordinate P (X, Y) in micro-optical shooting, and converting f (X)0,y0) Corresponding to P (x, y) one by one to obtain a focusing position set f (x, y);
and 5: fitting a complete focusing topographic Map Z-Map (x, y) of the biological slide to be detected by a Kriging (Kriging) interpolation method according to the focusing position set f (x, y);
step 6: according to the focusing topographic Map Z-Map (x, y), the positions of the scanning movement device 116 and the focusing movement device 115 are synchronously controlled, the scanning is completed, and the under-lens field image of the microscope objective lens is shot through the digital camera device 113.
As shown in fig. 4, in step 1 of the biological microscopic vision pre-focusing method based on the spectral confocal principle, the calibration method for establishing the spatial transformation relationship between the object focus of the microscope objective 108 and the measured value of the spectral confocal sensing module has the following steps:
step 1-1, acquiring a whole preview image 401 of a biological slice for calibration, and presetting N characteristic mark points 402;
step 1-2, adjusting the relative distance between the dispersion lens 101 and the calibration biological slide, so that two spectral peaks 408 and 409 of return light of the upper surface and the lower surface of the cover glass of the calibration biological slide are within the range of the positions of all characteristic mark points;
step 1-3, driving the calibration biological slide by the scanning movement device 116 to make each characteristic mark point in the light spot area of the dispersion lens one by one; recording a first spectrum peak p corresponding to each characteristic mark point by a spectrum confocal sensing module1408. Second spectral peak p 2409. Third spectral peak p 3410. Fourth spectral peak p 4411, recording the nth feature mark point data as Ln(p1,p2,p3,p4)[x0,y0]；
Step 1-4, driving the calibration biological slide by the scanning movement device 116 to make each characteristic mark point under the visual field of the microscope objective one by one; at each characteristic mark point, controlling a focusing movement device 115 to perform stepping movement so that the distance between the microscope objective 108 and the biological slide for calibration changes from large to small; the number of steps is M, the image under the microscope objective lens 108 at each step position z is obtained through the digital camera device 113, the sharpness of the corresponding image is calculated, and the nth characteristic mark point data is recorded as Sn(z1,z2,z3,…zM)[x,y]；
Step 1-5, the plane position coordinates [ x, y ] of each characteristic mark point in step 1-3 and step 1-4]And [ x ]0,y0]One-to-one correspondence, calculating [ x, y ] by a robust algorithm based on RANSAC]And [ x ]0,y0]In betweenHomography matrix (Homography matrix) H, which is a 3 × 3 matrix, then:
step 1-6, focusing image sharpness data S of each characteristic mark pointn(z1,z2,z3,…zM) Analyzing to find two peak P-S with maximum valuen(zm) And P-Sn(zk) I.e., 412 and 413 in FIG. 4, and zm<zk(ii) a Making the wavelength L of the first spectral peakn(p1) And zmOne-to-one correspondence, fitting to determine formula z using least squaresm＝a Ln(p1) The parameters a and b to be calibrated in the + b; the wavelengths corresponding to the second, third and fourth spectral peaks are made to be zkOne-to-one correspondence, fitting to determine a formula using least squares
zk＝c[Ln(p2)-Ln(p1)]+d[Ln(p4)-Ln(p3)]+e+zmC, d, e to be calibrated.
The parameters a, b, c, d, e to be calibrated shown as 407 in fig. 4 have the following characteristics:
a. b, the chromatic dispersion lens or the microscope objective is needed to be calibrated again when the installation of the chromatic dispersion lens or the microscope objective is changed; c. d, e are generally needed to be calibrated again when the material (especially the refractive index) of the biological slide is changed; when the measuring range of the spectrum confocal sensing module is more than 20 times of the thickness of the cover glass, d can be a constant of 0; from the use point of view, the calibration frequency of a, b is much higher than that of c, d, e.
The invention also provides a digital section scanner based on the device and the method, and the focusing motion device comprises a Z-direction piezoelectric motion platform, a spectrum confocal sensing module, a plane reflecting mirror and a piezoelectric drive controller. The planar reflector is arranged on the movable side of the Z-direction piezoelectric motion platform and is used for reflecting dispersed light emitted by the spectrum confocal sensing module; the stroke of the Z-direction piezoelectric motion platform is smaller than the measuring range of the spectrum confocal sensing module; and the piezoelectric driving controller utilizes the displacement information of the plane reflector acquired by the spectrum confocal sensing module to carry out closed-loop focusing motion control.
The above-described embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the invention. Various changes and modifications may be made by one of ordinary skill in the pertinent art without departing from the spirit and scope of the present invention. Therefore, the technical scheme obtained by adopting the mode of equivalent replacement or equivalent transformation is within the protection scope of the invention.
Claims (8)
1. A biological microscopic vision pre-focusing device based on a spectral confocal principle is characterized by comprising a spectral confocal sensing module, a microscopic optical module, a slide clamping device, a focusing movement device and a scanning movement device;
the spectrum confocal sensing module comprises a dispersion lens, a wide spectrum light source, an optical fiber light path, a spectrum measurement module and a control system; the microscopic optical module comprises a microscope objective, an imaging lens, a multiplying power conversion lens, a condenser lens, a light source and a digital camera device; the slide clamping device is used for fixing the standard biological slide and ensuring that the standard biological slide does not slide when moving horizontally; the focusing motion device is used for driving the slide clamping device and the microscope objective to move mutually; and the scanning motion device is used for driving the slide clamping device to move in an X-Y plane so as to complete the scanning of the standard biological slide.
2. The biological microscopic vision pre-focusing device based on the spectral confocal principle as claimed in claim 1, wherein the range of the spectral confocal sensing module is between 0.1mm and 1.5 mm; the longitudinal resolution is less than 0.2 μm.
3. The biological micro-vision pre-focusing device based on the spectral confocal principle as claimed in claim 1, wherein the dispersion lens is used for respectively focusing lights with different wavelengths to realize axial dispersion of the polychromatic lights; the wide-spectrum light source consists of a white light LED and a collimating lens and generates multi-wavelength composite light; the optical fiber optical path consists of a Y-shaped optical fiber, a light guide optical fiber and an optical fiber connector; the spectrum measurement module comprises a first lens, a grating light splitter, a second lens and a light detection sensor; the control system consists of a logic control circuit, a light intensity driving and controlling circuit and a digital signal processor; the logic control circuit controls the light detection sensor to acquire spectral information; the light intensity driving and controlling circuit controls the wide-spectrum light source to work and adjusts the luminous intensity of the wide-spectrum light source; and the digital signal processor converts the spectral information into the distance between the upper surface and the lower surface of the cover glass and the thickness information of the biological tissue layer to be observed.
4. The biological microscopic vision pre-focusing device based on the spectral confocal principle as claimed in claim 3, wherein the Y-shaped optical fiber has three ports, i.e. an optical inlet, an optical outlet and a beam combining port, and forms a small-hole confocal structure, the optical inlet is connected to the broad-spectrum light source, the optical outlet is connected to the optical inlet of the spectral measuring device, and the beam combining port is connected to one end of the optical fiber connector; the other end of the optical fiber connector is connected with a light guide optical fiber, and the other end of the light guide optical fiber is connected with the dispersion lens.
5. The biological microscopic vision pre-focusing device based on the spectral confocal principle as claimed in claim 1, 3 or 4, wherein the white light emitted from the broad spectrum light source sequentially passes through the light inlet of the Y-shaped optical fiber, the optical fiber connector and the light guide optical fiber, enters the dispersion lens, is divided into lights with different wavelengths, and is focused at different axial positions respectively; light focused on the upper surface of the cover glass, the lower surface of the cover glass, the upper surface of the tissue and the lower surface of the tissue is reflected to enter the dispersion lens, sequentially passes through the light guide optical fiber, the optical fiber connector and the beam combining port of the Y-shaped optical fiber and then enters the light inlet of the spectrum measuring device from the light outlet of the Y-shaped optical fiber; the light entering the spectrum measuring device is collimated by the first lens and then enters the grating light splitter; the grating light splitter reflects incident light at different angles according to different wavelengths, and the incident light forms a dispersed light beam through the second lens and finally enters the optical detection sensor; the light detection sensor is a linear array detector and can adopt a CMOS or CCD image sensor.
6. A biological microscopic vision pre-focusing method based on a spectral confocal principle is characterized by comprising the following steps:
step 1, calibrating and determining a spatial conversion relation between a microscope objective object focus and a spectrum confocal sensing module measurement value;
step 2, obtaining an overall preview image of the standard biological slide, and generating a plurality of characteristic points on the preview image;
step 3, driving the biological slide to be detected to complete scanning movement under the dispersive lens by the scanning movement device, and recording the measured value of the spectrum confocal sensing module and the corresponding X-Y plane position coordinate;
step 4, converting the measured value of the spectrum confocal sensing module into a focusing position f (x) of the moving device in the axial direction of the microscope objective by using the space conversion relation determined in the step 10,y0) The position coordinates P (X) of the X-Y plane recorded in the step 20,y0) Converting into X-Y plane position coordinate P (X, Y) in micro-optical shooting, and converting f (X)0,y0) Corresponding to P (x, y) one by one to obtain a set f (x, y) of focusing positions;
step 5, fitting a complete focusing topographic Map Z-Map (x, y) of the biological slide to be detected by a Kriging interpolation method according to the focusing position set f (x, y);
and 6, synchronously controlling the positions of the scanning motion device and the scanning motion device according to the focusing topographic Map Z-Map (x, y), finishing scanning, and synchronously shooting the under-lens field image of the microscope objective by the digital camera device.
7. The method as claimed in claim 6, wherein in step 1, the calibration method for establishing the spatial transformation relationship between the objective focus of the microscope and the measurement value of the spectral confocal sensing module comprises the following steps:
step 1-1, acquiring a whole preview image of a biological slice for calibration, and presetting N characteristic mark points;
step 1-2, adjusting the relative distance between the dispersion lens and the calibration biological slide to ensure that two spectral wave crests of return light of the upper surface and the lower surface of a cover glass of the calibration biological slide are within a measuring range at the positions of all characteristic mark points;
step 1-3, driving the biological slide for calibration by a scanning movement device to enable each characteristic mark point to be in a light spot area of the dispersion lens one by one; recording a first spectrum peak p corresponding to each characteristic mark point by a spectrum confocal sensing module1Second spectral peak p2The third spectral peak p3Fourth spectral peak p4Recording the nth feature mark point data as L corresponding to the wavelengthn(p1,p2,p3,p4)[x0,y0]；
Step 1-4, driving a calibration biological slide by a scanning movement device to enable each characteristic mark point to be positioned below the visual field of a microscope objective one by one; controlling a focusing movement device to perform stepping movement at each mark point, so that the distance between the microscope objective and the calibration biological slide is changed from large to small; the number of steps is M, the image under the microscope objective lens of each step position z is obtained through a digital camera device, the sharpness of the corresponding image is calculated, and the nth characteristic mark point data is recorded as Sn(z1,z2,z3,…zM)[x,y]；
Step 1-5, the position coordinates [ X, Y ] of each characteristic mark point in the X-Y plane in step 1-3 and step 1-4]And [ x ]0,y0]One-to-one correspondence, calculating [ x, y ] by a robust algorithm based on RANSAC]And [ x ]0,y0]The homography matrix H, H is a 3 × 3 matrix with:
step 1-6, marking each characteristicFocused image sharpness data S of pointsn(z1,z2,z3,…zM) Analyzing to find two peak P-S with maximum valuen(zm) And P-Sn(zk) And z ism<zk(ii) a Making the wavelength L of the first spectral peakn(p1) And zmOne-to-one correspondence, fitting to determine formula z using least squaresm＝aLn(p1) The parameters a and b to be calibrated in the + b; the wavelengths corresponding to the second, third and fourth spectral peaks are made to be zkAnd correspondingly determining the parameters c, d and e to be calibrated in the formula by using a least square method, wherein the determined formula is as follows:
zk＝c[Ln(p2)-Ln(p1)]+d[Ln(p4)-Ln(p3)]+e+zm
the parameters a, b, c, d and e to be calibrated have the following characteristics: a. b, the chromatic dispersion lens or the microscope objective is needed to be calibrated again when the installation of the chromatic dispersion lens or the microscope objective is changed; c. d, e generally needs to be calibrated again when the material of the biological slide is changed; when the measuring range of the spectrum confocal sensing module is more than 20 times of the thickness of the cover glass, d is a constant of 0; from the use point of view, the calibration frequency of a, b is much higher than that of c, d, e.
8. A digital slice scanner, characterized by: the device comprises a Z-direction piezoelectric motion platform, a spectrum confocal sensing module, a plane reflector and a piezoelectric drive controller;
the planar reflector is arranged on the movable side of the Z-direction piezoelectric motion platform and is used for reflecting dispersed light emitted by the spectrum confocal sensing module;
the stroke of the Z-direction piezoelectric motion platform is smaller than the measuring range of the spectrum confocal sensing module;
and the piezoelectric driving controller utilizes the displacement information of the plane reflector acquired by the spectrum confocal sensing module to carry out closed-loop focusing motion control.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110896664.0A CN113589506B (en) | 2021-08-05 | 2021-08-05 | Biological microscopic vision pre-focusing device and method based on spectrum confocal principle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110896664.0A CN113589506B (en) | 2021-08-05 | 2021-08-05 | Biological microscopic vision pre-focusing device and method based on spectrum confocal principle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113589506A true CN113589506A (en) | 2021-11-02 |
| CN113589506B CN113589506B (en) | 2022-07-29 |
Family
ID=78255438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110896664.0A Active CN113589506B (en) | 2021-08-05 | 2021-08-05 | Biological microscopic vision pre-focusing device and method based on spectrum confocal principle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113589506B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114236799A (en) * | 2021-12-17 | 2022-03-25 | 西安交通大学 | Real-time sample focusing device and method for super-oscillation annular-band confocal imaging system |
| CN115150519A (en) * | 2022-09-05 | 2022-10-04 | 武汉精立电子技术有限公司 | Automatic focusing system and method based on line scanning and application |
| CN117095065A (en) * | 2023-09-18 | 2023-11-21 | 合肥埃科光电科技股份有限公司 | Calibration method, system and equipment for linear spectrum copolymerization Jiao Weiyi sensor |
| CN116754565B (en) * | 2023-08-04 | 2024-04-26 | 哈尔滨工业大学 | Automatic focusing detection method for optical element full-caliber surface micro-defect photo-induced fluorescence detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009186517A (en) * | 2008-02-01 | 2009-08-20 | Nidec-Read Corp | Drive unit for movable part of optical device |
| US20150177149A1 (en) * | 2012-07-18 | 2015-06-25 | The Trustees Of Princeton University | Multiscale spectral nanoscopy |
| CN106802129A (en) * | 2016-12-30 | 2017-06-06 | 中国科学院光电研究院 | A kind of high resolution and self calibration Spectral Confocal displacement measurement system |
| CN109781015A (en) * | 2019-01-03 | 2019-05-21 | 西安交通大学 | A kind of method of Spectral Confocal line scanning rapid survey body surface step |
-
2021
- 2021-08-05 CN CN202110896664.0A patent/CN113589506B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009186517A (en) * | 2008-02-01 | 2009-08-20 | Nidec-Read Corp | Drive unit for movable part of optical device |
| US20150177149A1 (en) * | 2012-07-18 | 2015-06-25 | The Trustees Of Princeton University | Multiscale spectral nanoscopy |
| CN106802129A (en) * | 2016-12-30 | 2017-06-06 | 中国科学院光电研究院 | A kind of high resolution and self calibration Spectral Confocal displacement measurement system |
| CN109781015A (en) * | 2019-01-03 | 2019-05-21 | 西安交通大学 | A kind of method of Spectral Confocal line scanning rapid survey body surface step |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114236799A (en) * | 2021-12-17 | 2022-03-25 | 西安交通大学 | Real-time sample focusing device and method for super-oscillation annular-band confocal imaging system |
| CN114236799B (en) * | 2021-12-17 | 2022-09-16 | 西安交通大学 | Real-time sample focusing device and method for super-oscillation annular-band confocal imaging system |
| CN115150519A (en) * | 2022-09-05 | 2022-10-04 | 武汉精立电子技术有限公司 | Automatic focusing system and method based on line scanning and application |
| CN116754565B (en) * | 2023-08-04 | 2024-04-26 | 哈尔滨工业大学 | Automatic focusing detection method for optical element full-caliber surface micro-defect photo-induced fluorescence detection |
| CN117095065A (en) * | 2023-09-18 | 2023-11-21 | 合肥埃科光电科技股份有限公司 | Calibration method, system and equipment for linear spectrum copolymerization Jiao Weiyi sensor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113589506B (en) | 2022-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113589506B (en) | Biological microscopic vision pre-focusing device and method based on spectrum confocal principle | |
| JP5489392B2 (en) | Optical system evaluation apparatus, optical system evaluation method, and optical system evaluation program | |
| US8599372B2 (en) | Linear chromatic confocal microscopic system | |
| JP6899963B2 (en) | Real-time autofocus scanning | |
| JP4883751B2 (en) | 3D shape measuring apparatus and 3D shape measuring method | |
| US11454781B2 (en) | Real-time autofocus focusing algorithm | |
| JP6940696B2 (en) | Two-dimensional and three-dimensional fixed Z-scan | |
| CN110849289A (en) | Double-camera parallel confocal differential microscopic 3D morphology measurement device and method | |
| TW202140995A (en) | Apparatus for surface profile measurement | |
| CN211876977U (en) | Line focusing differential color confocal three-dimensional surface topography measuring system | |
| CN216117309U (en) | Biological microscopic vision pre-focusing device based on spectrum confocal principle | |
| CN114441531B (en) | Automatic focusing method with image recognition, device, computer and storage medium | |
| US11422349B2 (en) | Dual processor image processing | |
| CN112857262A (en) | Improved oblique illumination type color confocal measurement system and detection method | |
| Wu et al. | Auto-focus pathology microscope using sub-array sampling | |
| CN214470666U (en) | Improved oblique illumination type color confocal measuring system | |
| CN219475904U (en) | Imaging device and detection system thereof | |
| JP2004078175A (en) | Confocal microscope | |
| CN116045842A (en) | Differential parallel confocal-based 3D measurement error correction method and system | |
| CN117130144A (en) | Automatic focusing system and focusing method based on optical low coherence interference | |
| JP2003042720A (en) | Height measuring apparatus | |
| CN116819748A (en) | Condensing lens and optical system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |