CN112053304A - Rapid focusing restoration method for single shooting of full-slice digital imaging - Google Patents

Rapid focusing restoration method for single shooting of full-slice digital imaging Download PDF

Info

Publication number
CN112053304A
CN112053304A CN202010934440.XA CN202010934440A CN112053304A CN 112053304 A CN112053304 A CN 112053304A CN 202010934440 A CN202010934440 A CN 202010934440A CN 112053304 A CN112053304 A CN 112053304A
Authority
CN
China
Prior art keywords
focus
quasi
image
full
digital imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010934440.XA
Other languages
Chinese (zh)
Inventor
刘贤明
李强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN202010934440.XA priority Critical patent/CN112053304A/en
Publication of CN112053304A publication Critical patent/CN112053304A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/20Design optimisation, verification or simulation
    • G06F30/27Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Abstract

The invention is based on the biomedical instrument field, particularly relate to a full-section digital imaging single shooting fast accurate focus recovery method, mainly design the virtual auto-focus network of single shooting; the method comprises the following steps that a pathology scanner pre-scans an image block in the middle of a pathology slice view field, an axial three-dimensional image stack is constructed through axial movement, and an initial in-focus image is found through an evaluation function; under the condition of predicting the quasi-focal distance, carrying out single photographing on all sub-images to obtain a tentative out-of-focus image for single photographing as network input; the output of the network is a restored quasi-focus image; through the end-to-end network design of a single image, the method realizes the quick restoration of the single-shot quasi-focus image under full-slice digital imaging, and replaces the traditional self-quasi-focus method of firstly compensating the distance and then shooting. Meanwhile, in the scanning strategy, a quasi-focus measurement diagram and quasi-focus scanning photographing are not required to be constructed, and the method has the advantages of high speed, low cost, offline processing and the like.

Description

Rapid focusing restoration method for single shooting of full-slice digital imaging
Technical Field
The invention relates to a method for fast quasi-focus restoration by single shooting of full-section digital imaging based on the field of biomedical instruments and computational imaging and taking a deep learning technology as a core, and can be widely applied to the research in the fields of microscopic instruments, artificial intelligence, medical images, automation and the like.
Background
In recent years, advanced digital pathology imaging techniques have been widely studied and applied. The full-slice digital imaging technology (WSI, white Slide Images), i.e., virtual microscopy, can acquire traditional microscopic slices in the form of digitized Images, and can realize arbitrary computer access, easy storage, and remote transmission between researchers and doctors, etc. Full-slice digital imaging techniques are of critical importance in biological imaging research, such as in the fields of cancer analysis and disease prediction. The U.S. food and drug administration has currently adopted the full-slice digital imaging system of philips as the primary means of pathological analysis.
Full-slice digital imaging typically requires a two-step implementation: (1) scanning pathological images according to the sequence of the subareas, and then splicing the pathological images together to generate a complete pathological section image of the full view field; (2) purpose-built software is used to identify and analyze these digital images. Wherein the first step is crucial for the quality of the acquired image. At present, the challenge in full-slice digital imaging techniques is mainly how to quickly produce high quality in-focus images. Generally, full-slice scanning requires a high resolution and micron-length depth-of-field objective lens, which results in defocus due to the non-uniform three-dimensional distribution of the sample obtained during scanning of the system. This out-of-focus phenomenon is a major cause of degradation in full-slice digital imaging performance.
At present, a widely used method is to obtain high-quality full-slice digital images by adopting a method of quasi-focal image matching. The quasi-focus image matching method provides a quasi-focus priori, for the out-of-focus image at each position, a series of out-of-focus images (z-stack) at different quasi-focus distances can be obtained by moving the pathological sample along the optical axis, and finally the corresponding quasi-focus image is determined by maximizing the image contrast of each out-of-focus image or other image quality evaluation methods. This approach requires a one-by-one use for each sequentially scanned sub-region. However, this method results in a significant reduction in imaging speed due to repeated axial measurements. For other methods, for example, a double-camera device can be adopted to realize the self-focusing function, so that the axial layer-by-layer scanning of pathological images is avoided. However, this method is not suitable for adding a new imaging module in a conventional microscopy instrument due to problems of hardware incompatibility, high cost and the like. In the current relevant research, the full-slice digital imaging system mainly focuses on high-precision prediction of in-focus distance, and no relevant research currently performs in-focus restoration on out-of-focus images.
Quasi-focal restoration of out-of-focus images can change the scanning strategy of traditional pathological scanners. For a traditional pathological scanning method, firstly, when a scanner scans each image block, an axial image three-dimensional stack needs to be constructed, an in-focus image is found through evaluation functions such as maximized contrast and the like, and the in-focus position is recorded; secondly, for image blocks at other different positions, a block-by-block scanning method is adopted, and each image block executes a process of acquiring a quasi-focus position; then, integrating the quasi-focal distances corresponding to different image blocks of the whole pathological image into a quasi-focal measurement image according to positions; and finally, according to the coordinate position and the quasi-focus distance of the quasi-focus measurement image, moving a microscope objective lens to perform distance compensation and scanning and photographing to obtain a complete pathological digital image. The conventional scanning method inevitably consumes a lot of time cost because of the need to construct a large number of axial three-dimensional image stacks and the necessary axial mechanical displacement. Therefore, a new method is found for restoring the defocused image, a novel scanner scanning strategy is designed, and the method has great scientific research significance and clinical application value.
Disclosure of Invention
In consideration of the limitations of the conventional method, the invention utilizes an advanced machine learning algorithm to solve the problem of quasi-focus restoration of full-slice digital imaging. The invention discloses a full-slice digital imaging single-shot fast focusing restoration method based on deep learning. The method provides a full-slice digital imaging single-shot fast focusing restoration method based on deep learning, and mainly designs a virtual self-focusing network for single-shot; the pathological scanner pre-scans an image block in the middle of a pathological section view field, an axial three-dimensional image stack is constructed through axial movement, an initial quasi-focus image is found through an evaluation function, and a predicted quasi-focus distance of the quasi-focus image is obtained; under the condition of predicting the quasi-focal distance, carrying out single photographing on all sub-images to obtain a tentative out-of-focus image for single photographing as network input; the output of the network is a restored quasi-focus image; through the end-to-end network design of a single tentative out-of-focus image, the method realizes the quick restoration of the single-shot in-focus image under full-slice digital imaging, replaces the traditional self-in-focus method of firstly compensating the distance and then shooting, and performs software virtualization on the traditional full-slice digital pathological imaging hardware. Meanwhile, in the scanning strategy, a quasi-focus measurement diagram and quasi-focus scanning photographing are not required to be constructed, and the method has the advantages of high speed, low cost, offline processing and the like.
The purpose of the invention is realized as follows:
a quick quasi-focus restoration method for single shooting of full-slice digital imaging comprises the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
Further, in the step a, a sub-image position of the pathological section view field center is selected, the objective lens is moved in an axial scanning mode to obtain a z-stack, an axial image stack is constructed, a predicted quasi-focus image can be found by adopting evaluation functions such as contrast and the like, and the corresponding predicted quasi-focus position is recorded.
Further, the position of the selected first prediction quasi-focus image is the photographing position of all the sub-images, and other sub-images are photographed at the photographing position, so that a tentative out-of-focus image for photographing in a single time is obtained.
Further, the input to the network is a single shot tentative through-focus image.
Further, the output of the network is a restored quasi-in-focus image.
Further, after the tentative out-of-focus image for single photographing is processed through the neural network, an in-focus image is directly obtained.
Has the advantages that:
the invention realizes a quick focusing restoration method for single shooting of full-slice digital imaging, which is embodied in the following aspects:
firstly, the invention adopts a single-shot virtual self-focusing network, and realizes the function of restoring the focus-focusing image by carrying out end-to-end network processing on a single tentative out-of-focus image.
Secondly, the invention adopts a deep learning network algorithm, the method adopts a digital network structure to carry out simulation modeling, adopts a software method to realize the virtualization of a full-slice digital imaging system, directly realizes the restoration of a quasi-focus image, replaces the traditional method of firstly predicting the distance and then taking a picture by compensation, and effectively saves the cost of instrument experiment.
Thirdly, in the scanning strategy, the invention does not need to construct a quasi-focus measurement diagram and quasi-focus scanning photographing, and has the advantages of high speed, low cost, offline processing and the like.
Drawings
FIG. 1 is a flow chart of a single-shot fast in-focus restoration method for full-slice digital imaging according to the present invention;
fig. 2 is a schematic diagram of a scanning strategy of the pathology scanning system.
In the figure: microscope objective 1, single shot position 2, scan direction 3.
Detailed Description
The following further illustrates embodiments of the process of the present invention.
Referring to fig. 1 and fig. 2, a fast quasi-focus restoration method for full-slice digital imaging single-shot includes the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
Specifically, in the step a, a sub-image position of the pathological section view field center is selected, a z-stack is obtained through axial scanning movement of an objective lens, a predicted quasi-focus image can be found by adopting evaluation functions such as contrast and the like through constructing an axial image stack, and the corresponding predicted quasi-focus position is recorded.
Specifically, the position of the selected first prediction quasi-focus image is the photographing position of all sub-images, and other sub-images are photographed at the photographing position, so that a tentative out-of-focus image for single photographing is obtained.
Specifically, the input to the network is a single shot tentatively out-of-focus image.
Specifically, the output of the network is a restored quasi-in-focus image.
Specifically, a quasi-focus image is directly obtained after a tentative out-of-focus image for single photographing is processed through a neural network.
The flow chart of the quick focusing restoration method for single photographing of the full-slice digital imaging is known, and the algorithm comprises the following steps: predicting an in-focus image, photographing a tentative out-of-focus image for a single time, photographing a virtual self-in-focus network for a single time, and restoring an in-focus image.
In the training process, the predicted quasi-focus image is obtained through automatic quasi-focus, namely, in the central view field of the pathological section, a plurality of sub-images are continuously moved axially to obtain z-stack, and the predicted quasi-focus image is determined through maximizing contrast or other evaluation functions. (2) The tentative out-of-focus image for single photographing is obtained by photographing all image blocks with the position of the predicted in-focus image as a reference. (3) The single-shot virtual auto-focus network adopts a U-net mode. The input of the network is a tentative out-of-focus image taken once, and a restored in-focus image is finally output through network convolution. The method is only exemplified by U-net, including but not limited to various networks, and the quasi-focus image restoration is realized by using the single-shot tentative out-of-focus image.
In actual operation of the pathological scanning device, the scanning strategy is shown in fig. 2. The microscope objective 1 takes the position of the predicted in-focus image as a reference, and is used as a single photographing position 2, and exposure photographing is carried out to obtain a single photographing tentative out-of-focus image. The above procedure is then repeated for a second sub-image, at this position with x-y mechanical movements in the scanning direction 3.
In this embodiment, a method for fast focus restoration by single-shot full-slice digital imaging includes: the system comprises a prediction in-focus image module, a single-shot tentative out-of-focus image module, a single-shot virtual self-in-focus network module and a restoration in-focus image module. Wherein: the pathological scanner pre-scans an image block in the middle of a pathological section view field, an axial three-dimensional image stack is constructed through axial movement, an initial quasi-focus image is found through an evaluation function, and a predicted quasi-focus distance of the quasi-focus image is obtained; and under the condition of predicting the quasi-focal distance, photographing all the sub-images for a single time to obtain a tentative out-of-focus image for the single-time photographing, wherein the tentative out-of-focus image is used as an input end of a back-end neural network.
In the training process, the single-shot virtual self-focusing network performs network training and feature extraction according to the single-shot tentative out-of-focus image; the restored quasi-focus image is output of the neural network; and performing end-to-end network training through a single tentative out-of-focus image and the corresponding quasi-focus image to obtain a single-shot virtual self-quasi-focus network.
In the testing stage, a predicted quasi-focus sub-image is found at the center of a sample to obtain a corresponding predicted quasi-focus position; then, all other sub-images are photographed at the position to obtain a single photographing tentative out-of-focus image which is used as a test image of the rear-end single photographing virtual self-focusing network; and finally, processing through a single-time photographing virtual auto-focus network to obtain a restored auto-focus image.
In the scanning strategy, the specific process is as follows: firstly, selecting a sample center as an initial position to obtain a predicted in-focus position; secondly, taking the predicted quasi-focal distance as a reference position, and photographing all sub-images at the reference position to obtain a single tentative out-of-focus image; and finally, obtaining a plurality of sub-images of the quasi-focus, and splicing to obtain the pathological full-slice digital image of the quasi-focus.

Claims (6)

1. A quick quasi-focus restoration method for single shooting of full-slice digital imaging is characterized by comprising the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
2. The method for rapid quasi-focal restoration through single-shot full-slice digital imaging according to claim 1, wherein a sub-image position of the center of a pathological section field of view is selected in the step a, a z-stack is obtained through axial scanning movement of an objective lens, and by constructing an axial image stack, a predicted quasi-focal image can be found by using evaluation functions such as contrast and the like, and the corresponding predicted quasi-focal position is recorded.
3. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: and the position of the selected first prediction quasi-focus image is the photographing position of all sub-images, and other sub-images are photographed at the position to obtain a tentative out-of-focus image for single photographing.
4. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: the input to the network is a single shot tentative out-of-focus image.
5. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: the output of the network is a restored quasi-in-focus image.
6. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: and processing the tentative defocused image obtained by single photographing through a neural network to directly obtain a quasi-focus image.
CN202010934440.XA 2020-09-08 2020-09-08 Rapid focusing restoration method for single shooting of full-slice digital imaging Pending CN112053304A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010934440.XA CN112053304A (en) 2020-09-08 2020-09-08 Rapid focusing restoration method for single shooting of full-slice digital imaging

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010934440.XA CN112053304A (en) 2020-09-08 2020-09-08 Rapid focusing restoration method for single shooting of full-slice digital imaging

Publications (1)

Publication Number Publication Date
CN112053304A true CN112053304A (en) 2020-12-08

Family

ID=73610360

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010934440.XA Pending CN112053304A (en) 2020-09-08 2020-09-08 Rapid focusing restoration method for single shooting of full-slice digital imaging

Country Status (1)

Country Link
CN (1) CN112053304A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070660A (en) * 2020-09-08 2020-12-11 哈尔滨工业大学 Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180191948A1 (en) * 2017-01-03 2018-07-05 University Of Connecticut Single-Frame Autofocusing Using Multi-LED Illumination
CN108848302A (en) * 2018-05-24 2018-11-20 杭州电子科技大学 A kind of acquisition of micro-imaging, control and automated processing system and processing method
CN109410130A (en) * 2018-09-28 2019-03-01 华为技术有限公司 Image processing method and image processing apparatus
CN111462075A (en) * 2020-03-31 2020-07-28 湖南国科智瞳科技有限公司 Rapid refocusing method and system for full-slice digital pathological image fuzzy area

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180191948A1 (en) * 2017-01-03 2018-07-05 University Of Connecticut Single-Frame Autofocusing Using Multi-LED Illumination
CN108848302A (en) * 2018-05-24 2018-11-20 杭州电子科技大学 A kind of acquisition of micro-imaging, control and automated processing system and processing method
CN109410130A (en) * 2018-09-28 2019-03-01 华为技术有限公司 Image processing method and image processing apparatus
CN111462075A (en) * 2020-03-31 2020-07-28 湖南国科智瞳科技有限公司 Rapid refocusing method and system for full-slice digital pathological image fuzzy area

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
QIANG LI等: "Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing", 《ARXIV》 *
葛云皓: "基于卷积神经网络的病理显微镜自动对焦与全局精准成像研究", 《中国优秀硕士学位论文全文数据库 (工程科技II辑)》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070660A (en) * 2020-09-08 2020-12-11 哈尔滨工业大学 Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning
CN112070660B (en) * 2020-09-08 2022-08-12 哈尔滨工业大学 Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning

Similar Documents

Publication Publication Date Title
JP6437947B2 (en) Fully automatic rapid microscope slide scanner
CN109597195B (en) Super-resolution three-dimensional light field microscopic imaging system and method based on lattice illumination
Yazdanfar et al. Simple and robust image-based autofocusing for digital microscopy
EP2273302B1 (en) Image acquiring apparatus, image acquiring method and image acquiring program
EP1336888B1 (en) Microscopy imaging system and data acquisition method
US6816606B2 (en) Method for maintaining high-quality focus during high-throughput, microscopic digital montage imaging
US9297995B2 (en) Automatic stereological analysis of biological tissue including section thickness determination
CN109085695B (en) Method for quickly focusing and photographing plane sample
RU2734447C2 (en) System for forming a synthesized two-dimensional image of a biological sample with high depth of field
CN111429562B (en) Wide-field color light slice microscopic imaging method based on deep learning
CN110411946B (en) Method for focusing camera image in digital slice scanner
CN109272575B (en) Method for improving modeling speed of digital slice scanner
He et al. Microscope images automatic focus algorithm based on eight-neighborhood operator and least square planar fitting
CN111220615A (en) Inclined three-dimensional scanning microscopic imaging system and method
CN112053304A (en) Rapid focusing restoration method for single shooting of full-slice digital imaging
CN112070887A (en) Depth learning-based full-slice digital imaging depth of field extension method
US11356593B2 (en) Methods and systems for single frame autofocusing based on color- multiplexed illumination
CN113705298A (en) Image acquisition method and device, computer equipment and storage medium
CN112070660B (en) Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning
CN112069735B (en) Full-slice digital imaging high-precision automatic focusing method based on asymmetric aberration
CN110349237B (en) Fast volume imaging method based on convolutional neural network
CN112070661A (en) Full-slice digital imaging rapid automatic focusing method based on deep learning
CN112037154A (en) High-precision quasi-focus restoration method for full-slice digital imaging and double photographing
CN112037153A (en) Full-slice digital imaging quasi-focus restoration method based on quasi-focus distance prior
CN112037152A (en) Full-slice digital imaging two-step quasi-focus restoration method based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20201208

WD01 Invention patent application deemed withdrawn after publication