CN110992408B - Digital section processing method and system based on pathological microscope - Google Patents

Digital section processing method and system based on pathological microscope Download PDF

Info

Publication number
CN110992408B
CN110992408B CN201911125442.8A CN201911125442A CN110992408B CN 110992408 B CN110992408 B CN 110992408B CN 201911125442 A CN201911125442 A CN 201911125442A CN 110992408 B CN110992408 B CN 110992408B
Authority
CN
China
Prior art keywords
boundary
image
sample
slice
slice sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911125442.8A
Other languages
Chinese (zh)
Other versions
CN110992408A (en
Inventor
盛志华
史本庆
李淑玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dippers Medical Technology Shandong Co ltd
Original Assignee
Dippers Medical Technology Shandong Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dippers Medical Technology Shandong Co ltd filed Critical Dippers Medical Technology Shandong Co ltd
Priority to CN201911125442.8A priority Critical patent/CN110992408B/en
Publication of CN110992408A publication Critical patent/CN110992408A/en
Application granted granted Critical
Publication of CN110992408B publication Critical patent/CN110992408B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

The digital section processing method and system based on the pathological microscope control the movement speed of the glass slide according to the image acquisition frame rate by presetting the image acquisition frame rate; acquiring a motion track of a glass slide, wherein the glass slide moves at a motion speed according to the motion track, and the motion track covers a slice sample on the glass slide; carrying out focal length adjustment of the microscope in the moving process of the glass slide, wherein the focal length adjustment comprises intermittent focusing and continuous focusing, and acquiring an image of the sliced sample when the sliced sample is in a preset ambiguity range; preprocessing the collected slice sample images to remove motion blur, carrying out image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample. The technical scheme of the invention ensures that the acquisition of the digital section does not depend on a digital pathology full-section scanner any more, does not need to have extremely high requirements on mechanical precision like the digital pathology full-section scanner, and reduces the digital processing cost of the pathology section.

Description

Digital section processing method and system based on pathological microscope
Technical Field
The embodiment of the invention relates to the technical field of section processing, in particular to a digital section processing method and a digital section processing system based on a pathological microscope.
Background
Pathological section is usually made by embedding pathological tissues in paraffin blocks, slicing the paraffin blocks with a microtome, staining the paraffin blocks with hematoxylin-eosin (H-E), and further examining the pathological development process with a microscope. The digital section is to scan the whole glass slide in full information and all directions quickly, so that the traditional materialized glass slide is changed into a new generation of digital pathological section, a pathological doctor can be separated from a microscope, pathological diagnosis can be realized through a network at any time and any place, and global online synchronous remote consultation or offline remote consultation can be realized.
In the traditional technical scheme, the acquisition of the digital section needs to depend on a digital pathology full-section scanner, and the cost of a single digital pathology full-section scanner is far higher than that of a common pathology microscope at present. The expensive cost is mainly due to the requirement of extremely high mechanical precision of the automatic slice scanning mechanism, the microscope field of view is generally less than 0.5mm × 0.5mm, and the full-slice scanning area is generally more than 20mm × 20mm, so that thousands of fields of view need to be scanned by one full slice. The field-of-view scanning requires the slide system to be loaded with slides, the motors and control systems to be positioned with high precision and moved quickly, and customizing these specialized mechanical structures results in extremely high system costs. A digital section processing technical scheme based on a pathological microscope is needed.
Disclosure of Invention
Therefore, the invention provides a pathological microscope-based digital section processing method and system, which can be applied to refit and upgrade the traditional optical microscope and realize the acquisition of digital full sections on the basis of the optical microscope.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
in a first aspect, a pathological microscope-based digital section processing method includes the following steps:
presetting an image acquisition frame rate, and controlling the movement speed of the glass slide according to the image acquisition frame rate;
acquiring a motion track of a slide, wherein the slide moves at the motion speed according to the motion track, and the motion track covers a slice sample on the slide;
carrying out focal length adjustment of the microscope in the slide moving process, wherein the focal length adjustment comprises intermittent focusing and continuous focusing, and acquiring an image of a sliced sample when the sliced sample is in a preset ambiguity range;
preprocessing the collected slice sample images to remove motion blur, carrying out image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample.
As a preferable scheme of the digital section processing method based on the pathological microscope, the intermittent focusing judges whether the focal length needs to be readjusted by calculating the ambiguity of the given image area, and the image of the section sample is acquired when the ambiguity of the given image area is within a preset ambiguity range.
As a preferable scheme of the digital section processing method based on the pathological microscope, the change trend of the blur degree of the given image area is judged in the process of continuously focusing on the slide glass, and the up-down position of the slide glass is adjusted according to the change trend of the blur degree to compensate so that the blur degree of the section sample is maintained in a preset blur degree range.
As a preferable scheme of the digital section processing method based on the pathological microscope, a motion convolution kernel is obtained according to the motion direction and the motion speed of the slide glass and the exposure time of the camera, and a deconvolution algorithm is adopted to remove motion blur.
As a preferred solution of the pathological microscope-based digital slice processing method, the image registration comprises the following steps:
extracting feature points of adjacent slice sample images, and calculating the matching degree between the feature points of the adjacent images;
selecting matching points according to the matching degree, and calculating a single mapping matrix of the matching points;
warping and stitching adjacent slice sample images according to the single mapping matrix and camera parameters.
As a preferable scheme of the digital section processing method based on the pathological microscope, a splicing gap adjacent to a section sample image is removed by adopting a mixing process, and the mixing process comprises the following steps: and for the splicing area of the adjacent slice sample images, calculating the weighted sum of the pixel values according to the distance from the spliced pixels to the center of each image to obtain the effect of seamless splicing.
As a preferable scheme of the digital section processing method based on the pathological microscope, the determination of the motion trail comprises boundary wandering and filling wandering;
the step of boundary walking comprises:
manually selecting an upper left corner field of the trimming sample as an origin, wherein the field comprises a trimming sample region and a blank region;
judging the boundaries of the trimming sample areas and the blank areas, and calculating the average value of the boundary in the tangential direction;
judging to carry out transverse migration or longitudinal migration according to the tangent average value, when a blank field of view appears or all samples in the field of view are samples, migrating along the direction vertical to the original movement direction to enable the boundary of the sliced sample to be in the field of view, and when the blank field of view appears again, migrating along the direction opposite to the original movement direction to enable the boundary of the sliced sample to appear in the field of view again;
detecting whether the boundary walks to an origin area of the field of view, and finishing boundary walking when the boundary walks to the origin area;
the step of filling the walk comprises:
and taking the closed ring formed by the boundary wandering as a boundary, and sequentially acquiring the slice sample regions in the closed ring.
As a preferable embodiment of the digital section processing method based on the pathological microscope, the boundary determining step includes:
dividing virtual lattice points of the slice sample image, and searching a local boundary of the slice sample image along a row or a column from an initial point according to the virtual lattice points; moving along a local boundary places the slice sample image on one side of the direction of movement, forming a global boundary for the slice sample when returning again to the initial point.
In a second aspect, a pathological microscope-based digital section processing system is provided, comprising:
the frame rate presetting module is used for presetting an image acquisition frame rate and controlling the movement speed of the glass slide according to the image acquisition frame rate;
the movement track acquisition module is used for acquiring the movement track of the glass slide, the glass slide moves at the movement speed according to the movement track, and the movement track covers the slice sample on the glass slide;
the focal length adjusting module is used for adjusting the focal length of the microscope in the moving process of the glass slide, and the focal length adjustment comprises an intermittent focusing unit and a continuous focusing unit;
the sample acquisition module is used for acquiring an image of the slice sample when the slice sample is within a preset ambiguity range;
the preprocessing module is used for preprocessing the acquired slice sample image to remove motion blur;
and the registration and splicing module is used for carrying out image registration on the preprocessed slice sample images and splicing the registered adjacent slice sample images to obtain a full image of the slice sample.
As a preferable scheme of the digital section processing system based on the pathological microscope, the intermittent focusing unit judges whether the focal length needs to be readjusted by calculating the ambiguity of the given image area, and acquires the image of the section sample when the ambiguity of the given image area is within a preset ambiguity range.
As a preferable scheme of the digital section processing system based on the pathological microscope, the continuous focusing unit judges the change trend of the blur degree of the given image area during the movement of the slide glass, and adjusts the upper and lower positions of the slide glass according to the change trend of the blur degree to compensate so that the blur degree of the section sample is maintained in a preset blur degree range.
As a preferable scheme of the digital section processing system based on the pathological microscope, the preprocessing module acquires a motion convolution kernel according to the motion direction and the motion speed of the slide glass and the exposure time of the camera, and removes motion blur by adopting a deconvolution algorithm.
As a preferred solution of the digital section processing system based on the pathology microscope, the registration stitching module includes:
a feature point extraction unit: the method is used for extracting feature points of adjacent slice sample images and calculating the matching degree between the feature points of the adjacent images;
a matching unit: the single mapping matrix is used for selecting matching points according to the matching degree and calculating the single mapping matrix of the matching points;
splicing unit: for warping and stitching adjacent slice sample images according to the single-map matrix and camera parameters.
As a preferable solution of the digital section processing system based on the pathological microscope, the registration stitching module further includes a blending processing unit for removing a stitching slit adjacent to the section sample image; and calculating the weighted sum of the pixel values according to the distance from the spliced pixel to the center of each image in the spliced area of the adjacent slice sample images in the mixing processing unit to obtain the effect of seamless splicing.
As a preferable scheme of the digital section processing system based on the pathological microscope, the motion trail acquisition module comprises a boundary walking unit and a filling walking unit;
the boundary wander unit includes:
the origin point selection submodule is used for artificially selecting a field of view at the upper left corner of the trimming sample as an origin point, and the field of view comprises a trimming sample region and a blank region;
the boundary judgment submodule is used for judging the boundaries of the trimming sample areas and the blank areas and calculating the average value of the boundary in the tangential direction;
the boundary wandering submodule is used for judging transverse wandering or longitudinal wandering according to the tangent average value, wandering along the direction vertical to the original moving direction to enable the boundary of the sliced sample to be in the field of view when a blank field of view appears or all samples in the field of view appear, and wandering along the direction opposite to the original moving direction to enable the boundary of the sliced sample to appear in the field of view again when the blank field of view appears again;
the origin detection submodule is used for detecting whether the boundary walks to an origin area of the view field or not, and ending boundary walking when the boundary walks to the origin area;
the filling wander unit includes:
and the filling migration submodule is used for taking the closed ring formed by the boundary migration as a boundary and sequentially acquiring the slice sample regions in the closed ring.
As a preferable embodiment of the digital section processing system based on the pathological microscope, the motion trajectory acquisition module further includes a boundary determination unit, and the boundary determination unit includes:
the grid point division submodule is used for carrying out virtual grid point division on the slice sample image;
the local boundary acquisition submodule is used for searching a local boundary of the slice sample image from an initial point along a row or a column according to the virtual lattice point;
and the global boundary acquisition sub-module is used for moving along the local boundary to enable the slice sample image to be positioned at one side of the moving direction, and forming the global boundary of the slice sample when the initial point is returned again.
In a third aspect, a computer-readable storage medium is provided, in which program code for pathological full-slice processing is stored, the program code comprising instructions for performing the pathological full-slice processing method of the first aspect or any possible implementation thereof.
In a fourth aspect, an electronic device is provided, which includes a processor coupled with a storage medium, and when the processor executes instructions in the storage medium, the electronic device is caused to perform the pathology full-slice processing method in the first aspect or any possible implementation manner thereof.
According to the embodiment of the invention, the movement speed of the glass slide is controlled according to the image acquisition frame rate by presetting the image acquisition frame rate; acquiring a motion track of a glass slide, wherein the glass slide moves at a motion speed according to the motion track, and the motion track covers a slice sample on the glass slide; carrying out focal length adjustment of the microscope in the moving process of the glass slide, wherein the focal length adjustment comprises intermittent focusing and continuous focusing, and acquiring an image of the sliced sample when the sliced sample is in a preset ambiguity range; preprocessing the collected slice sample images to remove motion blur, carrying out image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample. The technical scheme of the invention ensures that the acquisition of the digital section does not depend on the traditional digital pathology full-section scanner, the image acquisition is carried out by controlling the movement of the glass slide on the basis of the traditional optical microscope, and the image deblurring, the registration and splicing treatment are carried out on the basis of the computer technology to obtain the full-width digital image of the section sample, so that the digital pathology full-section scanner does not have extremely high requirement on mechanical precision, and the digital treatment cost of the pathology section is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
Fig. 1 is a schematic flow chart of a pathological microscope-based digital section processing method provided in an embodiment of the present invention;
FIG. 2 is a flowchart of an intermittent focusing process in the digital section processing method based on a pathological microscope according to an embodiment of the present invention;
FIG. 3 is a flow chart of continuous focusing in the digital section processing method based on pathological microscope provided in the embodiment of the present invention;
FIG. 4 is a diagram illustrating an image stitching deblurring effect in the digital slice processing method based on a pathological microscope according to an embodiment of the present invention;
fig. 5 is an image stitching process in the digital section processing method based on the pathological microscope provided in the embodiment of the present invention;
fig. 6 is a diagram illustrating an image stitching effect in the digital slice processing method based on the pathological microscope according to the embodiment of the present invention;
fig. 7 is a flowchart of processing a motion trajectory in the digital section processing method based on a pathological microscope according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a pathological microscope-based digital section processing system provided in an embodiment of the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an embodiment of the present invention provides a pathological microscope-based digital section processing method, including the following steps:
s1: presetting an image acquisition frame rate, and controlling the movement speed of the glass slide according to the image acquisition frame rate;
s2: acquiring a motion track of a slide, wherein the slide moves at the motion speed according to the motion track, and the motion track covers a slice sample on the slide;
s3: carrying out focal length adjustment of the microscope in the slide moving process, wherein the focal length adjustment comprises intermittent focusing and continuous focusing, and acquiring an image of a sliced sample when the sliced sample is in a preset ambiguity range;
s4: preprocessing the collected slice sample images to remove motion blur, carrying out image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample.
In one embodiment of the digital section processing method based on the pathological microscope, the slide movement is realized by controlling the up-and-down position of the slide platform by a servo motor, and the continuous change of the working distance from the microscope lens to the section sample can be realized, so as to realize the automatic focusing in the image acquisition process. The automatic focusing can be carried out when the image area fuzziness is beyond the requirement, or can be continuously carried out by the control system to maintain the fuzziness within the required range all the time.
Referring to fig. 2, in an embodiment of the digital section processing method based on the pathological microscope, the intermittent focusing determines whether the focus needs to be adjusted again by calculating the ambiguity of a given image region, and the image of the section sample is acquired when the ambiguity of the given image region is within a preset ambiguity range.
Specifically, the intermittent focusing method comprises the following steps: it is first determined whether refocusing is required by calculating the blur level of the image. If focusing is required, the slide platform can stop moving, the focusing motor is adjusted upwards and downwards by taking the current focal distance as a center, images are collected simultaneously, and each collected image is analyzed to determine the current optimal focal distance position.
Referring to fig. 3, in an embodiment of the digital section processing method based on the pathological microscope, the change trend of the blur degree of a given image area is judged during the movement of the slide glass by continuously focusing, and the up-down position of the slide glass is adjusted according to the change trend of the blur degree to compensate so that the blur degree of the section sample is maintained within a preset blur degree range.
Specifically, the method of continuous focusing is as follows: and judging the change trend of the ambiguity in the continuous motion process of the slide glass platform, and adjusting the upper and lower positions of the slide glass platform in real time to compensate so as to maintain the ambiguity within an acceptable range. The degree of blur is a measure, such as the standard deviation of the entire image, or a particular region of the image.
In one embodiment of the digital section processing method based on the pathological microscope, a motion convolution kernel is obtained according to the motion direction and the motion speed of the slide glass and the exposure time of the camera, and a deconvolution algorithm is adopted to remove motion blur.
Specifically, to obtain the best image quality, preprocessing is performed before the images are spliced. The pre-processing includes de-motion blur. The continuous movement of the slide stage during the camera exposure causes motion blur that can be removed with software algorithms. Knowing the motion direction, the motion speed and the camera exposure time, the motion convolution kernel can be correctly estimated, and then the motion blur can be removed by utilizing a deconvolution algorithm. Referring to fig. 4, a comparison of the effect before and after the use of the deblurring algorithm is shown.
Referring to fig. 5, in one embodiment of the pathological microscope-based digital slice processing method, the image registration comprises the steps of:
extracting feature points of adjacent slice sample images, and calculating the matching degree between the feature points of the adjacent images;
selecting matching points according to the matching degree, and calculating a single mapping matrix of the matching points;
warping and stitching adjacent slice sample images according to the single mapping matrix and camera parameters.
In one embodiment of the digital slice processing method based on the pathological microscope, a blending process is adopted to remove the splicing gaps of the adjacent slice sample images, and the blending process comprises the following steps: and for the splicing area of the adjacent slice sample images, calculating the weighted sum of the pixel values according to the distance from the spliced pixels to the center of each image to obtain the effect of seamless splicing.
Specifically, after a clear image is obtained, any image registration technology may be used to stitch the adjacent slice sample images, for example, feature points (e.g., corner points) are extracted from the adjacent images based on a feature point matching algorithm, and the matching degree between the feature points of the adjacent images is calculated. And selecting matching points and calculating a single mapping matrix. The images are warped and stitched according to the single-mapping matrix and the gantry geometry.
Referring to fig. 6, an example of stitching is shown, where the slide stage is moved horizontally and the digital camera captures two pictures from the left and right, with some overlap between the pictures. The single mapping matrix is solved through feature point matching, proper image distortion is carried out, then the image is spliced into a picture with a larger view field, the spliced image is from two times of photographing, any slight difference can cause splicing gaps, and the gaps can be effectively removed by using a hybrid processing method. The method is described as follows: for the stitching area, the distance from the stitching pixel to the center of each image is obtained and used for calculating the weighted sum of the pixel values to obtain better seamless stitching effect. It is readily appreciated that other weighting and weighted averaging methods may be employed.
In one embodiment of the pathology microscope based digital slide processing method, one core issue that needs to be addressed is the control of the movement of the slide platform. To save scanning time, it is desirable to reduce the distance of movement of the slide platform. But the range of motion needs to ensure that the collection of microscope fields of view at least covers the sample. The method comprises the following steps: an additional camera is installed, and can shoot a panoramic view of the sample slice, locate the sample boundary by using the picture and determine the moving track of the object stage. The second method comprises the following steps: the microscope camera is directly used for shooting a view field image without an additional camera, a control signal is generated through image processing and fed back to the slide glass platform control circuit, and the motion trail of the slide glass platform is controlled.
Specifically, the boundary definition and motion direction detection method in the embodiment of the invention is divided into two steps: boundary wander and fill wander. The motor allows both lateral and longitudinal directions of motion.
Referring to fig. 7, the boundary wandering step includes:
manually selecting an upper left corner field of the trimming sample as an origin, wherein the field comprises a trimming sample region and a blank region;
judging the boundaries of the trimming sample areas and the blank areas, and calculating the average value of the boundary in the tangential direction;
judging to carry out transverse migration or longitudinal migration according to the tangent average value, when a blank field of view appears or all samples in the field of view are samples, migrating along the direction vertical to the original movement direction to enable the boundary of the sliced sample to be in the field of view, and when the blank field of view appears again, migrating along the direction opposite to the original movement direction to enable the boundary of the sliced sample to appear in the field of view again;
detecting whether the boundary walks to an origin area of the field of view, and finishing boundary walking when the boundary walks to the origin area;
the step of filling the walk comprises:
and taking the closed ring formed by the boundary wandering as a boundary, and sequentially acquiring the slice sample regions in the closed ring.
In an embodiment of the digital section processing method based on the pathological microscope, the boundary determining step is as follows:
dividing virtual lattice points of the slice sample image, and searching a local boundary of the slice sample image along a row or a column from an initial point according to the virtual lattice points; moving along a local boundary places the slice sample image on one side of the direction of movement, forming a global boundary for the slice sample when returning again to the initial point.
Specifically, a set of virtual grid points is preset on the image, particularly the low-magnification image. First find the edge of the image along a row or column and then move along the edge so that the image is always on one side of the direction of movement, when returning to the initial point, the sample boundary of the image is found.
Referring to fig. 8, an embodiment of the present invention further provides a pathological microscope-based digital section processing system, including:
the frame rate presetting module 1 is used for presetting an image acquisition frame rate and controlling the movement speed of the glass slide according to the image acquisition frame rate;
the motion trail acquisition module 2 is configured to acquire a motion trail of a slide, the slide moves at the motion speed according to the motion trail, and the motion trail covers a slice sample on the slide;
the focal length adjusting module 3 is used for adjusting the focal length of the microscope in the slide moving process, and the focal length adjustment comprises an intermittent focusing unit 301 and a continuous focusing unit 302;
the sample acquisition module 4 is used for acquiring an image of the slice sample when the slice sample is within a preset ambiguity range;
the preprocessing module 5 is used for preprocessing the acquired slice sample image to remove motion blur;
and the registration and splicing module 6 is used for performing image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample.
In one embodiment of the digital section processing system based on the pathological microscope, the intermittent focusing unit 301 determines whether the focus needs to be adjusted again by calculating the blur degree of a given image region, and acquires the image of the section sample when the blur degree of the given image region is within a preset blur degree range.
In an embodiment of the digital section processing system based on the pathological microscope, the continuous focusing unit 302 determines a variation trend of the blur degree of a given image area during the movement of the slide glass, and adjusts the up-down position of the slide glass according to the variation trend of the blur degree to compensate so that the blur degree of the section sample is maintained within a preset blur degree range.
In an embodiment of the digital section processing system based on the pathological microscope, the preprocessing module 5 obtains a motion convolution kernel according to the motion direction and the motion speed of the slide and the exposure time of the camera, and removes motion blur by using a deconvolution algorithm.
In one embodiment of the digital section processing system based on a pathology microscope, the registration stitching module 6 comprises:
feature point extraction section 601: the method is used for extracting feature points of adjacent slice sample images and calculating the matching degree between the feature points of the adjacent images;
the matching unit 602: the single mapping matrix is used for selecting matching points according to the matching degree and calculating the single mapping matrix of the matching points;
splicing unit 603: for warping and stitching adjacent slice sample images according to the single-map matrix and camera parameters.
In an embodiment of the digital slice processing system based on a pathology microscope, the registration stitching module 6 further comprises a blending processing unit 604, the blending processing unit 604 is configured to remove stitching gaps adjacent to the slice sample image; for the spliced area of the adjacent slice sample images in the mixing processing unit 604, the weighted sum of the pixel values is calculated according to the distance from the spliced pixel to the center of each image, so as to obtain the effect of seamless splicing.
In an embodiment of the digital slice processing system based on the pathological microscope, the motion trajectory acquisition module 2 includes a boundary walking unit 201 and a filling walking unit 202;
the boundary wander unit 201 includes:
an origin selecting submodule 2011, configured to manually select an upper left corner field of the trimmed sample as an origin, where the field includes a trimmed sample region and a blank region;
the boundary judgment sub-module 2012 is configured to judge the boundary between the trimming sample region and the blank region, and calculate the average value in the tangential direction of the boundary;
a boundary wandering submodule 2013, configured to determine to perform horizontal wandering or longitudinal wandering according to the tangent average value, wander in a direction perpendicular to an original moving direction to make a boundary of the sliced sample in the field of view when a blank field of view appears or all samples in the field of view appear, and wander in a direction opposite to the original moving direction to make the boundary of the sliced sample reappear in the field of view when the blank field of view appears again;
an origin detection sub-module 2014, configured to detect whether to wander to an origin area of the field of view, and end boundary wandering when wandering to the origin area;
the fill-walk unit 202 includes:
the fill wander sub-module 2021 is configured to sequentially acquire the slice sample regions within the closed loop bounded by the closed loop formed by the boundary wander.
In an embodiment of the digital section processing system based on a pathological microscope, the motion trajectory acquiring module 2 further includes a boundary determining unit 203, and the boundary determining unit 203 includes:
a grid point division submodule 2031 configured to perform virtual grid point division on the slice sample image;
a local boundary obtaining sub-module 2032, configured to find a local boundary of the slice sample image along a row or a column from an initial point according to the virtual grid point;
the global boundary acquisition sub-module 2033 is configured to move along the local boundary so that the slice sample image is located at one side of the moving direction, and form a global boundary of the slice sample when returning to the initial point again.
It should be noted that, for the information interaction, execution process, and other contents between the modules/units of the system, since the same concept is based on the method embodiment in the embodiment of the present application, the technical effect brought by the information interaction, execution process, and other contents is the same as that of the method embodiment of the present application, and specific contents may refer to the description in the foregoing method embodiment of the present application, and are not described herein again.
An embodiment of the present invention further provides a computer-readable storage medium, in which a program code for pathology full-slice processing is stored, where the program code includes instructions for executing the pathology full-slice processing method in the first aspect or any possible implementation manner thereof.
The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
An embodiment of the present invention further provides an electronic device, where the electronic device includes a processor, and the processor is coupled to a storage medium, and when the processor executes instructions in the storage medium, the electronic device is caused to execute the pathology full-slice processing method in the first aspect or any possible implementation manner of the first aspect.
Specifically, the processor may be implemented by hardware or software, and when implemented by hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor, located external to the processor, or stand-alone.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).
In the case where the above-described series of processes is realized by software, a program constituting the software is installed from a network such as the internet or a storage medium such as a removable medium.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
Although the invention has been described in detail above with reference to a general description and specific examples, it will be apparent to one skilled in the art that modifications or improvements may be made thereto based on the invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.

Claims (7)

1. The digital section processing method based on the pathological microscope is characterized by comprising the following steps: the method comprises the following steps:
presetting an image acquisition frame rate, and controlling the movement speed of the glass slide according to the image acquisition frame rate;
acquiring a motion track of a slide, wherein the slide moves at the motion speed according to the motion track, and the motion track covers a slice sample on the slide;
carrying out focal length adjustment of the microscope in the slide moving process, wherein the focal length adjustment comprises intermittent focusing and continuous focusing, and acquiring an image of a sliced sample when the sliced sample is in a preset ambiguity range;
preprocessing the collected slice sample images to remove motion blur, carrying out image registration on the preprocessed slice sample images, and splicing the registered adjacent slice sample images to obtain a full image of the slice sample;
the image registration comprises the steps of:
extracting feature points of adjacent slice sample images, and calculating the matching degree between the feature points of the adjacent images;
selecting matching points according to the matching degree, and calculating a single mapping matrix of the matching points;
warping and splicing the adjacent slice sample images according to the single mapping matrix and the camera parameters;
removing a stitching seam adjacent to a sliced sample image using a blending process, the blending process comprising: for the splicing area of the adjacent slice sample images, calculating the weighted sum of pixel values according to the distance from the spliced pixels to the center of each image to obtain the seamless splicing effect;
the determination of the motion trail comprises boundary wandering and filling wandering;
the step of boundary walking comprises:
manually selecting an upper left corner field of the trimming sample as an origin, wherein the field comprises a trimming sample region and a blank region;
judging the boundaries of the trimming sample areas and the blank areas, and calculating the average value of the boundary in the tangential direction;
judging to carry out transverse migration or longitudinal migration according to the tangent average value, when a blank field of view appears or all samples in the field of view are samples, migrating along the direction vertical to the original movement direction to enable the boundary of the sliced sample to be in the field of view, and when the blank field of view appears again, migrating along the direction opposite to the original movement direction to enable the boundary of the sliced sample to appear in the field of view again;
detecting whether the boundary walks to an origin area of the field of view, and finishing boundary walking when the boundary walks to the origin area;
the step of filling the walk comprises:
taking a closed ring formed by the boundary wandering as a boundary, and sequentially acquiring slice sample regions in the closed ring;
the boundary determining step comprises:
dividing virtual lattice points of the slice sample image, and searching a local boundary of the slice sample image along a row or a column from an initial point according to the virtual lattice points; moving along a local boundary places the slice sample image on one side of the direction of movement, forming a global boundary for the slice sample when returning again to the initial point.
2. The pathology microscope-based digital slice processing method of claim 1, wherein: and judging whether the focal length needs to be readjusted or not by calculating the ambiguity of the given image area in the intermittent focusing process, and acquiring the image of the slice sample when the ambiguity of the given image area is within a preset ambiguity range.
3. The pathology microscope-based digital slice processing method of claim 1, wherein: and continuously focusing on the slide glass in the moving process to judge the change trend of the fuzziness of the given image area, and adjusting the upper and lower positions of the slide glass according to the change trend of the fuzziness to compensate so as to maintain the fuzziness of the sliced sample in a preset fuzziness range.
4. The pathology microscope-based digital slice processing method of claim 1, wherein: and acquiring a motion convolution kernel according to the motion direction and the motion speed of the glass slide and the exposure time of the camera, and removing motion blur by adopting a deconvolution algorithm.
5. Digital section processing system based on pathology microscope, its characterized in that: the method comprises the following steps:
the frame rate presetting module is used for presetting an image acquisition frame rate and controlling the movement speed of the glass slide according to the image acquisition frame rate;
the movement track acquisition module is used for acquiring the movement track of the glass slide, the glass slide moves at the movement speed according to the movement track, and the movement track covers the slice sample on the glass slide;
the focal length adjusting module is used for adjusting the focal length of the microscope in the moving process of the glass slide, and the focal length adjustment comprises an intermittent focusing unit and a continuous focusing unit;
the sample acquisition module is used for acquiring an image of the slice sample when the slice sample is within a preset ambiguity range;
the preprocessing module is used for preprocessing the acquired slice sample image to remove motion blur;
the registration splicing module is used for carrying out image registration on the preprocessed slice sample images and splicing the adjacent slice sample images after registration to obtain a full image of the slice sample;
the intermittent focusing unit judges whether the focal length needs to be adjusted again or not by calculating the ambiguity of a given image area, and acquires the image of the slice sample when the ambiguity of the given image area is within a preset ambiguity range;
the continuous focusing unit judges the change trend of the fuzziness of a given image area in the moving process of the glass slide, and adjusts the upper position and the lower position of the glass slide according to the change trend of the fuzziness to compensate so that the fuzziness of the sliced sample is maintained in a preset fuzziness range;
the preprocessing module acquires a motion convolution kernel according to the motion direction and the motion speed of the glass slide and the exposure time of the camera, and removes motion blur by adopting a deconvolution algorithm;
the registration stitching module includes:
a feature point extraction unit: the method is used for extracting feature points of adjacent slice sample images and calculating the matching degree between the feature points of the adjacent images;
a matching unit: the single mapping matrix is used for selecting matching points according to the matching degree and calculating the single mapping matrix of the matching points;
splicing unit: warping and stitching adjacent slice sample images according to the single-mapping matrix and camera parameters;
the registration stitching module further comprises a mixing processing unit for removing stitching gaps adjacent to the sliced sample image; for a splicing area of adjacent slice sample images in the mixing processing unit, calculating the weighted sum of pixel values according to the distance from a spliced pixel to the center of each image to obtain a seamless splicing effect;
the motion trail acquisition module comprises a boundary walking unit and a filling walking unit;
the boundary wander unit includes:
the origin point selection submodule is used for artificially selecting a field of view at the upper left corner of the trimming sample as an origin point, and the field of view comprises a trimming sample region and a blank region;
the boundary judgment submodule is used for judging the boundaries of the trimming sample areas and the blank areas and calculating the average value of the boundary in the tangential direction;
the boundary wandering submodule is used for judging transverse wandering or longitudinal wandering according to the tangent average value, wandering along the direction vertical to the original moving direction to enable the boundary of the sliced sample to be in the field of view when a blank field of view appears or all samples in the field of view appear, and wandering along the direction opposite to the original moving direction to enable the boundary of the sliced sample to appear in the field of view again when the blank field of view appears again;
the origin detection submodule is used for detecting whether the boundary walks to an origin area of the view field or not, and ending boundary walking when the boundary walks to the origin area;
the filling wander unit includes:
the filling migration submodule is used for taking a closed ring formed by the boundary migration as a boundary and sequentially acquiring the slice sample regions in the closed ring;
the motion trail acquisition module further comprises a boundary judgment unit, and the boundary judgment unit comprises:
the grid point division submodule is used for carrying out virtual grid point division on the slice sample image;
the local boundary acquisition submodule is used for searching a local boundary of the slice sample image from an initial point along a row or a column according to the virtual lattice point;
and the global boundary acquisition sub-module is used for moving along the local boundary to enable the slice sample image to be positioned at one side of the moving direction, and forming the global boundary of the slice sample when the initial point is returned again.
6. A computer-readable storage medium characterized by: the computer readable storage medium having stored therein program code for digital full slice processing, the program code comprising instructions for performing the digital slice processing method of any of claims 1 to 4.
7. An electronic device, characterized in that: the electronic device includes a processor coupled with a storage medium that, when executing instructions in the storage medium, causes the electronic device to perform the digital slice processing method of any of claims 1 to 4.
CN201911125442.8A 2019-11-18 2019-11-18 Digital section processing method and system based on pathological microscope Active CN110992408B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911125442.8A CN110992408B (en) 2019-11-18 2019-11-18 Digital section processing method and system based on pathological microscope

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911125442.8A CN110992408B (en) 2019-11-18 2019-11-18 Digital section processing method and system based on pathological microscope

Publications (2)

Publication Number Publication Date
CN110992408A CN110992408A (en) 2020-04-10
CN110992408B true CN110992408B (en) 2021-08-17

Family

ID=70084743

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911125442.8A Active CN110992408B (en) 2019-11-18 2019-11-18 Digital section processing method and system based on pathological microscope

Country Status (1)

Country Link
CN (1) CN110992408B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112632661A (en) * 2020-12-14 2021-04-09 河南工业大学 SBS modified asphalt three-dimensional microstructure reconstruction method based on intelligent recognition algorithm
CN113487508B (en) * 2021-07-08 2024-03-26 山东志盈医学科技有限公司 Method and device for dynamically adjusting picture definition of digital slice scanner

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10317663B2 (en) * 2015-02-27 2019-06-11 General Electric Company Determination of deflection of a microscope slide
CN106873142B (en) * 2017-03-15 2019-04-12 北方工业大学 High-quality image acquisition device and method of tubercle bacillus detector
CN109978771A (en) * 2019-04-08 2019-07-05 哈尔滨理工大学 Cell image rapid fusion method based on content analysis
CN110146974B (en) * 2019-05-29 2020-10-23 电子科技大学 Intelligent biological microscope

Also Published As

Publication number Publication date
CN110992408A (en) 2020-04-10

Similar Documents

Publication Publication Date Title
US9224063B2 (en) Apparatus and method for digital microscopy imaging
TWI673750B (en) Sample observation device and sample observation method
KR101891364B1 (en) Fast auto-focus in microscopic imaging
US9307134B2 (en) Automatic setting of zoom, aperture and shutter speed based on scene depth map
US6816606B2 (en) Method for maintaining high-quality focus during high-throughput, microscopic digital montage imaging
CN110992408B (en) Digital section processing method and system based on pathological microscope
US8824735B2 (en) Multi-hypothesis projection-based shift estimation
CN104038691A (en) Auto Focus method and auto focus apparatus
CN112415733B (en) Method, system, device and medium for controlling microscope to shoot sample image
CN112505910B (en) Method, system, apparatus and medium for taking image of specimen with microscope
JP2016125913A (en) Image acquisition device and control method of image acquisition device
CN114463231A (en) Intelligent splicing method, device, medium and equipment for microscope images
JP4477863B2 (en) Cell measurement support system and cell observation apparatus
EP3709258B1 (en) Generating composite image from multiple images captured for subject
CN105651699B (en) It is a kind of based on the dynamic of area array cameras with burnt method
CN116313710A (en) Focusing control method and device, scanning electron microscope and storage medium
CN112839168B (en) Method for automatically adjusting camera imaging resolution in AOI detection system
US20140210980A1 (en) Image acquisition apparatus and image acquisition method
US11631183B2 (en) Method and system for motion segmentation
CN115578291A (en) Image brightness correction method, storage medium and electronic device
US20230258919A1 (en) A method for analysing scanning efficacy
Sánchez et al. Automatization techniques. Slide scanning
CN114663323B (en) Image range identification focusing method based on multiple samples of single slide and application thereof
CN117452617A (en) Focusing method, focusing device, electronic equipment and storage medium
Vázquez Marino Single-Camera 3D Microscope Scanner

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant