WO2022121284A1 - 大视场高通量高分辨病理切片分析仪 - Google Patents

大视场高通量高分辨病理切片分析仪 Download PDF

Info

Publication number
WO2022121284A1
WO2022121284A1 PCT/CN2021/102379 CN2021102379W WO2022121284A1 WO 2022121284 A1 WO2022121284 A1 WO 2022121284A1 CN 2021102379 W CN2021102379 W CN 2021102379W WO 2022121284 A1 WO2022121284 A1 WO 2022121284A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pathological
pathological slice
slice
resolution
Prior art date
Application number
PCT/CN2021/102379
Other languages
English (en)
French (fr)
Inventor
陈雪利
康欢
曾琦
徐欣怡
罗锡鑫
陈多芳
谢晖
Original Assignee
西安电子科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西安电子科技大学 filed Critical 西安电子科技大学
Publication of WO2022121284A1 publication Critical patent/WO2022121284A1/zh
Priority to US18/104,812 priority Critical patent/US20230177645A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/0008—Microscopes having a simple construction, e.g. portable microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/008—Details of detection or image processing, including general computer control
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/24—Base structure
    • G02B21/26—Stages; Adjusting means therefor
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/16—Image acquisition using multiple overlapping images; Image stitching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/955—Computational photography systems, e.g. light-field imaging systems for lensless imaging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the invention belongs to the technical field of medical imaging, and in particular relates to a high-throughput and high-resolution pathological slice analyzer with a large field of view.
  • Pathological sections are taken from a certain size of diseased tissue and made into pathological sections by histopathological methods. Machine cut into thin slices, and then stained with hematoxylin-eosin, with a microscope to further examine the lesions, as well as its occurrence and development process, and finally make a pathological diagnosis.
  • the existing pathological slice imaging methods and pathological slice analysis processes still have the following shortcomings: (1) The field of view of traditional optical microscopy technology The resolution is mutually limited, and the acquisition of high-resolution images is bound to be accompanied by a smaller field of view, which consumes a lot of time when a large amount of data is required; in addition, the marking of pathological slices by doctors is a highly repetitive process, which takes a long time.
  • the present invention provides a large field of view, high-throughput, and high-resolution pathological slice analyzer.
  • the technical problem to be solved by the present invention is realized by the following technical solutions:
  • the present invention provides a high-throughput and high-resolution pathological slice analyzer with a large field of view, comprising:
  • a pathological slice fixing module used for fixing and adjusting the position of the pathological slice, so that the pathological slice is located at the position of the imaging field of view;
  • a data acquisition module configured to collect the scattered light carrying wavefront information after monochromatic light passes through the pathological slice, and the interference image information formed by the transmitted light that does not carry the pathological slice information;
  • the control processing module is configured to perform image reconstruction on the interference image information to obtain a reconstructed image, and analyze the reconstructed image based on a pre-trained pathological slice analysis model to obtain an analysis result of the pathological slice.
  • the lighting module includes a monochromatic light source and a microporous structure located below the monochromatic light source, the monochromatic light source outputs monochromatic light with adjustable wavelength, and the microporous structure is used for illuminating the Monochromatic light is spatially filtered and diffracted.
  • the pathological slice fixing module includes a pathological slice holder and a pathological slice position adjuster, the pathological slice holder is used for fixing the pathological slice, and the pathological slice position adjuster is used for adjusting the pathological slice position
  • the position of the pathological slice is positioned so that the pathological slice is located at the position of the imaging field of view.
  • the data acquisition module includes a pixel detector, the imaging field of view of the pixel detector is >20 mm 2 , and the size of a single pixel of the pixel detector is ⁇ 1.4 ⁇ m.
  • control processing module includes a control unit and a data processing unit, the control unit is configured to control the lighting module to generate monochromatic light, and control the pathological slice fixing module to make the pathological slice is located at the position of the imaging field of view, and controls the data acquisition module to collect the interference image information;
  • the data processing unit searches the interference image information by using the image self-focusing algorithm of the structure tensor to obtain reconstruction parameters, and uses angular spectrum propagation theory to reconstruct the interference image information according to the reconstruction parameters to obtain the reconstruction parameters.
  • image using the image high-resolution algorithm based on generative adversarial network to improve the resolution of the reconstructed image, using the image analysis network based on the deep convolutional neural network to analyze the reconstructed image, and determine the tumor type of the pathological section , tumor malignancy and cancer type.
  • the data processing unit includes a reconstruction parameter acquisition subunit, an image reconstruction subunit, and an image analysis subunit, wherein,
  • the reconstruction parameter acquisition subunit is configured to search the interference image information according to the image self-focusing algorithm of the structure tensor to obtain a reconstruction parameter, and the reconstruction parameter is the distance z between the pathological slice and the data acquisition module;
  • the image reconstruction subunit is configured to perform image reconstruction on the interference image information by using the angular spectrum propagation theory according to the reconstruction parameters to obtain a reconstructed image;
  • the image analysis subunit is configured to analyze the reconstructed image based on a pre-trained pathological slice analysis model to obtain an analysis result of the pathological slice;
  • the analysis result includes the tumor type, tumor malignancy and cancer type of the pathological section.
  • the reconstruction parameter acquisition subunit is used for:
  • the image structure tensor for each search step is computed according to:
  • U(x,y) represents the interference image information collected by the data acquisition module, represents the 2D spatial gradient in the x and y directions, G(x, y) represents the non-negative convolution kernel, represents the gradient operator, and T represents the transpose;
  • a structure tensor curve is drawn to find the maximum point, and the maximum point is used as the distance z between the pathological slice and the data acquisition module.
  • the image reconstruction subunit is configured to perform image reconstruction according to the following image reconstruction formula according to the angular spectrum propagation theory, and obtain the reconstructed image:
  • ⁇ represents Fourier transform
  • ⁇ -1 represents inverse Fourier transform
  • i represents complex symbol
  • k 2 ⁇ / ⁇
  • ⁇ represents wavelength of monochromatic light
  • f x , f y represent spatial frequency
  • z represents pathological slice Distance from the data acquisition module.
  • the training method of the pathological slice analysis model includes:
  • Step a Acquire a data set, the data set includes several histopathological section images of different suspicious lesion areas, and the histopathological section images of each suspicious lesion area include a labeled microscopic image of the suspicious lesion area, a first reconstruction image and a second reconstructed image, where,
  • the labeled microscopic image is a microscopic image of the entire histopathological slice obtained by using a clinical microscope after hematoxylin-eosin staining is performed on the histopathological section of the suspicious lesion area, and then the microscopic image is marked. owned;
  • the first reconstructed image is obtained by using the large field of view, high-throughput and high-resolution pathological section analyzer after hematoxylin-eosin staining is performed on the histopathological section of the suspicious lesion area;
  • the second reconstructed image is obtained by using the large field of view, high-throughput and high-resolution pathological section analyzer for the histopathological section of the suspicious lesion area without hematoxylin-eosin staining;
  • Step b grouping the data sets as training data sets and test data sets
  • Step c input the training data set into a deep learning network for training and network parameter optimization, and the deep learning network includes a high-resolution image reconstruction network based on a confrontation generation network and an image analysis network based on a deep convolutional neural network;
  • Step d Repeat step c until the deep learning network reaches a preset number of training iterations, or the error is less than a preset threshold, to obtain a pathological slice analysis model;
  • Step e Perform performance test on the pathological section analysis model by using the test data set to determine the performance of the pathological section analysis model.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of the present invention has large field of view, high-throughput imaging, and the imaging field of view is hundreds of times that of a traditional optical microscope.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of the present invention greatly reduces the cost of the instrument because it does not rely on the objective lens to obtain images, and is suitable for clinical diagnosis, especially in areas where resources are scarce.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of the present invention adopts a deep learning network to analyze the pathological situation, does not need to change the existing instrument structure, and intelligently analyzes the pathological images, avoiding the problems caused by a large number of repeated operations by doctors. Errors and subjective factors of different doctors lead to inconsistent judgments of pathological conditions.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of the present invention can analyze and judge the tumor type, tumor malignancy and cancer classification of the pathological slice without staining the pathological slice, which simplifies the pathological slice. analysis process.
  • FIG. 1 is a structural block diagram of a large field of view high-throughput high-resolution pathological slice analyzer provided by an embodiment of the present invention
  • FIG. 2 is a structural block diagram of another large field of view high-throughput high-resolution pathological slice analyzer provided by an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a large field of view, high-throughput, and high-resolution pathological slice analyzer according to an embodiment of the present invention.
  • Icon 1-illumination module; 101-monochromatic light source; 102-micropore structure; 2-pathological slice fixation module; 201-pathological slice holder; 202-pathological slice position adjuster; 3-data acquisition module; 301-image 4-control processing module; 401-control unit; 402-data processing unit; 4021-reconstruction parameter acquisition sub-unit; 4022-image reconstruction sub-unit; 4023-image analysis sub-unit.
  • FIG. 1 is a structural block diagram of a large field of view, high-throughput, and high-resolution pathological slice analyzer provided by an embodiment of the present invention.
  • the large field of view high-throughput high-resolution pathological slice analyzer according to the embodiment of the present invention includes:
  • Lighting module 1 for generating monochromatic light
  • the pathological slice fixing module 2 is used to fix and adjust the position of the pathological slice, so that the pathological slice is located at the position of the imaging field of view;
  • the data acquisition module 3 is used to collect the interference image information formed by the scattered light carrying the wavefront information after the monochromatic light passes through the pathological section and the transmitted light not carrying the pathological section information;
  • the control processing module 4 is configured to perform image reconstruction on the interference image information to obtain a reconstructed image, and analyze the reconstructed image based on a pre-trained pathological slice analysis model to obtain an analysis result of the pathological slice.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of this embodiment greatly reduces the cost of the instrument because it does not rely on the objective lens to obtain images, and is suitable for clinical diagnosis, especially in areas where resources are scarce.
  • FIG. 2 is a structural block diagram of another large-field high-throughput and high-resolution pathological slice analyzer provided by an embodiment of the present invention
  • the lighting module 1 includes a monochromatic light source 101 and a microporous structure 102 located therebelow.
  • the monochromatic light source 101 outputs monochromatic light with adjustable wavelength, and the microporous structure 102 is used to perform the processing on the monochromatic light. spatially filtered and diffracted.
  • the monochromatic light source 101 is a laser diode.
  • the pathological slice fixing module 2 is located below the lighting module 1, and the pathological slice fixing module 2 includes a pathological slice holder 201 and a pathological slice position adjuster 202.
  • the pathological slice holder 201 is used for fixing the pathological slice, and the pathological slice position adjuster. 202 is used to adjust the position of the pathological slice so that the pathological slice is located at the position of the imaging field of view.
  • the data acquisition module 3 is located below the pathological slice fixing module 2, the data acquisition module 3 includes a pixel detector 301, the imaging field of view of the pixel detector is >20 mm 2 , and the size of a single pixel of the pixel detector is ⁇ 1.4 ⁇ m.
  • the pixel detector 301 may be a linear array CCD, an area array CCD or a CMOS.
  • CMOS is selected as the pixel detector 301 .
  • the imaging field of view can usually exceed 20mm 2 , and the imaging speed is fast, usually reaching 10-20fps.
  • the scattered light carrying the wavefront information after the monochromatic light passes through the pathological section forms an optical interference signal on the CMOS plane with the transmitted light that does not carry the pathological section information, and the CMOS converts the optical interference signal into an electrical signal , and transmitted to the control processing module 4 .
  • control processing module 4 includes a control unit 401 and a data processing unit 402 , wherein the control unit 401 is respectively connected with the monochromatic light source 101 , the pathological slice position adjuster 202 and the pixel detector 301 .
  • the control unit 401 is used to control the monochromatic light source 101 to output monochromatic light, control the pathological slice position adjuster 202 to locate the pathological slice at the best imaging visual field position, and control the pixel detector 301 to collect interference image information.
  • the data processing unit 402 is connected to the pixel detector 301.
  • the data processing unit 402 searches the received interference image information by using the image self-focusing algorithm of the structure tensor to obtain reconstruction parameters, and uses the angular spectrum propagation theory to analyze the interference image information according to the reconstruction parameters.
  • Perform image reconstruction to obtain the reconstructed image use the image high-resolution algorithm based on generative adversarial network to improve the resolution of the reconstructed image, use the image analysis network based on the deep convolutional neural network to analyze the reconstructed image, and determine the tumor type of the pathological slice , tumor malignancy and cancer type.
  • the large field of view, high-throughput, and high-resolution pathological slice analyzer of this embodiment has large field of view and high-throughput imaging.
  • the imaging field of view is hundreds of times that of traditional optical microscopes.
  • the pixel detector is used to collect signals, and the imaging field of view is equal to pixel detection. The size of the active area of the detector, and the image can be acquired quickly.
  • the data processing unit 402 includes a reconstruction parameter acquisition subunit 4021, an image reconstruction subunit 4022 and an image analysis subunit 4023, wherein,
  • the reconstruction parameter acquisition subunit 4021 is used for searching the interference image information according to the image self-focusing algorithm of the structure tensor to obtain the reconstruction parameter, and the reconstruction parameter is the distance z between the pathological slice and the data acquisition module;
  • the image reconstruction subunit 4022 is used for performing image reconstruction on the interference image information by using the angular spectrum propagation theory according to the reconstruction parameters to obtain a reconstructed image;
  • the image analysis subunit 4023 is used to analyze the reconstructed image based on the pre-trained pathological slice analysis model to obtain the analysis result of the pathological slice;
  • the input data of the pathological slice analysis model is the reconstructed image of the pathological slice
  • the output data of the pathological slice analysis model is the analysis result of the pathological slice
  • the analysis result includes the tumor type of the pathological slice, the degree of tumor malignancy, and the cancer type.
  • the reconstruction parameter acquisition subunit 4021 is used to: determine the search range and search step size of the distance between the pathological slice sample and the data acquisition module according to the prior information;
  • the prior information is an estimated value of the distance between the pathological slice sample and the pixel detector when building a large field of view, high-throughput, and high-resolution pathological slice analyzer.
  • the image structure tensor for each search step is computed according to:
  • U(x,y) represents the interference image information collected by the data acquisition module, Represents the 2D spatial gradient in the x and y directions, G(x, y) represents the non-negative convolution kernel, usually selected as a two-dimensional Gaussian function, represents the gradient operator, and T represents the transpose;
  • the structure tensor curve is drawn, and the maximum value point is found, and the maximum value point is used as the distance z between the pathological slice and the data acquisition module.
  • the image reconstruction subunit 4022 is configured to perform image reconstruction according to the following image reconstruction formula according to the angular spectrum propagation theory, and obtain the reconstructed image:
  • ⁇ represents Fourier transform
  • ⁇ -1 represents inverse Fourier transform
  • i represents complex symbol
  • k 2 ⁇ / ⁇
  • ⁇ represents wavelength of monochromatic light
  • f x , f y represent spatial frequency
  • z represents pathological slice Distance from the data acquisition module.
  • the training method of the pathological slice analysis model includes:
  • Step a Acquire a data set, the data set includes several histopathological section images of different suspicious lesion areas, and the histopathological section images of each suspicious lesion area include a labeled microscopic image, a first reconstructed image and a labeled microscopic image of the suspicious lesion area. the second reconstructed image, where,
  • the marked microscopic images are obtained by performing hematoxylin-eosin staining on the histopathological sections of the suspicious lesion area, and then using a clinical microscope to obtain the microscopic images of the whole histopathological sections, and then marking the microscopic images;
  • the first reconstructed image is obtained by performing hematoxylin-eosin staining on the histopathological section of the suspicious lesion area using a high-throughput and high-resolution pathological section analyzer with a large field of view;
  • the second reconstructed image is obtained by using a large field of view high-throughput high-resolution pathological section analyzer on the histopathological section of the suspicious lesion area without hematoxylin-eosin staining;
  • the method for acquiring the histopathological slice image of each suspicious lesion area includes: continuously cutting two histopathological slices at the same position of the suspicious lesion area, one of which is stained with hematoxylin-eosin, and one of which is not subjected to hematoxylin -Eosin staining.
  • a clinical microscope combined with spatial translation is used to obtain a high-resolution image of the whole histopathological section stained with hematoxylin-eosin, that is, a microscopic image, and then the microscopic image is marked by an experienced clinician. Microscopic images with markers were obtained.
  • the reconstructed image of the histopathological slice stained with hematoxylin-eosin is obtained as the first reconstructed image.
  • the reconstructed image of the pathological slice without hematoxylin-eosin staining is obtained as the second reconstructed image.
  • two histopathological sections were continuously cut from the same position of the suspicious lesion area, one for hematoxylin-eosin staining and one for no hematoxylin-eosin staining, and the stained histopathological sections were used to obtain the suspicious lesions.
  • the labeled microscopic image and the first reconstructed image of the area, and the unstained histopathological section was used to obtain the second reconstructed image of the suspicious lesion area.
  • the signature of the microscopic image includes the tumor type in the image, the degree of tumor malignancy, and the cancer type.
  • Step b Group the datasets as training datasets and test datasets
  • Step c input the training data set into the deep learning network for training and network parameter optimization, and the deep learning network includes a high-resolution image reconstruction network based on a confrontational generation network and an image analysis network based on a deep convolutional neural network;
  • Step d Repeat step c until the deep learning network reaches a preset number of training iterations, or the error is less than a preset threshold, and a pathological slice analysis model is obtained;
  • Step e Use the test data set to test the performance of the pathological section analysis model to determine the performance of the pathological section analysis model.
  • the large field of view, high-throughput, and high-resolution pathological slice analysis system of this embodiment can analyze and judge the tumor type, tumor malignancy, and cancer type of the pathological slice without staining the pathological slice, which simplifies the analysis of the pathological slice. Analysis process.
  • the deep learning network is used to analyze the pathological situation, without changing the existing instrument structure, and intelligently analyze the pathological image, which avoids the errors caused by a large number of repeated operations by doctors and the inconsistency in the judgment of the pathological situation caused by the subjective factors of different doctors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Optics & Photonics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Computational Linguistics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Molecular Biology (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)

Abstract

一种大视场高通量高分辨病理切片分析仪，包括：照明模块（1），用于产生单色光；病理切片固定模块（2），用于固定并调整病理切片位置，以使病理切片位于成像视野位置处；数据采集模块（3），用于采集单色光经过病理切片后携带波前信息的散射光，与未携带病理切片信息的透射光形成的干涉图像信息；控制处理模块（4），用于对干涉图像信息进行图像重建，得到重建图像，基于预先训练完成的病理切片分析模型，对重建图像进行分析，得到病理切片的分析结果。该病理切片分析仪的成像视野为传统光学显微镜的数百倍，不需要对病理切片进行染色，采用深度学习网络分析病理情况，简化了病理切片的分析过程。

Description

大视场高通量高分辨病理切片分析仪 技术领域
本发明属于医学影像技术领域，具体涉及一种大视场高通量高分辨病理切片分析仪。
背景技术
病理切片显微特征的光学检查是诊断疾病的金标准之一，病理切片是取一定大小的病变组织，用病理组织学方法制成病理切片，通常将病变组织包埋在石蜡块里，用切片机切成薄片，再用苏木精-伊红染色，用显微镜进一步检查病变，以及其发生发展过程，最后作出病理诊断。
在通过病理切片进行诊断时，需要使用切片系统对切片进行观察，然后对病理切片进行分析，现有的病理切片成像方式和病理切片分析过程还存在以下不足：(1)传统光学显微技术视野与分辨率互相限制，获取高分辨图像必然伴随着一个较小的视野，在需要大量获取数据时会消耗大量的时间；此外，医生对病理切片的标记是一个重复性高的过程，长时间进行不能保证标记的准确性；(2)病理学检查流程中为了更好观察需要对切片染色，该过程是不可逆且耗时的；(3)受激拉曼显微成像技术可以为未经处理的生物组织提供快速、免标记、亚微米分辨率成像，但是受激拉曼显微成像技术视野较小，系统复杂，不适合于临床，虽然计算显微成像技术成像视野大，但是超分辨过程需要消耗大量时间。
发明内容
为了解决现有技术中存在的上述问题，本发明提供了一种大视场高通量高分辨病理切片分析仪。本发明要解决的技术问题通过以下技术方案实现：
本发明提供了一种大视场高通量高分辨病理切片分析仪，包括：
照明模块，用于产生单色光；
病理切片固定模块，用于固定并调整病理切片位置，以使所述病理切片位于成像视野位置处；
数据采集模块，用于采集单色光经过所述病理切片后携带波前信息的散射光，与未携带所述病理切片信息的透射光形成的干涉图像信息；
控制处理模块，用于对所述干涉图像信息进行图像重建，得到重建图像，基于预先训练完成的病理切片分析模型，对所述重建图像进行分析，得到所述病理切片的分析结果。
在本发明的一个实施例中，所述照明模块包括单色光源以及位于其下方的微孔结构，所述单色光源输出波长可调的单色光，所述微孔结构用于对所述单色光进行空间滤波，并且使其发生衍射。
在本发明的一个实施例中，所述病理切片固定模块包括病理切片固定器和病理切片位置调整器，所述病理切片固定器用于固定所述病理切片，所述病理切片位置调整器用于调整所述病理切片位置，以使所述病理切片位于成像视野位置处。
在本发明的一个实施例中，所述数据采集模块包括像元探测器，所述像元探测器的成像视场＞20mm 2，所述像元探测器的单个像元尺寸＜1.4μm。
在本发明的一个实施例中，所述控制处理模块包括控制单元和数据处理单元，所述控制单元用于控制所述照明模块产生单色光，控制所述病理 切片固定模块使所述病理切片位于成像视野位置处，以及控制数据采集模块采集所述干涉图像信息；
所述数据处理单元利用结构张量的图像自聚焦算法对所述干涉图像信息进行搜寻得到重建参数，利用角谱传播理论根据所述重建参数对所述干涉图像信息进行图像重建，得到所述重建图像，利用基于生成对抗网络的图像高分辨算法提高所述重建图像的分辨率，利用基于深度卷积神经网络的图像分析网路，对所述重建图像进行分析，判断所述病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
在本发明的一个实施例中，所述数据处理单元包括重建参数获取子单元、图像重建子单元和图像分析子单元，其中，
所述重建参数获取子单元，用于根据结构张量的图像自聚焦算法对所述干涉图像信息进行搜寻得到重建参数，所述重建参数为所述病理切片与数据采集模块之间的距离z；
所述图像重建子单元，用于根据所述重建参数利用角谱传播理论对所述干涉图像信息进行图像重建，得到重建图像；
所述图像分析子单元，用于基于预先训练完成的病理切片分析模型，对所述重建图像进行分析，得到所述病理切片的分析结果；
其中，所述分析结果包括所述病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
在本发明的一个实施例中，所述重建参数获取子单元用于：
根据先验信息确定病理切片样本与所述数据采集模块的距离的搜索范围以及搜索步长；
根据下式计算每个搜索步长的图像结构张量：
Figure PCTCN2021102379-appb-000001
其中，U(x,y)表示数据采集模块采集的干涉图像信息，
Figure PCTCN2021102379-appb-000002
表示在x、y方向的2D空间梯度，G(x,y)表示非负卷积核，
Figure PCTCN2021102379-appb-000003
表示梯度算子，T表示转置；
根据计算得到的图像结构张量，绘制结构张量曲线，寻找最值点，所述最值点作为所述病理切片与所述数据采集模块的距离z。
在本发明的一个实施例中，所述图像重建子单元用于根据角谱传播理论，按照下述的图像重建公式进行图像重建，获取所述重建图像：
Figure PCTCN2021102379-appb-000004
其中，ξ表示傅里叶变换，ξ -1表示傅里叶反变换，i表示复数符号，k＝2π/λ，λ表示单色光波长，f x,f y表示空间频率，z表示病理切片与数据采集模块的距离。
在本发明的一个实施例中，所述病理切片分析模型的训练方法包括：
步骤a：获取数据集，所述数据集包括若干不同可疑病变区域的组织病理切片图像，每个可疑病变区域的组织病理切片图像包括该可疑病变区域的带有标记的显微图像、第一重建图像和第二重建图像，其中，
所述带有标记的显微图像为对可疑病变区域的组织病理切片进行苏木精-伊红染色后使用临床显微镜获取组织病理切片全片的显微图像，再对所述显微图像进行标记得到的；
所述第一重建图像为对可疑病变区域的组织病理切片进行苏木精-伊红染色后使用所述大视场高通量高分辨病理切片分析仪获取的；
所述第二重建图像为对未进行苏木精-伊红染色的可疑病变区域的组织病理切片使用所述大视场高通量高分辨病理切片分析仪获取的；
步骤b：将所述数据集进行分组作为训练数据集和测试数据集；
步骤c：将所述训练数据集输入深度学习网络进行训练和网络参数优化，所述深度学习网络包括基于对抗生成网络的高分辨率图像重建网络和基于深度卷积神经网络的图像分析网络；
步骤d：重复步骤c直至所述深度学习网络达到预设的训练迭代次数，或者误差小于预设阈值，得到病理切片分析模型；
步骤e：利用所述测试数据集对所述病理切片分析模型进行性能测试，确定所述病理切片分析模型的性能。
与现有技术相比，本发明的有益效果在于：
1.本发明的大视场高通量高分辨病理切片分析仪，大视野、高通量成像，成像视野为传统光学显微镜的数百倍，采用像元探测器收集信号，成像视野等于像元探测器的活跃区域大小，而且能够快速获取图像。
2.本发明的大视场高通量高分辨病理切片分析仪，由于不依靠物镜获取图像，极大降低了仪器的成本消耗，适合用于临床诊断，尤其是资源较匮乏的地区。
3.本发明的大视场高通量高分辨病理切片分析仪，采用深度学习网络分析病理情况，不需要改变现有的仪器结构，智能地对病理图像分析，避免了医生大量重复操作带来的误差以及不同医生主观因素导致对病理情况判断不一致。
4.本发明的大视场高通量高分辨病理切片分析仪，不需要对病理切片进行染色，就可以对病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型进 行分析判断，简化了病理切片的分析过程。
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例，并配合附图，详细说明如下。
附图说明
图1是本发明实施例提供的一种大视场高通量高分辨病理切片分析仪的结构框图；
图2是本发明实施例提供的另一种大视场高通量高分辨病理切片分析仪的结构框图；
图3是本发明实施例提供的一种大视场高通量高分辨病理切片分析仪的结构示意图。
图标：1-照明模块；101-单色光源；102-微孔结构；2-病理切片固定模块；201-病理切片固定器；202-病理切片位置调整器；3-数据采集模块；301-像元探测器；4-控制处理模块；401-控制单元；402-数据处理单元；4021-重建参数获取子单元；4022-图像重建子单元；4023-图像分析子单元。
具体实施方式
为了进一步阐述本发明为达成预定发明目的所采取的技术手段及功效，以下结合附图及具体实施方式，对依据本发明提出的一种大视场高通量高分辨病理切片分析仪及分析方法进行详细说明。
有关本发明的前述及其他技术内容、特点及功效，在以下配合附图的具体实施方式详细说明中即可清楚地呈现。通过具体实施方式的说明，可对本发明为达成预定目的所采取的技术手段及功效进行更加深入且具体地 了解，然而所附附图仅是提供参考与说明之用，并非用来对本发明的技术方案加以限制。
实施例一
请参见图1，图1是本发明实施例提供的一种大视场高通量高分辨病理切片分析仪的结构框图。如图所示，本发明实施例的大视场高通量高分辨病理切片分析仪，包括：
照明模块1，用于产生单色光；
病理切片固定模块2，用于固定并调整病理切片位置，以使所述病理切片位于成像视野位置处；
数据采集模块3，用于采集单色光经过所述病理切片后携带波前信息的散射光，与未携带所述病理切片信息的透射光形成的干涉图像信息；
控制处理模块4，用于对所述干涉图像信息进行图像重建，得到重建图像，基于预先训练完成的病理切片分析模型，对所述重建图像进行分析，得到所述病理切片的分析结果。
本实施例的大视场高通量高分辨病理切片分析仪，由于不依靠物镜获取图像，极大降低了仪器的成本消耗，适合用于临床诊断，尤其是资源较匮乏的地区。
进一步地，请结合参见图2和图3，图2是本发明实施例提供的另一种大视场高通量高分辨病理切片分析仪的结构框图；图3是本发明实施例提供的一种大视场高通量高分辨病理切片分析仪的结构示意图。如图所示，具体地，照明模块1包括单色光源101以及位于其下方的微孔结构102，单色光源101输出波长可调的单色光，微孔结构102用于对单色光进行空间滤波，并且使其发生衍射。可选地，单色光源101为激光二极管。
进一步地，病理切片固定模块2位于照明模块1的下方，病理切片固定模块2包括病理切片固定器201和病理切片位置调整器202，病理切片固定器201用于固定病理切片，病理切片位置调整器202用于调整病理切片位置，以使病理切片位于成像视野位置处。
进一步地，数据采集模块3位于病理切片固定模块2的下方，数据采集模块3包括像元探测器301，像元探测器的成像视场＞20mm 2，像元探测器的单个像元尺寸＜1.4μm。可选地，像元探测器301可以是线阵CCD、面阵CCD或CMOS。
由于线阵CCD获取信息少，需要机械扫描操作增加获取的信息量，面阵CCD信息输出速率较慢且价格昂贵，而CMOS具有较低价格，且阵列的面积较大，因此，在本实施例中选取CMOS作为像元探测器301。另外，CMOS成像，其成像视野通常可超过20mm 2，而且成像速度快，通常可达10-20fps。
在本实施例中，单色光经过病理切片后携带波前信息的散射光，与未携带所述病理切片信息的透射光在CMOS平面形成光干涉信号，CMOS将该光干涉信号转化为电信号，传输至控制处理模块4。
进一步地，控制处理模块4包括控制单元401和数据处理单元402，其中，控制单元401分别与单色光源101、病理切片位置调整器202以及像元探测器301连接。
控制单元401用于控制单色光源101输出单色光，控制病理切片位置调整器202使病理切片位于最佳成像视野位置处，以及控制像元探测器301采集干涉图像信息。
数据处理单元402与像元探测器301连接，数据处理单元402利用结 构张量的图像自聚焦算法对接收到的干涉图像信息进行搜寻得到重建参数，利用角谱传播理论根据重建参数对干涉图像信息进行图像重建，得到重建图像，利用基于生成对抗网络的图像高分辨算法提高重建图像的分辨率，利用基于深度卷积神经网络的图像分析网路，对重建图像进行分析，判断病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
本实施例的大视场高通量高分辨病理切片分析仪，大视野、高通量成像，成像视野为传统光学显微镜的数百倍，采用像元探测器收集信号，成像视野等于像元探测器的活跃区域大小，而且能够快速获取图像。
进一步地，数据处理单元402包括重建参数获取子单元4021、图像重建子单元4022和图像分析子单元4023，其中，
重建参数获取子单元4021，用于根据结构张量的图像自聚焦算法对干涉图像信息进行搜寻得到重建参数，重建参数为病理切片与数据采集模块之间的距离z；
图像重建子单元4022，用于根据重建参数利用角谱传播理论对干涉图像信息进行图像重建，得到重建图像；
图像分析子单元4023，用于基于预先训练完成的病理切片分析模型，对重建图像进行分析，得到病理切片的分析结果；
在本实施例中，病理切片分析模型的输入数据为病理切片的重建图像，病理切片分析模型的输出数据为病理切片的分析结果，分析结果包括病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
具体地，重建参数获取子单元4021用于：根据先验信息确定病理切片样本与数据采集模块的距离的搜索范围以及搜索步长；
在本实施例中，先验信息为搭建大视场高通量高分辨病理切片分析仪 时病理切片样本到像元探测器之间的距离估计值。
根据下式计算每个搜索步长的图像结构张量：
Figure PCTCN2021102379-appb-000005
其中，U(x,y)表示数据采集模块采集的干涉图像信息，
Figure PCTCN2021102379-appb-000006
表示在x、y方向的2D空间梯度，G(x,y)表示非负卷积核，通常选择为一个二维高斯函数，
Figure PCTCN2021102379-appb-000007
表示梯度算子，T表示转置；
根据计算得到的图像结构张量，绘制结构张量曲线，寻找最值点，最值点作为病理切片与数据采集模块的距离z。
进一步地，图像重建子单元4022用于根据角谱传播理论，按照下述的图像重建公式进行图像重建，获取所述重建图像：
Figure PCTCN2021102379-appb-000008
其中，ξ表示傅里叶变换，ξ -1表示傅里叶反变换，i表示复数符号，k＝2π/λ，λ表示单色光波长，f x,f y表示空间频率，z表示病理切片与数据采集模块的距离。
进一步地，在本实施例中，病理切片分析模型的训练方法包括：
步骤a：获取数据集，数据集包括若干不同可疑病变区域的组织病理切片图像，每个可疑病变区域的组织病理切片图像包括该可疑病变区域的带有标记的显微图像、第一重建图像和第二重建图像，其中，
带有标记的显微图像为对可疑病变区域的组织病理切片进行苏木精-伊红染色后使用临床显微镜获取组织病理切片全片的显微图像，再对显微图像进行标记得到的；
第一重建图像为对可疑病变区域的组织病理切片进行苏木精-伊红染色后使用大视场高通量高分辨病理切片分析仪获取的；
第二重建图像为对未进行苏木精-伊红染色的可疑病变区域的组织病理切片使用大视场高通量高分辨病理切片分析仪获取的；
具体地，每个可疑病变区域的组织病理切片图像的获取方法包括：在可疑病变区域的同一位置连续切取两片组织病理切片，其中一片进行苏木精-伊红染色，一片不进行苏木精-伊红染色。首先，使用临床显微镜结合空间平移，获取经过苏木精-伊红染色的组织病理切片全片的高分辨率图像，也就是显微图像，然后由经验丰富临床医师对该显微图像进行标记，得到带有标记的显微图像。其次，使用本实施例的大视场高通量高分辨病理切片分析仪，获取经过苏木精-伊红染色的组织病理切片的重建图像，作为第一重建图像。最后，使用本实施例的大视场高通量高分辨病理切片分析仪，获取未经苏木精-伊红染色的病理切片的重建图像，作为第二重建图像。
即就是在可疑病变区域的同一位置连续切取两片组织病理切片，一片进行苏木精-伊红染色，一片不进行苏木精-伊红染色，进行染色的组织病理切片用于获取该可疑病变区域的带有标记的显微图像和第一重建图像，未进行染色的组织病理切片用于获取该可疑病变区域的第二重建图像。
在本实施例中，需要采集不少于200例的不同可疑病变区域的组织病理切片图像。显微图像的标记包括图像中的肿瘤类型、肿瘤恶性程度以及癌症分型。
步骤b：将数据集进行分组作为训练数据集和测试数据集；
步骤c：将训练数据集输入深度学习网络进行训练和网络参数优化，深度学习网络包括基于对抗生成网络的高分辨率图像重建网络和基于深度卷 积神经网络的图像分析网络；
步骤d：重复步骤c直至深度学习网络达到预设的训练迭代次数，或者误差小于预设阈值，得到病理切片分析模型；
步骤e：利用测试数据集对病理切片分析模型进行性能测试，确定病理切片分析模型的性能。
本实施例的大视场高通量高分辨病理切片分析系统，不需要对病理切片进行染色，就可以对病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型进行分析判断，简化了病理切片的分析过程。另外，采用深度学习网络分析病理情况，不需要改变现有的仪器结构，智能地对病理图像分析，避免了医生大量重复操作带来的误差以及不同医生主观因素导致对病理情况判断不一致。
应当说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的物品或者设备中还存在另外的相同要素。“连接”或者“相连”等类似的词语并非限定于物理的或者机械的连接，而是可以包括电性的连接，不管是直接的还是间接的。“上”、“下”、“左”、“右”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不 能理解为对本发明的限制。
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (9)

  1. 一种大视场高通量高分辨病理切片分析仪，其特征在于，包括：
    照明模块，用于产生单色光；
    病理切片固定模块，用于固定并调整病理切片位置，以使所述病理切片位于成像视野位置处；
    数据采集模块，用于采集单色光经过所述病理切片后携带波前信息的散射光，与未携带所述病理切片信息的透射光形成的干涉图像信息；
    控制处理模块，用于对所述干涉图像信息进行图像重建，得到重建图像，基于预先训练完成的病理切片分析模型，对所述重建图像进行分析，得到所述病理切片的分析结果。
  2. 根据权利要求1所述的大视场高通量高分辨病理切片分析仪，其特征在于：所述照明模块包括单色光源以及位于其下方的微孔结构，所述单色光源输出波长可调的单色光，所述微孔结构用于对所述单色光进行空间滤波，并且使其发生衍射。
  3. 根据权利要求1所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述病理切片固定模块包括病理切片固定器和病理切片位置调整器，所述病理切片固定器用于固定所述病理切片，所述病理切片位置调整器用于调整所述病理切片位置，以使所述病理切片位于成像视野位置处。
  4. 根据权利要求1所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述数据采集模块包括像元探测器，所述像元探测器的成像视场＞20mm 2，所述像元探测器的单个像元尺寸＜1.4μm。
  5. 根据权利要求1所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述控制处理模块包括控制单元和数据处理单元，所述控制单元用于控制所述照明模块产生单色光，控制所述病理切片固定模块使所述病 理切片位于成像视野位置处，以及控制数据采集模块采集所述干涉图像信息；
    所述数据处理单元利用结构张量的图像自聚焦算法对所述干涉图像信息进行搜寻得到重建参数，利用角谱传播理论根据所述重建参数对所述干涉图像信息进行图像重建，得到所述重建图像，利用基于生成对抗网络的图像高分辨算法提高所述重建图像的分辨率，利用基于深度卷积神经网络的图像分析网路，对所述重建图像进行分析，判断所述病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
  6. 根据权利要求5所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述数据处理单元包括重建参数获取子单元、图像重建子单元和图像分析子单元，其中，
    所述重建参数获取子单元，用于根据结构张量的图像自聚焦算法对所述干涉图像信息进行搜寻得到重建参数，所述重建参数为所述病理切片与数据采集模块之间的距离z；
    所述图像重建子单元，用于根据所述重建参数利用角谱传播理论对所述干涉图像信息进行图像重建，得到重建图像；
    所述图像分析子单元，用于基于预先训练完成的病理切片分析模型，对所述重建图像进行分析，得到所述病理切片的分析结果；
    其中，所述分析结果包括所述病理切片的肿瘤类型、肿瘤恶性程度以及癌症分型。
  7. 根据权利要求6所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述重建参数获取子单元用于：
    根据先验信息确定病理切片样本与所述数据采集模块的距离的搜索范 围以及搜索步长；
    根据下式计算每个搜索步长的图像结构张量：
    Figure PCTCN2021102379-appb-100001
    其中，U(x,y)表示数据采集模块采集的干涉图像信息，
    Figure PCTCN2021102379-appb-100002
    表示在x、y方向的2D空间梯度，G(x,y)表示非负卷积核，
    Figure PCTCN2021102379-appb-100003
    表示梯度算子，T表示转置；
    根据计算得到的图像结构张量，绘制结构张量曲线，寻找最值点，所述最值点作为所述病理切片与所述数据采集模块的距离z。
  8. 根据权利要求7所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述图像重建子单元用于根据角谱传播理论，按照下述的图像重建公式进行图像重建，获取所述重建图像：
    Figure PCTCN2021102379-appb-100004
    其中，ξ表示傅里叶变换，ξ -1表示傅里叶反变换，i表示复数符号，k＝2π/λ，λ表示单色光波长，f x,f y表示空间频率，z表示病理切片与数据采集模块的距离。
  9. 根据权利要求6所述的大视场高通量高分辨病理切片分析仪，其特征在于，所述病理切片分析模型的训练方法包括：
    步骤a：获取数据集，所述数据集包括若干不同可疑病变区域的组织病理切片图像，每个可疑病变区域的组织病理切片图像包括该可疑病变区域的带有标记的显微图像、第一重建图像和第二重建图像，其中，
    所述带有标记的显微图像为对可疑病变区域的组织病理切片进行苏木 精-伊红染色后使用临床显微镜获取组织病理切片全片的显微图像，再对所述显微图像进行标记得到的；
    所述第一重建图像为对可疑病变区域的组织病理切片进行苏木精-伊红染色后使用所述大视场高通量高分辨病理切片分析仪获取的；
    所述第二重建图像为对未进行苏木精-伊红染色的可疑病变区域的组织病理切片使用所述大视场高通量高分辨病理切片分析仪获取的；
    步骤b：将所述数据集进行分组作为训练数据集和测试数据集；
    步骤c：将所述训练数据集输入深度学习网络进行训练和网络参数优化，所述深度学习网络包括基于对抗生成网络的高分辨率图像重建网络和基于深度卷积神经网络的图像分析网络；
    步骤d：重复步骤c直至所述深度学习网络达到预设的训练迭代次数，或者误差小于预设阈值，得到病理切片分析模型；
    步骤e：利用所述测试数据集对所述病理切片分析模型进行性能测试，确定所述病理切片分析模型的性能。
PCT/CN2021/102379 2020-12-07 2021-06-25 大视场高通量高分辨病理切片分析仪 WO2022121284A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/104,812 US20230177645A1 (en) 2020-12-07 2023-02-02 Pathological section analyzer with large field of view, high throughput and high resolution

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011420323.8A CN112712877B (zh) 2020-12-07 2020-12-07 大视场高通量高分辨病理切片分析仪
CN202011420323.8 2020-12-07

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/104,812 Continuation US20230177645A1 (en) 2020-12-07 2023-02-02 Pathological section analyzer with large field of view, high throughput and high resolution

Publications (1)

Publication Number Publication Date
WO2022121284A1 true WO2022121284A1 (zh) 2022-06-16

Family

ID=75542675

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102379 WO2022121284A1 (zh) 2020-12-07 2021-06-25 大视场高通量高分辨病理切片分析仪

Country Status (3)

Country Link
US (1) US20230177645A1 (zh)
CN (1) CN112712877B (zh)
WO (1) WO2022121284A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117970618A (zh) * 2024-03-28 2024-05-03 西北工业大学 一种动态高分辨率表面等离子体共振全息显微成像方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112712877B (zh) * 2020-12-07 2024-02-09 西安电子科技大学 大视场高通量高分辨病理切片分析仪

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103322941A (zh) * 2013-07-09 2013-09-25 河北工程大学 一种准确获取三维显微图像的方法
US20130274119A1 (en) * 2010-05-25 2013-10-17 Arryx, Inc. Methods and apparatuses for detection of positional freedom of particles in biological and chemical analyses and applications in immunodiagnostics
CN107064019A (zh) * 2017-05-18 2017-08-18 西安交通大学 用于无染色病理切片高光谱图像采集及分割的装置及方法
CN107967703A (zh) * 2018-01-05 2018-04-27 电子科技大学 一种基于连通域的光学扫描全息的自聚焦和重建方法
CN109240064A (zh) * 2018-08-06 2019-01-18 浙江工业大学 基于am335x处理器的便携式数字全息显微系统
CN110927115A (zh) * 2019-12-09 2020-03-27 杭州电子科技大学 基于深度学习的无透镜双型融合目标检测装置及方法
CN111833348A (zh) * 2020-08-10 2020-10-27 上海工程技术大学 一种基于图像处理的血管沉积物自动检测方法
CN112712877A (zh) * 2020-12-07 2021-04-27 西安电子科技大学 大视场高通量高分辨病理切片分析仪

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110570379B (zh) * 2019-09-11 2023-03-24 重庆大学 一种基于结构张量的非局部均值ct图像降噪方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130274119A1 (en) * 2010-05-25 2013-10-17 Arryx, Inc. Methods and apparatuses for detection of positional freedom of particles in biological and chemical analyses and applications in immunodiagnostics
CN103322941A (zh) * 2013-07-09 2013-09-25 河北工程大学 一种准确获取三维显微图像的方法
CN107064019A (zh) * 2017-05-18 2017-08-18 西安交通大学 用于无染色病理切片高光谱图像采集及分割的装置及方法
CN107967703A (zh) * 2018-01-05 2018-04-27 电子科技大学 一种基于连通域的光学扫描全息的自聚焦和重建方法
CN109240064A (zh) * 2018-08-06 2019-01-18 浙江工业大学 基于am335x处理器的便携式数字全息显微系统
CN110927115A (zh) * 2019-12-09 2020-03-27 杭州电子科技大学 基于深度学习的无透镜双型融合目标检测装置及方法
CN111833348A (zh) * 2020-08-10 2020-10-27 上海工程技术大学 一种基于图像处理的血管沉积物自动检测方法
CN112712877A (zh) * 2020-12-07 2021-04-27 西安电子科技大学 大视场高通量高分辨病理切片分析仪

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117970618A (zh) * 2024-03-28 2024-05-03 西北工业大学 一种动态高分辨率表面等离子体共振全息显微成像方法

Also Published As

Publication number Publication date
CN112712877B (zh) 2024-02-09
US20230177645A1 (en) 2023-06-08
CN112712877A (zh) 2021-04-27

Similar Documents

Publication Publication Date Title
JP6333326B2 (ja) スペクトルイメージングによる生体試料の分析方法
CN109670510B (zh) 一种基于深度学习的胃镜活检病理数据筛查系统
JP6366556B2 (ja) 生物学的試片をスペクトル画像により分析する方法
US10229310B2 (en) High throughput partial wave spectroscopic microscopy and associated systems and methods
US6181414B1 (en) Infrared spectroscopy for medical imaging
Bredfeldt et al. Computational segmentation of collagen fibers from second-harmonic generation images of breast cancer
US20180108163A1 (en) Method for analyzing biological specimens by spectral imaging
WO2020087966A1 (zh) 基于led阵列编码照明的三维衍射层析显微成像方法
WO2022121284A1 (zh) 大视场高通量高分辨病理切片分析仪
EP3009832A1 (en) A cytological system and method for analyzing a biological sample by raman spectroscopy
BR112021011719A2 (pt) Sistema baseado em microscopia computacional e método para geração de imagem automatizada e análise de espécimes de patologia
KR20140104946A (ko) 스펙트럼 이미징에 의해 생물학적 표본을 분석하는 방법 및 시스템
CN109297949B (zh) 显微影像结合透射拉曼光谱的肿瘤细胞检测方法及装置
CN112798504B (zh) 大视场高通量流式细胞分析系统及分析方法
CA2279945A1 (en) Infrared spectroscopy for medical imaging
Gradov Lensless holographic trichoscopy and trichometry [for medical specialists]
AU2017204736A1 (en) Method for analyzing biological specimens by spectral imaging
WO2023119267A1 (en) Method and system for visualization of the structure of biological cells
Wang et al. Classification of Benign–Malignant Thyroid Nodules Based on Hyperspectral Technology
Sindhoora et al. Machine learning aided classification and grading of biopsy sample using discrete wavelet transform and gray level co-occurrence matrix
CN117036269A (zh) 一种基于尿液样品高光谱图像的肾小球疾病分类诊断模型构建方法
Tang Label-free Investigation of Cells with Phase Imaging, Diffraction Tomography, and Raman Spectroscopy

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21901996

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21901996

Country of ref document: EP

Kind code of ref document: A1