WO2019191697A1 - Method and system for digital staining of label-free fluorescence images using deep learning - Google Patents
Method and system for digital staining of label-free fluorescence images using deep learning Download PDFInfo
- Publication number
- WO2019191697A1 WO2019191697A1 PCT/US2019/025020 US2019025020W WO2019191697A1 WO 2019191697 A1 WO2019191697 A1 WO 2019191697A1 US 2019025020 W US2019025020 W US 2019025020W WO 2019191697 A1 WO2019191697 A1 WO 2019191697A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- sample
- trained
- neural network
- fluorescence
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H70/00—ICT specially adapted for the handling or processing of medical references
- G16H70/60—ICT specially adapted for the handling or processing of medical references relating to pathologies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Definitions
- H&E pseudo-Hematoxylin and Eosin
- a system and method are provided that utilizes a trained deep neural network that is used for the digital or virtual staining of label-free thin tissue sections or other samples using their fluorescence images obtained from chemically unstained tissue (or other samples).
- Chemically unstained tissue refers to the lack of standard stains or labels used in histochemical staining of tissue.
- the fluorescence of chemically unstained tissue may include auto-fluorescence of tissue from naturally occurring or endogenous fluorophores or other endogenous emitters of light at frequencies different from the illumination frequency (i.e., frequency-shifted light). Fluorescence of chemically unstained tissue may further include fluorescence of tissue from exogenously added fluorescent labels or other exogenous emitters of light.
- Samples are imaged with a fluorescence microscope such as a wi de-field fluorescence microscope (or a standard fluorescence microscope).
- the microscope may utilize a standard near-UV excitation/emission filter set or other excitation/emission light source/filter sets that are known to those skilled in the art.
- the digital or virtual staining is performed, in some embodiments, on a single fluorescence image obtained of the sample by using, in on preferred embodiment, a trained deep neural network.
- the network inference performed by the trained neural network is fast, taking in some embodiments, less than a second using a standard desktop computer for an imaging field-of-view of ⁇ 0.33 mm x 0.33 mm using e.g., a 40x objective lens.
- a 20x objective for scanning tissue a network inference time of 1.9 seconds/mm 2 was achieved.
- the deep learning-based digital/ virtual histology staining method using auto fluorescence has been demonstrated by imaging label-free human tissue samples including salivary gland, thyroid, kidney, liver, lung and skin, where the trained deep neural network output created equivalent images, substantially matching with the images of the same samples that were labeled with three different stains, i.e., H&E (salivary gland and thyroid), Jones stain (kidney) and Masson’s Tri chrome (liver and lung). Because the trained deep neural network’s input image is captured by a conventional fluorescence microscope with a standard filter set, this approach has transformative potential to use unstained tissue samples for pathology and histology applications, entirely bypassing the histochemical staining process, saving time and the attendant costs.
- a method of generating a digitally stained microscopic image of a label-free sample includes providing a trained, deep neural network that is run using image processing software executed using one or more processors of a computing device, wherein the trained, deep neural network is trained with a plurality of matched chemically stained images or image patches and their corresponding fluorescence images or image patches of the same sample.
- the label-free sample may include tissues, cells, pathogens, biological fluid smears, or other micro-objects of interest.
- the deep neural network may be trained using one or more tissue type/chemical stain type
- this may include tissue type A with stain #1, stain #2, stain #3, etc.
- the deep neural network may be trained using tissue that has been stained with multiple stains.
- one fluorescence image may be obtained at a first filtered wavelength or wavelength range while another fluorescence image may be obtained at a second filtered wavelength or wavelength range.
- These two fluorescence images are then input into the trained, deep neural network to output a single digitally/virtually stained image.
- the obtained fluorescence image may be subject to one or more linear or non-linear pre-processing operations selected from contrast enhancement, contrast reversal, image filtering which may be input alone or in combination with the obtained fluorescence image into the trained, deep neural network.
- FIGS. 3G and 3H the H&E stains demonstrate infiltrating squamous cell carcinoma.
- the desmoplastic reaction with edematous myxoid change (asterisk in FIGS. 3G and 3H) in the adjacent stroma is clearly identifiable in both stains/panels.
- FIG. 6A illustrates a graph of combined loss function vs. number of iterations for random initialization and transfer learning initialization.
- FIG. 6A illustrates how superior convergence is achieved using transfer learning.
- a new deep neural network is initialized using the weights and biases learned from the salivary gland tissue sections to achieve virtual staining of thyroid tissue with H&E. Compared to random initialization, transfer learning enables much faster convergence, also achieving a lower local minimum.
- the results of the trained deep neural network 10 were quantified by first calculating the pixel- level differences between the brightfield images 48 of the chemically stained samples 22 and the digitally /virtually stained images 40 that are synthesized using the deep neural network 10 without the use of any labels/stains.
- Table 4 summarizes this comparison for different combinations of tissue types and stains, using the YCbCr color space, where the chroma components Cb and Cr entirely define the color, and Y defines the brightness component of the image.
- the system 2 and methods described herein show the ability to digitally /virtually stain label-free tissue sections 22, using a supervised deep learning technique that uses a single fluorescence image 20 of the sample as input, captured by a standard fluorescence microscope 110 and filter set (in other embodiments multiple fluorescence images 20 are input when multiple fluorescence channels are used).
- This statistical learning-based method has the potential to restructure the clinical workflow in histopathology and can benefit from various imaging modalities such as fluorescence microscopy, non-linear microscopy, holographic microscopy, stimulated Raman scattering microscopy, and optical coherence tomography, among others, to potentially provide a digital alternative to the standard practice of histochemical staining of tissue samples 22.
- an elastic image registration which matches the local features of both sets of images (auto-fluorescence 20b vs. brightfield 48e), by hierarchically matching the corresponding blocks, from large to small.
- a neural network 71 is used to learn the transformation between the roughly matched images. This network 71 uses the same structure as the network 10 in FIG. 10. A low number of iterations is used so that the network 71 only leams the accurate color mapping, and not any spatial transformations between the input and label images. The calculated transformation map from this step is finally applied to each brightfield image patch 48e.
- the generator loss function balances the pixel-wise mean squared error (MSE) of the generator network output image with respect to its label, the total variation (TV) operator of the output image, and the discriminator network prediction of the output image, using the regularization parameters (/.. a) that are empirically set to different values, which
- the up-sampling path consists of four, symmetric, up-sampling steps (#1, #2, #3, #4), with each step containing one convolutional block.
- the convolutional block operation which maps feature map y k into feature map y k+1 , is given by:
Abstract
Description
Claims
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP19776693.4A EP3776342A4 (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescence images using deep learning |
CN201980029172.2A CN112106061A (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of unlabeled fluorescent images using deep learning |
CA3095739A CA3095739A1 (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescence images using deep learning |
BR112020019896-0A BR112020019896A2 (en) | 2018-03-30 | 2019-03-29 | METHOD AND SYSTEM FOR DIGITAL COLORING OF FLUORESCENCE IMAGES WITHOUT LABELS USING DEEP LEARNING |
US17/041,447 US11893739B2 (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescence images using deep learning |
JP2020552396A JP7344568B2 (en) | 2018-03-30 | 2019-03-29 | Method and system for digitally staining label-free fluorescent images using deep learning |
KR1020207031116A KR20200140301A (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescent images using deep learning |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862651005P | 2018-03-30 | 2018-03-30 | |
US62/651,005 | 2018-03-30 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/041,447 A-371-Of-International US11893739B2 (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescence images using deep learning |
US18/543,168 Continuation US20240135544A1 (en) | 2023-12-18 | Method and system for digital staining of label-free fluorescence images using deep learning |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019191697A1 true WO2019191697A1 (en) | 2019-10-03 |
Family
ID=68060845
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2019/025020 WO2019191697A1 (en) | 2018-03-30 | 2019-03-29 | Method and system for digital staining of label-free fluorescence images using deep learning |
Country Status (8)
Country | Link |
---|---|
US (1) | US11893739B2 (en) |
EP (1) | EP3776342A4 (en) |
JP (1) | JP7344568B2 (en) |
KR (1) | KR20200140301A (en) |
CN (1) | CN112106061A (en) |
BR (1) | BR112020019896A2 (en) |
CA (1) | CA3095739A1 (en) |
WO (1) | WO2019191697A1 (en) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020144012A (en) * | 2019-03-06 | 2020-09-10 | 学校法人 埼玉医科大学 | Stained image estimator learning device, image processing device, stained image estimator learning method, image processing method, stained image estimator learning program, and image processing program |
KR20210054183A (en) * | 2019-11-05 | 2021-05-13 | 주식회사 토모큐브 | Method and Apparatus for Generating 3D Fluorescent Label Image of Label-Free using 3D Refractive Index Tomography and Deep Learning |
WO2021076075A3 (en) * | 2019-10-14 | 2021-06-17 | İzmi̇r Yüksek Teknoloji̇ Ensti̇tüsü | Cell viability analysis and counting from holograms by using deep learning and appropriate lensless holographic microscope |
EP3839884A1 (en) * | 2019-12-20 | 2021-06-23 | Fundación Tecnalia Research & Innovation | Method, system and computer program product for image transformation |
WO2021133847A1 (en) * | 2019-12-23 | 2021-07-01 | The Regents Of The University Of California | Method and system for digital staining of microscopy images using deep learning |
WO2021198247A1 (en) * | 2020-03-30 | 2021-10-07 | Carl Zeiss Ag | Optimal co-design of hardware and software for virtual staining of unlabeled tissue |
US11262286B2 (en) | 2019-04-24 | 2022-03-01 | The Regents Of The University Of California | Label-free bio-aerosol sensing using mobile microscopy and deep learning |
EP4002267A1 (en) | 2020-11-19 | 2022-05-25 | Prospective Instruments GmbH | Imaging method and system for generating a digitally stained image, training method for training an artificial intelligence system, and non-transitory storage medium |
US11460395B2 (en) | 2019-06-13 | 2022-10-04 | The Regents Of The University Of California | System and method for measuring serum phosphate levels using portable reader device |
JP2022548760A (en) * | 2019-09-19 | 2022-11-21 | ザ ホンコン ユニヴァーシティ オブ サイエンス アンド テクノロジー | Slide-free histological imaging methods and systems |
EP4083907A4 (en) * | 2019-12-26 | 2024-01-17 | Japanese Found For Cancer Res | Pathological diagnosis assisting method using ai, and assisting device |
EP4070232A4 (en) * | 2019-12-05 | 2024-01-31 | Tempus Labs Inc | Systems and methods for high throughput drug screening |
US11946854B2 (en) | 2018-12-26 | 2024-04-02 | The Regents Of The University Of California | Systems and methods for two-dimensional fluorescence wave propagation onto surfaces using deep learning |
Families Citing this family (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7181001B2 (en) * | 2018-05-24 | 2022-11-30 | 日本電子株式会社 | BIOLOGICAL TISSUE IMAGE PROCESSING SYSTEM AND MACHINE LEARNING METHOD |
CN113167783A (en) | 2018-10-18 | 2021-07-23 | 加利福尼亚大学董事会 | Apparatus and method for motion-based label-free detection of moving objects in a fluid sample |
JP7370158B2 (en) * | 2019-04-03 | 2023-10-27 | 株式会社Preferred Networks | Information processing device and information processing method |
EP3953861A1 (en) | 2019-04-11 | 2022-02-16 | Agilent Technologies, Inc. | User interface configured to facilitate user annotation for instance segmentation within biological sample |
SG11202112935YA (en) * | 2019-05-24 | 2021-12-30 | PharmaNest LLC | Quantitative phenotyping of fibrosis from images of biological tissue samples |
GB2585005B (en) * | 2019-06-24 | 2021-06-23 | Roadmetric Ltd | Training a machine to recognize a motor vehicle driver using a mobile device |
US11423678B2 (en) | 2019-09-23 | 2022-08-23 | Proscia Inc. | Automated whole-slide image classification using deep learning |
DE102019130032B4 (en) * | 2019-11-07 | 2022-04-14 | Ford Global Technologies, Llc | Method, computer program product and system for generating an image data set for a computer-implemented simulation |
CN111474701B (en) * | 2019-12-19 | 2021-11-30 | 上海杏脉信息科技有限公司 | Pathological microscopic image real-time acquisition and analysis system, method, device and medium |
DE102020105697B3 (en) * | 2020-03-03 | 2021-03-04 | Carl Zeiss Meditec Ag | Computer-implemented method for finding possible artifacts in a virtually colored histological image |
US11189060B2 (en) * | 2020-04-30 | 2021-11-30 | Adobe Inc. | Generating procedural materials from digital images |
US11393182B2 (en) | 2020-05-29 | 2022-07-19 | X Development Llc | Data band selection using machine learning |
AU2021339584A1 (en) * | 2020-09-08 | 2023-05-18 | Insitro, Inc. | Biological image transformation using machine-learning models |
US11250199B1 (en) * | 2020-09-16 | 2022-02-15 | Center For Deep Learning In Electronics Manufacturing, Inc. | Methods and systems for generating shape data for electronic designs |
IL301593B1 (en) | 2020-09-23 | 2024-03-01 | Proscia Inc | Critical component detection using deep learning and attention |
KR102462847B1 (en) * | 2021-01-29 | 2022-11-04 | 한국광기술원 | Whole slide imaging device for correcting shadow distortion and method therefor |
EP4323951A1 (en) * | 2021-04-14 | 2024-02-21 | Ventana Medical Systems, Inc. | Transformation of histochemically stained images into synthetic immunohistochemistry (ihc) images |
CN113515077A (en) * | 2021-04-23 | 2021-10-19 | 重庆德方信息技术有限公司 | System and method for monitoring human body cell staining process |
US20220343473A1 (en) * | 2021-04-27 | 2022-10-27 | City University Of Hong Kong | System and method for generating a stained image |
DE102021204805A1 (en) * | 2021-05-11 | 2022-11-17 | Carl Zeiss Microscopy Gmbh | Evaluation method for simulation models in microscopy |
CN113469864A (en) * | 2021-06-28 | 2021-10-01 | 平湖莱顿光学仪器制造有限公司 | Method and device for acquiring microscopic image |
WO2023061162A1 (en) * | 2021-10-12 | 2023-04-20 | The Hong Kong University Of Science And Technology | Systems and methods for label-free multi-histochemical virtual staining |
CN114119609B (en) * | 2022-01-25 | 2022-04-26 | 深圳佑驾创新科技有限公司 | Method, device and equipment for detecting image stain concentration and storage medium |
WO2023154925A1 (en) * | 2022-02-14 | 2023-08-17 | Genetic Innovations, Inc. | Medical spectroscopy and imaging analysis |
CN114596298B (en) | 2022-03-16 | 2022-11-15 | 华东师范大学 | Hyperspectral imaging-based automatic generation method of fine-labeled digital pathological data set |
KR20230160440A (en) * | 2022-05-17 | 2023-11-24 | 계명대학교 산학협력단 | Device and method for digital pathological dyeing restoration using generative adversarial network |
WO2024036020A1 (en) * | 2022-08-10 | 2024-02-15 | Verily Life Sciences Llc | Generating images of virtually stained biological samples |
WO2024063119A1 (en) * | 2022-09-21 | 2024-03-28 | 国立大学法人大阪大学 | Image translation device, diagnostic imaging system, image translation method, control program, and recording medium |
CN115565698B (en) * | 2022-10-26 | 2024-03-29 | 南方医科大学珠江医院 | Method and system for evaluating kidney supply quality by artificial intelligence |
CN116503858B (en) * | 2023-05-06 | 2024-01-23 | 华东师范大学 | Immunofluorescence image classification method and system based on generation model |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040093166A1 (en) * | 2002-09-13 | 2004-05-13 | Kil David H. | Interactive and automated tissue image analysis with global training database and variable-abstraction processing in cytological specimen classification and laser capture microdissection applications |
US20120083678A1 (en) * | 2010-09-30 | 2012-04-05 | Chemimage Corporation | System and method for raman chemical analysis of lung cancer with digital staining |
US20130077875A1 (en) * | 2005-01-27 | 2013-03-28 | Richard Levenson | Classifying image features |
US20140270457A1 (en) * | 2013-03-15 | 2014-09-18 | The Board Of Trustees Of The University Of Illinois | Stain-free histopathology by chemical imaging |
EP2921990A2 (en) * | 2014-03-20 | 2015-09-23 | Rudjer Boskovic Institute | Method and apparatus for unsupervised segmentation of microscopic color image of unstained specimen and digital staining of segmented histological structures |
US20180286038A1 (en) * | 2015-09-23 | 2018-10-04 | The Regents Of The University Of California | Deep learning in label-free cell classification and machine vision extraction of particles |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1556379A (en) * | 2003-12-30 | 2004-12-22 | 天津大学 | Optical fiber continuous exciting microfluorescent light spectrum imaging instrument |
US8269827B2 (en) * | 2009-09-29 | 2012-09-18 | General Electric Company | System and methods for mapping fluorescent images into a bright field color space |
CN102143133B (en) | 2010-08-05 | 2013-12-18 | 华为技术有限公司 | Method, device and system for supporting advertisement content in hyper text transport protocol (HTTP) stream playing manner |
US8725237B2 (en) * | 2012-03-19 | 2014-05-13 | Genetic Innovations, Inc. | Devices, systems, and methods for virtual staining |
WO2013187148A1 (en) * | 2012-06-15 | 2013-12-19 | オリンパス株式会社 | Image processing device, microscope system, endoscope system, and image processing method |
US20150226743A1 (en) * | 2012-10-28 | 2015-08-13 | Clarient Diagnostics Services, Inc. | Multiplexed method for diagnosing classical hodgkin lymphoma |
WO2015168026A2 (en) | 2014-04-28 | 2015-11-05 | The Broad Institute, Inc. | Method for label-free image cytometry |
US9625387B2 (en) | 2014-09-16 | 2017-04-18 | Lawrence Livermore National Security, Llc | System and method for controlling depth of imaging in tissues using fluorescence microscopy under ultraviolet excitation following staining with fluorescing agents |
US9964489B2 (en) * | 2014-09-16 | 2018-05-08 | Lawrence Livermore National Security, Llc | System and method for controlling depth of imaging in tissues using fluorescence microscopy under ultraviolet excitation following staining with fluorescing agents |
US9767555B2 (en) * | 2015-01-05 | 2017-09-19 | Case Western Reserve University | Disease characterization from fused pathology and radiology data |
US9984462B2 (en) * | 2015-01-05 | 2018-05-29 | Case Western Reserve University | Disease characterization from fused pathology and radiology data |
US9971966B2 (en) | 2016-02-26 | 2018-05-15 | Google Llc | Processing cell images using neural networks |
WO2017196885A1 (en) | 2016-05-10 | 2017-11-16 | The Regents Of The University Of California | Method and device for high-resolution color imaging using merged images from holographic and lens-based devices |
JP6795624B2 (en) * | 2016-05-27 | 2020-12-02 | ヴェリリー ライフ サイエンシズ エルエルシー | Spatial Light Modulator Based Hyperspectral Confocal Microscope and Usage |
JP6810167B2 (en) * | 2016-05-27 | 2021-01-06 | ヴェリリー ライフ サイエンシズ エルエルシー | Systems and methods for 4D hyperspectral imaging |
WO2017216123A1 (en) | 2016-06-13 | 2017-12-21 | Nanolive Sa | Method of characterizing and imaging microscopic objects |
CA3021481A1 (en) | 2016-07-29 | 2018-02-01 | Novadaq Technologies ULC | Methods and systems for characterizing tissue of a subject utilizing machine learning |
US20180064409A1 (en) | 2016-09-05 | 2018-03-08 | Koninklijke Philips N.V. | Simultaneously displaying medical images |
WO2018156133A1 (en) | 2017-02-23 | 2018-08-30 | Google Llc | Method and system for assisting pathologist identification of tumor cells in magnified tissue images |
EP3593111A4 (en) | 2017-03-10 | 2020-02-26 | The Regents of the University of California, A California Corporation | Mobile microscopy system for air quality monitoring |
EP3662407A2 (en) | 2017-08-03 | 2020-06-10 | Nucleai Ltd | Systems and methods for analysis of tissue images |
WO2019032723A1 (en) | 2017-08-09 | 2019-02-14 | Allen Institute | Systems, devices, and methods for image processing to generate an image having predictive tagging |
WO2019118544A1 (en) * | 2017-12-15 | 2019-06-20 | Verily Life Sciences Llc | Generating virtually stained images of unstained samples |
US11403735B2 (en) * | 2018-01-25 | 2022-08-02 | King Abdullah University Of Science And Technology | Deep-learning based structure reconstruction method and apparatus |
US11222415B2 (en) * | 2018-04-26 | 2022-01-11 | The Regents Of The University Of California | Systems and methods for deep learning microscopy |
US11030744B2 (en) | 2018-06-26 | 2021-06-08 | Astrazeneca Computational Pathology Gmbh | Deep learning method for tumor cell scoring on cancer biopsies |
WO2020018154A1 (en) | 2018-07-19 | 2020-01-23 | The Regents Of The University Of California | Method and system for digital staining of label-free phase images using deep learning |
US11354804B1 (en) * | 2019-09-27 | 2022-06-07 | Verily Life Sciences Llc | Transforming multispectral images to enhanced resolution images enabled by machine learning |
BR112022011915A2 (en) * | 2019-12-23 | 2022-09-06 | Univ California | MICROSCOPIC IMAGES DIGITAL DYEING METHOD AND SYSTEM USING DEEP LEARNING |
-
2019
- 2019-03-29 CN CN201980029172.2A patent/CN112106061A/en active Pending
- 2019-03-29 EP EP19776693.4A patent/EP3776342A4/en active Pending
- 2019-03-29 BR BR112020019896-0A patent/BR112020019896A2/en unknown
- 2019-03-29 KR KR1020207031116A patent/KR20200140301A/en active Search and Examination
- 2019-03-29 CA CA3095739A patent/CA3095739A1/en active Pending
- 2019-03-29 JP JP2020552396A patent/JP7344568B2/en active Active
- 2019-03-29 US US17/041,447 patent/US11893739B2/en active Active
- 2019-03-29 WO PCT/US2019/025020 patent/WO2019191697A1/en unknown
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040093166A1 (en) * | 2002-09-13 | 2004-05-13 | Kil David H. | Interactive and automated tissue image analysis with global training database and variable-abstraction processing in cytological specimen classification and laser capture microdissection applications |
US20130077875A1 (en) * | 2005-01-27 | 2013-03-28 | Richard Levenson | Classifying image features |
US20120083678A1 (en) * | 2010-09-30 | 2012-04-05 | Chemimage Corporation | System and method for raman chemical analysis of lung cancer with digital staining |
US20140270457A1 (en) * | 2013-03-15 | 2014-09-18 | The Board Of Trustees Of The University Of Illinois | Stain-free histopathology by chemical imaging |
EP2921990A2 (en) * | 2014-03-20 | 2015-09-23 | Rudjer Boskovic Institute | Method and apparatus for unsupervised segmentation of microscopic color image of unstained specimen and digital staining of segmented histological structures |
US20180286038A1 (en) * | 2015-09-23 | 2018-10-04 | The Regents Of The University Of California | Deep learning in label-free cell classification and machine vision extraction of particles |
Non-Patent Citations (2)
Title |
---|
BAYRAMOGLU ET AL.: "Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks", IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW, 2017 - 22 October 2017 (2017-10-22), pages 64 - 71, XP055497287 * |
RIVENSON ET AL.: "Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue", NATURE BIOMEDICAL ENGINEERING, 30 March 2018 (2018-03-30) - 2019, XP081136022 * |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11946854B2 (en) | 2018-12-26 | 2024-04-02 | The Regents Of The University Of California | Systems and methods for two-dimensional fluorescence wave propagation onto surfaces using deep learning |
JP7470339B2 (en) | 2019-03-06 | 2024-04-18 | 学校法人 埼玉医科大学 | Dye image estimator learning device, image processing device, dye image estimator learning method, image processing method, dye image estimator learning program, and image processing program |
JP2020144012A (en) * | 2019-03-06 | 2020-09-10 | 学校法人 埼玉医科大学 | Stained image estimator learning device, image processing device, stained image estimator learning method, image processing method, stained image estimator learning program, and image processing program |
US11262286B2 (en) | 2019-04-24 | 2022-03-01 | The Regents Of The University Of California | Label-free bio-aerosol sensing using mobile microscopy and deep learning |
US11460395B2 (en) | 2019-06-13 | 2022-10-04 | The Regents Of The University Of California | System and method for measuring serum phosphate levels using portable reader device |
JP7431472B2 (en) | 2019-09-19 | 2024-02-15 | ザ ホンコン ユニヴァーシティ オブ サイエンス アンド テクノロジー | Slide-free histological imaging method and system |
JP2022548760A (en) * | 2019-09-19 | 2022-11-21 | ザ ホンコン ユニヴァーシティ オブ サイエンス アンド テクノロジー | Slide-free histological imaging methods and systems |
WO2021076075A3 (en) * | 2019-10-14 | 2021-06-17 | İzmi̇r Yüksek Teknoloji̇ Ensti̇tüsü | Cell viability analysis and counting from holograms by using deep learning and appropriate lensless holographic microscope |
KR102302333B1 (en) * | 2019-11-05 | 2021-09-16 | 주식회사 토모큐브 | Method and Apparatus for Generating 3D Fluorescent Label Image of Label-Free using 3D Refractive Index Tomography and Deep Learning |
US11450062B2 (en) | 2019-11-05 | 2022-09-20 | Tomocube, Inc. | Method and apparatus for generating 3-D molecular image based on label-free method using 3-D refractive index image and deep learning |
KR20210054183A (en) * | 2019-11-05 | 2021-05-13 | 주식회사 토모큐브 | Method and Apparatus for Generating 3D Fluorescent Label Image of Label-Free using 3D Refractive Index Tomography and Deep Learning |
EP4070232A4 (en) * | 2019-12-05 | 2024-01-31 | Tempus Labs Inc | Systems and methods for high throughput drug screening |
EP3839884A1 (en) * | 2019-12-20 | 2021-06-23 | Fundación Tecnalia Research & Innovation | Method, system and computer program product for image transformation |
WO2021133847A1 (en) * | 2019-12-23 | 2021-07-01 | The Regents Of The University Of California | Method and system for digital staining of microscopy images using deep learning |
EP4083907A4 (en) * | 2019-12-26 | 2024-01-17 | Japanese Found For Cancer Res | Pathological diagnosis assisting method using ai, and assisting device |
WO2021198247A1 (en) * | 2020-03-30 | 2021-10-07 | Carl Zeiss Ag | Optimal co-design of hardware and software for virtual staining of unlabeled tissue |
WO2022106593A1 (en) | 2020-11-19 | 2022-05-27 | Prospective Instruments Gmbh | Imaging method and system for generating a digitally stained image, training method for training an artificial intelligence system, and non-transitory storage medium |
EP4002267A1 (en) | 2020-11-19 | 2022-05-25 | Prospective Instruments GmbH | Imaging method and system for generating a digitally stained image, training method for training an artificial intelligence system, and non-transitory storage medium |
Also Published As
Publication number | Publication date |
---|---|
EP3776342A1 (en) | 2021-02-17 |
BR112020019896A2 (en) | 2021-01-05 |
US11893739B2 (en) | 2024-02-06 |
CN112106061A (en) | 2020-12-18 |
EP3776342A4 (en) | 2021-06-09 |
CA3095739A1 (en) | 2019-10-03 |
JP2021519924A (en) | 2021-08-12 |
KR20200140301A (en) | 2020-12-15 |
US20210043331A1 (en) | 2021-02-11 |
JP7344568B2 (en) | 2023-09-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11893739B2 (en) | Method and system for digital staining of label-free fluorescence images using deep learning | |
US20230030424A1 (en) | Method and system for digital staining of microscopy images using deep learning | |
Rivenson et al. | Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning | |
Rivenson et al. | Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue | |
CA2966555C (en) | Systems and methods for co-expression analysis in immunoscore computation | |
WO2020018154A1 (en) | Method and system for digital staining of label-free phase images using deep learning | |
EP2972223B1 (en) | Spectral unmixing | |
US20230186659A1 (en) | Machine learning models for cell localization and classification learned using repel coding | |
WO2023121846A1 (en) | Adversarial robustness of deep learning models in digital pathology | |
Sikkandar et al. | Three factor nonnegative matrix factorization based HE stain unmixing in histopathological images | |
Abraham et al. | Applications of artificial intelligence for image enhancement in pathology | |
US20240079116A1 (en) | Automated segmentation of artifacts in histopathology images | |
WO2021198252A1 (en) | Virtual staining logic | |
Hoque et al. | Stain normalization methods for histopathology image analysis: A comprehensive review and experimental comparison | |
WO2023107844A1 (en) | Label-free virtual immunohistochemical staining of tissue using deep learning | |
US20240135544A1 (en) | Method and system for digital staining of label-free fluorescence images using deep learning | |
WO2021198241A1 (en) | Multi-input and/or multi-output virtual staining | |
WO2021198247A1 (en) | Optimal co-design of hardware and software for virtual staining of unlabeled tissue | |
Mannam et al. | Improving fluorescence lifetime imaging microscopy phasor accuracy using convolutional neural networks | |
Crouzet et al. | Spectroscopic and deep learning-based approaches to identify and quantify cerebral microhemorrhages | |
JPWO2020075226A1 (en) | Image processing device operation method, image processing device, and image processing device operation program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19776693 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2020552396 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 3095739 Country of ref document: CA |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112020019896 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 20207031116 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2019776693 Country of ref document: EP Effective date: 20201030 |
|
ENP | Entry into the national phase |
Ref document number: 112020019896 Country of ref document: BR Kind code of ref document: A2 Effective date: 20200929 |