WO2020146037A1 - Machine de microdissection par capture laser à réalité augmentée - Google Patents

Machine de microdissection par capture laser à réalité augmentée Download PDF

Info

Publication number
WO2020146037A1
WO2020146037A1 PCT/US2019/059215 US2019059215W WO2020146037A1 WO 2020146037 A1 WO2020146037 A1 WO 2020146037A1 US 2019059215 W US2019059215 W US 2019059215W WO 2020146037 A1 WO2020146037 A1 WO 2020146037A1
Authority
WO
WIPO (PCT)
Prior art keywords
cells
interest
microscope
operator
machine
Prior art date
Application number
PCT/US2019/059215
Other languages
English (en)
Inventor
Jason Hipp
Martin STUMPE
Original Assignee
Google Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google Llc filed Critical Google Llc
Priority to US17/295,353 priority Critical patent/US20220019069A1/en
Publication of WO2020146037A1 publication Critical patent/WO2020146037A1/fr

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/32—Micromanipulators structurally combined with microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B2090/364—Correlation of different images or relation of image positions in respect to the body
    • A61B2090/365—Correlation of different images or relation of image positions in respect to the body augmented reality, i.e. correlating a live optical image with another image

Definitions

  • This disclosure is directed to a laser capture microdissection (LCM) machine which includes an augmented reality (AR) viewing feature and automated identification of potential cells of interest for excision and capture by the LCM machine.
  • LCM laser capture microdissection
  • AR augmented reality
  • Laser capture microdissection is a technique for isolating and extracting specific groups of cells of interest from microscopic regions in tissue samples.
  • Special purpose LCM machines are currently commercially available from several different manufacturers and described in the patent and scientific literature. Such machines typically include a microscope for allowing the operator to view the specimen under magnification. After an operator has designated specific groups of cells of interest, typically on a workstation monitor, lasers are guided to the corresponding location in the microscope slide to excise the area of interest from the slide.
  • a capture mechanism captures the groups of cells and places them in a suitable medium or on a substrate.
  • an augmented reality laser capture microdissection machine configured for capturing cells of interest from a sample.
  • the machine includes: a) a microscope having an eyepiece and a camera configured to capture images of the field of view of the microscope as seen through the eyepiece; b) an augmented reality subsystem configured to receive the images from the camera, the subsystem including a machine learning model stored on a machine readable medium identifying cells of potential interest in the images and a optics module projecting into the view of the microscope as seen through the eyepiece an outline of cells of potential interest identified by the machine learning model; c) a laser capture and microdissection subsystem configured for excising the cells of interest from the sample with one or more lasers and placing such cells of interest on a suitable medium; and d) an operator-activated input mechanism configured to provide input to the laser capture and microdissection subsystem whereby operator activation of the input mechanism while viewing the specimen and the outline of cells identified by the machine leaning model at the eyepiece invokes the laser capture and
  • the input mechanism is activated while the operator is viewing the specimen and the outline of cells identified by the machine leaning model at the eyepiece, e.g., by activating a voice command, activating a foot switch, clicking a mouse, or taking some other specified action.
  • a method for capturing cells of interest from a sample with the aid of a microscope having an eyepiece.
  • the method includes the steps of a) projecting an augmented reality image into the field of view of the microscope as seen through the eyepiece, the augmented reality image identifying cells of potential interest for laser capture microdissection, typically in the form of a border or outline; and b) invoking a laser capture and microdissection subsystem coupled to the microscope so as to excise the cells of potential interest, as presented in the augmented reality image, from the sample in response to an operator instruction and place them on a suitable medium.
  • method includes repeating steps a) and b) as an operator of the microscope changes magnification or navigates to different areas of the sample with the microscope.
  • step b) the operator instruction is received while the operator is viewing the specimen through the eyepiece of the microscope.
  • the operator does not have to look away from the specimen and perform a manual annotation task on an external device to draw a circle or specify a cluster of cells of interest; rather the augmented reality outline of the cells of interest are presented to the operator while they view the specimen through the eyepiece and they invoke the LCM subsystem to capture the cells of interest within the outline by a simple command, such as voice activated switch, foot switch, clicking a mouse, or other direct operation.
  • an improvement to a machine including a microscope having an eyepiece and a laser capture and microdissection subsystem configured to excise cells of interest from a sample and place them on a suitable medium.
  • the improvement takes the form of providing the machine with an augmented reality subsystem including a) a camera configured for capturing images of the field of view of the microscope; b) a machine learning model for identifying cells of potential interest within the images; c) an AR image generation unit and an optics module for overlaying an augmented reality image, e.g., enhancement onto the field of view as seen through the eyepiece in the form of an outline of potential cells of interest for LCM; and d) an operator activated input mechanism (e.g., switch) for invoking the laser capture and microdissection subsystem to excise the potential cells of interest in accordance with the augmented reality image from the sample and place the cells of interest in a suitable medium.
  • an operator activated input mechanism e.g., switch
  • Figure 1 is a schematic view of an LCM machine which is configured with an AR subsystem.
  • Figure 2 is a schematic view of an augmented reality microscope which is configured with an LCM subsystem.
  • Figure 3A is an illustration of the view through the eyepiece of an LCM machine which does not have the AR aspect of this disclosure.
  • Figure 3B is an illustration of the view through the eyepiece of an LCM machine which does have the AR aspect of this disclosure, e.g., the instrument of either Figure 1 or Figure 2, showing the cells of potential interest for LCM surrounded by a border; if the operator is satisfied that the cells of interest should be captured they activate a switch or other suitable control device to trigger operation of the LCM subsystem.
  • Figure 3C is the view of the specimen after the microdissection operation.
  • a system and method which includes an augmented reality (AR) subsystem including one or more machine learning models, which operates to automatically overlay an augmented reality image, e.g., a border or outline, that identifies cells of potential interest, in the field of view of the specimen as seen through the eyepiece of LCM microscope.
  • AR augmented reality
  • the operator does not have to manually identify the cells of interest for subsequent LCM, e.g., on a workstation monitor, as in the prior art. Rather, the areas of potential interest are identified automatically by the machine learning models in the AR subsystem and an augmented reality overlay image (e.g., border or outline) is provided in the field of view of the microscope so that the operator can see the tissue directly as well as the overlay identifying the cells or tissue of interest for LCM.
  • an augmented reality overlay image e.g., border or outline
  • the operator is provided with a switch, operator interface tool or other mechanism to select the identification of the cells, that is, indicate approval of the identification of the cells, while they view the specimen through the eyepiece.
  • Activation of the switch or other mechanism invokes laser excising and capture of the cells of interest via a known and conventional LCM subsystem.
  • this disclosure marries together two distinct technologies (1) a augmented reality microscope (and automatic identification regions which may be of interest for LCM), and (2) laser capture microdissection.
  • the data representing the outline or border in the augmented reality image is supplied to the LCM subsystem, thereby providing the X/Y coordinates within the sample for the cells of interest and therefore allow the laser to correctly execute the excising operation and capture of the cells of interest and place them on a suitable medium.
  • an AR subsystem to an existing LCM machine.
  • This option requires the addition of the AR components, including a camera optically coupled to the LCM machine eyepiece capturing a magnified digital image of the field of view of the sample as seen through the eyepiece of the microscope (if not already present), and a computing unit including a machine learning model (artificial intelligence pattern recognizer) that receives the images from the camera and identifies areas or cells of interest in the sample from the data in the digital image.
  • the LCM machine also is fitted with an optics module which incorporates a component, such as a semitransparent mirror or beam combiner/splitter, for overlaying an enhancement generated by the compute unit onto the field of view through the eyepiece.
  • the optics module allows the operator to see the field of view of the microscope as they would in a conventional microscope of an LCM machine, and, on demand or automatically, see an enhancement (in this application, a boundary or outline of LCM tissue or cells of interest) as an overlay on the field of view which is projected into the field of view by an AR display generation unit and lens.
  • an enhancement in this application, a boundary or outline of LCM tissue or cells of interest
  • new images are captured by the camera and supplied to the machine learning pattern recognizer, and new region of interest boundaries are overlaid onto the field of view through the eyepiece.
  • This display of new enhancements, superimposed on the field of view happens in substantial real time (i.e., within a few seconds or even fraction of a second) as the operator moves the slide relative to the microscope optics, changes focus, or changes magnification and continues to observe the specimen through the eyepiece.
  • the enhancement/boundary is of an area of interest, e.g., a cluster of cells, that the operator wants to be captured by LCM, they activate a switch or other suitable input mechanism (which can vary depending on the LCM configuration) which then causes laser excision of the cells of interest and subsequent capture of the cells in or on a suitable medium in the usual manner provided by the LCM machine.
  • Figure 1 illustrates a LCM machine 10 which includes a augmented reality microscope subsystem 100 including binocular eyepieces 104 and a LCM subsystem 200 which performs laser capture and transport of excised cells from a sample in conventional manner.
  • the specific techniques for laser capture, microdissection and transfer onto a suitable substrate or medium can vary widely and use any of the currently known LCM formats of the various manufacturers and described in the scientific and technical literature. Since these methods and the LCM subsystem (200) is known, a detailed description is omitted for the sake of brevity.
  • the LCM machine 10 includes a camera 124 which captures images of the field of view of the microscope as seen through the eyepieces.
  • This image 125 (and in practice, typically a steady stream of images) is supplied to a compute unit 126, which may take the form of a general purpose computer which is equipped with one or more machine learning models. These models have been trained to identify cells of interest in the type of issue currently being examined, and at the current magnification of the microscope subsystem 100.
  • a digital image 125 from the camera is fed to the appropriate model and the model identifies one or more clusters of cells of potential interest.
  • the model may take the form of a deep convolutional neural network.
  • the compute unit 126 generates an augmented reality enhancement to the field of view in the form of an image of a boundary or outline surrounding the cluster of cells of potential interest in the field of view.
  • This enhancement is referred to as an‘annotation map” 300 in Figure 1.
  • This annotation map is superimposed on the current field of view as seen through the eyepieces by projecting the annotation map into the field of view using an optics module as explained in detail in Figure 2.
  • this annotation map is provided to the LCM subsystem 200.
  • the LCM subsystem uses the annotation map as input of the X/Y regions of the sample which are to be excised upon the receipt of a command or instruction from the operators.
  • the operator can decide to initiate LCM of the identified cluster of cells for LCM operation by activating an input mechanism, shown as a switch 400, or taking other operator interface action to initiate LCM action.
  • the LCM machine 10 includes an input mechanism in the form of a mouse 400 which is connected to the LCM subsystem and activating a click of the mouse the LCM is triggered.
  • the input mechanism 400 could take the form of a foot switch, which when depressed by the operator triggers initiation of LCM.
  • the input mechanism 400 could be integrated into the LCM machine 10 and be voice activated, e.g., by the operator speaking a designated command such as “cut” or“select.”
  • FIG. 2 is a schematic illustration of an augmented reality microscope 100 which is fitted with an LCM subsystem 200.
  • the microscope 100 is described in detail in the patent literature, see WO 2018/231204 and in U.S. provisional application serial no. 62/656557 filed April 12, 2018, the content of which is incorporated by reference herein.
  • Figure 2 shows the augmented reality microscope 100 in conjunction with an optional connected pathologist workstation 140.
  • the microscope includes an eyepiece 104 (optionally a second eyepiece in the case of a stereoscopic microscope).
  • a stage 110 supports a slide 114 containing a biological sample.
  • An illumination source projects light through the sample.
  • a microscope objective lens 108 directs an image of the sample as indicated by the arrow 106 to an optics module 120. Additional lenses 108A and 108B are provided in the microscope for providing different levels of magnification.
  • a focus adjustment knob 160 allows the operator to change the depth of focus of the lens 108.
  • the optics module 120 incorporates a component, such as a semitransparent mirror 122 or beam combiner/splitter for overlaying an enhancement onto the field of view through the eyepiece.
  • the optics module 120 allows the operator to see the field of view of the microscope as he would in a conventional microscope, and, on demand or automatically, see an enhancement (heat map, boundary or outline, annotations, "annotation map” of Figure 1 , etc.) as an overlay on the field of view which is projected into the field of view by an augmented reality (AR) display generation unit 128 and a lens 130.
  • the image generated by the display unit 128 is combined with the microscope field of view by the semitransparent mirror 122.
  • a liquid crystal display (LCD) could be placed in the optical path that uses a transmissive negative image to project the enhancement into the optical path.
  • the optics module 120 can take a variety of different forms, and various
  • the semi-transparent mirror 122 directs the field of view of the microscope to both the eyepiece 104 and also to a digital camera 124.
  • a lens for the camera is not shown but is conventional.
  • the camera may take the form of a high resolution (e.g., 16 megapixel) video camera operating at say 10 or 30 frames per second.
  • the digital camera captures magnified images of the sample as seen through the eyepiece of the microscope.
  • Digital images captured by the camera are supplied to a compute unit 126.
  • the camera may take the form of an ultra- high resolution digital camera such as APS-H-size (approx. 29.2 x 20.2 mm) 250 megapixel CMOS sensor developed by Cannon and announced in September 2015.
  • the compute unit 126 includes a machine learning pattern recognizer which receives the images from the camera.
  • the machine learning pattern recognizer may take the form of a deep convolutional neural network which is trained on a large set of microscope slide images of the same type as the biological specimen under examination. Additionally, the pattern recognizer will preferably take the form of an ensemble of pattern recognizers, each trained on a set of slides at a different level of magnification, e.g., 5X,
  • the pattern recognizer is trained to identify regions of interest in an image (the exact type which will vary depending on the specific application of LCM under consideration) in biological samples of the type currently placed on the stage.
  • the pattern recognizer recognizes regions of interest on the image captured by the camera 124.
  • the compute unit 126 generates data representing an enhancement to the view of the sample as seen by the operator, in this example in the form of a closed curve or boundary, which is generated and projected by the AR display unit 128 and combined with the eyepiece field of view by the semitransparent mirror 122.
  • the essentially continuous capture of images by the camera 124, rapid performance of interference on the images by the pattern recognizer, and generation and projection of enhancements as overlays onto the field of view, enables the microscope of Figure 2 to continue to provide enhancements to the field of view and assist the pathologist in selecting cells of interest in the specimen for LCM in substantial real time as the operator navigates around the slide (e.g., by use of a motor driving the stage), by changing magnification by switching to a different objective lens 108A or 108B, or by changing depth of focus by operating the focus knob 160.
  • substantially real time we mean that an enhancement or overlay is projected onto the field of view within 10 seconds of changing magnification, changing depth of focus, or navigating and then stopping at a new location on the slide.
  • inference accelerators we expect that in most cases the new overlay can be generated and projected onto the field of view within a matter of a second or two or even a fraction of a second of a change in focus, change in magnification, or change in slide position.
  • the enhancement or annotation map generated in the compute unit 126 (or data representing the outline of the annotation map) is also provided to the LCM subsystem 200 so as to enable the LCM subsystem to perform laser excision of the cells of interest at the correct X/Y location in the specimen and capture of the cells in a suitable medium in accordance with the enhancement on the field of view as seen by the operator.
  • the operator then continues to navigate around the slide, change focus, change magnification, etc. as they see fit and meanwhile the compute unit continues to generate new boundaries of cells of potential interest which are displayed in the field of view of the microscope.
  • the operator continues to activate the LCM subsystem as desired to capture additional cells of interest by activating the switch 400.
  • the operator is done with a particular specimen (microscope slide) they select a new one, it is placed on the stage 110 and the process continues as explained above.
  • Figure 3A is an illustration of the view through the eyepiece of an LCM machine which does not have the AR aspect of this disclosure.
  • the operator views a sample 300, and a cluster of cells in roughly the center of Figure 3, but they still have to manually draw the boundary over cluster of cells of interest, e.g., with the aid of drawing tools and an external monitor.
  • Figure 3B is an illustration of the view through the eyepiece of an LCM machine which does have the AR aspect of this disclosure, e.g., the instrument of either Figure 1 or Figure 2, showing the cells of potential interest for LCM surrounded by a border 302; if the operator is satisfied that the cells of interest should be captured they activate the switch 400 or other suitable control device to trigger operation of the LCM subsystem of Figure 1 or Figure 2.
  • Figure 3C is the view of the specimen after the microdissection operation.
  • the cells within the border 300 are now removed from the view through the microscope eyepiece as indicated by the vacant area 304, indicating that the LCM operation to remove the cells from the sample slide and place them in a suitable medium has occurred.
  • the compute unit 126 of Figures 1 and 2 includes a deep convolutional neural network pattern recognizer in the form of a memory storing processing instructions and parameters for the neural network and a central processing unit or performance of inference on a captured image.
  • the module may also include a graphics card generating overlay digital data (e.g. annotations, outlines, etc.) based on the inference results from the pattern recognizer.
  • a memory includes processing instructions for selecting the appropriate machine learning model based on the current magnification level.
  • the compute unit may also include an inference accelerator to speed up the performance of inference on captured images.
  • the compute unit further includes various interfaces to other components of the system including an interface, not shown, to receive the digital images from the camera, such as a USB port, an interface (e.g., network cable port or HDMI port) to send digital display data to the AR display unit 128, an interface (e.g., network cable port) 216 to the workstation 140 or to the LCM subsystem 200 and an interface (e.g., SC card reader) enabling the compute unit to receive and download portable media containing additional pattern recognizers to expand the capability of the system to perform pattern recognition and overlay generation for different pathology applications.
  • an interface e.g., network cable port or HDMI port
  • SC card reader e.g., SC card reader
  • the compute unit 126 could take the form of a general purpose computer (e.g., PC) augmented with the pattern recognizer(s) and accelerator, and graphics processing modules.
  • the personal computer has an interface to the camera (e.g., a USB port receiving the digital image data from the camera), an interface to the AR projection unit 126, such as an HDMI port, and a network interface to enable downloading of additional pattern recognizers and/or communicate with a remote workstation as shown in Figure 2.
  • an automatic specimen type detector or manual selector switches between the specimen dependent pattern recognition models (e.g. prostate cancer vs breast cancer vs organ bud detection), and based on that the proper machine learning pattern recognizer or model is chosen. Movement of the slide to a new location (e.g., by use of a motor driving the stage) or switching to another microscope objective 108 (i.e. magnification) triggers an update of the enhancement, as explained previously.
  • magnification i.e. magnification
  • an ensemble of different models operating at different magnification levels performs inference on the specimen and inference results could be combined on the same position of the slide. Further details on how this operation could be performed are described in the pending PCT application entitled "Method and System for Assisting Pathologist Identification of Tumor Cells in Magnified Tissue Images”, serial no.
  • Deep convolutional neural network pattern recognizers of the type used in the compute unit of Figures 1 and 2, are widely known in the art of pattern recognition and machine vision, and therefore a detailed description thereof is omitted for the sake of brevity.
  • the Google lnception-v3 deep convolutional neural network architecture upon which the present pattern recognizers are based, is described in the scientific literature. See the following references, the content of which is incorporated by reference herein: C. Szegedy et al., Going Deeper with Convolutions, arXiv:1409.4842 [cs.CV] (September 2014); C.
  • suitable medium in intended to refer broadly to known substrates, containers and/or and media which are specifically adapted for use in LCM, such as for example the transfer films described in U.S. patents 9,279,749 and 9,103,757.
  • suitable medium in intended to refer broadly to known substrates, containers and/or and media which are specifically adapted for use in LCM, such as for example the transfer films described in U.S. patents 9,279,749 and 9,103,757.
  • some embodiments of laser microdissection employ a polymer transfer film that is placed on top of the tissue sample.
  • the transfer film may or may not contact the tissue sample.
  • This transfer film is typically a thermoplastic manufactured containing organic dyes that are chosen to selectively absorb in the near infrared region of the spectrum overlapping the emission region of common AIGaAs infrared laser diodes.
  • Thermoplastic transfer films such as a 100 micron thick ethyl vinyl acetate (EVA) film available from Electroseal Corporation of Pompton Lakes, N.J. (type E540) have been used in LCM applications.
  • EVA ethyl vinyl acetate
  • the film is chosen due to its low melting point of about 90° C.
  • the suitable medium may be a surface or solution present in a receiving container, as described in U.S.

Abstract

La présente invention porte sur un sous-système de réalité augmentée (AR) doté d'un ou plusieurs modèles d'apprentissage machine, qui superpose automatiquement une image de réalité augmentée, par exemple une bordure ou un contour, identifiant des cellules d'intérêt potentiel, dans le champ de vision de l'échantillon tel que vu par l'oculaire d'un microscope LCM. L'utilisateur n'a pas à identifier manuellement les cellules d'intérêt pour un LCM ultérieur, par exemple, sur un moniteur de poste de travail, comme dans l'état de la technique. L'utilisateur est équipé d'un commutateur, d'un outil d'interface opérateur ou d'un autre mécanisme permettant de sélectionner l'identification des cellules, c'est-à-dire d'indiquer l'approbation de l'identification des cellules, tandis qu'elles visualisent l'échantillon par l'oculaire. L'activation du commutateur ou d'un autre mécanisme invoque l'excision et la capture laser des cellules d'intérêt par l'intermédiaire d'un sous-système LCM connu et classique.
PCT/US2019/059215 2019-01-09 2019-10-31 Machine de microdissection par capture laser à réalité augmentée WO2020146037A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/295,353 US20220019069A1 (en) 2019-01-09 2019-10-31 Augmented Reality Laser Capture Microdissection Machine

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962790221P 2019-01-09 2019-01-09
US62/790,221 2019-01-09

Publications (1)

Publication Number Publication Date
WO2020146037A1 true WO2020146037A1 (fr) 2020-07-16

Family

ID=68655680

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/059215 WO2020146037A1 (fr) 2019-01-09 2019-10-31 Machine de microdissection par capture laser à réalité augmentée

Country Status (2)

Country Link
US (1) US20220019069A1 (fr)
WO (1) WO2020146037A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173072A (zh) * 2023-11-03 2023-12-05 四川大学 一种基于深度学习的弱激光图像增强方法及装置

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5998129A (en) 1996-02-05 1999-12-07 P.A.L.M. Gmbh Method and device for the contactless laser-assisted microinjection, sorting and production of biological objects generated in a planar manner
WO2009086521A2 (fr) * 2007-12-28 2009-07-09 Carl Zeiss Microimaging Ais, Inc. Système, dispositif et procédé de microdissection laser
US7915016B2 (en) 2004-05-28 2011-03-29 Philipps-Universitat Marburg cDNA production from cells after laser microdissection
US8664002B2 (en) 2005-05-20 2014-03-04 Motic China Group Co., Ltd. Method and system for collecting cells following laser microdissection
US9103757B2 (en) 2000-04-26 2015-08-11 Life Technologies Corporation Laser capture microdissection (LCM) extraction device and device carrier, and method for post-LCM fluid processing
US20150262329A1 (en) * 2012-10-03 2015-09-17 KONINKLIJKE PHILIPS N.V. a corporation Combined sample examinations
US9279749B2 (en) 2004-09-09 2016-03-08 Life Technologies Corporation Laser microdissection method and apparatus
US20160183779A1 (en) 2014-12-29 2016-06-30 Novartis Ag Magnification in Ophthalmic Procedures and Associated Devices, Systems, and Methods
WO2016130424A1 (fr) 2015-02-09 2016-08-18 The Arizona Board Of Regents Of Regents On Behalf Of The University Of Arizona Microscopie stéréoscopique augmentée
US9664599B2 (en) 2011-03-22 2017-05-30 Carl Zeiss Microscopy Gmbh Laser microdissection method and laser microdissection device
US9804144B2 (en) 2012-10-09 2017-10-31 Leica Microsystems Cms Gmbh Method for defining a laser microdissection region, and associated laser microdissection system
WO2018019051A1 (fr) 2016-07-29 2018-02-01 林光榕 Atomiseur de cigarette électronique
US20180114317A1 (en) 2016-10-21 2018-04-26 Nantomics, Llc Digital histopathology and microdissection
US20180149561A1 (en) * 2015-05-20 2018-05-31 Leica Microsystems Cms Gmbh Method and examination system for examining and processing a microscopic sample
WO2018231204A1 (fr) 2017-06-13 2018-12-20 Google Llc Microscope à réalité augmentée destiné à une pathologie

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3776458B1 (fr) * 2018-04-12 2022-05-04 Google LLC Microscope à réalité augmentée pour pathologie avec superposition de données quantitatives de biomarqueurs

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5998129A (en) 1996-02-05 1999-12-07 P.A.L.M. Gmbh Method and device for the contactless laser-assisted microinjection, sorting and production of biological objects generated in a planar manner
US9103757B2 (en) 2000-04-26 2015-08-11 Life Technologies Corporation Laser capture microdissection (LCM) extraction device and device carrier, and method for post-LCM fluid processing
US7915016B2 (en) 2004-05-28 2011-03-29 Philipps-Universitat Marburg cDNA production from cells after laser microdissection
US9279749B2 (en) 2004-09-09 2016-03-08 Life Technologies Corporation Laser microdissection method and apparatus
US8664002B2 (en) 2005-05-20 2014-03-04 Motic China Group Co., Ltd. Method and system for collecting cells following laser microdissection
WO2009086521A2 (fr) * 2007-12-28 2009-07-09 Carl Zeiss Microimaging Ais, Inc. Système, dispositif et procédé de microdissection laser
US9664599B2 (en) 2011-03-22 2017-05-30 Carl Zeiss Microscopy Gmbh Laser microdissection method and laser microdissection device
US20150262329A1 (en) * 2012-10-03 2015-09-17 KONINKLIJKE PHILIPS N.V. a corporation Combined sample examinations
US9804144B2 (en) 2012-10-09 2017-10-31 Leica Microsystems Cms Gmbh Method for defining a laser microdissection region, and associated laser microdissection system
US20160183779A1 (en) 2014-12-29 2016-06-30 Novartis Ag Magnification in Ophthalmic Procedures and Associated Devices, Systems, and Methods
WO2016130424A1 (fr) 2015-02-09 2016-08-18 The Arizona Board Of Regents Of Regents On Behalf Of The University Of Arizona Microscopie stéréoscopique augmentée
US20180149561A1 (en) * 2015-05-20 2018-05-31 Leica Microsystems Cms Gmbh Method and examination system for examining and processing a microscopic sample
WO2018019051A1 (fr) 2016-07-29 2018-02-01 林光榕 Atomiseur de cigarette électronique
US20180114317A1 (en) 2016-10-21 2018-04-26 Nantomics, Llc Digital histopathology and microdissection
WO2018231204A1 (fr) 2017-06-13 2018-12-20 Google Llc Microscope à réalité augmentée destiné à une pathologie

Non-Patent Citations (13)

* Cited by examiner, † Cited by third party
Title
A. MADABHUSHI ET AL.: "Image analysis and machine learning in digital pathology: Challenges and opportunities", MEDICAL IMAGE ANALYSIS, vol. 33, 2016, pages 170 - 175, XP029703320, DOI: 10.1016/j.media.2016.06.037
A. SCHUAMBERG ET AL.: "H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer", BIORXIV, Retrieved from the Internet <URL:http:/.bioRxiv.or/content/early/2016/07/17/064279>
BRASKO ET AL.: "intelligent image-based in situ single-cell isolation", NATURE COMMUNICATIONS, vol. 9, no. 226, January 2018 (2018-01-01)
C. SZEGEDY ET AL.: "Going Deeper with Convolutions", ARXIV:1409.4842 [CS.CV, September 2014 (2014-09-01)
C. SZEGEDY ET AL.: "Rethinking the Inception Architecture for Computer Vision", ARXIV: 1512.00567 [CS.CV, December 2015 (2015-12-01)
C. SZEGEDY: "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning", ARXIV:1602.0761 [CS.CV, February 2016 (2016-02-01)
D. WANG ET AL.: "Deep Learning for Identifying Metastatic Breast Cancer", ARXIV:1606.05718V1, June 2016 (2016-06-01)
EDWARDS ET AL.: "Augmentation of Reality Using an Operating Microscope", J. IMAGE GUIDED SURGERY, vol. 1, no. 3, 1995
EDWARDS ET AL.: "Medicine Meets Virtual Reality", IOS PRESS, article "Stereo augmented reality in the surgical microscope", pages: 102
G. LITJENS ET AL., DEEP LEARNING AS A TOOL FOR INCREASING ACCURACY AND EFFICIENCY OF HISTOPATHOLOGICAL DIAGNOSIS, vol. 6, May 2016 (2016-05-01), pages 26286, Retrieved from the Internet <URL:www.nature.com/scientificreports>
VINCENZO PADUANO ET AL.: "Fully automated organ bud detection and segmentation for Laser Capture Microdissection applications", 2011 IEEE INTERNATIONAL CONFERENCE ON IMAGING SYSTEMS AND TECHNIQUES PENANG, MALAYSIA, 17 May 2011 (2011-05-17)
VINCENZO PADUANO ET AL: "Fully automated organ bud detection and segmentation for Laser Capture Microdissection applications", IMAGING SYSTEMS AND TECHNIQUES (IST), 2011 IEEE INTERNATIONAL CONFERENCE ON, IEEE, 17 May 2011 (2011-05-17), pages 118 - 123, XP031907210, ISBN: 978-1-61284-894-5, DOI: 10.1109/IST.2011.5962211 *
WATSON: "Augmented microscopy: real-time overlay of bright-field and near-infrared fluorescence images", JOURNAL OF BIOMEDICAL OPTICS, vol. 20, no. 10, October 2015 (2015-10-01), XP060071804, DOI: 10.1117/1.JBO.20.10.106002

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173072A (zh) * 2023-11-03 2023-12-05 四川大学 一种基于深度学习的弱激光图像增强方法及装置
CN117173072B (zh) * 2023-11-03 2024-02-02 四川大学 一种基于深度学习的弱激光图像增强方法及装置

Also Published As

Publication number Publication date
US20220019069A1 (en) 2022-01-20

Similar Documents

Publication Publication Date Title
JP6947841B2 (ja) 病理学用の拡張現実顕微鏡
US11636627B2 (en) System for histological examination of tissue specimens
CN107255863B (zh) 数字显微镜
RU2553078C2 (ru) Способ микродиссекции и система обработки информации
JP2021515240A (ja) 定量的バイオマーカデータのオーバレイを有する病理学用拡張現実顕微鏡
JP6310671B2 (ja) レーザー顕微解剖領域の画定方法及び関連するレーザー顕微解剖システム
EP1860481B1 (fr) Système et procédé de microscope pour synthétiser des images microscopiques
JP6333145B2 (ja) 画像処理方法および画像処理装置
US6813008B2 (en) Microdissection optical system
US8699129B2 (en) Microscope system, storage medium storing control program, and control method
US11869166B2 (en) Microscope system, projection unit, and image projection method
US20110134517A1 (en) Microscope controller and microscope system comprising microscope controller
WO2021057422A1 (fr) Système de microscope, dispositif médical intelligent, procédé de mise au point automatique et support de stockage
US11662565B2 (en) Microscope system, projection unit, and image projection method
US20160062101A1 (en) Method and apparatus for small and large format histology sample examination
US9678326B2 (en) Generating perspective views in microscopy
CN115485602A (zh) 显微镜系统、投影单元以及精子筛选辅助方法
JP2021515912A (ja) ディジタル病理スキャニング・インターフェースおよびワークフロー
US20220019069A1 (en) Augmented Reality Laser Capture Microdissection Machine
KR20200052157A (ko) 고배율 이미지가 저배율 이미지에 의해 가이드되는 디지털 현미경 및 디지털 현미경 시스템
US10475198B2 (en) Microscope system and specimen observation method
EP2322967A1 (fr) Système microscope
CN111656247B (zh) 一种细胞图像处理系统、方法、自动读片装置与存储介质
WO2022023881A1 (fr) Collecte de données d&#39;annotation à l&#39;aide d&#39;un suivi basé sur le regard
US20140253599A1 (en) Display data generating apparatus and control method for the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19809265

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19809265

Country of ref document: EP

Kind code of ref document: A1