US20210181110A1 - Method of Determining Surgical Margins Using Fluorescence Biopsy Specimen Imager - Google Patents
Method of Determining Surgical Margins Using Fluorescence Biopsy Specimen Imager Download PDFInfo
- Publication number
- US20210181110A1 US20210181110A1 US17/177,583 US202117177583A US2021181110A1 US 20210181110 A1 US20210181110 A1 US 20210181110A1 US 202117177583 A US202117177583 A US 202117177583A US 2021181110 A1 US2021181110 A1 US 2021181110A1
- Authority
- US
- United States
- Prior art keywords
- fluorescence
- images
- imaging
- tissue sample
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 19
- 208000035346 Margins of Excision Diseases 0.000 title abstract 2
- 238000001574 biopsy Methods 0.000 title description 4
- 238000003384 imaging method Methods 0.000 claims abstract description 90
- 230000005284 excitation Effects 0.000 claims abstract description 23
- 206010028980 Neoplasm Diseases 0.000 claims description 44
- 238000002073 fluorescence micrograph Methods 0.000 claims description 22
- MOFVSTNWEDAEEK-UHFFFAOYSA-M indocyanine green Chemical compound [Na+].[O-]S(=O)(=O)CCCCN1C2=CC=C3C=CC=CC3=C2C(C)(C)C1=CC=CC=CC=CC1=[N+](CCCCS([O-])(=O)=O)C2=CC=C(C=CC=C3)C3=C2C1(C)C MOFVSTNWEDAEEK-UHFFFAOYSA-M 0.000 claims description 16
- 229960004657 indocyanine green Drugs 0.000 claims description 16
- 239000007850 fluorescent dye Substances 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 11
- 238000001356 surgical procedure Methods 0.000 claims description 11
- 239000012216 imaging agent Substances 0.000 claims description 5
- 108010041308 Endothelial Growth Factors Proteins 0.000 claims description 3
- 102000009465 Growth Factor Receptors Human genes 0.000 claims description 3
- 108010009202 Growth Factor Receptors Proteins 0.000 claims description 3
- 102000005789 Vascular Endothelial Growth Factors Human genes 0.000 claims description 3
- 108010019530 Vascular Endothelial Growth Factors Proteins 0.000 claims description 3
- 230000002137 anti-vascular effect Effects 0.000 claims description 3
- 229960000397 bevacizumab Drugs 0.000 claims description 3
- 229960005395 cetuximab Drugs 0.000 claims description 3
- 229940116977 epidermal growth factor Drugs 0.000 claims description 3
- 102000052116 epidermal growth factor receptor activity proteins Human genes 0.000 claims description 3
- 108700015053 epidermal growth factor receptor activity proteins Proteins 0.000 claims description 3
- YOHYSYJDKVYCJI-UHFFFAOYSA-N n-[3-[[6-[3-(trifluoromethyl)anilino]pyrimidin-4-yl]amino]phenyl]cyclopropanecarboxamide Chemical compound FC(F)(F)C1=CC=CC(NC=2N=CN=C(NC=3C=C(NC(=O)C4CC4)C=CC=3)C=2)=C1 YOHYSYJDKVYCJI-UHFFFAOYSA-N 0.000 claims description 3
- 229960001972 panitumumab Drugs 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims 2
- 239000004065 semiconductor Substances 0.000 claims 2
- 239000000523 sample Substances 0.000 abstract 2
- 239000012472 biological sample Substances 0.000 abstract 1
- 210000001519 tissue Anatomy 0.000 description 31
- 238000005286 illumination Methods 0.000 description 15
- 238000003325 tomography Methods 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 9
- 239000000463 material Substances 0.000 description 8
- 201000010099 disease Diseases 0.000 description 7
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 7
- 238000000799 fluorescence microscopy Methods 0.000 description 7
- 230000003190 augmentative effect Effects 0.000 description 6
- 238000001816 cooling Methods 0.000 description 6
- 230000007170 pathology Effects 0.000 description 6
- 239000000975 dye Substances 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 5
- 239000002184 metal Substances 0.000 description 5
- 230000035945 sensitivity Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 201000011510 cancer Diseases 0.000 description 4
- 239000011521 glass Substances 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 238000012552 review Methods 0.000 description 4
- 238000005096 rolling process Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 239000002872 contrast media Substances 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 210000004185 liver Anatomy 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 230000001629 suppression Effects 0.000 description 3
- 230000009885 systemic effect Effects 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 241001465754 Metazoa Species 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 2
- 239000007864 aqueous solution Substances 0.000 description 2
- 230000008033 biological extinction Effects 0.000 description 2
- 210000004027 cell Anatomy 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 2
- 230000021615 conjugation Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000007788 liquid Substances 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000002595 magnetic resonance imaging Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000002603 single-photon emission computed tomography Methods 0.000 description 2
- 229910001220 stainless steel Inorganic materials 0.000 description 2
- 239000010935 stainless steel Substances 0.000 description 2
- 230000001954 sterilising effect Effects 0.000 description 2
- 238000004659 sterilization and disinfection Methods 0.000 description 2
- 231100000041 toxicology testing Toxicity 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- ZOOGRGPOEVQQDX-UUOKFMHZSA-N 3',5'-cyclic GMP Chemical compound C([C@H]1O2)OP(O)(=O)O[C@H]1[C@@H](O)[C@@H]2N1C(N=C(NC2=O)N)=C2N=C1 ZOOGRGPOEVQQDX-UUOKFMHZSA-N 0.000 description 1
- 208000019901 Anxiety disease Diseases 0.000 description 1
- 241000699666 Mus <mouse, genus> Species 0.000 description 1
- 241000699660 Mus musculus Species 0.000 description 1
- 206010029098 Neoplasm skin Diseases 0.000 description 1
- 206010067482 No adverse event Diseases 0.000 description 1
- 238000012879 PET imaging Methods 0.000 description 1
- 241000283984 Rodentia Species 0.000 description 1
- 208000000453 Skin Neoplasms Diseases 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000036506 anxiety Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012620 biological material Substances 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 210000000481 breast Anatomy 0.000 description 1
- 150000007942 carboxylates Chemical class 0.000 description 1
- 230000000747 cardiac effect Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 239000002826 coolant Substances 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000002224 dissection Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 210000005002 female reproductive tract Anatomy 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 125000000524 functional group Chemical group 0.000 description 1
- 201000010536 head and neck cancer Diseases 0.000 description 1
- 208000014829 head and neck neoplasm Diseases 0.000 description 1
- 230000002440 hepatic effect Effects 0.000 description 1
- 206010073071 hepatocellular carcinoma Diseases 0.000 description 1
- 231100000844 hepatocellular carcinoma Toxicity 0.000 description 1
- 230000002209 hydrophobic effect Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 239000003562 lightweight material Substances 0.000 description 1
- 230000003908 liver function Effects 0.000 description 1
- 210000001165 lymph node Anatomy 0.000 description 1
- 230000003692 lymphatic flow Effects 0.000 description 1
- 210000004324 lymphatic system Anatomy 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010172 mouse model Methods 0.000 description 1
- 238000003333 near-infrared imaging Methods 0.000 description 1
- 230000009871 nonspecific binding Effects 0.000 description 1
- 231100000252 nontoxic Toxicity 0.000 description 1
- 238000011580 nude mouse model Methods 0.000 description 1
- 238000002559 palpation Methods 0.000 description 1
- 210000004738 parenchymal cell Anatomy 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000002600 positron emission tomography Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000002285 radioactive effect Effects 0.000 description 1
- 238000002601 radiography Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 210000005005 sentinel lymph node Anatomy 0.000 description 1
- 239000000243 solution Substances 0.000 description 1
- 206010041823 squamous cell carcinoma Diseases 0.000 description 1
- 238000010186 staining Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007920 subcutaneous administration Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 231100000419 toxicity Toxicity 0.000 description 1
- 230000001988 toxicity Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 239000012780 transparent material Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6486—Measuring fluorescence of biological material, e.g. DNA, RNA, cells
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/16—Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/50—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H04N5/2251—
-
- H04N5/2256—
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N2021/1765—Method using an image detector and processing of image signal
- G01N2021/177—Detector of the video camera type
- G01N2021/1776—Colour camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03545—Pens or stylus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10064—Fluorescence image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Definitions
- a primary purpose of cancer surgery and surgical pathology is to remove tumor tissues. Complete removal of tumors and all tumor tissue is important. If a tumor is not completely removed, then a second surgery may be required. This produces unneeded patient anxiety and risk, delays the start of second stage therapies, and incurs additional financial costs.
- Tumor margins are the healthy tissue surrounding the tumor, more specifically, the distance between the tumor tissue and the edge of the surrounding tissue removed along with the tumor. Ideally, the margins are selected so that the risk of leaving tumor tissue within the patient is low.
- Intraoperative pathology consultation guides the cancer surgeon during surgery to assess if the tumor and margin have not been completely excised.
- Current surgical pathology consultation relies on palpation, visualization, and anatomical features to select sampling areas for frozen section analysis, which is often limited to 20-30 minutes.
- Intraoperative pathology consultation starts with gross examination of the excised tissues to choose areas for sampling. This is followed by dissection, embedding, frozen sectioning, and staining of 5- ⁇ m slice sections of the excised tissue.
- the limited surgical consultation timeframe allows only a small portion of the resected tissue to be interpreted by the pathologist. This time crunch can result in sampling errors which negatively impact the patient outcome.
- a device that provides a nearly full, 360° rotation plus limited elevation-angle, three-dimensional (3D) images of a tumor sample that has been systemically labeled using a fluorescence dye.
- a first camera takes full-color pictures of the tumor sample at different azimuth and elevation angles; a second camera captures fluorescence images of the sample at the same angles while the sample is illuminated at an excitation frequency.
- sensitivity-improving approaches such as active cooling, imaging with a monochrome sensor, long exposure durations, and electronic noise suppression methods, can be used with the second camera.
- the resulting full-color and fluorescence images are either overlaid or used to make 3D computer models that are overlaid in space.
- calibration marks on a window platform help the software generate precise 3D grids and sub-volumes in order to determine the position of each surface in the images.
- the imager can generate a full rotation 3D image of a sample and present it in interactive animation as well as in an augmented realty 3D model to help localize diseased regions around the whole surgical sample.
- GUI graphical user interface
- a touch pen comprising a pen body and a pen tip.
- the pen tip is attached to an end of the pen body.
- the touch pen is sterile and does not dispense ink.
- the pen body comprises stainless steel.
- the pen tip is detachable from the pen body and replaceable with a second pen tip.
- the touch pen further comprises a pen cover. The pen cover encloses the pen body and the pen tip, and is sterile.
- excitation light excites fluorescence less at farther points on the sample. This can be corrected for by determining the distance of each surface from the fluorescence excitation light source and adjusting the resulting reading's intensity.
- the device will help localize disease tissues to reduce sampling errors and re-examination, and ultimately reduce local cancer recurrence and second surgery costs.
- the device can be used for image-guided pathology to screen resected tissue or biopsy triage in conjunction with image-guide surgery.
- FIG. 1 illustrates a multiple-camera imaging system in accordance with an embodiment.
- FIG. 2 illustrates a switchable camera configuration in accordance with an embodiment.
- FIG. 3 illustrates a beam splitting configuration in accordance with an embodiment.
- FIG. 4 illustrates a dichroic mirror configuration in accordance with an embodiment.
- FIG. 5 illustrates a flip mirror configuration in accordance with an embodiment.
- FIG. 6 illustrates a second flip mirror configuration in accordance with an embodiment.
- FIG. 7 illustrates transparent window with scale marks in accordance with an embodiment.
- FIG. 8 illustrates using the use of scale marks on the transparent window of FIG. 7 for object registration in accordance with an embodiment.
- FIG. 9 illustrates fluorescence image correction of a rotated sample in accordance with an embodiment.
- FIG. 10 illustrates fluorescence image correction of two points on a sample in accordance with an embodiment.
- FIG. 11A illustrates images of xenograft tumor tissue.
- FIG. 11B illustrates a filmstrip view of 3D animation presenting images of the tumor of FIG. 11A in accordance with an embodiment.
- FIG. 12 illustrates the imaging of different tumor sizes and a filmstrip presentation manner of a graphical user interface (GUI) in accordance with an embodiment.
- GUI graphical user interface
- FIG. 13A illustrates touch screen manipulation of a GUI in a rolling mode in accordance with an embodiment.
- FIG. 13B illustrates touch screen manipulation of a GUI in a pitching mode in accordance with an embodiment.
- FIGS. 14-41 illustrate overlaid full-color and fluorescence images of an object at different angles in accordance with an embodiment.
- FIG. 42 is an illustration of a touch pen in accordance with an embodiment.
- Intraoperative consultations are among the most stressful parts of a pathologist's duties. With a strict time limit, little or no ancillary studies available, and using only touch and visual cues to select a few tissue sections to analyze, the pathologist must make a diagnosis where clinical consequences are quite significant.
- Imaging modalities currently used in patient care include magnetic resonance imaging (MRI), ultrasound, radiography (X-ray), single-photon emission computed tomography (SPECT), and positron emission tomography (PET). These imaging techniques are not generally used to localize disease in tissues for margin determinations. The reason for this is in part that the resolution offered by these five imaging technologies is not sufficient.
- MRI magnetic resonance imaging
- X-ray radiography
- SPECT single-photon emission computed tomography
- PET positron emission tomography
- non-specific contrast agents typically used by MRI, ultrasound, and X-ray computed tomography do not target tumors.
- the use of radioactive isotopes in SPECT and PET imaging introduces safety and wellness concerns and is financially costly for both patients and care providers.
- An alternative pseudo tomography imaging system embodiment can be used to satisfy the critical criteria for clinical utility.
- the proposed technology can provide the speed and throughput, the resolution, and the sensitivity and specificity needed to offer a solution for enhancing intraoperative pathology consultation.
- the described embodiments can also be used within the current work flow of surgeries without significant alterations.
- An imaging device as described herein can relieve the diagnosis stress while helping to visualize and identify suspicious areas in lesion samples. It is foreseeable that the device will be used to assist intraoperative consultation of primary tumors and biopsies of head and neck cancers, lymph nodes, hepatobiliary, breast, female genital tract, skin tumors, etc.
- the proposed technology would also provide cost savings to the healthcare system by reducing reoperations due to recurrence. It is notable that this invention can provide a new paradigm in cancer surgery without causing a deviation from current work flow.
- FIG. 1 illustrates a multiple-camera imaging system in accordance with an embodiment.
- the imaging system comprises an optical engine 101 for excitation and signal collection; a pseudo-tomography platform 102 comprising a rotatable stage 108 for positioning and rotation of a sample 103 ; and software (not shown) for operation, imaging, and generation of 3D results.
- the optical engine 101 comprises a first imaging camera 104 and a second imaging camera 105 .
- the first 104 and second 105 imaging cameras record information using two different imagining modalities.
- one of the two cameras can be a color camera used for reflected light imaging.
- the other camera can be a monochromatic camera for fluorescence signal collection.
- a monochromatic camera includes an imaging device having a sensitivity to a particular wavelength or range of wavelengths while having an insensitivity to a second range of wavelengths, or as otherwise known in the art.
- the optical engine 101 can also comprise a preview camera 106 .
- the preview camera 106 can be a true-color zoom camera module used to take preview images and videos in real-time at user-selected view angles.
- a laser system 107 can be positioned and configured to illuminate sample 103 with an excitation wavelength.
- the excitation wavelength is within the second range of wavelengths that the monochromatic camera is insensitive to.
- All three of the cameras 104 , 105 , and 106 can be configured to have a depth of focus that coincides with the location of the sample 103 on the pseudo-tomography platform 102 .
- the illumination light fields and fields-of view of the cameras 104 , 105 , and 106 can further be configured along with the depths of focus to provides an imaging resolution of 100 line pairs per inch.
- the optical engine 101 can also comprise a cooling line 109 .
- the cooling line 109 can be used provide liquid or other coolant to camera 105 in order to maintain a preferred camera temperature and increase camera sensitivity.
- the tomography platform 102 allows nearly full-rotation sample scanning and positioning for tomography imaging and sample loading. This is accomplished through tilting and rotation of the sample stage 107 holding the sample 103 . Tilting of the sample stage 107 allows projection views of the top and bottom of the sample 103 .
- the system comprises two imaging cameras as well as a preview camera.
- the preview camera can be used for taking a ‘preview’ picture of the entire sample for annotation of a respective data file storing image information.
- the preview pictures can also be used to determine if the sample is properly within the imaging volume.
- An embodiment can include a full-color camera to provide a sample preview (at, for example, 1280 ⁇ 720 pixels, 15 frame/second, >4-inch field of view) and an isometric picture of a sample for documentary requirement before beginning the 3D tomographic imaging.
- a software application can be used for a real-time preview function.
- a first imaging camera is used for collecting an image over a broad spectrum of visible wavelengths.
- the first camera is a grayscale camera.
- the first camera is a full color camera.
- the second imaging camera is a cooled camera configured to capture fluorescence images at very low light levels.
- a cooling fan, liquid cooling elements, and other cooling systems can be used to cool the second imaging camera.
- the camera can be optimized for collecting near infrared wavelength photons.
- the imaging cameras are configured to take pictures of the sample from the same orientation and at the same focal depth as one another. There are several different ways to achieve this.
- FIG. 2 illustrates a switchable camera configuration in accordance with an embodiment.
- the first 201 and second 202 imaging cameras can be rotated or translated into a position such that their optics focus at the same depth within the imaging volume. This can be accomplished by alternately positioning the first 201 and second 202 cameras such that one of these cameras at a time is positioned to receive light 203 reflecting or emitting from the sample 204 and reflecting off of a mirror 205 .
- a pivot location offset from the first 201 and second 202 cameras is used to rotate the cameras.
- a slide with detents is used to translate the first 201 and second 202 cameras,
- the slide can be made from metal, plastic, or other suitable rigid materials.
- FIG. 3 illustrates a beam splitting configuration in accordance with an embodiment.
- the cameras 301 and 302 remain fixed in place.
- a beam splitter 303 is used to receive incident light 304 reflecting or emitting from the sample 305 and split the incident light into reflected light 306 and transmitted light 307 .
- the reflected light 306 is captured by the first camera 301
- the transmitted light 307 is captured by the second camera 302 after reflecting off of a mirror 308 .
- FIG. 4 illustrates a dichroic mirror configuration in accordance with an embodiment.
- the cameras 401 and 402 remain fixed in place.
- a dichroic mirror 403 is used to receive incident light 404 reflecting or emitting from the sample 405 and split the incident light into reflected light 406 and transmitted light 407 .
- the reflected light 406 is captured by the first camera 401
- the transmitted light 407 is captured by the second camera 402 after reflecting off of a fully reflective mirror 408 .
- the dichroic mirror 403 can be configured to reflect a majority of light outside of a particular range of wavelengths.
- the dichroic mirror 403 can transmit light within the particular range of wavelengths to camera 402 , which can be a cooled monochromatic camera.
- FIG. 5 illustrates a flip mirror configuration in accordance with an embodiment.
- the cameras 501 and 502 remain fixed in place.
- a fully reflective first mirror 508 can be moved between a first 503 and second 504 position.
- light 505 reflected or emitted by the sample 506 is captured by the first camera 501 .
- light 505 reflected or emitted by the sample 506 is reflected by a second mirror 507 and captured by the second camera 502 .
- a pivot location offset from the first mirror 508 is used to move the first mirror between its first 503 and second 504 positions.
- a slide with detents is used to move the first mirror 508 between its first 503 and second 504 positions,
- the slide can be made from metal, plastic, or other suitable rigid materials.
- FIG. 6 illustrates a flip mirror configuration in accordance with an embodiment.
- the cameras 601 and 602 remain fixed in place.
- a fully reflective mirror 608 can be moved between a first 603 and second 604 position.
- the mirror 608 is in the first position 603
- light 605 reflected or emitted by the sample 606 is captured by the first camera 601 .
- the mirror 608 is in the second position 604
- light 605 reflected or emitted by the sample 606 is captured by the second camera 602 .
- a pivot location offset from the first mirror 608 is used to move the first mirror between its first 603 and second 604 positions.
- a slide with detents is used to move the first mirror 608 between its first 603 and second 604 positions,
- the slide can be made from metal, plastic, or other suitable rigid materials.
- the sample is imaged with a near infrared imaging system that provides a nearly full-rotation 3D tomographic image of surgical samples.
- the tomographic image can allow for the surgeon and pathologist to efficiently review putative disease areas in an augmented reality display with both fluorescence and reflective imaging modalities.
- the imaging device can incorporate one or both of true color and fluorescence imaging information with tomographic imaging information. Information associated with each imaging modality can be captured with a nearly 360° sample rotation stage and a 4-degree-of-freedom motion control system.
- the nearly full-rotation 3D pseudo-tomography imager can provide an efficient visualization of fluorescence signal at nearly every corner and nearly every facet on a true color 3D sample model.
- An animation of rotation images representing a 3D reconstructed model displayed on an interactive interface can present to the surgeon and pathologist the results in augmented reality to quickly identify disease areas on a surgical sample such as a tumor margin or biopsy.
- the proposed near-infrared (NIR) fluorescence pseudo-tomography imaging system comprises an NIR optical engine for excitation and image collection, a pseudo tomography platform for sample positioning to generate 3D images, a sample delivery system to optimize throughput, and an interactive user interface for efficient image review.
- NIR near-infrared
- Short wavelength NIR imaging around 700 to 800 nm has been well accepted in research applications due to advantages such as the availability of clinical and translational optical agents, high signal-to-noise ratios, low native tissue/bio-material fluorescence (autofluorescence), and low extinction at the excitation and emission wavelengths.
- Commercial products have been developed for small animal fluorescence imaging in the NIR range.
- the imaging system optical engine can also include laser sources in dual directions, white light LED illumination, specialized filter systems, a cooled imaging camera, and noise suppression technology.
- the laser sources can have wavelengths of, for example, one or more of 488 nm, 512 nm, 533 nm, 550 nm, 580 nm, 633 nm, 650 nm, 675 nm, 690 nm, 743 nm, 785 nm, or 806 nm.
- the laser sources are 690 nm and 785 nm for 700-nm and 800-nm channels, respectively.
- LED illumination can have wavelengths in one or more ranges from 450 nm to 850 nm, from 450 nm to 650 nm, from 550 nm to 750 nm, from 650 nm to 850 nm, from 450 nm to 550 nm, from 500 nm to 600 nm, from 550 nm, to 650 nm, from 600 nm to 700 nm, from 650 nm to 750 nm, from 700 nm to 800 nm, or from 750 nm to 850 nm.
- the LED illumination is at 670 nm and 760 nm for NIR dye excitation.
- the LED illumination is broad band illumination at visible wavelengths for white light illumination. Filter sets are designed to isolate excitation wavelength(s) with optimized emission collection for corresponding imaging agents (i.e. IRDye 800CW and ICG).
- the imaging system can offer high dynamic range and optimized signal-to-noise ratios with uniform illumination of the imaging field.
- the imaging dynamic range can be >6 logs (22-bit) in one imaging capture cycle. This dynamic range is higher than those (8-16 bit) commonly offered by other fluorescence imaging technologies.
- floating point values instead of long integer values
- imaging and processing speed is competitive.
- dark noise calibration is used to give imaging results high signal linearity for quantifiable evaluation.
- subcutaneous tumors IBDye 800CW-labeled agent, 15 nmol
- P-value P-value
- uniform illumination and field flattening correction technologies for laser(s) and LED (light emitting diode) white light channels are used to ensure the reliability and reproducibility of imaging throughout the whole imaging field.
- the uniformity and reliability can be such that coefficients of variation of approximately 0.04% are observed for repeated measurements at the same positions and ⁇ 3% for measurements at different positions in the illumination field.
- a pseudo-tomography imager can be capable of detecting indocyanine green (ICG), IRDye 800CW, and other dyes with emission in the 800-nm range.
- ICG is a clinically approved NIR fluorescence imaging contrast agent. In a dilute aqueous solution, ICG shows an absorption peak around the wavelength 780 nm and an emission peak around 822 nm.
- ICG As a systemically administrated imaging contrast agent, ICG offers a favorable safety profile and has been approved by U.S. Food and Drug Administration (FDA) for imaging of lymphatic flow, evaluation of hepatic function and liver blood flow, cardiac output, and other indications.
- FDA Food and Drug Administration
- Clinical trials for intraoperative imaging of tumors with ICG have been conducted on a variety of cancers. The use of ICG in tumor detection may be limited because ICG does not contain a reactive functional group for conjugation to target molecules specific to tumor tissues. Nevertheless, clinical studies show ICG may be beneficial for tumor tissue localization in sentinel lymph nodes and in liver, due to pooling effects of dye in the lymphatic system and uptake of ICG by hepatic parenchymal cells in liver.
- Focal accumulation or fluorescent rim in hepatocellular carcinoma can be visualized by an embodiment imaging device to indicate regions of interest in resected tissues prior to frozen analysis for intraoperative consultation.
- an embodiment imaging device to indicate regions of interest in resected tissues prior to frozen analysis for intraoperative consultation.
- NIR imaging agent currently approved by FDA
- ICG permits rapid clinical translation of the proposed imaging device to intraoperative consultation.
- IRDye 800CW has been used in clinical studies and trials for the detection of tumors. It offers reactive groups for easy conjugation with antibodies and superior solubility in aqueous solutions. IRDye 800CW advantages include that the dye is manufactured under cGMP and suitable for human use currently in investigational studies. A rodent toxicity study with IRDye 800CW carboxylate showed no toxic effects under the conditions of the study. A non-human primate toxicity study was completed in 2013 and showed no clinically significant toxicities from cetuximab-IRDye 800CW. Multiple human studies and clinical trials of IRDye 800CW-conjugates as imaging agents are being conducted. To date, the study results and clinical trial outcomes with IRDye 800CW-conjugates are encouraging.
- EGFR epidermal growth factor receptor
- VEGF anti-vascular endothelial growth factor
- Sensitivity and specificity are key metrics to validate the imaging technologies applied in a 3D imager.
- nude mice (Charles River Lab) received HNSCC (head and neck squamous carcinoma) cells to generate xenograft tumors. Imaging experiments were performed 48-96 hours following a systemic injection of panitumumab-IRDye 800CW. IgG-IRDye 800CW was used to assess non-specific binding. It was shown that the new imaging technology gives high tumor-to-background ratio with low nonspecific or native fluorescence background. The high contrast and tumor-to-background ratio is due at least in part to linear background suppression, high dynamic range technologies, and other imaging advances.
- a substantially uniform illumination area (3% variation) and imaging depth of focus will accommodate a 3-inch cube with resolution of ⁇ 125 ⁇ m.
- Other embodiments establish an imaging volume of a >4-inch cube.
- the disclosed systems, devices, and methods can offer nearly full 3D imaging, presenting nearly full or Euler animated rotations of the resulting reconstructed model.
- the pseudo-tomography imager provides fluorescence imaging and reflective white light imaging co-localized in a nearly full-rotation 3D image.
- a tomography platform is coordinated with the optical engine to provide the nearly full-rotation 3D imaging result.
- the imaging volume of the tomography system is a 3D space that accommodates the resected tissue sample.
- the imaging volume of this system is defined by the illumination fields of the excitation light, the depth of focus of the objective, and the field of view of the imaging head.
- the imaging volume can have a depth (in the x-axis) of, for example, 1 inch, 1.5 inches, 2 inches, 2.5 inches, 3 inches, 3.5 inches, 4 inches, 4.5 inches, or 5 inches.
- the imaging volume can have a cross-section (in the x-y plane) of, for example, 7 ⁇ 8 cm 2 , 7 ⁇ 10 cm 2 , 7 ⁇ 12 cm 2 , 8.5 ⁇ 8 cm 2 , 8.5 ⁇ 10 cm 2 , 8.5 ⁇ 12 cm 2 , 10 ⁇ 8 cm 2 , 10 ⁇ 10 cm 2 , or 10 ⁇ 12 cm 2 .
- the imaging resolution can be, for example, about 50 line pairs per inch, about 60 line pairs per inch, about 70 line pairs per inch, about 80 line pairs per inch, about 90 line pairs per inch, about 100 line pairs per inch, about 110 line pairs per inch, about 120 line pairs per inch, about 130 line pairs per inch, about 140 line pairs per inch, about 150 line pairs per inch, about 200 line pairs per inch, about 250 line pairs per inch, about 300 line pairs per inch, about 350 line pairs per inch, about 400 line pairs per inch, about 450 line pairs per inch, about 500 line pairs per inch, about 550 line pairs per inch, or about 600 line pairs per inch.
- the tomography platform is equipped with rotational motors and stages to control the view angle and position of a sample within the imaging volume.
- an imager can efficiently provide a nearly full rotation 3D image.
- the first rotation is a nearly 360-degree movement along the z-axis (roll) relative to the sample to collect images at serial view angles.
- the second rotation is tilting along y-axis (pitch) for imaging at different perspectives. Tilting of the sample stage allows projection views from the top and bottom of the sample via a transparent glass window. Rotation combinations allow nearly the entire sample to be imaged. Translational movements of the sample stage in X-Y plane allow the registration of the sample to the center of the imaging volume.
- the tomography platform rotates the object in two degrees-of-freedom.
- the tilting angle is typically in the range from 7.5 degrees to 45 degrees, depending on the complexity of the sample.
- sample holding structures such as pins, clamps, or stops, larger tilting angle can be achieved.
- a rolling step of 22.5 degree and a tilting angle at ⁇ 35 degrees in an embodiment can offer a nearly full rotation animation for 3D inspection.
- an imaging device should be capable of imaging a tissue having a size of ⁇ 3-inch diameter or larger with resolution better than 150 ⁇ m.
- the sample stage can be designed with lightweight material to accommodate large samples.
- the sample stage has a custom-made glass window with marking dimensions at its edges.
- the sample to be imaged is supported by a platform or stage having a transparent portion or window. In some embodiments, the entire platform or stage is transparent.
- FIG. 7 illustrates a transparent window 701 for object registration in accordance with an embodiment.
- the window 701 is marked with scale marks 702 and a circle 703 .
- the circle 703 marks the suggested border within which a sample to be imaged should be placed.
- the window can be transparent at the working wavelengths for both reflective light and fluorescence imaging. To accommodate a large size sample, the window can be custom made to a shape that is wider than the projection size of the imaging volume or the footprint of a target sample.
- the material of the window can be borosilicate based glass, or other transparent material.
- the surface could be treated or coated for optical (anti-reflection, transparency, absorption purposes) or surface functional (hydrophobic or hydrophilic properties, marks, barcodes, etc.) requirements.
- a circle on the window can be used to mark the border of suggested imaging area.
- Tick marks along the window can provide reference scales to users.
- the tick marks also can allow the construction software to identify and calculate the dimensions with references extending into the spatial volume.
- FIG. 8 (left side) illustrates the virtual reference marks extended from the glass window surface at a known rotation angles (along x, y, and z axis).
- the scale references can be used to help the software to recognize the dimension of an object.
- the software is shown to be able to generate precise 3D grids and sub-volumes.
- the information at each projection view and each view angle allows the software to label the dimension, position, points, features, voxels at different facets of a sample.
- the generated 3D grid renders the interception points of an object inside the confined virtual volume. The information assists the reconstruction of the object and the rendering of signal distributions.
- the imaging analysis may need to apply a correction factor in volume to compensate for decay along the sample.
- FIGS. 9 and 10 illustrate fluorescence image corrections of a rotated sample at one or two points, respectively, in accordance with an embodiment.
- FIG. 9 demonstrates that the excitation intensity of a divergent beam 901 at a signal spot 902 on an object 903 varies when the object rotates or moves to a different position.
- FIG. 10 demonstrates that a controlled diverging light 1001 would give different excitation energy density at a first 1002 and second 1003 spot on an object 1004 because the distances from the excitation source to the spots are different even absent rotation of the object.
- the excitation light goes along the propagation axis towards an object with controlled divergence or defined illumination pattern.
- the excitation radiation comes from a point source and is diverging toward the imaging volume for fluorescence excitation. If a signal spot is located at one end of an object, the energy density of the excitation wavefront at the area of the signal spot varies as the object is rotated to a different view angle.
- the function of light intensity along the propagation axis can be specified or can be measured.
- a correction factor can be applied into the signal reconstruction model.
- the 3D information collected via the reflective light imaging gives the geometric information at the spots of interest.
- the correction factor of fluorescence signal can then be implemented to improve the reliability and quantifiability of the imaging result.
- scattering and extinction of the signal light going through a part of the tissue object can be considered to further improve the fidelity of reconstructed results under superficial layers.
- a tomographic imager can generate a series of images corresponding to different view angles automatically in short period of time.
- Collocated serial images of two modalities are obtained by overlaying images of reflective light and fluorescence together.
- Software code is used to compile the serial imaging results into an animation to demonstrate the nearly full-rotation 3D views of a sample.
- the animation offers real-time 3D perspectives of nearly the entire sample.
- commercial 3D reconstruction algorithms can be used.
- the imaging process of an entire sample at 85 ⁇ m resolution can be done in 20 min or less.
- a 3D animation can be generated within a minute or less.
- FIGS. 11-13 illustrate a GUI and its touch screen manipulation in accordance with an embodiment.
- a tablet display can be used by a user to browse views interactively in two rotation axes (roll and pitch).
- a frame rate of 3-10 frames/second which may be changeable for a dynamic animation preview
- a user can scroll among different frames on a touch screen for static review of each frame with zoom-in up to 5-fold, ( ⁇ 12.5 ⁇ m/pixel display resolution) for detailed inspection. This can further help sample inspection and disease tissue localization.
- FIGS. 11A and 11B a nearly full-rotation imaging display is used to localize disease tissue.
- FIG. 11A a 3D imaging result of a xenograft tumor tissue on a mouse limp with (upper panel) and without (bottom panel) skin is shown.
- FIG. 11B an example of selected 3D animation shows that the top and bottom views of a full rotation 3D imaging result help localize the disease tissue.
- a pseudo color map for fluorescence is provided.
- FIG. 12 example images of human samples to evaluate the minimal amount of detectable tumor are shown.
- the upper row of FIG. 12 shows images of tissues collected from a systemic labeled patient cohort.
- the bottom row of FIG. 12 shows images of selected 3D rotation animation frames from an example 3D visualization result.
- a pseudo color map for fluorescence is provided.
- Gray scale is for reflective light.
- FIGS. 13A and 13B an interactive animation allows the user to scroll through the 3D result and take a specific view for detail inspection.
- FIG. 13A shows an example of a user scrolling in a rolling mode. That is, a user can apply touch gestures to scroll through different azimuth viewpoint angles around a single elevation.
- FIG. 13B shows an example of a user scrolling in a pitching/tilting mode. That is, a user can apply touch gestures to scroll through different elevation viewpoint angles along a single azimuth.
- an augmented reality display interface will allow the user to review 3D views by rolling and tilting the tablet display in which a gyroscope package senses the motion and provides the information to interact with the image output.
- a 3D reconstruction algorithm can offer full-rotation 3D model for an augmented reality application.
- the reconstruction algorithm one can apply volume carving to form spatial volume, together with feature recognition techniques to locate common features.
- a projection method known as “silhouette projection” takes an image into a virtual volume by carving away the voxels outside the silhouette projection at each view angle. Overall geometry of an object can then be derived. By matching common features, an object can be verified and incorporated into the model.
- an intensity scalar related to the adjusted fluorescence color map of each view is retrieved for 3D rendering. Coordinates, features, and scalars defined in the volume reconstruction process are used to match 2D and 3D results in rendering the signal distribution. Tissue mimicking phantoms and tissue constructs are accounted for in algorithm validation.
- the input lagging of an augmented reality 3D model is determined by the application algorithm.
- An input lag can be measured and expected to achieve ⁇ 100 ms lagging for a typical 3D model so that user will not experience serious human detectable latency while viewing a model interactively.
- FIGS. 14 through 40 illustrate overlaid full-color and fluorescence images of an object at different angles in accordance with an embodiment. Flipping through the figures rapidly shows a short animation. The first 17 images complete a 360° circuit around the sample at 22.5° increments, ending at the same azimuth angle as the beginning image. The next 11 images show the sample from higher and lower elevations (i.e., higher and lower tilts) at a constant azimuth angle.
- the animation can be automatically played or can be interactively controlled. For example, a surgeon can stop the animation at a selected frame, zoom in, or tilt up and down to see more of the sample.
- a surgeon who operates a surgery only touches tools that are sterilized.
- a technologist or other staff member assists a surgeon by helping to manipulate information presented on a display of any instrument.
- actions taken by the staff may not accurately or effectively accord with the verbal commands and requests from a surgeon.
- Touching of instruments such as a computer, keyboards, display panels, or a cabinet imager may break the sterilization, though, and create contamination problems.
- the use of a sterile touch pen to operate a display or interface on a screen can therefore assist in maintaining a sterile environment in an operating room.
- FIG. 42 illustrates one embodiment as a descriptive example. Shown is a touch pen 4200 comprising a pen body 4201 and a pen tip 4202 .
- the touch pen can also comprise a pen cover 4203 that encloses the pen body 4201 and the pen tip 4202 .
- the pen body 4201 can be made of disposable and pre-sterilized material intended for one-time or limited-time use.
- the pen body 4201 can be or made of sterilizable material intended for repeated use with sterilization occurring prior to each use.
- one or both of the pen body 4201 and the pen tip 4202 comprise a metal.
- the metal is stainless steel.
- the pen tip 4202 is detachable from the pen body 4201 .
- the pen tip 4202 can be made of disposable and pre-sterilized material intended for one-time or limited-time use.
- the touch pen can be enclosed in a pen cover 4203 that is made of disposable and pre-sterilized material intended for one-time or limited-time use.
- the pen body 4201 and pen tip 4202 are not sterile, but the pen cover 4203 is sterile.
- the touch pen can dispense ink. In some embodiments, the touch pen does not dispense ink.
- An embodiment can use a high-performance monochrome imaging camera with a CCD sensor.
- CMOS sensors may become the norm in fluorescence applications.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Biochemistry (AREA)
- Optics & Photonics (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Investigating Or Analysing Biological Materials (AREA)
- Medical Informatics (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
Abstract
Description
- This application is a continuation of U.S. patent application Ser. No. 16/529,463, filed Aug. 1, 2019, which is a continuation of U.S. patent application Ser. No. 15/192,771, filed Jun. 24, 2016 (U.S. Pat. No. 10,379,048, issued Aug. 13, 2019), which claims the benefit of U.S. Provisional Patent Application Nos. 62/185,407, filed Jun. 26, 2015, 62/325,588, filed Apr. 21, 2016, and 62/339,657, filed May 20, 2016, all of which are incorporated by reference in their entireties for all purposes.
- NOT APPLICABLE
- A primary purpose of cancer surgery and surgical pathology is to remove tumor tissues. Complete removal of tumors and all tumor tissue is important. If a tumor is not completely removed, then a second surgery may be required. This produces unneeded patient anxiety and risk, delays the start of second stage therapies, and incurs additional financial costs.
- In order to completely remove all tumor tissue, surgeons and pathologists work together during surgery to assess tumor margins such that no malignant cells remain in the wound bed. Tumor margins are the healthy tissue surrounding the tumor, more specifically, the distance between the tumor tissue and the edge of the surrounding tissue removed along with the tumor. Ideally, the margins are selected so that the risk of leaving tumor tissue within the patient is low.
- Intraoperative pathology consultation guides the cancer surgeon during surgery to assess if the tumor and margin have not been completely excised. Current surgical pathology consultation relies on palpation, visualization, and anatomical features to select sampling areas for frozen section analysis, which is often limited to 20-30 minutes. Intraoperative pathology consultation starts with gross examination of the excised tissues to choose areas for sampling. This is followed by dissection, embedding, frozen sectioning, and staining of 5-μm slice sections of the excised tissue.
- The limited surgical consultation timeframe allows only a small portion of the resected tissue to be interpreted by the pathologist. This time crunch can result in sampling errors which negatively impact the patient outcome.
- There exists a need in the art for better intraoperative tumor tissue analysis.
- Generally disclosed is a device that provides a nearly full, 360° rotation plus limited elevation-angle, three-dimensional (3D) images of a tumor sample that has been systemically labeled using a fluorescence dye. A first camera takes full-color pictures of the tumor sample at different azimuth and elevation angles; a second camera captures fluorescence images of the sample at the same angles while the sample is illuminated at an excitation frequency.
- Because of the low light levels of fluorescence, sensitivity-improving approaches, such as active cooling, imaging with a monochrome sensor, long exposure durations, and electronic noise suppression methods, can be used with the second camera.
- The resulting full-color and fluorescence images are either overlaid or used to make 3D computer models that are overlaid in space. For making 3D models, calibration marks on a window platform help the software generate precise 3D grids and sub-volumes in order to determine the position of each surface in the images.
- The imager can generate a full rotation 3D image of a sample and present it in interactive animation as well as in an augmented realty 3D model to help localize diseased regions around the whole surgical sample.
- A graphical user interface (GUI) aids a surgeon by presenting the images in a filmstrip manner on a touch screen device. A surgeon can swipe the filmstrip left or right in order to rotate views in azimuth around the sample. By swiping up or down, the surgeon can access views of the sample taken from higher or lower in elevation.
- Also provided is a touch pen comprising a pen body and a pen tip. The pen tip is attached to an end of the pen body. The touch pen is sterile and does not dispense ink. In some embodiments, the pen body comprises stainless steel. In some embodiments, the pen tip is detachable from the pen body and replaceable with a second pen tip. In some embodiments, the touch pen further comprises a pen cover. The pen cover encloses the pen body and the pen tip, and is sterile.
- Due to divergence, excitation light excites fluorescence less at farther points on the sample. This can be corrected for by determining the distance of each surface from the fluorescence excitation light source and adjusting the resulting reading's intensity.
- The device will help localize disease tissues to reduce sampling errors and re-examination, and ultimately reduce local cancer recurrence and second surgery costs. The device can be used for image-guided pathology to screen resected tissue or biopsy triage in conjunction with image-guide surgery.
-
FIG. 1 illustrates a multiple-camera imaging system in accordance with an embodiment. -
FIG. 2 illustrates a switchable camera configuration in accordance with an embodiment. -
FIG. 3 illustrates a beam splitting configuration in accordance with an embodiment. -
FIG. 4 illustrates a dichroic mirror configuration in accordance with an embodiment. -
FIG. 5 illustrates a flip mirror configuration in accordance with an embodiment. -
FIG. 6 illustrates a second flip mirror configuration in accordance with an embodiment. -
FIG. 7 illustrates transparent window with scale marks in accordance with an embodiment. -
FIG. 8 illustrates using the use of scale marks on the transparent window ofFIG. 7 for object registration in accordance with an embodiment. -
FIG. 9 illustrates fluorescence image correction of a rotated sample in accordance with an embodiment. -
FIG. 10 illustrates fluorescence image correction of two points on a sample in accordance with an embodiment. -
FIG. 11A illustrates images of xenograft tumor tissue. -
FIG. 11B illustrates a filmstrip view of 3D animation presenting images of the tumor ofFIG. 11A in accordance with an embodiment. -
FIG. 12 illustrates the imaging of different tumor sizes and a filmstrip presentation manner of a graphical user interface (GUI) in accordance with an embodiment. -
FIG. 13A illustrates touch screen manipulation of a GUI in a rolling mode in accordance with an embodiment. -
FIG. 13B illustrates touch screen manipulation of a GUI in a pitching mode in accordance with an embodiment. -
FIGS. 14-41 illustrate overlaid full-color and fluorescence images of an object at different angles in accordance with an embodiment. -
FIG. 42 is an illustration of a touch pen in accordance with an embodiment. - Intraoperative consultations are among the most stressful parts of a pathologist's duties. With a strict time limit, little or no ancillary studies available, and using only touch and visual cues to select a few tissue sections to analyze, the pathologist must make a diagnosis where clinical consequences are quite significant.
- Imaging modalities currently used in patient care include magnetic resonance imaging (MRI), ultrasound, radiography (X-ray), single-photon emission computed tomography (SPECT), and positron emission tomography (PET). These imaging techniques are not generally used to localize disease in tissues for margin determinations. The reason for this is in part that the resolution offered by these five imaging technologies is not sufficient.
- Further, the non-specific contrast agents typically used by MRI, ultrasound, and X-ray computed tomography do not target tumors. Additionally, the use of radioactive isotopes in SPECT and PET imaging introduces safety and wellness concerns and is financially costly for both patients and care providers.
- An alternative pseudo tomography imaging system embodiment can be used to satisfy the critical criteria for clinical utility. The proposed technology can provide the speed and throughput, the resolution, and the sensitivity and specificity needed to offer a solution for enhancing intraoperative pathology consultation. The described embodiments can also be used within the current work flow of surgeries without significant alterations.
- An imaging device as described herein can relieve the diagnosis stress while helping to visualize and identify suspicious areas in lesion samples. It is foreseeable that the device will be used to assist intraoperative consultation of primary tumors and biopsies of head and neck cancers, lymph nodes, hepatobiliary, breast, female genital tract, skin tumors, etc. The proposed technology would also provide cost savings to the healthcare system by reducing reoperations due to recurrence. It is notable that this invention can provide a new paradigm in cancer surgery without causing a deviation from current work flow.
-
FIG. 1 illustrates a multiple-camera imaging system in accordance with an embodiment. The imaging system comprises anoptical engine 101 for excitation and signal collection; apseudo-tomography platform 102 comprising arotatable stage 108 for positioning and rotation of asample 103; and software (not shown) for operation, imaging, and generation of 3D results. Theoptical engine 101 comprises afirst imaging camera 104 and asecond imaging camera 105. Typically the first 104 and second 105 imaging cameras record information using two different imagining modalities. For example, one of the two cameras can be a color camera used for reflected light imaging. The other camera can be a monochromatic camera for fluorescence signal collection. A monochromatic camera includes an imaging device having a sensitivity to a particular wavelength or range of wavelengths while having an insensitivity to a second range of wavelengths, or as otherwise known in the art. - The
optical engine 101 can also comprise apreview camera 106. Thepreview camera 106 can be a true-color zoom camera module used to take preview images and videos in real-time at user-selected view angles. - A
laser system 107 can be positioned and configured to illuminatesample 103 with an excitation wavelength. In some embodiments, the excitation wavelength is within the second range of wavelengths that the monochromatic camera is insensitive to. - All three of the
cameras sample 103 on thepseudo-tomography platform 102. The illumination light fields and fields-of view of thecameras - The
optical engine 101 can also comprise acooling line 109. Thecooling line 109 can be used provide liquid or other coolant tocamera 105 in order to maintain a preferred camera temperature and increase camera sensitivity. - The
tomography platform 102 allows nearly full-rotation sample scanning and positioning for tomography imaging and sample loading. This is accomplished through tilting and rotation of thesample stage 107 holding thesample 103. Tilting of thesample stage 107 allows projection views of the top and bottom of thesample 103. - In some embodiments, and as shown in
FIG. 1 , the system comprises two imaging cameras as well as a preview camera. The preview camera can be used for taking a ‘preview’ picture of the entire sample for annotation of a respective data file storing image information. The preview pictures can also be used to determine if the sample is properly within the imaging volume. An embodiment can include a full-color camera to provide a sample preview (at, for example, 1280×720 pixels, 15 frame/second, >4-inch field of view) and an isometric picture of a sample for documentary requirement before beginning the 3D tomographic imaging. A software application can be used for a real-time preview function. - In some embodiments, a first imaging camera is used for collecting an image over a broad spectrum of visible wavelengths. In some embodiments, the first camera is a grayscale camera. In some embodiments, the first camera is a full color camera.
- In some embodiments, the second imaging camera is a cooled camera configured to capture fluorescence images at very low light levels. In some embodiments, a cooling fan, liquid cooling elements, and other cooling systems can be used to cool the second imaging camera. The camera can be optimized for collecting near infrared wavelength photons.
- The imaging cameras are configured to take pictures of the sample from the same orientation and at the same focal depth as one another. There are several different ways to achieve this.
-
FIG. 2 illustrates a switchable camera configuration in accordance with an embodiment. The first 201 and second 202 imaging cameras can be rotated or translated into a position such that their optics focus at the same depth within the imaging volume. This can be accomplished by alternately positioning the first 201 and second 202 cameras such that one of these cameras at a time is positioned to receive light 203 reflecting or emitting from thesample 204 and reflecting off of amirror 205. In some embodiments, a pivot location offset from the first 201 and second 202 cameras is used to rotate the cameras. In some embodiments, a slide with detents is used to translate the first 201 and second 202 cameras, - The slide can be made from metal, plastic, or other suitable rigid materials.
-
FIG. 3 illustrates a beam splitting configuration in accordance with an embodiment. In this configuration, thecameras beam splitter 303 is used to receive incident light 304 reflecting or emitting from thesample 305 and split the incident light into reflected light 306 and transmittedlight 307. The reflectedlight 306 is captured by thefirst camera 301, and the transmittedlight 307 is captured by thesecond camera 302 after reflecting off of amirror 308. -
FIG. 4 illustrates a dichroic mirror configuration in accordance with an embodiment. In this configuration, thecameras dichroic mirror 403 is used to receive incident light 404 reflecting or emitting from thesample 405 and split the incident light into reflected light 406 and transmittedlight 407. The reflectedlight 406 is captured by thefirst camera 401, and the transmittedlight 407 is captured by thesecond camera 402 after reflecting off of a fullyreflective mirror 408. - The
dichroic mirror 403 can be configured to reflect a majority of light outside of a particular range of wavelengths. Thedichroic mirror 403 can transmit light within the particular range of wavelengths tocamera 402, which can be a cooled monochromatic camera. -
FIG. 5 illustrates a flip mirror configuration in accordance with an embodiment. In this configuration, thecameras first mirror 508 can be moved between a first 503 and second 504 position. When thefirst mirror 508 is in thefirst position 503, light 505 reflected or emitted by thesample 506 is captured by thefirst camera 501. When thefirst mirror 508 is in thesecond position 504, light 505 reflected or emitted by thesample 506 is reflected by asecond mirror 507 and captured by thesecond camera 502. - In some embodiments, a pivot location offset from the
first mirror 508 is used to move the first mirror between its first 503 and second 504 positions. In some embodiments, a slide with detents is used to move thefirst mirror 508 between its first 503 and second 504 positions, The slide can be made from metal, plastic, or other suitable rigid materials. -
FIG. 6 illustrates a flip mirror configuration in accordance with an embodiment. In this configuration, thecameras reflective mirror 608 can be moved between a first 603 and second 604 position. When themirror 608 is in thefirst position 603, light 605 reflected or emitted by thesample 606 is captured by thefirst camera 601. When themirror 608 is in thesecond position 604, light 605 reflected or emitted by thesample 606 is captured by thesecond camera 602. - In some embodiments, a pivot location offset from the
first mirror 608 is used to move the first mirror between its first 603 and second 604 positions. In some embodiments, a slide with detents is used to move thefirst mirror 608 between its first 603 and second 604 positions, The slide can be made from metal, plastic, or other suitable rigid materials. - In some embodiments, the sample is imaged with a near infrared imaging system that provides a nearly full-rotation 3D tomographic image of surgical samples. The tomographic image can allow for the surgeon and pathologist to efficiently review putative disease areas in an augmented reality display with both fluorescence and reflective imaging modalities. The imaging device can incorporate one or both of true color and fluorescence imaging information with tomographic imaging information. Information associated with each imaging modality can be captured with a nearly 360° sample rotation stage and a 4-degree-of-freedom motion control system.
- The nearly full-rotation 3D pseudo-tomography imager can provide an efficient visualization of fluorescence signal at nearly every corner and nearly every facet on a true color 3D sample model. An animation of rotation images representing a 3D reconstructed model displayed on an interactive interface can present to the surgeon and pathologist the results in augmented reality to quickly identify disease areas on a surgical sample such as a tumor margin or biopsy.
- The proposed near-infrared (NIR) fluorescence pseudo-tomography imaging system comprises an NIR optical engine for excitation and image collection, a pseudo tomography platform for sample positioning to generate 3D images, a sample delivery system to optimize throughput, and an interactive user interface for efficient image review.
- Short wavelength NIR imaging around 700 to 800 nm has been well accepted in research applications due to advantages such as the availability of clinical and translational optical agents, high signal-to-noise ratios, low native tissue/bio-material fluorescence (autofluorescence), and low extinction at the excitation and emission wavelengths. Commercial products have been developed for small animal fluorescence imaging in the NIR range.
- The imaging system optical engine can also include laser sources in dual directions, white light LED illumination, specialized filter systems, a cooled imaging camera, and noise suppression technology. The laser sources can have wavelengths of, for example, one or more of 488 nm, 512 nm, 533 nm, 550 nm, 580 nm, 633 nm, 650 nm, 675 nm, 690 nm, 743 nm, 785 nm, or 806 nm. In some embodiments, the laser sources are 690 nm and 785 nm for 700-nm and 800-nm channels, respectively. LED illumination can have wavelengths in one or more ranges from 450 nm to 850 nm, from 450 nm to 650 nm, from 550 nm to 750 nm, from 650 nm to 850 nm, from 450 nm to 550 nm, from 500 nm to 600 nm, from 550 nm, to 650 nm, from 600 nm to 700 nm, from 650 nm to 750 nm, from 700 nm to 800 nm, or from 750 nm to 850 nm. In some embodiments, the LED illumination is at 670 nm and 760 nm for NIR dye excitation. In some embodiments, the LED illumination is broad band illumination at visible wavelengths for white light illumination. Filter sets are designed to isolate excitation wavelength(s) with optimized emission collection for corresponding imaging agents (i.e. IRDye 800CW and ICG).
- The imaging system can offer high dynamic range and optimized signal-to-noise ratios with uniform illumination of the imaging field. The imaging dynamic range can be >6 logs (22-bit) in one imaging capture cycle. This dynamic range is higher than those (8-16 bit) commonly offered by other fluorescence imaging technologies. To relieve the processing loads from high dynamic range imaging, one can apply floating point values (instead of long integer values) for each pixel to maximize dynamic range without increasing image size. Thus imaging and processing speed is competitive.
- In some embodiments, dark noise calibration is used to give imaging results high signal linearity for quantifiable evaluation. For example, testing on subcutaneous tumors (IRDye 800CW-labeled agent, 15 nmol) of a mouse model has demonstrated linear imaging signal versus tumor tissue weight (Pearson r=0.97 with P-value<0.0001).
- In some embodiments, uniform illumination and field flattening correction technologies for laser(s) and LED (light emitting diode) white light channels are used to ensure the reliability and reproducibility of imaging throughout the whole imaging field. The uniformity and reliability can be such that coefficients of variation of approximately 0.04% are observed for repeated measurements at the same positions and <3% for measurements at different positions in the illumination field.
- In some embodiments, a pseudo-tomography imager can be capable of detecting indocyanine green (ICG), IRDye 800CW, and other dyes with emission in the 800-nm range. ICG is a clinically approved NIR fluorescence imaging contrast agent. In a dilute aqueous solution, ICG shows an absorption peak around the wavelength 780 nm and an emission peak around 822 nm.
- As a systemically administrated imaging contrast agent, ICG offers a favorable safety profile and has been approved by U.S. Food and Drug Administration (FDA) for imaging of lymphatic flow, evaluation of hepatic function and liver blood flow, cardiac output, and other indications. Clinical trials for intraoperative imaging of tumors with ICG have been conducted on a variety of cancers. The use of ICG in tumor detection may be limited because ICG does not contain a reactive functional group for conjugation to target molecules specific to tumor tissues. Nevertheless, clinical studies show ICG may be beneficial for tumor tissue localization in sentinel lymph nodes and in liver, due to pooling effects of dye in the lymphatic system and uptake of ICG by hepatic parenchymal cells in liver. Focal accumulation or fluorescent rim in hepatocellular carcinoma can be visualized by an embodiment imaging device to indicate regions of interest in resected tissues prior to frozen analysis for intraoperative consultation. As the only NIR imaging agent currently approved by FDA, ICG permits rapid clinical translation of the proposed imaging device to intraoperative consultation.
- LI-COR's IRDye 800CW has been used in clinical studies and trials for the detection of tumors. It offers reactive groups for easy conjugation with antibodies and superior solubility in aqueous solutions. IRDye 800CW advantages include that the dye is manufactured under cGMP and suitable for human use currently in investigational studies. A rodent toxicity study with IRDye 800CW carboxylate showed no toxic effects under the conditions of the study. A non-human primate toxicity study was completed in 2013 and showed no clinically significant toxicities from cetuximab-IRDye 800CW. Multiple human studies and clinical trials of IRDye 800CW-conjugates as imaging agents are being conducted. To date, the study results and clinical trial outcomes with IRDye 800CW-conjugates are encouraging.
- FDA approved anti-epidermal growth factor receptor (EGFR) monoclonal antibodies, cetuximab and panitumumab, and anti-vascular endothelial growth factor (VEGF) antibody, bevacizumab, have been conjugated to IRDye 800CW for tumor targeted imaging. A systemic dose escalation study has shown safety and efficacy in a clinical trial. Other similar moieties can also be attached to the dye. Some of these additional moiety attachments can also be undergoing current clinical trials.
- Sensitivity and specificity (reducing false negatives) are key metrics to validate the imaging technologies applied in a 3D imager. One study demonstrated that the presently disclosed imaging module can produce superior sensitivity, specificity, detection limits and signal contrast. In this study, nude mice (Charles River Lab) received HNSCC (head and neck squamous carcinoma) cells to generate xenograft tumors. Imaging experiments were performed 48-96 hours following a systemic injection of panitumumab-IRDye 800CW. IgG-IRDye 800CW was used to assess non-specific binding. It was shown that the new imaging technology gives high tumor-to-background ratio with low nonspecific or native fluorescence background. The high contrast and tumor-to-background ratio is due at least in part to linear background suppression, high dynamic range technologies, and other imaging advances.
- In some embodiments, a substantially uniform illumination area (3% variation) and imaging depth of focus will accommodate a 3-inch cube with resolution of ˜125 μm. Other embodiments establish an imaging volume of a >4-inch cube.
- In contrast with conventional small animal fluorescence imaging which provide images of partial 3D faces, the disclosed systems, devices, and methods can offer nearly full 3D imaging, presenting nearly full or Euler animated rotations of the resulting reconstructed model. In some embodiments, the pseudo-tomography imager provides fluorescence imaging and reflective white light imaging co-localized in a nearly full-rotation 3D image.
- In some embodiments, a tomography platform is coordinated with the optical engine to provide the nearly full-rotation 3D imaging result. The imaging volume of the tomography system is a 3D space that accommodates the resected tissue sample. The imaging volume of this system is defined by the illumination fields of the excitation light, the depth of focus of the objective, and the field of view of the imaging head. The imaging volume can have a depth (in the x-axis) of, for example, 1 inch, 1.5 inches, 2 inches, 2.5 inches, 3 inches, 3.5 inches, 4 inches, 4.5 inches, or 5 inches. The imaging volume can have a cross-section (in the x-y plane) of, for example, 7×8 cm2, 7×10 cm2, 7×12 cm2, 8.5×8 cm2, 8.5×10 cm2, 8.5×12 cm2, 10×8 cm2, 10×10 cm2, or 10×12 cm2. The imaging resolution can be, for example, about 50 line pairs per inch, about 60 line pairs per inch, about 70 line pairs per inch, about 80 line pairs per inch, about 90 line pairs per inch, about 100 line pairs per inch, about 110 line pairs per inch, about 120 line pairs per inch, about 130 line pairs per inch, about 140 line pairs per inch, about 150 line pairs per inch, about 200 line pairs per inch, about 250 line pairs per inch, about 300 line pairs per inch, about 350 line pairs per inch, about 400 line pairs per inch, about 450 line pairs per inch, about 500 line pairs per inch, about 550 line pairs per inch, or about 600 line pairs per inch.
- The tomography platform is equipped with rotational motors and stages to control the view angle and position of a sample within the imaging volume. By rotating a sample in two degrees of freedom, an imager can efficiently provide a nearly full rotation 3D image. The first rotation is a nearly 360-degree movement along the z-axis (roll) relative to the sample to collect images at serial view angles. The second rotation is tilting along y-axis (pitch) for imaging at different perspectives. Tilting of the sample stage allows projection views from the top and bottom of the sample via a transparent glass window. Rotation combinations allow nearly the entire sample to be imaged. Translational movements of the sample stage in X-Y plane allow the registration of the sample to the center of the imaging volume.
- To collect pertinent imaging projections along a sample for 3D reconstruction, the tomography platform rotates the object in two degrees-of-freedom. To provide comprehensive coverage of sample features, the tilting angle is typically in the range from 7.5 degrees to 45 degrees, depending on the complexity of the sample. With sample holding structures, such as pins, clamps, or stops, larger tilting angle can be achieved. A rolling step of 22.5 degree and a tilting angle at ±35 degrees in an embodiment can offer a nearly full rotation animation for 3D inspection.
- For many practical applications, an imaging device should be capable of imaging a tissue having a size of ˜3-inch diameter or larger with resolution better than 150 μm. The sample stage can be designed with lightweight material to accommodate large samples. In some embodiments, the sample stage has a custom-made glass window with marking dimensions at its edges.
- In some embodiments, the sample to be imaged is supported by a platform or stage having a transparent portion or window. In some embodiments, the entire platform or stage is transparent.
-
FIG. 7 illustrates atransparent window 701 for object registration in accordance with an embodiment. Thewindow 701 is marked with scale marks 702 and acircle 703. Thecircle 703 marks the suggested border within which a sample to be imaged should be placed. - The window can be transparent at the working wavelengths for both reflective light and fluorescence imaging. To accommodate a large size sample, the window can be custom made to a shape that is wider than the projection size of the imaging volume or the footprint of a target sample.
- The material of the window can be borosilicate based glass, or other transparent material. The surface could be treated or coated for optical (anti-reflection, transparency, absorption purposes) or surface functional (hydrophobic or hydrophilic properties, marks, barcodes, etc.) requirements.
- A circle on the window can be used to mark the border of suggested imaging area. Tick marks along the window can provide reference scales to users. The tick marks also can allow the construction software to identify and calculate the dimensions with references extending into the spatial volume.
-
FIG. 8 (left side) illustrates the virtual reference marks extended from the glass window surface at a known rotation angles (along x, y, and z axis). The scale references can be used to help the software to recognize the dimension of an object. In the middle figure, the software is shown to be able to generate precise 3D grids and sub-volumes. The information at each projection view and each view angle allows the software to label the dimension, position, points, features, voxels at different facets of a sample. In the rightmost figure ofFIG. 8 , with the information from a deck of rotation angles, the generated 3D grid renders the interception points of an object inside the confined virtual volume. The information assists the reconstruction of the object and the rendering of signal distributions. - Due to the divergent propagation of excitation light along the depth of imaging volume (x-axis), the imaging analysis may need to apply a correction factor in volume to compensate for decay along the sample.
-
FIGS. 9 and 10 illustrate fluorescence image corrections of a rotated sample at one or two points, respectively, in accordance with an embodiment.FIG. 9 demonstrates that the excitation intensity of adivergent beam 901 at asignal spot 902 on anobject 903 varies when the object rotates or moves to a different position.FIG. 10 demonstrates that a controlled diverging light 1001 would give different excitation energy density at a first 1002 and second 1003 spot on anobject 1004 because the distances from the excitation source to the spots are different even absent rotation of the object. - For large-area illumination, the excitation light goes along the propagation axis towards an object with controlled divergence or defined illumination pattern. In the figures, the excitation radiation comes from a point source and is diverging toward the imaging volume for fluorescence excitation. If a signal spot is located at one end of an object, the energy density of the excitation wavefront at the area of the signal spot varies as the object is rotated to a different view angle. For a point source (isotropic radiator), the energy density at a distance y follows the inverse-square law (I=1/y2). For a controlled divergence to achieve uniform illumination at an x-y plane, the function of light intensity along the propagation axis can be specified or can be measured. With a known function of energy density at a distance y, a correction factor can be applied into the signal reconstruction model. The 3D information collected via the reflective light imaging gives the geometric information at the spots of interest. The correction factor of fluorescence signal can then be implemented to improve the reliability and quantifiability of the imaging result. For a complicated model, scattering and extinction of the signal light going through a part of the tissue object can be considered to further improve the fidelity of reconstructed results under superficial layers.
- In some embodiments, a tomographic imager can generate a series of images corresponding to different view angles automatically in short period of time. Collocated serial images of two modalities are obtained by overlaying images of reflective light and fluorescence together. Software code is used to compile the serial imaging results into an animation to demonstrate the nearly full-rotation 3D views of a sample. The animation offers real-time 3D perspectives of nearly the entire sample. For the 3D reconstruction of the sample model and the interactive displaying of the 3D results, commercial 3D reconstruction algorithms can be used.
- In some embodiments, the imaging process of an entire sample at 85 μm resolution can be done in 20 min or less. In some embodiments, a 3D animation can be generated within a minute or less.
-
FIGS. 11-13 illustrate a GUI and its touch screen manipulation in accordance with an embodiment. A tablet display can be used by a user to browse views interactively in two rotation axes (roll and pitch). In addition to a frame rate of 3-10 frames/second, which may be changeable for a dynamic animation preview, a user can scroll among different frames on a touch screen for static review of each frame with zoom-in up to 5-fold, (˜12.5 μm/pixel display resolution) for detailed inspection. This can further help sample inspection and disease tissue localization. - In
FIGS. 11A and 11B , a nearly full-rotation imaging display is used to localize disease tissue. InFIG. 11A , a 3D imaging result of a xenograft tumor tissue on a mouse limp with (upper panel) and without (bottom panel) skin is shown. InFIG. 11B , an example of selected 3D animation shows that the top and bottom views of a full rotation 3D imaging result help localize the disease tissue. A pseudo color map for fluorescence is provided. - In
FIG. 12 , example images of human samples to evaluate the minimal amount of detectable tumor are shown. The upper row ofFIG. 12 shows images of tissues collected from a systemic labeled patient cohort. The bottom row ofFIG. 12 shows images of selected 3D rotation animation frames from an example 3D visualization result. A pseudo color map for fluorescence is provided. Gray scale is for reflective light. - In
FIGS. 13A and 13B , an interactive animation allows the user to scroll through the 3D result and take a specific view for detail inspection.FIG. 13A shows an example of a user scrolling in a rolling mode. That is, a user can apply touch gestures to scroll through different azimuth viewpoint angles around a single elevation.FIG. 13B shows an example of a user scrolling in a pitching/tilting mode. That is, a user can apply touch gestures to scroll through different elevation viewpoint angles along a single azimuth. - Furthermore, an augmented reality display interface will allow the user to review 3D views by rolling and tilting the tablet display in which a gyroscope package senses the motion and provides the information to interact with the image output.
- A 3D reconstruction algorithm can offer full-rotation 3D model for an augmented reality application. In the reconstruction algorithm one can apply volume carving to form spatial volume, together with feature recognition techniques to locate common features. A projection method known as “silhouette projection” takes an image into a virtual volume by carving away the voxels outside the silhouette projection at each view angle. Overall geometry of an object can then be derived. By matching common features, an object can be verified and incorporated into the model. After the model is created, an intensity scalar related to the adjusted fluorescence color map of each view is retrieved for 3D rendering. Coordinates, features, and scalars defined in the volume reconstruction process are used to match 2D and 3D results in rendering the signal distribution. Tissue mimicking phantoms and tissue constructs are accounted for in algorithm validation.
- As current tablet displays commonly have a refresh rate of >60 Hz and a response time of <8 ms, the input lagging of an augmented reality 3D model is determined by the application algorithm. An input lag can be measured and expected to achieve <100 ms lagging for a typical 3D model so that user will not experience serious human detectable latency while viewing a model interactively.
- FIGS. 14 through 40 illustrate overlaid full-color and fluorescence images of an object at different angles in accordance with an embodiment. Flipping through the figures rapidly shows a short animation. The first 17 images complete a 360° circuit around the sample at 22.5° increments, ending at the same azimuth angle as the beginning image. The next 11 images show the sample from higher and lower elevations (i.e., higher and lower tilts) at a constant azimuth angle.
- The animation can be automatically played or can be interactively controlled. For example, a surgeon can stop the animation at a selected frame, zoom in, or tilt up and down to see more of the sample.
- In a surgical workflow, a surgeon who operates a surgery only touches tools that are sterilized. In some surgical procedures, a technologist or other staff member assists a surgeon by helping to manipulate information presented on a display of any instrument. However, actions taken by the staff may not accurately or effectively accord with the verbal commands and requests from a surgeon. As a result, there can be a benefit to enabling surgeons to work with a display or instrument directly. Touching of instruments such as a computer, keyboards, display panels, or a cabinet imager may break the sterilization, though, and create contamination problems. The use of a sterile touch pen to operate a display or interface on a screen can therefore assist in maintaining a sterile environment in an operating room.
-
FIG. 42 illustrates one embodiment as a descriptive example. Shown is atouch pen 4200 comprising apen body 4201 and apen tip 4202. The touch pen can also comprise apen cover 4203 that encloses thepen body 4201 and thepen tip 4202. - The
pen body 4201 can be made of disposable and pre-sterilized material intended for one-time or limited-time use. Thepen body 4201 can be or made of sterilizable material intended for repeated use with sterilization occurring prior to each use. In some embodiments, one or both of thepen body 4201 and thepen tip 4202 comprise a metal. In some embodiments, the metal is stainless steel. In some embodiments, thepen tip 4202 is detachable from thepen body 4201. Thepen tip 4202 can be made of disposable and pre-sterilized material intended for one-time or limited-time use. The touch pen can be enclosed in apen cover 4203 that is made of disposable and pre-sterilized material intended for one-time or limited-time use. In some embodiments, thepen body 4201 andpen tip 4202 are not sterile, but thepen cover 4203 is sterile. In some embodiments, the touch pen can dispense ink. In some embodiments, the touch pen does not dispense ink. - An embodiment can use a high-performance monochrome imaging camera with a CCD sensor. With advances in CMOS imaging sensors, CMOS sensors may become the norm in fluorescence applications.
- Although the foregoing invention has been described in some detail by way of illustration and example for purposes of clarity of understanding, one of skill in the art will appreciate that certain changes and modifications may be practiced within the scope of the appended claims. In addition, each reference provided herein is incorporated by reference in its entirety to the same extent as if each reference was individually incorporated by reference.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/177,583 US20210181110A1 (en) | 2015-06-26 | 2021-02-17 | Method of Determining Surgical Margins Using Fluorescence Biopsy Specimen Imager |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562185407P | 2015-06-26 | 2015-06-26 | |
US201662325588P | 2016-04-21 | 2016-04-21 | |
US201662339657P | 2016-05-20 | 2016-05-20 | |
US15/192,771 US10379048B2 (en) | 2015-06-26 | 2016-06-24 | Fluorescence biopsy specimen imager and methods |
US16/529,463 US10948415B2 (en) | 2015-06-26 | 2019-08-01 | Method of determining surgical margins using fluorescence biopsy specimen imager |
US17/177,583 US20210181110A1 (en) | 2015-06-26 | 2021-02-17 | Method of Determining Surgical Margins Using Fluorescence Biopsy Specimen Imager |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/529,463 Continuation US10948415B2 (en) | 2015-06-26 | 2019-08-01 | Method of determining surgical margins using fluorescence biopsy specimen imager |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210181110A1 true US20210181110A1 (en) | 2021-06-17 |
Family
ID=57585893
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/192,771 Expired - Fee Related US10379048B2 (en) | 2015-06-26 | 2016-06-24 | Fluorescence biopsy specimen imager and methods |
US16/529,463 Active 2036-10-09 US10948415B2 (en) | 2015-06-26 | 2019-08-01 | Method of determining surgical margins using fluorescence biopsy specimen imager |
US17/177,583 Abandoned US20210181110A1 (en) | 2015-06-26 | 2021-02-17 | Method of Determining Surgical Margins Using Fluorescence Biopsy Specimen Imager |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/192,771 Expired - Fee Related US10379048B2 (en) | 2015-06-26 | 2016-06-24 | Fluorescence biopsy specimen imager and methods |
US16/529,463 Active 2036-10-09 US10948415B2 (en) | 2015-06-26 | 2019-08-01 | Method of determining surgical margins using fluorescence biopsy specimen imager |
Country Status (6)
Country | Link |
---|---|
US (3) | US10379048B2 (en) |
EP (2) | EP3314234B1 (en) |
JP (1) | JP6732807B2 (en) |
CN (1) | CN107709968A (en) |
CA (1) | CA2984438A1 (en) |
WO (1) | WO2016210340A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118212660A (en) * | 2024-05-22 | 2024-06-18 | 四川省医学科学院·四川省人民医院 | Intelligent hip bath system based on image recognition |
Families Citing this family (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10013808B2 (en) | 2015-02-03 | 2018-07-03 | Globus Medical, Inc. | Surgeon head-mounted display apparatuses |
EP3262454A4 (en) | 2015-02-23 | 2019-02-27 | Li-Cor, Inc. | Fluorescence biopsy specimen imager and methods |
CA2984438A1 (en) | 2015-06-26 | 2016-12-29 | Li-Cor, Inc. | Fluorescence biopsy specimen imager and methods |
US10489964B2 (en) | 2016-04-21 | 2019-11-26 | Li-Cor, Inc. | Multimodality multi-axis 3-D imaging with X-ray |
US10278586B2 (en) | 2016-06-23 | 2019-05-07 | Li-Cor, Inc. | Complementary color flashing for multichannel image presentation |
EP3545488A1 (en) | 2016-11-23 | 2019-10-02 | Li-Cor, Inc. | Motion-adaptive interactive imaging method |
JP6708143B2 (en) * | 2017-02-07 | 2020-06-10 | 株式会社島津製作所 | Time intensity curve measuring device |
JP6939000B2 (en) * | 2017-03-23 | 2021-09-22 | 株式会社Jvcケンウッド | Imaging device and imaging method |
EP3616158A1 (en) | 2017-04-25 | 2020-03-04 | Li-Cor, Inc. | Top-down and rotational side view biopsy specimen imager and methods |
DE102017113960A1 (en) * | 2017-06-23 | 2018-12-27 | Cruse Spezialmaschinen GmbH | Optical recording system |
US20190254753A1 (en) | 2018-02-19 | 2019-08-22 | Globus Medical, Inc. | Augmented reality navigation systems for use with robotic surgical systems and methods of their use |
EP3773153A4 (en) * | 2018-03-30 | 2021-11-17 | Perkinelmer Health Sciences Inc. | Systems and methods for 3d reconstruction of anatomical organs and inclusions using short-wave infrared (swir) projection tomography |
JP7098146B2 (en) * | 2018-07-05 | 2022-07-11 | 株式会社Iddk | Microscopic observation device, fluorescence detector and microscopic observation method |
US20220141445A1 (en) * | 2018-08-30 | 2022-05-05 | Gene Malkin | Calibration of depth-sensing computer vision systems |
EP4322176A3 (en) * | 2019-01-17 | 2024-03-20 | University Health Network | Systems, methods, and devices for three-dimensional imaging, measurement, and display of wounds and tissue specimens |
CN114126490A (en) * | 2019-07-29 | 2022-03-01 | 深圳帧观德芯科技有限公司 | System and method for three-dimensional imaging |
CN114868009A (en) * | 2019-10-14 | 2022-08-05 | 南加利福尼亚大学 | Systems, methods, and assays for Outlier Cluster Unsupervised Learning Automatic Reporting (OCULAR) |
CN110784646A (en) * | 2019-10-28 | 2020-02-11 | 中科智云科技有限公司 | System and method for sample collection |
CN110720985A (en) * | 2019-11-13 | 2020-01-24 | 安徽领航智睿科技有限公司 | Multi-mode guided surgical navigation method and system |
US11992373B2 (en) | 2019-12-10 | 2024-05-28 | Globus Medical, Inc | Augmented reality headset with varied opacity for navigated robotic surgery |
US11317884B2 (en) | 2019-12-20 | 2022-05-03 | GE Precision Healthcare LLC | Methods and systems for mammography and biopsy workflow optimization |
US11464581B2 (en) | 2020-01-28 | 2022-10-11 | Globus Medical, Inc. | Pose measurement chaining for extended reality surgical navigation in visible and near infrared spectrums |
US11382699B2 (en) | 2020-02-10 | 2022-07-12 | Globus Medical Inc. | Extended reality visualization of optical tool tracking volume for computer assisted navigation in surgery |
US11207150B2 (en) | 2020-02-19 | 2021-12-28 | Globus Medical, Inc. | Displaying a virtual model of a planned instrument attachment to ensure correct selection of physical instrument attachment |
US11607277B2 (en) | 2020-04-29 | 2023-03-21 | Globus Medical, Inc. | Registration of surgical tool with reference array tracked by cameras of an extended reality headset for assisted navigation during surgery |
US11382700B2 (en) | 2020-05-08 | 2022-07-12 | Globus Medical Inc. | Extended reality headset tool tracking and control |
US11510750B2 (en) | 2020-05-08 | 2022-11-29 | Globus Medical, Inc. | Leveraging two-dimensional digital imaging and communication in medicine imagery in three-dimensional extended reality applications |
US11153555B1 (en) | 2020-05-08 | 2021-10-19 | Globus Medical Inc. | Extended reality headset camera system for computer assisted navigation in surgery |
CN113995376A (en) * | 2020-07-14 | 2022-02-01 | 上海勤翔科学仪器有限公司 | Rotary imaging system, plant imager, animal imager and animal and plant imager |
US11737831B2 (en) | 2020-09-02 | 2023-08-29 | Globus Medical Inc. | Surgical object tracking template generation for computer assisted navigation during surgical procedure |
CN112505082A (en) * | 2020-12-18 | 2021-03-16 | 福州大学 | X-ray near-infrared two-region luminous biological imaging device |
FR3122734B1 (en) * | 2021-05-07 | 2023-08-25 | Algoscope | Pathological anatomy and cytology station for macroscopic examination |
CN117957575A (en) * | 2021-09-14 | 2024-04-30 | 深圳帧观德芯科技有限公司 | X-ray imaging in an extended microscope |
CN115202027B (en) * | 2022-08-26 | 2024-06-14 | 徕卡显微系统科技(苏州)有限公司 | Dual-camera switching device, switching method and microscope system |
US11860098B1 (en) * | 2023-01-13 | 2024-01-02 | Tencent America LLC | Method and device for three-dimensional object scanning with invisible markers |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120105600A1 (en) * | 2008-06-20 | 2012-05-03 | Meyer Michael G | Functional imaging of cells with optical projection tomography |
US20120206448A1 (en) * | 2011-02-11 | 2012-08-16 | Embrey Cattle Co. | System and method for modeling a biopsy specimen |
US20140276008A1 (en) * | 2013-03-15 | 2014-09-18 | The Regents Of The University Of California | Imaging system and method for fluorescence guided surgery |
Family Cites Families (137)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US2792502A (en) | 1953-02-06 | 1957-05-14 | Donald T O'connor | High sensitivity fluoroscope |
JPH02304366A (en) | 1989-05-18 | 1990-12-18 | Tosoh Corp | Manipulator for analyzing device |
US5103338A (en) | 1990-10-04 | 1992-04-07 | Crowley Kevin D | Apparatus for positioning objects for microscopic examination |
US5224141A (en) | 1991-02-06 | 1993-06-29 | General Electric Company | Fluoroscopic method with reduced x-ray dosage |
JP2830492B2 (en) | 1991-03-06 | 1998-12-02 | 株式会社ニコン | Projection exposure apparatus and projection exposure method |
US5408294A (en) * | 1993-05-28 | 1995-04-18 | Image Technology International, Inc. | 3D photographic printer with direct key-subject alignment |
EP0685709B1 (en) | 1994-05-31 | 2001-12-05 | Japan Em Co., Ltd. | Apparatus for measuring dimension of article and scale to be used in the same |
US5782762A (en) | 1994-10-27 | 1998-07-21 | Wake Forest University | Method and system for producing interactive, three-dimensional renderings of selected body organs having hollow lumens to enable simulated movement through the lumen |
JP3602207B2 (en) * | 1995-07-12 | 2004-12-15 | 富士写真フイルム株式会社 | Surgical fluorescent imaging device |
JPH09236755A (en) | 1996-02-29 | 1997-09-09 | Jeol Ltd | Method for correcting position of sample stage of microscope and sample stage |
DE19628765B4 (en) * | 1996-07-17 | 2007-04-05 | Braun, Paul-Wilhelm, Dipl.-Ing. | Method and device for determining the position of non-rectilinearly moving, in particular rotating machine parts |
KR19980019031A (en) | 1996-08-27 | 1998-06-05 | 고노 시게오 | A stage device (A STAGE APPARATUS) |
CN1131495C (en) | 1996-08-29 | 2003-12-17 | 三洋电机株式会社 | Texture information giving method, object extracting method, three-D model generating method and apparatus for same |
JPH10123054A (en) | 1996-10-23 | 1998-05-15 | Kagaku Gijutsu Shinko Jigyodan | Method and apparatus for observing sample |
US6165170A (en) | 1998-01-29 | 2000-12-26 | International Business Machines Corporation | Laser dermablator and dermablation |
AU5920899A (en) | 1998-09-14 | 2000-04-03 | Lucid, Inc. | Imaging of surgical biopsies |
JP2000304688A (en) * | 1999-04-16 | 2000-11-02 | Canon Inc | Substrate measuring method and device |
US6473489B2 (en) | 1999-09-30 | 2002-10-29 | Siemens Corporate Research, Inc | Apparatus for superimposition of X-ray and video images |
US6711433B1 (en) | 1999-09-30 | 2004-03-23 | Siemens Corporate Research, Inc. | Method for providing a virtual contrast agent for augmented angioscopy |
US7065242B2 (en) | 2000-03-28 | 2006-06-20 | Viewpoint Corporation | System and method of three-dimensional image capture and modeling |
US7118710B2 (en) * | 2000-10-30 | 2006-10-10 | Sru Biosystems, Inc. | Label-free high-throughput optical technique for detecting biomolecular interactions |
GB0112392D0 (en) | 2001-05-22 | 2001-07-11 | Medical Res Council | Optical imaging appartus and associated specimen support means |
US7113217B2 (en) | 2001-07-13 | 2006-09-26 | Xenogen Corporation | Multi-view imaging apparatus |
KR100411631B1 (en) * | 2001-10-18 | 2003-12-18 | 주식회사 메디미르 | Fluorescence endoscope apparatus and a method for imaging tissue within a body using the same |
JP2004163413A (en) | 2002-08-28 | 2004-06-10 | Carl-Zeiss-Stiftung Trading As Carl Zeiss | Microscope system and microscope inspection method |
US20110229023A1 (en) | 2002-11-01 | 2011-09-22 | Tenebraex Corporation | Technique for enabling color blind persons to distinguish between various colors |
US20040101088A1 (en) | 2002-11-27 | 2004-05-27 | Sabol John Michael | Methods and apparatus for discriminating multiple contrast agents |
GB0301775D0 (en) * | 2003-01-25 | 2003-02-26 | Wilson John E | Device and method for 3Dimaging |
JP4043417B2 (en) | 2003-08-28 | 2008-02-06 | シスメックス株式会社 | Particle size measuring device |
JP4634304B2 (en) * | 2003-10-10 | 2011-02-16 | 浜松ホトニクス株式会社 | Method and system for quantifying fluorescent dye concentration |
ATE383817T1 (en) | 2004-06-17 | 2008-02-15 | Cadent Ltd | METHOD FOR PROVIDING DATA RELATED TO THE ORAL CAVITY |
EP1824379B1 (en) | 2004-12-08 | 2017-04-12 | The General Hospital Corporation | System and method for normalized fluorescence or bioluminescence imaging |
US7355700B2 (en) | 2004-12-10 | 2008-04-08 | International Business Machines Corporation | Automated inspection system and method |
US7822174B2 (en) | 2005-04-20 | 2010-10-26 | The Regents Of The University Of California | Cryotomography x-ray microscopy state |
US8351026B2 (en) | 2005-04-22 | 2013-01-08 | Affymetrix, Inc. | Methods and devices for reading microarrays |
US8504140B2 (en) | 2008-04-08 | 2013-08-06 | Bruker Biospin Corporation | Apparatus and method for fluorescence imaging and tomography using spatially structured illumination |
WO2007030424A2 (en) | 2005-09-08 | 2007-03-15 | Matthew Joseph Kuhn | Digital blink comparator apparatus and software and methods for operation |
US20070106111A1 (en) | 2005-11-07 | 2007-05-10 | Eli Horn | Apparatus and method for frame acquisition rate control in an in-vivo imaging device |
US8953909B2 (en) | 2006-01-21 | 2015-02-10 | Elizabeth T. Guckenberger | System, method, and computer software code for mimic training |
WO2007091530A1 (en) * | 2006-02-07 | 2007-08-16 | The Furukawa Electric Co., Ltd. | Photodetector and measurement object reader |
US7286232B2 (en) | 2006-02-15 | 2007-10-23 | Li-Cor, Inc. | Fluorescence filtering system and method for molecular imaging |
US8241573B2 (en) * | 2006-03-31 | 2012-08-14 | Illumina, Inc. | Systems and devices for sequence by synthesis analysis |
JP4979271B2 (en) | 2006-05-29 | 2012-07-18 | オリンパス株式会社 | ENDOSCOPE SYSTEM AND ENDOSCOPE OPERATING METHOD |
US20080025475A1 (en) | 2006-07-25 | 2008-01-31 | Synarc, Inc. | Apparatus for determining the position and orientation in medical imaging |
WO2008019355A2 (en) | 2006-08-07 | 2008-02-14 | Xoran Technologies, Inc. | Ct scanner including a camera to obtain external images of a patient |
US8792968B2 (en) * | 2006-09-25 | 2014-07-29 | Song Xiao | System and method for health evaluation |
US7715523B2 (en) | 2006-09-28 | 2010-05-11 | Lafferty Peter R | System and apparatus for rapid stereotactic breast biopsy analysis |
US8503602B2 (en) | 2006-09-28 | 2013-08-06 | Peter R. Lafferty | System and apparatus for rapid stereotactic breast biopsy analysis |
CN101301192B (en) | 2007-05-10 | 2010-06-23 | 中国科学院自动化研究所 | Multimode autofluorescence tomography molecule image instrument and rebuilding method |
EP1998205B1 (en) * | 2007-05-29 | 2011-07-06 | Olympus Corporation | Observation apparatus |
JP2009008739A (en) * | 2007-06-26 | 2009-01-15 | Olympus Corp | Living body observation apparatus |
JP2008298861A (en) * | 2007-05-29 | 2008-12-11 | Olympus Corp | Observation device |
DE102007030768A1 (en) | 2007-07-02 | 2009-01-08 | Sirona Dental Systems Gmbh | Measuring device and method for 3D measurement of tooth models |
CN100593389C (en) | 2007-07-10 | 2010-03-10 | 清华大学 | Continuous dynamic gathering type beastie inducing fluorescence molecule imaging system |
US8220415B2 (en) | 2007-09-05 | 2012-07-17 | Li-Cor, Inc. | Modular animal imaging apparatus |
DE102007047461A1 (en) | 2007-09-28 | 2009-04-02 | Carl Zeiss Microimaging Gmbh | Method and optical arrangement for examining a sample |
US7929743B2 (en) | 2007-10-02 | 2011-04-19 | Hologic, Inc. | Displaying breast tomosynthesis computer-aided detection results |
US20090234225A1 (en) | 2008-01-10 | 2009-09-17 | The Ohio State University Research Foundation | Fluorescence detection system |
US9332942B2 (en) | 2008-01-28 | 2016-05-10 | The General Hospital Corporation | Systems, processes and computer-accessible medium for providing hybrid flourescence and optical coherence tomography imaging |
US8143600B2 (en) | 2008-02-18 | 2012-03-27 | Visiongate, Inc. | 3D imaging of live cells with ultraviolet radiation |
CZ301826B6 (en) | 2008-03-21 | 2010-06-30 | Biologické centrum AV CR, v.v.i., Ústav molekulární biologie rostlin | Method of investigating dynamics of volume changes of physiologically, particularly photo synthetically active samples and apparatus for making the same |
US8167793B2 (en) | 2008-04-26 | 2012-05-01 | Intuitive Surgical Operations, Inc. | Augmented stereoscopic visualization for a surgical robot using time duplexing |
ES2772073T3 (en) | 2008-07-25 | 2020-07-07 | Sloan Kettering Inst Cancer Res | Rapid confocal microscopy to support surgical procedures |
CN101401722B (en) * | 2008-11-07 | 2012-07-25 | 上海奥通激光技术有限公司 | Multi-mode co-focusing imaging method and apparatus |
WO2010102164A1 (en) | 2009-03-06 | 2010-09-10 | The Trustees Of Columbia University In The City Of New York | Systems, methods and computer-accessible media for hyperspectral excitation-resolved fluorescence tomography |
US9155471B2 (en) * | 2009-05-27 | 2015-10-13 | Lumicell, Inc'. | Methods and systems for spatially identifying abnormal cells |
US8310531B2 (en) * | 2009-08-03 | 2012-11-13 | Genetix Corporation | Methods and apparatuses for processing fluorescence images |
SG187478A1 (en) * | 2009-10-19 | 2013-02-28 | Ventana Med Syst Inc | Imaging system and techniques |
US8911360B2 (en) | 2009-11-20 | 2014-12-16 | Given Imaging Ltd. | System and method for controlling power consumption of an in vivo device |
US8235530B2 (en) | 2009-12-07 | 2012-08-07 | C-Rad Positioning Ab | Object positioning with visual feedback |
ES2364916B1 (en) | 2010-03-05 | 2012-09-04 | Consejo Superior De Investigaciones Científicas (Csic) | INSTRUMENT FOR THE IMPLEMENTATION OF WIDE FIELD IMAGES TO DIFFERENT DEPTHS OF A SPECIMEN |
US8575570B2 (en) * | 2010-08-25 | 2013-11-05 | California Institute Of Technology | Simultaneous orthogonal light sheet microscopy and computed optical tomography |
US20120065518A1 (en) | 2010-09-15 | 2012-03-15 | Schepens Eye Research Institute | Systems and methods for multilayer imaging and retinal injury analysis |
CN101984928B (en) * | 2010-09-29 | 2012-06-13 | 北京大学 | Multi-mode molecular tomography system |
US9345389B2 (en) * | 2010-11-12 | 2016-05-24 | Emory University | Additional systems and methods for providing real-time anatomical guidance in a diagnostic or therapeutic procedure |
EP2455891A1 (en) * | 2010-11-23 | 2012-05-23 | Synoptics Limited | Methods and systems for automatic capture of an image of a faint pattern of light emitted by a specimen |
WO2012071682A1 (en) | 2010-11-30 | 2012-06-07 | 中国科学院自动化研究所 | System and method for multimode three dimensional optical tomography based on specificity |
JP5677864B2 (en) | 2011-01-17 | 2015-02-25 | オリンパス株式会社 | Microscope imaging apparatus and microscope observation method |
CN102048525B (en) | 2011-01-26 | 2012-05-30 | 浙江大学 | Organism fluorescent three-dimensional imaging system and application thereof |
JP5636993B2 (en) | 2011-02-03 | 2014-12-10 | ソニー株式会社 | Image processing method, image processing apparatus, and display apparatus |
US9077910B2 (en) | 2011-04-06 | 2015-07-07 | Dolby Laboratories Licensing Corporation | Multi-field CCD capture for HDR imaging |
US9557281B2 (en) | 2011-06-09 | 2017-01-31 | The Regents Of The University Of California | Excised specimen imaging using a combined PET and micro CT scanner |
DE102011104216A1 (en) | 2011-06-15 | 2012-12-20 | Scanbull Software Gmbh | Method for three-dimensional acquisition of object to be utilized in entertainment field, involves creating and storing image data set comprising coding for images of objects, captured by optical acquisition unit at different polar angles |
WO2013016651A1 (en) | 2011-07-28 | 2013-01-31 | Massachusetts Institute Of Technology | Camera configuration for three-dimensional imaging of interior spaces |
JP5796423B2 (en) | 2011-09-06 | 2015-10-21 | リコーイメージング株式会社 | Imaging device |
WO2013082538A1 (en) | 2011-11-30 | 2013-06-06 | Waba Fun, Llc | Systems and methods for authenticating objects using ir |
CA2858166C (en) | 2011-12-05 | 2020-10-20 | Commonwealth Scientific And Industrial Research Organisation | Method and system for characterising plant phenotype |
US10006922B2 (en) * | 2011-12-22 | 2018-06-26 | Massachusetts Institute Of Technology | Raman spectroscopy for detection of glycated analytes |
WO2015023990A1 (en) * | 2013-08-15 | 2015-02-19 | The Trustees Of Dartmouth College | Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance |
US20140378843A1 (en) | 2012-01-20 | 2014-12-25 | The Trustees Of Dartmouth College | Method And Apparatus For Quantitative Hyperspectral Fluorescence And Reflectance Imaging For Surgical Guidance |
KR101320712B1 (en) | 2012-02-23 | 2013-10-21 | 인하대학교 산학협력단 | Method for aligning rotation axis of two-axis rotation stage using alignment mark and apparatus thereof |
US9715727B2 (en) | 2012-02-23 | 2017-07-25 | Smith & Nephew, Inc. | Video endoscopic system |
US9395303B2 (en) | 2012-03-12 | 2016-07-19 | Mitsubishi Rayon Co., Ltd. | Fluorescence detection device and fluorescence detection method |
WO2013166497A1 (en) | 2012-05-04 | 2013-11-07 | University Of Rochester | Method and device for preserving and imaging specimens while retaining information on the spatial orientation of specimens with respect to reference objects |
US20150105283A1 (en) | 2012-05-30 | 2015-04-16 | Clarient Diagnostics Services, Inc. | Multiplexed diagnosis method for classical hodgkin lymphoma |
US8741232B2 (en) | 2012-09-05 | 2014-06-03 | Faxitron Bioptics, Llc | Specimen imaging device and methods for use thereof |
MX2018016101A (en) | 2012-11-07 | 2022-05-19 | Modulated Imaging Inc | Efficient modulated imaging. |
US20140125790A1 (en) | 2012-11-08 | 2014-05-08 | Wisconsin Alumni Research Foundation | Device And Method For Three Dimensional Imaging Of Biological Sample |
US9824440B2 (en) | 2012-11-20 | 2017-11-21 | Vanderbilt University | Methods and systems for three-dimensional real-time intraoperative surgical margin evaluation of tumor tissues |
JP2014115151A (en) | 2012-12-07 | 2014-06-26 | Shimadzu Corp | Optical imaging device |
US8988574B2 (en) | 2012-12-27 | 2015-03-24 | Panasonic Intellectual Property Corporation Of America | Information communication method for obtaining information using bright line image |
WO2014094142A1 (en) * | 2012-12-21 | 2014-06-26 | Quantum Dental Technologies Inc. | Apparatus for in-vitro imaging and analysis of dental samples |
CN103082997B (en) | 2013-01-28 | 2015-10-28 | 中国科学院自动化研究所 | Drum-type multimodality fusion three-dimension disclocation imaging system and method |
US9936858B2 (en) | 2013-02-04 | 2018-04-10 | Orpheus Medical Ltd | Color reduction in images of an interior of a human body |
EP2765591B1 (en) | 2013-02-08 | 2016-07-13 | FEI Company | Sample preparation stage |
US9654704B2 (en) | 2013-03-15 | 2017-05-16 | Infrared Integrated Systems, Ltd. | Apparatus and method for multispectral imaging with three dimensional overlaying |
GB2514125B (en) | 2013-05-13 | 2016-10-05 | Nikon Metrology Nv | X-ray imaging system with climate control |
US9632187B2 (en) | 2013-06-12 | 2017-04-25 | The Regents Of The University Of California | Modular positron emission tomography kit |
US9651525B2 (en) | 2013-06-27 | 2017-05-16 | TecScan Systems Inc. | Method and apparatus for scanning an object |
JP5726956B2 (en) * | 2013-07-09 | 2015-06-03 | オリンパス株式会社 | Method and apparatus for analyzing faint light sample |
KR101514204B1 (en) | 2013-07-12 | 2015-04-23 | 한국전기연구원 | Apparatus and method for detecting NIR fluorescence at Sentinel Lymph Node |
US10539772B2 (en) | 2013-10-09 | 2020-01-21 | Howard Hughes Medical Institute | Multiview light-sheet microscopy |
CN203677064U (en) * | 2014-01-23 | 2014-07-02 | 清华大学 | Reflection-type fluorescence tomography system with linear optical fibers adopted |
US20150257653A1 (en) | 2014-03-14 | 2015-09-17 | Elwha Llc | Device, system, and method for determining blood pressure in a mammalian subject |
US20150373293A1 (en) | 2014-06-24 | 2015-12-24 | Sony Corporation | Video acquisition with adaptive frame rate |
WO2016014252A1 (en) | 2014-07-24 | 2016-01-28 | Apple Inc. | Invisible optical label for transmitting information between computing devices |
US10113910B2 (en) | 2014-08-26 | 2018-10-30 | Digimarc Corporation | Sensor-synchronized spectrally-structured-light imaging |
CN104299599B (en) | 2014-11-04 | 2017-05-24 | 深圳市华星光电技术有限公司 | Conversion system and conversion method from RGB data to WRGB data |
WO2016073569A2 (en) | 2014-11-05 | 2016-05-12 | Carestream Health, Inc. | Video detection of tooth condition using green and red fluorescence |
US9570106B2 (en) | 2014-12-02 | 2017-02-14 | Sony Corporation | Sensor configuration switching for adaptation of video capturing frame rate |
EP4226887A1 (en) | 2014-12-16 | 2023-08-16 | Intuitive Surgical Operations, Inc. | Ureter detection using waveband-selective imaging |
US10013796B2 (en) | 2015-01-22 | 2018-07-03 | Ditto Technologies, Inc. | Rendering glasses shadows |
EP3262454A4 (en) | 2015-02-23 | 2019-02-27 | Li-Cor, Inc. | Fluorescence biopsy specimen imager and methods |
WO2016154589A1 (en) | 2015-03-25 | 2016-09-29 | Camplex, Inc. | Surgical visualization systems and displays |
CA2984438A1 (en) | 2015-06-26 | 2016-12-29 | Li-Cor, Inc. | Fluorescence biopsy specimen imager and methods |
US20180209924A1 (en) | 2015-07-27 | 2018-07-26 | Hitachi High-Technologies Corporation | Defect Determining Method and X-Ray Inspection Device |
KR101849705B1 (en) | 2015-11-13 | 2018-05-30 | 한국전기연구원 | Method and system for generating 3D image using spectral x-ray and optical image |
US11147457B2 (en) | 2015-11-18 | 2021-10-19 | The Board Of Trustees Of The Leland Stanford Junior University | Method and systems for measuring neural activity |
US10386300B2 (en) | 2015-12-21 | 2019-08-20 | Verily Life Sciences Llc | Spectrally and spatially multiplexed fluorescent probes for in situ cell labeling |
US11579088B2 (en) | 2016-03-14 | 2023-02-14 | Massachusetts Institute Of Technology | Device and method for imaging shortwave infrared fluorescence |
US10489964B2 (en) | 2016-04-21 | 2019-11-26 | Li-Cor, Inc. | Multimodality multi-axis 3-D imaging with X-ray |
US10768114B2 (en) | 2016-04-29 | 2020-09-08 | Synaptive Medical (Barbados) Inc. | Multi-modal optical imaging system for tissue analysis |
WO2017200801A1 (en) | 2016-05-20 | 2017-11-23 | Li-Cor, Inc. | X-ray biopsy specimen imager and methods |
JP6898946B2 (en) | 2016-06-17 | 2021-07-07 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | Systems and methods for determining patient hemodynamic parameters |
US10278586B2 (en) | 2016-06-23 | 2019-05-07 | Li-Cor, Inc. | Complementary color flashing for multichannel image presentation |
US10709333B2 (en) | 2016-07-25 | 2020-07-14 | PhotoSound Technologies, Inc. | Instrument for acquiring co-registered orthogonal fluorescence and photoacoustic volumetric projections of tissue and methods of its use |
EP3545488A1 (en) | 2016-11-23 | 2019-10-02 | Li-Cor, Inc. | Motion-adaptive interactive imaging method |
-
2016
- 2016-06-24 CA CA2984438A patent/CA2984438A1/en not_active Abandoned
- 2016-06-24 US US15/192,771 patent/US10379048B2/en not_active Expired - Fee Related
- 2016-06-24 EP EP16815434.2A patent/EP3314234B1/en active Active
- 2016-06-24 JP JP2017562263A patent/JP6732807B2/en not_active Expired - Fee Related
- 2016-06-24 WO PCT/US2016/039382 patent/WO2016210340A1/en active Application Filing
- 2016-06-24 EP EP21165337.3A patent/EP3869184A1/en not_active Withdrawn
- 2016-06-24 CN CN201680037184.6A patent/CN107709968A/en active Pending
-
2019
- 2019-08-01 US US16/529,463 patent/US10948415B2/en active Active
-
2021
- 2021-02-17 US US17/177,583 patent/US20210181110A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120105600A1 (en) * | 2008-06-20 | 2012-05-03 | Meyer Michael G | Functional imaging of cells with optical projection tomography |
US20120206448A1 (en) * | 2011-02-11 | 2012-08-16 | Embrey Cattle Co. | System and method for modeling a biopsy specimen |
US20140276008A1 (en) * | 2013-03-15 | 2014-09-18 | The Regents Of The University Of California | Imaging system and method for fluorescence guided surgery |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118212660A (en) * | 2024-05-22 | 2024-06-18 | 四川省医学科学院·四川省人民医院 | Intelligent hip bath system based on image recognition |
Also Published As
Publication number | Publication date |
---|---|
JP6732807B2 (en) | 2020-07-29 |
JP2018529929A (en) | 2018-10-11 |
EP3314234B1 (en) | 2021-05-19 |
US20160377545A1 (en) | 2016-12-29 |
CA2984438A1 (en) | 2016-12-29 |
CN107709968A (en) | 2018-02-16 |
US10379048B2 (en) | 2019-08-13 |
EP3314234A1 (en) | 2018-05-02 |
EP3869184A1 (en) | 2021-08-25 |
EP3314234A4 (en) | 2019-06-19 |
WO2016210340A1 (en) | 2016-12-29 |
US20190353595A1 (en) | 2019-11-21 |
US10948415B2 (en) | 2021-03-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10948415B2 (en) | Method of determining surgical margins using fluorescence biopsy specimen imager | |
US10489964B2 (en) | Multimodality multi-axis 3-D imaging with X-ray | |
US10816788B2 (en) | Imaging systems with micro optical element arrays and methods of specimen imaging | |
EP1968431B1 (en) | Combined x-ray and optical tomographic imaging system | |
CN104470458B (en) | For the augmented reality imaging system of operation instrument guiding | |
US8041409B2 (en) | Method and apparatus for multi-modal imaging | |
US20130345509A1 (en) | System and method for endoscopic measurement and mapping of internal organs, tumors and other objects | |
US20170336706A1 (en) | X-ray biopsy specimen imager and methods | |
JP2005169116A (en) | Fused image displaying method | |
US20150071513A1 (en) | Method and apparatus for analyzing images | |
US20200237229A1 (en) | Apparatus and method for image-guided interventions with hyperspectral imaging | |
US9361726B2 (en) | Medical image diagnostic apparatus, medical image processing apparatus, and methods therefor | |
FR2908628A1 (en) | METHOD AND SYSTEM FOR CONTROLLING A MEDICAL INSTRUMENT | |
US10342633B2 (en) | Medical image data processing system and method | |
CN111970960A (en) | System and method for 3D reconstruction of anatomical organs and contents using Short Wave Infrared (SWIR) projection tomography | |
US20200257098A1 (en) | Imaging systems with micro optical element arrays and methods of specimen imaging | |
Liu et al. | A co-axial projection surgical navigation system for breast cancer sentinel lymph node mapping: system design and clinical trial | |
US11039803B1 (en) | System and method for cabinet radiography incorporating a gamma or other probe | |
Daly | An Image-Guided Computational Framework for Intraoperative Fluorescence Quantification | |
Baghdadchi | Spectral and Spatial Light Field Encoding for Tissue Imaging, Drug Delivery and Focused Field Shaping |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: LI-COR, INC., NEBRASKA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WANG, HAN-WEI;REEL/FRAME:055295/0766 Effective date: 20160624 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: MIDCAP FINANCIAL TRUST, MARYLAND Free format text: SECURITY INTEREST;ASSIGNOR:LI-COR, INC.;REEL/FRAME:058293/0889 Effective date: 20211201 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |