WO2024099594A1

WO2024099594A1 - Imaging device and method

Info

Publication number: WO2024099594A1
Application number: PCT/EP2023/063671
Authority: WO
Inventors: Gareth GALLAGHER; Ra'ed MALALLAH; Ronan CAHILL
Original assignee: University College Dublin
Priority date: 2022-11-09
Filing date: 2023-05-22
Publication date: 2024-05-16
Also published as: GB202216723D0

Abstract

An imaging device comprising: an electromagnetic radiation, EMR, source, configured to provide first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 μm to 3 μm, preferably in a range from 0.4 μm to 1.4 μm, more preferably in range from 0.75 μm to 0.85 μm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; a set of flexible fibre-optic bundles, including a first fibre-optic bundle, having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end of the first fibre-optic bundle is flexibly positionable and/or orientable to illuminate a target with the first EMR, incident thereupon, provided by the EMR source; and an EMR detector configured to detect second EMR, having a second wavelength in a range from 0.1 μm to 3 μm, preferably in a range from 0.4 μm to 1.4 μm, more preferably in range from 0.80 μm to 0.85 μm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and to output signals corresponding to the detected second EMR.

Description

IMAGING DEVICE AND METHOD

Field

The present invention relates to an imaging device, for example a fluorescence imaging device, and a method of imaging, for example fluorescence imaging.

Background to the invention

Current near infrared (NIR) fluorescence imaging devices are restricted to open and/or laparoscopic modalities, thereby limiting their anatomical reach.

Hence, there is a need to improve the anatomical reach of NIR fluorescence imaging, more generally fluorescence imaging.

Summary of the Invention

It is one aim of the present invention, amongst others, to provide an imaging device which at least partially obviates or mitigates at least some of the disadvantages of the prior art, whether identified herein or elsewhere. For instance, it is an aim of embodiments of the invention to provide an imaging device, for example a fluorescence imaging device, and a method of imaging, for example fluorescence imaging, having an improved anatomical reach compared with conventional imaging devices, for example to extend fluorescence imaging to relatively more tortuous regions of the gastrointestinal (Gl) tract.

A first aspect provides an imaging device comprising: an electromagnetic radiation, EMR, source, configured to provide first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; a set of flexible fibre-optic bundles, including a first fibre-optic bundle, having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end of the first fibre-optic bundle is flexibly positionable and/or orientable to illuminate a target with the first EMR, incident thereupon, provided by the EMR source; and an EMR detector configured to detect second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1.4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and to output signals corresponding to the detected second EMR.

A second aspect provides an endoscope comprising the imaging device according to the first aspect.

A third aspect provides a method of fluorescence imaging a target, for example an organ of a patient, comprising: flexibly positioning and/or orienting a distal end of a first fibre-optic bundle; illuminating, via the distal end of the first fibre-optic bundle, the target with first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; and detecting second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and outputting signals corresponding to the detected second EMR.

Detailed Description of the Invention

According to the present invention there is provided an imaging device, as set forth in the appended claims. Also provided is a method of imaging. Other features of the invention will be apparent from the dependent claims, and the description that follows.

Imaging device

The first aspect provides an imaging device comprising: an electromagnetic radiation, EMR, source, configured to provide first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; a set of flexible fibre-optic bundles, including a first fibre-optic bundle, having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end of the first fibre-optic bundle is flexibly positionable and/or orientable to illuminate a target with the first EMR, incident thereupon, provided by the EMR source; and an EMR detector configured to detect second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1.4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and to output signals corresponding to the detected second EMR.

In this way, in vivo fluorescence imaging of patients’ organs may be performed by minimally invasive procedures, for example minimally invasive gastrointestinal procedures, since the set of fibre-optic bundles is flexible, allowing the set of fibre-optic bundles to be curved and/or bent around and/or within the patients’ bodies i.e. anatomically conformable. Particularly, the distal end (i.e. the illumination and collection end) of the set of flexible fibre-optic bundles, for example of the first fibre-optic bundle, is flexibly positionable and/or orientable, for example relative to the proximal end thereof, enabling the distal end to be guided though a patient’s body towards a target and positioned and/or oriented for imaging thereof. In this way, the anatomical reach of the fluorescence imaging is improved. For example, fluorescence imaging of the whole of the colorectal and internal gastrointestinal (Gl) tract may be thus performed. For example, fluorescence imaging of the oesophagus, stomach, bile duct and/or pancreas may be thus performed.

An aim of the imaging device is to facilitate endoscopic near-infrared imaging of the perfusion of Indocyanine Green (ICG) in tissue, particularly, but not restricted to, the colon. ICG is a fluorescent dye, which emits fluorescence on excitation by a NIR light source at a wavelength of approximately 785 nm. The emitted fluorescence (approximate wavelength band of 800-850 nm) can be captured (imaged) and processed. This emission intensity signal can then be used to accurately classify cancerous tissue, through the use of biophysical modelling and image analysis techniques (Artificial intelligence indocyanine green (ICG) perfusion for colorectal cancer intra-operative tissue classification; R. A. Cahill, D. F. O’Shea, M. F. Khan, H. A. Khokhar, J. P. Epperlein, P. G. Mac Aonghusa, R. Nair and S. M. Zhuk; BJS, 2020, 00, 1-5; DOI: 10.1093/bjs/znaa004).

Other fluorescent dyes are known. Fluorophore molecules may be either utilized alone, or serve as a fluorescent motif of a functional system. Based on molecular complexity and synthetic methods, fluorophore molecules may be generally classified into four categories: proteins and peptides, small organic compounds, synthetic oligomers and polymers, and multi-component systems. See, for example, https://en.wikipedia.org/wiki/Fluorophore.

The three fundamental elements of operation of the imaging device are:

1 . Energy (NIR) delivery to site (dye excitation);

2. Image (fluorescence emission) capture and delivery to detector; and

3. Image detection and optionally, signal interpretation.

To achieve these operations four system (device) components are required: 1 . An energy (for example laser) source (also known as an excitation source) for the delivery of wavelength specific NIR light;

2. An endoscope compatible, fibre optic probe for delivery of the NIR (also known as light display optics) and transfer of fluorescence emitted data (image) (also known as light assortment optics or collector), which may incorporate one or more lenses at the tip and/or detector end to enhance and focus the image;

3. An image detector, such as a photomultiplier (PMT), a charge-couple device (CCD) or a complementary metal-oxide-semiconductor (CMOS) detector, for interpretation and conversion of the analogue image to a digital signal, positioned at the distal tip (source) or externally whereby the image fibre directs from the source to the sensor (detector);

4. Optionally, one or more optical wavelength filters (NIR) to remove undesired noise from the detected light signal; and

5. Optionally, a processor (computer) to read the digital signal and process the information such that it can be interpreted by the user/AI algorithm.

EMR source

The imaging device comprises the EMR source (also known as an excitation source), configured to provide the first EMR. In one example, the EMR source comprises and/or is a broadband (also known as broad) wavelength source, for example an IR lamp. In one example, the EMR source comprises and/or is a narrowband (also known as narrow) wavelength source, for example a laser. Suitable EMR sources are known. In one example, the first EMR comprises and/or is monochromatic EMR, consisting of a single wavelength, a discrete number of wavelengths or a narrow range of wavelengths, for example having a wavelength range of at most 10 nm, preferably at most 5 nm, more preferably at most 2 nm, most preferably at most 1 nm. It should be understood that the first wavelength is selected to match absorption by fluorescent molecules, such as endogenous proteins or exogenous dyes, on and/or in the target and the preferred ranges may be shifted accordingly. For example, the US Food and Drug Administration (FDA) has approved two NIR-I dyes for clinical use, namely, indocyanine green (ICG, emission at -800 nm) and methylene blue (MB, emission at -700 nm). These two dyes are primarily used for NIR fluorescence-based intraoperative imaging for structural visualization of anatomical features, such as tract blood and lymphatic vessels, the gastrointestinal (Gl) tract, bile duct, and ureters. Other dyes are known, as shown in Table 1A. The first wavelength is in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1.4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm, for ICG. More preferable and most preferable ranges for suitable dyes are shown in Table 1A. That is, the first wavelength (i.e. the excitation wavelength) is in the visible, nearinfrared (NIR) or in the short-wavelength infrared (SWIR), preferably in the near-infrared (NIR), for example for ICG. In one example, the EMR source comprises and/or is a multispectral source, configured to provide the first EMR and visible light (i.e. 380 to 750 nm), for example white light. In this way, fluorescence imaging and visible light imaging may be provided, for example simultaneously or successively (i.e. alternating).

Table 1A: Suitable dyes and respective excitation (i.e. first) and emission (i.e. second) wavelengths. Generally, more preferable and most preferable wavelength ranges are 100 nm and 30 nm about the respective excitation (i.e. first) and emission (i.e. second) wavelengths and may be included additionally and/or alternatively in the subject matter of the first aspect and/or the third aspect for the specific dyes.

Table 1 B: References for suitable dyes of Table 1A.

In one example, the EMR source is configured to provide the first EMR, having the first wavelength, at power in a range from 100 mW to 5000 mW, preferably in a range from 200 mW to 1000 mW, for example about 300 mW or about 400 mW. In this way, sufficient power may be transmitted via the set of flexible fibre-optic bundles, including the first fibre-optic bundle, accounting for transmission losses such that the illumination power of the first EMR, having the first wavelength, at the distal end is in a range from 10 mW to 500 mW, preferably in a range from 20 mW to 100 mW, for example about 30 mW or about 40 mW (i.e. about 10% of the EMR source power).

In one example, the EMR source is configured to provide the first EMR, having the first wavelength, as pulsed EMR or as continuous EMR. The peak power of the pulsed EMR may be relatively greater than the constant power of the continuous EMR, the latter attenuated to avoid tissue damage. Pulsed EMR may be provided simultaneously or successively (i.e. alternately) with white light for visible imaging, for example for simultaneous fluorescence and visible imaging.

Flexible fibre-optic bundles

The imaging device comprises the set of flexible fibre-optic bundles, including the first fibre-optic bundle (also known as light display optics), having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end (i.e. the illumination and collection end) of the first fibre-optic bundle is flexibly (c.f. rigidly) positionable and/or orientable, for example with respect to the proximal end thereof, to illuminate a target with the first EMR, incident thereupon, provided by the EMR source. In this way, the target is illuminated with the first EMR that is transmitted along the set of flexible fibre-optic bundles, for example the first fibre-optic bundle. Particularly, the distal end (i.e. the illumination and collection end) of the set of flexible fibre-optic bundles, for example of the first fibre-optic bundle, is flexibly positionable and/or orientable, for example relative to the proximal end thereof, enabling the distal end to be guided though a patient’s body towards a target and positioned and/or oriented for imaging thereof. In this way, the anatomical reach of the fluorescence imaging is improved. In contrast, conventional fluorescence imaging devices are rigid (i.e. not flexible) and are not flexibly positionable and/or orientable. In one example, the set of flexible fibre-optic bundles, for example the first fibre-optic bundle, has a minimum bend radius in a range from 5 mm to 500 mm, preferably in a range from 10 mm to 250 mm, more preferably in a range from 20 mm to 100 mm, most preferably in a range from 25 mm to 50 mm, optionally along a section of the length thereof, between the proximal end and the distal end, in a range from 50% to 100%, preferably in a range from 75% to 99.9%, more preferably in a range from 90% to 99% of the length thereof, for example relatively more proximate and/or measured from the distal end. In this way, the distal end of the the set of flexible fibre-optic bundles, for example of the first fibre-optic bundle, may be guided though a patient’s body, for example through the relatively tortuous Gl tract, towards a target and positioned and/or oriented for imaging thereof. In this way, the anatomical reach of the fluorescence imaging is further improved. In one example, the set of flexible fibre-optic bundles, for example the first fibre-optic bundle, has a length, between the proximal end and the distal end thereof, in a range from 0.5 m to 10 m, preferably in a range from 1 m to 5 m, more preferably in a range from 1 .5 m to 3 m, for example about 2 m or 2.5 m. In this way, the anatomical reach of the fluorescence imaging is further improved. In one example, the first fibre-optic bundle comprises single-mode fibres. In one example, the first fibre-optic bundle comprises multi-mode fibres. Multi-mode fibres are preferred. In one example, the first fibre-optic bundle consists of an individual (i.e. a single, only one) optic fibre, for example an individual single-mode fibre or an individual multi-mode fibre, preferably an individual multi-mode fibre. In one example, fibres included in the first fibre-optic bundle have a diameter in a range from 10 pm to 500 pm, preferably in a range from 50 pm to 300 pm, more preferably in a range from 100 pm to 250 pm.

In one example, the set of flexible fibre-optic bundles includes a second fibre-optic bundle; and wherein the proximal end of the second fibre-optic bundle is optically coupled to the EMR detector. In this way, the second EMR emitted by the target is transmitted from the distal end of the second fibre-optic bundle to the proximal end thereof, for detection by the EMR detector optically coupled thereto. That is, the EMR detector is external relative to the target (c.f. at the distal end, as described below).

In one example, the second fibre-optic bundle comprises N fibre-optics, where N is a number in a range from 10² to 10⁸, preferable in a range from 10³ to 10⁶, more preferably in a range from 10⁴ to 10⁵. By increasing the number N of fibre-optics of the second fibre-optic bundle, a resolution of the image generated from the output signals may be increased.

In one example, the first fibre-optic bundle and the second fibre-optic bundle are mutually coaxial, for example wherein the second fibre-optic bundle surrounds the first fibre-optic bundle. In one example, the set of flexible fibre-optic bundles has an outside diameter in a range from 0.5 mm to 10 mm, preferably in a range from 1 mm to 7 mm for example 6.4 mm, more preferably in a range from 2 mm to 4 mm, for example 2.8 mm or 3.2 mm. In this way, the set of flexible fibre-optic bundles may be inserted into the working channel (also known as the instrument channel) of a conventional endoscope.

EMR detector

The imaging device comprises the EMR detector configured to detect the second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 820 nm or about 825 nm for ICG, emitted by the illuminated target and to output signals corresponding to the detected second EMR. In this way, the second EMR emitted by the illuminated target (for example, by fluorescence) is detected by the EMR detector and the output signals may be used for image generation. It should be understood that the second wavelength is selected to match emission by fluorescent molecules, such as endogenous proteins or exogenous dyes, on and/or in the target and the preferred ranges may be shifted accordingly. For example, the US Food and Drug Administration (FDA) has approved two NIR-I dyes for clinical use, namely, indocyanine green (ICG, emission at -800 nm) and methylene blue (MB, emission at -700 nm). These two dyes are primarily used for NIR fluorescence-based intraoperative imaging for structural visualization of anatomical features, such as tract blood and lymphatic vessels, the gastrointestinal (Gl) tract, bile duct, and ureters. Other dyes are known, as shown in Table 1A. It should be understood that the second wavelength is greater than the first wavelength. Suitable EMR detectors, for example photomultipliers (PMTs), charge-couple devices (CCDs) and complementary metal-oxide- semiconductor (CMOS) detectors, are known.

In one example, the EMR detector is disposed proximate or at the proximal end of the first fibreoptic bundle, wherein the set of flexible fibre-optic bundles includes a second fibre-optic bundle; and wherein the proximal end of the second fibre-optic bundle is optically coupled to the EMR detector. That is, the EMR detector is external relative to the target (c.f. at the distal end, as described below).

In one example, the EMR detector comprises and/or is a monochromatic EMR detector, configured to detect only the second EMR and/or a relatively narrow band around the second EMR. Monochromatic EMR detectors may more sensitively detect infrared light (i.e. fluorescence emission) and thus, can more effectively delineate fluorescence. In one example, the EMR detector is disposed proximate or at the distal end of the first fibreoptic bundle; and wherein the imaging device comprises an electrical cable electrically coupled, for example at a distal end thereof, to the EMR detector. That is, the EMR detector is internal relative to the target (c.f. at the proximal end, as described above) and the output signals are electrically communicated therefrom to the proximal end. In one example, the proximal end of the electrical cable is electrically coupled to an amplifier, a circuit or a computer.

In one example, the EMR detector is configured to detect the second EMR and to detect visible light (i.e. 380 to 750 nm), for example white light. In this way, fluorescence imaging and visible light imaging may be provided, for example simultaneously or successively (i.e. alternating).

Flexible conduit

In one example, the imaging device comprises a flexible conduit containing the set of flexible fibre-optic bundles.

In one example, the flexible conduit has an outside diameter in a range from 0.5 mm to 10 mm, preferably in a range from 1 mm to 7 mm for example 6.4 mm, more preferably in a range from 2 mm to 4 mm, for example 2.8 mm or 3.2 mm. In this way, flexible conduit containing the set of flexible fibre-optic bundles may be inserted into the working channel (also known as the instrument channel) of a conventional endoscope.

Lenses

In one example, the imaging device comprises a first optical lens (or a first optical lens assembly) optically coupled to the distal end of the first fibre-optic bundle. In this way, transmission of the first EMR along the first fibre-optic bundle is improved.

In one example, the imaging device comprises a second optical lens (or a second optical lens assembly) optically coupled to the EMR detector. In this way, focussing of the second EMR into the EMR detector is improved, thereby increasing a sensitivity (signal to noise ratio) of the EMR detector with respect to the second EMR.

Filters

In one example, the imaging device comprises an optical filter, optically coupled to the EMR detector, configured to attenuate the first EMR having the first wavelength. In this way, an intensity of the first EMR detected by the EMR detector is reduced, thereby increasing a sensitivity (signal to noise ratio) of the EMR detector with respect to the second EMR. Computer

In one example, the imaging device comprises a computer having a processor and a memory, wherein the computer is configured to generate a first image, for example a fluorescence image, of the illuminated target using the signals output from the EMR detector. Generation of images of illuminated targets using the signals output from the EMR detector is known. In one example, the computer is configured to translate an intensity of the output signals and hence of the second EMR into a grey scale, for example 0-255 grayscale intensity, and to generate the first image using the grey-scale intensities.

In one example, the computer is configured to generate a second image, for example a visible image, of the illuminated target using signals output from the EMR detector. In one example, the computer is configured to display the first image and optionally the second image. In one example, the computer is configured to generate the first image in correspondence with the second image, for example having the same or similar field of view and/or to track matching regions of interest therein.

In one example, the computer is configured to identify a feature of the generated image and optionally, to classify the identified feature. Artificial intelligence, machine learning

Endoscope

The second aspect provides an endoscope comprising the imaging device according to the first aspect.

Method

The third aspect provides a method of fluorescence imaging a target, for example an organ of a patient, comprising: flexibly positioning and/or orienting a distal end of a first fibre-optic bundle; illuminating, via the distal end of the first fibre-optic bundle, the target with first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; and detecting second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and outputting signals corresponding to the detected second EMR. Definitions

Throughout this specification, the term “comprising” or “comprises” means including the component(s) specified but not to the exclusion of the presence of other components. The term “consisting essentially of’ or “consists essentially of’ means including the components specified but excluding other components except for materials present as impurities, unavoidable materials present as a result of processes used to provide the components, and components added for a purpose other than achieving the technical effect of the invention, such as colourants, and the like.

The term “consisting of’ or “consists of’ means including the components specified but excluding other components.

Whenever appropriate, depending upon the context, the use of the term “comprises” or “comprising” may also be taken to include the meaning “consists essentially of’ or “consisting essentially of’, and also may also be taken to include the meaning “consists of’ or “consisting of’.

The optional features set out herein may be used either individually or in combination with each other where appropriate and particularly in the combinations as set out in the accompanying claims. The optional features for each aspect or exemplary embodiment of the invention, as set out herein are also applicable to all other aspects or exemplary embodiments of the invention, where appropriate. In other words, the skilled person reading this specification should consider the optional features for each aspect or exemplary embodiment of the invention as interchangeable and combinable between different aspects and exemplary embodiments.

Brief description of the drawings

For a better understanding of the invention, and to show how exemplary embodiments of the same may be brought into effect, reference will be made, by way of example only, to the accompanying diagrammatic Figures, in which:

Figure 1 schematically depicts an imaging device according to an exemplary embodiment, for ICG/bovine colon-tissue tests;

Figure 2 shows (a) 3D printed designed tissue test fixture, (b) image of providing a polyp-tissue with injected by ICG; Figure 3 shows graphs of fluorescent intensity results for ICG /DMSO solution as function of (a) concentrations (b) light wavelengths;

Figure 4 shows graphs of UV spectral intensity results of (a) Absorption (b) Fluorescence for 7.5pM of ICG /DMSO solution;

Figure 5 shows graphs of normalized fluorescence intensity as function of wavelengths for different concentrations using (a) Band-Pass filter (b) Long-Pass filter;

Figure 6 shows graphs of normalized fluorescence intensity as function of (a) wavelengths (b) different concentrations using double filters;

Figure 7 shows a graph of the Pre and Post-Photobleaching test for 30 pM concentration;

Figure 8 shows graphs for the distance test of 30 pM concentration;

Figure 9 shows graphs for the power intensity test of 30 pM concentration;

Figure 11 shows IBM Fluorescence Tracker;

Figure 12 shows fluorescence video capture and gray scaling;

Figure 13 shows RGB breakdown and fluorescence contrasting;

Figure 14 shows NIR Imaging Scope Compatibility with IBM Tracker; and

Figure 15 schematically depicts a method according to an exemplary embodiment.

Detailed Description of the Drawings

Introduction

Multispectral imaging has been widely adopted for medical sensing with both diagnostic and therapeutic applications. Near-Infrared (NIR) imaging, using fluorescent contrast agents intravenously injected, is one such example that has been widely accepted. The US Food and Drug Administration (FDA) approved fluorophore, Indocyanine Green (ICG), is a water soluble tricarbocyanine molecule of molecular mass 775 g/mol. It readily binds to blood plasma proteins, in particular the beta-lipoprotein, albumin, without altering its molecular structure and it is excreted unchanged via the liver making it highly safe to use. Its optical absorption (780-805nm) and peak fluorescence (~820-830nm) provides specific enhancement of optical contrast compared to the relatively transparent blood and tissue within the NIR range. These aspects make ICG a suitable agent for NIR imaging although it has limitations such as low quantum yield.

Validated applications of NIR imaging using ICG, such as perfusion quantification in colorectal anastomosis and axillary staging in breast cancer among others have been well documented, while new exciting uses, which include cancerous lesion identification, are currently in development. Modern machine learning technologies have assisted in quantifying perfusion for subtle differences in dynamic perfusion patterns for these purposes with high accuracy in research studies enabling distinction. However, the internal anatomical reach of the current NIR imaging modalities and subsequent application of such tissue classification approaches, are limited by the rigidity and size of the physical devices which have been developed for open and laparoscopic rather than intraluminal interventions. Thus, natural orifice and transluminal NIR imaging is impeded by the current commercial technologies.

Endoscopic screening has become the standard of care for gastrointestinal cancer diagnosis. Visualization of the colon is typically achieved using micro-camera sensors located at the tip of a flexible endoscope, translating a digital white/visible light signal to the user. For more miniature luminal imaging applications, in regions like the bile duct and pancreas, fibre optic bundles are used to collect and transmit the visible light energy to a proximal, extracorporeal camera, similar to a traditional rigid laparoscope configuration. Although examples of multispectral compatible endoscopic devices can be found in the literature and efforts have been made in the characterization of colorectal cancer lesions via adaptation of clinically available gastrointestinal fibrescopes for NIR fluorescence imaging, no NIR capable flexible endoscopes are available commercially, thus, presenting the need for a device to enhance the reach and extend NIR visualization to more tortuous regions of the gastrointestinal (Gl) tract. The narrower calibre needed for such instruments also means the necessary illumination and sensing capability is more challenging then with existing larger calibre devices instruments.

Fluorescent dye imaging typically necessitates three fundamental elements for performance: excitation light energy transmittance to target site; fluorescence emission collection and transfer to sensor/detector; signal capture/sensing and image interpretation. To achieve the aforementioned performance of each element, dye specificity may dictate system/equipment specifications. The contents of this disclosure therefore describe the characterization of ICG fluorescence detection from both ICG solutions and bovine colon tissue (to replicate the known spectral shifts that occur with ICG in biological tissue) using flexible bundle-fibre optics, comprising the initial step in the development of a clinically appropriate flexible NIR endoscopic imaging probe capable of use in conjunction with conventional colonoscopy. In this endeavour, an experimental setup was developed and validated using ICG with a Dimethyl Sulfoxide (DMSO) binding agent. Subsequently, this system was used to investigate the fluorescence emission of ICG dye (in bovine albumin solutions) concentrations along with the emission performance relationship to physical parameters in the aim of optimizing probe design in a clinically relevant colonic polyp model.

Methodology

ICG/DMSO Fluorescence Investigation

To define these specifications needed to inform flexible bundle-fibre optics, the inventors first investigated the optical fluorescent properties of ICG of varied concentrations in a Dimethyl sulfoxide (DMSO) solution using UV-spectroscopy (Cary 60 UV-VIS Spectrophotmeter, Agilent Technologies, US) since it is known that Indocyanine Green (ICG) is ineffective without the presence of an excitatory agent. Through previous chemical experiments, dimethyl sulfoxide (DMSO) was identified as a highly stable and superior solvent for ICG. For the spectroscopy investigation of homogenous ICG/DMSO solution, a stock solution was prepared of 100pM by dissolving 1 mg ICG (Verdye, Diagnostic Green, Germany) into 12.9mL DMSO (D8418-50ML, Sigma Aldrich, US), and further diluting to formulate the desired test concentrations. Molar concentrations were chosen by taking the manufacturer recommended ICG dosage (0.1- 0.3mg/Kg) and incrementing two fold, five times either side, resulting in 11 ICG/DMSO solution concentrations. Spectral fluorescence intensities were compared and the optimal concentration identified. For the optimal solution, peak absorption and fluorescence wavelengths were empirically derived, allowing for the excitation source, fibre bundle and light filters to be specified for the experimental setup.

ICG/BSA Fluorescence Experimental Setup

A flexible bundle fibre optic system was set-up as shown in Figure 1 , using commercial sourced components comprising a multi-arm fibre optic bundle, illumination source, wavelength filters and photon detectors. The Y-bundle fibre optic was 2 meters long to replicate that of the clinically used endoscopic systems and capable of transmitting light in the visible and NIR spectral bands. The illumination source providing 300mW max power similar to that used clinically as well as in previous NIR fluorescence colonoscope development. This experimental set-up, although not yet fully clinically compatible (6.4mm outer diameter), is reflective of the intended use case being similar in concept to a probe that could be placed down the working channel of a endoscope (mother-daughter technique) (c. 3.2 mm channel). The experimental setup was validated and optimized using the ICG/DMSO concentration with the highest fluorescence intensity performance as described above. A 785nm laser source (Roithner Lasertechnik, RLTMDL-785- 2.5W-3) was selected as the excitation light, given its proximity to the peak absorption of the optimal ICG/DMSO concentration. The bundle fibre optic (Thorlabs, RP22) acts to both deliver illumination, collect and transfer fluorescent emission, via its three arm configuration: single mode optical fibre transmits the adjusted (macro objective lens, Olympus, MPLFLN10X, NA 0.3, and 3D adjusted stage, OP Mount, SF-1T) 785nm excitation wavelength to the target (tissue); multi-mode bundle fibre optics transmit collected fluorescence to a photon detector; combined single mode and multi-mode bundle arm that targets the site, both delivering and collecting light/emission energy. The interchangeable photon detectors consisted of a power meter (Thorlabs, PM100D/S121 C) and a spectrometer (Thorlabs, CCS200/M). Before the collected light reaches the detectors, the emission fluorescence is isolated using wavelength filters (Thorlabs, Long-pass, FEL0800 and Band-pass filters, FB840-10). The power meter was intermittently placed at the fibre illumination tip for source intensity measurements at the target site and also at the collection tip to determine power attenuation and fluorescence quantum yield.

ICG/BSA 4% Fluorescence Investigation and Device Optical Specification Optimization

Albumin is the constituent binding protein for ICG in blood plasma, typically making up ~4% weight/volume (3.5-5g/dL) in healthy blood. For this investigation, bovine serum albumin (BSA, A3059, Sigma Aldrich, US) was utilized as a substitute for the human serum albumin (HAS), due to HAS’s human product status and restricted commercial sale. BSA has shown to exhibit similar size and binding structures when compared to its human counterpart and thus, was deemed acceptable for use. To mimic the 4% weight/volume ratio of albumin in blood, BSA was dissolved in a phosphate buffered saline (BR0014G, Fisher Scientific, US) solution. ICG/BSA solutions of concentrations mirroring those used in the DMSO investigation, including an additional typically used clinical (assessment of liver function) concentration (0.25mg/kg), were formulated.

Table 2: Concentration values of ICG dissolved in 4% Bovine Serum Albumin (BSA) buffer. or a 70kg patient with a conventional blood volume estimate of 70mL/kg.

Bovine colon tissue, while being more clinically relevant, simulating the target transluminal environment and biological dye interaction, also minimizes source reflection and thus, was the media used. To isolate each of the concentrations in a single, splayed opened colon section, a custom tissue test fixture was designed and 3D printed (Figure 2(a)). The double layer design invokes circumferential channel and inverted wall separation at each sample slot (total quantity 12). This compression barrier prevents dye dispersion and cross-sample contamination, while also facilitating quick and simple transition of the excitation laser between samples. This is essential to avoid degradation of ICG samples due to photobleaching over time. It must be noted that all dye sample preparation and testing was performed in a dark room setting to elude environmental light noise and photobleaching.

A 0.2mL volume of each ICG/BSA dye concentration was tangentially injected into individual tissue apertures of the test fixture assembly, forming a pseudo polyp (ICG blebs) within the submucosal layer. Fluorescence intensity tests were performed by sequentially directing the laser (785nm/60mW) tip, fixed 0.5cm from each polyp prominence (Figure 2(b), while collecting emission spectral measurements for a 3000ms exposure time (time at which the laser is directed at the target).

Dye concentration comparison testing was repeated with variation in filter configuration (single filter and double filter). Exposure time was reduced to 750ms when testing the long-pass filter. On emergence of the optimal dye concentration with respect to fluorescence intensity, further testing was performed solely on this concentration to investigate the dye’s behaviour to exposure of excitation light for longer periods of time, stimulating photobleaching. This was achieved by taking an initial emission spectrum reading (t=0.3s), allowing the excitation source to continually target the prominent tissue sample and re-test the spectral fluorescence after 5 minutes (t~5 mins). The final testing focused on the fluorescence intensity relationship with varied distance to target (tissue) and excitation power intensity.

Results

ICG/DMSO Fluorescence Investigation

The UV-spectroscopy fluorescent intensity results for ICG/DMSO solution as a function of concentrations and light wavelengths are shown in Figure 3. As can be seen in Figure 3(a), the fluorescence intensity increases with concentration up to the optimal quantity of 7.5pM, subsequently, decreasing exponentially with increased concentration. Figure 3(b) shows the spectral emissions for each of the concentrations, noting the shift in peak fluorescence intensity for a given concentration. Table 3 lists the maximum fluorescence value across the spread of ICG/DMSO dye concentrations.

Table 3: ICG /DMSO concentrations vs. fluorescent wavelengths results.

Focusing on the spectral intensity results for the 7.5pM concentration (Figure 4), the highest peak value for light absorption intensity is at wavelength 795nm, while the highest peak value for fluorescence intensity is at wavelength 835nm.

ICG/BSA Fluorescence Investigation

Figures 5 (a), (b) shows the normalized fluorescence intensity as a function of wavelengths for different concentrations using a Band-Pass (840nm) and Long-Pass (800nm) filter, respectively. Both results demonstrated a variation in fluorescence intensity performance with dye concentration, while also exhibiting a saturated excitation source reflection signal, centred at 785nm. Consistency of performance of 30pM in producing the maximum fluorescence intensity (820- 840nm range for long-pass) is apparent across both filter tests.

Subsequent plots (Figure 6) reveal the concentration performance using a double filter configuration (Long and Band-pass). The reflectance noise eradication is clearly evident, denoting the effectiveness of the dual filter combination. The relationship between fluorescent intensity, centred at 840nm, and dye concentration remains apparent and consistent with the previous single filter results, with 30pM performing optimally. This is shown in Figure 6(b).

The 30pM was adopted and examined for the remaining investigations, considering its optimal performance. Figure 7 shows the fluorescent light intensity from the tissue at two different time stamps (0.3 sec. and ~5 min), during fixed, continuous excitation illumination. An approximate six-fold reduction (0.882 to 0.146 Fl) in emission intensity is observed after 5 minutes.

Analysis into the effects of excitation distance on the produced fluorescence intensity for the optimal concentration unveils an exponential relationship (Figure 8). The rate at which the fluorescence tends towards zero reduces the further the illumination source is from the target tissue. From this perspective, the inventors can take into account effective distances in the imaging process, as well as choose the appropriate focal lens type at the collector tip.

In order to avoid the tissue damage due to the high-power light intensity (threshold), the effect of power intensity changes on emission intensity have been studied. The relationship between source excitation power intensity and fluorescence exhibits proportional linearity, as shown in Figure 9.

Discussion

Fluorescence imaging in colonoscopy garners potential to provide clinicians with additional and valuable information about the underlying Gl tissue, yet commercial flexible endoscope systems with simultaneous visible and NIR fluorescence imaging remain unavailable. Initial strides have been made towards the development of a flexible endoscopic NIR imaging device by first focusing on the target NIR fluorescent agent, ICG. The described experimental model (Figure 1) using a flexible Y-fibre optic bundle facilitated ICG fluorescence characterization in a bovine colon tissue sample, allowing for investigations of dye concentration performance and their relationships to physical excitation parameters. Prior UV spectral analysis of ICG/DMSO concentrations provided valuable insight into the dye’s fluorescent behaviour, permitting optimization of the optical equipment.

BSA, in protein levels imitating those found in healthy human blood provided an effective binding agent for ICG across the concentrations tested, with fluorescent emission observed in all test spectral results. Isolation of these emission signals was effectively achieved using a dual filter (long and band-pass) combination. However, single filter testing presented clear laser source reflectance noise signals, centered at 785nm (Figure 5). The filter’s specified transmission in the rejection regions of -0.01 % highlights the weak quantum yield of ICG fluorescence with respect to excitation source reflection intensity. Looking ahead, from an imaging perspective it is important that specified sensors exhibit maximal quantum efficiencies over ICG’s fluorescent range (800-850nm). Reflectance noise minimization is fundamental to true fluorescence imaging. The current y-fibre bundle tip lens configuration allows lens internal surface reflectance before leaving the flexible conduit. One possible design solution for this issue would be to separate the illumination fibre(s) and its respective tip lens from the emission collection fibres- lens assembly.

Dye concentration performance is assessed by comparing fluorescence intensities centered at 840nm. The optimal dye concentration was consistent throughout, with 30pM displaying the greatest capacity to produce fluorescence (Figure 6). From a clinical perspective, a 30pM concentration equates to a 1.628mg/kg dose of ICG, where the maximum daily allowance is 5mg/kg (adult). Although this is well within the acceptable limit, this dose requires 114g (~4.5x 25mg vials) of ICG for a 70kg patient, representing a 16 fold increase from doses commonly used in clinical practice. It must be noted that the peak fluorescence wavelength can shift with concentrations, as seen with ICG/DMSO (Table 3), whereby the peak wavelength shifts up the spectrum with increased concentration. Thus, centering the comparison at 840nm is a potential limitation to the study. Filter specification for a NIR imaging device should in practice maximize the potential for imaging ICG fluorescence and thus, a wider spectral band should be facilitated to reach the image sensor.

The process of photobleaching expends the fluorescent capacity of fluorophore molecules leaving them no longer optically valuable. The obtained results (Figure 7) show the extent to which the fluorescent capacity of 30pM ICG diminishes over a 5 minute period, rendering static NIR measurement sensitivity over longer periods reduced. AIM tissue classifiers tracked NIR intensity signals in patients over a 5 to 10 minute period, however, this classifier like many others rely on the dynamic turnover of fluorescent agents, resulting from its perfusion. Thus, circumventing static photobleaching. However, for direct target site submucosal applications, it may cause issues over longer exposure times.

The tortuous nature of the Gl tract means endoscopes now possess focal lengths and lens systems facilitating 1.5 to 100 mm depth of field (DoF) and 90 to 170° field of view (FoV), enhancing the surgical view. The observed exponential decay of fluorescence with distance (Figure 8(b)) may limit the sensitivity of multi-depth view NIR imaging, particularly for the case of NIR intensity quantification application. However, optical fibre optic fabrication via multicore designs may help to minimize DoF sensitivity decay. Not only for fluorescence collection, excitation illumination diffusion over the full FOV stresses the significance of fibre numerical aperture specification. Illumination power intensity can also play its part, whereby maximizing the power will result in improved dye activation and subsequent fluorescence (Figure 9). However, guidelines for laser exposure safety defined in IEC 60825-1 may limit the permissible excitation power. Fibre material effects light absorption and subsequently, power attenuation over probe length. The developed system observed fluorescence collection attenuations of 47.6dB (post filter) from a 55.17mW target tip excitation power. Excitation source attenuation (7dB) was however comparable to a commercial white light fibrescope (SpyGlass Direct Visualization Probe, Boston Scientific, US) where 7.3dB was observed. Interestingly, a commercial laparoscopic NIR capable system (Pinpoint, Endoscopic Fluorescence Imaging, Stryker, US) displayed greater power attenuation (9.7dB) measured across the scope length (438.5mW at light guide tip versus 45.25mW at scope distal tip power recording centered at 830nm).The work undertaken in [21] presents the current state of the art in simultaneous visible and NIR fluorescence imaging via a flexible endoscopic paradigm, successfully exploiting it for early colorectal lesion characterization. Unlike this work, which chose to adapt a commercially available fibrescope in combination with a colonoscope (mother-daughter technique) and a miniature endoscope, the aim of the work described here is to develop a custom flexible endoscopic probe specific for NIR fluorescence imaging. This provides opportunities to optimize the optical components and fibre (imaging and illumination) specifications for dynamic, perfusion based ICG fluorescence. This lends potential for added sensitivity and integration of perfusion- based AIM in tissue classification. Also in contrast is the dye selection, whereby ICG is a widely accepted and FDA approved fluorophore, facilitating early clinical adoption unlike the non-FDA approved fluorophore investigated in [21],

Figure 10(a) schematically depicts an imaging device according to an exemplary embodiment, generally as described with respect to the imagining device of Figure 1 (a). In this example, the set of flexible fibre-optic bundles includes a second fibre-optic bundle; and wherein the proximal end of the second fibre-optic bundle is optically coupled to the EMR detector. That is, the EMR detector is external relative to the target. A lens in included to increase dispersion of the first EMR, to maximize field of view. The number N of fibre-optics of the second fibre-optic bundle is optimized for collection. Reprocessing of the images may be required since the VIS and NIR images may not be aligned.

Figure 10(a) schematically depicts an imaging device according to an exemplary embodiment, generally as described with respect to the imagining device of Figure 1 (a). In this example, the EMR detector is disposed proximate or at the distal end of the first fibre-optic bundle; and wherein the imaging device comprises an electrical cable electrically coupled, for example at a distal end thereof, to the EMR detector. That is, the EMR detector is internal relative to the target. Particularly, the EMR detector comprises and/or is a miniature CCD/CMOS camera, configured to detect the second EMR and to detect visible light. In this way, reprocessing of the images is not required and the VIS and NIR images are aligned. A digital/micro wavelength filter is included.

Raw Fluorescence Detection Experimental Setup

Figure 1 (a) schematically depicts an imaging device according to an exemplary embodiment, comprising: an electromagnetic radiation, EMR, source, configured to provide first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; a set of flexible fibre-optic bundles, including a first fibre-optic bundle, having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end of the first fibre-optic bundle is flexibly positionable and/or orientable to illuminate a target with the first EMR, incident thereupon, provided by the EMR source; and an EMR detector configured to detect second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1.4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and to output signals corresponding to the detected second EMR.

The first step in the iterative development was to build a version of the set-up as shown in Figure 1 , to measure raw ICG fluorescence signals, substituting photon detection equipment in place of the camera sensor. Figure 1 (a), 1 (b) shows is a schematic design of the imaging device. The design utilises the “mother-daughter” technique, whereby the imaging device works in parallel with a conventional colonoscope, accessing the target site by being fed down the working channel of the colonoscope. The rest of the setup was built using commercial sourced components comprising a multi-arm fibre optic bundle, illumination source and wavelength filters. The Y-bundle fibre optic was 2 meters long, replicating the clinically used endoscopic systems, and capable of transmitting light in the visible and NIR spectral bands. The illumination source providing 300mW max power similar to that used clinically as well as in previous NIR fluorescence colonoscope development. This experimental set-up, although not yet fully clinically compatible (6.4mm outer diameter), is reflective of the intended use case being similar in concept to a probe that could be placed down the working channel of a endoscope (motherdaughter technique)(c. 3.2 mm channel).

A 785nm laser source (Roithner Lasertechnik, RLTMDL-785-2.5W-3) was selected as the excitation light, given its proximity to the peak absorption of the optimal ICG/DMSO concentration. The bundle fibre optic (Thorlabs, RP22) acts to both deliver illumination, collect and transfer fluorescent emission, via its three arm configuration: single mode optical fibre transmits the adjusted (macro objective lens, Olympus, MPLFLN10X, NA 0.3, and 3D adjusted stage, OP Mount, SF-1T) 785nm excitation wavelength to the target (tissue); multi-mode bundle fibre optics transmit collected fluorescence to a photon detector; combined single mode and multi-mode bundle arm that targets the site, both delivering and collecting light/emission energy. The interchangeable photon detectors consisted of a power meter (Thorlabs, PM100D/S121 C) and a spectrometer (Thorlabs, CCS200/M). Before the collected light reaches the detectors, the emission fluorescence is isolated using wavelength filters (Thorlabs, Long-pass, FEL0800 and Band-pass filters, FB840-10). The power meter was intermittently placed at the fibre illumination tip for source intensity measurements at the target site and also at the collection tip to determine power attenuation and fluorescence quantum yield.

This experimental setup was used to investigate the fluorescence emission of ICG dye (in bovine albumin solutions) concentrations along with the emission performance relationship to physical parameters in the aim of optimizing NIR imaging device design in a clinically relevant colonic polyp model.

Table 4: Bill of Materials

Flexible Image Fibre Guide Setup

Unfiltered Camera Setup

Moving from a fluorescence measurement setup to an imaging configuration required the replacement of the photon detector(s) with a camera (Raspberry Pi HQ Camera with Sony IMX477R sensor). The IR cut filter was removed to allow wavelengths greater than 650nm to reach the sensor wafer, while a 16mm telephoto lens (Raspberry Pi HQ Camera lens) focuses the target image to the sensor.

An imaging guide replaced the bundle fibre optics as the fluorescence and visible light signal conduit. The imaging guide fibre bundle (Fujikura FIGH-10-500N) provides a 10,000 pixel resolution. The original y-bundle fibre optic was retained as the excitation illumination guide. An image processor (Raspberry PI 4B) integrates directly to the camera and lens assembly via a ribbon cable. A display is connected to the processor allowing the user to control and visualise the camera feed. Using an aliquoted well of ICG/BSA (bovine serum albumin) solution, a static image of unfiltered fluorescence (high excitation reflection noise) was successfully captured with the camera setup.

Table 5: Bill of Materials

Filtered Camera Setup

To clean the fluorescence and remove the reflection noise of the excitation laser, a multibandpass filter was introduced into the camera lens assembly. To further improve and better simulate the intraoperative conditions, a perfusion model was designed and built into the setup. The model allowed the dynamic introduction of ICG/DMSO, and subsequently water to flush out the dye, into the focused area of the image guide, facilitating an increase and subsequent decrease in observed fluorescence through the perfusion tubing. To increase the laser source excitation target area, the y-bundle fibre was removed and a series of lenses (Fourier assembly) and a mirror were introduced into the setup to expand the excitation beam (Gaussian) over a widened area of the perfusion model.

Main idea: using fibre optics for illuminating and imaging target.

The additions to the setup allowed for the capture of a colour image of pure (filtered) ICG fluorescence and its dynamic change overtime, due to the increase and decrease ofthe quantity of dye through the perfusion model.

Table 6: Bill of Materials

Flexible Endoscopic NIR Imaging Setup

The objective of this iteration was to integrate the illumination/excitation and imaging into a more clinically appropriate paradigm. To achieve this, the polyscope (PolyDiagnost, PD-PS-0144p) flexible micro-endoscope was incorporated. The flexible scope is 185cm in length and has an outer diameter of 2.8mm, making it compatible with most colonoscope working channels. The scope incorporates an illumination fibre guide, which was adjusted to our laser source. An imaging fibre guide of is fed through the scope optic channel, providing the light conduit for translating both the visible light and NIR fluorescent information. A LED white light source (Thorlabs, OSL2) was also introduced into the setup to provide a clear white light image. An added infusion syringe pump (Harvard Apparatus, PHD Ultra) provides consistent and controllable delivery of dye to the perfusion model.

Table 7: Bill of Materials Image Processor Development

A major component in the development of a NIR imaging device is the digital sensing and interpretation of fluorescent images, which requires image processing techniques. Furthermore, integration of Al methods for tissue classification and perfusion quantification, whose inputs are video derived fluorescence intensity tracking, necessitates post-processing of the aforementioned flexible NIR imaging setup captured videos. The following section details the initial attempts to develop a video processing methodology facilitating compatibility with fluorescence tracking.

IBM Fluorescence Tracker

The fluorescence tracker in question is a video analysis software package developed by IBM (DTIF Consortium member). It works by recording the fluorescence intensity of pre-selected regions of interest (ROI) and plotting the intensity results over time. The software was developed and tailored for use with specific fluorescent videos from the Novadaq (Stryker) Pinpoint Fluorescence imaging stack, with the aim of assessing the behaviour of ICG (indocyanine green) fluorescence intra-operatively for tissue classification.

As the tracking software was developed for a different imaging system than the flexible system developed, there are certain functional features which must be met for compatibility. The system uses colour images/video feed to positionally track the ROI, such that it can account for motion of the camera or target. While the fluorescence intensity tracking uses a monochrome (grayscale) video feed to record the changes in the fluorescence. Monochrome camera sensors can more sensitively detect infrared light (i.e. fluorescence emission) and thus, can more effectively delineate fluorescence. In this single colour channel format, fluorescence intensity can be translated to a grey scale derived from the monochrome image pixel information (i.e. 0- 255 grayscale intensity).

The developed flexible imaging system incorporates a single colour image sensor, therefore, producing only a single colour image/video, unlike the dual feeds produced by the commercial Stryker system. Therefore, in order to track the fluorescence or grayscale intensity from the image produced, the fluorescence information from the original colour image must be isolated and subsequently converted to grayscale (Figure 11).

Static Fluorescence Imaging

Initial image processor development consisted of a writing a simple python script whose function was to read RGB (red, green, blue) .mp4 video files and convert them to grey. The OpenCV python library, which was developed for computer vision applications, was utilised. Once armoured with this, static ICG fluorescent videos were produced to test the compatibility with the IBM tracker. The videos were of the NIR scope being directed at an aliquoted sample filled with the optimum ICG/BSA (bovine serum albumin) concentration. The scope excitation fibres deliver the 785nm source and the imaging fibre bundle transmits the emitted fluorescence to the camera. The captured result was then processed in python, converting it to grayscale such that the IBM tracker can record the pixel intensity scale (0-255 grayscale). The tracker was successfully able to produce a graph of fluorescence intensity.

Further testing of static ICG fluorescence in more complex items (geometry, surface, colour, etc.) revealed limitations in the grayscale conversion. The fluorescence contrast, which is so clear in the colour image, is lost in the grayscale image where the fluorescent boundary and intensity are diminished (see Figure 12). This is down to the nature of the python grayscale conversion, whereby it takes the RBG components of an image and assigns them a weighted average value on the grayscale, thus the contrast is lost as different colours in the colour image can be assigned the same grayscale value. Therefore, the grayscale image is not a true representation of fluorescence in the same way that can be said about the commercial stack NIR video feed.

This presents an issue in fluorescence intensity differentiation from a colour image. The commercial stacks use specific NIR sensitive monochrome camera sensors to produce the NIR video feed and as the IBM tracker records the change in the grayscale intensity over time, which from the results observed may struggle to differentiate fluorescence in brighter images converted to grayscale. To better represent the fluorescence and improve image contrast in grayscale, the image processor was developed further to break down and split the image into RGB components, with the aim to assess whether one of the channels produces a better contrast (e.g. the blue component of the image), and subsequently use this channel to convert to grayscale (Figure 13).

In this case the blue representation provides the best contrast and strongest visual delineation of ICG fluorescence. Although the red image also shows the fluorescence more effectively than the straight grayscale conversion, the blue image simulates the monochrome IR feed from the commercial stacks, whereby the areas (tissue) that haven’t been perfused with ICG remain black (no fluorescence). The areas of foam without ICG in the red image are visible due to the red colour component of the yellow foam. When looking towards a clinical setting, blue would be the clear choice given its minimal translation within the colon. With that being said, it would be interesting to retry this test using ICG injected into animal tissue.

Dynamic Fluorescence Imaging With the developed image processor in place, the goal shifted to producing videos of dynamically changing fluorescence that simulated the in vivo perfusion, which could be captured and recorded by the IBM tracker. A perfusion model fixture was designed and integrated into the flexible NIR imaging setup. Firstly, the videos are recorded, where the inventors gradually introduce the dye, by injecting into the perfusion model. Once captured, the inventors passed the videos through our python software extracting the blue component of the video, subsequently gray scaling them and finally the inventors use the “separate-vids” function of the tracker to track the colour image and record the grayscale from the “blue” image (Figure 14, left). Below (Figure 14, right) shows the IBM tracking output intensity plot and ROI selection from two separate perfusion test video captures.

Improving the test setup and simplifying the perfusion model led to more representative plots from the IBM tracker, effectively simulating the wash in and wash out phase of ICG intra operatively, see Figure 14 (right). This provides a strong indication that the processed image is indeed compatible with the IBM fluorescence tracker and is sensitive enough to translate changes in fluorescence over time. To further progress this, more anatomically correct and potentially live animal trials must be undertaken.

Figure 15 schematically depicts a method according to an exemplary embodiment.

The method is of fluorescence imaging a target, for example an organ of a patient, comprising: flexibly positioning and/or orienting a distal end of a first fibre-optic bundle (S1501); illuminating, via the distal end of the first fibre-optic bundle, the target with first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm (S1502); and detecting second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and outputting signals corresponding to the detected second EMR (S1503).

The method may include any of the steps described herein.

Summary

In summary, the invention provides an imaging device, for example a fluorescence imaging device, and a method of imaging, for example fluorescence imaging. Raw ICG fluorescence performance for varied dye concentrations was successfully characterized using flexible fibre optics, yielding potential for its use as a NIR imaging modality in endoscopic and colorectal applications. Initial design considerations are indicated from the aforementioned findings for such a device. However, fibre optic image translation for human and computer interpretation of NIR signalling presents significant development challenges in the form of signal sensing sensitivity, noise filtering and optical performance in the development of a flexible NIR imaging device compatible with endoscopic systems.

Although a preferred embodiment has been shown and described, it will be appreciated by those skilled in the art that various changes and modifications might be made without departing from the scope of the invention, as defined in the appended claims and as described above.

Definitions

At least some of the example embodiments described herein may be constructed, partially or wholly, using dedicated special-purpose hardware. Terms such as ‘component’, ‘module’ or ‘unit’ used herein may include, but are not limited to, a hardware device, such as circuitry in the form of discrete or integrated components, a Field Programmable Gate Array (FPGA) or Application Specific Integrated Circuit (ASIC), which performs certain tasks or provides the associated functionality. In some embodiments, the described elements may be configured to reside on a tangible, persistent, addressable storage medium and may be configured to execute on one or more processors. These functional elements may in some embodiments include, by way of example, components, such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Although the example embodiments have been described with reference to the components, modules and units discussed herein, such functional elements may be combined into fewer elements or separated into additional elements. Various combinations of optional features have been described herein, and it will be appreciated that described features may be combined in any suitable combination. In particular, the features of any one example embodiment may be combined with features of any other embodiment, as appropriate, except where such combinations are mutually exclusive. Throughout this specification, the term “comprising” or “comprises” means including the components) specified but not to the exclusion of the presence of others.

Attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of the foregoing embodiment(s). The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

Claims

1. An imaging device comprising: an electromagnetic radiation, EMR, source, configured to provide first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; a set of flexible fibre-optic bundles, including a first fibre-optic bundle, having respective proximal ends and distal ends, wherein the proximal end of the first fibre-optic bundle is optically coupled to the EMR source and wherein the distal end of the first fibre-optic bundle is flexibly positionable and/or orientable to illuminate a target with the first EMR, incident thereupon, provided by the EMR source; and an EMR detector configured to detect second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1.4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and to output signals corresponding to the detected second EMR.

2. The imaging device according to claim 1 , wherein the set of flexible fibre-optic bundles includes a second fibre-optic bundle; and wherein the proximal end of the second fibre-optic bundle is optically coupled to the EMR detector.

3. The imaging device according to claim 2, wherein the second fibre-optic bundle comprises N fibre-optics, where N is a number in a range from 10² to 10⁸, preferable in a range from 10³ to 10⁶, more preferably in a range from 10⁴ to 10⁵.

4. The imaging device according to claim 1 , wherein the EMR detector is disposed proximate the distal end of the first fibre-optic bundle; and wherein the imaging device comprises an electrical cable electrically coupled to the EMR detector.

5. The imaging device according to any previous claim, comprising a flexible conduit containing the set of flexible fibre-optic bundles.

6. The imaging device according to claim 5, wherein the flexible conduit has an outside diameter in a range from 0.5 mm to 10 mm, preferably in a range from 1 mm to 7 mm for example 6.4 mm, more preferably in a range from 2 mm to 4 mm, for example 2.8 mm or 3.2 mm.

7. The imaging device according to any previous claim, comprising a first optical lens optically coupled to the distal end of the first fibre-optic bundle.

8. The imaging device according to any previous claim, comprising a second optical lens optically coupled to the EMR detector.

9. The imaging device according to any previous claim, comprising an optical filter, optically coupled to the EMR detector, configured to attenuate the first EMR having the first wavelength.

10. The imaging device according to any previous claim, wherein the EMR source is configured to provide the first EMR, having the first wavelength, at power in a range from 100 mW to 5000 mW, preferably in a range from 200 mW to 1000 mW, for example about 300 mW.

11 . The imaging device according to any previous claim, wherein the EMR source is configured to provide the first EMR, having the first wavelength, as pulsed EMR.

12. The imaging device according to any previous claim, comprising a computer having a processor and a memory, wherein the computer is configured to generate a first image of the illuminated target using the signals output from the EMR detector.

13. The imaging device according to claim 12, wherein the computer is configured to identify a feature of the generated image and optionally, to classify the identified feature.

14. An endoscope comprising the imaging device according to any previous claim.

15. A method of fluorescence imaging a target, for example an organ of a patient, comprising: flexibly positioning and/or orienting a distal end of a first fibre-optic bundle; illuminating, via the distal end of the first fibre-optic bundle, the target with first EMR, preferably monochromatic EMR, having a first wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.75 pm to 0.85 pm, most preferably in range from 780 nm to 805 nm, for example about 785 nm; and detecting second EMR, having a second wavelength in a range from 0.1 pm to 3 pm, preferably in a range from 0.4 pm to 1 .4 pm, more preferably in range from 0.80 pm to 0.85 pm, most preferably in range from 810 nm to 830 nm, for example about 825 nm, emitted by the illuminated target and outputting signals corresponding to the detected second EMR.