AU751456B2 - Multispectral/hyperspectral medical instrument - Google Patents

Description

WO 99/22640 WO 9922640PCTIUS98/2296 1 MIJLTISPECTRAL/HYPERSPECTRAL MEDICAL INSTRUMENT Field of the Invention The invention relates to a surgical and diagnostic instrument for performing real-time general-purpose imaging during surgery, clinical procedures, or other medical evaluations.

Background of the Invention Spectroscopic imaging devices which employ Acousto-Optic Tunable Filters (AOTF), Liquid Crystal Tunable Filters (LCTF), or dispersive gratings are known. Such devices have been used for microscopy and remote sensing.

Summar of the Invention Generally, the invention features a medical instrument that includes an optic responsive to a surface of tissue of a patient, a spectral separator optically responsive to the optic, and an imaging sensor optically responsive to the spectral separator. The instrument also includes a diagnostic processor having an image acquisition interface responsive to the imaging sensor and a filter control interface to which the spectral separator is responsive.

The spectral separator can be a tunable filter, such as a liquid crystal tunable filter, and the imaging sensor can be a two-dimensional imaging array, such as a charge coupled device.

The optic can include a macro lens, an adjustable lens, or a probe that includes an imaging fiber optic cable, and a stand can be connected relative to the optic to position the optic relative to the patient. The control interface can be operable to adjust the filter at least twenty times to acquire hyperspectral data for redisplay in real time. The medical instrument can perform diagnostic processing for images acquired exclusively under visible light.

The diagnostic processor can also include a general-purpose processing module and diagnostic protocol modules, which can each include filter transfer functions and an image processing protocol. The general-purpose processing module can be operative to instruct the WO 99/22640 PCTIUS98/22961 filter to successively apply the filter transfer functions to light collected from the patient, to acquire from the imaging sensor a number of images of the collected light each obtained after one of the filter transfer functions is applied, and to process the acquired images according to the image processing protocol to obtain a processed display image. The general-purpose processor can be a real-time processor operative to generate a processed display image within a time period on the order of the persistence of human vision. It may also be operative to acquire some images more slowly depending on the number of wavelengths and complexity of diagnostic processing protocol. The sensor and filter can be operative in the visible, infra-red, and UV regions.

Instruments according to the invention are advantageous in that they can permit a surgeon or a physician to diagnose a medical condition or develop a surgical strategy based on real-time images obtained during surgery or in the course of performing clinical procedures or other medical evaluations. The physician may therefore be able to obtain significantly more information about a patient's condition than he might otherwise have been able to assemble by presenting an interactive interface. This additional information may permit a given surgical procedure to be carried out more precisely and may lead to more successful surgical results. It may also enhance the precision and results of other types of medical evaluations and procedures.

The general-purpose nature of the instrument can also help the surgeon develop significant amounts of medical information in time-critical surgical situations. For example, a patient may undergo relatively straight-forward surgery during which the surgeon may discover a tumor or another internal condition. With an instrument according to the invention, the physician can spend a small amount of additional time with the patient under anesthesia and determine the nature and extent of the tumor. This can be particularly beneficial during major surgery, where extending surgery duration poses a potential morbidity and mortality risk.

Because the procedure is rapid and noninvasive, the patient is exposed to little additional risk.

The benefit of immediate diagnosis and evaluation is significant.

An instrument according to the invention may also be able to provide a wide variety of diagnostic capabilities, allowing a physician to enhance the capabilities of his or her practice substantially in a variety of different realms, without investing in a number of instruments. The physician can then enhance or update the instrument by the adding software modules that are specifically targeted towards certain conditions of particular tissues, subsystems, or disease states. This can allow a single base instrument to be configured for a variety of different types of practices, and priced according to the type of practice to be served by the instrument. For example, a general-purpose instrument to be used by a general surgeon could include a package of diagnostic protocols that would permit the diagnosis of a variety of conditions that a general surgeon might encounter, while a neurosurgeon's module might be added to allow a specialist to detect particular conditions within the brain. Electronic and optical upgrades may also be provided to update, specialize, or improve the performance of the instrument. Such upgrades can include processing modules, memory boards, lenses, and the like.

In one aspect the invention there is provided a medical instrument, including a first stage optic responsive to a tissue surface of a patient; a programmable spectral filter having a control input; an imaging sensor optically responsive to the spectral filter and having an image data output; and a diagnostic processor having a) an image acquisition i **interface with an input responsive to the imaging sensor and b) a filter control interface having a control output for controlling the programmable spectral filter, wherein the filter control interface can adjust the spectral filter at least twenty times to acquire twenty different spectral images in real time faster than the persistence of human vision.

In another aspect of the invention there is provided a multi-spectral diagnostic imaging method comprising the steps of collecting light from a patient surface area; acquiring a number of images, wherein each said number is equal to or greater than two and each image is acquired by an acquisition method comprising the step of applying a particular filter-transfer function to filter out all but a particular wavelength °region of interest from the light collected from the patient surface; specifying said number, said particular filter-transfer functions, and said particular wavelength region of interest according to a diagnostic protocol, wherein said diagnostic protocol is adapted to detect characteristics of said patient surface area; and processing said number images to obtain a display image.

In a further aspect, there is provided a medical instrument, including a first stage optic responsive to a tissue of a patient; a spectral separator optically responsive to the first stage optic and having a control input; an imaging sensor optically responsive to the spectral separator and having an image data output; and a diagnostic processor having an image acquisition interface with an input responsive to the imaging sensor, a filter control interface having a control output provided to the control input of the spectral separator, a general purpose operating module, and a plurality of diagnostic protocol modules, wherein each diagnostic protocol module is organ or tissue specific and contains a set of instructions for operating the spectral separator via the filter control interface and for operating the image acquisition interface.

Brief Description of the Drawings Fig. 1 is a perspective diagram of a macroscopic instrument according to the invention; Fig. 2 is a perspective diagram of a rigid or flexible probe-based instrument according to the invention; .ooo Fig. 3 is a block diagram of the instrument of Fig. 1; and Fig. 4 is a flowchart illustrating the operation of the system of Fig. 1.

Detailed Description of an Illustrative Embodiment Referring to Fig. 1, an instrument according to the invention 10 includes an imaging module 12 mounted on a surgical stand 14. In this embodiment, the surgeon can direct the imaging portion 12 towards a patient 20 by manipulating a control 16 that 2. 5 adjusts the attitude of the imaging portion through a positioning mechanism 18.

Referring to Fig. 2, an alternative embodiment of the invention 22 may include a °oo: probe such as a rigid or flexible endoscopic, thoracoscopic, laproscopic, or angioscopic 0 probe 24 connected to an imaging station 30 via a fiber-optic cable 26. The surgeon can manipulate the probe within the patient in a minimally-invasive surgical procedure and derive images from a WO 99/22640 PCT/US98/22961 portion of the patient and display these images on a display 28. A medical implement 32, such as a laser, can also be provided through the probe. For example, after diagnosing a particular condition, a physician can begin laser ablation therapy to remedy it.

Referring to Fig. 3, a medical instrument according to the invention 34 may include an optical acquisition system 36 and a diagnostic processor 38. The acquisition system 36 includes a first-stage imaging optic 40, a Liquid Crystal Tunable Filter (LCTF) 42, a second-stage optic 44, and an imaging element 46. The first-stage optic receives light collected from the patient and focuses it onto the surface of the LCTF. The first-stage optic can be a simple or compound macro lens in the case of a macroscopic instrument (Fig. In a probe-based instrument (Fig.

the first stage optic can include imaging optics within a probe such as a endoscopic, laproscopic, thoracoscopic, or angioscopic probe. The first stage lens can also be adjustable, allowing a physician to scan larger areas of tissue and then zoom into particular regions.

The LCTF 42 is a programmable filter that filters out all but a wavelength region of interest from the light collected from the patient. The second-stage optic 44 receives the remaining light from the LCTF and focuses it onto the image sensor 46. The image sensor is preferably, although not necessarily, a two-dimensional array sensor, such as a charge-coupled device array (CCD), which delivers an image signal to the diagnostic processor 38.

The diagnostic processor 38 includes an image acquisition interface 50, that has an input responsive to an output of the image sensor 46 and an output provided to a general-purpose operating module 54. The general-purpose operating module includes routines that perform image processing, and that operate and control the various parts of the system. It has a control output provided to a filter control interface 52, which in turn has an output provided to the LCTF 42. The general-purpose operating module also interacts with a number of diagnostic protocol modules 56A, 56B, 54N, and has an output provided to the video display 12. The diagnostic processor can include special purpose hardware, general-purpose hardware with special-purpose software, or a combination of the two. The diagnostic processor also includes an input device 58, which is operatively connected to the general-purpose operating module. A storage device and a printer are also operatively connected to the general-purpose operating module.

WO 99/22640 PCT/US98/22961 In operation, referring to Figs. 3 and 4, a surgeon employing the instrument begins by selecting a diagnostic protocol module using the instrument's input device (step 100). Each diagnostic protocol module is adapted to detect particular characteristics of the surface of one or more types of tissue. For example, the surgeon might select a module which enhances the visibility of cancerous tissue. The surgeon would then direct the camera at the area of interest and begin inspecting it either under ambient light or with the aid of a supplemental light source, which can be filtered to emphasize particular special characteristics of the light it emits.

The diagnostic processor 38 responds to the surgeon's input by obtaining a series of filter transfer functions and an image processing protocol from the selected diagnostic protocol module 56. The diagnostic processor provides the filtering transfer functions to the LCTF 42 via its filter control interface 52 (step 102) and then instructs the image acquisition interface 50 to acquire and store the resulting filtered image from the image sensor 46 (step 104). The general-purpose operating module 54 repeats these filtering and acquiring steps one or more times, depending on the number of filter transfer functions stored in the selected diagnostic protocol module (see step 106). The filtering transfer functions can represent bandpass, multiple bandpass, or other filter characteristics.

Once the image acquisition interface 50 has stored images for all of the image planes specified by the diagnostic protocol chosen by the surgeon, it begins processing these image planes based on the image processing protocol from the selected diagnostic protocol module 56N (step 108). Processing operations can include general image processing of combined images, such as comparing the relative amplitude of the collected light at different wavelengths, adding amplitudes of the collected light at different wavelengths, or computing other combinations of signals corresponding to the acquired planes. The processing operations can also include more complex multivariate statistical techniques to compute the image chemometrics). The computed image is displayed on the display 12. It can also be stored in the storage device 60 or printed out on the printer 62.

The processing operations can also be based on a diagnostic knowledge base. This database can include data resulting from the comparison between optical and actual diagnoses.

WO 99/22640 PCT/US98/22961 Each instrument can also continuously update its database as it is used to perform diagnoses, thereby steadily expanding its diagnostic capabilities.

In order to provide a real-time or near-real-time image to the surgeon, the instrument repeatedly acquires planes and processes them to develop an image to be displayed to the surgeon. This allows the surgeon to move the instrument, or to view moving organs, such as a beating heart. This constant acquisition and processing continues until the surgeon either turns the instrument off (step 110) or selects a different imaging mode (step 112). The diagnostic processor 38 preferably has sufficient processing power to update the screen in this way at video rates about 30 frames per second), although rates as low as a few frames per second may work quite well, and rates as low as one frame per minute may be adequate for many purposes.

On slower instruments, general lock-in schemes or other tracking modalities, such as cardiac gating, can be used to remove motion artifacts due to breathing or heart beat. Frame rate may also be variable, depending on the number of wavelengths and the complexity of the diagnostic procedure.

Preferably, the instrument can operate in multispectral, and hyperspectral, or even ultraspectral imaging modes. Multispectral modes involve image processing derived from a relatively small number of spectral image planes (two wavelengths to about twenty wavelengths). Hyperspectral and ultra spectral imaging modes involve at least twenty image planes and can produce significantly more accurate and informative results. Ultraspectral modes involve hundreds of wavelengths, and may be able to produce even further information about the patient. Hyperspectral and ultraspectral imaging may include selecting specific wavelength bands for discrimination of a particular diseased states, or it may also allow the instrument to scan for multiple conditions at the same time.

It is also contemplated that both types of instrument can operate in connection with an excitation source, such as an ultraviolet lamp and IR source, or other means of spectral illumination or a laser to enhance the received images. Although such excitation may not be necessary, it may allow for the examination of different optical phenomenon and provide additional diagnostic information. And both emission and reflectance modes can be combined in WO 99/22640 PCT/US98/22961 a diagnostic procedure either simultaneously or sequentially. Relative utilization of different emission or reflection measurements involved in the same diagnostic procedure can be obtained by modulating the different sources. The instrument can also develop light from bioluminescent sources introduced into the patient.

Instruments according to the invention can also operate to process images from image planes acquired at wavelengths outside of the visible region. In one particular embodiment, the instrument is sensitive to the visible and near infra-red regions. It is also contemplated that far infra-red be included to allow the instrument to sense molecular-specific rotational modes.

An example of operation would include the use of a diagnostic protocol module that examined a first wavelength of about 550 and a second wavelength of about 575 associated with oxy- and deoxy-hemoglobin to determine blood oxygenation. The relationship between these wavelengths is described in "Hemoglobin: Molecular Genetics and Clinical Aspects," by H.

Franklin Bunn and Bernard Forget, W. B. Sanders, 1986. Another example would include the use of a diagnostic protocol module for examining the Fourier transform infra-red spectra of the colon and rectum as described in "Human Colorectal Cancers Display Abnormal Fourier Transform Spectra, by Basil Rigas et al., Proceedings of the National Academy of Science, pp.

8140-8144, 1987.

Surgical and medical applications of instruments according to the invention can include, but are not limited to, determining tissue viability whether tissue is dead or living tissue and whether it is predicted to remain living), detecting tissue ischemia in heart, or in leg after a gunshot wound, differentiating between normal and malignant cells and tissues delineating tumors, dysplasias and precaucerous tissue, detecting metastasis), differentiating between of infected and normal (but inflamed) tissue extent of aortic root infection), quantification and identification of pathogens, bacterial count of bum wounds and differentiating and delineating other pathologic states. Application can also include tissue, blood chemistry, and blood flow (including oxy- and deoxyhemoglobin, myoglobin deoxymyoglobin, cytochrome, pH, glucose, calcium and other elements or biological compounds alone or in combination). The WO 99/22640 PCT/US98/22961 instrument can also be applied by veterinarians to animals and by dentists to dental applications, such as peridental disease.

The present invention has now been described in connection with a number of specific embodiments thereof. However, numerous modifications which are contemplated as falling within the scope of the present invention should now be apparent to those skilled in the art.

Therefore, it is intended that the scope of the present invention be limited only by the scope of the claims appended hereto. In addition, the order of presentation of the claims should not be construed to limit the scope of any particular term in the claims.

What is claimed is:

Claims (20)

1. A medical instrument, including: a first stage optic responsive to a tissue surface of a patient; a programmable spectral filter having a control input; an imaging sensor optically responsive to the spectral filter and having an image data output; and a diagnostic processor having a) an image acquisition interface with an input responsive to the imaging sensor and b) a filter control interface having a control output for controlling the programmable spectral filter, wherein the filter control interface can adjust the spectral filter at least twenty times to acquire twenty different spectral images in real time faster than the persistence of human vision. cs

2. The medical instrument of claim 1, wherein the spectral filter is a liquid crystal tunable filter.

3. The medical instrument of either claim 1 or claim 2, wherein the imaging sensor is a two-dimensional imaging array. 20

4. The medical instrument of any one of claims 1 to 3, further comprising a °supplemental light source. o.

The medical instrument of any preceding claim, wherein the diagnostic processor includes a general-purpose processing module and a plurality of diagnostic protocol modules.

6. The medical instrument of any preceding claim, wherein the filter and sensor are operable in the visible and far infra-red regions.

7. The medical instrument of any one of claims 1 to 5, wherein the filter and sensor are operable in the ultra-violet, visible and far infra-red regions.

8. The medical instrument of any preceding claim, wherein the spectral filter comprises multiple bandpass filters.

9. A multi-spectral diagnostic imaging method comprising the steps of: collecting light from a patient surface area; acquiring a number of images, wherein each said number is equal to or greater than two and each image is acquired by an acquisition method comprising the step of: applying a particular filter-transfer function to filter out all but a particular 15 wavelength region of interest from the light collected from the patient surface; specifying said number, said particular filter-transfer functions, and said particular wavelength region of interest according to a diagnostic protocol, wherein said diagnostic protocol is adapted to detect characteristics of said patient surface area; and processing said number images to obtain a display image. '..600 S. S. 0 tunable filter.

The method of claim 9, wherein the spectral filter is a liquid crystal

11. The method of either claim 9 or claim 10, wherein the imaging sensor is a two-dimensional imaging array.

12. The method of any one of claims 9 to 11, further comprising a supplemental light source.

13. The method of any preceding claim, wherein the diagnostic processor includes a general-purpose processing module and a plurality of diagnostic protocol modules.

14. The method of any preceding claim, wherein the filter and sensor are operable in the visible and far infra-red regions.

The method of any one of claims 9 to 13, wherein the filter and sensor are operable in the ultra-violet, visible and far infra-red regions.

16. The method of any preceding claim, wherein the spectral filter comprises multiple bandpass filters. 20

17. The method of any preceding claim, wherein the images are collected at a rate of at least 20 per second.

18. A medical instrument, including: _a first stage optic responsive to a tissue of a patient; a spectral separator optically responsive to the first stage optic and having a control input; an imaging sensor optically responsive to the spectral separator and having an image data output; and a diagnostic processor having an image acquisition interface with an input responsive to the imaging sensor, a filter control interface having a control output provided to the control input of the spectral separator, a general purpose operating module, and a plurality of diagnostic protocol modules, wherein each diagnostic protocol module is organ or tissue specific and contains a set of instructions for operating the spectral separator via the filter control interface and for operating the image acquisition interface.

19. A medical instrument substantially as hereinbefore described and/or illustrated with reference to any one or more of the accompanying drawings.

20. A multi-spectral diagnostic method substantially as hereinbefore described and/or illustrated with reference to any one or more of the accompanying drawings. DATED THIS TWENTY-SEVENTH DAY OF MAY 2002 S HYPERMED IMAGING, INC. and UNITED STATES GOVERNMENT AS REPRESENTED BY SECRETARY 25 DEPARTMENT OF HEALTH AND HUMAN SERVICES gS By PIZZEYS Patent and Trade Mark Attorneys