US20040208390A1 - Methods and apparatus for processing image data for use in tissue characterization - Google Patents

Methods and apparatus for processing image data for use in tissue characterization Download PDF

Info

Publication number
US20040208390A1
US20040208390A1 US10/418,975 US41897503A US2004208390A1 US 20040208390 A1 US20040208390 A1 US 20040208390A1 US 41897503 A US41897503 A US 41897503A US 2004208390 A1 US2004208390 A1 US 2004208390A1
Authority
US
United States
Prior art keywords
mask
image
data
tissue
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/418,975
Inventor
Chunsheng Jiang
Christopher Griffin
Ross Flewelling
Peter Costa
Stephen Sum
Jean-Pierre Schott
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Medispectra Inc
Original Assignee
Medispectra Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Medispectra Inc filed Critical Medispectra Inc
Priority to US10/418,975 priority Critical patent/US20040208390A1/en
Assigned to MEDISPECTRA, INC. reassignment MEDISPECTRA, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: COSTA, PETER J., GRIFFIN, CHRISTOPHER E., JIANG, CHUNSHENG, SCHOTT, JEAN-PIERRE, SUM, STEPHEN T., FLEWELLING, ROSS F.
Priority to AU2003259095A priority patent/AU2003259095A1/en
Priority to CA002491703A priority patent/CA2491703A1/en
Priority to EP03763350A priority patent/EP1532431A4/en
Priority to PCT/US2003/021347 priority patent/WO2004005895A1/en
Publication of US20040208390A1 publication Critical patent/US20040208390A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7253—Details of waveform analysis characterised by using transforms
    • A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • MDS-035F entitled, “Methods and Apparatus for Processing Spectral Data for Use in Tissue Characterization”
  • Attorney Docket No. MDS-035G entitled, “Methods and Apparatus for Evaluating Image Focus”
  • MDS-035H entitled, “Methods and Apparatus for Calibrating Spectral Data,” all of which are filed on even date herewith.
  • This invention relates generally to image processing. More particularly, in certain embodiments, the invention relates to methods of processing image data for use in a tissue classification scheme.
  • a chemical agent such as acetic acid
  • acetic acid is applied to enhance the differences in appearance between normal and pathological tissue.
  • Such acetowhitening techniques may aid a colposcopist in the determination of areas in which there is a suspicion of pathology.
  • Colposcopic techniques are not perfect. They generally require analysis by a highly-trained physician. Colposcopic images may contain complex and confusing patterns and may be affected by glare, shadow, or the presence of blood or other obstruction, rendering an indeterminate diagnosis.
  • Optical analysis methods have increasingly been used to diagnose disease in tissue. Optical analysis is based on the principle that the intensity of light that is transmitted from an illuminated tissue sample may indicate the state of health of the tissue. As in colposcopic examination, optical analysis of tissue may be conducted using a contrast agent such as acetic acid. The contrast agent is used to enhance differences in the light that is transmitted from normal and pathological tissues.
  • a contrast agent such as acetic acid. The contrast agent is used to enhance differences in the light that is transmitted from normal and pathological tissues.
  • Optical analysis offers the prospect of at least partially-automated diagnosis of tissue using a classification algorithm.
  • examinations using optical analysis may be adversely affected by glare, shadow, or the presence of blood or other obstruction, rendering an indeterminate diagnosis.
  • the invention provides methods for processing tissue-derived optical data for use in a classification algorithm.
  • Methods of the invention comprise application of image masks for identifying ambiguous or unclassifiable optical data.
  • the optical data may comprise, for example, spectral data and/or acetowhitening kinetic data used in a tissue classification scheme.
  • the invention improves the accuracy of tissue classification by properly identifying and accounting for optical data that are not representative of a zone of interest of a tissue sample.
  • non-representative data include, for example, data from tissue regions that are affected by an obstruction and/or regions that lie outside a diagnostic zone of interest.
  • tissue examination may include data from portions of the tissue sample that lie outside an identified zone of interest.
  • Regions that lie outside a zone of interest include, for example, a tissue wall (e.g., a vaginal wall), an os, an edge surface of a tissue (e.g., a cervical edge), tissue in the vicinity of a smoke tube, and non-tissue portions of the sample.
  • a preferred method of the invention comprises applying image masks to automatically identify data from regions of a tissue sample that are obstructed or that lie outside a zone of interest. Regions from which such data are obtained are then identified and characterized as being indeterminate. Optical data from these regions may be disqualified from further use in the tissue classification algorithm.
  • tissue classification scheme for example, to determine tissue-class probabilities.
  • Those probabilities may be “soft masked”—that is, weighted according to a likelihood the region (or a point within the region) is affected by an obstruction and/or lies outside a zone of interest.
  • the invention may also comprise applying image masks to identify regions of a tissue sample providing superior tissue classification data.
  • image masks to identify regions of a tissue sample providing superior tissue classification data.
  • soft masking of optical data from identified regions affords them greater weight in the tissue classification algorithm, compared with data from other regions.
  • An image mask as applied in the present invention may comprise a combination of image processing steps designed to isolate a particular feature of a tissue sample.
  • Exemplary image masks presented herein include a blood mask, a mucus mask, a speculum mask, a pooled fluid and foam mask, a glare mask, an os mask, a smoke tube mask, a vaginal wall mask, and a region-of-interest mask.
  • the area of a tissue sample identified by an image mask is considered to be “masked.”
  • the masked area may be represented as ones or zeros in a binary image, or, alternatively, the masked areas may simply be represented as a set of points or pixels.
  • An image mask of the invention may operate on a complete image of the tissue sample, or on parts of the image.
  • the invention provides a glare mask which is applied by dividing an image into blocks, determining a histogram for one or more of the blocks, and computing thresholds for each block based on its histogram. This compensates for variations in overall brightness levels in the image when computing intensity thresholds indicative of glare.
  • the invention comprises applying an image mask by determining one or more intermediate images before computing a final binary image.
  • the invention comprises applying a vaginal wall mask by determining a gradient image of the tissue sample, determining a skeletonized image from the gradient image, and performing edge linking and edge extension to obtain a final binary image mask.
  • Image masking techniques of the invention work particularly well when applied in tissue classification schemes which use spectral data.
  • tissue classification based on a principal component analysis method or a feature coordinate extraction method produces more accurate results when input spectral data are processed via image masking.
  • Accuracy may be further increased by employing a tissue classification scheme based on both a principal component analysis method and a feature coordinate extraction method.
  • the invention provides methods of performing fast and accurate image and spectral scans of the tissue, such that both image and spectral data are obtained from each of a plurality of regions of the tissue sample. Each data point is keyed to its respective region, and the data are used to characterize the condition of each of the regions of interest.
  • spectral and image data are acquired from a tissue sample over an approximately 10 to 15 second interval of time. In other embodiments, the scanning time may be longer or shorter.
  • the invention comprises compensating for image misalignment caused by patient movement during data acquisition. Furthermore, validating misalignment corrections improves the accuracy of diagnostic procedures that utilize data obtained over an interval of time, particularly where the misalignments are small and the need for accuracy is great. Methods of the invention may be performed in real time by determining misalignment corrections, validating them, and adjusting for them at the same time that optical data are being obtained.
  • the invention comprises providing image data from an area of a tissue sample, applying image masks to identify regions of the tissue that are outside a zone of interest or are affected by an obstruction, and processing optical data from the identified regions in a tissue classification scheme.
  • the step of providing image data may comprise the physical act of obtaining a video image of the tissue sample. Alternatively, simply supplying image data otherwise obtained from the tissue sample may encompass the providing step according to the invention.
  • FIG. 1 is a block diagram featuring components of a tissue characterization system according to an illustrative embodiment of the invention.
  • FIG. 2 is a schematic representation of components of the instrument used in the tissue characterization system of FIG. 1 to obtain spectral data and image data from a tissue sample according to an illustrative embodiment of the invention.
  • FIG. 3 is a block diagram of the instrument used in the tissue characterization system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 4 depicts a probe within a calibration port according to an illustrative embodiment of the invention.
  • FIG. 5 depicts an exemplary scan pattern used by the instrument of FIG. 1 to obtain spatially-correlated spectral data and image data from a tissue sample according to an illustrative embodiment of the invention.
  • FIG. 6 depicts front views of four exemplary arrangements of illumination sources about a probe head according to various illustrative embodiments of the invention.
  • FIG. 7 depicts exemplary illumination of a region of a tissue sample using light incident to the region at two different angles according to an illustrative embodiment of the invention.
  • FIG. 8 depicts illumination of a cervical tissue sample using a probe and a speculum according to an illustrative embodiment of the invention.
  • FIG. 9 is a schematic representation of an accessory device for a probe marked with identifying information in the form of a bar code according to an illustrative embodiment of the invention.
  • FIG. 10 is a block diagram featuring spectral data calibration and correction components of the tissue characterization system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 11 is a block diagram featuring the spectral data pre-processing component of the tissue characterization system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 12 shows a graph depicting reflectance spectral intensity as a function of wavelength using an open air target according to an illustrative embodiment of the invention.
  • FIG. 13 shows a graph depicting reflectance spectral intensity as a function of wavelength using a null target according to an illustrative embodiment of the invention.
  • FIG. 14 shows a graph depicting fluorescence spectral intensity as a function of wavelength using an open air target according to an illustrative embodiment of the invention.
  • FIG. 15 shows a graph depicting fluorescence spectral intensity as a function of wavelength using a null target according to an illustrative embodiment of the invention.
  • FIG. 16 is a representation of regions of a scan pattern and shows values of broadband reflectance intensity at each region using an open air target according to an illustrative embodiment of the invention.
  • FIG. 17 shows a graph depicting as a function of wavelength the ratio of reflectance spectral intensity using an open air target to the reflectance spectral intensity using a null target according to an illustrative embodiment of the invention.
  • FIG. 18 shows a graph depicting as a function of wavelength the ratio of fluorescence spectral intensity using an open air target to the fluorescence spectral intensity using a null target according to an illustrative embodiment of the invention.
  • FIG. 19 is a photograph of a customized target for factory/preventive maintenance calibration and for pre-patient calibration of the instrument used in the tissue characterization system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 20 is a representation of the regions of the customized target of FIG. 19 that are used to calibrate broadband reflectance spectral data according to an illustrative embodiment of the invention.
  • FIG. 21 shows a graph depicting as a function of wavelength the mean reflectivity of the 10% diffuse target of FIG. 19 over the non-masked regions shown in FIG. 20, measured using the same instrument on two different days according to an illustrative embodiment of the invention.
  • FIG. 22A shows a graph depicting, for various individual instruments, curves of reflectance intensity (using the BB 1 light source), each instrument curve representing a mean of reflectance intensity values for regions confirmed as metaplasia by impression and filtered according to an illustrative embodiment of the invention.
  • FIG. 22B shows a graph depicting, for various individual instruments, curves of reflectance intensity of the metaplasia-by-impression regions of FIG. 22A, after adjustment according to an illustrative embodiment of the invention.
  • FIG. 23 shows a graph depicting the spectral irradiance of a NIST traceable Quartz-Tungsten-Halogen lamp, along with a model of a blackbody emitter, used for determining an instrument response correction for fluorescence intensity data according to an illustrative embodiment of the invention.
  • FIG. 24 shows a graph depicting as a function of wavelength the fluorescence intensity of a dye solution at each region of a 499-point scan pattern according to an illustrative embodiment of the invention.
  • FIG. 25 shows a graph depicting as a function of scan position the fluorescence intensity of a dye solution at a wavelength corresponding to a peak intensity seen in FIG. 24 according to an illustrative embodiment of the invention.
  • FIG. 26 shows a graph depicting exemplary mean power spectra for various individual instruments subject to a noise performance criterion according to an illustrative embodiment of the invention.
  • FIG. 27A is a block diagram featuring steps an operator performs in relation to a patient scan using the system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 27B is a block diagram featuring steps that the system of FIG. 1 performs during acquisition of spectral data in a patient scan to detect and compensate for movement of the sample during the scan.
  • FIG. 28 is a block diagram showing the architecture of a video system used in the system of FIG. 1 and how it relates to other components of the system of FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 29A is a single video image of a target of 10% diffuse reflectivity upon which an arrangement of four laser spots is projected in a target focus validation procedure according to an illustrative embodiment of the invention.
  • FIG. 29B depicts the focusing image on the target in FIG. 29A with superimposed focus rings viewed by an operator through a viewfinder according to an illustrative embodiment of the invention.
  • FIG. 30 is a block diagram of a target focus validation procedure according to an illustrative embodiment of the invention.
  • FIG. 31 illustrates some of the steps of the target focus validation procedure of FIG. 30 as applied to the target in FIG. 29A.
  • FIG. 32A represents the green channel of an RGB image of a cervical tissue sample, used in a target focus validation procedure according to an illustrative embodiment of the invention.
  • FIG. 32B represents an image of the final verified laser spots on the cervical tissue sample of FIG. 32A, verified during application of the target focus validation procedure of FIG. 30 according to an illustrative embodiment of the invention.
  • FIG. 33 depicts a cervix model onto which laser spots are projected during an exemplary application of the target focus validation procedure of FIG. 30, where the cervix model is off-center such that the upper two laser spots fall within the os region of the cervix model, according to an illustrative embodiment of the invention.
  • FIG. 34 shows a graph depicting, as a function of probe position, the mean of a measure of focus of each of the four laser spots projected onto the off-center cervix model of FIG. 33 in the target focus validation procedure of FIG. 30, according to an illustrative embodiment of the invention.
  • FIG. 35 shows a series of graphs depicting mean reflectance spectra for CIN 2/3 and non-CIN 2/3 tissues at a time prior to application of acetic acid, at a time corresponding to maximum whitening, and at a time corresponding to the latest time at which data was obtained—used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 36 shows a graph depicting the reflectance discrimination function spectra useful for differentiating between CIN 2/3 and non-CIN 2/3 tissues, used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 37 shows a graph depicting the performance of two LDA (linear discriminant analysis) models as applied to reflectance data obtained at various times following application of acetic acid, used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 38 shows a series of graphs depicting mean fluorescence spectra for CIN 2/3 and non-CIN 2/3 tissues at a time prior to application of acetic acid, at a time corresponding to maximum whitening, and at a time corresponding to the latest time at which data was obtained, used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 39 shows a graph depicting the fluorescence discrimination function spectra useful for differentiating between CIN 2/3 and non-CIN 2/3 tissues in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 40 shows a graph depicting the performance of two LDA (linear discriminant analysis) models as applied to fluorescence data obtained at various times following application of acetic acid, used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 41 shows a graph depicting the performance of three LDA models as applied to data obtained at various times following application of acetic acid, used in determining an optimal window for obtaining spectral data according to an illustrative embodiment of the invention.
  • FIG. 42 shows a graph depicting the determination of an optimal time window for obtaining diagnostic optical data using an optical amplitude trigger, according to an illustrative embodiment of the invention.
  • FIG. 43 shows a graph depicting the determination of an optimal time window for obtaining diagnostic data using a rate of change of mean reflectance signal trigger, according to an illustrative embodiment of the invention.
  • FIG. 44A represents a 480 ⁇ 500 pixel image from a sequence of images of in vivo human cervix tissue and shows a 256 ⁇ 256 pixel portion of the image from which data is used in determining a correction for a misalignment between two images from a sequence of images of the tissue in the tissue characterization system of FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 44B depicts the image represented in FIG. 44A and shows a 128 ⁇ 128 pixel portion of the image, made up of 16 individual 32 ⁇ 32 pixel validation cells, from which data is used in performing a validation of the misalignment correction determination according to an illustrative embodiment of the invention.
  • FIG. 45 is a schematic flow diagram depicting steps in a method of determining a correction for image misalignment in the tissue characterization system of FIG. 1, according to an illustrative embodiment of the invention.
  • FIGS. 46A and 46B show a schematic flow diagram depicting steps in a version of the method shown in FIG. 45 of determining a correction for image misalignment according to an illustrative embodiment of the invention.
  • FIGS. 47A and 47B show a schematic flow diagram depicting steps in a version of the method shown in FIG. 45 of determining a correction for image misalignment according to an illustrative embodiment of the invention.
  • FIGS. 48 A-F depict a subset of adjusted images from a sequence of images of a tissue with an overlay of gridlines showing the validation cells used in validating the determinations of misalignment correction between the images according to an illustrative embodiment of the invention.
  • FIG. 49A depicts a sample image after application of a 9-pixel size (9 ⁇ 9) Laplacian of Gaussian filter (LoG 9 filter) on an exemplary image from a sequence of images of tissue, used in determining a correction for image misalignment, according to an illustrative embodiment of the invention.
  • Laplacian of Gaussian filter LiG 9 filter
  • FIG. 49B depicts the application of both a feathering technique and a Laplacian of Gaussian filter on the exemplary image used in FIG. 49A to account for border processing effects, used in determining a correction for image misalignment according to an illustrative embodiment of the invention.
  • FIG. 50A depicts a sample image after application of a LoG 9 filter on an exemplary image from a sequence of images of tissue, used in determining a correction for image misalignment according to an illustrative embodiment of the invention.
  • FIG. 50B depicts the application of both a Hamming window technique and a LoG 9 filter on the exemplary image in FIG. 50A to account for border processing effects in the determination of a correction for image misalignment according to an illustrative embodiment of the invention.
  • FIGS. 51 A-F depict the determination of a correction for image misalignment using methods including the application of LoG filters of various sizes, as well as the application of a Hamming window technique and a feathering technique according to illustrative embodiments of the invention.
  • FIG. 52 shows a graph depicting exemplary mean values of reflectance spectral data as a function of wavelength for tissue regions affected by glare, tissue regions affected by shadow, and tissue regions affected by neither glare nor shadow according to an illustrative embodiment of the invention.
  • FIG. 53 shows a graph depicting mean values and standard deviations of broadband reflectance spectral data using the BB 1 channel light source for regions confirmed as being obscured by blood, obscured by mucus, obscured by glare from the BB 1 source, obscured by glare from the BB 2 source, or unobscured, according to an illustrative embodiment of the invention.
  • FIG. 54 shows a graph depicting mean values and standard deviations of broadband reflectance spectral data using the BB 2 channel light source for regions confirmed as being obscured by blood, obscured by mucus, obscured by glare from the BB 1 source, obscured by glare from the BB 2 source, or unobscured, according to an illustrative embodiment of the invention.
  • FIG. 55 shows a graph depicting the weighted difference between the mean 20 reflectance values of glare-obscured regions and unobscured regions of tissue as a function of wavelength used in determining metrics for application in the arbitration step in FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 56 shows a graph depicting the weighted difference between the mean reflectance values of blood-obscured regions and unobscured regions of tissue as a function of wavelength used in determining metrics for application in the arbitration step in FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 57 shows a graph depicting the weighted difference between the mean reflectance values of mucus-obscured regions and unobscured regions of tissue as a function of wavelength, used in determining metrics for application in the arbitration step in FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 58 shows a graph depicting a ratio of the weighted differences between the mean reflectance values of glare-obscured regions and unobscured regions of tissue at two wavelengths, used in determining metrics for application in the arbitration step in FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 59 shows a graph depicting a ratio of the weighted differences between the mean reflectance values of blood-obscured regions and unobscured regions of tissue at two wavelengths, used in determining metrics for application in the arbitration step in FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 60 shows a graph depicting a ratio of the weighted differences between the mean reflectance values of mucus-obscured regions and unobscured regions of tissue at two wavelengths, used in determining metrics for application in the arbitration step in FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 61 shows a graph depicting as a function of wavelength mean values and confidence intervals of a ratio of BB 1 and BB 2 broadband reflectance spectral values for regions confirmed as being either glare-obscured or shadow-obscured tissue, used in determining metrics for application in the arbitration step in FIG. 1 according to an illustrative embodiment of the invention.
  • FIG. 62 shows a graph depicting BB 1 and BB 2 broadband reflectance spectral data for a region of tissue where the BB 1 data is affected by glare but the BB 2 data is not, according to an illustrative embodiment of the invention.
  • FIG. 63 shows a graph depicting BB 1 and BB 2 broadband reflectance spectral data for a region of tissue where the BB 2 data is affected by shadow but the BB 1 data is not, according to an illustrative embodiment of the invention.
  • FIG. 64 shows a graph depicting BB 1 and BB 2 broadband reflectance spectral data for a region of tissue that is obscured by blood, according to an illustrative embodiment of the invention.
  • FIG. 65 shows a graph depicting BB 1 and BB 2 broadband reflectance spectral data for a region of tissue that is unobscured, according to an illustrative embodiment of the invention.
  • FIG. 66 shows a graph depicting the reduction in the variability of broadband reflectance measurements of CIN 2/3-confirmed tissue produced by applying the metrics in the arbitration step 128 of FIG. 1 to remove data affected by an artifact, according to an illustrative embodiment of the invention.
  • FIG. 67 shows a graph depicting the reduction in the variability of broadband reflectance measurements of tissue classified as “no evidence of disease confirmed by pathology” produced by applying the metrics in the arbitration step 128 of FIG. 1 to remove data affected by an artifact, according to an illustrative embodiment of the invention.
  • FIG. 68 shows a graph depicting the reduction in the variability of broadband reflectance measurements of tissue classified as “metaplasia by impression” produced by applying the metrics in the arbitration step 128 of FIG. 1 to remove data affected by an artifact, according to an illustrative embodiment of the invention.
  • FIG. 69 shows a graph depicting the reduction in the variability of broadband reflectance measurements of tissue classified as “normal by impression” produced by applying the metrics in the arbitration step 128 of FIG. 1 to remove data affected by an artifact, according to an illustrative embodiment of the invention.
  • FIG. 70A depicts an exemplary image of cervical tissue divided into regions for which two types of reflectance spectral data and one type of fluorescence spectral data are obtained, according to an illustrative embodiment of the invention.
  • FIG. 70B is a representation of the regions depicted in FIG. 70A and shows the categorization of each region using the metrics in the arbitration step 128 of FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 71A depicts an exemplary image of cervical tissue divided into regions for which two types of reflectance spectral data and one type of fluorescence spectral data are obtained, according to an illustrative embodiment of the invention.
  • FIG. 71B is a representation of the regions depicted in FIG. 71A and shows the categorization of each region using the metrics in the arbitration step 128 of FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 72A depicts an exemplary image of cervical tissue divided into regions for which two types of reflectance spectral data and one type of fluorescence spectral data are obtained, according to an illustrative embodiment of the invention.
  • FIG. 72B is a representation of the regions depicted in FIG. 72A and shows the categorization of each region using the metrics in the arbitration step 128 of FIG. 1, according to an illustrative embodiment of the invention.
  • FIG. 73 is a block diagram depicting steps in a method of processing and combining spectral data and image data obtained in the tissue characterization system of FIG. 1 to determine states of health of regions of a tissue sample, according to an illustrative embodiment of the invention.
  • FIG. 74 is a block diagram depicting steps in the method of FIG. 73 in further detail, according to an illustrative embodiment of the invention.
  • FIG. 75 shows a scatter plot depicting discrimination between regions of normal squamous tissue and CIN 2/3 tissue for known reference data, obtained by comparing fluorescence intensity at about 460 nm to a ratio of fluorescence intensities at about 505 nm and about 410 nm, used in determining an NED spectral mask (NED spec ) according to an illustrative embodiment of the invention.
  • FIG. 76 shows a graph depicting as a function of wavelength mean broadband reflectance values for known normal squamous tissue regions and known CIN 2/3 tissue regions, used in determining an NED spectral mask (NED spec ) according to an illustrative embodiment of the invention.
  • FIG. 77 shows a graph depicting as a function of wavelength mean fluorescence intensity values for known squamous tissue regions and known CIN 2/3 tissue regions, used in determining an NED spectral mask (NED spec ) according to an illustrative embodiment of the invention.
  • FIG. 78 shows a graph depicting values of a discrimination function using a range of numerator wavelengths and denominator wavelengths in the discrimination analysis between known normal squamous tissue regions and known CIN 2/3 tissue regions, used in determining an NED spectral mask (NED spec ) according to an illustrative embodiment of the invention.
  • FIG. 79A depicts an exemplary reference image of cervical tissue from a patient scan in which spectral data is used in arbitration, NED spectral masking, and statistical classification of interrogation points of the tissue sample, according to an illustrative embodiment of the invention.
  • FIG. 79B is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 79A and shows points classified as “filtered” following arbitration, “masked” following NED spectral masking with two different sets of parameters, and “CIN 2/3” following statistical classification, according to an illustrative embodiment of the invention.
  • FIG. 79C is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 79A and shows points classified as “filtered” following arbitration, “masked” following NED spectral masking with two different sets of parameters, and “CIN 2/3” following statistical classification, according to an illustrative embodiment of the invention.
  • FIG. 79D is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 79A and shows points classified as “filtered” following arbitration, “maske” following NED spectral masking with two different sets of parameters, and “CIN 2/3” following statistical classification, according to an illustrative embodiment of the invention.
  • FIG. 80 shows a graph depicting fluorescence intensity as a function of wavelength from an interrogation point confirmed as invasive carcinoma by pathology and necrotic tissue by impression, used in determining a Necrosis spectral mask according to an illustrative embodiment of the invention.
  • FIG. 81 shows a graph depicting broadband reflectance BB 1 and BB 2 as functions of wavelength from an interrogation point confirmed as invasive carcinoma by pathology and necrotic tissue by impression, used in determining a Necrosis spectral mask according to an illustrative embodiment of the invention.
  • FIG. 82A depicts an exemplary reference image of cervical tissue from the scan of a patient confirmed as having advanced invasive cancer in which spectral data is used in arbitration, Necrosis spectral masking, and statistical classification of interrogation points of the tissue sample, according to an illustrative embodiment of the invention.
  • FIG. 82B is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 82A and shows points classified as “filtered” following arbitration, “masked” following application of the “Porphyrin” and “FAD” portions of the Necrosis spectral mask, and “CIN 2/3” following statistical classification, according to an illustrative embodiment of the invention.
  • FIG. 83 shows a graph depicting as a function of wavelength mean broadband reflectance values for known cervical edge regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a cervical edge/vaginal wall ([CE] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 84 shows a graph depicting as a function of wavelength mean fluorescence intensity values for known cervical edge regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a cervical edge/vaginal wall ([CE] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 85 shows a graph depicting as a function of wavelength mean broadband reflectance values for known vaginal wall regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a cervical edge/vaginal wall ([CE] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 86 shows a graph depicting as a function of wavelength mean fluorescence intensity values for known vaginal wall regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a cervical edge/vaginal wall ([CE] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 87A depicts an exemplary reference image of cervical tissue from a patient scan in which spectral data is used in arbitration and cervical edge/vaginal wall ([CE] spec ) spectral masking, according to an illustrative embodiment of the invention.
  • FIG. 87B is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 87A and shows points classified as “filtered” following arbitration and “masked” following cervical edge/vaginal wall ([CE] spec ) spectral masking, according to an illustrative embodiment of the invention.
  • FIG. 88 shows a graph depicting as a function of wavelength mean broadband reflectance values for known pooling fluids regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a fluids/mucus ([MU] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 89 shows a graph depicting as a function of wavelength mean fluorescence intensity values for known pooling fluids regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a fluids/mucus ([MU] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 90 shows a graph depicting as a function of wavelength mean broadband reflectance values for known mucus regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a fluids/mucus ([MU] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 91 shows a graph depicting as a function of wavelength mean fluorescence intensity values for known mucus regions and known CIN 2/3 tissue regions, used in a discrimination analysis to determine a fluids/mucus ([MU] spec ) spectral mask according to an illustrative embodiment of the invention.
  • FIG. 92A depicts an exemplary reference image of cervical tissue from a patient scan in which spectral data is used in arbitration and fluids/mucus ([MU] spec ) spectral masking, according to an illustrative embodiment of the invention.
  • FIG. 92B is a representation (obgram) of the interrogation points (regions) of the tissue sample depicted in FIG. 92A and shows points classified as “filtered” following arbitration and “masked” following fluids/mucus ([MU] spec ) spectral masking, according to an illustrative embodiment of the invention.
  • FIG. 93 depicts image masks determined from an image of a tissue sample and shows how the image masks are combined with respect to each spectral interrogation point (region) of the tissue sample, according to an illustrative embodiment of the invention.
  • FIG. 94A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding glare image mask, Glare vid , according to an illustrative embodiment of the invention.
  • FIG. 94B represents a glare image mask, Glare vid , corresponding to the exemplary image in FIG. 94A, according to an illustrative embodiment of the invention.
  • FIG. 95 is a block diagram depicting steps in a method of determining a glare image mask, Glare vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 96 shows a detail of a histogram used in a method of determining a glare image mask, Glare vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 97A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding region-of-interest image mask, [ROI] vid , according to an illustrative embodiment of the invention.
  • FIG. 97B represents a region-of-interest image mask, [ROI] vid , corresponding to the exemplary image in FIG. 120A, according to an illustrative embodiment of the invention.
  • FIG. 98 is a block diagram depicting steps in a method of determining a region-of-interest image mask, [ROI] vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 99A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding smoke tube image mask, [ST] vid , according to an illustrative embodiment of the invention.
  • FIG. 99B represents a smoke tube image mask, [ST] vid , corresponding to the exemplary image in FIG. 99A, according to an illustrative embodiment of the invention.
  • FIG. 100 is a block diagram depicting steps in a method of determining a smoke tube image mask, [ST] vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 101A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding os image mask, Os vid , according to an illustrative embodiment of the invention.
  • FIG. 101B represents an os image mask, Os vid , corresponding to the exemplary image in FIG. 101A, according to an illustrative embodiment of the invention.
  • FIG. 102 is a block diagram depicting steps in a method of determining an os image mask, Os vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 103A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding blood image mask, Blood vid , according to an illustrative embodiment of the invention.
  • FIG. 103B represents a blood image mask, Blood vid , corresponding to the exemplary image in FIG. 103A, according to an illustrative embodiment of the invention.
  • FIG. 104 is a block diagram depicting steps in a method of determining a blood image mask, Blood vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 105A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding mucus image mask, Mucus vid , according to an illustrative embodiment of the invention.
  • FIG. 105B represents a mucus image mask, Mucus vid , Corresponding to the exemplary reference image in FIG. 105A, according to an illustrative embodiment of the invention.
  • FIG. 106 is a block diagram depicting steps in a method of determining a mucus image mask, Mucus vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 107A depicts an exemplary reference image of cervical tissue obtained during a patient examination and used in determining a corresponding speculum image mask, [SP] vid , according to an illustrative embodiment of the invention.
  • FIG. 107B represents a speculum image mask, [SP] vid , corresponding to the exemplary image in FIG. 107A, according to an illustrative embodiment of the invention.
  • FIG. 108 is a block diagram depicting steps in a method of determining a speculum image mask, [SP] vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 109A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a vaginal wall image mask, [VW] vid , according to an illustrative embodiment of the invention.
  • FIG. 109B represents the image of FIG. 109A overlaid with a vaginal wall image mask, [VW] vid , following extension, determined according to an illustrative embodiment of the invention.
  • FIG. 110 is a block diagram depicting steps in a method of determining a vaginal wall image mask, [VW] vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIG. 111A depicts an exemplary image of cervical tissue obtained during a patient examination and used in determining a corresponding fluid-and-foam image mask, [FL] vid , according to an illustrative embodiment of the invention.
  • FIG. 111B represents a fluid-and-foam image mask, [FL] vid , corresponding to the exemplary image in FIG. 111A, according to an illustrative embodiment of the invention.
  • FIG. 112 is a block diagram depicting steps in a method of determining a fluid-and-foam image mask, [FL] vid , for an image of cervical tissue, according to an illustrative embodiment of the invention.
  • FIGS. 113 A-C show graphs representing a step in a method of image visual enhancement in which a piecewise linear transformation of an input image produces an output image with enhanced image brightness and contrast, according to one embodiment of the invention.
  • FIG. 114A depicts an exemplary image of cervical tissue obtained during a patient examination and used as a reference (base) image in a method of disease probability display, according to one embodiment of the invention.
  • FIG. 114B depicts the output overlay image corresponding to the reference image in FIG. 114A, produced using a method of disease probability display according to one embodiment of the invention.
  • FIG. 115A represents a disease display layer produced in a method of disease probability display for the reference image in FIG. 114A, wherein CIN 2/3 probabilities at interrogation points are represented by circles with intensities scaled by CIN 2/3 probability, according to one embodiment of the invention.
  • FIG. 115B represents the disease display layer of FIG. 114B following filtering using a Hamming filter, according to one embodiment of the invention.
  • FIG. 116 represents the color transformation used to determine the disease display layer image in a disease probability display method, according to one embodiment of the invention.
  • FIG. 117A depicts an exemplary reference image of cervical tissue having necrotic regions, obtained during a patient examination and used as a reference (base) image in a method of disease probability display, according to one embodiment of the invention.
  • FIG. 117B depicts the output overlay image corresponding to the reference image in FIG. 117A, including necrotic regions, indeterminate regions, and CIN 2/3 regions, and produced using a method of disease probability display according to one embodiment of the invention.
  • the invention provides systems and methods for obtaining spectral data and image data from a tissue sample, for processing the data, and for using the data to diagnose the tissue sample.
  • spectral data from a tissue sample includes data corresponding to any wavelength of the electromagnetic spectrum, not just the visible spectrum. Where exact wavelengths are specified, alternate embodiments comprise using wavelengths within a ⁇ 5 nm range of the given value, within a ⁇ 10 nm range of the given value, and within a ⁇ 25 nm range of the given value.
  • image data from a tissue sample includes data from a visual representation, such as a photo, a video frame, streaming video, and/or an electronic, digital or mathematical analogue of a photo, video frame, or streaming video.
  • a tissue sample may comprise, for example, animal tissue, human tissue, living tissue, and/or dead tissue.
  • a tissue sample may be in vivo, in situ, ex vivo, or ex situ, for example.
  • a tissue sample may comprise material in the vacinity of tissue, such as non-biological materials including dressings, chemical agents, and/or medical instruments, for example.
  • Embodiments of the invention include obtaining data from a tissue sample, determining which data are of diagnostic value, processing the useful data to obtain a prediction of disease state, and displaying the results in a meaningful way.
  • spectral data and image data are obtained from a tissue sample and are used to create a diagnostic map of the tissue sample showing regions in which there is a high probability of disease.
  • the systems and methods of the invention can be used to perform an examination of in situ tissue without the need for excision or biopsy.
  • the systems and methods are used to perform in-situ examination of the cervical tissue of a patient in a non-surgical setting, such as in a doctor's office or examination room.
  • the examination may be preceded or accompanied by a routine pap smear and/or colposcopic examination, and may be followed-up by treatment or biopsy of suspect tissue regions.
  • FIG. 1 depicts a block diagram featuring components of a tissue characterization system 100 according to an illustrative embodiment of the invention. Each component of the system 100 is discussed in more detail herein.
  • the system includes components for acquiring data, processing data, calculating disease probabilities, and displaying results.
  • an instrument 102 obtains spectral data and image data from a tissue sample.
  • the instrument 102 obtains spectral data from each of a plurality of regions of the sample during a spectroscopic scan of the tissue 104 .
  • video images of the tissue are also obtained by the instrument 102 .
  • one or more complete spectroscopic spectra are obtained for each of 500 discrete regions of a tissue sample during a scan lasting about 12 seconds.
  • any number of discrete regions may be scanned and the duration of each scan may vary.
  • a detected shift is compensated for in real time 106 .
  • one or more components of the instrument 102 may be automatically adjusted during the examination of a patient while spectral data are obtained in order to compensate for a detected shift caused by patient movement.
  • the real-time tracker 106 provides a correction for patient movement that is used to process the spectral data before calculating disease probabilities.
  • the illustrative system 100 of FIG. 1 uses image data to identify regions that are obstructed or are outside the areas of interest of a tissue sample 108 . This feature of the system 100 of FIG. 1 is discussed herein in more detail.
  • the system 100 shown in FIG. 1 includes components for performing factory tests and periodic preventive maintenance procedures 110 , the results of which 112 are used to preprocess patient spectral data 114 .
  • reference spectral calibration data are obtained 116 in an examination setting prior to each patient examination, and the results 118 of the pre-patient calibration are used along with the factory and preventive maintenance results 112 to preprocess patient spectral data 114 .
  • the instrument 102 of FIG. 1 includes a frame grabber 120 for obtaining a video image of the tissue sample.
  • a focusing method 122 is applied and video calibration is performed 124 .
  • the corrected video data may then be used to compensate for patient movement during the spectroscopic data acquisition 104 .
  • the corrected video data is also used in image masking 108 , which includes identifying obstructed regions of the tissue sample, as well as regions of tissue that lie outside an area of diagnostic interest.
  • image masking 108 which includes identifying obstructed regions of the tissue sample, as well as regions of tissue that lie outside an area of diagnostic interest.
  • a single image is used to compute image masks 108 and to determine a brightness and contrast correction 126 for displaying diagnostic results.
  • more than one image is used to create image masks and/or to determine a visual display correction.
  • spectral data are acquired 104 within a predetermined period of time following the application of a contrast agent, such as acetic acid, to the tissue sample.
  • a contrast agent such as acetic acid
  • four raw spectra are obtained for each of approximately 500 regions of the tissue sample and are processed.
  • a fluorescence spectrum, two broadband reflectance (backscatter) spectra, and a reference spectrum are obtained at each of the regions over a range from about 360 nm to about 720 nm wavelength.
  • the period of time within which a scan is acquired is chosen so that the accuracy of the resulting diagnosis is maximized.
  • a spectral data scan of a cervical tissue sample is performed over an approximately 12-second period of time within a range between about 30 seconds and about 130 seconds following application of acetic acid to the tissue sample.
  • the illustrative system 100 includes data processing components for identifying data that are potentially non-representative of the tissue sample.
  • potentially non-representative data are either hard-masked or soft-masked.
  • Hard-masking of data includes eliminating the identified, potentially non-representative data from further consideration. This results in an indeterminate diagnosis in the corresponding region.
  • Hard masks are determined in components 128 , 130 , and 108 of the system 100 .
  • Soft masking includes applying a weighting function or weighting factor to the identified, potentially non-representative data. The weighting is taken into account during calculation of disease probability 132 , and may or may not result in an indeterminate diagnosis in the corresponding region.
  • Soft masks are determined in component 130 of the system 100 .
  • Soft masking provides a means of weighting spectral data according to the likelihood that the data is representative of clear, unobstructed tissue in a region of interest. For example, if the system 100 determines there is a possibility that one kind of data from a given region is affected by an obstruction, such as blood or mucus, that data is “penalized” by attributing a reduced weighting to that data during calculation of disease probability 132 . Another kind of data from the same region that is determined by the system 100 not to be affected by the obstruction is more heavily weighted in the diagnostic step than the possibly-affected data, since the unaffected data is attributed a greater weighting in the calculation of disease probability 132 .
  • an obstruction such as blood or mucus
  • soft masking is performed in addition to arbitration of two or more redundant data sets.
  • Arbitration of data sets is performed in component 128 .
  • this type of arbitration employs the following steps: obtaining two sets of broadband reflectance (backscatter) data from each region of the tissue sample using light incident to the region at two different angles; determining if one of the data sets is affected by an artifact such as shadow, glare, or obstruction; eliminating one of the redundant reflectance data sets so affected; and using the other data set in the diagnosis of the tissue at the region. If both of the data sets are unaffected by an artifact, a mean of the two sets is used.
  • the instrument 102 obtains both video images and spectral data from a tissue sample.
  • the spectral data may include fluorescence data and broadband reflectance (backscatter) data.
  • the raw spectral data are processed and then used in a diagnostic algorithm to determine disease probability for regions of the tissue sample.
  • both image data and spectral data are used to mask data that is potentially non-representative of unobstructed regions of interest of the tissue.
  • both the image data and the spectral data are alternatively or additionally used in the diagnostic algorithm.
  • the system 100 also includes a component 132 for determining a disease probability at each of a plurality of the approximately 500 interrogation points using spectral data processed in the components 128 and 130 and using the image masks determined in component 108 .
  • the disease probability component 132 processes spectral data with statistical and/or heuristics-based (non-statistically-derived) spectral classifiers 134 , incorporates image and/or spectral mask information 136 , and assigns a probability of high grade disease, such as CIN 2+, to each examined region of the tissue sample.
  • the classifiers use stored, accumulated training data from samples of known disease state.
  • the disease display component 138 graphically presents regions of the tissue sample having the highest probability of high grade disease by employing a color map overlay of the cervical tissue sample.
  • the disease display component 138 also displays regions of the tissue that are necrotic and/or regions at which a disease probability could not be determined.
  • FIG. 2 is a schematic representation of components of the instrument 102 used in the tissue characterization system 100 of FIG. 1 to obtain spectral data and image data from a tissue sample according to an illustrative embodiment of the invention.
  • the instrument of FIG. 2 includes a console 140 connected to a probe 142 by way of a cable 144 .
  • the cable 144 carries electrical and optical signals between the console 140 and the probe 142 .
  • signals are transmitted between the console 140 and the probe 142 wirelessly, obviating the need for the cable 144 .
  • the probe 142 accommodates a disposable component 146 that comes into contact with tissue and may be discarded after one use.
  • the console 140 and the probe 142 are mechanically connected by an articulating arm 148 , which can also support the cable 144 .
  • the console 140 contains much of the hardware and the software of the system, and the probe 142 contains the necessary hardware for making suitable spectroscopic observations. The details of the instrument 100 are further explained in conjunction with FIG. 3.
  • FIG. 3 shows an exemplary operational block diagram 150 of an instrument 102 of the type depicted in FIG. 2.
  • the instrument 102 includes features of single-beam spectrometer devices, but is adapted to include other features of the invention.
  • the instrument 102 is substantially the same as double-beam spectrometer devices, adapted to include other features of the invention.
  • the instrument 102 employs other types of spectroscopic devices.
  • the console 140 includes a computer 152 , which executes software that controls the operation of the instrument 102 .
  • the software includes one or more modules recorded on machine-readable media such as magnetic disks, magnetic tape, CD-ROM, and semiconductor memory, for example.
  • the machine-readable medium is resident within the computer 152 .
  • the machine-readable medium can be connected to the computer 152 by a communication link.
  • one can substitute computer instructions in the form of hardwired logic for software or one can substitute firmware (i.e., computer instructions recorded on devices such as PROMs, EPROMS, EEPROMs, or the like) for software.
  • firmware i.e., computer instructions recorded on devices such as PROMs, EPROMS, EEPROMs, or the like
  • machine-readable instructions as used herein is intended to encompass software, hardwired logic, firmware, object code and the like.
  • the computer 152 of the instrument 102 is preferably a general purpose computer.
  • the computer 152 can be, for example, an embedded computer, a personal computer such as a laptop or desktop computer, or another type of computer, that is capable of running the software, issuing suitable control commands, and recording information in real-time.
  • the illustrative computer 152 includes a display 154 for reporting information to an operator of the instrument 102 , a keyboard 156 for enabling the operator to enter information and commands, and a printer 158 for providing a print-out, or permanent record, of measurements made by the instrument 102 and for printing diagnostic results, for example, for inclusion in the chart of a patient.
  • some commands entered at the keyboard 156 enable a user to perform certain data processing tasks, such as selecting a particular spectrum for analysis, rejecting a spectrum, and/or selecting particular segments of a spectrum for normalization.
  • Other commands enable a user to select the wavelength range for each particular segment and/or to specify both wavelength contiguous and non-contiguous segments.
  • data acquisition and data processing are automated and require little or no user input after initializing a scan.
  • the illustrative console 140 also includes an ultraviolet (UV) source 160 such as a nitrogen laser or a frequency-tripled Nd:YAG laser, one or more white light sources 162 such as one, two, three, four, or more Xenon flash lamps, and control electronics 164 for controlling the light sources both as to intensity and as to the time of onset of operation and the duration of operation.
  • UV ultraviolet
  • One or more power supplies 166 are included in the illustrative console 140 to provide regulated power for the operation of all of the components of the instrument 102 .
  • the illustrative console 140 of FIG. 3 also includes at least one spectrometer and at least one detector (spectrometer and detector 168 ) suitable for use with each of the light sources.
  • a single spectrometer operates with both the UV light source 160 and the white light source(s) 162 .
  • the same detector may record both UV and white light signals.
  • different detectors are used for each light source.
  • the illustrative console 140 further includes coupling optics 170 to couple the UV illumination from the UV light source 160 to one or more optical fibers in the cable 144 for transmission to the probe 142 , and coupling optics 172 for coupling the white light illumination from the white light source(s) 162 to one or more optical fibers in the cable 144 for transmission to the probe 142 .
  • the spectral response of a specimen to UV illumination from the UV light source 160 observed by the probe 142 is carried by one or more optical fibers in the cable 144 for transmission to the spectrometer and detector 168 in the console 140 .
  • the spectral response of a specimen to the white light illumination from the white light source(s) 162 observed by the probe 142 is carried by one or more optical fibers in the cable 144 for transmission to the spectrometer and detector 168 in the console 140 .
  • the console 140 includes a footswitch 174 to enable an operator of the instrument 102 to signal when it is appropriate to commence a spectral scan by stepping on the switch. In this manner, the operator has his or her hands free to perform other tasks, for example, aligning the probe 142 .
  • the console 140 additionally includes a calibration port 176 into which a calibration target may be placed for calibrating the optical components of the instrument 102 .
  • a calibration target may be placed for calibrating the optical components of the instrument 102 .
  • an operator places the probe 142 in registry with the calibration port 176 and issues a command that starts the calibration operation.
  • a calibrated light source provides a calibration signal in the form of an illumination of known intensity over a range of wavelengths, and/or at a number of discrete wavelengths.
  • the probe 142 detects the calibration signal, and transmits the detected signal through the optical fiber in the cable 144 to the spectrometer and detector 168 . A test spectral result is obtained.
  • a calibration of the spectral system can be computed as the ratio of the amplitude of the known illumination at a particular wavelength divided by the test spectral result at the same wavelength.
  • Calibration may include factory calibration 110 , preventive maintenance calibration 110 , and/or pre-patient calibration 116 , as shown in the system 100 of FIG. 1.
  • Pre-patient calibration 116 may be performed to account for patient-to-patient variation, for example.
  • FIG. 4 depicts the illustrative probe 142 of FIG. 2 resting within a calibration port 176 according to an illustrative embodiment of the invention.
  • the illustrative calibration port 176 is adjustably attached to the probe 142 or the console 140 to allow an operator to perform pre-patient calibration without assembling detachable parts.
  • the pre-patient calibration port may contain one or more pre-positioned calibration targets, such as a customized target 426 (see also FIG. 19) and a null target 187 , both described in more detail below.
  • factory and/or preventive maintenance calibration includes using a portable, detachable calibration port to calibrate any number of individual units, allowing for a standardized calibration procedure among various instruments.
  • the calibration port 176 is designed to prevent stray room light or other external light from affecting a calibration measurement when a calibration target is in place in the calibration port 176 .
  • the null target 187 can be positioned up against the probe head 192 by way of an actuator 189 such that the effect of external stray light is minimized.
  • the null target 187 is positioned out of the path of light between the customized target 426 and the collection optics 200 , as depicted in FIG. 4.
  • An additional fitting may be placed over the probe head 192 to further reduce the effect of external stray light.
  • the target 187 in the calibration port 176 is located approximately 100 mm from the probe head 192 ; and the distance light travels from the target 187 to the first optical component of the probe 142 is approximately 130 mm.
  • the location of the target (in relation to the probe head 192 ) during calibration may approximate the location of tissue during a patient scan.
  • the illustrative probe 142 includes probe optics 178 for illuminating a specimen to be analyzed with UV light from the UV source 160 and for collecting the fluorescent and broadband reflectance (backscatter) illumination from the specimen being analyzed.
  • the illustrative probe 142 of FIGS. 2 and 3 includes a scanner assembly 180 that provides illumination from the UV source 160 , for example, in a raster pattern over a target area of the specimen of cervical tissue to be analyzed.
  • the probe 142 also includes a video camera 182 for observing and recording visual images of the specimen under analysis.
  • the probe 142 also includes a targeting source 184 for determining where on the surface of the specimen to be analyzed the probe 142 is pointing.
  • the probe 142 also includes white light optics 186 to deliver white light from the white light source(s) 162 for recording the reflectance data and to assist the operator in visualizing the specimen to be analyzed.
  • the computer 152 controls the actions of the light sources 160 , 162 , the coupling optics 170 , 172 , the transmission of light signals and electrical signals through the cable 144 , the operation of the probe optics 178 and the scanner assembly 180 , the retrieval of observed spectra, the coupling of the observed spectra into the spectrometer and detector 168 via the cable 144 , the operation of the spectrometer and detector 168 , and the subsequent signal processing and analysis of the recorded spectra.
  • FIG. 4 depicts the probe 142 having top and bottom illumination sources 188 , 190 according to an illustrative embodiment of the invention.
  • the illumination sources 188 , 190 are situated at an upper and a lower location about the perimeter of a probe head 192 such that there is illuminating light incident to a target area at each of two different angles.
  • the target area is a tissue sample.
  • the probe head 192 contains probe optics 178 for illuminating regions of tissue and for collecting illumination reflected or otherwise emitted from regions of tissue.
  • the probe optics for collecting the illumination 200 are located between the top and bottom illumination sources 188 , 190 .
  • illuminating and collecting probe optics 178 are used that allow the illumination of a given region of tissue with light incident to the region at more than one angle.
  • One such arrangement includes the collecting optics 200 positioned around the illuminating optics.
  • the top and bottom illumination sources 188 , 190 are alternately turned on and off in order to sequentially illuminate the tissue at equal and opposite angles relative to the collection axis.
  • the top illumination source 188 is turned on while the bottom illumination source 190 is turned off, such that spectral measurements may be obtained for light reflected from a region of the tissue sample 194 illuminated with light incident to the region at a first angle. This angle is relative to the surface of the tissue sample at a point on the region, for example.
  • the top illumination source 188 is turned off while the bottom illumination source 190 is turned on, such that spectral measurements may be obtained using light incident to the region at a second angle.
  • the spectral measurements can include reflectance and/or fluorescence data obtained over a range of wavelengths.
  • the top and the bottom illumination sources 188 , 190 may be alternately cycled on and off more than once while obtaining data for a given region. Also, cycles of the illumination sources 188 , 190 may overlap, such that more than one illumination source is on at one time for at least part of the illumination collection procedure. Other illumination alternation schemes are possible, depending at least in part on the arrangement of illumination sources 188 , 190 in relation to the probe head 192 .
  • the scanner assembly 180 illuminates a target area of the tissue sample region-by-region.
  • a first region is illuminated using light incident to the region at more than one angle as described above, then the probe optics 178 are automatically adjusted to repeat the illumination sequence at a different region within the target area of the tissue sample.
  • the illustrative process is repeated until a desired subset of the target area has been scanned.
  • a desired subset of the target area preferably about five hundred regions are scanned within a target area having a diameter of about 25-mm.
  • the scan of the aforementioned five hundred regions takes about 12 seconds.
  • the number of regions scanned, the size of the target area, and/or the duration of the scan vary from the above.
  • FIG. 5 depicts an exemplary scan pattern 202 used by the instrument 102 to obtain spatially-correlated spectral data and image data from a tissue sample according to an illustrative embodiment of the invention.
  • spectral data are obtained at 499 regions of the tissue sample, plus one region out of the field of view of the cervix obtained, for example, for calibration purposes.
  • the exemplary scan pattern 202 of FIG. 5 includes 499 regions 204 whose centers are inside a circle 206 that measures about 25.8 mm in diameter. The center of each region is about 1.1 mm away from each of the nearest surrounding regions.
  • each scan line may be offsetting by about 0.9527 mm in the y-direction and by staggering each scan line in the x-direction by about 0.55 mm.
  • Each of the 499 regions is about 0.7 mm in diameter. In other illustrative embodiments, other geometries are used.
  • the spectral data acquisition component 104 of the system 100 depicted in FIG. 1 is performed using the scan pattern 202 shown in FIG. 5.
  • a fluorescence spectrum, two broadband reflectance spectra, and a reference spectrum are obtained at each region 204 .
  • the two broadband reflectance spectra use light incident to the sample at two different angles.
  • a scan preferably begins at the center region 208 , which corresponds to a pixel in a 500 ⁇ 480 pixel video image of the tissue sample at location 250 , 240 .
  • a sequence of video images of the tissue sample may be taken during a scan of the 499 regions shown in FIG. 5 and may be used to detect and compensate for movement of the tissue sample during the scan.
  • the real-time tracker component 106 of the system 100 shown in FIG. 1 performs this motion detection and compensation function.
  • the scanner assembly 180 of FIG. 3 includes controls for keeping track of the data obtained, detecting a stalled scan process, aborting the scan if the tissue is exposed to temperature or light outside of acceptable ranges, and/or monitoring and reporting errors detected by the spectral data acquisition component 104 of the system of FIG. 1.
  • FIG. 6 depicts front views of four exemplary arrangements 210 , 212 , 214 , 216 of illumination sources about a probe head 192 according to various illustrative embodiments of the invention.
  • the drawings are not to scale; they serve to illustrate exemplary relative arrangements of illumination sources about the perimeter of a probe head 192 .
  • Other arrangements include positioning collecting optics 200 around the perimeter of the probe head 192 , about the illumination sources, or in any other suitable location relative to the illumination sources.
  • the first arrangement 210 of FIG. 6 has one top illumination source 218 and one bottom illumination source 220 , which are alternately cycled on and off as described above.
  • the illumination sources are arranged about the collecting optics 200 , which are located in the center of the probe head 192 . Light from an illumination source is reflected from the tissue and captured by the collecting optics 200 .
  • the second arrangement 212 of FIG. 6 is similar to the first arrangement 210 , except that there are two illumination sources 222 , 224 in the top half of the probe head 192 and two illumination sources 226 , 228 in the bottom half of the probe head 192 .
  • the two lights above the midline 230 are turned on and the two lights below the midline 230 are turned off while obtaining a first set of spectral data; then the lights above the midline 230 are turned off and the lights below the midline 230 are turned on while obtaining a second set of spectral data.
  • only one of the four illumination sources are turned on at a time to obtain four sets of spectral data for a given region.
  • Other illustrative embodiments include turning the illumination sources on and off in other patterns.
  • Other alternative embodiments include using noncircular or otherwise differently shaped illumination sources, and/or using a different number of illumination sources.
  • the third arrangement 214 of FIG. 6 includes each illumination source 232 , 234 positioned on either side of the probe head 192 .
  • the sources 232 , 234 may be alternated in a manner analogous to those described for the first arrangement 210 .
  • the fourth arrangement 216 of FIG. 6 is similar to the second arrangement 212 , except that the illumination sources 236 , 238 on the right side of the probe head 192 are turned off and on together, alternately with the illumination sources 240 , 242 on the left side of the probe head 192 .
  • two sets of spectral data may be obtained for a given region, one set using the illumination sources 236 , 238 on the right of the midline 244 , and the other set using the illumination sources 240 , 242 on the left of the midline 244 .
  • FIG. 7 depicts exemplary illumination of a region 250 of a tissue sample 194 using light incident to the region 250 at two different angles 252 , 254 according to an illustrative embodiment of the invention.
  • FIG. 7 demonstrates that source light position may affect whether data is affected by glare.
  • the probe head 192 of FIG. 7 is depicted in a cut-away view for illustrative purposes.
  • the top illumination source 188 and bottom illumination source 190 are turned on sequentially and illuminate the surface of a tissue sample 194 at equal and opposite angles relative to the collection axis 256 .
  • Arrows represent the light emitted 252 from the top illumination source 188 , and the light specularly reflected 258 from the surface of the region 250 of the tissue sample 194 .
  • the emitted light 254 from the 20 bottom illumination source 190 reaches the surface of the region 250 of the tissue 194 and is specularly reflected into the collecting optics 200 , shown by the arrow 260 .
  • Data obtained using the bottom illumination source 190 in the example pictured in FIG. 7 would be affected by glare. This data may not be useful, for example, in determining a characteristic or a condition of the region 250 of the tissue 194 . In this example, it would be advantageous to instead use the set of data obtained using the top illumination source 188 since it is not affected by glare.
  • the position of the collection optics 200 may affect whether or not data is affected by glare.
  • light 252 with illumination intensity I o ( ⁇ ) strikes a tissue surface at a given region 250 .
  • a fraction of the initial illumination intensity, ⁇ I o ( ⁇ ) is specularly reflected from the surface 258 , where ⁇ is a real number between 0 and 1.
  • An acceptance cone 268 is the space through which light is diffusely reflected from the tissue 194 into the collecting optics 200 , in this embodiment.
  • Light may also be emitted or otherwise transmitted from the surface of the tissue.
  • the diffusely reflected light is of interest, since spectral data obtained from diffusely reflected light can be used to determine the condition of the region of the sample.
  • I t ( ⁇ ) is the intensity of light diffusely reflected from the region 250 on the surface of the tissue.
  • the collection optics 200 are off-center, light incident to the tissue surface may specularly reflect within the acceptance cone 268 .
  • light with illumination intensity I o ( ⁇ ) strikes the surface of the tissue.
  • Light with a fraction of the initial illumination intensity, ⁇ I o ( ⁇ ) from a given source is specularly reflected from the surface 266 , where ⁇ is a real number between 0 and 1.
  • ⁇ I o ( ⁇ ) is specularly reflected from the surface 266 , where ⁇ is a real number between 0 and 1.
  • the collected signal corresponds to an intensity represented by the sum I t ( ⁇ )+ ⁇ I o ( ⁇ ). It may be difficult or impossible to separate the two components of the measured intensity, thus, the data may not be helpful in determining the condition of the region of the tissue sample due to the glare effect.
  • FIG. 8 is a diagram 284 depicting illumination of a region 250 of a cervical tissue sample 194 using a probe 142 and a vaginal speculum 286 according to an illustrative embodiment of the invention.
  • the illuminating light incident to the tissue sample 194 is depicted by the upper and lower intersecting cones 196 , 198 .
  • the probe 142 operates without physically contacting the tissue being analyzed.
  • a disposable sheath 146 is used to cover the probe head 192 , for example, in case of incidental contact of the probe head 192 with the patient's body.
  • FIG. 9 is a schematic representation of an accessory device 290 that forms at least part of the disposable sheath 146 for a probe head 192 according to an illustrative embodiment of the invention.
  • the entire sheath 146 including the accessory device 290 , if present, is disposed of after a single use on a patient.
  • the disposable sheath 146 and/or the accessory device 290 have a unique identifier, such as a two-dimensional bar code 292 .
  • the accessory device 290 is configured to provide an optimal light path between the optical probe 142 and the target tissue 194 .
  • Optional optical elements in the accessory device 290 may be used to enhance the light transmitting and light receiving functions of the probe 142 .
  • tissue types may be analyzed using these methods, including, for example, colorectal, gastroesophageal, urinary bladder, lung, skin tissue, and/or any tissue comprising epithelial cells.
  • FIG. 10 is a block diagram 300 featuring components of the tissue characterization system 100 of FIG. 1 that involve spectral data calibration and correction, according to an illustrative embodiment of the invention.
  • the instrument 102 of FIG. 1 is calibrated at the factory, prior to field use, and may also be calibrated at regular intervals via routine preventive maintenance (PM). This is referred to as factory and/or preventive maintenance calibration 110 . Additionally, calibration is performed immediately prior to each patient scan to account for temporal and/or intra-patient sources of variability. This is referred to as pre-patient calibration 116 .
  • the illustrative embodiment includes calibrating one or more elements of the instrument 102 , such as the spectrometer and detector 168 depicted in FIG. 3.
  • Calibration includes performing tests to adjust individual instrument response and/or to provide corrections accounting for individual instrument variability and/or individual test (temporal) variability.
  • data is obtained for the pre-processing of raw spectral data from a patient scan.
  • the tissue classification system 100 of FIG. 1 includes determining corrections based on the factory and/or preventive maintenance calibration tests, indicated by block 112 in FIG. 10 and in FIG. 1. Where multiple sets of factory and/or preventive maintenance (PM) data exists, the most recent set of data is generally used to determine correction factors and to pre-process spectral data from a patient scan. Corrections are also determined based on pre-patient calibration tests, indicated by block 118 of FIG. 10.
  • the correction factors are used, at least indirectly, in the pre-processing ( 114 , FIG. 1) of fluorescence and reflectance spectral data obtained using a UV light source and two white light sources.
  • Block 114 of FIG. 11 corresponds to the pre-processing of spectral data in the overall tissue classification system 100 of FIG. 1, and is further discussed herein.
  • Calibration accounts for sources of individual instrument variability and individual test variability in the preprocessing of raw spectral data from a patient scan.
  • Sources of instrument and individual test variability include, for example, external light (light originating outside the instrument 102 , such as room light) and internal stray light. Internal stray light is due at least in part to internal “cross talk,” or interaction between transmitted light and the collection optics 200 .
  • Calibration also accounts for the electronic background signal read by the instrument 102 when no light sources, internal or external, are in use.
  • calibration accounts for variations in the amount of light energy delivered to a tissue sample during a scan, spatial inhomogeneities of the illumination source(s), chromatic aberration due to the scanning optics, variation in the wavelength response of the collection optics 200 , and/or the efficiency of the collection optics 200 , for example, as well as other effects.
  • factory and preventive maintenance calibration tests are performed to determine correction factors 112 to apply to raw fluorescence and reflectance spectral data obtained during patient scans.
  • the 20 factory/preventive maintenance calibration tests 110 include a wavelength calibration test 302 , a “null” target test 304 , a fluorescent dye cuvette test 306 , a tungsten source test 308 , an “open air” target test 310 , a customized target test 312 , and a NIST standard target test 314 .
  • the wavelength calibration test 302 uses mercury and argon spectra to convert a CCD pixel index to wavelengths (nm). A wavelength calibration and interpolation method using data from the mercury and argon calibration test 302 is described below.
  • the null target test 304 employs a target having about 0% diffuse reflectivity and is used along with other test results to account for internal stray light. Data from the factory/PM null target test 304 are used to determine the three correction factors shown in block 316 for fluorescence spectral measurements (F) obtained using a UV light source, and broadband reflectance measurements (BB 1 , BB 2 ) obtained using each of two white light sources. In one embodiment, these three correction factors 316 are used in determining correction factors for other tests, including the factory/PM fluorescent dye cuvette test 306 , the factory/PM open air target test 310 , the factory/PM customized target test 312 , and the factory/PM NIST standard target test 314 .
  • F fluorescence spectral measurements
  • BB 1 , BB 2 broadband reflectance measurements
  • the open air target test 310 , the customized target test 312 , and the NIST standard target test 314 are used along with the null target test 304 to correct for internal stray light in spectral measurements obtained using a UV light source and one or more white light sources.
  • the open air target test 310 is performed without a target and in the absence of external light (all room lights turned off).
  • the customized target test 312 employs a custom-designed target including a material of approximately 10% diffuse reflectivity and is performed in the absence of external light.
  • the custom-designed target also contains phosphorescent and fluorescent plugs that are used during instrument focusing and target focus validation 122 .
  • the custom-designed target is also used during pre-patient calibration testing ( 116 , 330 ) to monitor the stability of fluorescence readings between preventive maintenance procedures and/or to align an ultraviolet (UV) light source 160 —for example, a nitrogen laser or a frequency-tripled Nd:YAG laser.
  • UV ultraviolet
  • standard target test 314 employs a NIST-standard target comprising a material of approximately 60% diffuse reflectivity and is performed in the absence of external light. Correction factors determined from the “open air” target test 310 , the custom target test 312 , and the NIST-standard target test 314 are shown in blocks 322 , 324 , and 326 of FIG. 10, respectively. The correction factors are discussed in more detail below.
  • the fluorescent dye cuvette test 306 accounts for the efficiency of the collection optics 200 of a given unit.
  • the illustrative embodiment uses data from the fluorescent dye cuvette test 306 to determine a scalar correction factor 318 for fluorescence measurements (F) obtained using a UV light source.
  • the tungsten source test 308 uses a quartz-tungsten-halogen lamp to account for the wavelength response of the fluorescence collection optics 200 , and data from this test are used to determine a correction factor 320 for fluorescence measurements (F) obtained using a UV light source.
  • pre-patient calibration 116 is performed immediately before each patient scan.
  • the pre-patient calibration 116 includes performing a null target test 328 and a customized target test 330 before each patient scan. These tests are similar to the factory/PM null target test 304 and the factory/PM custom target test 312 , except that they are each performed under exam room conditions immediately before a patient scan is conducted.
  • the correction factors shown in blocks 332 and 334 of FIG. 10 are determined from the results of the pre-patient calibration tests.
  • correction factors ( 316 , 322 ) from the factory/PM null target test 304 and the factory/PM open air test 310 are used along with pre-patient calibration data to determine the pre-patient correction factors 118 , which are used, in turn, to pre-process raw spectral data from a patient scan, as shown, for example, in FIG. 11.
  • FIG. 11 is a block diagram 340 featuring the spectral data pre-processing component 114 of the tissue characterization system 100 of FIG. 1 according to an illustrative embodiment of the invention.
  • F represents the fluorescence data obtained using the UV light source 160
  • BB1 represents the broadband reflectance data obtained using the first 188 of the two white light sources 162
  • BB2 represents the broadband reflectance data obtained using the second 190 of the two white light sources 162 .
  • Blocks 342 and 344 indicate steps undertaken in pre-processing raw reflectance data obtained from the tissue using each of the two white light sources 188 , 190 , respectively.
  • Block 346 indicates steps undertaken in pre-processing raw fluorescence data obtained from the tissue using the UV light source 160 . These steps are discussed in more detail below.
  • the instrument 102 detailed in FIG. 3 features a scanner assembly 180 which includes a CCD (charge couple device) detector and spectrograph for collecting fluorescence and reflectance spectra from tissue samples. Because a CCD detector is used, the system employs a calibration procedure to convert a pixel index into wavelength units. Referring to FIG. 10, the pixel-to-wavelength calibration 302 is performed as part of factory and/or preventive maintenance calibration procedures 110 .
  • CCD charge couple device
  • the tissue classification system 100 uses spectral data obtained at wavelengths within a range from about 360 nm to about 720 nm.
  • the pixel-to-wavelength calibration procedure 302 uses source light that produces peaks near and/or within the 360 nm to 720 nm range.
  • a mercury lamp produces distinct, usable peaks between about 365 nm and about 578 nm
  • an argon lamp produces distinct, usable peaks between about 697 nm and about 740 nm.
  • the illustrative embodiment uses mercury and argon emission spectra to convert a pixel index from a CCD detector into units of wavelength (run).
  • a low-pressure pen-lamp style mercury lamp is used as source light, and intensity is plotted as a function of pixel index.
  • the pixel indices of the five largest peaks are correlated to ideal, standard Hg peak positions in units of nanometers.
  • a pen-lamp style argon lamp is used as source light and intensity is plotted as a function of pixel index. The two largest peaks are correlated to ideal, standard Ar peak positions in units of nanometers.
  • the seven total peaks provide a set of representative peaks well-distributed within a range from about 365 nm to about 738 nm—comparable to the range from about 360 nm to about 720 nm that is used for data analysis in the tissue classification system 100 .
  • the calibration procedure in block 302 of FIG. 10 includes retrieving the following spectra: a spectrum using a mercury lamp as light source, a mercury background spectrum (a spectrum obtained with the mercury source light turned off), a spectrum using an argon lamp as light source, and an argon background spectrum.
  • the respective Hg and Ar background spectra are subtracted from the Hg and Ar spectra, producing the background-corrected Hg and Ar spectra.
  • centroid ⁇ p max - 5 p max + 5 ⁇ p ⁇ ⁇ I p ⁇ ⁇ ⁇ p ⁇ p max + 5 ⁇ ⁇ I p ⁇ ⁇ ⁇ p , ( 1 )
  • p is pixel value
  • I p is the intensity at pixel p
  • p max is the pixel value corresponding to each peak maximum.
  • a polynomial function correlating pixel value to wavelength value is determined by performing a least-squares fit of the peak data.
  • the polynomial function is of fourth order.
  • the polynomial is of first order, second order, third order, fifth order, or higher order.
  • the pixel-to-wavelength calibration procedure 302 includes fitting a second order polynomial to the signal intensity versus pixel index data for each of the seven peaks around the maximum ⁇ 3 pixels (range including 7 pixels); taking the derivative of the second order polynomial; and finding the y-intercept to determine each p max .
  • the resulting polynomial function correlating pixel value to wavelength value is validated, for example, by specifying that the maximum argon peak be located within a given pixel range, such as [300:340] and/or that the intensity count at the peak be within a reasonable range, such as between 3000 and 32,000 counts. Additionally, the maximum mercury peak is validated to be between pixel 150 and 225 and to produce an intensity count between 3000 and 32,000 counts. Next, the maximum difference between any peak wavelength predicted by the polynomial function and its corresponding ideal (reference) peak is required to be within about 1.0 nm. Alternatively, other validation criteria may be set.
  • Additional validation procedures may be performed to compare calibration results obtained for different units, as well as stability of calibration results over time.
  • the pixel-to-wavelength calibration 302 and/or validation is performed as part of routine preventive maintenance procedures.
  • the illustrative system 100 standardizes spectral data in step 302 of FIG. 10 by determining and using values of spectral intensity only at designated values of wavelength.
  • Spectral intensity values are standardized by interpolating pixel-based intensities such that they correspond to wavelengths that are spaced every 1 nm between about 360 nm and about 720 nm. This may be done by linear interpolation of the pixel-based fluorescence and/or reflectance values.
  • Other illustrative embodiments use, for example, a cubic spline interpolation procedure instead of linear interpolation.
  • spectral data acquisition during patient scans and during the calibration procedures of FIG. 10 includes the use of a CCD array as part of the scanner assembly 180 depicted in FIG. 3.
  • the CCD array may contain any number of pixels corresponding to data obtained at a given time and at a given interrogation point.
  • the CCD array contains about 532 pixels, including unused leading pixels from index 0 to 9, relevant data from index 10 to 400, a power monitor region from index 401 to 521, and unused trailing pixels from index 522 to 531.
  • One embodiment includes “power correcting” or “power monitor correcting” by scaling raw reflectance and/or fluorescence intensity measurements received from a region of a tissue sample with a measure of the intensity of light transmitted to the region of the tissue sample.
  • the instrument 102 directs a portion of a light beam onto the CCD array, for example, at pixel indices 401 to 521 , and integrates intensity readings over this portion of the array.
  • both factory/PM 110 and pre-patient 116 calibration accounts for chromatic, spatial, and temporal variability caused by system interference due to external stray light, internal stray light, and electronic background signals.
  • External stray light originates from sources external to the instrument 102 , for example, examination room lights and/or a colposcope light.
  • the occurrence and intensity of the effect of external stray light on spectral data is variable and depends on patient parameters and the operator's use of the instrument 102 . For example, as shown in FIG. 8, the farther the probe head 192 rests from the speculum 286 in the examination of cervical tissue, the greater the opportunity for room light to be present on the cervix.
  • the configuration and location of a disposable component 146 on the probe head 192 also affects external stray light that reaches a tissue sample. Additionally, if the operator forgets to turn off the colposcope light before taking a spectral scan, there is a chance that light will be incident on the cervix and affect spectral data obtained.
  • Electronic background signals are signals read from the CCD array when no light sources, internal or external, are in use.
  • both external stray light and electronic background signals are taken into account by means of a background reading.
  • a background reading is obtained in which all internal light sources (for example, the Xenon lamps and the UV laser) are turned off.
  • the background reading immediately precedes the fluorescence and broadband reflectance measurements at each scan location, and the system 100 corrects for external stray light and electronic background by subtracting the background reading from the corresponding spectral reading at a given interrogation point.
  • each calibration test including 304 , 306 , 308 , 310 , 312 , 314 , 328 , and 330 —includes obtaining a background reading at each interrogation point and subtracting it from the test reading to account for external stray light and electronic background signals.
  • background subtraction is a step in the spectral data preprocessing 114 methods in FIG. 11, for the pre-processing of raw BB 1 and BB 2 reflectance data 342 , 344 as well as the pre-processing of raw fluorescence data 346 .
  • Equation 2 shows the background correction for a generic spectral measurement from a tissue sample, S tissue+ISL+ESL+EB (i, ⁇ )
  • i corresponds to a scan location
  • ⁇ is wavelength or its pixel index equivalent
  • subscripts denote influences on the spectral measurement—where “tissue” represents the tissue sample, “ISL” represents internal stray light (internal to the instrument 102 ), “ESL” represents external stray light, and “EB” represents electronic background.
  • S tissue+ISL+ESL+EB (i, ⁇ ) is a two-dimensional array (which may be power-monitor corrected) of spectral data obtained from the tissue at each interrogation point (region) i as a function of wavelength ⁇ ; and Bk EB+ESL (i, ⁇ ) is a two-dimensional array representing values of the corresponding background spectral readings at each point i as a function of wavelength ⁇ .
  • S tissue+ISL (i, ⁇ ) is the background-subtracted spectral array that is thereby corrected for effects of electronic background (EB) and external stray light (ESL) on the spectral data from the tissue sample.
  • the electronic background reading is subtracted on a wavelength-by-wavelength, location-by-location basis. Subtracting the background reading generally does not correct for internal stray light (ISL), as denoted in the subscript of S tissue+ISL (i, ⁇ ).
  • Internal stray light includes internal cross talk and interaction between the transmitted light within the system and the collection optics.
  • a primary source of internal stray light is low-level fluorescence of optics internal to the probe 142 and the disposable component 146 .
  • a primary source of internal stray light is light reflected off of the disposable 146 and surfaces in the probe 142 that is collected through the collection optics 200 .
  • the positioning of the disposable 146 can contribute to the effect of internal stray light on reflectance measurements.
  • the internal stray light effect may vary over interrogation points of a tissue sample scan in a non-random, identifiable pattern due to the position of the disposable during the test.
  • the factory/PM null target test 304 , the factory/PM open air target test 306 , the factory/PM custom target test 312 , the factory/PM NIST target test 314 , the pre-patient null target test 328 , and the pre-patient custom target test 330 provide correction factors to account for internal stray light effects on fluorescence and reflectance spectral measurements. In an alternative illustrative embodiment, a subset of these tests is used to account for internal stray light effects.
  • the null target test 304 , 328 performed in factory/preventive maintenance 110 , and pre-patient 116 calibration procedures, uses a target that has a theoretical diffuse reflectance of 0%, although the actual value may be higher. Since, at least theoretically, no light is reflected by the target, the contribution of internal stray light can be measured for a given internal light source by obtaining a spectrum from a region or series of regions of the null target with the internal light source turned on, obtaining a background spectrum from the null target with the internal light source turned off, and background-subtracting to remove any effect of electronic background signal or external stray light. The background-subtracted reading is then a measure of internal stray light.
  • the pre-patient null target test 328 takes into account spatially-dependent internal stray light artifacts induced by the position of a disposable 146 , as well as temporal variability induced, for example, by the aging of the instrument and/or dust accumulation.
  • the factory/PM null target test 304 is used in calculating correction factors from other factory and/or preventive maintenance calibration procedures.
  • the null target tests 304 , 328 are not perfect, and improved measurements of the effect of internal stray light on spectral data can be achieved by performing additional tests.
  • the open air target test 310 is part of the factory preventive maintenance (PM) calibration procedure 110 of FIG. 10 and provides a complement to the null target tests 304 , 328 .
  • the open air target test 310 obtains data in the absence of a target with the internal light sources turned on and all light sources external to the device turned off, for example, in a darkroom.
  • the null target test 304 by contrast, does not have to be performed in a darkroom since it uses a target in place in the calibration port, thereby sealing the instrument such that measurements of light from the target are not affected by external light.
  • a disposable 146 is in place during open air test measurements, the factory/PM open air target test 310 does not account for any differences due to different disposables used in each patient run.
  • the open air measurements are important in some embodiments, however, since they are performed under more controlled conditions than pre-patient calibration tests 116 , for example, the open air tests may be performed in a darkroom. Also, the factory/PM calibration 110 measurements account for differences between individual instruments 102 , as well as the effects of machine aging—both important factors since reference data obtained by any number of individual instruments 102 are standardized for use in a tissue classification algorithm, such as the one depicted in block 132 of FIG. 1.
  • FIGS. 12, 13, 14 , and 15 show graphs demonstrating mean background-subtracted, power-monitor-corrected intensity readings from a factory open air target test 310 and a null target test 304 using a BB 1 reflectance white light source and a UV light source (laser).
  • FIG. 12 shows a graph 364 of mean intensity 366 from an open air target test over a set of regions as a function of wavelength 368 using a BB 1 reflectance white light source 188 —the “top” source 188 as depicted in FIGS. 4, 7, and 8 .
  • FIG. 13 shows a graph 372 of mean intensity 366 from a null target test over the set of regions as a function of wavelength 368 using the same BB 1 light source. Curves 370 and 374 are comparable but there are some differences.
  • FIG. 14 shows a graph 376 of mean intensity 378 from an open air target test over a set of regions as a function of wavelength 380 using a UV light source
  • FIG. 15 shows a graph 384 of mean intensity 378 from a null target test over the set of regions as a function of wavelength 380 using the UV light source.
  • curves 382 and 386 are comparable, but there are some differences between them. Differences between the open air test intensity and null target test intensity are generally less than 0.1% for reflectance data and under 1 count/ ⁇ J for fluorescence data.
  • the open air target test measurement has a spatial profile that is dependent on the position of the disposable.
  • FIG. 16 shows a representation 390 of regions of an exemplary scan performed in a factory open air target test.
  • the representation 390 shows that broadband intensity readings can vary in a non-random, spatially-dependent manner.
  • Other exemplary scans performed in factory open air target tests show a more randomized, less spatially-dependent variation of intensity readings than the scan shown in FIG. 16.
  • the system 100 of FIG. 1 accounts for internal stray light by using a combination of the results of one or more open air target tests 310 with one or more null target tests 304 , 328 .
  • open air target test data is not used at all to correct for internal stray light, pre-patient null target test data being used instead.
  • FIG. 17 shows a graph 402 depicting as a function of wavelength 406 the ratio 404 of the background-corrected, power-monitor-corrected reflectance spectral intensity at a given region using an open air target to the reflectance spectral intensity at the region using a null target according to an illustrative embodiment of the invention.
  • the raw data 407 is shown in FIG. 17 fit with a second-order polynomial 412 , and fit with a third-order polynomial without filtering 410 , and with filtering 408 .
  • FIG. 18 shows a graph 414 depicting as a function of wavelength 418 the ratio 416 of fluorescence spectral intensity using an open air target to the fluorescence spectral intensity using a null target according to an illustrative embodiment of the invention.
  • the raw data 420 does not display a clear wavelength dependence, except that noise increases at higher wavelengths.
  • a mean 422 based on the ratio data 420 over a range of wavelengths is plotted in FIG. 18. Where a ratio of open air target to null target data is used to correct for internal stray light in fluorescence measurements, using a mean value calculated from raw data over a stable range of wavelength reduces noise and does not ignore any clear wavelength dependence.
  • FIG. 10 shows correction factors corresponding to open air 310 and null target 304 , 328 calibration tests in one embodiment that compensates spectral measurements for internal stray light effects.
  • spectral measurements There are three types of spectral measurements in FIG. 10—fluorescence (F) measurements and two reflectance measurements (BB 1 , BB 2 ) corresponding to data obtained using a UV light source and two different white light sources, respectively.
  • the corrections in blocks 316 , 322 , and 332 come from the results of the factory/PM null target test 304 , the factory/PM open air target test 310 , and the pre-patient null target test 328 , respectively, and these correction factors are applied in spectral data pre-processing (FIG. 11) to compensate for the effects of internal stray light.
  • F fluorescence
  • BB 1 , BB 2 two reflectance measurements
  • the corrections in blocks 316 , 322 , and 332 come from the results of the factory/PM null target test 304
  • Block 316 in FIG. 10 contains correction factors computed from the results of the null target test 304 , performed during factory and/or preventive maintenance (PM) calibration.
  • the null target test includes obtaining a one-dimensional array of mean values of spectral data from each channel—F, BB 1 , and BB 2 —corresponding to the three different light sources, as shown in Equations 3, 4, and 5:
  • FCNULLFL ⁇ I nt,F ( i, ⁇ ,t o )> i (3)
  • FCNULLBB 1 ⁇ I nt,BB1 ( i, ⁇ ,t o )> i (4)
  • FCNULLBB 2 ⁇ I nt,BB2 ( i, ⁇ ,t o )> i (5)
  • I nt refers to a background-subtracted, power-monitor-corrected two-dimensional array of spectral intensity values
  • subscript F refers to intensity data obtained using the fluorescence UV light source
  • subscripts BB 1 and BB 2 refer to intensity data obtained using the reflectance BB 1 and BB 2 white light sources, respectively
  • i refers to interrogation point “i” on the calibration target
  • ⁇ refers to a wavelength at which an intensity measurement corresponds or its approximate pixel index equivalent
  • t o refers to the fact the measurement is obtained from a factory or preventive maintenance test, the “time” the measurement is made
  • ⁇ > i represents a one-dimensional array (spectrum) of mean values computed on a pixel-by-pixel basis for each interrogation point, i.
  • data from an additional interrogation point is obtained from a region outside the target 206 .
  • Each of the reflectance intensity spectra is obtained over the same wavelength range as the fluorescence intensity spectra, but the BB 1 data is obtained at each of 250 interrogation points over the bottom half of the target and the BB 2 data is obtained at each of 249 interrogation points over the top half of the target.
  • the pre-patient null target test shown in block 328 of FIG. 10, is similar to the factory/PM null target test 304 , except that it is performed just prior to each patient test scan.
  • Each pre-patient null target test 328 produces three arrays of spectral data as shown below:
  • t′ refers to the fact the measurements are obtained just prior to the test patient scan, as opposed to during factory/PM testing (t o ).
  • Block 332 in FIG. 10 contains correction factors from the open air target test 310 , preformed during factory and/or preventive maintenance (PM) calibration 110 .