KR20130056886A - Method for analyzing biological specimens by spectral imaging - Google Patents

Abstract

A method of analyzing a biological sample by spectral imaging to provide a medical diagnosis includes acquiring a spectral and visual image of the biological sample, and registering the image to detect cell abnormalities, cells before cancer, and cancer cells. It includes. This method eliminates the bias and distrust of diagnosis inherent in standard histopathology and other spectral methods. In addition, a method of correcting confounding spectral contributions frequently observed in the infrared spectra of microscopically acquired cells and tissues includes performing phase corrections on the spectral data. This phase correction method can be used to correct various types of absorption spectra contaminated by reflective elements.

Description

METHOOD FOR ANALYZING BIOLOGICAL SPECIMENS BY SPECTRAL IMAGING

Aspects of the invention relate to a method of analyzing a biological sample by spectral imaging to provide a medical diagnosis. The biological sample may include medical specimens obtained by surgical methods, biopsies, and cultured samples.

Various pathological methods are used to analyze biological samples to detect abnormal or cancerous cells. For example, standard histopathology involves visual analysis of colored tissue portions by pathologists using a microscope. Typically, the tissue portion is removed from the patient by biopsy and the sample is frozen and split in minutes by freeze-microtome, or formalin-fixed, paraffin-containing and microtome splitting. The tissue portion is then mounted on a suitable substrate. The paraffin-containing tissue portion is then deparaffinized. For example, the tissue portion is colored and coverslips using hematoxylin-eosin (H & E) staining.

The tissue sample is then visually irradiated at 10 to 40 times magnification. Magnified cells are compared to a visual database in pathologist memory. Visual analysis of colored tissue sections by pathologists involves precise features such as nuclear and cell morphology, tissue structure, pigmentation patterns, and penetration of immune reactive cells to detect the presence of abnormal or cancerous cells.

If you suspect an initial metastasis or micrometastasis, which is a small cluster of cancer cells measured less than 0.2 to 2 mm in size, adjacent tissue parts are stained with an immunohistochemical (IHC) agent / counter stain, such as cytokeratin-specific staining. Can be. Since normal tissues, such as lymph node tissues, do not respond to this pigmentation, this method increases the sensitivity of histopathology. Eventually, the difference between uninfected and infected tissue can be large.

Histopathology was the primary method of microtransition discovery. For example, the discovery of micrometastasis in lymph nodes by standard histopathology is a tremendous task because of its small size and lack of ability to distinguish abnormalities in lymph node tissue. Since the spread of cancer can be inhibited if lymph nodes are not related to metastatic cells, the discovery of these micrometastasis is still important in the spread of disease. On the other hand, false negative diagnosis resulting from the absence of micrometastasis in the lymph nodes is very positive and more aggressive treatment should be recommended.

Although standard histopathology is well established in the diagnosis of advanced disease, it has a number of disadvantages. Because diagnosis and grading of diseases by this method are based on comparisons of samples with a pathological memory database that is subjective, it is common for each pathology to differ in the diagnosis of the same tissue part in particular. Differences in diagnosis occur especially in the diagnosis of rare cancers or in the early stages of cancer. In addition, standard histopathology is time consuming and expensive, and results are difficult to reproduce, depending on the human eye upon discovery. Furthermore, surgical fatigue and varying levels of pathologist expertise can influence the diagnosis.

In addition, if the tumors are very diverse, a lot of immunohistochemical staining may be needed to differentiate cancer types. Such staining can be performed on multiple parallel cell blocks. This staining process can be very expensive and the cell sample provides only a few diagnostic cells in one cell block.

To overcome the differences in diagnosis by standard histopathology, which mainly depends on cell morphology and tissue structure characteristics, spectroscopy is used to capture snapshots of biological components of cells and tissues. This allows for the discovery of changes in the biological composition of biological specimens caused by various conditions and diseases. As the tissue or cell sample uses spectroscopy, changes in the chemical makeup of a portion of the sample may be found, which may indicate the presence of abnormal or cancerous cells. Infrared cytopathology (cell disease research) application of spectroscopy is called spectral cytopathology (SCP), and histopathology (tissue disease research) application of infrared cytopathology is called spectral histopathology (SHP).

SCPs on individual urinary tract and cultured cells are described in B. Bird et al., Vibr. Spectros, 48, 10 (2008) and M. Romeo et al., Biochim Biophys Acta, 1758, 915 (2006). SCPs based on imaging data sets and applied to oral mucosa and cervical cells are discussed in WO 2009/146425. Description of disease progression through SCPs in oral mucosal cells is discussed in K. Papamarkakis et al., Laboratory Investigations, 90, 589 (2010). A description of the sensitivity of SCP to detecting cancer area effects and sensitivity to viral infections in cervical cells is discussed in K. Papamarkakis et al., Laboratory Investigations, 90, 589, (2010).

A description of the first unsupervised imaging of tissue using SHP of liver tissue via hierarchical cluster analysis (HCA) is discussed in M. Diem et al., Biopolymers, 57, 282 (2000). The discovery of metastatic cancer in lymph nodes is discussed in M. J. Romeo et al., Vibrational Spectrosc, 38, 1 15 (2005) and M. Romeo et al., Vibrational Microspectroscopy of Cells and Tissues, Wiley-lnterscience, Hoboken, NJ (2008). The use of trained neural networks in HCA-derived data to detect cancer in colon tissue is discussed in P. Lasch et al., J. Chemometrics, 20, 209 (2007). The discovery of micro-metastasis and individual metastatic cancer cells in lymph nodes is described by B. Bird et al., The Analyst, 134, 1067 (2009), B. Bird et al., BMC J. Clin. Pathology, 8, 1 (2008), and B. Bird et al., Tech. Cancer Res. Treatment, 10, 35 (201).

Spectroscopy is desirable in that it has prompted pathologists to be alert to small changes in chemical composition in biological samples that may show early stages of disease. Conversely, morphological changes in tissue that are evident from standard histopathology take longer, which makes early detection of the disease more difficult. In addition, spectroscopy allows pathologists to spend less time inspecting large samples of tissue or cellular material than visually examining the same sample. Furthermore, spectroscopy relies on device-based measurement tools that are objective, digitally recorded and stored, reproducible, and capable of mathematical and statistical analysis. As a result, the results derived from spectroscopy are more accurate and detailed than those derived from existing histopathological methods.

Various techniques can be used to obtain spectral data. For example, Raman spectroscopy, which assesses the molecular vibration of a system using scattering effects, can be used to analyze a cell or tissue sample. This method is described in N. Stone et al., Vibrational Spectroscopy for Medical Diagnosis, J. Wiley & Sons (2008), and C. Krafft et al., Vibrational Spectrosc (2011).

The Raman scattering effect is considered weak in that only about 1 out of 10 ¹⁰ incident photons withstand Raman scattering. Thus, Raman spectroscopy is most effective when using strongly focused visible or near-IR laser beams for stimulation. As a result, this indicates the point at which spectral information is collected. Depending on the numerical aperture of the microscope objective lens and the wavelength of the laser used, the size of this spot may range from about 0.3 μm to about 2 μm. Because the data set contains numerous spectra and may require long data recognition time, this small size makes it impossible to collect data from large tissue parts. After all, SHP using Raman spectroscopy requires an operator to select that small area. This approach negates the benefits of spectral imaging, such as non-polarized analysis of large areas of tissue.

SHP using infrared spectroscopy has been used to detect abnormalities in tissues including the brain, lungs, oral mucosa, cervical mucosa, thyroid, colon, skin, breast, esophagus, prostate and lymph nodes. Infrared spectroscopy, such as Raman spectroscopy, is based on molecular vibrations but is an absorption effect, and between 1% and 50% of incident infrared photons are easily absorbed if certain conditions are met. As a result, when compared to Raman spectroscopy, data can be recognized faster and with perfect spectrum quality by infrared spectroscopy. In addition, infrared spectroscopy is extremely sensitive to detecting small constitutive changes in tissues. After all, SHP using infrared spectroscopy is particularly advantageous for the diagnosis, treatment and prognosis of cancers, such as breast cancer, which are not frequently found until metastasis, since the micro-metastasis can be easily found. It can also find a small population of metastatic cancer cells that are as small as a few individual cells. Furthermore, the spatial resolution that can be obtained using infrared spectroscopy is comparable to the size of human cells, and commercial instruments incorporating large infrared detectors can collect tens of thousands of pixel spectra in minutes.

Methods of SHP using infrared spectroscopy are described in Bird et al., “Spectral detection of micro-metastates in lymph node histo-pathology”, J. Biophoton. 2, No. 1-2, 37-46 (2009) (hereinafter "Bird"). This method uses infrared micro-spectroscopy (IRMSP) and multivariate analysis to pinpoint micro-metastasis and individual metastatic cells in lymph nodes.

Bird 700 and 4000㎝ ^- studied the non-processing hyperspectral image data set containing 25,600 spectrum with 1650 points each spectral intensity between the ^first. This data set, which takes up about 400 MBytes each, is loaded and preprocessed. It includes a wave number range limits, and other processes of the ^first-data processing 900 - 1800㎝. The "fingerprint" infrared spectral region is then divided into a "protein region" between 1700 and 1450 cm ^-1 and governed by the amide I and amide II vibration bands of the peptide linkage of the protein. This region is very sensitive to different protein secondary and tertiary structures and can be used to cause certain phenomena of cell biology with different protein contents. The “phosphate region”, which ranges from 900 to 1350 cm ⁻¹ , has a low oscillation frequency, which, like DNA and RNA, includes some vibrations of phosphodiester linkages found in phospholipids.

In Bird, the minimum intensity criterion for incorporating amide I bonds was established to remove pixels without tissue integrity. Subsequently, vector normalization and spectral vector conversion to secondary derivatives are performed. The data sets are then subjected to hierarchical clustering analysis (HCA) individually using Euclidean distance to define spectral similarity and Ward's integration algorithm. Pixel set qualification is converted to pseudo-color spectroscopic images.

According to Bird's method, the indicators are placed on slides with colored tissue parts to highlight areas that match the non-pigmented adjacent tissue parts where spectral analysis will be performed. By the user aligning the distinctive features on the spectral image and the visual image, the spectral and visual image are thus physically connected to cover the spectral and visual image.

According to Bird's method, portions corresponding to the spectral image and the visual image are examined to determine the correlation between the visual observation and the spectral data. In particular, when investigating a ratio consistent with the spectral image covering the colored visual image, abnormal or cancer cells observed by the pathologist within the colored visual image may be observed. As a result, the contours of the pattern in the pseudo-color spectral image can match the known abnormal or cancer cells in the colored visual image. Potential abnormalities or cancer cells observed by the pathologist in the colored visual image can be used to verify the accuracy of the pseudo-color spectral image.

Bird's method is unclear, however, because it relies on the user's skills to visually link specific indicators on the spectrum and visual images. This method is often ambiguous. In addition, Bird's method allows the visual and spectral images to be physically linked to cover them, but not the data of the two images. Since the images are only physically coated, the overlapping images are not stored together for future analysis.

Furthermore, the covered image of the bird does not show the same tissue part because other adjacent parts of the tissue undergo spectral and visual imaging. This makes it difficult to link the spectrum with the visual image because there may be differences within the shape of the visual image and the color pattern of the spectral image.

Another problem with Bird's overlay method is that the visual image is not in the same spatial domain as the infrared spectral image. As a result, the spatial resolution of Bird's visual and spectral images is different. Typically, the spatial resolution of the infrared image is lower than the resolution of the visual image. To account for this difference in resolution, the data used in the infrared region can be expanded by selecting an area around the corresponding visual point and diagnosing that area rather than one point. At every point in the visual image there is an area of the infrared image that is larger than the point that must be entered for the diagnostic result to be produced. This process of explaining the difference in resolution is not done by Bird. Instead, Bird assumes that when selecting a point in the visual image, it is the same point of the information in the spectral image through overlay, and the corresponding diagnosis comes out. While the images are visually identical, they are not diagnostically identical.

In order to make a diagnostic connection, the spectral image used must come from a supervised diagnostic algorithm trained to recognize the corresponding diagnostic signature. Eventually, in order to create a diagnostic link that is not a user-selectable link, the spectral image cluster will be limited by an algorithmic classification scheme according to biochemical classification. In contrast, Bird uses "no supervision" HCA images when compared to "supervision" colored visual images for diagnosis. Based on the rules and limitations established for clustering, the HCA image identifies areas of common spectral features that have not yet been diagnosed, and allows boundary (geometric) connections to be visually accepted by the pathologist to outline the area of cancer. Manually cutting the plant until it is included. This method only provides a visual comparison.

Other methods based on fluorescence data analysis exist and generally use changes in intrinsic fluorescence, also known as self-fluorescence, based on the distribution of external tags such as coloration or labels. In recognizing biochemical makeup and changes in makeup, these methods are generally less diagnostic. In addition, these methods lack the fingerprint sensitivity of vibration spectroscopy techniques such as Raman and infrared.

A common problem with spectral recognition techniques is that large amounts of spectral data are collected when testing biological samples. As a result, the data analysis process becomes computationally complex and time consuming. Spectral data often include entangled spectral functions that are frequently observed within the infrared spectrum of cells and tissues obtained microscopically, such as scattering and baseline artificial products. In the end, it is helpful to examine the spectral data in advance to isolate the corresponding cellular material and remove the disturbing spectral function.

One type of entanglement spectral feature is Mie scattering, which is a sample form-dependent effect. If the sample is non-uniform and contains particles of a size close to the wavelength of light acting on the sample, this effect interferes with infrared absorption and reflection measurements. Microscattering is manifested by a massive and undulating scattering function, overlaid by infrared absorption.

Non-scattering may mediate mixing of absorptive and reflective linear shapes. In principle, a pure absorbent linear corresponds to the frequency-dependent absorbency and is mostly Gaussian, Lorentian, or a mixture of the two. The absorption curve corresponds to the imaginary part of the composite refractive index. The reflective contribution corresponds to the real part of the composite index of refraction and is dispersed in a linear fashion. By numerical KK-transformation or as a real part of the composite Fourier transform (FT), the distributive contribution can be obtained from the absorbent linear.

The resonance RMie function comes from the mixing of absorbent and reflective contention forms and occurs when the absorptivity is at its maximum because the refractive index undergoes anomalous dispersion (ie on the profile of the absorption band). Other optical effects that depend on non-scattering or index of refraction will mix reflective and absorptive linearity, causing distortion of the band profile and apparent frequency shift.

1 shows the contamination of the absorption pattern by the dispersion band form observed in both SCP and SHP. The bottom trace of FIG. 1 shows the general absorption spectrum of the biological tissue, while the top trace shows the spectrum severely contaminated by the dispersive element through the RMie effect. The spectral distortion appears to be unrelated to the chemical makeup but depends on the shape of the sample. The resulting band intensities and frequency shifts exacerbate spectral analysis so that the non-contamination and contamination spectra are classified into different groups because of band shift occurrences. The vast and undulating background function is shown in FIG. When superimposed on an infrared micro-spectrometry (IR-MSP) pattern of cells, these functions are due to scattering by globular particles such as cell nuclei or globular cells.

The dispersive linear appearance of FIG. 1 overlaid on IR-MSP was reported according to theoretical analysis in M. Romeo, et al., Vibrational Spectroscopy, 38, 129 (2005) (hereafter “Romeo 2005”). From superposition of dissipative (reflective) elements to the absorbing function of the infrared spectrum, Romeo 2005 distinguishes between distorted band shapes. These effects are due to incorrect phase correction in the instrument control software. In particular, the original interferogram obtained within FTIR spectroscopy is frequently “chirped” or asymmetric and must be symmetrical before FT. This is done by gathering the interferograms on both sides on the shorter interferometer stroke and calculating the phase correction to yield a symmetric interferogram.

In Romeo 2005, it was assumed that this process did not work properly, which caused it to produce a distorted spectral function. Attempts are made to correct the distorted spectral function by calculating the phase between the real and imaginary parts of the distorted spectrum and reconstructing the power spectrum from the phases of the corrected real and imaginary parts. Romeo 2005 also reports that the refractive index undergoes anomalous dispersion in each absorption band of the observed infrared spectrum. In certain circumstances, various amounts of dispersible linear may overlap or mix with the absorbance spectrum.

The mathematical relationship between the absorptive and reflective bands is presented by the Kramers-Kronig (KK) transformation, which is related to two physical phenomena. Mixing of dispersive (reflective) and absorptive effects in the observed spectra has been identified and a method of correcting the effects through a procedure called "phase correction (PC)" is discussed in Romeo 2005. Although the cause of the mix of dispersibility and absorbency contributions is incorrectly due to instrument software malfunction, the principle of entanglement effect has been well identified. However, because of the incomplete understanding of basic medicine, the correction method presented above does not work correctly.

P. Bassan et al., Analyst, 134, 1586 (2009) and P. Bassan et al., Analyst, 134, 1 171 (2009) suggest that the dispersibility and absorbency effects can be mixed through the “resonance misscattering” (RMieS) effect. Explain. Algorithms and methods for correcting spectral distortion have been described in P. Bassan et al., "Resonant Mie Scattering (RMieS) correction of infrared spectra from highly scattering biological samples", Analyst, 135, 268-277 (2010). This method is described in A. Kohler et al., Appl. Spectrosc, 59, 707 (2005) and A. Kohler et al., Appl. It is an extension of the "Extended Multiplicative Signal Correction" (EMSC) method reported in Spectrosc, 62, 259 (2008).

By including reflective elements obtained through the KK-transformation of the pure absorption spectrum in the multiple linear regression model, this method eliminates non-resonant microscattering from the infrared spectral database. This method uses the original data set and the "Reference" spectrum as the normalization function of EMSC scaling and input information used to calculate the reflectance contribution of the reference spectrum. Since the reference spectra were not a priori recognized, Bassan et al. Used artificial spectra, such as the mean spectra of the entire dataset or the spectra of pure protein substrates, as the "seed" reference spectra. After the initial pass through the algorithm, each corrected spectrum can be used in an iterative manner to correct all the spectra in the accessory pass. Eventually, requiring 1,000,000 calibration runs, a dataset of 1000 spectra will produce 1000 RMieS-EMSC corrected spectra, each of which will be used as an independent new reference spectrum for the next pass. Running this algorithm, called the "RMieS-EMSC" algorithm at a stable level of the calibrated output spectrum, requires several passes (~ 10) and computational time measured over several days.

Discussed in B. Bird, M. Miljkovic and M. Diem, "Two step resonant Mie scattering correction of infrared micro-spectral data: human lymph node tissue", J. Biophotonics, 3 (8-9) 597-608 (2010). As the RMieS-EMSC algorithm requires hours or days of computation time, a fast two-step method of removing scattering and dispersive linearity from the spectrum has been developed. This method, like the unscattered curve calculated by the van Hulst equation (see HC Van De Hulst, Light Scattering by Small Particles, Dover, Mineola, NY, (1981)), extends from the KK-transformation of the pure absorption spectrum to the Extended Multiplicative Signal. Fit multiple variance components obtained up to all spectra in the dataset by a procedure known as Correction (EMSC) (see A. Kohler et al., Appl. Spectrosc, 62, 259 (2008)), and correct all spectra without these disturbances. Involves reconstruction.

This algorithm avoids the iterative approach used in the RMieS-EMSC algorithm by using a non-polluting reference spectrum from the dataset. This non-contamination reference spectrum was found by performing a preliminary cluster analysis of the dataset and selecting the spectrum with the highest amide I frequency in each cluster as the "non-contamination" spectrum. Along the micro curves compressed for RMieS correction as described above, this spectrum was converted to purely reflective spectra through a numerical KK transform and used as an interference spectrum. This approach is fast, but only effective for datasets that contain some spectral classification.

But for spectral datasets with many tissue types, the extraction of non-pollutant spectra can be tedious. In addition, in such a situation, it is uncertain to ensure that all spectra in the data are fitted to most suitable interference spectra. In addition, this algorithm requires reference data for correction and works best with large datasets.

In view of the above, methods for analyzing biological samples enhanced by spectral images are still needed to provide a medical diagnosis. Furthermore, an improved dictionary that is based on a modified phase correction scheme, requires no input data, is computationally fast, and takes into account the various types of entangled spectral contributions that are frequently observed within the infrared spectrum of cells and tissues obtained microscopically. A treatment method is necessary.

The present invention has been made to solve the problems of the conventional methods for medical diagnosis, to analyze a biological sample by spectral imaging, and to provide a medical diagnosis based thereon.

One aspect of the present invention relates to a method of analyzing a biological sample by spectral imaging to provide a medical diagnosis. The method includes acquiring a spectral and visual image of a biological specimen and registering the image to detect cell abnormalities, cells before cancer, and cancer cells. This method overcomes the obstacles described above in that it eliminates the bias and distrust of diagnosis inherent in standard histopathology and other spectral methods.

Another aspect of the invention relates to a method for correcting confounding spectral contributions frequently observed in the infrared spectrum of microscopically acquired cells and tissues by performing phase corrections on the spectral data. This phase correction method can be used to correct various kinds of absorption spectra contaminated by reflective elements.

According to an aspect of the present invention, a method of analyzing a biological specimen by spectral imaging may include obtaining a spectral image of the biological specimen, acquiring a visual image of the biological specimen, and registering the visual and spectral images. It includes.

According to an aspect of the present invention, a method of developing a data store includes: identifying an area of a visual image indicative of a disease or condition, connecting the area of the visual image to spectral data corresponding to the area, and Storing the association between the corresponding statement or state.

A method for providing a medical diagnosis in accordance with an aspect of the present invention includes obtaining spectral data for a biological sample, comparing the spectral data for the biological sample with data in a reservoir related to a disease or condition, data in the reservoir and Determining a correlation between the spectral data for the biological sample and outputting a diagnosis associated with the determination.

A system for providing medical diagnostics in accordance with aspects of the present invention includes a processor, a user interface functioning through the processor, and a repository accessible by the processor, wherein spectral data of a biological sample is obtained, and spectral data of the biological sample is obtained. Is compared with data in a reservoir related to a disease or condition, a correlation between the data in the reservoir and spectral data for a biological sample is determined, and outputs a diagnosis associated with the determination.

A computer program product according to an aspect of the present invention includes a computer-used medium that stores control logic therein for the computer to provide a medical diagnosis. The control logic includes first computer readable program code for obtaining spectral data for a biological sample, second computer readable program code for comparing spectral data for the biological sample with data in a reservoir associated with a disease or condition, the reservoir A third computer readable program code for determining a correlation between the data and the spectral data for the biological sample, and a fourth computer readable program code for outputting a diagnosis associated with the determination.

The present invention provides a biological diagnosis by spectral imaging to provide a medical diagnosis comprising acquiring a spectral and visual image of a biological sample, and registering the image to detect cell abnormalities, cells before cancer, and cancer cells. A method for analyzing samples, which has the effect of eliminating the bias and distrust of diagnosis inherent in standard histopathology and other spectral methods.

In addition, a method for correcting the entanglement spectral contribution that is frequently observed in the infrared spectra of microscopically acquired cells and tissues includes performing phase corrections on the spectral data, which phase correction methods are various types contaminated by reflective elements. There is an effect that can correct the absorption spectrum of.

1 shows the contamination of the absorption pattern by the dispersion band shape generally observed in both SCP and SHP.
FIG. 2 shows broad, tall background features generally observed on the IR-MSP spectra of cells that contribute to Mie scattering by spherical particles.
3 is a flow chart illustrating a method of analyzing a biological sample by spectral imaging in accordance with an aspect of the present invention.
4 is a flow chart illustrating the steps of a method of obtaining a spectral image in accordance with an aspect of the present invention.
5 is a flow chart illustrating the steps of a method of preprocessing spectral data in accordance with aspects of the present invention.
6A shows a general spectrum superimposed on a linear background in accordance with aspects of the present invention.
6B shows an example of secondary derivative spectra in accordance with aspects of the present invention.
7 shows the real part of the interferogram according to aspects of the present invention.
8 shows that the phase angle which produces the maximum intensity after phase correction according to aspects of the present invention is considered an intact spectrum.
9A shows absorption spectra contaminated by the dispersing effect that mimics the baseline slope in accordance with aspects of the present invention.
9B shows that the imaginary part of the front FT exhibits a strong curve effect at the boundary of the spectrum, resulting in contamination of the corrected spectrum according to aspects of the present invention.
10A is H & E-based histopathology showing lymph nodes with confirmed breast cancer micrometastasis in the capsule in accordance with aspects of the present invention.
FIG. 10B shows data segmentation by Hierarchical Cluster Analysis (HCA) performed in the lymph node portion of FIG. 10A in accordance with aspects of the present invention. FIG.
FIG. 10C shows the peak frequencies of amide I vibrational bands in each spectrum in accordance with aspects of the present invention. FIG.
10D shows an image of the same lymph node portion of FIG. 10A after phase-calibration using RMieS correction in accordance with aspects of the present invention.
11A shows the results of HCA after phase-correction using the RMieS correction of FIG. 10D in accordance with aspects of the present invention.
11B is H & E-based histopathology of the lymph node portion of FIG. 11A in accordance with aspects of the present invention.
12A is a visual microscopic image of a colored cervical image portion.
12B is an infrared spectral image generated from hierarchical clustering analysis of an infrared dataset collected prior to tissue staining according to aspects of the present invention.
FIG. 13A is a visual microscopic image of a portion of an H & E-colored armpit lymph node in accordance with aspects of the present invention. FIG.
FIG. 13B is an infrared spectral image generated from an artificial neural network (ANN) analysis of an infrared dataset collected prior to tissue staining according to aspects of the present invention.
14A is a small cell lung cancer tissue visual image in accordance with aspects of the present invention.
FIG. 14B is an HCA-based spectral image of the tissue shown in FIG. 14A in accordance with aspects of the present invention. FIG.
14C is a registered image of the visual image of FIG. 14A and the spectral image of FIG. 14B in accordance with an aspect of the present invention.
14D is an example of a graphical user interface (GUI) for the registered image of FIG. 14C in accordance with an aspect of the present invention.
15A is a visual microscopic image of H & E-stained lymph node tissue sections according to aspects of the present invention.
FIG. 15B is a global digital staining image of the portion shown in FIG. 15A in accordance with aspects of the present invention, which distinguishes the membrane sac from the interior of the lymph node.
FIG. 15C is a diagnostic digitally pigmented image of the portion indicated in FIG. 15A in accordance with aspects of the present invention, distinguishing meninges, metastatic breast cancer, histocytes, activated B-lymphocytes, and T-lymphocytes.
16 is a schematic diagram of the relationship between global and diagnostic digital staining in accordance with aspects of the present invention.
17A is a visual image of H & E-colored tissue sections in axillary lymph nodes in accordance with aspects of the present invention.
17B is an SHP-based digitally colored region of breast cancer micrometastasis in accordance with aspects of the present invention.
17C is an SHP-based digitally colored region occupied by B-lymphocytes in accordance with aspects of the present invention.
17D is an SHP-based digitally colored region occupied by tissue cells according to aspects of the present invention.
18 illustrates the detection of individual cancer cells and a small population of cancer cells via SHP in accordance with aspects of the present invention.
19A shows a raw spectral data set consisting of cell spectra recorded from lung adenocarcinoma, small cell carcinoma, squamous cell carcinoma cells in accordance with aspects of the present invention.
19B shows a calibrated spectral data set consisting of cell spectra recorded from lung adenocarcinoma, small cell carcinoma and squamous cell carcinoma cells in accordance with aspects of the present invention.
19C shows standard spectra for lung adenocarcinoma, small cell carcinoma and squamous cell carcinoma cells in accordance with aspects of the present invention.
19D shows the KK transform spectrum calculated from the spectrum of FIG. 19C.
19E shows a PCA score plot of a multi-class data set before EMSC correction in accordance with aspects of the present invention.
19F shows a PCA score plot of a multiclass data set after EMSC correction in accordance with aspects of the present invention.
20A shows the mean absorbance spectra of lung adenocarcinoma, small cell carcinoma and squamous epithelial carcinoma in accordance with aspects of the present invention.
20B shows a second derivative spectrum of the absorbance spectrum shown in FIG. 20A in accordance with aspects of the present invention.
FIG. 21A shows four stitched fine R & E-colored images of 1 mm × 1 mm tissue area comprising adenocarcinoma, small cell carcinoma and squamous cell carcinoma cells in accordance with aspects of the present invention.
21B is 1350㎝ ^-1 of the four stitches of the raw infrared image recorded from the tissue area, shown in Figure 21A in accordance with the aspect of the invention a binary mask was made by analyzing the performance of the RCA which decreases rapidly from 900㎝ ^-1 spectral region Image.
FIG. 21C is a 6-group RCA image of minor corrected spectral data recorded from a region of diagnostic cell material in accordance with aspects of the present invention. FIG.
22 illustrates various features of a computer system that can be used with aspects of the present invention.
23 shows a computer system that can be used with aspects of the present invention.

Unless defined otherwise, all technical and scientific terms used herein generally have the same meaning as understood by one of ordinary skill in the art to which aspects of the present invention pertain. Similar or equivalent methods and materials described herein may be used for practice or testing, but appropriate methods and materials are described below. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety. In the case of a conflict, this specification, the definitions containing it, will control. In addition, the materials, methods, and examples are not limited to explanations.

One aspect of the present invention relates to a method of analyzing a biological sample by spectral imaging to provide a medical diagnosis. The biological sample may be a medical sample obtained by surgical method, biopsy and cultured sample. The method includes acquiring a spectral and visual image of a biological specimen and registering the image to detect cell abnormalities, cells before cancer, and cancer cells. The biological sample may include tissue or cell samples, but in some applications tissue samples are preferred. Such methods include inflammation, necrosis and apoptosis, as well as cancer and other diseases that are abnormal or include, but are not limited to, breast, uterus, kidney, testes, ovarian or prostate cancer, pulmonary cell carcinoma, non-small cell lung cancer, and melanoma. Identify non-impacted effects that are not limited to one.

One method according to aspects of the present invention overcomes the obstacles discussed above in that it eliminates or overall reduces the bias and distrust of diagnosis inherent in standard histopathology and other spectral methods. It is also possible to access a spectral database of tissue types generated by quantitative and reproducible measurements and analyzed by calibrated algorithms for classical histopathology. Through this method, for example, abnormal and cancerous cells can be found earlier than can be identified by related techniques, including standard histopathology or other spectral techniques.

The method according to aspects of the present invention is shown in the flowchart of FIG. 3. As indicated in FIG. 3, the method generally includes obtaining a biological portion 301, obtaining a spectral image of the biological portion 302, and obtaining a visual image of the same biological portion 303. And performing 304 an image registration. The registered image may be selectively trained (305) and a medical diagnosis may be obtained (306).

Creature part

Acquiring the biological portion 301 according to the exemplary method of the present invention shown in FIG. 3 means extraction of tissue or cellular material from an individual such as a human or an animal. Tissue sections can be obtained by methods including, but not limited to, core and punch biopsy and incision. Cellular material may be obtained by methods including swabbing (exfoliation), washing (lavages), and fine needle aspiration (FNA) It is not limited.

The portion of tissue to acquire spectral and visual images can be prepared from frozen or paraffin-containing tissue blocks, depending on the method used in standard histopathology. The portion can be mounted on a slide that can be used for spectral data acquisition and visual pathology. For example, the tissue may be mounted on an infrared transparent microscope slide composed of a material including, but not limited to, infrared reflective slides such as calcium fluoride (CaF ₂ ) or commercially available "low-e" slides. . After mounting, the paraffin-comprising sample may be subjected to deparaffinization.

Spectrum image

According to an aspect of the present invention, the step 302 of obtaining a spectral image of the biological part shown in FIG. 3 comprises the steps of acquiring 401 spectral data from the biological part, as shown in the flowchart of FIG. Step 402, performing multivariate analysis 403, and generating 404 a grayscale or pseudo-color image of the biological part. can do.

Spectral data

As indicated in FIG. 4, spectral data from the biological portion can be obtained in step 401. Spectral data from unstained biological samples, such as tissue samples, can be obtained to capture a snapshot of the chemical composition of the sample. The spectral data can be collected from the tissue portion of the pixel information, each pixel being about the size of the cell nucleus. Each pixel has a unique spectral pattern and can show small but repeated differences in the biochemical makeup of the tissue when the spectral patterns of the samples are compared.

The spectral data may be collected by methods including but not limited to infrared, Raman, visible, terahertz and fluorescence spectroscopy. Infrared spectroscopy may include, but is not limited to, attenuated total reflectance (ATR) and attenuated total reflectance Fourier transform infrared spectroscopy (ATR-FTIR). In general, infrared spectroscopy can be used for reasons of fingerprint sensitivity that may also be exhibited by Raman spectroscopy. Infrared spectroscopy can be used with large tissue parts and to provide datasets of a more manageable size than Raman spectroscopy. In addition, infrared spectroscopy data can better handle fully automatic data collection and interpretation. In addition, infrared spectroscopy can have the sensitivity and specificity necessary for the detection of various tissue structures and diagnoses of the disease.

In general, the intensity axis of the spectral data represents another suitable measure of absorption, reflection, emittance, scattering intensity or light power. The wavelength may be related to the actual wavelength, wavenumber, frequency or energy of electromagnetic radiation.

Acquisition of infrared data can be performed using currently available Fourier transform infrared imaging microspectrometers, tunable laser-based imaging devices or other technologies such as quantum cascades or non-linear optical devices. It can be performed using other devices of equivalent function based on the above. The acquisition of the spectral data using an adjustable laser is further described in US patent application Ser. No. 13 / 084,287 entitled "Tunable Laser-Based Infrared Imaging System and Method of Use Thereof", which is hereby incorporated by reference in its entirety.

According to one method according to an aspect of the present invention, a pathologist or technician can select a region of pigmented tissue portions, and in real time according to a hyperspectral dataset collected on the tissue prior to pigmentation. Spectroscopic-based evaluation can be taken. Spectral data can be collected for each of the pixels in a selected uncolored tissue sample. Each spectrum collected includes a fingerprint of the chemical component of each tissue pixel. The acquisition of spectral data is described in WO 2009/146425, which is hereby incorporated by reference in its entirety.

In general, the spectral data comprises a hyperspectral dataset, where N = nm, a structure comprising individual spectra or spectral vectors (absorption, emission, reflection, etc.), where n and m are each the x and The number of pixels in the y dimension. Each spectrum is associated with a unique pixel of the sample and can be located at its coordinates x and y, where 1 <x ≤ n, 1 <y ≤ m. Each vector has k intensity data points, which are usually equally spaced in the frequency or wave domain.

The pixel size of the spectral image can generally be chosen smaller than the typical cell size to achieve sub-cell resolution. The size may be determined by the diffraction limit of the light, which is generally about 5 μm to about 7 μm of infrared light. Thus, individual pixel infrared spectra of about 140 ² to about 200 ² can be collected for a 1 mm ² portion of tissue. N pixels of the spectrum "hypercube", their x and y coordinates and their intensity vectors (intensity versus wavelength) are each stored.

Preprocessing

Dependency of the spectral data in the form of pretreatment may aid in the separation of data related to the cellular material of interest and removal of entangled spectral functions. Referring to FIG. 4, once the spectral data is collected, it may be subjected to such preprocessing as specified in step 402.

Pretreatment can include the generation of a binary mask that separates and diagnoses from the non-diagnostic region of the sampling area to separate the cellular data of interest. A method of generating a binary mask is disclosed in WO 2009/146425, which is hereby incorporated by reference in its entirety.

The preprocessing method according to another aspect of the present invention allows for the correction of the scattered line shape of the observed absorption spectrum by an "phase correction" algorithm that adjusts the phase angle between them to optimize the separation of the real and imaginary parts of the spectrum. do. This computationally fast method is based on a calibrated phase correction scheme that requires no input data. Phase correction, in which an appropriate phase angle can be determined experimentally, is used for pre-processing the raw interferograms of FTIR and NMR spectroscopy (in the latter case the interferogram is usually referred to as "free induced decay, FID"). The method of this aspect of the invention differs from previous phase correction schemes in that it considers mitigating factors such as Mie, RMie and other effects due to anomalous dispersion of the refractive index. It can be applied retrospectively to datasets.

The preprocessing method of this aspect of the invention transforms the damaged spectrum into Fourier space through inverse FT transform. The inverse FT is the result in real and imaginary interferograms. The second half of each said interferogram is individually zero-filled and forward FT transformed. This process yields the real part of the spectrum with the same dispersion band shape and the imaginary part including the absorption line shape obtained through the numerical KK transformation. By recombining the real and imaginary parts with the correct phase angle and phase correction between them, a spectrum free of artifacts is obtained.

Since the phase for which correction of the contaminated spectrum is required cannot be determined experimentally and varies from spectrum to spectrum, the phase angle is determined using a stepwise manner between -90 degrees and 90 degrees in the user selection step. The "optimal" spectrum is determined by analysis of the peak position and intensity criteria that vary during phase correction. Mie scattering contributions with a wide range of elevations are not explicitly corrected for this apparent method, and they disappear by performing phase correction calculations on the second derivative spectrum, which represents a non-dispersed background.

According to an aspect of the present invention, the preprocessing step 402 of FIG. 4 includes selecting the spectral range 501 as shown in the flow chart of FIG. 5, and calculating 502 a secondary derivative of the spectrum. An inverse Fourier transform step 503 of the data, a zero-fill and forward Fourier transform step 504 of the interferogram, and a step 505 of correcting the phase of the real and imaginary part results of the spectrum. have.

Spectral range

In step 501, each spectrum of the hyperspectral dataset is preprocessed to select the most appropriate spectral range (fingerprint region). For example, this range can be from about 800 to about 1800 cm ⁻¹ , which includes XH (X: heavy atom with atomic number greater than 12) deformation modes as well as elongated heavy atoms. An example of a typical spectrum superimposed on a linear background is shown in FIG. 6A.

2nd order of the spectrum derivative

The secondary derivative of each spectrum is then calculated at step 502 of the flowchart of FIG. 5. The second derivative spectrum is derived from the original spectral vector by the second derivative of the intensity versus wave constant. Second-order derivative spectra can be calculated using the Savitzky-Golay sliding window algorithm, or they can be calculated in Fourier space by multiplying the interferogram by a properly truncated quadratic function.

Secondary derivative spectra can have the advantage of being independent of the baseline slope and include a slowly changing unscattered background. The secondary derivative spectra may almost completely lack baseline effects due to scattering and non-resonant non-scattering, but still include the RMieS effect. If desired, the secondary derivative spectrum can be vector normalized to compensate for changes in sample thickness. An example of the secondary derivative spectrum is shown in FIG. 6B.

Inverse Fourier Transform

In step 503 of the flowchart of FIG. 5, each spectrum of the data set is inverse Fourier transformed (FT). Inverse FT means the spectral shift from the intensity versus wave domain to the intensity versus phase domain. Since the FT routines only function with spectral vectors of integer square lengths, the spectra are interpolated or truncated to 512, 1024, or 2048 (NFT) data point lengths before the FT. The inverse FT yields the real (RE) and imaginary (IM) interferograms of the NFT / 2 points. The real part of this interferogram is shown in FIG.

Zero-Fill and Forward Fourier Transform

The second half of both the real and imaginary interferograms for each spectrum are then zero-filled in step 504. These zero-filled interferograms are then transformed forward Fourier to yield real and imaginary spectral components in the form of dispersion and absorption bands, respectively.

Phase correction

The real (RE) and imaginary (IM) portions resulting from Fourier analysis are then phase corrected, as shown in step 505 of the flowchart of FIG. 5. This yields phase shifted real RE 'and imaginary number IM', as specified in the formula below.

Is the phase angle.

Since the phase angle φ for the phase correction is unknown, the phase angle can be varied from -π / 2 <φ <π / 2 in a user defined unit, and a minimum residual scattering line spectrum can be selected. . The phase angle that produces the maximum intensity after phase correction can be considered an intact spectrum, as shown in FIG. 8. The thick record, indicated by the arrow and called the “original spectrum”, is the spectrum that is contaminated by the RMieS contribution. Thin records show how the spectrum changes during phase correction with various phase angles. The second thick record is the recovered spectrum, which is in good agreement with the unpolluted spectrum. As shown in FIG. 8, the optimal calibrated spectrum shows the highest amide I strength at about 1655 cm ⁻¹ . This peak position coincides with the position before the spectrum is contaminated.

The phase correction method according to aspects of the invention described in steps 501-505 works well for both absorption and derived spectra. Even this approach shows that if the absorption spectrum is contaminated by scattering effects that mimic the baseline slope, as shown schematically in Figure 9A, the imaginary part of the front FT shows a strong curve effect at the spectral boundary, as shown in Figure 9B. This also solves the problems that can arise when the absorption spectrum is used in that the corrected spectrum is contaminated. The use of a second derivative spectrum can eliminate this effect because the derivative eliminates the inclined background, and thus a spectrum free of artifacts can be obtained. Since subsequent analysis of spectral data-sets by hierarchical cluster analysis or other appropriate classification or diagnostic algorithm is performed on the secondary derivative spectrum anyway, it is likewise desirable to perform variance correction on the secondary derivative spectrum. Secondary derivative spectra show the inversion of the spectral peak signal. Thus, the phase angle is pursued to generate the largest negative intensity. The value of this approach can be seen from the artificially contaminated with Spectrum: pollution caused by the reflective element is always or not the contamination because it is possible to reduce the intensity "corrected" spectrum 1650 and 1660㎝ ^- amide between ¹ It will have the largest (negative) band strength in the I band.

Example 1-Working with a Phase Correction Algorithm

Examples of such phase correction algorithm operation are provided in FIGS. 10 and 11. This example is based on a dataset collected from human lymph node tissue parts. The lymph nodes shown by the black arrows in FIG. 10A have confirmed breast cancer micrometastasis in the meninges. This micrograph shows the high cytoplasm of activated lymphocyte regions, as well as the cell nuclei that distinguish in the cancerous region, and is represented by gray arrows. All of these sample heterogeneities contribute to large RMieS effects.

When data segmentation by hierarchical clustering analysis (HCA) was first performed on this exemplary lymph node portion, the image shown in FIG. 10B is obtained. Ten clusters are required to distinguish the cancerous tissue (dark green and yellow) from the mesothelial sac (red) and lymphocytes (the rest of the color), and the distinction of these tissue types is poor. In FIG. 10B, the capsular bag, shown in red, contains one or more spectral classes, which are combined into one cluster.

The difficulty of classifying this dataset can be measured by the inspection of FIG. 10C. This figure shows the peak frequency of the amide I vibrational band of each spectrum. Color scale of the drawing is the right side peak is about 1630 and the lymph nodes 1665㎝ body ^- indicating that occurred between the between the ^first, the capsule 1635 and 1665㎝ ^-1. Since it is well known that the amide I frequency for peptides and proteins occurs in the region of 1650 to 1660 cm ⁻¹ depending on the protein secondary structure, the diffusion of the amide I frequency is due to the dataset significantly contaminated by the RMieS effect. It is common. 10D shows an image of the same tissue part after phase-correction based on RMieS correction. The frequency deviation of the amide I peak in the lymph node body ranges from 1650 to 1654 cm ⁻¹ , and for the capsular bag, 1657 to 1665 cm ⁻¹ (fibro-connective proteins in the membrane capsule are mainly in the high amide I band position). It is known that it is composed of collagen, a protein known to show a).

The results of the HCA are then shown in FIG. 11. In FIG. 11A, cancerous tissue is shown in red; The contour of the cancerous region shown in FIG. 11B (this figure is the same as 10A) is in good agreement with H & E-based histopathology. The meninges appear in two different tissue classes (light blue and purple) with activated B-lymphocytes shown in bright green. Histiocytes and T-lymphocytes are shown as dark green, gray and blue areas. The region shown in FIG. 11A is in good agreement with visual histopathology, and the phase correction method discussed herein represents a significant quality improvement of the spectral histopathology method.

Advantages of the preprocessing according to aspects of the present invention over conventional spectral correction methods include that the method provides a fast execution time of about 5000 spectra / second and that no prior information on the dataset is required. The phase correction algorithm can also be integrated into spectral imaging and "digital staining" diagnostic routines for automatic cancer detection and diagnosis of SCP and SHP. In addition, phase correction greatly improves the quality of the image, which aids in image registration accuracy and diagnostic alignment and boundary representation.

In addition, the pretreatment method according to aspects of the present invention can be used to correct a wide range of absorption spectra contaminated by reflective elements. These contaminations are distorted band shapes, diffuse reflectance Fourier Transform Spectroscopy (DRIFTS), Attenuated Total Reflection (ATR), and other forms of mixing or dielectric susceptibility of real and imaginary parts of complex refractive indices. It occurs frequently in other types of spectroscopy, such as the spectroscopy of, and to a considerable extent as can be represented by Coherent Anti-Stokes Raman Spectroscopy (CARS).

Multivariate analysis

As described in step 403 of the flowchart of FIG. 4, multivariate analysis may be performed on the preprocessed spectral data to detect spectral differences. In certain multivariate analyzes, the spectra are grouped together based on similarity. The number of groups can be selected according to the level of differentiation required for a given biological sample. In general, the greater the number of groups, the more evident the details of the spectral image will be. If fewer details are required, fewer groups may be used. According to aspects of the present invention, the user can adjust the number of groups for the desired level of spectral differentiation.

For example, unsupervised methods such as HCA and principal component analysis, artificial neural networks (ANNs), hierarchical artificial neural networks (hANN), support vector machines ( Supervised methods such as, but not limited to, support vector machines (SVM) and / or “random forest” algorithms can be used. The unsupervised method is based on the similarity or difference of the datasets, respectively, and the dataset portion or clustering by these criteria, and does not require any information except for datasets for segmentation or clustering. Thus, this non-supervised method produces an image due to the natural similarity or dissimilarity (difference) of the dataset. On the other hand, supervised algorithms require a reference spectrum, such as, for example, a spectrum representing cancer, muscle, or bone, and classify the dataset according to certain similarity criteria of this reference spectrum.

HCA technology is disclosed in Bird (Bird et al., "Spectral detection of micro-metastates in lymph node histo-pathology", J. Biophoton. 2, No. 1-2, 37-46 (2009)) Are integrated. PCA is published in WO 2009/146425, which is hereby incorporated by reference in its entirety.

Examples of supervision methods for use in accordance with aspects of the present invention include Lasch et al. "Artificial neural networks as supervised techniques for FT-IR microspectroscopic imaging" J. Chemometrics 2006 (hereinafter "Lasch"); 20: 209-220, M. Miljkovic et al. "Label-free imaging of human cells: algorithms for image reconstruction of Raman hyperspectral datasets" ("Miljkovic"), Analyst, 2010, x, 1-13 and A. Dupuy et al. " Critical Review of Published Microarray Studies for Cancer Outcome and Guidelines on Statistical Analysis and Reporting ", JNCI, Vol. 99, Issue 2 | January 17, 2007 ("Dupuy"), each of which is hereby incorporated by reference in its entirety.

Grayscale or pseudo-color spectrum image

Similarly grouped data from the multivariate analysis can be assigned the same color code. As indicated at step 404 of the flowchart of FIG. 4, the grouped data can be used to construct "digital colored" grayscale or pseudo-color maps. Thus, the method can provide images of biological samples based solely or primarily on chemical information contained in the spectral data.

Examples of spectral images prepared after multivariate analysis by HCA are provided in FIGS. 12A and 12B. 12A is a visual microscopic image of the colored cervical image portion measured for 0.5 mm x 1 mm. Typical layers of squamous epithelium appear. 12B is a pseudo-color infrared spectral image constructed after multivariate analysis by HCA before staining of the tissue. This image is generated by mathematical correlation between spectra in the dataset and is based entirely on spectral similarity; Reference spectra are not provided to computer algorithms. As shown in FIG. 12B, the HCA spectral image can reproduce the tissue structure visible after proper staining (eg, H & E staining) using a standard microscope such as that shown in FIG. 12A. In addition, FIG. 12B shows a function not readily detectable in FIG. 12A, which includes infiltration by immune cells into the deposits of keratin in (a) and (b).

The construction of pseudo-color spectral images by HCA analysis is discussed in Bird.

Examples of spectral images prepared after analysis by ANN are provided in FIGS. 13A and 13B. 13A is a visual microscopic image of H & E-colored axillary lymph node portion. FIG. 13B is an infrared spectral image generated from ANN analysis of an infrared dataset collected before the tissue of FIG. 13A is colored.

Visual image

As represented by step 303 of FIG. 3, a visual image of the same biological part obtained in step 302 may be obtained. The biological sample applied to the slide of step 301 described above may be unstained or colored by well-known and suitable methods used in standard histopathology, such as by one or more H & E and / or IHC staining, and coverslips May be applied. Examples of visual images are shown in FIGS. 12A and 13A.

Visual images of histopathological samples can be obtained using standard visual microscopes such as those commonly used in pathology laboratories. The microscope can be connected to a high resolution digital camera that captures the field of view of the digital microscope. This real-time digital image is based on standard microscopic perspectives of colored tissue and represents tissue structure, cell morphology and pigmentation patterns. For example, the digital image may include many pixel tiles that are combined via image stitching for the generation of a photo. According to an aspect of the invention, the digital image used for analysis may comprise individual tiles or several tiles that are stitched and combined into a photograph. This digital image can be stored and displayed on a computer screen.

Registration of spectral and visual images

According to one method according to an aspect of the present invention, as shown in step 304 of the flowchart of FIG. 3, after the spectral and visual images are acquired, the visual image of the colored tissue is digitally colored grayscale or pseudo- It can be registered as a color spectrum image. In general, image registration is the process of transforming or matching different data sets in one coordinate system. Image registration involves spatially matching or transforming the first image to align with the second image. The image may include other types of data, and image registration may allow for matching or converting other types of data.

According to an aspect of the present invention, image registration may be performed in various ways. For example, a general coordinate system can be established for the visual and spectral images. If it is impossible or undesirable to establish a general coordinate system, the image can be registered by point mapping to import the image to align with another image. In point mapping, control points on both images are selected that identify the same feature or landmark in the image. Depending on the location of the control point, spatial mapping of the two images may be performed. For example, at least two control points can be used. In order to register the image, the control points of the visible image can be associated with control points corresponding to the spectral image and aligned together.

In one variation according to aspects of the present invention, the control point may be selected by placing reference marks on the slide containing the biological specimen. Reference numerals may include, but are not limited to, portions of materials including, but not limited to, inks, paints, and polyethylene. The reference numerals may have a suitable shape or size and may be located at the central portion, edge or side corner as long as they are in view. The reference sign can be added to the slide while the biological sample is being prepared. If spectral patterns, including but not limited to chemicals such as polyethylene and known biological substances, are used in the reference numerals, they may also be used as calibration marks to confirm the accuracy of the spectral data of the biological specimen. .

In another variation according to aspects of the present invention, a user, such as a pathologist, can select control points of the spectral and visual images. The user can select a control point based on his or her knowledge of the distinction of their visual or spectral image, including but not limited to edges and boundaries. Control points for biological images such as cells and tissues can be selected from biological features of the images. For example, these biological features may include clumps of cells, mitotic features, sample pores such as cords or nests of cells, alveolars, and bronchi. (sample voids) and irregular sample edges may include, but are not limited to. The selection of control points in the spectrum and visual image of the user may be stored in a repository used to provide training of correlation for personal and / or customized use. This approach allows subjective best practices to be integrated into the control point selection process.

In another variation according to aspects of the present invention, software-based recognition of the discriminating function in the spectral and visual images may be used for the selection of control points. The software may find a control point corresponding to the distinguishing function of at least one visual or spectral image. For example, the control points of a particular clustered area can be selected from the spectral image. The cluster pattern can be used to identify similar features of the visual image. The features of the two images can be aligned through deformation, rotation and scaling. For example, deformation, rotation, and scaling can also be automated or semi-automated by choosing a function selection and then developing relationships or model mappings. This automated process can provide an approximation of the mapping relationship that can then be resampled and transformed, for example for optimization registration. Resampling techniques include, but are not limited to, nearest neighbors, linear, and cubic interpolation.

Once the control points are aligned, the pixels of the spectral image with P ₁ (x ₁ , y ₁ ) coordinates can be aligned with the corresponding pixels of the visual image with P ₂ (x ₂ , y ₂ ) coordinates. This alignment process can be applied to all or selected portions of the pixels in the spectral and visual images. Once aligned, the pixels in each spectrum and visual image can be registered together. By this registration process, the pixel in each spectrum and visual image can be digitally connected with the pixel in the corresponding image. Since the method according to aspects of the invention allows the same biological sample to be spectroscopically and visually tested, the visual and spectral images can be registered accurately.

Identification marks, such as numeric codes and bar codes, can be added to the slide to verify that the correct sample is being used. The reference and identification code may be recognized by a computer displaying or storing a visual image of the biological specimen. The computer may include software for use in image registration.

Examples of image registration according to aspects of the present invention are shown in Figures 14A-14C. 14A is a visual image of a small cell lung cancer tissue sample, and FIG. 14B is a spectral image of the same tissue sample that received HCA. FIG. 14B contains spectral data of the upper right most of the visual image of FIG. 14A. If the visual image of FIG. 14A is registered with the spectral image of FIG. 14B, the results are shown in FIG. 14C. As shown in FIG. 14C, the circular portion comprising points and contours 1-4 that are easily visible in the spectral image of FIG. 14B closely matches the points and contours that are visible in the fine image of FIG. 14A.

Once the coordinates of the pixels in the spectral and visual images are registered, they can be stored digitally together. The whole image or partial image may be stored. For example, the diagnostic region may be digitally stored instead of an image of the entire sample. This can greatly reduce data storage requirements.

A user viewing a particular pixel region in either the spectral or visual image can immediately access the corresponding pixel region of another image. For example, a pathologist can select an area of the spectral image by clicking the mouse or adjusting the joystick and can view the corresponding area of the visual image registered to the spectral image. 14D is an illustration of a graphical user interface (GUI) for the registered image of FIG. 14C in accordance with an aspect of the present invention. The GUI shown in FIG. 14D allows to toggle between visual, spectral, and registered images, and to inspect specific portions of interest.

In addition, the pathologist's image movement or manipulation allows him / she to also access the corresponding portion of the other images registered. For example, if a pathologist magnifies a particular portion of the spectral image, he / she can access the same portion of the visual image at the same level of magnification.

Operational parameters of the visual microscope system, as well as microscope magnification, magnification change, and the like, may be stored in an instrument specific log file. The log file can later be used to select annotation records and corresponding spectral pixels for algorithm training. Thus, a pathologist can manipulate the spectral image, and later both the spectral image and the digital image registered to it are displayed at an appropriate magnification. This function can be useful, for example, because it allows a user to digitally store an image that has been manipulated and registered for later electronic transmission for remote viewing or remote viewing.

Image registration can be used in tissue parts with known diagnostics to extract the training spectrum in the training phase of the method according to aspects of the present invention. In the training phase, the visual image of the colored tissue may be registered as an unsupervised spectral image such as HCA. Image registration can also be used when diagnosing tissue parts. For example, a supervised spectral image of the tissue portion may be registered in a corresponding visual image. Therefore, the user may be diagnosed according to the point of the selected registered image.

Image registration in accordance with aspects of the present invention provides many advantages over earlier methods of biological sample analysis. For example, when analyzing biological materials, pathologists rely on spectral images, which reflect the very sensitive biochemical content of biological samples. For example, it provides significantly better accuracy than the related art in the detection of small abnormalities such as micrometastasis, pre-cancerous or cancer cells. Thus, the pathologist need not base his / her subjective observation of the biological sample visual image on his / her sample analysis. Thus, for example, the pathologist may simply study the spectral image and refer to the relevant portion of the registered visual image as needed to confirm his / her results.

In addition, since the image registration method according to an aspect of the present invention is based on the correlation of digital data, that is, the pixels of the spectrum and the visual image, Bird (Bird et al., "Spectral detection of micro-metastates in lymph node histo-pathology" ", J. Biophoton. 2, No. 1-2, 37-46 (2009)). Bird does not correlate digital data from the image, but instead relies on the user's ability to visually match the spectrum and visual image of adjacent tissue parts by physically coating the image. Thus, the image registration method according to aspects of the present invention provides a more accurate and reproducible diagnosis of abnormal or cancerous cells. This may help to provide an accurate diagnosis, for example, in the early stages of a disease where abnormalities and signs of cancer are difficult to detect.

training

As indicated at step 305 in the method provided in the flowchart of FIG. 3, a training set may optionally be developed. According to aspects of the present invention, a training set includes spectral data related to a particular disease or condition among others. The association of the spectral data of the training set with a disease or condition may be based on the correlation of classical pathology to spectral patterns depending on the morphological functions found in common pathological specimens. The diseases and conditions may include, but are not limited to, cellular abnormalities, inflammation (inflammation), infections (infections), pre-cancer and cancer.

According to one aspect of the present invention, in the training step, the training set identifies a region of a visual image comprising a disease or condition, correlates the region of the visual image with spectral data corresponding to the region, and corresponds to the spectral data. It can be developed by storing associations between diseases or conditions. The training set can then be stored in a repository, such as a database, and made available to machine learning algorithms that provide output and diagnostic algorithms derived from the training set. The diagnostic algorithm may also be stored in a repository, such as a database, for later use.

For example, the visual image of the tissue portion may be registered as a corresponding unsupervised spectral image as prepared by HCA. The user can then select a characteristic region of the visual image. This area may be classified and / or annotated by the user to specify a disease or condition. The spectral data underlying the feature region in the corresponding registered unsupervised spectral image may be classified and / or annotated for the disease or condition.

The spectral data, classified and / or annotated for a disease or condition, provides a training set in which a supervised analytical method such as ANN can be used for training. For example, this method is also described in Lasch, Miljkovic, and Dupuy. The trained supervisory analysis method may provide a diagnostic algorithm.

Disease or condition information is based on algorithms provided by the instrument, algorithms trained by the user, or a combination thereof. For example, the algorithm provided by the instrument can be improved by the user.

An advantage of the training step according to aspects of the present invention is that the registered image can receive the best possible training of the consensus-based "optimal criterion", which evaluates the spectral data by reproducible and repetitive criteria. Thus, after proper instrument verification and algorithm training, the method according to aspects of the present invention is more similar worldwide than relying on visually assigned criteria such as normal, atypical, low grade tumors, high grade tumors and cancer. Can produce results. The results for each cell can be represented by the overall results as appropriately adjusted numerical indexes or classification matchability. Thus, the method according to aspects of the present invention may have sensitivity and specificity for the detection of various biological structures and disease diagnoses.

The diagnostic limit of the training set may be limited by the extent to which the spectral data is classified for the disease or condition and / or annotated. As indicated above, this training set can be augmented by the user's own interests and expertise. For example, a user may prefer to color something else, such as one or more IHC colors over H & E coloring. In addition, algorithms can be trained to recognize certain conditions, such as, for example, breast cancer metastasis of armpit lymph nodes. The algorithm can be trained not to classify the occurrence of different normal tissue types such as meninges, B- and T-lymphocytes, so as to only display normal versus abnormal tissue types or binary outputs such as adenocarcinoma versus non-adenocarcinoma. The area of a particular tissue type or disease state obtained by the SHP can be represented by "digital staining" superimposed on the real time microscopic representation of the tissue part.

Diagnosis

As described in step 306 in the flowchart of FIG. 3, once the spectral and visual images have been registered, they can be used for the formation of a medical diagnosis. The diagnosis may include a disease or condition, including but not limited to cellular abnormalities, inflammation, infection, precancerous cancer, cancer, and gross anatomical features. In a method according to an aspect of the present invention, the spectral data of a biological specimen spectral image of an unknown disease or condition registered as its visual image may be input to a trained diagnostic algorithm, as described above. Based on the similarity to the training set used to prepare the diagnostic algorithm, the spectral data of the biological specimen can be correlated to the disease or condition. The disease or condition may be output through diagnosis.

For example, spectral data and visual images can be obtained from biological specimens of unknown disease or condition. The spectral data can be analyzed by an unsupervised method such as HCA, which can then be used with spatial reference data to prepare an unsupervised spectral image. This unsupervised spectral image can be registered with the visual image, as discussed above. The spectral data analyzed by the unsupervised method can then be input into a trained supervision algorithm. For example, the trained supervisory algorithm may be an ANN, as described in the training phase above. The output of the trained supervision algorithm may be spectral data comprising one or more labels corresponding to the classification and / or annotation of the disease or condition based on the training set.

To extract a diagnosis according to the label, the labeled spectral data can be used to prepare a supervised spectral image that can be registered with the visual image and / or the unsupervised spectral image of the biological specimen. For example, when the supervised spectral image is registered with the visual image and / or the unsupervised spectral image via a GUI, the user is placed on a label at the point of interest of the visual or unsupervised spectral image and at the point of the supervised spectral image. The point at which the corresponding disease or condition is provided may be selected. Alternatively, a user may request a software program to retrieve a registered image for a particular disease or condition, which may highlight portions of the visual, unsupervised spectrum and supervised spectrum images labeled with the particular disease or condition. . This preferably allows the user to get a diagnosis in real time while accessing very sensitive and finely acquired data and also allows the user to see the visual image he / she is familiar with.

The diagnosis may include a binary output, such as an output of the "yes / no" type, which indicates the presence or absence of a disease or condition. In addition, the diagnosis may include, but is not limited to, adjunctive reports, indexes, or relative composition ratios, such as the probability of matching a disease or condition.

According to an aspect of the method of the invention, gross architectural features of the tissue portion may be analyzed via spectral patterns to distinguish gross anatomical features that are not necessarily disease related. This process, known as global digital staining (GDS), can utilize a combination of supervised and unsupervised multivariate methods. GDS is used to analyze anatomical features, including but not limited to glandular and squamous epithelium, endothelial, connective tissue, bone and fatty tissue. Can be used.

In GDS, supervised diagnostic algorithms can be built from training datasets containing several samples of a given disease from different patients. Each individual tissue portion of the patient can be analyzed using spectral image data acquisition, preprocessing of the resulting dataset and interpretation by an unsupervised algorithm such as HCA, as described above. The HCA image can be registered in the corresponding colored tissue and annotated by a pathologist. This annotation step, shown in Figures 15A-C, allows for the extraction of spectra corresponding to tissue types or stages of disease and typical signs of conditions or other desired features. As a result, the typical spectrum along with annotated medical diagnostics can then be used to train supervisory algorithms such as ANN that are particularly suited to detecting features trained to recognize.

According to the GDS method, the sample can be stained using classical stains or immunohistochemical agents. When the pathologist receives the colored sample and examines it using a computerized imaging microscope, the spectral results can be used in a computer controlling the visual microscope. The pathologist can select a tissue point on the sample and receive a spectroscopic-based diagnosis. This diagnosis can apply a grayscale or pseudo-color image to the visual image representing all areas having the same spectral diagnostic classification.

15A is a visual microscopic image of H & E-stained lymph node tissue sections. FIG. 15B shows a typical illustration of global discrimination of total anatomical features, such as the membrane sac and the interior of a lymph node. FIG. 15B is a global digital staining image of the portion shown in FIG. 15A, which distinguishes the membrane sac from the inside of the lymph node.

The total anatomical feature area registered in the corresponding visual image can be selected for analysis according to more sophisticated criteria of the spectral pattern dataset. The next level of diagnosis can be based on a diagnostic marker digital staining (DMDS) database, which can only be based on SHP results, for example, or by using the results of immunohistochemistry (IHC). It may include spectral information that may be collected. For example, by using many diagnostic databases to scan selected areas, epithelial tissue portions can be selected for analysis of the presence of spectral patterns indicative of abnormality and / or cancer. An example of this approach is shown schematically in FIG. 15C, which utilizes the full discriminatory power of SHP and utilizes lymph node internal tissue features as can only be used after immuno-histochemical staining in classical histopathology. Calculate details (cancer, lymphocytes, etc.) FIG. 15C is a DMDS image of the portion shown in FIG. 15A, which distinguishes the meninges, metastatic breast cancer, histiocytes, activated B-lymphocytes and T lymphocytes.

The relationship between the GDS and the DMDS is shown by the horizontal progression indicated by dark blue and purple, respectively, in the schematic diagram of FIG. 16. Both GDS and DMDS are based on spectral data but may include other information such as IHC data. The actual diagnosis may be performed by the same or similarly trained diagnostic algorithm, such as hANN. These hANNs can first analyze tissue portions for total anatomical features that detect large changes in the dataset of patterns collected for the tissue (dark blue tracks). “Diagnostic Factor” analysis can then be performed by hANN using some of the spectral information indicated in the purple track. For example, a multi-layer algorithm in binary format can be implemented. In order to reach each diagnosis, both GDS and DMDS can use different database details, indicated in Figure 16 as Total Tissue Database and Diagnostic Tissue Database, and the result can be superimposed on the colored image after proper image registration.

According to an example of a method according to an aspect of the present invention, the pathologist may provide certain inputs to ensure that an accurate diagnosis is made. For example, the pathologist can visually check the quality of the colored image. The pathologist can also perform selective interrogation to alter the magnification or field of view of the sample.

The method according to aspects of the invention may be practiced by a pathologist looking at the biological specimen and performing image registration. Alternatively, the method can be performed remotely because the registered image contains digital data which can be transmitted electronically.

The method can be shown by the following non-limiting example.

Example 2-lymph node part

FIG. 17 shows a visual image of the H & E-colored armpit lymph node area measured 1 mm × 1 mm and includes breast cancer micrometastasis on the top of the left quadrant. 17B is an SHP-based digitally colored region of breast cancer micrometastasis. For example, by clicking to select using a mouse-controlled cursor in the general area of microtransition, the area identified as cancer by SHP is highlighted in red as shown in FIG. 17B. 17C is an SHP-based digitally colored region occupied by B-lymphocytes. By pointing towards the lower right corner, the area occupied by the B-lymphocytes is indicated in light blue as shown in FIG. 17C. 17D is an SHP-based digitally colored area showing the area occupied by tissue cells, which is identified by arrows.

Because the SHP-based digital coloration is based on a trained and verified repository or database containing spectra and diagnostics, the given digital coloration can be applied directly to the diagnostic category, such as in the case of "metastatic breast cancer" Lt; / RTI &gt; Although diagnostic analysis can be performed by SHP, the system can first be used by a pathologist as a supplement or supplement. As an additional tool, for example, the output may be a match probability rather than a binary report. 18 shows the detection of SHP and cancer cells of individual and small populations.

Example 3-Fine Needle Aspiration in the Lung Part ( fine needle aspirate ) Sample

The sample portion is cut from formalin fixed paraffin embedded cell blocks prepared by microneedle aspiration of a suspect population located in the lung. Cell blocks are selected based on criteria for which previous histological analysis identified adenocarcinoma, small cell carcinoma (SCC) or squamous cell carcinoma of the lung. Samples are cut using a microtome to provide a thickness of about 5 μm and then mounted on low-e microscope slides (Kevley Technologies, Ohio, USA). The portion is then deparaffinized using standard protocols. After spectroscopic data collection, the tissue portions are hematoxylin and eosin (H & E) stained for morphological interpretation by histopathologists.

Perkin Elmer Spectrum 1 / Spotlight 400 Imaging Spectrometer (Perkin Elmer Corp, Shelton, CT, USA) is used in this study. Infrared micro-spectral images were captured before Norton-Beer apodization (see, eg, Naylor, et al. J Opt. Soc. Am., A24: 3644-3648 (2007)) and Fourier transform With a pixel resolution of 6.25 μm x 6.25 μm, a spectral resolution of 4 cm ⁻¹ and a joint addition of 8 interferograms from the 1 mm x 1 mm tissue area in the transflection mode Is recorded. Appropriate background spectra are collected outside the sample area in proportion to a single beam spectrum. As a result, the ratio spectra are then converted into absorbance. Each 1 mm x 1 mm infrared image contains 160 x 160 or 25,600 spectra.

Initially, raw infrared micro-spectral data sets are imported and processed using software written in Matlab (version R2009a, Mathworks, Natick, MA, USA). Spectrum quality tests are performed to remove all spectra that are recorded from areas where tissue is not present or that show poor signals for noise. All spectra that pass the test are then baseline off-set normalize (minimum reduction of absorbance intensity in the entire spectral vector), converted to secondary derivatives (Savitzy-Golay algorithm (see. ... eg, Savitzky, et al Anal Chem, 36: 1627 (1964)), 13 smoothing points), 1350㎝ -1 - 900㎝ -1 is cut so as to include only the intensity value recorded in the spectral domain, the last vector Normalize to normal.

The processed data set is HCA performed using Euclidean distance to define software system and spectral similarity, and Ward's algorithm for clustering (see, eg, Ward, J Am. Stat. Assoc., 58: 236 (1963). Pseudo-colored cluster images describing pixel cluster membership are then aggregated and compared directly with H & E images captured from the same sample. HCA images between 2 and 15 clusters describing different clustering structures are collected by cutting the HCA dendrogram computed at different levels. This cluster image is then provided to collaborative pathologists who identify clustering structures that are well replicated in morphological interpretation with the formation of H & E-stained tissue.

The infrared spectrum is a LambertBeer law (corrected by a sub-spatial model version of the EMSC for fundamental baseline shifts, unknown signal intensity changes, peak position shifts, or non-scattering and reflection contributions to the recorded spectra). see B. Bird, M. Miljkovic and M. Diem, "Two step resonant Mie scattering correction of infrared micro-spectral data: human lymph node tissue", J. Biophotonics, 3 (8-9) 597-608 (2010)) Not contaminated from or contaminated by general characteristics that obey. Initially, 1000 recorded spectra for each cancer type are collected into a separate data set from the infrared images shown in FIGS. 19A-19F.

This data set is then searched for minimum scattering contributions and spectra, as shown in FIGS. 19A and 19B, and averages for each cancer type are calculated for each signal type to increase the signal to noise, and each cell The KK transform is calculated for the type. 19A shows a raw spectral data set consisting of cell spectra recorded from lung adenocarcinoma, small cell carcinoma and squamous cell carcinoma cells. 19B shows a calibrated spectral data set consisting of cell spectra recorded from lung adenocarcinoma, small cell carcinoma, and squamous cell carcinoma cells, respectively. 19C shows standard spectra for lung adenocarcinoma, small cell carcinoma and squamous cell carcinoma.

The sub space model for non-scattering contributions is the Van de Hulst approximation formula (see, eg, Brussard, et al., Rev. Mod. Phys., 34: 507 (1962)). Using to calculate the 340 non-scattering curve to form a nuclear region radius range of 6 μm-40 μm and a refractive index range of 1.1-1.5. The first 10 principal components accounting for more than 95% of the differences made up of these scattering curves are used in addition to the KK transformation for each cancer type, such as interference of the first stage EMSC correction of the data set. The EMSC calculation takes approximately 1 second per 1000 spectra. 19D shows the KK transform spectrum calculated from the spectrum of FIG. 19C. 19E shows a PCA score plot of a multiclass data set before EMSC correction. 19F shows a PCA score plot of a multiclass data set after EMSC correction. This assay is 1800㎝ ^-1 - is performed on the normalized vector from 900㎝ ^-1 spectral region.

20A shows the average absorbance spectra of lung adenocarcinoma, small cell carcinoma and squamous carcinoma, respectively. This is calculated from the 1000 minor corrected cell spectra of each cell type. FIG. 20B shows the secondary derivative spectrum of the absorbance spectrum shown in FIG. 20A. In general, adenocarcinoma and squamous cell carcinoma have similar spectral profiles in the low wavenumber region of the spectrum. However, the squamous cell carcinoma exhibits a generally low wavenumber shoulder for the amide I band, which is observed in spectral data recorded from squamous cell carcinoma of the oral cavity (Papamarkakis, et al. (2010), Lab. Invest., 90: 589-598). The small cell carcinoma displays very strong symmetrical and asymmetrical phosphate bands that are slightly shifted to high wavenumbers and exhibits a strong contribution of phospholipids to the observed spectrum.

Since most of the sample area consists of blood and non-diagnostic materials, the data contain only diagnostic material and are pre-processed to correct scattering contribution. In addition, the HCA is used to generate a binary mask and finally classify the data. This result is shown in FIGS. 21A-21C. 21A shows four stitched fine R & E-stained images of 1 mm × 1 mm tissue area, each containing adenocarcinoma, small cell carcinoma and squamous cell carcinoma cells. 21B is 1350㎝ ^-1 of the four stitches of the raw infrared image recorded from the tissue region indicated in Figure 21A - a binary mask image made by the performance analysis of the RCA which decreases rapidly from 900㎝ ^-1 spectral region. Diagnostic cell material and areas of blood cells are indicated. 21C is a 6-group RCA image of a small amount of modified spectral data recorded from a region of diagnostic cell material. This assay is 1800㎝ ^-1 - are performed on 900㎝ ^-1 spectral region. Areas of squamous cell carcinoma, adenocarcinoma, small cell carcinoma and various reverse desmoplastic tissue responses are shown. Otherwise, this process may be replaced with a supervisory algorithm such as ANN.

The results presented in the examples above show that analysis of raw measured spectral data enables differentiation of SCC and non-small cell carcinoma (NSCC). After the raw measured spectrum is corrected for scattering contribution, the two subtypes of NSCC, adenocarcinoma and squamous cell carcinoma according to the method according to the aspect of the present invention, are clearly distinguished. Thus, these examples provide strong evidence that this spectral imaging method can be used to identify and accurately classify three major types of lung cancer.

22 illustrates various features of an example computer system 100 that may be used with methods such as image registration and training in accordance with aspects of the present invention. As shown in FIG. 22, the computer system 100 may be a personal computer (PC), a minicomputer, a mainframe computer, a microcomputer, a telephone device, a PDA. It may be used by the requestor 101 via a terminal 102, such as a personal digital assistant or other device with input capabilities. The server module may include, for example, a PC, minicomputer, mainframe computer, microcomputer, or other device having a processor and storage of data or having access to the data storage. For example, the server module 106 may be connected with an accessible repository of disease based data for use in diagnosis.

For example, information related to the diagnosis may be transmitted between the analyst 101 and the server module 106 via a network 110 such as the Internet. Communication may be via, for example, couplings 111, 113, such as wired, wireless or fiberoptic links.

The invention may be implemented using hardware, software or a combination thereof, and may be implemented in one or more computer systems or other processing systems. In one variation, aspects of the present invention refer to one or more computer systems capable of performing the functions described herein. An example of such a computer system 200 is shown in FIG. 23.

Computer system 200 includes one or more processing devices, such as processor 204. The processor 204 is connected with a communication infrastructure (eg, communication bus, cross-over bar or network) 206. Various software aspects are described in terms of this exemplary computer system. After reading this description, it will be apparent to those skilled in the art how to implement aspects of the present invention using other computer systems and / or architectures.

The computer system 200 includes a display interface 202 for conveying graphics, text and other data in the communication infrastructure 206 (or frame buffer, not shown) for display on the display unit 230. It may include. Computer system 200 also includes main memory 208, preferably random access memory (RAM), and may also include secondary memory 210. The secondary memory 210 may be a hard disk drive 212 and / or represented by, for example, a floppy disk drive, a magnetic tape drive, an optical disk drive, or the like. It may include a removable storage drive 214. The removable storage drive 214 reads from and / or writes to the removable storage unit 218 in a well known manner. Removable storage unit 218 represents a floppy disk, magnetic tape, optical disk, and the like, which is read from and written to removable storage drive 214. The removable storage unit 218 includes a computer usable storage medium having computer software and / or data stored therein, which will be useful later.

In alternative variations, secondary memory 210 may include other similar devices to permit computer programs or other instructions to be loaded into computer system 200. For example, such a device may include a removable storage unit 222 and an interface 220. Such examples include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (EPROM or PROM) and associated sockets, and other removable storage units 222 and interfaces 220. ), Which allows software and data to be transferred from the removable storage unit 222 to the computer system 200.

Computer system 200 may also include a communication interface 224. The communication interface 224 allows software and data to be transferred between the computer system 200 and the external device. Examples of communication interface 224 may include a modem, a network interface (such as an Ethernet card), a communication port, a Personal Computer Memory Card International Association (PCMCIA) slot and a card, and the like. have. Software and data transmitted over communication interface 224 are in the form of signals 228, which may be electronic, electromagnetic, optical or other signals that may be received by communication interface 224. This signal 228 is provided to the communication interface 224 via the communication path 226 (eg, channel). This path 226 carries the signal 228 and may be implemented using wires or cables, fiber optics, telephone lines, wireless links, radio frequency (RF) links, and / or other communication channels. In this document, the terms “computer program medium” and “computer usable medium” are used for general reference to media such as removable storage drive 214, hard disk 212 installed in hard disk drive, and signal 228. . This computer program product provides software to the computer system 200. Aspects of the present invention disclose such a computer program product.

Computer programs (also referred to as computer control logic) are stored in main memory 208 and / or secondary memory 210. The computer program may also be received via the communication interface 224. When executed as discussed herein, such a computer program enables the computer system 200 to perform functions in accordance with aspects of the present invention. In particular, the computer program, when executed, allows the processor 204 to perform such functions. Thus, such a computer program represents a controller of the computer system 200.

While various aspects of the present invention may be implemented using software, the software may be stored in a computer program product using the use of a removable storage drive 214, a hard drive 212, or a communication interface 224, and the computer system ( 200). When executed by the processor 204, the control logic (software) causes the processor 204 to perform a function as described herein. Another aspect of the invention of the change is implemented primarily in hardware using hardware elements such as application specific integrated circuits (ASICs). Implementation of a hardware state machine to perform the functions described herein will be apparent to those of ordinary skill in the art.

In another variation, aspects of the invention are implemented using a combination of both hardware and software.

Claims (34)

Obtaining a spectral image of biological specimens;
Obtaining a visual image of the biological specimen; And
Registering the visual and spectral images; Method of analyzing a biological sample by spectral imaging comprising a.
The method of claim 1,
Storing the registered visual and spectral images; Method of analyzing a biological specimen by spectral imaging further comprising.
The method of claim 1,
Wherein said biological sample comprises cells or tissue.
The method of claim 1, wherein the obtaining of the spectral image of the biological specimen comprises:
Obtaining spectral data from the biological sample;
Performing preprocessing of the spectral data;
Performing a multivariate analysis of the spectral data; And
Preparing a grayscale or pseudo-color spectral image; Method of analyzing a biological sample by spectral imaging comprising a.
The method of claim 4, wherein obtaining spectral data from the biological sample comprises:
Performing infrared spectroscopy, Raman spectroscopy, visible, terahertz, or fluorescence spectroscopy on the biological specimen. How to analyze a biological sample.
The method of claim 4, wherein obtaining spectral data from the biological sample comprises:
And performing infrared spectroscopy on the biological sample.
The preprocessing of the spectral data according to claim 4,
Selecting a spectral range;
Calculating a secondary derivative;
Performing a Reverse Fourier transformation;
Performing zero-filling and inverse Fourier transform; And
Performing phase correction; Method of analyzing a biological sample by spectral imaging comprising a.
The preprocessing of the spectral data according to claim 4,
Subjecting the spectral data to a binary mask.
The multivariate analysis of the spectral data according to claim 4,
A method of analyzing a biological specimen by spectral imaging, comprising performing an unsupervised analysis. The method of claim 9, wherein performing the unsupervised analysis comprises:
A method of analyzing a biological sample by spectral imaging, comprising performing hierarchical cluster analysis (HCA) or principal component analysis (PCA).
The multivariate analysis of the spectral data according to claim 4,
A method of analyzing a biological specimen by spectral imaging, comprising performing an analysis of said data via a supervised algorithm.
The method of claim 11, wherein performing the analysis of the data through the supervisory algorithm,
Machine learning algorithms selected from the group consisting of artificial neural networks (ANNs), hierarchical artificial neural networks (hANN), support vector machines (SVM), and random forest algorithms and analyzing the data via a machine learning algorithm.
The method of claim 1, wherein the obtaining of the visual image of the biological specimen comprises:
Obtaining a digital image of the biological sample.
The method of claim 2, wherein registering the visual image and the spectral image comprises:
Aligning control points corresponding to the spectral image and the visual image.
The method of claim 1,
A method of analyzing a biological sample by spectral imaging, further comprising providing a medical diagnosis.
The method of claim 15, wherein providing the medical diagnosis comprises:
Obtaining a selected region of the spectral image;
Comparing the data in the selection region with data in a repository related to a disease or condition;
Determining a correlation between data in the storage and data in a selection region; And
Outputting a diagnosis associated with the determination; Method of analyzing a biological sample by spectral imaging comprising a.
17. The method of claim 16,
Data in the storage is obtained for a plurality of images, and
Wherein each of the plurality of images of the reservoir is related to a disease or condition.
Identifying an area of the visual image indicating the disease or condition;
Connecting an area of the visual image to spectral data corresponding to the area; And
Storing a link between the spectral data and the disease or condition in question; How to develop a data store that includes.
Obtaining spectroscopic data for the biological sample;
Comparing the spectral data for the biological sample with data in a reservoir related to a disease or condition;
Determining a correlation between data in the reservoir and spectral data for a biological sample; And
Outputting a diagnosis associated with the determination; Method for providing a medical diagnosis comprising a.
20. The method of claim 19,
Data in the storage is obtained from a plurality of images, and
Wherein each of the plurality of images of the reservoir is related to a disease or condition.
The method of claim 19, wherein the outputting the diagnosis,
Displaying the diagnosis on a computer screen.
The method of claim 19, wherein the outputting the diagnosis,
Electronically storing the diagnosis.
A processor;
A user interface functioning through the processor; And
Storage accessible by the processor; Including;
The spectral data of the biological specimen is obtained,
The spectral data of the biological sample is compared with the data from the reservoir related to the disease or condition,
A correlation between the data in the reservoir and the spectral data for the biological sample is determined, and
And output a diagnosis associated with the determination.
24. The method of claim 23,
And the processor is located on a terminal.
25. The method of claim 24,
The terminal is from a group consisting of a personal computer, a minicomputer, a main frame computer, a microcomputer, a hand held device and a telephonic device. A system for providing a medical diagnosis, characterized in that selected.
24. The method of claim 23,
And the processor is located on a server.
27. The method of claim 26,
And wherein said server is selected from the group consisting of personal computer, minicomputer, microcomputer and mainframe computer.
27. The method of claim 26,
The server is connected to a network.
29. The method of claim 28,
And said network is the Internet.
29. The method of claim 28,
And the server is coupled to the network via a coupling.
31. The method of claim 30,
The coupling is selected from the group consisting of a wired connection, a wireless connection and a fiber optic connection.
24. The method of claim 23,
And the repository is located on a server.
The method of claim 32,
The server is connected to a network.
A computer program product comprising a computer-used medium for storing control logic therein to cause a computer to provide a medical diagnosis, the control logic including:
First computer readable program code for obtaining spectroscopic data on the biological sample;
Second computer readable program code for comparing spectral data for the biological sample with data in a reservoir associated with the disease or condition;
Third computer readable program code for determining a correlation between data in said reservoir and spectral data for a biological sample; And
Fourth computer readable program code for outputting a diagnosis related to the determination.

Images