WO2012097189A1

WO2012097189A1 - Systems and methods for multi-protocol registration and tissue classification using local morphologic scale (lms)

Info

Publication number: WO2012097189A1
Application number: PCT/US2012/021133
Authority: WO
Inventors: Anant Madabhushi; Andrew Janowczyk; Sharat Chandran
Original assignee: Rutgers State University of New Jersey
Current assignee: Rutgers State University of New Jersey
Priority date: 2011-01-13
Filing date: 2012-01-12
Publication date: 2012-07-19
Anticipated expiration: 2013-07-13

Abstract

The described invention provides systems and methods for classifying and registering a region of interest (ROI) in an image by utilizing local morphologic scale (LMS), which is not constrained by a pre-defined shape criterion.

Description

SYSTEMS AND METHODS FOR MULTI-PROTOCOL REGISTRATION AND TISSUE CLASSIFICATION USING LOCAL MORPHOLOGIC SCALE (LMS)

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This Application claims the benefit of priority to U.S. Provisional Application No. 61/432,493 (filed January 13, 2011) entitled "LOCAL MORPHOLOGIC SCALE:

APPLICATION TO MULTI-PROTOCOL REGISTRATION AND TISSUE

CLASSIFICATION," and U.S. Provisional Application No. 61/435,622 (filed January 24, 2011) entitled "LOCAL MORPHOLOGIC SCALE: APPLICATION TO MULTI-PROTOCOL REGISTRATION AND TISSUE CLASSIFICATION." The entire content of each of these applications is incorporated by reference in its entirety.

FIELD OF INVENTION

[0002] The described invention is related to applying local morphological scale (LMS) to image processing applications, including, but not limited to, image registration, image segmentation, and image classification.

BACKGROUND OF THE INVENTION

[0003] The notion of scale in the context of image processing has been routinely employed over the last few decades to facilitate multi-resolution feature analysis based on the assumption that certain pertinent image features are only discernible at certain image scales and hence a spectrum of image resolutions needs to be considered for object recognition. Multi-scale approaches, such as scale-space (Witkin, A., IJCAI, 1019-1022, 1983) and hierarchical pyramids (Burt, P. Comput. Graphics Image Processing 16, 25-51, 1981), envisioned image processing operations being applied on a single image at varying levels of resolution, with homogeneous regions being operated on at a lower resolution, and more heterogeneous regions being examined at higher resolutions. A limitation of these multi-scale techniques is that an "optimal" image resolution needs to be selected from within the image pyramid (Burt, P. Comput. Graphics Image Processing 16, 25-51, 1981). Additionally, some approaches (Doyle, S, et al, Detecting Prostatic Adenocarcinoma from Digitized Histology Using a Multi-Scale, Hierarchical Classification Approach. EMBS, 1 :4759-62, 2006) might require selection of multiple image scales for classification of a single image region.

[0004] To overcome these difficulties, the idea of locally adaptive scale emerged

(Lindeberg, T., Journal of Applied Statistics, 21(1), 225-270, 1994). The concept of local scale was introduced to characterize varying levels of image detail so that localized image processing tasks could be performed, yielding an optimal result globally. Pizer et al. (Pizer, Eberly, et al., Zoom invariant vision of gural shape: the mathematics of cores. Computer Vision and Image Understanding 69 ,1998 ) suggested that having a locally adaptive definition of scale was necessary even for moderately complex detailed images. By quantifying these images details, an adaptive local scale image could encode implicit information present in the image intensity values. Saha and Udupa introduced the notion of ball-scale (Saha, P. et al., Computer Vision and Image Understanding 77(2), 145-174, 2000), which, at every spatial location, was defined as the value corresponding to the radius of the largest ball encompassing all locations neighboring the location under consideration and satisfying some pre-defined homogeneity criterion. Saha et al., (Comput. Vis. Image Underst. 99(3), 384-413, 2005) extended the ball-scale idea to a tensor- scale (t-scale), where the t-scale was defined as the largest ellipse at every spatial location where the pixels within the ellipse satisfied some predefined homogeneity criterion. The shape constraints of both b-scale and t-scale were overcome by Madabhushi and Udupa with the introduction of generalized scale (g-scale). (Madabhushi, A. et al, Comput. Vis. Image Underst. 101(2), 100-121, 2006). G-scale is defined as the largest connected set associated with every spatial location, such that all spatial locations in this set satisfy a predefined homogeneity criterion.

[0005] Locally adaptive scale has seen application in a variety of image processing tasks including magnetic resonance imaging (MRI) bias field correction (Madabhushi, A. and Udupa, J., Med Phys 33(9), 3426-3434, 2006), image segmentation (Madabhushi, A. et al, Comput. Vis. Image Underst. 101(2), 100-121, 2006), image registration (Laaszlo, N. et al., IEEE Trans. Med. Imaging 22(2), 228-237, 2003), and image coding (Hontsch, I. and Karam, L., IEEE

Transactions on Image Processing 9(9), 1472-1483, 2000). The common thread between these local scale concepts was that they were defined based on some homogeneity criterion linking the pixels neighboring the spatial location under consideration. For example, as shown in Figure 1, both the b-scale ((a)-(d)) and g-scale ((e)-(h)) representations for a specific spatial location (located in the center of the image) attempt to identify the largest ball and set of pixels, respectively, that is homogeneous with respect to the pixel under consideration. The initial motivation of both b-scale, and g-scale was from the perspective of noise filtering and bias field correction (Madabhushi. et al, Comput. Vis. Image Underst. 101(2), 100-121, 2006), image processing operations that warranted identification of locally connected homogeneous regions. Consequently, the scale definitions are not necessarily optimized to capture local heterogeneity, except as a very small ball (in the case of b-scale) or set (in the case of g-scale) of homogeneous pixels in image regions with significant complexity. However, in the context of biological images (Kim, J. et al, Eur J Radiol 52(1), 78-83, 2004), such as in microscopy applications or histopathology imagery, the objective is often to identify local regions of heterogeneity (e.g. cancer nuclei, lymphocytes), whereas larger homogeneous regions (e.g. benign stroma) can be less interesting or informative from a diagnostic or prognostic perspective. (Madabhushi, A. et al, Clin Chem Lab Med 48(7), 989-998, 2010; Madabhushi, A. et al, Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multimodal data," Computerized Medical Imaging and Graphics, 2011). Additionally, the shape or architecture of this local heterogeneity may be highly predictive of a pathologic process (e.g., architectural arrangement of nuclei and glands in prostate cancer reflects the Gleason grade and hence aggressiveness of the disease (Gleason, D., Cancer Chemother. Rep (50), 125-128, 1966)).

[0006] It is important to note, however, that the b-, t-, and g-scale formulations were not devised with the purpose of object classification in mind. While both the b- and t- scale definitions assign a feature vector (ball radius and parameters of ellipse respectively) to each image location, which potentially could be used to perform pixel-level classification, it is not clear whether these b-, t- scale related parameters are discriminatory enough. For instance, for an ovarian cancer (OCa) biopsy image (Figure 1), the b-scale at two different locations in the image (Figures 1 (c), (d)), are identical in spite of significantly different local structural attributes. Similarly, the g-scale representations at 4 different image locations from within the OCa biopsy image (Figure l(e)-(h)) does not appear to yield a signature amenable to object or pixel level classification. Therefore, for images where the most interesting information is encoded in the local heterogeneity, and where the objective is to spatially assign distinctive quantitative scale signatures to characterize and classify such regions, a new local scale definition is warranted. [0007] Local morphologic scale (LMS), as disclosed herein, differs from previous locally adaptive scale definitions in that it is not shape constrained as with tensor or ball-scale definitions. Additionally, LMS is driven by heterogeneity considerations as opposed to the homogeneity constraints as in the case of ball (b), tensor (t), and generalized- scales. While b- and t-scale definitions provide a feature space based on scale parameters, they are not discriminatory enough for pixel level classification, especially for biological and histological data, which can have a significant amount of structural complexity. The rich domain specific features, which can be derived from LMS, can be used successfully in conjunction with a supervised classifier to discriminate regions with different structure and heterogeneity.

SUMMARY OF THE INVENTION

[0008] According to one aspect, the described invention provides a method for classifying and registering a region of interest (ROI) in an image by utilizing local morphologic scale (LMS), the method comprising: (a) obtaining an image of a tissue sample comprising a region of interest (ROI), wherein each pixel of the image is associated with identical potential energy; (b) initializing a plurality of particles at the region of interest (ROI) in the image, wherein an initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories; (c) modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets the obstruction, each particle loses its velocity and changes its trajectory; (d) locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory; (e) connecting the catchment of each particle to yield a closed polygon, wherein the closed polygon represents a local morphologic scale (LMS) signature of the region of interest (ROI); and (f) classifying and registering the local morphologic scale (LMS) signature.

[0009] According to one embodiment of the method, the obstruction in (c) is in a form of high local gradients. According to another embodiment, the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving. According to another embodiment, the local morphologic scale (LMS) is classified and registered by parallelized computations. According to another embodiment, the image is a biological image. According to another embodiment, the image is a histological image. According to another embodiment, the image comprises a magnetic resonance image, a positron emission tomography (PET) image, and a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2 -weighted image, or a combination thereof. According to another embodiment, the tissue sample is a tissue microarray (TMA). According to another embodiment, the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample. According to another embodiment, the tissue sample is a cancer tissue sample. According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer. According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique. According to another embodiment, at least one of steps (a), (b), (c), (d), (e), and (f) is performed using a computer. According to another embodiment, at least one of steps (a), (b), (c), (d), (e), and (f) is performed automatically.

[00010] According to another aspect, the described invention provides a computing device comprising: a processor; and a storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising: (a) logic executed by the processor for obtaining an image of a tissue sample comprising a region of interest (ROI), wherein each pixel of the image is associated with identical potential energy; (b) logic executed by the processor for initializing a plurality of particles at the region of interest (ROI) in the image, wherein an initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories; (c) logic executed by the processor for modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets the obstruction, each particle loses its velocity and changes its trajectory; (d) logic executed by the processor for locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory; (e) logic executed by the processor for connecting the catchment of each particle to yield a closed polygon, wherein the closed polygon represents a local morphologic scale (LMS) signature of the region of interest (ROI); and (f) logic executed by the processor for classifying and registering the local morphologic scale (LMS) signature.

[00011] According to one embodiment, the computing device according to claim 16, wherein the obstruction in (c) is in a form of high local gradients. According to another embodiment, the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving. According to another embodiment, the local morphologic scale (LMS) signature is classified and registered by parallelized computations. According to another embodiment, the image is a biological image. According to another embodiment, the image is a histological image. According to another embodiment, the image comprises a magnetic resonance image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2-weighted image, or a combination thereof. According to another embodiment, the tissue sample is a tissue microarray (TMA). According to another embodiment, the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample. According to another embodiment, the tissue sample is a cancer tissue sample. According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer. According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

[00012] According to another aspect, the described invention provides a computer-readable storage medium tangibly storing thereon computer program instructions capable of being executed by a computer processor of a computing device, the computer program instructions defining steps of: (a) obtaining an image of a tissue sample comprising a region of interest (ROI), wherein each pixel of the image is associated with identical potential energy; (b) initializing a plurality of particles at the region of interest (ROI) in the image, wherein an initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories; (c) modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets the obstruction, each particle loses its velocity and changes its trajectory; (d) locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory; (e) connecting the catchment of each particle to yield a closed polygon, wherein the closed polygon represents a local morphologic scale (LMS) signature of the region of interest (ROI); and (f) classifying and registering the local morphologic scale (LMS) signature.

[00013] According to one embodiment, the obstruction in (c) is in a form of high local gradients. According to another embodiment, the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving. According to another embodiment, the local morphologic scale (LMS) signature is classified and registered by parallelized computations. According to another embodiment, the image is a biological image. According to another embodiment, the image is a histological image. According to another embodiment, the image comprises a magnetic resonance image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2 -weighted image, or a combination thereof. According to another embodiment, the tissue sample is a tissue microarray (TMA). According to another embodiment, the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample. According to another embodiment, the tissue sample is a cancer tissue sample. According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer. According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

BRIEF DESCRIPTION OF THE DRAWINGS

[00014] FIGURE 1 shows associated b-scale ((a)-(d)), g-scale ((e)-(h)), and local

morphologic scale (LMS) ((i)-(l)) for a candidate image location on an ovarian cancer (OCa) biopsy image. For lymphocyte in stromal regions (Figures 1 (a), (b)), the associated b-scale ((a), (b)) and g-scale ((e), (f)) regions are large, reflecting the relative image homogeneity in that location of the image. Note, however, that the corresponding g-scale set is affected by the presence of local heterogeneity (g-scale has multiple cavities). The LMS ((i), (j)) for stromal regions has particles radiating far out in certain directions, but also is constrained locally (to the right) because of neighboring nuclei. For tumor infilitrating leukocytes (TILs) in panels ((c), (d)), the corresponding b-scale is small, with g-scale resulting in an amorphous shape with multiple cavities. Additionally, the g-scale sets for the TIL regions in ((g), (h)) are different. The corresponding LMS ((k), (1)), while not constrained by a prior shape model, yields a local structural signature that is consistent across both ((k), (1)) and distinctly different from the corresponding non-TIL LMS signatures ((i), (j)).

[00015] FIGURE 2 shows a flow chart representing the local morphologic scale (LMS) signature creation algorithm. Particles emanate at each image location and proceed in radially divergent directions with the same initial conditions. As the particles traverse along their initial path, they can encounter three different situations: (a) a high velocity may allow them to overcome obstacles; (b) a low velocity and a relatively minor obstacle in which case the particle can navigate around the obstacle; and (c) low velocity and an insurmountable obstacle (e.g. a large image gradient) in which case the particle grinds to a halt. The lower right panel shows the final LMS, reflecting the final resting location of the particles. The set of pixels in the polygon, obtained by linking the final locations of the particles, is the final LMS corresponding to the spatial location under consideration.

[00016] FIGURE 3 shows a spatial location under consideration across all images, which is indicated by the "X" mark. The particle paths emanating from "X" are shown as a bold line. The arrows indicate the direction vector. In (a), the particle heading out at 15° hits an obstruction and comes to a halt (a circle showing the final resting location). In (b), the particle heading out at 285° goes around the obstruction. In (c), the particle has amassed enough velocity to proceed over the obstructions. The circle in each of ((a)-(c)) represents the final resting place for the particle p_; .

[00017] FIGURE 4 shows the effect of changing the number of radial particles emanating from each c e C . In the case of Θ = 180 (a), a particle is introduced at 2° intervals, and the computation time is higher, but the resulting local morphologic scale (LMS) L(c) is able to capture local heterogeneity with greater resolution. When Θ = 12 (c), the corresponding L(c) has poorer resolution, (b) Θ = 36 represents a good compromise between resolution and

computational complexity for L(c).

[00018] FIGURE 5 shows the relative invariance of the local morphologic scale (LMS) to the choice of color space. For some pixel c e C, the corresponding L(c) is shown in the original Red, Green, and Blue (RGB) color space. Note that the corresponding L(c) obtained following transformation of the image in (a) to alternative color spaces (b) Hue, Saturation, and Value (HSV), and (c) an arbitrary color transformation of RGB, is surprisingly consistent. This particular property of LMS has important implications in CAD application in histopathology, where staining and illumination difference can significantly alter visual appearance of the image scene.

[00019] FIGURE 6 shows (a) T1-, (b) T2-, and (c) Proton Density (PD)-magnetic resonance imaging (MRI) images of a representative 2D planar section from a synthetic 3D brain study obtained from the Brainweb repository (Collins, D. et al., IEEE Trans. Med. Imag. 17, 463-486, 1998). A visual representation of the corresponding principle component analysis (PCA) scenes was derived from the LMS signatures at every spatial location ((d), (e), and (f), respectively). Using the 9 LMS features presented in Table 1 , the RGB values were set to the three largest principal components. Note the similarity in visual appearance of the principle component analysis (PCA) representations of the LMS scenes derived from the three different MRI protocols ((a)-(c)). In each of Figures 5(d)-(f), every pixel is represented by the three principal eigenvectors, scaled to the RGB color space.

[00020] FIGURE 7 shows two images corresponding to different regions within the same ovarian cancer (OCa) histology image, (a) Non-stroma and (d) stroma regions; the corresponding particle trajectories are illustrated in ((b) and (e)). The shape of the local morphologic scale (LMS) signatures in the two different tissue regions was drastically different. The polygon containing the various particle signatures at an individual spatial location ((c), (f)) is created and a number of mophometric features extracted from it. Every spatial location in the image then can be associated with a multi-dimensional morphometric LMS feature vector.

[00021] FIGURE 8 shows representative tumor ((a), (e)) and stromal patches ((i), (m)) from an ovarian cancer (OCa) biopsy image. Panels (b), (f), (j), (n), and (c), (g), (k), (o), respectively, represent the local morphologic scale (LMS) parametric images obtained by assigning each spatial image location with the area and extent of the polygon obtained by connecting the particle trajectories. Panels (d), (h), (1), (p) represent the three principal eigenvectors scaled to RGB space obtained via application of principle component analysis (PCA) to a 24 dimensional LMS feature vector at each c e C . In the case of endothelial cells in the stroma class ((i), (m)), the LMS calculation was limited to the region inside of the cell, as opposed to outside.

[00022] FIGURE 9 shows the Receiver Operating Characteristic (ROC) curve for the Probabilistic Boosting Tree (PBT) classifier for discriminating between tumor and stromal tissue classes over 11,000 images of ovarian cancer (OCa) biopsy specimens.

DETAILED DESCRIPTION OF THE INVENTION

Glossary

[00023] The term "cartesian coordinates" as used herein refers to a system of coordinates for locating a point on a plane (Cartesian plane) by its distance from each of two intersecting lines, or in space by its distance from each of three planes intersecting at a point.

[00024] The terms "classify" or "classifying" as used herein refers to the labeling of one or more objects (e.g. images, regions, pixels) into one of a number of predefined categories.

[00025] The term "classifier" as used herein refers to a computational device capable of performing a classification function. A classification function is a discrete value output function, such as for example, a prediction of disease outcome. According to the described invention, each parameter has its own separate classifier. The plurality of individual classifiers are then combined to create a meta-classifier (combined classifier) which yields the final risk score. [00026] The term "eccentricity" as used herein refers to a ratio of the distance between the foci of an ellipse and its major axis length.

[00027] The term "eigenvector" as used herein refers to a special set of vectors associated with a linear system of equations (i.e., a matrix equation) that are sometimes also known as characteristic vectors, proper vectors, or latent vectors. Each eigenvector is paired with a corresponding so-called "eigenvalue."

[00028] The term "extent" as used herein refers to a ratio of pixels in a region to pixels in the total bounding box.

[00029] The term "final angles" as used herein refers to a difference between the start heading and end angle at end location.

[00030] The term "HSV color space" as used herein refers to a cylindrical-coordinate representation of points in an RGB color model, which rearrange the geometry of the RGB model.

[00031] The term "initializing" as used herein refers to assigning initial values to variables of a particle such as velocity and direction before they emanate from the region of interest (ROI). This initializing results in the movement with an initial velocity under different radial headings.

[00032] The term "modeling" as used herein refers to capturing and quantifying the magnitude and orientation of particles' paths on a per pixel basis using a physics-based system.

[00033] The term "Monte-Carlo" or "Monte-Carlo sampling technique" as used herein refers to an art of approximating an expectation by the sample mean of a function of simulated random variables. The Monte-Carlo sampling technique uses a repeated random sampling to compute increasingly accurate results.

[00034] The term "neighbor distance" as used herein refers to a distance between

neighboring particles ending locations.

[00035] The term "obstacle hits" as used herein refers to a density function of the number of obstructions hit by the particle set. [00036] The term "obstruction" as used herein refers to an area of high resistance to motion of a particle in a straight line.

[00037] The term "Oca" as used herein refers to ovarian cancer.

[00038] The term "parameter" as used herein refers to a variable, aspect or element.

[00039] The term "parametric image" as used herein refers to assigning algorithmically determined value(s) to a pixel which encodes some information regarding properties of the pixel and its local neighborhood

[00040] The term "particle" as used herein refers a small localized object to which can be ascribed several physical properties such as volume, mass, trajectory heading, etc.

[00041] The term "Probabilistic Boosting Tree" or "PBT" as used herein refers to a framework for learning two or more class discriminative models to allow unseen samples to be correctly classified. In the training stage, a tree is recursively constructed in which each tree node is a strong classifier. The input training set is divided into two new sets, left and right ones, according to the learned classifier, each of which is then used to train the left and right sub-trees recursively. Zhuowen Tu, "Probabilistic Boosting-Tree: learning Discriminative Models for Classification, Recognition and Clustering," Proceeding, ICCV '05 Proceedings of the Tenth IEEE Intl Conference on Computer Vision, Vol. 2 (2005).

[00042] The term "registering" as used herein refers to spatially aligning two distinct images so that anatomy of interest occupies the same pixels in both images.

[00043] The term "receiver operating characteristic (ROC) curve has the following meaning. The sensitivity of a diagnostic test is the proportion of patients for whom the outcome is positive that are correctly identified by the test. The specificity is the proportion of patients for whom the outcome is negative that are correctly identified by the test. When the cut-off value for a continuous diagnostic variable is increased (assuming that larger values indicate an increased chance of a positive outcome), the proportions of both true and false positives decreases. These proportions are the sensitivity and 1 - specificity, respectively. A graph of sensitivity against 1 - specificity is called a receiver operating characteristic (ROC) curve. The performance of a diagnostic variable can be quantified by calculating the area under the ROC curve (AUROC). Bewick, V.., et al.., Crit. Care 8(6): 508-512 (2004).

[00044] The term "RGB color space" as used herein refers to an additive color model in which red, green, and blue light is added together in various ways to reproduce a broad array of colors.

[00045] The term "ball scale" or "b-scale" as used herein refers to the value corresponding to the radius of the largest ball encompassing all locations neighboring the location under consideration and satisfying some pre-defined homogeneity criterion. . The ball scale takes into account the local scale information at every image element to adaptively control diffusion and extent of filtering (Saha, P. K. et al., "Scale-based fuzzy connected image segmentation:

Theory,algorithms, and validation," CVIU 77(2), 145 - 174, 2000).

[00046] The term "generalized scale" or "g-scale" as used herein refers to the largest connected set associated with every spatial location, such that all spatial locations in this set satisfy a pre-defined homogeneity criterion (Madabhushi, A., et al, "Generalized scale: theory, algorithms, and application to image inhomogeneity correction," CVIU 101(2), 100-121, 2006).

[00047] The term "tensor scale" or "t-scale" as used herein refers to a parametric

representation of local structure morphology that simultaneously describes its orientation, shape, and isotropic scale. At any image location, t-scale is the parametric representation of the largest ellipse (an ellipsoid in three dimensions (3D)). (Saha, P. K., "Tensor scale: a local morphometric parameter with applications to computer vision and image processing,". 99(3), 384-413, 2005).

[00048] The term "survival" as used herein refers to ratio of particles that make it to the end of the simulation time.

Local Morphologic Scale (LMS)

[00049] Local structure is characterized by the presence, strength, and orientation of local image gradients. These local gradients are a function of the precise location of neighborhood objects (pixel intensities in the case of images). The local morphologic scale (LMS) definition aims to capture and quantify the magnitude and orientation of the local gradients, computed on a per pixel basis. The definition of LMS, as described herein, assumes that every pixel is associated, initially, with identical potential energy. Each spatial location under consideration emanates multiple particles (in different radial directions) and moves them with an initial velocity in different trajectories (see Figure 2 for flowchart). If the particle moves along an unimpeded trajectory (i.e. it does not encounter an obstruction in the form of a location with a very different intensity compared to that at the current location) the particle gains velocity.

However, obstructions, in the form of high local gradients, reduce the particle's velocity and cause changes in the particle's trajectory. When the particle's velocity goes to zero (because of colliding with an insurmountable gradient or running out of time) it stops moving. The connection of the final particle trajectories yields a closed polygon, which is the LMS for the pixel under consideration.

Theory of Local Morphologic Scale (LMS) [00050] The term "image" as used herein is defined as€ = (C_> f) where C is an nD grid representing N spatial locations c £ C, f(c) e I^s where s represents the dimensionality of /(s = 1 for a gray scale image and s = 3 for an RGB color image. For a 2D image scene, C £ ^s and c = (x, y) represent the Cartesian coordinates of pixel c and f a color intensity function associated with c. For eachc e C , there are associated particles p_t , i e {l,..., m} , where m = 2π -θί and is a predefined parameter. Each particle p_i originates from c and travels along different radial directions (specified by 2π -θί) away from c. For each particle, initial particle energy ( u_{i 0} ) is the potential energy given to each particle, which causes it to start moving. Over the course of the particle's travel, the particle energy u_{i t} where t e {Ο,.,., Γ} , will change.

[00051] The term "catchment" or "catchment area", defined as i?(c, /?. ) e C for a pixel c and particle p i e (l,..., m), refers to a final image location where particle energy reaches zero

( u_{i t} = 0 ). A particle ( p_{ ) comes to a halt when either u_{i t} = 0 or when the particle has reached t

= T. For example, Figure 3 shows three different scenarios for a particle with different initial velocities and encountering impediments of varying magnitude. [00052] The term "trajectory" or ":T c₍ p,)" as used herein refers to the path described by a particle moving along an image. Trajectory (c p,) is associated with particle p_; , i e (l,..., m), originating at c e C is given as T(C, p₍) = [ά' d^A _{> ^},d^lxj<] , where d^(,) e C and t, k e {Ι,.,., Γ} .

Note that d^{k) is the position of the particle p_{ after its velocity u_{i k} = 0 or a certain length of time has elapsed since p_; began moving.

[00053] The system of a physics-based equation that dictates the trajectory J( ,j?_s) is given as follows. At the time step t + 1 , p 's velocity is updated as, u_t+i = u_t - Equation [1]

[00054] Since d⁽⁰⁾ = c, the subsequent location d^(l) is identified as d^(M) = arg - /(e) , Equation [2]

where N(d') represents the 8-connected neighborhood of d^(t) in W . is terminated when for any given iteration t, u_t≤ 0 or when t = T. Particle trajectories for pixel c e C for the ovarian cancer (OCa) image are shown for Figure 3.

[00055] The term "local morphologic scale" or "LMS" as used herein refers to a local scale definition that attempts to model local structural heterogeneity at every spatial location in an image. A set of LMS features derived from LMS at every image location can be used to train a supervised classifier to identify similar structural signatures in an image. The LMS for any c e C is obtained as the set of all pixels d e C contained in the polygon P, constructed by linearly connecting R(c, p_p ) and R(c, p_p+l ) where β e {l,..., m - 1} .

[00056] The term "particle" as used herein refers to Pi {c), i e {l,..., m} , which is given an initial particle energy (u_{p 0} ) . This energy moves to pull a particle towards a lower energy position.

[00057] The term "particle trajectory" as used herein refers to the k step path

T(c, p_t) = {d^l , d² ,...,d^k } , which a particle must take given u ₀ in order to reach its catchment region ^(c, /?,) .

[00058] The term "path of least resistance", T(c, p_i ), is computed such that it minimizes the overall rate of change of quantized intensity values over the trajectory k steps: Equation [2]

Algorithm

[00059] The table below represents the algorithm (Algorithm 1 LMS) for computing the LMS 1(c)) for every c e C .

rt:t urtl ί■ -: :

[00060] After initializing particle p_i in direction δ, where δ = 2π - θί,ί e {l,...,m} and with an initial velocity of u_{t 0} , the particle p_{ continues to propagate until either the simulation has ended or u_{i t} = 0

[00061] Step 6 sets the tentative next particle location d^(t+l) to the adjacent pixel in the direction of δ. Step 7 determines if p_; has sufficient velocity at d^{t) for it to be able to move to d^(t+l) . In the case where the particle is obstructed, N(d^(t) ) is examined to determine which of the locations e e N(d^(t)), p_i should move to next. For each e e N(d^(t)), p_i , the gradient with respect to d^(t) is determined, and p_i moves to the location determined by Equation 2. This intuitively coincides with a particle going around an obstruction, which results in the curved paths as seen in Figure 3(b).

[00062] Figure 3 illustrates the three typical situations encountered by a particle. A particle is placed at c, which in this case is marked with the "X". The particle moves in a direction determined via Equation 1. As seen in Figure 3(a), a particle that heads out at an angle 15° and continues unobstructed until its velocity goes to 0 and comes to a halt.

[00063] On the other hand, in Figure 3(c), a particle, which heads out with an orientation of 224° and continuing unobstructed until it arrives at a stained endothelial cell, can be observed. In spite of the high image gradient at this spatial location, the particle's velocity allows it to circumvent the endothelial cell and carry on, on its path. Finally, in Figure 3(b), a particle, which heads out at an initial orientation of 270° and continues unobstructed until it reaches a large tumor cell, can be observed. The particle's current velocity is insufficient for it to overcome the obstacle; it is, however, able to find a less resistant path and thus circumvents the obstruction, albeit at a reduced new velocity. The final location (catchment) where the particle comes to a halt (in each case) is indicated by a circle in Figures 3 ((a), (b), and (c)).

Implementation

[00064] The preprocessing consists of creating a quantized color scene (to reduce the color resolution on account of computational considerations) by using a standard k-means algorithm. (Seber, G., Multivariate Observations, Wiley, New York, 1984). Hence the total number of unique color/gray scale intensities in the image is reduced to q.

[00065] From Step 1 of the LMS algorithm, it can be seen that the Monte-Carlo (the use of repeated random sampling to compute increasingly accurate results) aspect of the algorithm allows for the selection of a distribution of values, which coincide with the resolution of L(c). By increasing the number of radial particles emanating from c (a direct result of setting to a higher value), the computation time is increased, but the resulting LMS L(c) is able to capture local heterogeneity with greater resolution (see Figure 4(a)). On the other hand, with a smaller value, L(c) losing its distinctiveness can be observed (Figure 4(c)). This allows the overall computation time of L(c) to be modulated as a function of the desired resolution.

[00066] The overall scene L is identical regardless of order of computation of both c and p. Evidence of this comes from L(Cj ) being computed independently iromc_k (j≠ k) , where k e {1,..., N} , and T^c. ^V {I_., ... ,m}, being computed deterministically. Additionally, due to the decoupling of both points and particle paths, LMS is a prime candidate for massive parallel computation.

Methods and Systems for Classifying and Registering a Region of Interest

(ROI) in an Image by Local Morphologic Scale (LMS)

[00067] According to one aspect, the described invention provides a method for classifying and registering a region of interest (ROI) in an image by utilizing local morphologic scale (LMS), the method comprising:

(a) obtaining an image of a tissue sample comprising a region of interest (ROI), wherein each pixel of the image is associated with identical potential energy;

(b) initializing a plurality of particles at the region of interest (ROI) in the image, wherein an initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories;

(c) modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system, such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets an obstruction, each particle loses its velocity and changes its trajectory;

(d) locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory;

(e) connecting the catchment of each particle to yield a closed polygon, wherein the closed polygon represents a local morphologic scale (LMS) signature of the region of interest (ROI); and

(f) classifying and registering the local morphologic scale (LMS) signature.

[00068] According to one embodiment of the method, the obstruction in (c) is in a form of high local gradients.

[00069] According to another embodiment, the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving.

[00070] According to another embodiment, the local morphologic scale (LMS) signature is classified and registered by parallelized computations.

[00071] According to one embodiment of the method, the image is a biological image.

[00072] According to another embodiment, the image is a histological image.

[00073] According to another embodiment, the image includes, but is not limited to, a magnetic resonance image, a positron emission tomography (PET) image, a single-photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, and a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprising a diffusion- weighted image, a Tl -weighted image, a T2- weighted image, or a combination thereof.

[00074] According to another embodiment, the tissue sample is a tissue microarray (TMA).

[00075] According to another embodiment, the tissue sample includes, but is not limited to, a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample.

[00076] According to another embodiment, the tissue sample is a cancer tissue sample.

[00077] According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

[00078] According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

[00079] According to another embodiment, at least one of steps (a), (b), (c), (d), (e) and (f) is performed using a computer.

[00080] According to another embodiment, at least one of steps (a), (b), (c), (d), (e) and (f) is performed automatically.

[00081] According to another aspect, the described invention provides a computing device comprising: a processor; and a storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

(a) logic executed by the processor for obtaining an image of a tissue sample comprising a region of interest (ROI), wherein each pixel of the image is associated with identical potential energy;

(b) logic executed by the processor for initializing a plurality of particles at the region of interest (ROI) in the image, wherein the initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories;

(c) logic executed by the processor for modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system, such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets an obstruction, the particle loses velocity and changes its trajectory;

(d) logic executed by the processor for locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory;

(e) logic executed by the processor for connecting the catchment of each particle to yield a closed polygon, wherein the closed polygon represents a local morphologic scale (LMS) signature of the region of interest (ROI); and

(f) logic executed by the processor for classifying and registering the local morphologic scale (LMS) signature.

[00082] According to one embodiment, the obstruction in (c) is in a form of high local gradients.

[00083] According to another embodiment, the modeling step (c) is repeated until the velocity of the particle reaches zero and the particle stops moving.

[00084] According to another embodiment, the local morphologic scale (LMS) signature is classified and registered by parallelized computations.

[00085] According to another embodiment of the method, the image is a biological image.

[00086] According to another embodiment, the image is a histological image.

[00087] According to another embodiment, the image includes, but is not limited to, a magnetic resonance image, a positron emission tomography (PET) image, a single-photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, and a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprising a diffusion- weighted image, a Tl -weighted image, a T2- weighted image, or a combination thereof. [00088] According to another embodiment, the tissue sample is a tissue microarray (TMA).

[00089] According to another embodiment, the tissue sample includes, but is not limited to, a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample.

[00090] According to another embodiment, the tissue sample is a cancer tissue sample.

[00091] According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

[00092] According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

[00093] According to another aspect, the described invention provides a computer-readable storage medium tangibly storing thereon computer program instructions capable of being executed by a computer processor of a computing device, the computer program instructions defining steps of:

(b) initializing a plurality of particles at the region of interest (ROI) in the image, wherein the initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories;

(c) modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on response of each particle to an obstruction in its via a physics-based system, such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets obstruction, the particle loses velocity and changes its trajectory; (d) locating a catchment of each particle, wherein the catchment represents a final location where each particle stops moving along its trajectory;

(f) classifying and registering the local morphologic scale (LMS) signature.

[00094] According to one embodiment, the obstruction in (c) is in a form of high local gradients.

[00095] According to another embodiment, the modeling step (c) is repeated until the velocity of the particle reaches zero and the particle stops moving.

[00096] According to another embodiment, the local morphologic scale (LMS) signature is classified and registered by parallelized computations.

[00097] According to another embodiment of the method, the image is a biological image.

[00098] According to another embodiment, the image is a histological image.

[00099] According to another embodiment, the image includes, but is not limited to, a magnetic resonance image, a positron emission tomography (PET) image, a single-photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, and a computed tomography (CT) image. According to another embodiment, the magnetic resonance imaging (MRI) image comprising a diffusion- weighted image, a Tl -weighted image, or a T2- weighted image.

[000100] According to another embodiment, the tissue sample is a tissue microarray (TMA).

[000101] According to another embodiment, the tissue sample includes, but is not limited to, a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample. [000102] According to another embodiment, the tissue sample is a cancer tissue sample.

[000103] According to another embodiment, the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

[000104] According to another embodiment, the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

[000105] The present invention is described below with reference to block diagrams and operational illustrations of methods and devices to select and present media related to a specific topic. It is understood that each block of the block diagrams or operational illustrations, and combinations of blocks in the block diagrams or operational illustrations, can be implemented by means of analog or digital hardware and computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, ASIC, or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implements the functions/acts specified in the block diagrams or operational block or blocks.

[000106] For the purposes of this disclosure a computer readable medium stores computer data, which data can include computer program code that is executable by a computer, in machine readable form. By way of example, and not limitation, a computer readable medium may comprise computer readable storage media, for tangible or fixed storage of data, or communication media for transient interpretation of code-containing signals. Computer readable storage media, as used herein, refers to physical or tangible storage (as opposed to signals) and includes without limitation volatile and non- volatile, removable and non-removable media implemented in any method or technology for the tangible storage of information such as computer-readable instructions, data structures, program modules or other data.

Computer readable storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other physical or material medium which can be used to tangibly store the desired information or data or instructions and which can be accessed by a computer or processor.

[000107] For the purposes of this disclosure a module is a software, hardware, or firmware (or combinations thereof) system, process or functionality, or component thereof, that performs or facilitates the processes, features, and/or functions described herein (with or without human interaction or augmentation). A module can include sub-modules. Software components of a module may be stored on a computer readable medium. Modules may be integral to one or more servers, or be loaded and executed by one or more servers. One or more modules may be grouped into an engine or an application.

[000108] Those skilled in the art will recognize that the methods and systems of the present disclosure may be implemented in many manners and as such are not to be limited by the foregoing exemplary embodiments and examples. In other words, functional elements being performed by single or multiple components, in various combinations of hardware and software or firmware, and individual functions, may be distributed among software applications at either the client or server or both. In this regard, any number of the features of the different

embodiments described herein may be combined into single or multiple embodiments, and alternate embodiments having fewer than, or more than, all of the features described herein are possible. Functionality may also be, in whole or in part, distributed among multiple components, in manners now known or to become known. Thus, myriad software/hardware/firmware combinations are possible in achieving the functions, features, interfaces and preferences described herein. Moreover, the scope of the present disclosure covers conventionally known manners for carrying out the described features and functions and interfaces, as well as those variations and modifications that may be made to the hardware or software or

firmware components described herein as would be understood by those skilled in the art now and hereafter.

[000109] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein also can be used in the practice or testing of the described invention, the preferred methods and materials are now described. All publications mentioned herein are incorporated herein by reference to disclose and describe the methods and/or materials in connection with which the publications are cited.

[000110] Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the invention. The upper and lower limits of these smaller ranges which may independently be included in the smaller ranges is also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either both of those included limits are also included in the invention.

[000111] It must be noted that as used herein and in the appended claims, the singular forms "a", "and", and "the" include plural references unless the context clearly dictates otherwise. All technical and scientific terms used herein have the same meaning.

[000112] The publications discussed herein are provided solely for their disclosure prior to the filing date of the present application. Nothing herein is to be construed as an admission that the described invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication provided may be different from the actual publication dates which may need to be independently confirmed.

[000113] The described invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof and, accordingly, reference should be made to the appended claims, rather than to the foregoing specification, as indicating the scope of the invention.

EXAMPLES

[000114] The following examples are put forth so as to provide those of ordinary skill in the art with a complete disclosure and description of how to make and use the present invention, and are not intended to limit the scope of what the inventors regard as their invention nor are they intended to represent that the experiments below are all or the only experiments performed. Efforts have been made to ensure accuracy with respect to numbers used (e.g. amounts, temperature, etc.) but some experimental errors and deviations should be accounted for. Unless indicated otherwise, parts are parts by weight, molecular weight is weight average molecular weight, temperature is in degrees Centigrade, and pressure is at or near atmospheric.

Property 1: Invariance of Local Morphologic Scale (LMS) to Color

Spaces

[000115] One of the key challenges in developing computer assisted diagnostic (CAD) methods for the automated detection and classification of objects in histopathology imagery is the ability to deal with changes in illumination and stain variations (Madabhushi, A. et al., Clin Chem Lab Med 48(7), 989-998, 2010; Madabhushi, A., et al, "Computer-aided prognosis:

Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data," CMIG, 2011; Madabhushi, A., "Digital pathology image analysis: opportunities and challenges," Imaging in Medicine 1,7-10(4), Oct 2009). LMS has the unique advantage of being invariant to the absolute color values associated with the image pixels. Since £ is computed from the image gradient (see Equation 1), a derivative of local structure in this case, £ encodes images of similar structure in a manner independent of their absolute intensity value characteristics.

[000116] Qualitatively, when this property is examined in Figure 5 the following are observed: (a) the original image; (b) the corresponding HSV (hue, saturation, and value) representation; and (c) an arbitrary color transformation of the original space. When a spatial location c is considered and the associated £ c} is computed across the 3 scenes, it can be observed that the

LMS signature is consistent, though the color spaces are significantly different.

Property 2: Invariance of LMS to different imaging protocols

[000117] To illustrate the invariance of LMS to different imaging protocols, a synthetic multimodal data set from Brain Web (Collins, D. et al., IEEE Trans. Med. Imag. 17, 463-486, 1998), which comprises corresponding multiprotocol (Tl, T2, and Proton Density (PD)) MRI slices, was considered. A single 2D slice (gray scale) from within this 3D volume was selected. Figure 6 shows three different MRI protocols (Tl (a), T2 (b), PD (c)) for a corresponding slice along with their corresponding LMS scenes ((d), (e), (f) respectively). At each c e C the LMS derived features (e.g. area, perimeter, orientation, major and minor axis length, eccentricity, equivalent diameter, and solidity) were extracted; then features were rescaled in order to lie in the range [0,1]. Principal component analysis (PC A) then was performed on the LMS matrix in order to obtain the top 3 eigenvectors for each pixel. After that, the first principal component was mapped to the Red channel, the second to the Green channel, and the third to Blue channel. Each image channel then was independently rescaled to the [0,255] range. Although the input images are all of different intensity values and contrast patterns, the LMS scenes were surprisingly similar.

[000118] In the described invention, local morphologic scale (LMS) was applied to classify regions as either stromal or endothelial in ovarian cancer (OCa) histology slides. Previous studies have suggested that tumor-infiltrating lymphocytes (TILs) serve as a prognostic image marker of disease outcome in OCa (Sato, E. et al, Proc Natl Acad Sci U S A 102(51), 18538-18543, 2005; Clarke, B. et al, Mod Pathology 22, 393-402, 2008; Zhang, L. et al, N Engl J Med 348(3), 203- 213, 2003). Since lymphocytes in both the stromal and tumor regions appear identical, identifying TIL's requires identification of a tissue that lymphocytes are embedded in. In other words, discriminating TILs from non-TILs has to do with quantifying the appearance of the local neighborhood within which a lymphocyte is present. A lymphocyte present within a largely homogeneous region is strongly suggestive of one embedded in stroma (a non-TIL), whereas a lymphocyte surrounded by tumor cells is one where the local neighborhood would be very heterogeneous.

[000119] As seen from Figures 1 ((i), (j)), the LMS of the described invention extends radially outwards for lymphocytes in the stroma region (on account of lack of impediments), while for TILs (Figures 1 (k), (1)), the particle trajectories are constrained on account of the local heterogeneity.

[000120] The difference in the LMS shape for the lymphocytes in Figures l(i), l(j) and Figures 1 (k), 1(1), respectively, can be exploited to distinguish all TILs and non-TILs in the image, a laborious task for a human to perform manually. The described invention provides that a Probabilistic Boosting Tree (PBT) classifier (Tu, Z., CCV '05: Proceedings of the Tenth IEEE International Conference on Computer Vision, 1589-1596, IEEE Computer Society, Washington, DC, USA, 2005) can be used in conjunction with the LMS features to accurately discriminate between stroma and non- stroma regions in ovarian cancer histology images.

[000121] In addition, LMS exhibits invariance to changes in object illumination and to image intensity differences across multiple imaging protocols for the same patient study. It was observed qualitatively that LMS scenes (obtained by replacing the image pixel intensities with corresponding LMS derived features) are nearly identical for the same image in different color spaces and across different imaging protocols, suggesting that LMS may have applications in other image processing domains (apart from pixel-level or object classification) such as image registration.

Data Description

[000122] The ability of the LMS features were evaluated to discriminate between stromal and tumor regions in ovarian cancer (OCa) biopsy images. Specifically, the LMS features on 1 1 different OCa histology images were evaluated qualitatively. These images were stained with an immunohistochemical stain for CD3 positive T cells, with a hematoxylin counterstain across 6 different patient studies. Ground truth for stromal and tumor regions on these OCa images was obtained via manual annotation. Each image was 1400 x 1400 pixels, and scanned at 40x optical resolution.

[000123] Table 1. A representative set of features derivable from L. In the case of a completely homogeneous region, without any gradient differences, L(c) is a circle as all the particles come to a halt at a distance T from c.

Experimental Design

[000124] The experiment described herein below was designed to determine how well local morphologic scale (LMS) features can be used to differentiate between stroma and tumor regions. The classification of these regions allows for the identification of a lymphocyte as either a tumor-infiltrating leukocyte (TIL) or a non-tumor infiltrating leukocyte (non-TIL).

[000125] The following LMS features were employed for this task: Area, Major Axis Length, Minor Axis Length, Eccentricity, Orientation, Convex Area, Unique Locations, Equiv Diameter, Solidity, Extent, Perimeter (see Table 1).

[000126] Figure 1 presents an example of the stroma ((a) and (b)) versus the tumor classes ((c) and (d)). The stroma region is smoother and less dense as compared to the more complex tumor region (see Figures 7 (a), (d)). With the point of interest located in the center of the image, particle paths are determined by obstructions caused by cells (blue), and particles either stop or move around based on their current velocity. Since the two classes have different particle path trajectories (the stroma class has longer particle paths while the tumor class has more restricted particle trajectories and as such is more dense and constrained), quantifying these patterns allows a supervised classifier to successfully distinguish unseen test cases.

[000127] Specifically, 1000 tissue locations were selected randomly from each of the images, creating 11,000 individual image patches. The training set was consisted of 80% of the image patches, and the test set was consisted of the remaining 20%. This process was repeated 5 times, with a new training and testing set being chosen at every iteration. The LMS variables employed were q = 2, = 36 and T = 100. The chosen supervised classifier was Probabilistic Boosting Trees (PBT), a popular approach that has the attractive property that the posterior probability can be used as a threshold to balance between sensitivity and specificity (Tu, Z. et al, CCV '05:

Proceedings of the Tenth IEEE International Conference on Computer Vision, 1589-1596, IEEE Computer Society, Washington, DC, USA, 2005). Additionally, the classifier is efficient and does not take very long to train.

Computational Considerations

[000128] All calculations and local morphologic scale (LMS) experiments were performed on an 8 core processor at 2.66 GHz with 72 gigabytes of RAM. Table 2 shows that the computation time per 2000 samples was proportional to the number of particles. Increasing the number of particles increased the area under the receiver operation characteristic curve (AUC) but at the cost of additional computation time. On average, the LMS computation for around 2000 spatial locations took 8 seconds. Additionally, due to the straightforward definition of a particle life, the computation of LMS can be performed on a graphics processing unit (GPU).

Example 1. Application of local LMS in Distinguishing Tumor and Stromal

Regions on Ovarian Cancer Biopsy Images.

[000129] Figure 8 shows two panels ((a), (e)) of tumor regions on ovarian cancer (OCa) biopsy samples, while panels ((i), (m)) represent candidate stromal regions. For each pixel in each of Figures 8 (a), (e), (i), (m), the polygon obtained by connecting corresponding particle trajectories was constructed and multiple local morphologic scale (LMS) features were extracted. Figures 8 (b), (f), (j), and (n) represent the LMS parametric scenes corresponding to (a), (e), (i), and (m), wherein the corresponding area of the polygon was assigned to every spatial location. Even this simple scalar representation of LMS yielded significantly improved contrast between the foreground and background objects. Similarly, Figures 8 (c), (g), (k), and (o) represent the corresponding parametric LMS images for the "extent" feature. An additional 24 LMS features were also extracted from each spatial location in each of (a), (e), (i), and (m). Principle component analysis (PCA) was applied to the 24 dimensional LMS feature vector. For each spatial location, the three principal eigenvectors were scaled into RGB space and the PCA representations were illustrated in Figures 8(d), (h), (1) and (p). Both the individual scalar and tensor based LMS representations of the OCa images in Figure 8 reveals a distinct separation between the stromal and tumor image patches, suggesting fundamental differences in local heterogeneity and structure between the 2 different tissue classes and successfully captured by LMS.

[000130] Quantitative evaluation of the LMS features in terms of their ability to discriminate between stromal and tumor ovarian cancer patches was performed via the Probabilistic Boosting Tree (PBT) classifier. Figures 9 and 10 show the corresponding the Receiver Operating

Characteristic (ROC) curves and area under the receive operation characteristic curve (AUC) values for different runs of the PBT classifier. For 5 different runs of randomized cross- validation, the variance in AUC was small (see Table 2).

[000131] Table 2. Area under the receive operation characteristic curve (AUC) and corresponding computation time for# e {72,72,24,24} and T e {l 00,200,100,200} . (Note that the variance in AUC is small for different parameter settings.)

[000132] While the described invention has been described with reference to the specific embodiments thereof it should be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the true spirit and scope of the invention. In addition, many modifications may be made to adopt a particular situation, material, composition of matter, process, process step or steps, to the objective spirit and scope of the described invention. All such modifications are intended to be within the scope of the claims appended hereto.

Claims

CLAIMS We claim:

1 . A method for classifying and registering a region of interest (ROI) in an image by utilizing local morphologic scale (LMS), the method comprising:

(c) modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets the obstruction, each particle loses its velocity and changes its trajectory;

(f) classifying and registering the local morphologic scale (LMS) signature.

2. The method according to claim 1, wherein the obstruction in (c) is in a form of high local gradients.

3. The method according to claim 1 , wherein the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving.

4. The method according to claim 1, wherein the local morphologic scale (LMS) is classified and registered by parallelized computations.

5. The method according to claim 1 , wherein the image is a biological image.

6. The method according to claim 1, wherein the image is a histological image.

7. The method according to claim 1 , wherein the image comprises a magnetic resonance image, a positron emission tomography (PET) image, and a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image.

8. The method according to claim 7, wherein the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2-weighted image, or a combination thereof.

9. The method according to claim 1, wherein the tissue sample is a tissue microarray (TMA).

10. The method according to claim 1 , wherein the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample.

1 1 . The method according to claim 1, wherein the tissue sample is a cancer tissue sample.

12. The method according to claim 11 , wherein the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

13. The method according to claim 1 , wherein the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

14. The method according to claim 1, wherein at least one of steps (a), (b), (c), (d), (e), and (f) is performed using a computer.

15. The method according to claim 1, wherein at least one of steps (a), (b), (c), (d), (e), and (f) is performed automatically.

16. A computing device comprising: a processor; and a storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

(b) logic executed by the processor for initializing a plurality of particles at the region of interest (ROI) in the image, wherein an initialization causes each particle of the plurality of particles to emanate from the region of interest (ROI) in a plurality of radial directions and to move with an initial velocity in a plurality of trajectories;

(c) logic executed by the processor for modeling the plurality of trajectories of the plurality of particles moving away from the region of interest (ROI) based on a response of each particle to any obstruction in its path via a physics-based system such that if the particle moves along an unimpeded trajectory, the particle gains velocity, whereas if the particle meets the obstruction, each particle loses its velocity and changes its trajectory;

17. The computing device according to claim 16, wherein the obstruction in (c) is in a form of high local gradients.

18. The computing device according to claim 16, wherein the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving.

19. The computing device according to claim 16, wherein the local morphologic scale (LMS) signature is classified and registered by parallelized computations.

20. The computing device according to claim 16, wherein the image is a biological image.

21 . The computing device according to claim 16, wherein the image is a histological image.

22. The computing device according to claim 16, wherein the image comprises a magnetic resonance image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image.

23. The computing device according to claim 22, wherein the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2 -weighted image, or a combination thereof.

24. The computing device according to claim 16, wherein the tissue sample is a tissue microarray (TMA).

25. The computing device according to claim 16, wherein the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample.

26. The computing device according to claim 16, wherein the tissue sample is a cancer tissue sample.

27. The computing device according to claim 26, wherein the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

28. The computing device according to claim 16, wherein the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.

29. A computer-readable storage medium tangibly storing thereon computer program instructions capable of being executed by a computer processor of a computing device, the computer program instructions defining steps of:

(f) classifying and registering the local morphologic scale (LMS) signature.

30. The computer readable storage medium according to claim 29, wherein the obstruction in (c) is in a form of high local gradients.

3 1 . The computer readable storage medium according to claim 29, wherein the modeling step (c) is repeated until the velocity of each particle reaches zero and the particle stops moving.

32. The computer readable storage medium according to claim 29, wherein the local morphologic scale (LMS) signature is classified and registered by parallelized computations.

33. The computer readable storage medium according to claim 29, wherein the image is a biological image.

34. The computer readable storage medium according to claim 29, wherein the image is a histological image.

35. The computer readable storage medium according to claim 29, wherein the image comprises a magnetic resonance image, a positron emission tomography (PET) image, a single photon emission computed tomography (SPECT) image, an ultrasound image, an x-ray image, or a computed tomography (CT) image.

36. The computer readable storage medium according to claim 35, wherein the magnetic resonance imaging (MRI) image comprises a diffusion- weighted image, a Tl -weighted image, a T2 -weighted image, or a combination thereof.

37. The computer readable storage medium according to claim 29, wherein the tissue sample is a tissue microarray (TMA).

38. The computer readable storage medium according to claim 29, wherein the tissue sample is selected from the group consisting of a brain tissue sample, a heart tissue sample, a liver tissue sample, a kidney tissue sample, an ovary tissue sample, a breast tissue sample, a lymph node tissue sample, a thyroid tissue sample, a stomach tissue sample, a lung tissue sample, and a prostate tissue sample.

39. The computer readable storage medium according to claim 29, wherein the tissue sample is a cancer tissue sample.

40. The computer readable storage medium according to claim 39, wherein the cancer is selected from the group consisting of an ovarian cancer, a brain cancer, a heart cancer, a liver cancer, a kidney cancer, a breast cancer, a thyroid cancer, a lymph node cancer, a stomach cancer, a lung cancer, and a prostate cancer.

41 . The computer readable storage medium according to claim 29, wherein the classifying and registering step (e) is carried out by a Monte-Carlo sampling technique.