JP7420401B2 - Computational System Pathology Spatial Analysis Platform for Multiparameter Cellular and Subcellular Image Data in Situ or In Vitro - Google Patents

Description

The present invention relates to digital pathology, and in particular to comprehensive computational system pathology spatial analysis (CSPSA) capable of integrating, visualizing, and modeling high-dimensional cellular and subcellular resolution image data in situ or in vitro. ) regarding computer platforms.

Digital pathology refers to the acquisition, storage, and display of histologically stained tissue samples and was initially developed for niche applications such as second opinion telepathology, immunostain interpretation, and intraoperative telepathology. It is attracting attention. A large amount of patient data, typically consisting of 3 to 50 slides, is generated from a biopsy specimen and evaluated visually by a pathologist under a microscope, but with digital technology, it can be evaluated by looking at a high-resolution monitor. can. Current workflow practices are time-consuming, error-prone, and subjective due to the manual labor involved.

Cancer is a heterogeneous disease. In hematoxylin and eosin (H&E) stained tissue images, heterogeneity is characterized by the presence of various histological structures such as carcinoma in situ, invasive carcinoma, adipose tissue, blood vessels and normal ducts. Furthermore, in many malignancies, molecular and cellular heterogeneity is a prominent feature between tumors in different patients, between different sites of neoplasia in the same patient, and within a single tumor. It has become. Intratumoral heterogeneity involves phenotypically distinct cancer cell clonal subpopulations and other cell types comprising the tumor microenvironment (TME). These cancer cell clonal subpopulations and other cell types include local bone marrow-derived stromal stem and progenitor cells, subclasses of immunoinflammatory cells that are either tumor-promoting or tumor-killing, and cancer-associated fibroblasts. There are cells, endothelial cells, and pericytes. The TME can be viewed as an evolving ecosystem in which cancer cells engage in heterotypic interactions with these other cell types, use available resources to proliferate, and survive. From this point of view, spatial interrelationships between cell types within the TME (ie, spatial heterogeneity) are considered to be one of the main factors of disease progression and treatment resistance. Therefore, it is imperative to characterize the spatial heterogeneity within the TME to correctly determine specific disease subtypes and identify the optimal treatment course for individual patients.

To date, intratumoral heterogeneity has been investigated using three major approaches. The first approach is to take a core sample from a specific area of the tumor and measure the population average. Sample heterogeneity is measured by analyzing multiple cores within the tumor using multiple techniques such as whole exome analysis, epigenetics, proteomics, and metabolomics. A second approach involves "single cell analysis" using the methods described above, RNASeq, imaging or flow cytometry after isolation of cells from tissue. A third approach utilizes the spatial resolution of light microscopy imaging to maintain spatial context and in combination with molecular-specific labels to measure cellular biomarkers in situ. Biomarkers can identify cell type, state of activation (eg, phosphorylation of target proteins), and intracellular function. While each of these approaches has a certain level of effectiveness, they have various drawbacks and limitations.

Furthermore, one of the biggest problems in assessing the clinical significance of tumor heterogeneity is the lack of sophisticated tools for spatial analysis of multiparametric cellular and subcellular image data.

In certain embodiments, methods are provided for analyzing disease progression from multiparameter cellular and subcellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. The method of the present invention is based on the steps of: generating a global quantification of spatial heterogeneity between cells of several different predetermined phenotypes in multiparameter cellular and subcellular image data; identifying a plurality of microdomains for a plurality of tissue samples, each microdomain being associated with a respective one of the plurality of tissue samples; constructing a weighted graph. The weighted graph has a plurality of nodes and a plurality of edges, each node being located between a pair of said nodes, in the weighted graph each node is a particular one of the microdomains, and the weighted graph The edge between each pair of microdomains in indicates the degree of similarity between that pair of microdomains.

In another embodiment, a system is provided for analyzing disease progression from multiparameter cellular and subcellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. . The system of the present invention comprises: (i) spatially configured to generate global quantification of spatial heterogeneity between cells of several different predetermined phenotypes in multi-parameter cellular and intracellular image data; (ii) a microdomain identification component configured to identify a plurality of microdomains for a plurality of tissue samples based on the global quantification, wherein each microdomain (iii) a weighted graph component that constructs a weighted graph for the multiparameter cellular and subcellular image data, the weighted graph being associated with a respective one of the tissue samples; , has a plurality of nodes and a plurality of edges, each located between a pair of nodes, in a weighted graph, each node being a particular one of the microdomains, and each node of the microdomains in the weighted graph The edges between the pairs include a processing unit that includes a weighted graph component indicating a degree of similarity between the pair of microdomains.

In yet another embodiment, a spatially informed representation of heterogeneous cell-to-cell communication is generated from multiparameter cellular and subcellular image data obtained from multiple patients or multiple tissue samples from several multicellular in vitro models. A method is provided. The method of the present invention generates a quantification of spatial heterogeneity between cells of several different predetermined phenotypes in multiparameter cellular and intracellular image data, and based on the quantification, the spatial heterogeneity between cells of several different predetermined phenotypes is identifying microdomains in one of the tissue samples, the step comprising performing phenotyping on multiparametric cellular and subcellular image data to identify the several different predetermined phenotypes; and constructing a communication graph for the microdomain. Each phenotype is a node in the communication graph, and the edges between each pair of phenotypes in the communication graph indicate the influence of one phenotype in the pair on the presence of the other phenotype in the pair. .

In yet another embodiment, a spatially informed representation of heterogeneous cell-to-cell communication is generated from multiparameter cellular and subcellular image data obtained from multiple patients or multiple tissue samples from several multicellular in vitro models. A computer system is provided for doing so. The system of the present invention comprises: (i) spatial heterogeneity configured to generate a quantification of spatial heterogeneity between cells of several different predetermined phenotypes in multiparameter cellular and intracellular image data; (ii) a spatial heterogeneity quantification component that performs cellular phenotyping on multiparameter cellular and subcellular image data to identify specific distinct predetermined phenotypes; a microdomain identification component configured to identify a plurality of microdomains for the number of tissue samples based on the number of tissue samples, each microdomain being associated with one of the number of tissue samples; a microdomain identification component; and (iii) a communication network component configured to construct a communication graph for a selected one of the plurality of microdomains, wherein each phenotype is a node of the communication graph. , an edge between each pair of phenotypes in the communication graph includes a processing unit that includes a communication network component that indicates the influence of one phenotype in the pair on the presence of the other phenotype in the pair.

In yet another form, a method of creating a personalized medical strategy for a particular patient, the particular patient being one of several patients, wherein the particular patient is one of several patients. A method is provided that is associated with multiparameter cellular and intracellular image data obtained from several tissue samples from a patient. The method of the invention includes identifying a plurality of microdomains in one of several tissue samples associated with a particular patient, the plurality of microdomains being identified based on multiparameter cellular and subcellular image data. and a step of generating a heterogeneous intercellular communication network for each of the plurality of microdomains, each heterogeneous intercellular communication network including a representation of heterogeneous intercellular communication based on spatial information of the microdomains. and the process of quantifying the spatial and temporal interdependence of microdomains based on heterogeneous intercellular communication networks, and developing medical strategies for specific patients based on the quantified results. process.

In yet another embodiment, a method is provided for representing the time evolution of a disease in a particular patient, where the particular patient is one of several patients, and where the several patients are one of several patients. associated with multi-parameter cellular and intracellular image data obtained from several tissue samples. The method of the invention includes the step of generating a geometric representation of a disease landscape for a particular patient, the geometric representation comprising a plurality of points, each point of the geometric representation having a specific (ii) describes the disease status of a particular patient at a time point in time and is based on a particular one of several tissue samples associated with the selected patient; and (ii) is based on multiparametric cellular and intracellular imaging data. (iii) a heterogeneous intercellular communication network for the microdomain that includes a representation of a heterogeneous intercellular communication network based on spatial information about the microdomain; including, including a process.

FIG. 1 is a schematic diagram of an exemplary Computational Systems Pathology Space Analysis (CSPSA) platform for in situ or in vitro multiparameter cellular and subcellular image data based on one embodiment of the disclosed concepts. FIG. 2 is a schematic diagram of an exemplary hyperplexed image stack. FIG. 3 is a flowchart illustrating a method for analyzing tumor progression/progression from multiparameter cellular and intracellular imaging data obtained from multiple tumor sections from a patient cohort, the method comprising: It can be implemented in the CSPSA platform of FIG. 1 based on the exemplary embodiment. FIG. 4 is a schematic diagram of an exemplary global spatial map. FIG. 5 is a schematic illustration of each of the predetermined dominant biomarker intensity patterns in an exemplary embodiment of the disclosed concept. FIG. 6 is a schematic diagram of a cell spatially dependent image according to one particular exemplary embodiment of the disclosed concept. FIG. 7 is a schematic diagram of an exemplary microdomain of an exemplary tissue section in accordance with one exemplary embodiment of the disclosed concepts. FIG. 8 is a schematic diagram of a weighted microdomain graph constructed for multiparameter cellular and subcellular image data in accordance with an exemplary embodiment of the disclosed concepts. FIG. 9 is a flowchart illustrating a method for generating a representation of spatially informed heterogeneous intercellular communication from multiparametric cellular and intracellular image data in accordance with another exemplary embodiment of the disclosed concepts. FIG. 10 is a schematic diagram of an example communication graph according to one example embodiment of the disclosed concepts. FIG. 11 is a schematic diagram of certain communication graphs generated based on certain exemplary embodiments in which the phenotypes are tumor cells, lymphocytes, macrophages, stroma, and necrosis. FIG. 12 is a schematic diagram of a personalized medicine strategy based on one aspect of the disclosed concepts. FIG. 13 is a schematic diagram of an exemplary cancer landscape in accordance with one aspect of the disclosed concepts.

As used herein, the singular forms "a" and "the" include plural references unless the context clearly dictates otherwise.

In this specification, the statement that two or more parts or components are "coupled" means that the connection occurs directly or indirectly, that is, through one or more intermediate parts or components. means that parts are connected or work together.

As used herein, the term "some" means one or an integer greater than one (ie, a plurality).

As used herein, the terms "component" and "system" are intended to refer to a computer-related entity that is either hardware, a combination of hardware and software, software, or running software. . For example, a component may be, and is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, and/or a computer. For example, both an application running on a server and a server may be components. One or more components may reside within a process and/or thread of execution, and a component may be local to one computer and/or distributed between two or more computers. . Although several ways of displaying information to a user have been illustrated and described as screenshots and in particular diagrams or graphs, those skilled in the relevant art will recognize that various other alternatives may be employed. Probably.

As used herein, the term "multiparameter cellular and intracellular image data" refers to data obtained by generating several images from several sections of tissue, including cells or cells in those sections of tissue. refers to data that provides information about multiple measurable parameters at an internal level. Multiparameter cellular and intracellular image data may be generated by several different imaging techniques, including (1) transmitted light imaging by IHC using several biomarkers; (2) multiparameter Immunofluorescence imaging, including both plex imaging (1-7 biomarkers) and hyperplex imaging (more than 7 biomarkers); (3) toponome imaging; (4) matrix-assisted laser desorption/ ionization mass spectrometry imaging (MALDI MSI), (5) complementary spatial imaging such as FISH, MxFISH, FISHSEQ, or CyTOF, (6) multiparameter ion beam imaging, and (7) in vitro imaging of experimental models. Not limited to these. In addition, generating several biomarker images from several sections of tissue, for example, but not limited to, by labeling each section of tissue with multiple different biomarkers, allows for multiparameter cellular and intracellular imaging. Image data may be generated.

As used herein, the term "spatial map" refers to a representation of some quantified spatial statistics that shows the relationship between cells of different predetermined phenotypes in a set of multiparameter cellular and intracellular imaging data, and/or the representation of spatial statistics in a visually perceptible form. For example, without limitation, the spatial map may be a pointwise mutual information (PMI) map generated in the manner described in PCT Application No. PCT/US2016/036825 and US Patent Application Publication No. 2018/0204085. , both of their names are “Systems and Methods for Finding Regions of Interest in Hematoxylin and Eosin (H&E) Stained Tissue Images and Quantify ing Intratumor Cellular Spatial Heterogeneity in Multiplexed/Hyperplexed Fluorescence Tissue Images, the disclosures of which are incorporated by reference. It becomes part of this specification.

As used herein, the term "microdomain" refers to a spatial arrangement (or motif) of phenotypically distinct cells in a tissue sample, which spatial arrangement is characterized by spatial intratumoral heterogeneity. occur and are associated with one or more outcome specific variables (eg, time to recurrence). Microdomains may be identified from multiparameter cellular and subcellular image data according to any of several known or later developed methods. Such methods include U.S. Provisional Application No. 62/675,832, “Predicting the Recurrence Risk of Cancer Patients From Primary Tumors with Multiplexed Immune offluorescence Biomarkers and Their Spatial Correlation Statistics” and PCT Application No. PCT/US2019/033662 “System and Method for Predicting the Risk of Cancer Recurrence From Spatial Multi-Parameter Cellular a nd Sub-Cellular Imaging Data for Tumors by Identifying Emergent Spatial Domain Networks Associated With Recurrence” and/or “Spag nolo , et al., Platform for Quantitative Evaluation of Spatial Intratumoral Heterogeneity in Multiplexed Fluorescence Images, Cancer Res. 2017 Nov 1;77(21):e71-e74'' and/or the method described in the Spagnolo et al. The method is implemented in the public domain THRIVE (Tumor Heterogenity Research Interactive Visualization Environment) software described, but not limited thereto. The method uses spatially resolved correlations between biomarkers as covariates in a multivariable survival model of prognostic data (e.g., recurrence) to construct a map of the spatial organization of cancer recurrence in multiplexed tissue samples. These maps depict microdomains associated with recurrence and metastatic progression.

Directional terms used herein, such as, for example, top, bottom, left, right, top, bottom, front, rear, and derivatives thereof, refer to the orientation of elements shown in the drawings and are explicitly It does not limit the scope of the claims unless explicitly stated otherwise.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS For purposes of explanation, the disclosed concepts are described in many specific details in order to provide a thorough understanding of the invention. It will be apparent, however, that the disclosed concepts may be practiced without these specific details without departing from the spirit and scope of the invention.

The disclosed concepts create a comprehensive computational systems pathology spatial analysis (CSPSA) platform that is capable of integrating, visualizing, and/or modeling high-dimensional in situ cellular and subcellularly resolved image data. provide. The CSPSA platform of the disclosed concept combines a core set of existing tools for cellular phenotyping and spatial analysis to infer spatially heterogeneous intercellular (intercellular) and intracellular communication patterns. And, in the case of cancer, it is combined with a sophisticated toolset for constructing evolutionary trees of tumors (or other diseases) from pre-cancerous origins to metastatic endpoints. These tools can be combined to quantify spatial intratumoral heterogeneity in tissue samples and in vitro models and correlate it with prognostic data, to build diagnostic and prognostic applications, and to develop personalized treatment strategies. It can be applied for both research and clinical purposes, such as designing and drug discovery. Furthermore, this platform can be enhanced by adding region-specific genomics and spatial transcriptomics data to in situ cellular and subcellular high-definition images.

FIG. 1 is a schematic diagram of an exemplary Computational Systems Pathology Spatial Analysis (CSPSA) platform 5 for in situ multiparameter cellular and subcellular image data based on embodiments of the disclosed concepts, in which Various methods described in the book can be carried out. As shown in FIG. 1, the CSPSA platform 5 provides specific multi-parameter analysis (e.g., for large patient cohorts including recurrent and non-recurrent cancer tumors, or for individual patients or individual tumors). A computing device configured to receive and store cellular and intracellular image data 10 and process the data 10 as described herein. In a non-limiting exemplary embodiment, multiparameter cellular and intracellular image data 10 is generated using multiplexed or hyperplexed immunofluorescence imaging, but as described elsewhere herein. It will be appreciated that other imaging techniques can also be used, such as those described. For example, multiparameter cellular and intracellular image data 10 may be based on an exemplary hyperplex image stack 12 for all patient data in a cohort, such as depicted in FIG.

CSPSA platform 5 may be, for example, a PC, laptop computer, tablet computer, smartphone, or other suitable computing device configured to perform the functions described herein. Not limited. CSPSA platform 5 includes an input device 15 (such as a keyboard), a display 20 (such as an LCD), and a processing device 25. A user may provide input to processing device 25 using input device 15, and processing device 25 may cause display 20 to provide information (e.g., The output signal is provided to a display 20 so that a spatial map or other spatially dependent image, microdomain image, weighted graph image and/or communication network graph image) can be displayed. The processing device 25 includes a processor and a memory. The processor may be, for example, but not limited to, a microprocessor (μP), microcontroller, application specific integrated circuit (ASIC), or other suitable processing device that interfaces with the memory. Memory refers to data storage such as the internal storage area of a computer, such as RAM, ROM, EPROM, EEPROM, FLASH, or anything else that provides storage registers, such as a computer readable medium. It may be one or more of various types of internal and/or external storage media, and may be volatile or non-volatile memory. The memory stores a number of routines executable by the processor, including routines for implementing the disclosed concepts as described herein. In particular, processing device 25 performs global quantification of intercellular spatial heterogeneity of several different predetermined phenotypes in multiparameter cellular and subcellular image data 10, as described herein. Based on the spatial heterogeneity quantification component 30 configured to generate and the global quantification generated by the spatial heterogeneity quantification component 30 from the multiparameter cellular and subcellular image data 10, a microdomain identification component 35 configured to identify a plurality of microdomains as described herein for a plurality of tumor sections; A weighted graph component 40 configured to construct a weighted graph of cellular and intracellular image data 10 and a communication network for multiparametric cellular and intracellular image data 10 in the manner described herein. a communication network component 45 configured to construct a graph.

FIG. 3 is a flowchart illustrating a method for analyzing tumor progression/progression from multiparameter cellular and intracellular imaging data 10 obtained from multiple tumor sections of a patient cohort, the method comprising: It can be implemented on the CSPSA platform 5 based on the exemplary embodiment. However, it will be appreciated that this is intended to be illustrative only, and the method steps illustrated in FIG. 3 may be implemented in other configurations and/or platforms.

The method begins at step 50, where the spatial heterogeneity quantification component 30 provides global quantification of spatial heterogeneity in the multiparameter cellular and subcellular image data 10. In an exemplary embodiment, the global quantification provided in step 50 is a global spatial map of the multiparameter cellular and subcellular image data 10. In certain embodiments, the global spatial map is the global PMI map 52, as shown in FIG. 4, mentioned elsewhere herein and incorporated by reference. PCT Application No. PCT/US2016/036825 and US Patent Application Publication No. 2018/0204085. In this exemplary embodiment, each section of tissue is labeled with a plurality of different biomarkers (e.g., ER, PR, and HER2) to generate several biomarker images from several sections of tissue. Multi-parameter cell and intracellular image data 10 are created. As described in the aforementioned application, PMI map 52 first performs cell segmentation on multiparameter cellular and intracellular image data 10 (i.e., for each "slide" thereof). generated by. Any of several suitable cell segmentation algorithms known or hereafter developed may be employed. Spatial location and biomarker intensity data are then obtained for each cell, and based on the biomarker intensity configuration of the cell, each cell is assigned one of the predetermined phenotypes (each phenotype is a predetermined dominant biotype). marker intensity pattern). FIG. 5 shows a schematic representation 56 of each of the predetermined predominant biomarker intensity patterns, numbered 1 to 8, in an exemplary embodiment. In an exemplary embodiment, each schematic representation 56 is provided with a unique color so that the schematic representations can be easily distinguished from each other. The cell assignments described herein and the schematic representation shown in FIG. 5 can be used to generate cell spatially dependent images that visually demonstrate the heterogeneity of a tissue sample of interest. FIG. 6 shows a cell spatially dependent image 58 according to one particular exemplary embodiment of the disclosed concepts. As shown in FIG. 6, the cell spatial dependence image 58 uses a schematic representation 56 to show the spatial dependence between cells of the subject's slide. A spatial network is then constructed to represent the composition of key biomarker intensity patterns in the subject's slides. Then, by generating a PMI map 52 as shown in FIG. 4, the non-uniformity of the subject's slides is quantified. In an exemplary embodiment, spatial network and PMI maps 52 are generated as shown below.

A network is constructed over the subject's slides to represent the spatial organization of biomarker patterns in the biomarker images (ie, tissue/tumor samples) of the subject's slides. The construction of a spatial network for tumor samples inherently links cellular biomarker intensity data (network nodes) and spatial data (network edges). The assumption in network construction is that cells have the ability to communicate with nearby cells up to a certain limit, e.g. up to 250 μm, and that the ability of cells to communicate within that limit depends on their distance. . Accordingly, a probability distribution in an exemplary embodiment is calculated for the distance between a cell and its 10 nearest neighbors in the subject's slide. A hard limit was chosen based on the median value of this distribution multiplied by 1.5 (to estimate the standard deviation), and the cells of the network were only connected within this limit. Edges between cells in the network are then weighted by the distance between adjacent cells.

In an exemplary embodiment, pointwise mutual information (PMI) is then used to evaluate the association between each pair of biomarker patterns in the dictionary, and thus between different cell phenotypes, for a subject's slides. be done. This metric captures common statistical associations, both linear and nonlinear, and previous studies have used linear metrics such as Spearman's rho coefficient. Once the PMI is calculated for each pair of biomarker patterns, all measures of association in the subject's slide data are displayed in the PMI map 52. The use of PMI in this embodiment is only exemplary and other methods of defining spatial relationships may be used, such as Spagnolo DM, Al-Kofahi Y, Zhu P, Lezon TR, Gough A, Stern AM, Lee AV. , Ginty F, Sarachan B, Taylor DL, Chennubhotla SC, Platform for Quantitative Evaluation of Spatial Intratumoral Heterogeneity i n Multiplexed Fluorescence Images, Cancer Res. 2017 Nov 1 and Nguyen, L. , Tosun, B. , Fine, J. , Lee, A. , Taylor, L. , Chennubhotla, C. (2017), Spatial statistics for segmenting histological structures in H&E stained tissue images, IEEE Trans Med Imaging. The methods described in 2017 Mar 16 may be employed with respect to the disclosed concepts.

The exemplary PMI map 52 describes the relationships of different cellular phenotypes within the microenvironment of a subject's slide. In particular, entries in the PMI map 52 indicate that a specific spatial interaction between two phenotypes (referenced by row and column numbers) is a random (or background) interaction across all phenotypes. It shows how often an interaction occurs in the dataset compared to the interaction predicted by the distribution. Entries in a first color, such as red, indicate a strong spatial association between phenotypes, and entries in a second color, such as black, indicate a lack of colocalization (a weak spatial association between phenotypes). Relevance). Other colors may be used to indicate other associations. For example, PMI entries colored in a third color, such as green, indicate no better association than a random distribution of cell phenotypes across the dataset. Additionally, the PMI map 52 identifies anti-associations with entries shown in a fourth color, such as blue (for example, if Phenotype 1 rarely occurs spatially near Phenotype 3). I can express it.

Referring again to FIG. 3, following step 50, the method then proceeds to step 55. In step 55, the microdomain identification component 35 determines whether (e.g., other The microdomain or microdomains are identified and located (as described in the section). For example, a cancer cell type and its co-occurring particular immune cell type may have high spatial co-occurrence values. Using cancer and immune cell combinations as seed points, mine global quantification (e.g., spatial maps) to propagate phenotypic associations to distance cutoffs in any tissue section. Microdomains can be generated by growing a spatial network of cells (e.g., about 100 cells) around a seed based on a cell. Such an exemplary microdomain 62 is illustrated in an exemplary tissue section 64 shown in FIG.

Next, in step 60, a local quantification of spatial heterogeneity (eg, a local spatial map such as a local PMI map) is determined for each microdomain identified in step 55. Furthermore, each local quantification determined in each tissue section is used to define a degree of similarity between pairs of microdomains in the tissue section. In particular, in an exemplary embodiment, given two microdomains A and B, (i) the difference in the relative abundance of cells in microdomain A and microdomain B, and (ii) their local A distance function is defined that is a combination of quantification (local spatial maps such as PMI _A and PMI _B , or other methods may be used, as mentioned herein). be done. This procedure is repeated for all pairs of microdomains across all multiparameter cellular and intracellular image data 10. This process determines a weighted similarity measure for each pair of microdomains in each tissue section.

Next, in step 65, a weighted microdomain graph 66, shown in exemplary form in FIG. 8, is constructed for the multiparameter cellular and intracellular image data 10. Each node 68 of the weighted microdomain graph 66 is one of the microdomains generated in step 55, and the edges 72 connecting each pair of microdomains are determined by their determined similarity (e.g., as described above). weighted similarity value). Weighted microdomain graph 66 may then be displayed on display 20.

The weighted microdomain graph 66 can then be further analyzed to understand the progression of the phenotype during the process of metastasis, as shown in step 70 of FIG. Although ground truth labels exist for each entire tissue section of multiparameter cellular and intracellular image data 10 as originating from either a recurrent (R) or non-recurrent (NR) patient, a priori it is important to know which microdomains Note that no knowledge is available to ascertain which is most discriminatory between the two categories. In an exemplary embodiment, label R and label NR are transferred from the tissue section to corresponding microdomains. The probability propagation is then repeated several times to improve the labels and separate cliques of microdomains in the weighted microdomain graph 66 that are uniformly R or NR. With the improved reliability of label assignments, a betweenness centrality measure can be used to identify the microdomains most commonly transferred on the NR to R path. To mimic the evolutionary trajectory, a random walk of the weighted microdomain graph 66 can be performed, noting that some microdomains are only found in either observed or unobserved metastasis data. It is being By comparing the relative probabilities of the NR to R pathway, classes of events important for metastasis can be identified. The average first passage time of the random walk is then used to identify microdomains that are prone to metastasis as the set of nodes for which the probability that the random walk reaches either R or NR is 0.5. be able to. Note that the potentially translocable microdomain is the midpoint of the NR to R evolutionary trajectory. Note that in the exemplary embodiment, the steps just described are performed by components of the CSPSA platform 5 and the results are displayed to the user on the display 20 of the CSPSA platform 5. This aspect of the disclosed concepts thus defines, identifies, and compares phenotypes important for metastasis (phenotypic evolution) as described herein. bring ability. In addition, it is understood that cancer is only one example of an area where the disclosed concepts are applicable, and that the disclosed concepts are also applicable to other diseased tissues, such as neurodegenerative diseases, metabolic diseases, inflammatory diseases, among others. There will be.

FIG. 9 is a flowchart illustrating a method for generating a representation of heterogeneous intercellular communication based on spatial information from multiparameter cellular and intracellular image data 10 that may be obtained from several tumor sections from a patient in this embodiment. may be implemented in the CSPSA platform 5 according to another exemplary embodiment of the disclosed concepts. However, it will be appreciated that this is intended to be illustrative only, and the method steps illustrated in FIG. 9 may be implemented in other configurations and/or platforms.

The method begins at step 75, where the spatial heterogeneity quantification component 30 performs spatial heterogeneity quantification in the multiparameter cellular and subcellular image data 10, as described in detail elsewhere herein. Provides quantification of heterogeneity. As mentioned elsewhere herein, the quantification of spatial heterogeneity performed in step 75 includes performing cell phenotyping on the multiparameter cellular and subcellular image data 10 to including identifying a predetermined phenotype. In an exemplary embodiment, the quantification provided in step 75 is a spatial map of the multiparameter cellular and subcellular image data 10. In certain embodiments, the spatial map is referred to elsewhere herein and is incorporated herein by reference in PCT Application No. PCT/US2016/036825 and U.S. Patent Application Publication No. 2018/ It is produced by the method described in No. 0204085. In this exemplary embodiment, several biomarker images are generated from several sections of tissue by labeling each section of tissue with multiple different biomarkers (e.g., ER, PR, and HER2). , multiparameter cellular and intracellular image data 10 may be generated. In a non-limiting exemplary embodiment, multiparameter cellular and intracellular image data 10 is generated using multiplexed or hyperplexed immunofluorescence imaging, but as described elsewhere herein. It will be appreciated that other imaging techniques can also be used, such as those described.

Next, at step 80, the microdomain identification component 35 determines, as described herein, in each of the tumor sections of the multiparameter cellular and intracellular image data 10 based on the quantification provided at step 75. The microdomain or microdomains are identified and located according to the method. Next, at step 85, one or more of the identified microdomains are selected. The method then proceeds to step 90.

At step 90, communication network component 45 constructs a communication graph for each of the selected microdomains. The communication graph may be displayed on display 20. An exemplary communication graph 92 is shown in FIG. As shown in FIG. 10, in the exemplary embodiment, each of the phenotypes is a node 94 of communication graph 92, and each of the phenotypes of each pair of phenotypes (each pair of nodes 94) of communication graph 92 is The edge 96 in between shows the influence of the phenotype of one of the pairs on the presence of the other phenotype of the pair. In the exemplary embodiment, each phenotype (each node 94) of communication graph 92 is represented by a data vector obtained from multiparameter cellular and intracellular image data 10. The edge 96 of the communication graph 92 of the exemplary embodiment also determines, for each pair of phenotypes, the value of a linear or non-linear correlation coefficient between the data vectors of the pair of phenotypes to (after removing the confounding effects of phenotypes other than the paired phenotypes). In addition, for each numerical relationship, a directionality is preferably established based on several receptor-ligand databases of relevant biomarkers. Thus, in this embodiment, the edges 96 between each pair of nodes 94 indicate a determined numerical relationship and a determined directionality. In an exemplary embodiment, edges 96 may be colored to represent a particular correlation coefficient value (eg, degree of positive or negative correlation).

Additionally, in the exemplary embodiment, each data vector of each node 94 is a vector of expression values for a number of predetermined biomarkers, where the predetermined biomarkers include multiparameter cellular and intracellular image data. The specific biomarkers used in generating 10. For example, each vector of expression values may be a biomarker intensity pattern for a given biomarker determined in step 75. Additionally, each data vector may be analyzed and biologically interpreted to determine a number of activation states 98 for each of the phenotypes (each node 94), which activation states 98 are connected to the communication graph 94. include. For example, the biomarker ALDH1 is a surrogate for tumor cell stemness, and the biomarker PD-L1 determines mutational burden.

FIG. 11 is a schematic diagram of a particular communication graph generated based on a particular exemplary embodiment in which the phenotypes (nodes 94) are tumor cells, lymphocytes, macrophages, stroma, and necrosis. Also shown are various edge effects 96 and various activation states 98.

One of the outcomes of the mapping described herein is the inference of disease progression pathways based on the local tumor microenvironment (TME) and subsequent listing of known molecular targets in that pathway. be. Following this, machine learning tools (eg, Balestra Web) can be used to predict drug target interactions based on spatial relationships. This could lead to the repurposing of drugs as well as the development of new treatments. Additionally, described in detail below are several exemplary clinical applications (e.g., applications of CSPA Platform 5) that may incorporate various aspects of the disclosed concepts, including: : (1) drug discovery and personalized medicine strategies using spatially modulated computational system pathology; (2) cancer/disease landscapes, which are geometric representations of multiparametric readouts of patient cancer/disease status; and (3) ) Application of the CSPA platform for in vitro models for basic research and clinical translation.

In certain exemplary embodiments, the disclosed concepts can be used for drug discovery and personalized medicine strategies using spatial modulation computational system pathology. More specifically, spatial intratumoral heterogeneity is an important feature in determining the temporal evolution of cancer and patient fate. This heterogeneity is reflected in the diversity of heterogeneous intercellular communication networks embedded in microdomains, making the resulting systems biology patient dependent. Current treatment strategies are designed for the average patient and do not reflect the patient's unique systems biology. Patient-specific systems pathology identifies microdomains/regions of interest, characterizes their underlying communication networks, and precisely quantifies microdomain interdependencies both spatially and temporally. It starts with doing. This knowledge is also essential for new drug design strategies. This strategy can also be enhanced by using region-specific genomics.

FIG. 12 is a schematic illustration of a personalized medicine strategy based on this aspect of the disclosed concept, ranging from tissue sections to personalized therapeutic treatments. Figure 12 shows, from left to right, i) multiparameter images at cell/subcellular resolution, ii) identification of regions of interest (microdomains) within the image, and iii) specific connection weights and spaces between cells/microdomains. iv) combining information to extract accurate systems biology; and v) defining personalized treatment strategies.

As shown in FIG. 12, in the illustrated exemplary embodiment, microdomain 1 and microdomain 2 of the sample were found to be phenotypically different for each individual patient. A phenotyping algorithm as described herein is applied to this tissue sample. A heterogeneous intercellular network is then constructed for each microdomain as described herein. Prognostic tests applied separately to microdomain 1 and microdomain 2 reveal that both microdomains have a low risk of recurrence. However, when the microdomains are in close proximity, the risk of recurrence increases dramatically, suggesting that information flows between microdomains 1 and 2. Thus, in this aspect of the disclosed concept, the heterogeneous intercellular network signals which pathways are activated and what the relationship is between the two networks. For example, WNT signaling is upregulated in microdomain 2 but downregulated in microdomain 1, and TGFβ is upregulated in microdomain 1 but downregulated in microdomain 2. Ru.

In this aspect of the disclosed concept, pathways and microdomain networks are identified, and thus also information flows that must be inhibited to slow the progression of cancer in a particular individual. Using this information, personalized medicine strategies can be developed. In another specific exemplary embodiment, the disclosed concepts can be used to accurately describe the temporal evolution of a disease, such as cancer, in a particular patient. Thus, the disclosed concept defines a cancer landscape, which is a geometric representation of a multi-parametric readout of a patient's cancer status. Each point in the cancer landscape represents the patient's cancer status, such as precancerous, early stage, or invasive. The path followed in this landscape describes the temporal evolution of that particular patient's disease. Current treatments use cancer landscapes averaged across patient populations. Unless an individual patient's profile perfectly matches the average profile, predictions made with the average model may be inaccurate.

FIG. 13 shows an example cancer landscape that includes a geometric representation, where locations on the landscape represent particular example patient conditions. In Figure 13, the progression of colorectal cancer is taken as an example, and the depression labeled 1 indicates a precancerous state, the depression labeled 2 indicates a small polyp state, and the depression labeled 3 indicates a large polyp state. , the depression labeled 4 represents an invasive colon cancer condition. The arrows within the geometric representation indicate the path the patient follows from pre-cancer to invasive cancer. From this model, the dynamics of cancer progression can be inferred.

In accordance with one aspect of this embodiment of the disclosed concept, a retrospective patient cohort was developed to construct a comprehensive library of microdomains as described herein for tissue evolution from precancerous to metastatic. is required. A specific heterologous intercellular communication network, as described herein, is associated with each microdomain. In addition, each heterogeneous intercellular communication network has a systems biology model in the form of a system of ordinary differential equations. A system of ordinary differential equations defines the kinetics of the cancer landscape. In prospective studies, this kinetics can help predict the temporal evolution of microdomains from premalignant state to metastasis. Kinetic models also serve as a substitute for models of synthetic tissue development.

In yet another specific exemplary embodiment, the disclosed concepts may be used in combination with in vitro models for basic research and clinical translation. More specifically, in two specific applications of the disclosed concept just described, the platform of the disclosed concept was applied to in situ hyperplex single cell resolution solid tumor images. In this embodiment, the same platform is applied to image data from an in vitro microphysiological model. Multicellular in vitro models summarize human tissues that are applicable for investigating mechanisms of disease progression in vitro, testing drugs, and characterizing the structural organization and content of these models for use in transplantation. It enables the study of spatiotemporal cellular heterogeneity and communication between different types of cells.

In vitro models include the form of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiological systems. The goal of these systems was to approach physiologically relevant biomimetics. 2D cultures are cell cultures that grow by surface attachment. Since they deploy 2D communication networks, the exchange of information is limited to two dimensions. 3D spheroids are clusters of cells that grow spatially. Although cells are grown in an artificial environment, they better mimic in vivo growth conditions since cells can interact and grow in all directions. Organoids are extremely small, self-organized, three-dimensional tissue cultures. They are usually made from stem cells. Organoids can be created to reproduce most of the complexity of an organ, or they can be induced to express selected aspects of an organ, such as the generation of only certain cell types. Regeneration of organ function, even if only partially, means regeneration of systems biology and heterologous cell-cell communication under controlled conditions in vitro. Biomimetic microphysiological systems are the first step in mimicking organ function with respect to organoids. The 3D microfluidic channel cell culture chip mimics all the activities, mechanics, and physiological responses of an entire organ. This makes it possible to more accurately represent systems biology and communication networks between different types of cells.

In one or more aspects of this particular embodiment, spatiotemporal cellular heterogeneity and heterogeneous cell-to-cell communication are monitored by constructing a timeline of images and comparing the evolution of inferred networks in the model. It's okay to be. The level of complexity of in vitro systems, from 2D cultures to biomimetic MPSs, is reflected in the complexity of systems biology models that can be developed. Although applicable to all in vitro models, biomimetic models constructed by layering or bioprinting various cell types in appropriate 3D tissues benefit most from defining spatial relationships within the model. will receive. The same type of system pathology defined in the two specific applications of the disclosed concept discussed earlier will be valid here as well. The goal is to demonstrate that the in vitro model reflects the organizational and spatial relationships identified in the in situ tissues/organs under consideration and that systems biology can be reproduced in vitro. Models can be established and studied in "normal" healthy conditions, or cells from patients with disease and/or induced pluripotency from patients that have differentiated and matured into the cell types required for the model. Stem cells (iPSCs) may be used to establish disease models. At different time points and after the selected treatments, the model is fixed, embedded, sectioned, and labeled in the same way as the patient's tissue. A hyperplex imaging method is then applied to the labeled tissue sections. The computational system pathology analysis of the disclosed concepts described herein may then be applied to tissue sections obtained from the model.

Finally, although we have discussed image data obtained from tumor sections, the disclosed concepts are applicable to image data obtained from other types of tissue sections, as well as from solid un-sectioned sections. It will be appreciated that it is also applicable to image data obtained from unsectioned samples using imaging modalities that can penetrate the sample.

In the claims, any signs placed between parentheses shall not be construed as limiting the claim. The word ``comprising'' or ``comprising'' does not exclude the presence of elements or steps other than those listed in a claim. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The word "a" preceding an element does not exclude the existence of more than one such element. In any device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain elements are recited in mutually different dependent claims does not indicate that a combination of these elements cannot be used.

Although the invention has been described in detail for the purpose of illustrating the invention in terms of what is presently believed to be the most practical and preferred embodiment, such details are only for that purpose. It is intended that the invention not be limited to the disclosed embodiments, but on the contrary, it is intended to cover modifications and equivalent constructions within the spirit and scope of the appended claims. Understand that. For example, it is understood that the invention contemplates that, to the extent possible, one or more features of any embodiment may be combined with one or more features of any other embodiment. .

Claims (43)

A method for analyzing disease progression from multiparameter cellular and subcellular imaging data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models, comprising:
generating a global quantification of spatial heterogeneity between cells of several different predetermined phenotypes in the multiparameter cellular and subcellular image data;
identifying a plurality of microdomains for a plurality of tissue sections from the multiparameter cellular and subcellular image data based on the global quantification, each microdomain being a respective one of the plurality of tissue sections; A process related to
constructing a weighted graph for the multi-parameter cellular and intracellular image data, the weighted graph showing the progression of the several different predetermined phenotypes, each comprising a plurality of nodes, a plurality of edges located between pairs of nodes, in the weighted graph, each node being a particular one of the plurality of microdomains; the edges between indicate the degree of similarity between the pair of microdomains, and the weighted graph can be analyzed to understand the progression of the disease;
method including. The method further includes determining, for each pair of microdomains in the plurality of microdomains, a weighted similarity between the pair of microdomains, and an edge between each pair of microdomains in the weighted graph 2. The method of claim 1, wherein the method is a weighted similarity determined for the pair of domains. For each identified microdomain, the method further includes determining a local quantification of intercellular spatial heterogeneity, and determining a weighted similarity between each pair of microdomains for each microdomain of the pair. 3. The method according to claim 2, which is based on local quantification of domains. 2. The method of claim 1, wherein the global quantification is a spatial map of the multiparameter cellular and subcellular image data. 5. The method of claim 4, wherein the spatial map is a pointwise mutual information (PMI) map. 2. The method of claim 1, wherein the multiparameter cellular and intracellular image data comprises multiplexed or hyperplexed immunofluorescence image data. 2. The method of claim 1, wherein the several predetermined phenotypes include a predetermined set of dominant biomarker intensity patterns. 2. The method of claim 1, wherein some of the specific microdomains are associated with a prognosis-specific variable, said prognosis-specific variable being in the form of time to recurrence or indicative of progression of said disease. . 2. The method of claim 1, wherein some of the specific microdomains are associated with prognostic specific variables indicative of disease progression. 10. The method of claim 9, further comprising proposing a treatment strategy based on the prognosis-specific variables. 10. The method of claim 9, wherein the plurality of tissue samples are tumor samples and the prognosis-specific variable is indicative of metastatic potential. A non-transitory computer-readable medium storing one or more programs containing instructions that, when executed by a computer, cause the computer to perform the method of claim 1. A system for analyzing disease progression from multiparameter cellular and subcellular imaging data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models, comprising:
A processing device is provided, and the processing device includes:
a spatial heterogeneity quantification component configured to generate a global quantification of spatial heterogeneity between cells of several different predetermined phenotypes in the multiparameter cellular and subcellular image data; ,
a microdomain identification component configured to identify a plurality of microdomains for a plurality of tissue sections from the multiparameter cellular and intracellular image data based on the global quantification, each microdomain being configured to identify a plurality of microdomains from the multiparameter cellular and subcellular image data; a microdomain identification component associated with an individual one of the tissue sections;
a weighted graph component for constructing a weighted graph for the multiparameter cellular and intracellular image data, the weighted graph showing the progression of the several different predetermined phenotypes, the weighted graph comprising a plurality of nodes and , each node being a particular one of the microdomains in the weighted graph, and each node being a particular one of the microdomains between each pair of microdomains in the weighted graph. a weighted graph component, the edges of which indicate a degree of similarity between the pair of microdomains, and the weighted graph can be analyzed to understand the progression of the disease;
system, including. The weighted graph component is further configured to determine, for each pair of microdomains in the plurality of microdomains, a weighted similarity between the pair of microdomains; 14. The system of claim 13, wherein the edge between each pair is a weighted similarity determined for that pair of microdomains. The weighted graph component is further configured to determine, for each particular microdomain, a local quantification of spatial heterogeneity between cells, and the weighted similarity between each pair of microdomains 15. The system of claim 14, wherein the degree determination is based on local quantification of the paired microdomains. 14. The system of claim 13, wherein the global quantification is a spatial map of the multiparameter cellular and subcellular image data. 17. The system of claim 16, wherein the spatial map is a pointwise mutual information (PMI) map. 14. The system of claim 13, wherein the multiparameter cellular and intracellular image data comprises multiplexed or hyperplexed immunofluorescence image data. 14. The system of claim 13, wherein the several predetermined phenotypes include a predetermined set of dominant biomarker intensity patterns. 14. The system of claim 13, wherein some of the specific microdomains are associated with a prognosis-specific variable, said prognosis-specific variable being in the form of time to recurrence or indicative of progression of said disease. . 14. The system of claim 13, wherein some of the specific microdomains are associated with prognostic specific variables indicative of disease progression. 22. The system of claim 21, further comprising a component configured to suggest a treatment strategy based on the prognosis-specific variable. 22. The system of claim 21, wherein the plurality of tissue samples are tumor samples and the prognosis-specific variable is indicative of metastatic potential. 22. The system of claim 21, further comprising a component configured to guide a treatment strategy based on identifying cell types and activation and evolutionary states based on the weighted graph. A method for generating a representation of heterogeneous intercellular communication based on spatial information from multiparameter cellular and intracellular image data obtained from several tissue samples from several patients or several multicellular in vitro models, comprising: ,
generating a quantification of spatial heterogeneity between cells of several different predetermined phenotypes in the multi-parameter cellular and intracellular image data, and determining one of the several tissue samples based on the quantification; identifying microdomains for a given number of different predetermined phenotypes by performing phenotyping on the multiparameter cellular and intracellular image data;
constructing a communication graph of the microdomain, wherein each phenotype is a node of the communication graph, and an edge between each pair of expressions of the communication graph corresponds to one phenotype in the pair; showing the effect of the expression on the presence of the other phenotype in the pair;
It contains
Each phenotype of the communication graph is represented by a data vector obtained from the multiparameter cell and intracellular image data,
establishing, for each pair of phenotypes, a numerical relationship between the pair of phenotypes by determining an association or strength of influence value between the data vectors of the pair of phenotypes; the edge between each pair of phenotypes indicates a numerical relationship between the phenotypes in that pair and a directionality of the numerical relationship between the phenotypes in that pair. Method. 26. The method of claim 25 , wherein each association or influence strength value is a linear or non-linear correlation coefficient. Each data vector is a vector of expression values of several predetermined biomarkers, said several predetermined biomarkers being used in generating said multiparameter cellular and intracellular image data. 26. The method of claim 25 . 28. The method of claim 27 , wherein each vector of expression values is a biomarker intensity pattern of the several predetermined biomarkers. 28. The method of claim 27 , wherein establishing directionality for each numerical relationship is performed based on a number of receptor-ligand databases for the number of predetermined biomarkers. 25. Claim 25 , further comprising: interpreting each data vector to determine a number of activation states for each phenotype; and including each of said number of activation states in said communication graph. The method described in. 26. The step of determining an association or influence strength value between the paired phenotypic data vectors includes removing confounding effects of phenotypes other than the paired phenotypes. Method. 26. A non-transitory computer-readable medium storing one or more programs containing instructions that, when executed by a computer, cause the computer to perform the method of claim 25. A computer system for generating spatially informed representations of heterogeneous intercellular communication from multiparameter cellular and intracellular image data obtained from several tissue samples from several patients or several multicellular in vitro models. And,
A processing device is provided, and the processing device includes:
a spatial heterogeneity quantification component configured to generate a quantification of spatial heterogeneity between cells of several different predetermined phenotypes in the multiparameter cellular and intracellular image data, the component comprising: a spatial heterogeneity quantification component that performs cellular phenotyping on the multiparameter cellular and subcellular image data to identify the several different predetermined phenotypes;
a microdomain identification component configured to identify a plurality of microdomains for the several tissue samples based on the quantification, each microdomain being associated with one of the several tissue samples; a microdomain identification component,
a communication network component configured to construct a communication graph for a selected one of the plurality of microdomains, wherein each representation is a node of the communication graph; an edge between each pair of communication network components indicating the influence of one phenotype in that pair on the presence of the other phenotype in that pair;
It contains
Each phenotype of the communication graph is represented by a data vector obtained from the multiparameter cellular and intracellular image data, and the communication network component comprises:
establishing, for each pair of phenotypes, a numerical relationship between the pair of phenotypes by determining an association or strength of influence value between the data vectors of the pair of phenotypes; the edge between each pair of phenotypes indicates the numerical relationship between the phenotypes in that pair and the directionality of the numerical relationship between the phenotypes in that pair. The system consists of : 34. The system of claim 33 , wherein each association or influence strength value is a linear or non-linear correlation coefficient. Each data vector is a vector of expression values of several predetermined biomarkers, said several predetermined biomarkers being used in generating said multiparameter cellular and intracellular image data. 34. The system of claim 33 . 36. The system of claim 35 , wherein each vector of expression values is a biomarker intensity pattern for the number of predetermined biomarkers. 36. The system of claim 35 , wherein establishing directionality for each numerical relationship is performed based on a number of receptor-ligand databases for the number of predetermined biomarkers. The communication network component includes the steps of: interpreting each data vector to determine a number of activation states for each phenotype; and including each of the number of activation states in the communication graph. 34. The system of claim 33 , further comprising: 34. The step of determining an association or influence strength value between the paired phenotypic data vectors includes removing confounding effects of phenotypes other than the paired phenotypes. system. 2. The method of claim 1, wherein the several multicellular in vitro models are selected from the group consisting of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiology systems. 14. The system of claim 13, wherein the several multicellular in vitro models are selected from the group consisting of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiology systems. 26. The method of claim 25, wherein the several multicellular in vitro models are selected from the group consisting of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiology systems. 34. The system of claim 33 , wherein the several multicellular in vitro models are selected from the group consisting of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiology systems.

Images