JP7336806B2 - 3D image analysis of cells and tissues for cellular and intracellular morphological modeling and classification - Google Patents

Description

The present disclosure relates to three-dimensional image analysis of cells and tissues for morphological modeling and classification within cells and cells.

Deformation within tissues and cells is governed by complex underlying biomechanisms that influence spatial and temporal morphological changes. Understanding these processes through quantitative analysis of three-dimensional changes in the size and shape of cells and subcellular structures is important not only for the study of cell tissues, but also for the detection and treatment of pathological conditions such as cancer. is. However, for a three-dimensional shape method that should be capable of accurate morphological analysis, scalable, noise tolerant, and sufficiently specific across multiple populations simultaneously, the dimensionality and quality of the imaging data, as well as the population The high diversity of cellular and subcellular shapes within populations presents several challenges.

"Robust Surface ReconStructure VIA LaPlace -Beltrami Egen -Projection And Boundary Deformation" (YONGGANG SHI et al.) ACTIONS ON MEDICAL IMaging, Vol. 29, No. 12 "Discrete Differential-Geometry Operators for Triangulated 2-Manifolds" (M. Meyer et al.), Visualization and Mathematics III, Berlin, Heidelberg Springer Berlin Heidelberg, 2003, pp. 35-57 "Segmentation and Recognition of 3D Point Clouds within Graph-theoretic and thermodynamic frameworks" (A. Jagannathan) (Doctoral dissertation, Northeastern University, 2005)

Thus, there is a strong need for robust, cellular and subcellular 3D morphometric techniques with high throughput capabilities for performing analyzes on populations.

This section provides background information regarding the present disclosure, which information is not necessarily prior art.

What is provided here is a general summary of the disclosure, and this summary is neither a comprehensive disclosure of its full scope nor all its features.

We present an automated method for analyzing living cells. The method includes staining a component of a biological sample including at least one cell; receiving image data of the biological sample that provides a three-dimensional representation of the biological sample; for each labeled nucleus in the image data, constructing a mathematical representation of the boundaries that define a given nucleus; for each labeled nucleus, for that nucleus Using mathematical expressions to extract features that are measures of the shape and size of the labeled cell nucleus and, for each labeled cell nucleolus, the boundary that defines the given cell nucleolus. constructing a mathematical representation and using the mathematical representation of the cell nucleolus from each of the labeled cell nucleoli a feature that is a measure of the shape and size of the labeled cell nucleolus , storing two or more models of cell classification, and comparing the extracted features of the labeled nuclei and labeled nucleoli with the stored models, and classifying the biological sample.

Image data may be generated by a confocal microscope or another type of imaging device.

In one embodiment, the mathematical representation is constructed using iterative Laplace-Betrami eigenprojections and boundary deformations.

In some embodiments, labeling the components includes dividing the image data into volumes representing the components.

Features extracted from labeled nuclei include volume of labeled nuclei, surface area of labeled nuclei, mean curvature of labeled nuclei, shape index of labeled nuclei, and The determination includes, but is not limited to, one or more of the nuclear curvature index labeled, and the labeled nuclear fractal dimension.

Features extracted from each labeled cell nucleolus include the number of cell nucleoli within the corresponding cell nucleus, the volume of the labeled cell nucleolus, the surface area of the labeled cell nucleolus, the labeled one or more of a labeled cell nucleolus mean curvature, a labeled cell nucleolus shape index, a labeled cell nucleolus curvature index, and a labeled cell nucleolus fractal dimension involved in determining including but not limited to.

In one embodiment, the biological sample is classified using a random forest taxonomy.

In other embodiments, the biological sample is classified using a classification method selected from the group consisting of linear classifiers, k-nearest neighbors, decision trees, neural networks, and support vector machines.

Further areas of applicability will become apparent from the description provided herein. The descriptions and specific examples in the Summary of the Invention are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

The drawings described herein are for the purpose of illustrating selected embodiments only, not all possible implementations, and are not intended to limit the scope of the disclosure.

FIG. 10 provides an overview of automated morphological modeling and classification methods; FIG. 4 is a flowchart illustrating an example method for segmenting image data including one or more cells; FIG. 1 is a flowchart illustrating an example method for resurfacing a cell or organelles or elements within a cell (eg, cell nucleus, cell nucleolus). Robust smooth surface reconstruction image showing 3D visualization of binary mask representation of cell nuclei segmented from fibroblast cell images. Robust smooth surface reconstruction images showing a three-dimensional visualization of a mesh representation of the reconstructed smooth surface of a cell. Robust smooth surface reconstruction images showing three-dimensional visualization of three binary mask representations of the segmented cell nucleolus in the segmented cell nucleus from fibroblast cell images. Images of robust smooth surface reconstruction showing three-dimensional visualization of three mesh representations of the cell nucleolus surface color-coded along the Z-axis. Fig. 2 shows a (local) geometry consisting of two manifolds, including the definition of curvature for a vertex-wise local coordinate frame; FIG. 10 is an exemplary start-to-finish graphical representation of a workflow within a client interface of a LONI pipeline, including a validated start-to-finish workflow protocol overview showing nested module groups; Exemplary graphical representation of the workflow within the client interface of the LONI pipeline, showing an expanded view of the Volume-Shape group for fine-tuning 3D shape modeling and an expansion of the Morphometrics group containing modules for the extraction of morphological measures. Fig. 3 shows a diagram. FIG. 10 is a graph showing fibroblast sorting results with mean AUC for different size cell populations. FIG. Graph showing the classification results of fibroblasts with the top 10 features for classification in order of importance score (mean, minimum, maximum, variance, suitable for classification). Somatic feature names that have also been reported among the top 10 PC3 cells are shown in blue font). FIG. 10 is a graph showing PC3 sorting results with average AUC for different size cell populations. FIG. PC3 classification results are graphs showing the top 10 features for classification in order of importance score (cell nucleolus with "mean", "minimum", "maximum", "variance" suitable for classification Feature names that have also been reported among the top ten fibroblasts are shown in blue font).

Corresponding reference numbers indicate corresponding parts throughout the several drawings.

Exemplary embodiments will now be described in more detail with reference to the accompanying drawings.

Tissue and cell morphology are governed by complex underlying biomechanisms associated with cell differentiation, development, proliferation and disease. Changes in cell shape reflect rearrangements of chromatin structure associated with altered functional attributes, such as gene regulation and expression. At the same time, numerous studies in mechanobiology have shown that the geometric constraints and mechanical forces used to deform cells inversely affect nuclear and chromatin dynamics, gene and pathway activation. ing. Thus, morphological quantification of cells or organelles and elements is of increasing importance as the study of structural rearrangements of chromatin and DNA within a spatial and temporal framework known as the 4D nucleome. Cellular structures of interest in the context of the 4D nucleome include not only the cell nucleus itself, but also the cell nucleolus and nucleolus-associated domains (NADs), chromosome territories, topologically-associated domains (TDA), lamina associations. Also included are sex domains (LADs), as well as loop domains within transcription factories. Furthermore, understanding these processes through quantitative analysis of morphological changes also has many medical implications for the detection, understanding, and treatment of pathological conditions such as cancer. In the literature, cell morphometry is often used to analyze shape, including quantitative measures of size and shape of cells, organelles, elements, and other cellular structures.

Although many efforts have been made to develop cell shape features in 2D or pseudo-3D, several studies have shown that 3D morphometric measures are better than 2D features for describing and identifying cell nucleus shape. get good results. However, 3D shape descriptors that enable robust morphological analysis and are easy to decode by humans are still under active investigation at this time. Furthermore, for 3D shape analysis, which should be scalable, robust to noise, and sufficiently specific across multiple populations simultaneously, the dimensionality and volume of the acquired data, various image acquisition conditions, and intra-population The great diversity of cell shapes in S. cerevisiae presents several challenges. Thus, there is a strong demand for robust 3D cytomorphometry techniques with high throughput capabilities for performing analyzes on populations.

This disclosure has two complementary purposes. The first objective is to assess and validate 3D morphometric metrics for shape description of cells or subcellular organelles and structures. First, the surface of the 3D mask extracted from the microscopic data is reconstructed using Laplace-Betrami eigenprojections and phase-preserving boundary deformations. Next, we compute intrinsic and extrinsic geometric metrics. These are used as derived signature vectors (shape biomarkers) to characterize the complexity of the 3D shape and discriminate observed clinical phenotypic traits. These metrics include volume, surface area, mean curvature, curvature, shape index, and fractal dimension. Shape quantification of cells, cell nuclei, and other subcellular organs and elements can be performed using recommended modeling and analysis methods.

Second, a reproducible pipeline that implements the entire process that can be customized and extended to deeply investigate shape-phenotype relationships in 3D cells and cells in healthy and diseased states. It is to develop a workflow. High-throughput imaging (HTI) can include automation of liquid handling, microscope-based image acquisition, image processing, and statistical data analysis. This work focuses on the last two aspects of this definition. A streamlined multi-step protocol is implemented that relies on a diverse set of tools and seamlessly connects to the workflow of the LONI pipeline. This workflow meets modern high-throughput imaging and analysis standards and is mostly automated with a focus on validity and reproducibility.

FIG. 1 shows a bird's-eye view of the protocol from start to finish. At 11 as a starting point, a biological sample containing at least one cell is imaged using a three-dimensional image acquisition technique. The techniques include microscopy (such as confocal microscopy, multiphoton microscopy, light sheet microscopy, and super-resolution microscopy), tomography (such as fluorescence activated cell sorting (FACS) tomography). imaging, holographic tomography, and optical coherence tomography), or optoacoustic imaging. In one embodiment, 3D confocal imaging is performed using a Zeiss LSM710 laser scanning confocal microscope with a 63x plan/apochromat 1.4na DIC objective. Microscopic imaging produces image data of the biological sample, which image data provides a three-dimensional representation of the biological sample. The biological sample preferably contains 3-20 cells to improve classification, as described in detail below. It is understood that the biological sample may be obtained from tissue or cell culture.

At 12, the image data is divided into separate volumes representing different components (ie, organelles) containing cells within the biological sample. For example, boundaries of cell nuclei within cells are labeled in the image data. Similarly, the nucleolus within each cell nucleus is labeled within the image data. A separate mask (ie, binary image) may be generated for each component element within the cell. In this example embodiment, the mask is converted to a standard file format such as NIFTI (Neuroimaging Informatics Technology Initiative). Splitting may include other filtering or signal processing, as described in detail below.

Surface reconstruction is performed at 13 for each of the labeled cells or subcellular organelles and elements in the image data. That is, construct a mathematical representation of the boundaries that define a given cell nucleus. As a result, the bounding surfaces of each labeled object are represented by polygon meshes. In one example embodiment, a mathematical representation of the surface of a given cell nucleus is constructed using iterative Laplace-Bestrami eigenprojections and boundary deformation methods. Further information on this example of resurfacing methods can be found in IEEE Transactions on Medical Imaging, December 2010, Vol. 29, No. 12, "Robust Surface Reconstruction via Laplace-Beltrami Eigen-Projection and Boundary Deformation" (Yonggang Shi et al.), which is incorporated herein in its entirety. Other methods of constructing the mathematical representation of the interface are also contemplated in this disclosure.

At 14, for each labeled cell or subcellular organelle or element, features are then extracted from the mathematical representation of the surface boundaries of those cells or subcellular organelles or elements. A feature is a measure of the shape and size of a labeled cell or subcellular organelle or element. For example, features extracted from labeled cells or intracellular organelles or elements include volume of cells or intracellular organelles or elements, surface area of cells or intracellular organelles or elements, or mean curvature of the element, shape index of the cell or intracellular organelle or element, tortuosity index of the cell or intracellular organelle or element, and fractal dimension of the cell or intracellular organelle or element. is not limited to

In this example embodiment, the feature is a measure of the shape and size of the labeled cell nuclei. For example, features extracted from labeled nuclei can include nucleus volume, nucleus surface area, nucleus mean curvature, nucleus shape index, nucleus curvature index, and nucleus fractal dimension. is not limited to Features may also be extracted for each labeled cell nucleolus. Again, the features are measures of the shape and size of the labeled cell nucleolus. Examples of features include number of cell nuclei in a labeled cell, cell nucleolus volume, cell nucleolus surface area, cell nucleolus mean curvature, cell nucleolus shape index, cell nucleolus curvature index, and the fractal dimension of the cell nucleolus. It is readily understood that other types of features may be extracted and used for classification.

Finally, at step 15 the cells within the biological sample are classified. In one embodiment, geometric features extracted from cells can be used to distinguish between healthy and cancerous cells. Two or more models for classification are determined prior to imaging. By comparing the extracted features (ie, features in the form of feature vectors) to a stored model, cells within the biological sample can be classified. In one embodiment, cells are classified using a random forest classification method, as described in detail below. Although particular taxonomies are mentioned, other taxonomies also fall within the scope of this disclosure, including linear classifiers, k-nearest neighbors, decision trees, neural networks, and support vector machines, among others.

An example embodiment is described in detail below. Components of the biological sample may be labeled (eg, stained) prior to imaging. In this example embodiment, cells were labeled with three different fluorophores. DAPI (4',6-diamidino-2-phenylindole) is a common stain for cell nuclei, and fibrillarin antibody (Fibrillarin) and ethidium bromide (EtBr) are both used to stain cell nucleoli. Although fibrillarin is a commonly used nucleolus label, it is not suitable for shape modeling purposes because the large local intensity variation within the detected cell nucleolus makes mask shape extraction difficult. It turned out to be too specific. It was found that ethidium bromide can be used to stain condensed chromatin, cell nucleoli, and ribosomes. The use of ethidium bromide was found to improve overall representation of cell nucleolus shape. Specific fibrillarin is combined with ethidium bromide through co-localization to ensure accurate detection of nucleolus location, as described below.

Splitting is described in more detail with reference to FIG. Based on the labeling, the image data can be separated into two or more channels, as shown at 21 . In this example embodiment, the channels were stored separately as separate volumes labeled c0, c1, c2 to represent the DAPI, fibrillarin, and EtBr channels, respectively. Each channel-specific volume was then resliced into a 1024×1024×Z grid (Z=[10,50]), with local subvolumes facilitating matching with the microscope's intrinsic tile size. All sub-volumes were transformed (losslessly) and saved as multi-image 3D TIFF volumes.

At 22, for every sub-volume, the accompanying microscopy metadata is extracted from the original data, including magnification and resolution in all three dimensions, and sent to further downstream modules to address image anisotropy. Ta. At 23, each sub-volume was further converted to grayscale (eg, 8-bit resolution). A noise reduction method such as despeckle may be applied to the sub-volume.

At 24, divide the nucleus and nucleolus from the volume. As an example, automated 3D segmentation of cell nuclei within a DAPI channel was performed by applying a cell segmentation algorithm such as that provided in the Farsight toolkit to subvolumes of the DAPI channel. There are several reasons for adopting the Farsight toolkit. First, it is a CLI kit specifically made for DAPI-stained cell nuclei in 2D or 3D and very easy to use. Also, the Farsight toolkit does not require a labeled training set and proven stable results on the data. The algorithm for nucleus segmentation implements multiple steps, including a graph cut algorithm to binarize the subvolume, a multi-scale Laplacian of Gaussian (LoG) filter to convert the nucleus to a blob mask, a fast algorithm to delineate the nucleus. Clustering and refinement of cell nucleus contours using graph cut and alpha dilation are included. After splitting, the data is converted to a 16-bit 3D TIFF file. Each segmented nucleus was represented as a mask and given its own index. It is readily understood that other cell nucleus division methods are also within the scope of this disclosure.

As an example, automated 3D segmentation of cell nucleoli was also performed in fibrillarin and ethidium bromide channels using the Weka Data Mining software package. In this example embodiment, the Trainable Weka Segmentation plugin was bundled with Fiji, an ImageJ-based image processing and visualization software package. Subnuclear segmentation was performed separately for ethidium bromide (EtBr)-stained and fibrillarin-stained nucleoli. The Trainable Weka Segmentation plugin is the most popular segmentation tool in the ImageJ ecosystem and is relatively easy to use for labeling biological structures in 3D images. A DAPI nuclear mask was used to define partition spaces in the EtBr and fibrillarin channels for the purpose of separating multiple divisions into nuclear objects. A random sample of 10% of the sub-volume within the channel being trained was used to create a classifier model for each channel. A random forest algorithm was then used to apply the model to all sub-volumes within the channel. A cell nucleolus mask was generated from the resulting probability map. Connected elements were then labeled using the "Find Connected Regions" ImageJ/Fiji plugin to uniquely index each cell nucleolus. Segmentation artifacts were filtered for the nucleus mask using the quality adjustment protocol described below. Finally, both EtBr and fibrillarin partition volumes were used as inputs for a colocalization algorithm to demonstrate partitioned EtBr-stained intracellular bodies based on the presence of fibrillarin. It is readily understood that other cell nucleolus splitting methods are also within the scope of this disclosure.

Non-uniform staining often causes occasional segmentation artifacts, and the resulting mask becomes complex, including, for example, "handles", "holes" or irregular boundary shapes with singularities. This problem is usually solved by applying various phase modification techniques, as shown in step 25. An exemplary post-processing step for the nucleus mask includes filling holes using a set of MATLAB functions. Artifact filtering, for example, measures the spherical compactness of an identified object, estimates the volume of that object in terms of voxel numbers, and identifies objects that straddle tile edges or connect to other objects. It can also be done using other measures, such as Java applications that detect. The compactness and voxel number cutoff values were empirically chosen to remove most artifacts. This filter, in conjunction with the hole fill modification, helped to generate a mask more suitable for morphometric analysis. Through this quality adjustment protocol, potentially non-genus-zero masks were corrected or screened and flagged for further fine-tuning. Finally, at 26, the quality-adjusted mask was converted to a neuroimaging format, such as the NIFTI format. These masks were used for shape morphometric analysis subsequently in the pipeline workflow.

The set that served as input to the morphometric feature extraction portion of the workflow consisted of the set of volumes from channel c0 representing the 3D segmented and uniquely labeled binary nucleus mask, as described above, and this The assembly involves a 3D mask of cell nucleoli from channel c2 for each cell filtered by a colocalization procedure using c1. Compressed NIFTI format (.nii.gz) volumes of segmented and quality-adjusted cell nuclei and nucleoli were converted from voxel masks to triangulated manifolds and processed automatically using the developed workflow. .

To model the 3D shape of cell nuclei and nucleoli, the boundaries of the 3D masks extracted from the microscopic data were modeled as 2D manifolds (topologically isomorphic to the 2D sphere S2). These are embedded as triangular surfaces using Laplace-Betrami eigenprojections and a phase-preserving boundary deformation algorithm. This approach is described in more detail in connection with FIG.

Objects are defined in the output received from the splitting process. At 33, for each object, construct a mesh representation from the boundary of the object's binary mask. The mask is binarized at 32 before constructing the mesh representation. The boundaries are projected onto the subspace of the Laplace-Bestrami eigenfunctions so that spurious features can be automatically located by computing the measured distortion of the eigenprojections. The Laplace-Betrami eigenfunctions are intrinsically defined and can be easily computed from the boundary surface without requiring any parameterization. Since these functions are isometrically invariant, they are robust to the jagged nature of interfaces, which is desirable for biomorphology analysis. The magnitude of the eigenvalues of the Laplace-Betrami operator intuitively corresponds to the frequency of the Fourier analysis, providing a convenient mechanism for adjusting the smoothness of the reconstructed surface.

The next step is the mask deformation process of step 34 that uses this information to remove spurious features while leaving the rest of the mask intact, thereby preventing unintended volume reduction. This deformation process is phase-preserving and well designed so that the boundary surface of the mask is manifold. Steps 33 and 34 are repeated until convergence at 35 . Finally at 36, the method produces the final surface as an eigenprojection of the mask boundary, which is a smooth surface with genus-zero phase. This manifold representation of the boundaries of the local organelles allows algorithmic computation of all shapes, including e.g. easier to understand. An exemplary result of this step is shown in FIG. Thus, from the segmented mask, an iterative mask filtering process was used to perform robust surface reconstruction of cell nuclei and cell nucleoli as genus-zero 2D manifolds. Further details regarding this process can be found in IEEE Transactions on Medical Imaging 2010, "Robust Surface Reconstruction via Laplace-Beltrami Eigen-Projection and Boundary Deformation" (YG S hen et al.), which is incorporated herein in its entirety. there is

In this example embodiment, six shape measures are used as features to quantify geometric features of 3D surfaces. To compute these measures, we use a model of triangular surfaces representing the boundaries of genus-zero solids to determine the principal curvatures (minimum and maximum)

Calculate The morphometric measures of shape, namely mean curvature, shape index and tortuosity, can then be expressed in terms of these principal curvatures. The average curvature is

and the shape index is

and the tortuosity is

is defined as The principal curvatures of the surface are the eigenvalues of the Hessian matrix (second fundamental form),

where I is the identity matrix. If S is the surface H(X,Y) of the second basic form, then

is a fixed point, and denoting by u,v the orthonormal basis of the tangent vector at p, the principal curvatures are the symmetric Hessian matrix

is the eigenvalue of As shown in FIG.

the surface

be a parametric representation of It represents a smooth vector-valued function of two variables, with partial derivatives with respect to u and _v , denoted by r _u and r v . Then the Hessian coefficient H _i,j at a given point (p) in the parametric u,v plane is the projection of the second partial derivative at that point onto the normal to S

and using the dot product operator,

can be calculated as

The other three shape measures are volume, surface area, and fractal dimension. Volume is the amount of 3D space enclosed by a closed bounding surface,

where I _D (x,y,z) represents the indicator function of the region of interest (D). r(u,v) is a continuously differentiable function whose normal vector to the surface over a suitable region D in the parametric u,v plane is

If indicated by

and

is the parametric surface representation of the domain boundary. Then the surface area is

can be expressed as The fractal dimension is calculated using the fractal reduction factor

and based on the number N of replacement moieties. "Discrete Differential-Geometry Operators for Triangulated 2-Manifolds" (M. Meyer et al.) Visualization and Mathematics III, Berlin, Heidelberg Springer Berlin Heidelberg, 2 2003, pp. 35-57, and "Segmentation and Recognition of 3D Point Clouds with Graph - theoretic and thermodynamic frameworks" (A. Jagannathan) (Ph.D. thesis, Northeastern University, 2005), using high-precision discrete approximations of these metrics to compute these shape measures on mesh-represented surfaces. was calculated.

The extracted 3D morphometric measures served as features in the feature vector. In one embodiment, the feature vector contains measures for each cell nucleus. Specifically, the features of each nucleus include nucleus volume, nucleus surface area, nucleus mean curvature, nucleus shape index, nucleus curvature index, and nucleus fractal dimension. The feature vectors are then used to train several machine learning classifiers, eg using the open source Python package scikit-learn 0.17.0. To improve the behavior of the classification algorithm, preprocessing of the data included standardization of the variables to zero mean and unit variance, and normalization to scale individual samples to the unitary norm, allowing each of the cross-validation Computed on the training set separately in steps.

In this example embodiment, morphometric measures were used to aggregate nucleolus data for each cell nucleus. For example, the number of nucleoli detected per cell nucleus was included as an individual feature in the feature vector. Various methods were explored to synthesize features at the cell nucleolus level. They include custom, nuclear-level dissimilarity metrics and multi-instance learning frameworks. Best results were obtained by summing the nucleolus data within each cell nucleus by estimating the density of each morphometric measure. For each nucleus, sample statistics (eg, mean, minimum, maximum, and higher moments) were calculated for each morphometric measure across the nucleolus therein. Using these statistics, the signature feature vectors of the corresponding parental nuclei were scaled so that all feature vectors were of the same length. Accordingly, cell nuclei in which no internally located cell nucleolus was automatically detected were excluded from further analysis, such that at least one cell nucleolus was present in each cell nucleus. In general, the correct classification (type, stage, treatment, etc.) of each individual cell is a difficult task due to the significant population heterogeneity of observed cell phenotypes. For example, the same sample may contain a tangled and dense mixture of “cancerous” and “non-cancerous” cell phenotypes, or both classes may contain apoptotic cells exhibiting similar shapes or sizes. Sometimes it happens. Given the nature of the samples, culture, preparation and collection, sorting of cell clusters rather than single cells were considered. The rationale is that even if the algorithm misclassifies a small number of cells in the sample, as long as the majority of cells are correctly classified, the final (cell population) label will be correctly assigned. based on the observation that Using this strategy, sorting is preferably performed on small groups of cells ranging from 3 to 19 cells per set. During each fold of internal cross-validation, this small cell population was randomized by bootstrapping with 1000 replicates.

The LONI pipeline is a popular tool in neuroimaging and bioinformatics, but has so far been overlooked in the cellular bioimaging community. This disclosure uses the LONI pipeline to implement a protocol that streamlines multiple steps. This protocol relies on a diverse set of tools and solutions, seamlessly connecting to the LONI pipeline workflow, as shown in FIG. At a high level, each step of the data processing and analysis protocol is wrapped in a workflow as a separate module that provides inputs and outputs, and the pipeline automatically connects these atomic modules together. can be managed. As a result, distributed and massively parallel protocol implementations can easily process thousands of cell nuclei and nucleoli simultaneously. The workflow does not depend on the total number of 3D objects, biological conditions, or number of running instances. The reason is that once the workflow settings, including job scheduling and resource allocation, are finalized, workflow execution is fully automated. During execution, the workflow provides the researcher with real-time information on progress and allows the researcher to view intermediate results for each individual step and easily inspect and restart failed modules or specific instances. This implementation is very flexible and is not limited to any particular rules contained in this workflow. By adjusting parameters and exchanging individual modules, it can be converted to a wide variety of experiments while maintaining high throughput capability.

This workflow is configured to make the data available in a 3-channel 1024x1024xZ 3D volume in the format used for sharing, ie a 16-bit 3D TIFF file. Each volume is processed individually in parallel, with the workflow automatically determining how many processes are required to parse all the input data. In addition, parallelized workflows automatically branch and collapse as needed. For example, from preprocessing to splitting, workflow execution initiates M processes, where M is the number of input volumes. After this step, the workflow automatically starts M×[N ₁ , _{N 2} , . is the number of divided objects (eg cell nuclei). In a post-processing step, some of the segmented objects are filtered by the curation module and excluded from further analysis. The workflow automatically reduces the number of processes to the number of masks that pass curation, with separate 3D shape modeling and morphometric feature extraction for each mask, which saves experimental computational time. Up to 1200 jobs can be run concurrently on the cluster while reducing the number of jobs exponentially. Finally, the workflow collects morphometric information from each individual mask and summarizes it into a result table, which is then used as input for the classification algorithm. This ability takes advantage of modern computing resources, frees researchers from the labor of low-level settings, simplifies control of execution processes, and improves reproducibility across processes.

To validate the shape morphometry metric, the metric was first applied to an artificially generated 3D mask. The scikit-image Python library was used to create 3D solids representing a cube, an octahedron, a sphere, an ellipsoid, and three overlapping spheres with linearly aligned centers. All these objects were processed and the resulting shape morphometric measures were compared. Specifically, the analytically derived (expected) volume and surface area calculated using the corresponding shape parameters (e.g. radius, size) and the computational It is desirable to check if there is a close relationship between the calculated volume and surface area. As a result, the computational error was within 2% for shapes similar to cell nuclei such as spheres and ellipsoids. For more geometric objects such as cubes and octahedra, the computational error was within 6%. The increased error in the latter can be explained by mesh smoothing applied to shape vertices to resolve singularities (eg, smooth but indistinguishable surface boundaries). To demonstrate the detection of shape differences between various types of 3D objects, overlapping spheres were compared with circumscribing ellipsoids. As expected, spheres had lower measures of mean curvature and tortuosity and higher values of shape index compared to ellipsoids. Comparing spheres, ellipsoids, and overlapping spheres, a gradual monotonic slope of shape morphometric measures was observed. This simulation confirmed the ability to accurately measure the size and shape characteristics of 3D objects based on their boundary shapes. This ability forms the basis for machine learning-based object classification.

To assess the chosen shape morphometric measure as a distinguishing feature, single fibroblast nuclear classification was compared with the SPHARM coefficient. Fibroblasts (neonatal male) were purchased from ATCC (BJ Fibro-blasts CRL-2522 normal) and subjected to G0/G1 phase Serum Starvation protocol. The imaging state or phenotype obtained by this protocol is cell cycle synchronized by serum starvation (SS) and proliferation (PROLIF). As a result, both approaches successfully processed 962 cell masks, of which 466 with PROLIF and 496 with SS.

To compare the methods, a binary mask of cell nuclei was extracted. Shape morphometric measures were calculated as described above. To obtain the SPHARM coefficients, we first use the popular SPHARM-MAT toolbox, which performs surface reconstruction and spherical parameterization using the CALD algorithm, and then transform the object surface into l= By expanding to the full set of thirteenth order (default setting) functions and finally using the SHREC method that minimizes the square root of the distance between the corresponding surface parts, we have obtained the "Spherical mapping for processing of 3D closed surfaces" L. We computed the SPHARM shape descriptor as described by Shen et al. and used it as the feature vector for classification.

The open-source Python package Scikit-learn 0.17.0 was adapted to test several commonly used machine learning classification methods using default parameters for each method and, where applicable, the same random The seed was used to test on the extracted feature vectors. Performance was compared using a 5-fold cross-validation technique and using area under the receiver operating characteristic curve (AUC) as the metric. As shown in Table 1 below, the 3D profilometry measure not only demonstrates discriminative performance comparable to, but outperforms the SPHARM coefficients in the use of all algorithms tested.

After segmentation and morphometric feature extraction of both cell nuclei and nucleoli as described above, the entire fibroblast classification data set contains a total of 965 nuclei (498 for SS and 470 for PROLIF) and 2181 nucleoli. (SS is 1151, PROLIF is 1030). The best results with a single classifier were obtained using a stochastic gradient boosting classifier with 1500 base learners, maximum decision tree depth of 9, learning rate of 0.01, subsampling rate of 0.5, leaf A minimum sample size of 3 nodes was achieved. Hyperparameters were fine-tuned by grid search combined with cross-validation. To evaluate these classification results, 7-fold internal statistical cross-validation was randomly repeated 10 times to measure accuracy, precision, sensitivity and AUC. The results are shown in Table 2 for the classification of single cells and clusters of 9 cells.

FIG. 7A shows average AUC values for cell populations ranging in size from 3-19. An accuracy rate of 95% was obtained when a group of 9 cells was classified, and an accuracy rate of 98% was obtained for a group of 15 or more cells. The gradient boosting classifier also computes cross-validated feature importance and results are reported as shown in FIG. 7B. Which measures differ significantly between the two cell states can be assessed from these figures, and novel research hypotheses may be proposed and tested using the prospective data. . Both nuclear morphometric features (top 3) and nucleolus morphometric features (top 5 of 10) are of high importance for distinguishing SS fibroblasts from PROLIF was reported. The nucleolus shape morphometric statistics with high moments contribute less to the classification, probably because the sample size is too small (most cells have only 1-3 nucleoli per nucleus). Since it does not, it was removed from the feature list when selecting features.

Malignant cancer cells undergo a series of reversible transitions between multiple intermediate phenotypic states surrounded by pure epithelium and pure mesenchyme during the process of metastatic progression. These transitions in prostate cancer are associated with quantifiable changes in nuclear structures. Microscopic slides of prostate cancer cell line P3 were cultured in two phenotypic states: epithelial (EPI) and mesenchymal transition (EMT). The derived dataset consisted of 458 nuclei (310 EPI, 148 EMT) and 1101 nucleolus (649 EPI, 452 EMT) masks. The sample size imbalance between the two classes was resolved using random uniform subsampling. There were up to 250 cells in the training set and up to 40 different cells in the test data for each of the 7-fold cross-validation processes.

In this case, the best classification with a single classifier is the result of applying a random forest model (number of decision trees 1000, maximum depth of decision trees 12, maximum number of features for best split 40%) Met. Hyperparameter fine-tuning, accuracy metrics, and cross-validation procedures were the same as reported for the fibroblast experiments above. Again, for fibroblast classification, the 9-cell classification set achieved an average accuracy rate of 95.4%, which improved to 95.4% for sets of 15 or more cells ( See Table 3). FIG. 8A shows the AUC of different size groups, and it can be seen that the accuracy of classification increases with the size of the cell population, reaching 98% for the population of 13 cells. This experiment also investigated the feature importance reported by the classifier. The top 10 significant features in this classification include nuclear shape morphometric features (3 of the top 10, 2 of which are the top 2 in fibroblasts) and nucleolus shape morphometric features. Features (top 5) were included.

This disclosure presents a protocol that provides a high-throughput, near-automated (human-involved) solution for 3D modeling, morphological feature extraction, and classification of cell types or therapeutic states. Compared to other studies using 2D projections, this technique operates natively in 3D space and also better represents the underlying real cell nucleus and nucleolus geometries for human interpretation. Make use of intrinsic and extrinsic morphometric measures that simplify Robust surface reconstruction allowed for accurate approximation of 3D object boundaries, which was verified with artificial data. The shape metrology measure proposed here outperforms another popular approach, demonstrating its versatility across different cell types, states, and even domains.

The resulting start-to-end protocol is highly parallel, and its throughput is effectively limited by the number of computational nodes, allowing thousands of objects to run simultaneously with minimal need for human intervention. can be processed. This pipeline workflow utilizes the diversity of bioimage analysis software and integrates several open source tools for various data processing and analysis steps. The modularity of workflows makes them more reusable and easier to change. This allows the same workflow to be used or customized and extended for other purposes or other studies requiring analysis of diverse arrays of cells, cell nuclei (e.g. specification of new datasets, replacement of certain atomic modules, etc.). A live demonstration through the LONI pipeline demonstrates the ease of use and high efficiency of parallel data processing.

To facilitate reproducibility of results, facilitate open-science development, and enable collaborative validation, we generated 3D imaging data for two publicly shared cell lines. This 3D imaging dataset is one of the largest published datasets of this type (3-channel raw data with up to 1500 nucleus masks and up to 2700 nucleolus masks). include). Classification results of these data comparing epithelial vs. mesenchymal human prostate cancer cell lines and comparing serum starved vs. proliferating fibroblast cell lines demonstrated the correct prediction of cell type using 3D morphometry. The rate was demonstrated to be high, especially when applied to aggregates of cells. Although different classification algorithms may be optimal for different experiments, we observed that morphometric measures of both cell nuclei and nucleoli are important features for distinguishing treatment status or cell phenotype. For fibroblast classification, the results show that nuclear morphometry, number of nucleoli per nucleus, and various inner cell morphometric measures are important. In PC3 cells, the top important classification feature was the moment of distribution of various nucleolus morphometric measures, in addition to the size and shape of the cell nucleus. Interestingly, 3 of the top 10 important morphometric features were common for both cell lines. This confirms and demonstrates the observation reported above that this method extracts relevant information from cell shape to correctly classify cells using a combination of criteria.

The proposed approach is scalable, can preprocess various complex and large-scale 3D image data, and is not limited to the shape of cell nuclei and nucleoli. This method can be applied to other cell and cell nuclear elements of interest with some modifications. Robust smooth surface reconstruction can be directly applied to any 3D shape as long as the phase is spherical-like. In conjunction with molecular-level techniques, this 3D shape morphometric workflow can form a powerful combination for interrogation of DNA structure within the spatial and temporal framework of the 4D nucleome. An example of a possible future application of this workflow is asymmetric cell division. Stem and progenitor cells are characterized by the ability to self-renew and give rise to differentiated progeny. Controlled asymmetric differentiation provides a balance between these processes that is essential for generating cellular diversity during development and for maintaining adult tissue homeostasis. . Disruption of this balance can lead to premature exhaustion of the stem/progenitor cell pool, ie, abnormal growth. In many tissues, dysregulation of asymmetric differentiation is associated with cancer. It is not known whether there is a fortuitous link between defects in asymmetric differentiation and precancerous lesions. It is expected that this shape analysis pipeline will aid in the study of 4D nucleomic phases of morphogenesis and precancerous lesions.

The ability to automate the process of specimen collection, image acquisition, data pre-processing, derived biomarker calculation, modeling, classification, and analysis to significantly impact clinical decision-making and the fundamental investigation of cellular alterations can be done. It combines 3D cell nuclear shape modeling with robust smooth surface reconstruction and shape morphometric measures for highly parallel start-to-end morphological analysis of thousands of cell nuclei and nucleoli in 3D. It seems to be the first attempt to create a typical pipeline workflow. This approach can efficiently and informatively assess cell shape within image data, and represents a reproducible technique that can be validated, modified, and transferred in the biomedical community. This facilitates reproducibility of results, joint validation of methods, and widespread knowledge dissemination.

Portions of the techniques described herein may be implemented by one or more computer programs executed by one or more processors. The computer program includes processor-executable instructions stored on a non-transitory, tangible computer-readable medium. This computer program also includes stored data. Non-limiting examples of non-transitory, tangible computer-readable media include non-volatile memory, magnetic storage, and optical storage.

Some portions of the above description present the techniques described herein in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. These operations, although described in terms of function or logic, are understood to be implemented by computer programs. Furthermore, it has proven convenient at times to refer to the organization of operations by module or function names without loss of generality.

Unless otherwise stated, throughout this description, discussion using terms such as "processing", "computing", "calculating", "determining", "displaying" refers to information storage, such as computer system memory or registers. It should be clear from the above discussion that it refers to the operations or processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in a device, transmitter or display device. be.

Certain aspects of the described techniques include process steps and instructions described herein in algorithmic form. It should be noted that the process steps and instructions described may be embodied in software, firmware, or hardware, and if embodied in software, may be implemented in a variety of applications used by real-time network operating systems. It may be downloaded and reside on platforms and operated from those platforms.

The present disclosure further relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may contain a computer which can be selectively activated or reconfigured by a computer program, which computer program is accessible by the computer. may be stored on any computer-readable medium. Such computer programs may be stored on tangible computer-readable storage media, which may be any type of disk including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory ( ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application specific integrated circuits (ASICs), or any medium suitable for storing electronic instructions. and each is coupled to a computer system bus. Further, the computers referred to herein may include a single processor, or may be architectures employing multi-processor designs for increased computing power.

The algorithms and operations presented herein are not inherently related to any particular computer or other apparatus. Additionally, various systems may be used with programs in accordance with the teachings herein, and it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure, along with equivalent variations, for a variety of these systems will be apparent to those skilled in the art. Additionally, no particular programming language is considered in the description of this disclosure. It will be appreciated that a variety of programming languages may be used to implement the techniques of this disclosure as described herein.

The above description of embodiments has been provided for purposes of illustration and is not intended to be exhaustive or limiting of the present disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment and, where applicable, are interchangeable even if not specifically shown or described and used in selected embodiments. can do. Also, individual elements or features of a particular embodiment may be varied in numerous ways. Such variations are not considered a departure from the present disclosure and all such modifications are intended to be included within the scope of the present disclosure.

Claims (21)

In an automated method for analyzing living tissue and cells,
storing two or more models of cell classification, each of the two or more models representing the shape and size of a component from a known biological sample; ,
staining a component of a biological sample, said biological sample comprising a tissue or cell culture containing at least one cell;
receiving image data of the biological sample, the image data providing a three-dimensional representation of the biological sample;
labeling at least one cell or subcellular organelle or element within the image data;
for each labeled cell or subcellular organelle or element within said image data, including a chromosomal territory, a topologically associated domain (TDA), a lamina associated domain (LAD), or a telomere boundary; constructing from the image data a mathematical representation of boundaries defining cell nuclei;
for each of said labeled cells or organelles or elements within said cells, using said mathematical representation of the boundary defining said cell nucleus, shape of said labeled cells or organelles or elements within said cells; and extracting features that are measures of size;
and classifying the biological sample by comparing the extracted features of the labeled cells or organelles or elements within cells to the stored model. 2. The method of claim 1, further comprising generating said image data using a three-dimensional image acquisition technique. 2. The method of claim 1, wherein labeling the components further comprises dividing the image data into volumes representing the components. 2. The method of claim 1, wherein constructing the mathematical representation further comprises using iterative Laplace-Betrami eigenprojections and boundary deformations. The feature extracted from the labeled cell or intracellular organelle or element is the volume of the labeled cell or intracellular organelle or element, the labeled cell or intracellular organelle or element surface area of the element, mean curvature of the labeled cell or subcellular organelle or element, shape index of the labeled cell or subcellular organelle or element, labeled cell or subcellular organelle or 2. The method of claim 1, comprising determining one or more of the curvature index of the element and the fractal dimension of the labeled cell or organelle or element within the cell. 2. The method of claim 1, further comprising classifying the biological sample using a random forest taxonomy. 2. The method of claim 1, further comprising classifying the biological sample using a classification method selected from the group consisting of linear classifiers, k-nearest neighbors, decision trees, neural networks, and support vector machines. Method. 2. The method of claim 1, comprising sorting the biological sample only if the biological sample contains three or more cells. staining a component of said biological sample using a label for cells or subcellular organelles or elements of one of fibrillarin antibodies, ethidium bromide, and 4',6-diamidino-2-phenylindole. A method according to claim 1. 2. The method of claim 1, wherein said biological sample is from cell culture or tissue. In an automated method for analyzing biological tissue,
storing two or more models of cell classification, each of the two or more models representing the shape and size of a component from a known biological sample; ,
staining a component of a biological sample containing at least one cell;
receiving image data of the biological sample, the image data providing a three-dimensional representation of the biological sample;
labeling nuclei and nucleoli of at least one cell in the image data;
constructing from the image data, for each labeled cell nucleus in the image data , a mathematical representation of boundaries defining a cell nucleus;
extracting, for each of the labeled nuclei, features that are measures of the shape and size of the labeled nuclei using the mathematical representation of the boundaries that define the nuclei;
constructing from the image data , for each labeled cell nucleolus, a mathematical representation of boundaries defining the cell nucleolus;
extracting from each of the labeled cell nucleoli a feature that is a measure of the labeled cell nucleolus shape and size using the mathematical representation of the boundaries defining the cell nucleolus process and
and classifying the biological sample by comparing the extracted features of the labeled nuclei and the labeled nucleoli with the stored model. 12. The method of claim 11, further comprising generating said image data using a confocal microscope. 12. The method of claim 11, wherein labeling the components further comprises dividing the image data into volumes representing the components. 12. The method of claim 11, wherein constructing the mathematical representation further comprises using iterative Laplace-Betrami eigenprojections and boundary deformations. The features extracted from the labeled cell nuclei are volume of the labeled cell nucleus, surface area of the labeled cell nucleus, mean curvature of the labeled cell nucleus, shape index of the labeled cell nucleus, labeling 12. The method of claim 11, comprising determining one or more of the curvature index of the labeled cell nucleus and the fractal dimension of the labeled cell nucleus. The features extracted from the labeled cell nucleolus are the number of cell nucleoli in the corresponding cell nucleus, the volume of the labeled cell nucleolus, the surface area of the labeled cell nucleolus, the labeled one or 16. The method of claim 15, comprising a plurality. 12. The method of claim 11, further comprising classifying the biological sample using a random forest taxonomy. 12. The method of claim 11, further comprising classifying the biological sample using a classification method selected from the group consisting of linear classifiers, k-nearest neighbors, decision trees, neural networks, and support vector machines. Method. 12. The method of claim 11, further comprising sorting the biological sample only if the biological sample contains three or more cells. 12. The method of claim 11, wherein one of a fibrillarin antibody, ethidium bromide, and 4',6-diamidino-2-phenylindole is used to stain a component of the biological sample. 12. The method of claim 11, wherein the biological sample is cell cultured.

Images