WO2018157381A1 - Method and apparatus for intelligently classifying pathological slice image - Google Patents

Abstract

Disclosed are a method and apparatus for intelligently classifying a pathological slice image. The method comprises: carrying out image processing on each pathological slice image in a pre-set normal sample and cancer sample, so as to obtain training data of the normal sample and the cancer sample, wherein the training data includes a mean value set, a variance set and an information entropy set of similarity indexes; and training a pre-set machine classification model based on the training data, so as to use the trained machine classification model to determine the type of a pathological slice image to be classified. By way of introducing an information entropy as an independent dimension of the image structure disorder degree, the purpose of quantitatively describing the differentiation degree of a tumour cell or a tissue can be achieved, and by way of using the training data including the information entropy set to train the machine classification model, the accuracy of the intelligent classification of the pathological slice image can be effectively improved.

Description

Pathological slice image intelligent classification method and device Technical field

The present invention relates to the field of image processing, and in particular, to an intelligent classification method and apparatus for pathological slice images.

Background technique

The difference in the morphological structure and functional stability of normal cells in the development of the individual is called cell differentiation. The higher the degree of differentiation, the greater the difference. Some cells in the body lose their normal death regulation due to gene mutation, and the division and proliferation are out of control, and disordered excessive proliferation leads to destruction of normal tissue structure and becomes cancer cells. Differentiation in tumor pathology often refers to the similarity between tumor cells and normal cells from which they originate. It is the main basis for the differentiation of benign and malignant tumors. The tumors with high differentiation have benign behavior, and the tumors with low differentiation have many malignant manifestations.

The main task of pathological section image analysis is to identify tumor cells or tissues under the microscope to show structural features different from normal cells or tissues, and usually need to be assisted by HE staining and other means of labeling. Light microscopy can only describe the morphology of the nucleus, which is subjective and lacks accurate and more objective quantitative criteria. In recent years, the advancement of science and technology has promoted the research methods of pathology far beyond the traditional morphological observations, and many new methods and technologies have emerged, fundamentally requiring the development of standards to be objective and quantitative. Quantitative quantitative analysis reflects the morphological structure of tissues and cells, and can exclude the influence of subjective factors. Image analysis in tumor pathology is mainly the determination of nuclear morphological parameters, distinguishing between precancerous lesions and cancer, distinguishing between benign and malignant tumors, and pathological grading of tumors. And judge the prognosis and so on. With the development of electronic computers, researchers began to try to convert medical analog images into digital images, and carried out preliminary research on computer-aided diagnosis, trying to assist doctors to read medical images to a certain extent, excluding human subjective factors, improving diagnostic accuracy and effectiveness.

The development of optical imaging technology has led to high dimensionality of data acquisition. It is difficult to understand such a large amount of information in traditional two-dimensional gray value images. Medical image analysis is no longer limited to diseases with obvious diagnostic features in the past, and has begun to expand into images of many different organs, anatomical and functional processes, attempting to exploit automated A certain amount of computer-aided image analysis helps clinicians and researchers to process massive image information efficiently and accurately. However, the current technology is still difficult to effectively determine whether the pathological slice image is a normal slice image or a cancer slice image with low accuracy.

Summary of the invention

The main object of the present invention is to provide an intelligent classification method and apparatus for pathological slice images, which aims to solve the technical problem of low accuracy in classifying pathological slice images in the prior art.

To achieve the above object, a first aspect of the present invention provides a method for intelligently classifying a pathological slice image, the method comprising:

Performing image processing on each of the preset normal sample and the cancer sample image to obtain training data of the normal sample and the cancer sample, the training data including a mean set of similarity indicators, a variance set, and Information entropy set;

And training the preset machine classification model based on the training data of the normal sample and the cancer sample to obtain a trained machine classification model;

The pathological slice image to be classified is input to the trained machine classification model, and the type of the trained machine classification model output is used as the type of the pathological slice image to be classified.

In order to achieve the above object, a second aspect of the present invention provides an apparatus for intelligently classifying a pathological slice image, the device comprising:

a processing module, configured to perform image processing on each of the preset normal sample and the cancer sample, to obtain training data of the normal sample and the cancer sample, wherein the training data includes an average of similarity indicators Sets, variance sets, and information entropy sets;

a training module, configured to train a preset machine classification model based on training data of the normal sample and the cancer sample, to obtain a trained machine classification model;

And a classification module, configured to input the pathological slice image to be classified into the trained machine classification model, and use the type of the trained machine classification model output as the type of the pathological slice image to be classified.

The invention provides an intelligent classification method for pathological slice images, which comprises: performing image processing on each of the preset normal samples and the cancer sample images, and obtaining training data of normal samples and cancer samples, wherein, training The data contains the mean set, the variance set, and the letter of similarity indicators. Entropy set, and training the preset machine classification model based on the training data of the normal sample and the cancer sample, obtaining the trained machine classification model, inputting the pathological slice image to be classified into the trained machine classification model, and The type of the machine classification model output after training is used as the type of the pathological slice image to be classified. Compared with the prior art, not only the mean and variance of the similarity index are used to discriminate the difference between the normal slice image and the cancer slice image, but also the information entropy is introduced as an independent dimension of the degree of image structure confusion, and the information is used. Entropy can achieve the purpose of quantitatively describing the degree of differentiation of tumor cells or tissues, and train the machine classification model through the training data including the mean set, the variance set and the information entropy set of the similarity index of the normal sample and the cancer sample, and pass the The machine classification model classifies the pathological slice patterns, which can effectively improve the accuracy of intelligent classification of pathological slice images.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and those skilled in the art can obtain other drawings according to these drawings without any creative work.

1 is a schematic flow chart of an intelligent classification method for pathological slice images according to a first embodiment of the present invention;

2 is a schematic flow chart of an intelligent classification method for pathological slice images according to a second embodiment of the present invention;

3 is a schematic diagram of functional modules of a pathological slice image intelligent classification device according to a third embodiment of the present invention;

4 is a schematic diagram of functional modules of an intelligent classification device for pathological slice images according to a fourth embodiment of the present invention;

Figure 5a is a three-dimensional spatial distribution of the fluorescence lifetime of the slice HE staining in the mean μ, the variance σ and the entropy value S;

Figure 5b is a support vector machine linear discrimination of the slice HE staining fluorescence lifetime in the μ_σ plane;

Figure 5c is a support vector machine linear discrimination of the slice HE staining fluorescence lifetime in the S_μ plane;

Figure 5d is a support vector machine linear discrimination of the slice HE staining fluorescence lifetime in the S_σ plane.

detailed description

In order to make the objects, features and advantages of the present invention more obvious and easy to understand, the following will be The technical solutions in the embodiments of the present invention are clearly and completely described in the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Due to the prior art, the accuracy of classifying pathological slice images is less technical.

In order to solve the above technical problem, the present invention proposes an intelligent classification method for pathological slice images, in which not only the mean value and the variance of the similarity index are used to discriminate the difference between the normal slice image and the cancer slice image, but also introduces As an independent dimension of the degree of image structure disorder, information entropy can achieve the purpose of quantitatively describing the degree of differentiation of tumor cells or tissues, and through the mean set, variance set and information entropy set of similarity indicators including normal samples and cancer samples. The training data is used to train the machine classification model, and the pathological slice pattern is classified by the machine classification model, so that the accuracy of the intelligent classification of the pathological slice image can be effectively improved.

1 is a flowchart of a method for intelligently classifying a pathological slice image according to a first embodiment of the present invention, the method comprising:

Step 101: Perform image processing on each of the preset normal sample and the cancer sample to obtain training data of the normal sample and the cancer sample, where the training data includes a mean set of similarity indicators, Variance set and information entropy set;

In the embodiment of the present invention, the pathological slice image intelligent classification method is implemented by a pathological slice image intelligent classification device (hereinafter referred to as: classification device).

In the embodiment of the present invention, it is necessary to train the machine classification model first to classify the classified pathological slice images using the machine classification model. In order to train the machine classification model, it is necessary to prepare a training sample in advance, the training sample includes a normal sample and a cancer sample, wherein the normal sample includes a pathological slice image that is diagnosed as normal, and the cancer sample includes a diagnosis that has been cancerous. Pathological slice image.

The classification device performs image processing on the preset normal sample and each pathological slice image in the cancer sample to obtain training data of the normal sample and the cancer sample, wherein the training data of the normal sample includes all normal pathological slice images. Mean set of similarity index, variance set of similarity index and information entropy set, the training data of cancer sample contains all pathological slice images of cancerous The mean set of similarity indicators, the variance set of similarity indicators, and the set of information entropy.

Step 102: Train a preset machine classification model based on training data of the normal sample and the cancer sample to obtain a trained machine classification model;

Step 103: Input the pathological slice image to be classified into the trained machine classification model, and use the type of the trained machine classification model output as the type of the pathological slice image to be classified.

In the embodiment of the present invention, the classification device trains the preset machine classification model based on the training data of the normal sample and the cancer sample, obtains the trained machine classification model, and inputs the pathological slice image to be classified into the trained machine. The classification model uses the type of the trained machine classification model as the type of the pathological slice image to be classified.

In the embodiment of the present invention, the classification device performs image processing on each of the preset normal sample and the cancer sample image to obtain training data of the normal sample and the cancer sample, wherein the training data includes the mean value of the similarity index. The set, the variance set and the information entropy set, and training the preset machine classification model based on the training data of the normal sample and the cancer sample, obtaining the trained machine classification model, and inputting the pathological slice image to be classified into the trained machine classification The model, and the type of the trained machine classification model output is taken as the type of the pathological slice image to be classified. Compared with the prior art, not only the mean and variance of the similarity index are used to discriminate the difference between the normal slice image and the cancer slice image, but also the information entropy is introduced as an independent dimension of the degree of image structure confusion, and the quantitative can be achieved. Describe the purpose of the degree of differentiation of tumor cells or tissues, and train the machine classification model through the training data including the mean set, the variance set and the information entropy set of the similarity index of the normal sample and the cancer sample, and through the machine classification model pair The classification of pathological slice patterns makes it possible to effectively improve the accuracy of intelligent classification of pathological slice images.

2 is a flowchart of a method for intelligently classifying a pathological slice image according to a second embodiment of the present invention, the method comprising:

In the embodiment of the present invention, in order to obtain the training data of the normal sample and the cancer sample, each pathological slice image in the normal sample and the cancer sample is processed according to steps 201 to 203, and each pathological slice image is obtained. Mean, variance and information entropy of similarity indicators. details as follows:

Step 201: Read a three-dimensional image of the pathological slice image containing structural information, where the three-dimensional image forms a third dimension by the photon number distribution of each pixel point;

In the embodiment of the present invention, for each pathological slice image, the classification device will read the three-dimensional image of the pathological slice image containing the structural information, and the three-dimensional image is composed of the coordinates of each pixel to form the first and second dimensions, and each pixel The photon number distribution of points constitutes the third dimension. Wherein, in the three-dimensional image, data containing structural information is stored in a third dimension perpendicular to the pixel coordinates, and is a photon attenuation sequence in fluorescence lifetime imaging and a Raman spectrum in Raman imaging.

Step 202: Extract a similarity indicator of each pixel point determined by the third dimension data set in the three-dimensional image.

In the embodiment of the present invention, the classification device extracts the similarity index of each pixel point determined by the third dimension data set in the three-dimensional image.

Wherein, the three-dimensional image may be fluorescence lifetime imaging or Raman imaging.

Among them, the related photon counting (TCSPC) promotes fluorescence lifetime imaging (FLIM), and the fluorescence lifetime of the fluorophore is considered to be only related to its structure and microenvironment, and is not affected by excitation light intensity, molecular concentration, etc., so it can be used to characterize The degree of similarity between the marked material structures. In addition, the Raman spectrum reflects the internal karyotype structure of the material. The Raman spectral correlation coefficient matrix is obtained by cross-correlation operation on the Raman spectrum. The Raman spectral correlation coefficient matrix can further reflect the similarity of the internal structure of the structure. Both the fluorescence lifetime and the Raman spectral correlation coefficient matrix can be used as a quantitative index for discriminating the degree of similarity between pixels of a medical image, that is, a calculation parameter as a similarity index. With quantitative indicators of image similarity, statistical analysis of similarity can be performed on medical images. In normal tissues, the degree of cell differentiation is high, and there is a wide similarity distribution between each pixel. The degree of differentiation of tumor tissue is manifested as the structural difference of each pixel is smaller, the similarity distribution is relatively narrow, and the tumor is tumor. The higher the malignancy, the lower the differentiation, and the similarity distribution is concentrated. In general, the similarity index of the pathological slice image can be averaged, and the mean and variance of the similarity index can be obtained to determine the difference between the normal cell or tissue and the cancer cell or tissue. Furthermore, information entropy can be further introduced as an independent dimension index of the degree of image structure disorder, which can quantitatively describe the degree of differentiation of tumor cells or tissues, and the mean and variance of the similarity index are used as the differentiation of tumor cells or tissues from normal cells or tissues. A set of criteria criteria to improve accuracy is the principle of the invention.

If the three-dimensional image is a fluorescence lifetime imaging, the fluorescence lifetime or the phase mapping coordinate may be calculated by a fitting or phase mapping algorithm. Specifically, the above step 202 may be the following step A, or step B:

In step A, the fluorescence lifetime of each pixel is obtained by least square fitting using the time decay curve corresponding to each pixel point, and the fluorescence lifetime of each pixel is used as the similarity index of each pixel.

Among them, the time decay curve of the pixel is:

I _i,j (t)=I ₀ ^i,j exp(-t/τ _i,j )

Where i and j represent the coordinates of the pixel in the three-dimensional image, I _i,j (t) represents the fluorescence intensity of the pixel point ij after decay at time t, t represents time, I ₀ ^i,j represents the total pixel point ij The fluorescence intensity, τ _i,j represents the fluorescence lifetime of the pixel point ij. Wherein, the total fluorescence intensity of the pixel point ij can be determined based on the pixel point ij photon number attenuation sequence in the three-dimensional image.

or,

In step B, the fluorescence lifetime of each pixel point is calculated by using a preset phase mapping algorithm, and the fluorescence lifetime of each pixel point is used as the similarity index of each pixel point.

Among them, the phase mapping algorithm includes:

Where ω represents the laser pulse angular frequency of the laser pulse used to achieve fluorescence lifetime imaging, and τ _i,j represents the fluorescence lifetime of the pixel point ij.

In the embodiment of the present invention, when the three-dimensional image is fluorescence lifetime imaging, the fluorescence lifetime of each pixel point can be obtained by the above method, and the fluorescence lifetime of each pixel point is used as the similarity index of each pixel point.

In addition, in the case that the three-dimensional image is Raman imaging, the similarity index of each pixel point can be obtained by the Pearson cross-correlation algorithm. Specifically, the above step 202 can be the following step C, specifically:

Step C: performing a pairwise cross-correlation operation on the Raman spectra of each pixel by using a preset Pearson cross-correlation algorithm to obtain a Raman spectral correlation coefficient matrix of each pixel point, and pulling the pixel points The MN spectral correlation coefficient matrix is used as an index of similarity of the respective pixel points.

Among them, in Raman imaging, the data of the three-dimensional image contains Raman spectroscopy, and the above Pearson cross-correlation algorithm is as follows:

Where C _l,m represents a Raman spectral correlation coefficient matrix, and R ^l and R ^m respectively represent Raman spectra of two different pixel points,

with

Representing the average of the two spectral lines, k is the kth data point in the Raman spectrum, and N is the total number of spectral data points, so the Raman spectral correlation coefficient matrix C _l,m is an N×N symmetric matrix.

In the embodiment of the present invention, after obtaining the Raman spectral coefficient matrix of each pixel, the analyzing device uses the Raman spectral coefficient matrix of each pixel as the similarity index of each pixel.

Step 203: Calculate a mean value, a variance, and an information entropy of the similarity index of the pathological slice image by using a similarity index of each pixel point;

In the embodiment of the present invention, after obtaining the similarity index of each pixel in the pathological slice image, the analyzing device performs averaging operation using the similarity index of each pixel point to obtain the similarity index of the pathological slice image. The mean value, and the variance index of each pixel is used to calculate the variance, and the variance of the similarity index of the pathological slice image is obtained, and the entropy operation is performed by using the similarity index of each pixel to obtain the information entropy of the pathological slice image. .

Specifically, if the three-dimensional image of the pathological slice image is fluorescence lifetime imaging, the fluorescence lifetime of each pixel point may be averaged, the variance calculation, and the entropy calculation performed, to obtain the mean, variance, and information entropy of the similarity index. Alternatively, if the three-dimensional image of the pathological slice image is Raman imaging, the upper triangular matrix element of the Raman spectral correlation coefficient matrix of each pixel point may be respectively used for averaging operation, variance calculation, and entropy calculation to obtain a similarity index. Mean, variance, and information entropy.

Among them, the calculation of information entropy can be defined by Shannon entropy definition or other information entropy. The definition formula of Shannon information entropy is as follows:

among them,

or

Where S denotes information entropy, p _i,j denotes the probability that the fluorescence lifetime of the pixel point ij occupies the sum of the fluorescence lifetimes of all the pixel points, or the Raman spectral correlation coefficient of the pixel point ij occupies the Raman spectral correlation coefficient of all the pixel points The probability of sum.

among them,

An upper triangular matrix element representing a matrix of Raman spectral correlation coefficients.

In the embodiment of the present invention, by processing each pathological slice image in the normal sample and the cancer sample according to the above steps 201 to 203, the mean, variance and information of the similarity index of each path slice image can be obtained. entropy.

Step 204: classify the mean value, the variance, and the information entropy of the similarity indicators of all the pathological slice images in the normal sample into the mean set, the variance set, and the information entropy set of the similarity index of the normal sample, respectively. The training data of the normal sample, and the mean, variance and information entropy of the similarity index of all the pathological slice images in the cancer sample are respectively classified into the mean set, the variance set and the information entropy of the similarity index of at least one category Assorted to serve as training data for the cancer sample;

In the embodiment of the present invention, the analyzing device obtains the mean set {μ _n }, the variance set {σ _n }, and the information entropy set {S _n } of the similarity index after the normal sample is classified, as the training of the normal sample. Data, and the cancer sample is classified into a mean set {μ _e } of the similarity index after at least one category, a variance set {σ _c }, and an information entropy set {S _e } as training data for the cancer sample. . It can be understood that since cancer can have many different periods, cancer samples can be classified based on different periods, for example, classified into 4 categories and the like.

Step 205: Train the preset machine classification model based on the training data of the normal sample and the cancer sample to obtain a trained machine classification model;

Step 206: Input the pathological slice image to be classified into the trained machine classification model, and use the type of the trained machine classification model output as the type of the pathological slice image to be classified.

In the embodiment of the present invention, the analyzing device trains the preset machine classification model based on the training data of the normal sample and the cancer sample, and inputs the pathological slice image to be classified into the trained machine classification model, by the trained The machine classification model performs classification to determine whether the pathological slice image is a normal slice image or a cancer slice image, and the type of the trained machine classification model output is used as the type of the pathological slice image, wherein the type of the pathological slice image may be It is a normal slice image or a cancer slice image.

Wherein, the above machine classification model may be a support vector machine neural network model, or a Bayesian linear or nonlinear classifier, or other linear or nonlinear classifier with machine learning function, in practical applications, The model used is selected according to specific needs, and is not limited herein.

In the embodiment of the present invention, the information entropy is introduced to characterize the similarity or chaos of the structure of the substance, and the degree of differentiation of the tissue cells is statistically and objectively quantitatively described, and the differentiation of the tissue cells can be directly reflected. The degree, combined with the mean and variance of the similarity index, as a set of criteria for distinguishing tumor cells or tissues from normal cells or tissues, can effectively improve the accuracy of intelligent classification of pathological slice images.

Please refer to FIG. 3 , which is a schematic diagram of functional modules of an intelligent classification device for pathological slice images according to a third embodiment of the present invention. The device includes:

The processing module 301 is configured to perform image processing on each of the preset normal sample and the cancer sample to obtain training data of the normal sample and the cancer sample, where the training data includes a similarity index Mean set, variance set and information entropy set;

In the embodiment of the present invention, it is necessary to train the machine classification model first to classify the classified pathological slice images using the machine classification model. In order to classify the machine classification model, it is necessary to prepare a training sample in advance, the training sample includes a normal sample and a cancer sample, wherein the normal sample includes a pathological slice image that is diagnosed as normal, and the cancer sample includes a diagnosis that has been cancerous. Pathological slice image.

The processing module 301 performs image processing on the preset normal sample and each pathological slice image in the cancer sample to obtain training data of the normal sample and the cancer sample, wherein the training data of the normal sample includes all normal pathological slices. The mean set of the similarity index of the image, the variance set of the similarity index, and the information entropy set. The training data of the cancer sample includes the mean set of the similarity index of all the cancerous pathological slice images, the variance set of the similarity index, and the information entropy. set.

The training module 302 is configured to train the preset machine classification model based on the training data of the normal sample and the cancer sample to obtain a trained machine classification model;

The classification module 303 is configured to input the pathological slice image to be classified into the trained machine classification model, and use the type of the trained machine classification model output as the type of the pathological slice image to be classified.

In the embodiment of the present invention, the training module 302 trains the preset machine classification model based on the training data of the normal sample and the cancer sample to obtain the trained machine classification model, and inputs the pathological slice image to be classified by the classification module 303. The trained machine classification model uses the type of the trained machine classification model as the type of the pathological slice image to be classified.

In the embodiment of the present invention, the classification device performs image processing on each of the preset normal samples and the cancer sample images, and obtains training data of the normal samples and the cancer samples, wherein, the training The data includes the mean set, the variance set and the information entropy set of the similarity index, and the preset machine classification model is trained based on the training data of the normal sample and the cancer sample, and the trained machine classification model is obtained, and the pathological slice image to be classified is to be classified. The trained machine classification model is input, and the type of the trained machine classification model output is used as the type of the pathological slice image to be classified. Compared with the prior art, not only the mean and variance of the similarity index are used to discriminate the difference between the normal slice image and the cancer slice image, but also the information entropy is introduced as an independent dimension of the degree of image structure confusion, and the quantitative can be achieved. Describe the purpose of the degree of differentiation of tumor cells or tissues, and train the machine classification model through the training data including the mean set, the variance set and the information entropy set of the similarity index of the normal sample and the cancer sample, and through the machine classification model pair The classification of pathological slice patterns makes it possible to effectively improve the accuracy of intelligent classification of pathological slice images.

FIG. 4 is a schematic diagram of functional modules of a path segmentation image intelligent classification device according to a fourth embodiment of the present invention. The device includes a processing module 301, a training module 302, and a classification module 303 in the third embodiment, and a third The content described in the embodiment is similar and will not be described here.

In the embodiment of the present invention, the processing module 301 includes: a reading module 401, an extracting module 402, a calculating module 403, and a categorizing module 404. The reading module 401, the extracting module 402, and the calculating module 403 are used for pairing Processing each of the normal sample and the pathological slice image of the cancer sample;

The reading module 401 is specifically configured to read a three-dimensional image of the pathological slice image containing structural information, where the three-dimensional image forms a third dimension by the photon number distribution of each pixel point;

In the embodiment of the present invention, for each pathological slice image, the reading module 401 will read the three-dimensional image of the pathological slice image containing the structural information, and the three-dimensional image is composed of the coordinates of each pixel to form the first and second dimensions, The photon number distribution of each pixel constitutes a third dimension. Wherein, in the three-dimensional image, data containing structural information is stored in a third dimension perpendicular to the pixel coordinates, and is a photon attenuation sequence in fluorescence lifetime imaging and a Raman spectrum in Raman imaging.

The extraction module 402 is specifically configured to extract a similarity indicator of each pixel determined by the third dimension data set in the three-dimensional image;

The calculating module 403 is specifically configured to calculate a mean value, a variance, and an information entropy of the similarity index of the pathological slice image by using a similarity index of each pixel point;

The categorization module 404 is configured to classify the mean value, the variance, and the information entropy of the similarity indicators of all the pathological slice images in the normal sample as the mean set, the variance set, and the information entropy of the similarity index of the normal sample, respectively. Collecting, as the training data of the normal sample, and classifying the mean, variance and information entropy of the similarity index of all pathological slice images in the cancer sample into a mean set of similarity indicators of at least one category, A set of variances and a set of information entropy are used as training data for the cancer sample.

In the embodiment of the present invention, the extraction module 402 will extract the similarity index of each pixel point determined by the third dimension data set in the three-dimensional image.

Wherein, the three-dimensional image may be fluorescence lifetime imaging or Raman imaging.

Among them, the related photon counting (TCSPC) promotes fluorescence lifetime imaging (FLIM), and the fluorescence lifetime of the fluorophore is considered to be only related to its structure and microenvironment, and is not affected by excitation light intensity, molecular concentration, etc., so it can be used to characterize The degree of similarity between the marked material structures. In addition, the Raman spectrum reflects the internal karyotype structure of the material. The Raman spectral correlation coefficient matrix is obtained by cross-correlation operation on the Raman spectrum. The Raman spectral correlation coefficient matrix can further reflect the similarity of the internal structure of the structure. Both the fluorescence lifetime and the Raman spectral correlation coefficient matrix can be used as a quantitative index for discriminating the degree of similarity between pixels of a medical image, that is, a calculation parameter as a similarity index. With quantitative indicators of image similarity, statistical analysis of similarity can be performed on medical images. In normal tissues, the degree of cell differentiation is high, and there is a wide similarity distribution between each pixel. The degree of differentiation of tumor tissue is manifested as the structural difference of each pixel is smaller, the similarity distribution is relatively narrow, and the tumor is tumor. The higher the malignancy, the lower the differentiation, and the similarity distribution is concentrated. In general, the similarity index of the pathological slice image can be averaged, and the mean and variance of the similarity index can be obtained as two independent indicators for discriminating the difference between normal cells or tissues and cancer cells and tissues. Furthermore, information entropy can be further introduced as an independent dimension index of the degree of image structure disorder, which can quantitatively describe the degree of differentiation of tumor cells or tissues, and the mean and variance of the similarity index are used as the differentiation of tumor cells or tissues from normal cells or tissues. A set of criteria criteria to improve accuracy is the principle of the invention.

If the three-dimensional image is fluorescence lifetime imaging, the fluorescence lifetime or phase mapping coordinates may be calculated by a fitting or phase mapping algorithm, and the extraction module 402 is specifically configured to:

The fluorescence lifetime of each pixel is obtained by least square fitting using the acquired time decay curve corresponding to each pixel point, and the fluorescence lifetime of each pixel is used as the similarity index of each pixel.

Among them, the time decay curve of the pixel is:

I _i,j (t)=I ₀ ^i,j exp(-t/τ _i,j )

Where i and j represent the coordinates of the pixel in the three-dimensional image, I _i,j (t) represents the fluorescence intensity of the pixel point ij after decay at time t, t represents time, I ₀ ^i,j represents the total pixel point ij The fluorescence intensity, τ _i,j represents the fluorescence lifetime of the pixel point ij. Wherein, the total fluorescence intensity of the pixel point ij can be determined based on the pixel point ij photon number attenuation sequence in the three-dimensional image.

or,

The extraction module 402 is specifically configured to calculate a fluorescence lifetime of each pixel by using a preset phase mapping algorithm, and use a fluorescence lifetime of each pixel as a similarity index of each pixel.

Among them, the phase mapping algorithm includes:

Where ω represents the laser pulse angular frequency of the laser pulse used to achieve fluorescence lifetime imaging, and τ _i,j represents the fluorescence lifetime of the pixel point ij.

In the embodiment of the present invention, when the three-dimensional image is fluorescence lifetime imaging, the fluorescence lifetime of each pixel point can be obtained by the above method, and the fluorescence lifetime of each pixel point is used as the similarity index of each pixel point.

In addition, in the case that the three-dimensional image is Raman imaging, the similarity index of each pixel point can be obtained by the Pearson cross-correlation algorithm. Specifically, the extraction module 402 is specifically configured to:

Using a preset Pearson cross-correlation algorithm to perform a pairwise cross-correlation operation on the Raman spectra of each pixel to obtain a Raman spectral correlation coefficient matrix of each pixel point, and correlate the Raman spectra of the respective pixel points. The coefficient matrix is used as an index of similarity of the respective pixel points.

Among them, in Raman imaging, the data of the three-dimensional image contains Raman spectroscopy, and the above Pearson cross-correlation algorithm is as follows:

Where C _l,m represents a Raman spectral correlation coefficient matrix, and R ^l and R ^m respectively represent Raman spectra of two different pixel points,

with

Representing the average of the two spectral lines, k is the kth data point in the Raman spectrum, and N is the total number of spectral data points, so the Raman spectral correlation coefficient matrix C _l,m is an N×N symmetric matrix. In the embodiment of the present invention, after obtaining the Raman spectral coefficient matrix of each pixel, the analyzing device uses the Raman spectral coefficient matrix of each pixel as the similarity index of each pixel.

In the embodiment of the present invention, after the analyzing device obtains the similarity index of each pixel in the pathological slice image, the calculating module 403 performs averaging operation using the similarity index of each pixel to obtain the pathological slice image. The mean value of the similarity index is calculated by using the similarity index of each pixel to calculate the variance of the similarity index of the pathological slice image, and the entropy operation is performed by using the similarity index of each pixel to obtain the pathological slice image. Information entropy.

Specifically, if the three-dimensional image of the pathological slice image is fluorescence lifetime imaging, the calculation module 403 can perform the averaging operation, the variance calculation, and the entropy calculation on the fluorescence lifetimes of the respective pixel points to obtain the mean value and the variance of the similarity index. The information entropy, or if the three-dimensional image of the pathological slice image is Raman imaging, the calculation module 403 can perform the averaging operation, the variance calculation, and the entropy by using the upper triangular matrix elements of the Raman spectral correlation coefficient matrix of each pixel point. The operation obtains the mean, variance and information entropy of the similarity index.

Among them, the calculation of information entropy can be defined by Shannon entropy definition or other information entropy. The definition formula of Shannon information entropy is as follows:

among them,

or

Where S denotes information entropy, p _i,j denotes the probability that the fluorescence lifetime of the pixel point ij occupies the sum of the fluorescence lifetimes of all the pixel points, or the Raman spectral correlation coefficient of the pixel point ij accounts for the Raman spectrum of all the pixel points The probability of the sum of the coefficients;

among them,

An upper triangular matrix element representing a matrix of Raman spectral correlation coefficients.

In the embodiment of the present invention, the categorization module 404 obtains the mean set {μ _n }, the variance set {σ _n }, and the information entropy set {S _n } of the similarity index after the normal sample is classified as the normal sample. Training data, and the cancer sample is classified into a mean set {μ _c }, a variance set {σ _c }, and an information entropy set {S _c } of the similarity index after at least one category, as a cancer sample. Training data. It can be understood that since cancer can have many different periods, cancer samples can be classified based on different periods, for example, classified into four categories and the like.

In the embodiment of the present invention, the training module 302 trains the preset machine classification model based on the training data of the normal sample and the cancer sample, and the classification module 303 inputs the pathological slice image to be classified into the trained machine classification model. Sorting by the trained machine classification model to determine whether the pathological slice image is a normal slice image or a cancer slice image, and the type of the trained machine classification model output is used as the type of the pathological slice image, wherein the pathology The type of the slice image may be a normal slice image or a cancer slice image.

Wherein, the above machine classification model may be a model in a support vector machine neural network, or a Bayesian linear or nonlinear classifier, or other linear or nonlinear classifier with machine learning function, in practical applications. The model used can be selected according to specific needs, and is not limited herein.

In the embodiment of the present invention, the information entropy is introduced to characterize the similarity or chaos of the structure of the substance, and the degree of differentiation of the tissue cells is statistically and objectively quantitatively described, and the degree of differentiation of the tissue cells can be directly reflected, and the similarity index is combined. Mean and variance are a set of criterion criteria for distinguishing tumor cells or tissues from normal cells or tissues, which can effectively improve the accuracy of intelligent classification of pathological slice images.

In order to verify that the above method and device can improve the accuracy of intelligent classification of pathological slice images, the following experiments are performed:

Pathological sections of normal skin tissue and skin cancer tissue samples stained with HE were obtained from the hospital dermatology, and normal samples and cancer samples were separately labeled.

A femtosecond laser with a wavelength of 780 nm and a repetition rate of 75.4 MHz was used as the excitation source. Two-photon fluorescence lifetime imaging analysis was performed on the LEICA DMIRE2 confocal microscope system with TCSPC from B&H. All samples were imaged using a 60x objective.

The acquired two-photon fluorescence lifetime image is imported into MATLAB, and a program (which can be a program corresponding to the intelligent classification method for pathological slice images in the embodiment of the present invention) is used to perform phase mapping calculation on the fluorescence lifetime image, and pixels with similar structures mean The fluorescence lifetimes are also similar. In the phase coordinates, the data points clustered in one block, and the structure in the fluorescence lifetime image can be segmented according to the phase coordinate clustering. The purpose of this step is to segment the tissue part of the melanocytes from the image and extract the fluorescence lifetime of the HE stain for the mean statistics and entropy calculation.

Statistical analysis of mean μ, variance σ and entropy S was performed on the collected 50 sets of cancer cell image data and 50 normal cell image data. The distribution of all data points in the three-dimensional space of μ_σ_S is shown in Figure 5a, which is convenient for observation. We labeled the cancer sample data in dark black, as indicated by number A, and grayed out normal sample data, as indicated by number B, and mapped the data distribution to three different planes.

The sample data is linearly classified by the support vector machine on the μ_σ plane, as shown in Fig. 5b. It can be seen from the figure that the data of the normal sample and the cancer sample are partially intertwined on the μ_σ plane, and cannot be linearly classified, indicating that the possibility of misjudgment can be made only when the fluorescence lifetime mean and the variance are used for the pathological slice diagnosis. great.

After introducing the entropy value, the sample data can obtain very obvious linear classification effects by performing support vector machine linear classification on the S_μ plane (Fig. 5c) or the S_σ plane (Fig. 5d).

Finally, we randomly selected the training samples and test samples from the normal sample and the cancer sample images in a 7:3 ratio for cross-validation experiments. The cross-verification result of support vector machine using linear kernel is that when only two parameters of μ and σ are used for training and discrimination of support vector machine, the linear discriminant accuracy is 85.9%; and the three parameters of μ, σ and S are adopted. When the support vector machine is trained and discriminated, the linear discriminant accuracy is 97.2%, which is obviously better than the linear discriminant result of the two parameters.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be another division manner, for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.

The modules described as separate components may or may not be physically separated. The components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. in. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

The integrated modules, if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

It should be noted that, for the foregoing method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are all focused, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

The above is a description of an intelligent classification method and apparatus for pathological slice images provided by the present invention. For those skilled in the art, according to the idea of the embodiments of the present invention, there will be changes in specific implementation modes and application scopes. In summary, the content of the specification should not be construed as limiting the invention.

Claims (10)

1. An intelligent classification method for pathological slice images, characterized in that the method comprises:

   Performing image processing on each of the preset normal sample and the cancer sample image to obtain training data of the normal sample and the cancer sample, the training data including a mean set of similarity indicators, a variance set, and Information entropy set;

   And training the preset machine classification model based on the training data of the normal sample and the cancer sample to obtain a trained machine classification model;

   The pathological slice image to be classified is input to the trained machine classification model, and the type of the trained machine classification model output is used as the type of the pathological slice image to be classified.
2. The method according to claim 1, wherein the image processing is performed on each of the preset normal sample and the cancer sample to obtain training data of the normal sample and the cancer sample. The steps include:

   Performing the following processing on each of the normal sample and the pathological slice image in the cancer sample:

   Reading a three-dimensional image of the pathological slice image containing structural information, the three-dimensional image being composed of a photon number distribution of each pixel point to form a third dimension;

   Extracting a similarity index of each pixel point determined by the third dimension data set in the three-dimensional image;

   Calculating a mean value, a variance and an information entropy of the similarity index of the pathological slice image by using the similarity index of each pixel point;

   The steps further include:

   Mean, variance and information entropy of the similarity index of all pathological slice images in the normal sample are respectively classified into a mean set, a variance set and an information entropy set of the similarity index of the normal sample, as the normal sample Training data, and averaging, variance, and information entropy of the similarity index of all pathological slice images in the cancer sample are respectively classified into a mean set, a variance set, and an information entropy set of the similarity index of at least one category, As training data for the cancer sample.
3. The method according to claim 2, wherein if the three-dimensional image is fluorescence lifetime imaging, the step of extracting the similarity index of each pixel determined by the third dimensional data set in the three-dimensional image comprises :

   Using the time decay curve corresponding to each pixel point collected, the least squares fitting is used to obtain each pixel. a fluorescence lifetime of a point, wherein a fluorescence lifetime of each pixel point is used as an index of similarity of each pixel point;

   or,

   The fluorescence lifetime of each pixel point is calculated by a preset phase mapping algorithm, and the fluorescence lifetime of each pixel point is used as an index of similarity of each pixel point.
4. The method according to claim 3, wherein if the three-dimensional image is Raman imaging, the step of extracting the similarity index of each pixel determined by the third dimensional data set in the three-dimensional image comprises :

   Using a preset Pearson cross-correlation algorithm to perform a pairwise cross-correlation operation on the Raman spectra of each pixel to obtain a Raman spectral correlation coefficient matrix of each pixel point, and correlate the Raman spectra of the respective pixel points. The coefficient matrix is used as an index of similarity of the respective pixel points.
5. The method according to any one of claims 1 to 3, characterized in that the machine classification model is a support vector machine neural network model or a Bayesian-based linear or nonlinear classifier.
6. An intelligent classification device for pathological slice images, characterized in that the device comprises:

   a processing module, configured to perform image processing on each of the preset normal sample and the cancer sample, to obtain training data of the normal sample and the cancer sample, wherein the training data includes an average of similarity indicators Sets, variance sets, and information entropy sets;

   a training module, configured to train a preset machine classification model based on training data of the normal sample and the cancer sample, to obtain a trained machine classification model;

   And a classification module, configured to input the pathological slice image to be classified into the trained machine classification model, and use the type of the trained machine classification model output as the type of the pathological slice image to be classified.
7. The apparatus according to claim 6, wherein the processing module comprises: a reading module, an extracting module, a calculating module, and a categorizing module, wherein the reading module, the extracting module, and the calculating module are used for pairing Processing each of the normal sample and the pathological slice image of the cancer sample;

   The reading module is specifically configured to read a three-dimensional image of the pathological slice image containing structural information, where the three-dimensional image forms a third dimension by the photon number distribution of each pixel point;

   The extraction module is specifically configured to extract a similarity indicator of each pixel determined by the third dimension data set in the three-dimensional image;

   The calculating module is specifically configured to calculate the pathological slice by using a similarity indicator of each pixel point Mean, variance and information entropy of the similarity index of the image;

   The categorization module is configured to classify the mean value, the variance, and the information entropy of the similarity index of all the pathological slice images in the normal sample as the mean set, the variance set, and the information entropy set of the similarity index of the normal sample, respectively. As the training data of the normal sample, and the mean, variance and information entropy of the similarity index of all the pathological slice images in the cancer sample are respectively classified into the mean set and the variance of the similarity index of at least one category. A set of sets of information and entropy is used as training data for the cancer sample.
8. The apparatus according to claim 7, wherein if the three-dimensional image is fluorescence lifetime imaging, the extraction module is specifically configured to:

   Performing a least squares fitting on the time decay curve corresponding to each pixel point to obtain a fluorescence lifetime of each pixel point, and using the fluorescence lifetime of each pixel point as a similarity index of each pixel point;

   or,

   The fluorescence lifetime of each pixel point is calculated by a preset phase mapping algorithm, and the fluorescence lifetime of each pixel point is used as an index of similarity of each pixel point.
9. The apparatus according to claim 7, wherein if the three-dimensional image is Raman imaging, the extraction module is specifically configured to:

   Using a preset Pearson cross-correlation algorithm to perform a pairwise cross-correlation operation on the Raman spectra of each pixel to obtain a Raman spectral correlation coefficient matrix of each pixel point, and correlate the Raman spectra of the respective pixel points. The coefficient matrix is used as an index of similarity of the respective pixel points.
10. The apparatus according to any one of claims 6 to 9, wherein the machine classification model is a support vector machine neural network model or a Bayesian-based linear or nonlinear classifier.

Images