CN110969204A - Sample classification system based on fusion of magnetic resonance image and digital pathology image - Google Patents
Sample classification system based on fusion of magnetic resonance image and digital pathology image Download PDFInfo
- Publication number
- CN110969204A CN110969204A CN201911199840.4A CN201911199840A CN110969204A CN 110969204 A CN110969204 A CN 110969204A CN 201911199840 A CN201911199840 A CN 201911199840A CN 110969204 A CN110969204 A CN 110969204A
- Authority
- CN
- China
- Prior art keywords
- image
- region
- interest
- module
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007170 pathology Effects 0.000 title claims abstract description 34
- 230000004927 fusion Effects 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 94
- 238000012545 processing Methods 0.000 claims abstract description 44
- 238000012216 screening Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 21
- 230000001575 pathological effect Effects 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000013145 classification model Methods 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000000926 separation method Methods 0.000 claims description 11
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000010801 machine learning Methods 0.000 claims description 7
- 210000004027 cell Anatomy 0.000 claims description 6
- 210000003855 cell nucleus Anatomy 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 6
- 210000000805 cytoplasm Anatomy 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 238000000528 statistical test Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000010186 staining Methods 0.000 claims description 3
- 238000011425 standardization method Methods 0.000 claims description 3
- 238000010998 test method Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000007473 univariate analysis Methods 0.000 claims description 3
- 230000010339 dilation Effects 0.000 claims description 2
- 238000004043 dyeing Methods 0.000 claims description 2
- 238000012552 review Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract description 3
- 206010028980 Neoplasm Diseases 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001086 cytosolic effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000035876 healing Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Abstract
The invention belongs to the technical field of medical image processing, and particularly relates to a sample classification system based on fusion of magnetic resonance images and digital pathological images, aiming at solving the problem of low sample classification precision caused by the limitation of single sample classification of the existing imaging or pathology. The system comprises an image acquisition module, a first image acquisition module, a second image acquisition module and a display module, wherein the image acquisition module is configured to acquire a first image and a second image; the preprocessing module is configured to preprocess the first image and the second image; the delineation module is configured to acquire the region of interest of each preprocessed image through a delineation method and further process the region of interest; the extraction characteristic module is configured to respectively extract the characteristics of the processed interested areas of the images; the screening and sorting module is configured to screen and sort the extracted features; and the classification output module is configured to obtain classification results of the screened and sequenced features through a classification model. The invention fuses the magnetic resonance image and the digital pathological image, solves the defect of single sample classification and improves the classification precision.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a sample classification system based on fusion of magnetic resonance images and digital pathological images.
Background
In recent years, image fusion classification has become a new technology with important application value in the field of image understanding and computer vision, especially in the field of medical engineering, different medical images provide different medical information for doctors due to different imaging mechanisms and application environments of medical instruments, and the image fusion classification can integrate various valuable information together and becomes an important means for clinical diagnosis and medical research. Among them, the multi-modality magnetic resonance image and the digital pathology image are the more commonly used images for clinical disease diagnosis.
Macroscopic multi-modal magnetic resonance images can comprehensively reflect sample information, but the accuracy is limited, and the method is different from the real histopathology 'gold standard'. The digital pathological images can reflect microscopic information such as tissue component change, cell activity, nuclear division degree and the like, but cannot reflect comprehensive overall information. Therefore, the traditional sample classification based on imaging or sample classification based on pathology only starts from the perspective of single subject knowledge, and observation information of different scales are isolated from each other, so that the final classification efficiency and robustness are influenced.
However, both the multi-modality mr images and the digital pathology image information contain a large amount of high-dimensional and quantifiable incompletely-explored information, and these high-dimensional feature information can be used to construct an accurate sample classification by means of pattern recognition. Therefore, observation information of different scales, namely macroscopic information and microscopic information, can be combined to effectively help reveal more comprehensive information, so that more accurate sample classification can be realized.
Disclosure of Invention
In order to solve the problems in the prior art, namely to solve the problem that the sample classification precision is low due to the limitation of single sample classification of the existing imaging or pathology, the invention provides a sample classification system based on the fusion of a magnetic resonance image and a digital pathology image, which comprises an image acquisition module, a preprocessing module, a delineation module, a feature extraction module, a screening and sorting module and a classification output module;
the image acquisition module is configured to acquire a first image and a second image; the first image is a multi-modality magnetic resonance image, and the second image is a digital pathology image;
the preprocessing module is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the delineation module is configured to respectively acquire a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image through a delineation method; performing expansion processing on the first region of interest to obtain a third region of interest; performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest;
the feature extraction module is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on a feature extraction method for presetting multiple preset categories, so as to construct a feature set;
the screening and sorting module is configured to acquire a refined feature set for the feature set through a preset feature screening method and an importance sorting method;
the classification output module is configured to obtain a classification result through a preset classification model based on the refined feature set.
In some preferred embodiments, the preprocessing module "preprocesses the first image and the second image to obtain a first preprocessed image and a second preprocessed image", and the method includes:
based on the first image and the second image, carrying out image standardization processing by a Z-score standardization method to obtain a third image and a fourth image;
carrying out gray distribution optimization processing on the third image by a window width and window level method to obtain a fifth image; carrying out color correction processing on the fourth image by a Macenko color correction method to obtain a sixth image;
and based on the fifth image and the sixth image, performing noise reduction processing by a super-resolution method based on a depth convolution neural network to obtain a first preprocessed image and a second preprocessed image.
In some preferred embodiments, the method of "performing dilation processing on the first region of interest to obtain a third region of interest" in the delineation module includes: and performing expansion processing on the first region of interest through a filter with a preset size of N x N to obtain a third region of interest.
In some preferred embodiments, the method of "performing rasterization processing, color channel separation processing, and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest" in the delineation module includes:
dividing the second region of interest into a plurality of M x M sized regions;
obtaining dyeing channels of each divided region by a color channel separation method;
a cell nucleus region, a cytoplasm region and a cell region are obtained by a threshold segmentation method based on each staining channel, and a region composed of the cell nucleus region, the cytoplasm region and the cell region is used as a fourth region of interest.
In some preferred embodiments, the second region of interest is a region determined after the second preprocessed image is subjected to delineation by a delineation method under L-magnification and boundary review on the delineation region under 2L-magnification.
In some preferred embodiments, the extracted features of the third region of interest include: first order statistical characteristics, morphological characteristics, textural characteristics, wavelet transformation characteristics.
In some preferred embodiments, the extracted features of the fourth region of interest include: the first-order statistical features, the morphological features and the features of which the morphological features are constructed by means of mean values, variances, median values, decile values and quartile values.
In some preferred embodiments, the classification output module "acquires a refined feature set by a preset feature screening method and an importance ranking method", and the method includes: and screening each feature in the feature set by a univariate analysis method and a collinearity test method, and sequencing the importance of the features by a statistical test method and a machine learning method after screening.
In some preferred embodiments, the preset classification model is obtained by training through a machine learning method.
The invention has the beneficial effects that:
the invention fuses the magnetic resonance image and the digital pathological image, solves the defect of single sample classification and improves the classification precision. The invention obtains standard images by preprocessing the multi-mode magnetic resonance images and the digital pathological images. And acquiring an interested region based on the preprocessed image by a delineation method, and further processing the interested regions of the multi-mode magnetic resonance image and the digital pathological image respectively to obtain an accurate interested region, thereby ensuring the accuracy and robustness of subsequent feature extraction. And acquiring and fusing the features of the images with different scales based on the extracted region of interest, and screening out redundant features and features which do not meet the requirements of statistical test through analysis and statistics to acquire refined features. And obtaining a classification result through a classification model trained based on a machine learning method according to the refined characteristics. The image is more comprehensively described, the defect of single sample classification is overcome, and therefore the classification precision and stability are improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a sample classification system based on fusion of magnetic resonance images and digital pathology images according to an embodiment of the present invention;
FIG. 2 is a schematic illustration of the effect of the method of the present invention compared to other methods according to one embodiment of the present invention;
FIG. 3 is a diagram illustrating the detailed processing procedure of the sample classification system based on the fusion of the magnetic resonance image and the digital pathology image according to an embodiment of the present invention;
fig. 4 is a flowchart of a sample classification method based on fusion of a magnetic resonance image and a digital pathology image according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
As shown in fig. 1, the system for classifying samples based on fusion of magnetic resonance images and digital pathology images of the present invention includes an image acquisition module 100, a preprocessing module 200, a delineation module 300, a feature extraction module 400, a screening and sorting module 500, and a classification output module 600;
the image acquiring module 100 is configured to acquire a first image and a second image; the first image is a multi-modality magnetic resonance image, and the second image is a digital pathology image;
the preprocessing module 200 is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the delineating module 300 is configured to obtain a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image respectively by a delineating method; performing expansion processing on the first region of interest to obtain a third region of interest; performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest;
the feature extraction module 400 is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on preset feature extraction methods of multiple preset categories to construct a feature set;
the screening and sorting module 500 is configured to obtain a refined feature set for the feature set through a preset feature screening method and an importance sorting method;
the classification output module 600 is configured to obtain a classification result through a preset classification model based on the refined feature set.
In order to more clearly describe the sample classification system based on the fusion of the magnetic resonance image and the digital pathology image, the following describes each module in various embodiments of the system in detail with reference to the accompanying drawings.
An image acquisition module 100 configured to acquire a first image and a second image; the first image is a multi-modality magnetic resonance image and the second image is a digital pathology image.
In this embodiment, first, a multi-modality magnetic resonance image and a digital pathology image of a corresponding part of a patient are acquired as a first image and a second image. The present invention is preferably used to treat a long tumor site of a patient, and in other embodiments, sites corresponding to other diseases of the patient may be selected.
The preprocessing module 200 is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image.
In this embodiment, an input image pair is processed by four image preprocessing methods, i.e., image normalization, color normalization, image gray scale distribution optimization, and image super-resolution enhancement, as shown in fig. 3: the specific treatment is as follows:
and carrying out image standardization processing by a Z-score standardization method based on the first image and the second image to obtain a third image and a fourth image.
And performing gray distribution optimization processing on the third image by a window width and window level method, fully utilizing a display effective value range between 0 and 255, and reducing loss caused by value range compression as much as possible to obtain a fifth image. And carrying out color correction processing on the fourth image by a Macenko color correction method, correcting the color space of the digital pathological image, and obtaining a sixth image.
Based on the fifth image and the sixth image, noise reduction processing is carried out through a super-resolution method based on a depth convolution neural network, noise influence is weakened, the signal to noise ratio of the magnetic resonance image and the digital pathological image data is improved, and a first preprocessing image and a second preprocessing image are obtained.
The rest of fig. 3 is detailed below.
The delineating module 300 is configured to obtain a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image respectively by a delineating method; performing expansion processing on the first region of interest to obtain a third region of interest; and performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest.
In this embodiment, based on the first preprocessed image and the second preprocessed image, the regions of interest of each image are respectively extracted by a delineation method. I.e. establishing tumor boundaries and regions of interest. The drawing method comprises automatic and semi-automatic drawing. Manual delineation may also be employed during the training phase.
And for the first preprocessed image, obtaining an interested region by a delineation method, namely segmenting the tumor region to be used as a first interested region, performing image expansion operation on the interested region by adopting a filter with a preset size of N x N, realizing outward expansion of the boundary, obtaining an edge interested region, and using the edge interested region as a third interested region. In the present invention, the size of the filter is preferably 3 × 3.
And for the second preprocessed image, obtaining a delineation area by a delineation method under the L multiplying power, and performing boundary rechecking on the delineation area under the 2L multiplying power to obtain a determined area as a second interested area. In the invention, preferably, the tumor region is delineated by a delineation method under 10 magnifications, and the delineation boundary rechecking is performed under 20 magnifications, but in other embodiments, the magnification can be selected according to actual requirements. And dividing the second region of interest into a plurality of M-by-M regions, namely performing rasterization operation to ensure that the regions are not overlapped, and performing color channel separation on the regions. Because the digital pathological image is acquired based on hematoxylin-eosin staining (HE), the digital pathological image is separated into an H channel and an E channel after channel separation. And in the separated H channel, obtaining a foreground object, namely a cell nucleus, by a threshold segmentation method, and obtaining a foreground object, namely a cytoplasm region, by an E channel by the threshold segmentation method. The intersection of the nuclear foreground and the cytoplasmic foreground constitutes the cellular region. The region of interest of the pathology image is composed of three parts, namely a cell nucleus region, a cytoplasm region and a cell region, namely a corresponding fourth region of interest. In the present invention, the size of the region is preferably 512 × 512. In fig. 3, in consideration of the layout of the map, the region-of-interest segmented portion of the second preprocessed image is included in the feature extraction, and does not substantially affect the feature extraction.
The feature extraction module 400 is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on feature extraction methods preset for multiple preset categories, so as to construct a feature set.
In this embodiment, different features are extracted for the third region of interest and the fourth region of interest.
The extraction of four features of the third region of interest can also be called image feature extraction, and the three features comprise first-order statistical features, morphological features, texture features and wavelet transformation features.
For the fourth region of interest, the extracted features, which may also be referred to as pathological features, include first-order statistical features, morphological features, and pathological image features of which the morphological features are constructed by mean, variance, median, decile value, and quartile value.
Based on the iconography characteristics and the pathology characteristics, namely the multi-modal magnetic resonance image characteristics and the digital pathology image characteristics are combined, a characteristics pool of the iconography characteristics and the pathology characteristics is constructed, and the combined characteristics set is obtained. In addition, in this embodiment, the feature extraction methods of multiple preset categories are preset as methods corresponding to the features extracted from the third region of interest and the fourth region of interest. For example, the first-order statistical features correspond to a first-order statistical feature extraction method, and other similar processes are not described in detail herein.
And the screening and sorting module 500 is configured to acquire a refined feature set for the feature set through a preset feature screening method and an importance sorting method.
In the embodiment, a univariate analysis method and a collinearity test method are adopted to eliminate the features which do not accord with the statistical test hypothesis and the redundancy, information strategy fusion is realized, feature screening is performed, feature importance ranking is obtained by adopting a mode of combining the statistical test method and a machine learning method, and a refined feature set is obtained after ranking is completed.
The classification output module 600 is configured to obtain a classification result through a preset classification model based on the refined feature set.
In this embodiment, the classification model obtained by the machine learning method pre-training obtains an accurate classification result, that is, determines the quality, the grade, and the type of classification of the tumor, and if the classification model is a multi-modal magnetic resonance image or a digital pathology image of the tumor to be treated, the obtained classification result further includes a treatment effect and a survival time after healing.
The filtering and sorting module 500-the classification output module 600 correspond to the information fusion and modeling part in fig. 4.
In addition, a refined feature extraction system based on the fusion of the magnetic resonance image and the digital pathological image can be constructed based on the image acquisition module 100 and the screening and sorting module 500, and the refined features after the fusion of the magnetic resonance image and the digital pathological image are extracted, so that the problem of low classification accuracy caused by the fact that the extracted features of a classification model are not accurate and comprehensive enough when samples are classified is solved.
It should be noted that, the sample classification system based on fusion of a magnetic resonance image and a digital pathology image provided in the above embodiment is only illustrated by dividing the above functional modules, and in practical applications, the above functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the above embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
A second embodiment of the present invention provides a sample classification method based on fusion of a magnetic resonance image and a digital pathology image, as shown in fig. 4, including the following steps:
step S100, acquiring a first image and a second image; the first image is a multi-modality magnetic resonance image, and the second image is a digital pathology image;
step S200, preprocessing the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
step S300, respectively acquiring a first region of interest and a second region of interest corresponding to the first preprocessing image and the second preprocessing image through a delineation method; performing expansion processing on the first region of interest to obtain a third region of interest; performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest;
step S400, respectively extracting the characteristics of multiple preset categories of the third region of interest and the fourth region of interest based on a characteristic extraction method for presetting multiple preset categories, and constructing a characteristic set;
step S500, for the characteristic set, obtaining a refined characteristic set through a preset characteristic screening method and an importance sorting method;
and S600, obtaining a classification result through a preset classification model based on the refined feature set.
For training and testing of the method of the invention, 981 real data sets with both multi-modal magnetic resonance images and digital pathology images were used for testing in the embodiment, 303 of which were used for training and 678 of which were used for testing the classification effect of the invention.
Compared with the existing classification methods (a pure image method and a pure pathology method), the method provided by the invention has the advantages that the Receiver Operating Characteristic (ROC) curve and the area under the curve (AUC) of the classification method are obtained, and the ROC curve and the AUC are used as the measurement of the performance of the classifier.
The classification ROC curves for both methods on the real experimental dataset are shown in fig. 2, respectively, where true positive rate refers to the percentage of actually positive and correctly determined as positive using the method, and false positive rate refers to the percentage of actually negative and incorrectly determined as positive according to the method. As shown in fig. 2, the ROC curve for this method is higher than the other two methods for most of the threshold range, and the AUC values compare: the AUC value of this method was 0.868, the AUC value of the pure image method was 0.752, the AUC of the pure pathology method was 0.743, and the AUC of the method of randomly classifying samples was 0.5. The area under the curve (AUC) measures the overall classification performance, with the greater the AUC value, the better the overall performance of the classification method. Therefore, the method has better effect than other two methods.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the method described above may refer to the corresponding process in the foregoing system embodiment, and are not described herein again.
A storage device according to a third embodiment of the present invention stores a plurality of programs, which are adapted to be loaded by a processor and to implement the above-described method for classifying a sample based on fusion of a magnetic resonance image and a digital pathology image.
A processing apparatus according to a fourth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is adapted to be loaded and executed by a processor to implement the above-described method of sample classification based on fusion of magnetic resonance images with digital pathology images.
It can be clearly understood by those skilled in the art that, for convenience and brevity, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method examples, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether these functions are performed in electronic hardware or software depends on the intended application of the solution and design constraints. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (9)
1. A sample classification system based on fusion of magnetic resonance images and digital pathological images is characterized by comprising an image acquisition module, a preprocessing module, a delineation module, a feature extraction module, a screening and sorting module and a classification output module;
the image acquisition module is configured to acquire a first image and a second image; the first image is a multi-modality magnetic resonance image, and the second image is a digital pathology image;
the preprocessing module is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the delineation module is configured to respectively acquire a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image through a delineation method; performing expansion processing on the first region of interest to obtain a third region of interest; performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest in sequence to obtain a fourth region of interest;
the feature extraction module is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on a feature extraction method for presetting multiple preset categories, so as to construct a feature set;
the screening and sorting module is configured to acquire a refined feature set for the feature set through a preset feature screening method and an importance sorting method;
the classification output module is configured to obtain a classification result through a preset classification model based on the refined feature set.
2. The system for classifying samples based on the fusion of magnetic resonance image and digital pathology image according to claim 1, wherein the preprocessing module "preprocesses the first image and the second image to obtain a first preprocessed image and a second preprocessed image" by:
based on the first image and the second image, carrying out image standardization processing by a Z-score standardization method to obtain a third image and a fourth image;
carrying out gray distribution optimization processing on the third image by a window width and window level method to obtain a fifth image; carrying out color correction processing on the fourth image by a Macenko color correction method to obtain a sixth image;
and based on the fifth image and the sixth image, performing noise reduction processing by a super-resolution method based on a depth convolution neural network to obtain a first preprocessed image and a second preprocessed image.
3. The system of claim 1, wherein the delineation module performs dilation on the first region of interest to obtain a third region of interest by: and performing expansion processing on the first region of interest through a filter with a preset size of N x N to obtain a third region of interest.
4. The sample classification system based on the fusion of the magnetic resonance image and the digital pathology image according to claim 1, wherein in the delineation module, "the second region of interest is subjected to rasterization, color channel separation, and threshold segmentation in sequence to obtain a fourth region of interest", and the method thereof is as follows:
dividing the second region of interest into a plurality of M x M sized regions;
obtaining dyeing channels of each divided region by a color channel separation method;
a cell nucleus region, a cytoplasm region and a cell region are obtained by a threshold segmentation method based on each staining channel, and a region composed of the cell nucleus region, the cytoplasm region and the cell region is used as a fourth region of interest.
5. The system according to claim 4, wherein the second region of interest is determined by performing a delineation method on the second preprocessed image at an L-magnification and performing a boundary review on the delineation region at a 2L-magnification.
6. The system of claim 1, wherein the extracted features of the third region of interest include: first order statistical characteristics, morphological characteristics, textural characteristics, wavelet transformation characteristics.
7. The system of claim 1, wherein the extracted features of the fourth region of interest include: the first-order statistical features, the morphological features and the features of which the morphological features are constructed by means of mean values, variances, median values, decile values and quartile values.
8. The system for classifying samples based on the fusion of the magnetic resonance image and the digital pathological image according to claim 1, wherein the classification output module "acquires the refined feature set by a preset feature screening method and an importance ranking method" includes: and screening each feature in the feature set by a univariate analysis method and a collinearity test method, and sequencing the importance of the features by a statistical test method and a machine learning method after screening.
9. The system for classifying samples based on the fusion of magnetic resonance images and digital pathology images according to claim 1, wherein the preset classification model is obtained by training through a machine learning method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199840.4A CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199840.4A CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110969204A true CN110969204A (en) | 2020-04-07 |
| CN110969204B CN110969204B (en) | 2023-05-12 |
Family
ID=70032062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911199840.4A Active CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969204B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111507431A (en) * | 2020-06-30 | 2020-08-07 | 嘉兴太美医疗科技有限公司 | Medical image classification method, device, system and readable medium |
| CN111833332A (en) * | 2020-07-15 | 2020-10-27 | 中国医学科学院肿瘤医院深圳医院 | Generation method and identification method of energy spectrum CT identification model of bone metastasis tumor and bone island |
| CN112150565A (en) * | 2020-09-25 | 2020-12-29 | 呙明 | Medical image management system |
| CN112257769A (en) * | 2020-10-19 | 2021-01-22 | 中国科学院自动化研究所 | Multilayer nuclear magnetic image classification method and system based on reinforcement learning type brain reading |
| CN112288683A (en) * | 2020-06-30 | 2021-01-29 | 深圳市智影医疗科技有限公司 | Pulmonary tuberculosis judgment device and method based on multi-mode fusion |
| WO2022100022A1 (en) * | 2020-11-16 | 2022-05-19 | 中山大学 | Digital pathological image analysis method and system, and device and storage medium |
| CN114897796A (en) * | 2022-04-22 | 2022-08-12 | 深圳市铱硙医疗科技有限公司 | Method, device, equipment and medium for judging stability of atherosclerotic plaque |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281856A (en) * | 2014-10-14 | 2015-01-14 | 中国科学院深圳先进技术研究院 | Image preprocessing method and system for brain medical image classification |
| WO2015054666A1 (en) * | 2013-10-10 | 2015-04-16 | Board Of Regents, The University Of Texas System | Systems and methods for quantitative analysis of histopathology images using multi-classifier ensemble schemes |
| CN110321943A (en) * | 2019-06-25 | 2019-10-11 | 中国科学院自动化研究所 | CT image classification method, system, device based on semi-supervised deep learning |
-
2019
- 2019-11-29 CN CN201911199840.4A patent/CN110969204B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015054666A1 (en) * | 2013-10-10 | 2015-04-16 | Board Of Regents, The University Of Texas System | Systems and methods for quantitative analysis of histopathology images using multi-classifier ensemble schemes |
| CN104281856A (en) * | 2014-10-14 | 2015-01-14 | 中国科学院深圳先进技术研究院 | Image preprocessing method and system for brain medical image classification |
| CN110321943A (en) * | 2019-06-25 | 2019-10-11 | 中国科学院自动化研究所 | CT image classification method, system, device based on semi-supervised deep learning |
Non-Patent Citations (1)
| Title |
|---|
| ZHU HANCAN 等: "\"Multi-atlas label fusion with random local binary pattern features: Application to hippocampus segmentation\"", 《SCIENTIFIC REPORTS》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111507431A (en) * | 2020-06-30 | 2020-08-07 | 嘉兴太美医疗科技有限公司 | Medical image classification method, device, system and readable medium |
| CN111507431B (en) * | 2020-06-30 | 2020-10-02 | 嘉兴太美医疗科技有限公司 | Medical image classification method, device, system and readable medium |
| CN112288683A (en) * | 2020-06-30 | 2021-01-29 | 深圳市智影医疗科技有限公司 | Pulmonary tuberculosis judgment device and method based on multi-mode fusion |
| CN111833332A (en) * | 2020-07-15 | 2020-10-27 | 中国医学科学院肿瘤医院深圳医院 | Generation method and identification method of energy spectrum CT identification model of bone metastasis tumor and bone island |
| CN112150565A (en) * | 2020-09-25 | 2020-12-29 | 呙明 | Medical image management system |
| CN112257769A (en) * | 2020-10-19 | 2021-01-22 | 中国科学院自动化研究所 | Multilayer nuclear magnetic image classification method and system based on reinforcement learning type brain reading |
| CN112257769B (en) * | 2020-10-19 | 2022-10-21 | 中国科学院自动化研究所 | Multilayer nuclear magnetic image classification method and system based on reinforcement learning type brain reading |
| WO2022100022A1 (en) * | 2020-11-16 | 2022-05-19 | 中山大学 | Digital pathological image analysis method and system, and device and storage medium |
| CN114897796A (en) * | 2022-04-22 | 2022-08-12 | 深圳市铱硙医疗科技有限公司 | Method, device, equipment and medium for judging stability of atherosclerotic plaque |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110969204B (en) | 2023-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110969204A (en) | Sample classification system based on fusion of magnetic resonance image and digital pathology image | |
| CN107680678B (en) | Thyroid ultrasound image nodule diagnosis system based on multi-scale convolution neural network | |
| Lee et al. | Segmentation of overlapping cervical cells in microscopic images with superpixel partitioning and cell-wise contour refinement | |
| CN112184617B (en) | Spine MRI image key point detection method based on deep learning | |
| CN111448582A (en) | System and method for single channel whole cell segmentation | |
| Zhang et al. | Automated semantic segmentation of red blood cells for sickle cell disease | |
| CN108537751B (en) | Thyroid ultrasound image automatic segmentation method based on radial basis function neural network | |
| CN111524137A (en) | Cell identification counting method and device based on image identification and computer equipment | |
| US11538261B2 (en) | Systems and methods for automated cell segmentation and labeling in immunofluorescence microscopy | |
| CN112348059A (en) | Deep learning-based method and system for classifying multiple dyeing pathological images | |
| CN114945941A (en) | Non-tumor segmentation for supporting tumor detection and analysis | |
| CN110310291A (en) | A kind of rice blast hierarchy system and its method | |
| CN114841947A (en) | Method and device for multi-scale feature extraction and prognosis analysis of H & E staining pathological image tumor region | |
| JP2024027078A (en) | Multi-scale whole slide pathological feature fusion extraction method, system, electronic equipment and storage medium | |
| CN110033448B (en) | AI-assisted male baldness Hamilton grading prediction analysis method for AGA clinical image | |
| Cao et al. | Automatic segmentation of pathological glomerular basement membrane in transmission electron microscopy images with random forest stacks | |
| CN113724235B (en) | Semi-automatic Ki67/ER/PR negative and positive cell counting system and method under condition of changing environment under mirror | |
| CN116959712A (en) | Lung adenocarcinoma prognosis method, system, equipment and storage medium based on pathological image | |
| CN114862799B (en) | Full-automatic brain volume segmentation method for FLAIR-MRI sequence | |
| Razavi et al. | An automated and accurate methodology to assess ki-67 labeling index of immunohistochemical staining images of breast cancer tissues | |
| Parvaze et al. | Extraction of multiple cellular objects in HEp-2 images using LS segmentation | |
| Rashid et al. | Unsupervised color image segmentation of red blood cell for thalassemia disease | |
| CN114862883A (en) | Target edge extraction method, image segmentation method and system | |
| Guatemala-Sanchez et al. | Nuclei segmentation on histopathology images of breast carcinoma | |
| CN110458042B (en) | Method for detecting number of probes in fluorescent CTC |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |