CN110969204B - Sample classification system based on fusion of magnetic resonance image and digital pathology image - Google Patents
Sample classification system based on fusion of magnetic resonance image and digital pathology image Download PDFInfo
- Publication number
- CN110969204B CN110969204B CN201911199840.4A CN201911199840A CN110969204B CN 110969204 B CN110969204 B CN 110969204B CN 201911199840 A CN201911199840 A CN 201911199840A CN 110969204 B CN110969204 B CN 110969204B
- Authority
- CN
- China
- Prior art keywords
- image
- region
- interest
- module
- magnetic resonance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007170 pathology Effects 0.000 title claims abstract description 33
- 230000004927 fusion Effects 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 98
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000012216 screening Methods 0.000 claims abstract description 23
- 230000001575 pathological effect Effects 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000013145 classification model Methods 0.000 claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims description 10
- 238000000926 separation method Methods 0.000 claims description 10
- 230000001086 cytosolic effect Effects 0.000 claims description 7
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 6
- 238000000528 statistical test Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000010186 staining Methods 0.000 claims description 3
- 238000010998 test method Methods 0.000 claims description 3
- 238000007473 univariate analysis Methods 0.000 claims description 3
- 238000004043 dyeing Methods 0.000 claims description 2
- 238000012552 review Methods 0.000 claims description 2
- 238000011425 standardization method Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 abstract description 7
- 230000007547 defect Effects 0.000 abstract description 3
- 206010028980 Neoplasm Diseases 0.000 description 7
- 210000004027 cell Anatomy 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000003855 cell nucleus Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000035876 healing Effects 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 210000004940 nucleus Anatomy 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000001225 therapeutic effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
- 238000002759 z-score normalization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention belongs to the technical field of medical image processing, in particular relates to a sample classification system based on fusion of a magnetic resonance image and a digital pathology image, and aims to solve the problem of lower sample classification precision caused by limitation of single sample classification in the existing imaging or pathology. The system comprises an image acquisition module, a first image acquisition module and a second image acquisition module, wherein the image acquisition module is configured to acquire a first image and a second image; the preprocessing module is configured to preprocess the first image and the second image; the sketching module is configured to acquire the interested areas of the preprocessed images through a sketching method and further process the interested areas; the feature extraction module is configured to extract features of the region of interest of each processed image respectively; the screening and sorting module is configured to screen and sort the extracted features; and the classification output module is configured to obtain classification results through the classification model by using the characteristics after screening and sorting. The invention fuses the magnetic resonance image and the digital pathological image, solves the defect of single sample classification, and improves the classification precision.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a sample classification system based on fusion of a magnetic resonance image and a digital pathological image.
Background
In recent years, image fusion classification has become a new technology with important application value in the fields of image understanding and computer vision, especially in the medical engineering field, different medical images provide different medical information for doctors due to different imaging mechanisms and application environments of medical instruments, and the image fusion classification can integrate various valuable information together, so that the image fusion classification becomes an important means for clinical diagnosis and medical research. Wherein the multi-modality magnetic resonance image and the digital pathology image are images that are more commonly used for clinical disease diagnosis.
Macroscopic multi-modal magnetic resonance images can reflect sample information overall, but their accuracy is limited and there is a gap from the true histopathological "gold standard". The digital pathological image can reflect microscopic information such as tissue component change, cell activity, nuclear division degree and the like, but cannot reflect comprehensive overall information. Therefore, the traditional sample classification based on imaging or the sample classification based on pathology is only from the single subject knowledge perspective, and the observation information of different scales is isolated from each other, so that the final classification efficiency and the robustness are affected.
However, both the multi-modality magnetic resonance image and the digital pathology image information contain a large amount of highly dimensional and quantifiable information that is not fully explored, and these highly dimensional feature information can be used to construct accurate sample classification by means of pattern recognition. Therefore, combining the observation information of different scales, namely macroscopic and microscopic information, can effectively help to reveal more comprehensive information, so that more accurate sample classification can be realized.
Disclosure of Invention
In order to solve the problems in the prior art, namely to solve the problem of lower sample classification precision caused by the limitation of single sample classification in the existing imaging or pathology, the invention provides a sample classification system based on the fusion of a magnetic resonance image and a digital pathology image, which comprises an image acquisition module, a preprocessing module, a sketching module, an extraction feature module, a screening and sorting module and a classification output module;
the image acquisition module is configured to acquire a first image and a second image; the first image is a multi-mode magnetic resonance image, and the second image is a digital pathological image;
the preprocessing module is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the sketching module is configured to respectively acquire a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image through a sketching method; performing expansion treatment on the first region of interest to obtain a third region of interest; sequentially performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest to obtain a fourth region of interest;
the feature extraction module is configured to respectively extract the features of multiple preset categories of the third region of interest and the fourth region of interest based on a feature extraction method of preset multiple preset categories, and construct a feature set;
the screening and sorting module is configured to acquire a refined feature set from the feature set through a preset feature screening method and an importance sorting method;
the classification output module is configured to obtain a classification result through a preset classification model based on the refined feature set.
In some preferred embodiments, the preprocessing module performs preprocessing on the first image and the second image to obtain a first preprocessed image and a second preprocessed image, and the method is as follows:
based on the first image and the second image, performing image standardization processing by a Z-score standardization method to obtain a third image and a fourth image;
carrying out gray level distribution optimization treatment on the third image by a window width and level method to obtain a fifth image; performing color correction processing on the fourth image by a Macenko color correction method to obtain a sixth image;
and carrying out noise reduction processing by a super-resolution method based on a depth convolution neural network based on the fifth image and the sixth image to obtain a first preprocessed image and a second preprocessed image.
In some preferred embodiments, the method of "expanding the first region of interest to obtain a third region of interest" in the sketching module is as follows: and performing expansion processing on the first region of interest through a filter with a preset N-by-N size to obtain a third region of interest.
In some preferred embodiments, in the sketching module, "the second region of interest is sequentially rasterized, color channel separated, and thresholded to obtain a fourth region of interest", the method includes:
dividing the second region of interest into a plurality of M x M sized regions;
obtaining dyeing channels of the divided areas by a color channel separation method;
based on each staining channel, a nuclear region, a cytoplasmic region, and a cell region are obtained by a threshold segmentation method, and a region composed of the nuclear region, the cytoplasmic region, and the cell region is taken as a fourth region of interest.
In some preferred embodiments, the second region of interest is a region determined after the second pre-processed image is subjected to boundary review at a magnification of 2L, and a sketched region is obtained by a sketching method at a magnification of L.
In some preferred embodiments, the third region of interest includes extracted features comprising: first order statistics, morphology, texture, wavelet transform.
In some preferred embodiments, the fourth region of interest includes extracted features comprising: first-order statistical features, morphological features and features constructed by means of mean, variance, median, decimal, quartile values of the morphological features.
In some preferred embodiments, the classification output module "acquires a refined feature set through a preset feature screening method and an importance ranking method", where the method is: and screening all the characteristics in the characteristic set by a univariate analysis method and a collinearity test method, and sorting the importance of the characteristics by a statistical test method and a machine learning method after screening.
In some preferred embodiments, the predetermined classification model is trained by a machine learning method.
The invention has the beneficial effects that:
the invention fuses the magnetic resonance image and the digital pathological image, solves the defect of single sample classification, and improves the classification precision. The standard image is obtained by preprocessing the multi-mode magnetic resonance image and the digital pathology image. The method comprises the steps of obtaining an interested region through a sketching method based on the preprocessed image, and further processing the interested region of the multi-mode magnetic resonance image and the digital pathological image respectively to obtain a precise interested region, so that the accuracy and the robustness of subsequent feature extraction are ensured. Based on the extracted interested region, acquiring and fusing the features of images with different scales, and analyzing and statistically screening out the features which are redundant and do not meet the statistical test requirement, so as to acquire the refined features. And obtaining a classification result through a classification model trained based on a machine learning method according to the refined characteristics. The method and the device realize more comprehensive description of the images, solve the defect of single sample classification, and further improve the classification precision and stability.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings.
FIG. 1 is a schematic diagram of a sample classification system based on fusion of magnetic resonance images with digital pathology images in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram showing the effect of the method of the present invention compared with other methods according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a detailed process of a sample classification system based on fusion of magnetic resonance images with digital pathology images according to one embodiment of the present invention;
fig. 4 is a flow chart of a sample classification method based on fusion of a magnetic resonance image and a digital pathology image according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
The invention discloses a sample classification system based on fusion of a magnetic resonance image and a digital pathological image, which is shown in fig. 1 and comprises an image acquisition module 100, a preprocessing module 200, a sketching module 300, a feature extraction module 400, a screening and sorting module 500 and a classification output module 600;
the image acquisition module 100 is configured to acquire a first image and a second image; the first image is a multi-mode magnetic resonance image, and the second image is a digital pathological image;
the preprocessing module 200 is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the sketching module 300 is configured to obtain a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image respectively by a sketching method; performing expansion treatment on the first region of interest to obtain a third region of interest; sequentially performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest to obtain a fourth region of interest;
the feature extraction module 400 is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on feature extraction methods of preset multiple preset categories, and construct a feature set;
the screening and sorting module 500 is configured to obtain a refined feature set from the feature set through a preset feature screening method and an importance sorting method;
the classification output module 600 is configured to obtain a classification result through a preset classification model based on the refined feature set.
In order to more clearly describe the sample classification system based on the fusion of the magnetic resonance image and the digital pathological image of the present invention, each module in one embodiment of the system of the present invention is described in detail below with reference to the accompanying drawings.
An acquisition image module 100 configured to acquire a first image, a second image; the first image is a multi-modal magnetic resonance image and the second image is a digital pathology image.
In this embodiment, a multi-mode magnetic resonance image and a digital pathology image of a corresponding portion of a patient are acquired first, and are used as a first image and a second image. The invention is preferably a long tumor site of a patient suspected of being treated, and in other embodiments other sites of the patient corresponding to other diseases may be selected.
The preprocessing module 200 is configured to perform preprocessing on the first image and the second image to obtain a first preprocessed image and a second preprocessed image.
In this embodiment, the input image pair is processed by four image preprocessing methods, such as image normalization, color normalization, image gray scale distribution optimization, and image super-resolution enhancement, as shown in fig. 3: the specific treatment is as follows:
and carrying out image normalization processing by a Z-score normalization method based on the first image and the second image to obtain a third image and a fourth image.
And carrying out gray level distribution optimization treatment on the third image by a window width and level method, fully utilizing the display effective value range between 0 and 255, and reducing the loss caused by value range compression as much as possible to obtain a fifth image. And performing color correction processing on the fourth image by a Macenko color correction method, and correcting the color space of the digital pathological image to obtain a sixth image.
And carrying out noise reduction processing by a super-resolution method based on a depth convolution neural network based on the fifth image and the sixth image, weakening noise influence, and improving the signal-to-noise ratio of the magnetic resonance image and the digital pathological image data to obtain a first preprocessing image and a second preprocessing image.
The other parts of fig. 3 are detailed below.
The sketching module 300 is configured to obtain a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image respectively through a sketching method; performing expansion treatment on the first region of interest to obtain a third region of interest; and sequentially performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest to obtain a fourth region of interest.
In this embodiment, the region of interest of each image is extracted by a sketching method based on the first preprocessed image and the second preprocessed image. I.e. establish the tumor boundary with the region of interest. The sketching method comprises automatic and semi-automatic sketching. Manual sketching may also be used during the training phase.
And for the first preprocessing image, obtaining an interested region by a sketching method, namely dividing a tumor region, taking the tumor region as the first interested region, and adopting a filter with a preset N-by-N size to perform image expansion operation on the interested region so as to realize outward expansion of a boundary and obtain an edge interested region, and taking the edge interested region as a third interested region. In the present invention, the size of the filter is preferably 3*3.
And for the second preprocessed image, obtaining a sketched area by a sketching method under the L multiplying power, and checking the boundary of the sketched area under the 2L multiplying power to determine the area as a second interested area. In the invention, the tumor area is preferably sketched by a sketching method under the 10 multiplying power, and the sketching boundary rechecking is assisted by 20 multiplying power, and in other embodiments, the multiplying power can be selected according to the actual requirement. Dividing the second region of interest into a plurality of M-sized regions, namely performing rasterization operation to ensure that the plurality of regions are not overlapped, and performing color channel separation on the plurality of regions. Because the digital pathological image is acquired based on hematoxylin-eosin staining (HE), the digital pathological image is separated into an H channel and an E channel after channel separation. In the separated H channel, a foreground object, namely a cell nucleus, is obtained through a threshold segmentation method, and in the E channel, the foreground object obtained through the threshold segmentation method is a cytoplasmic area. The intersection of the nuclear and cytoplasmic prospects constitutes the cell area. The region of interest of the pathology image is composed of three parts, namely a nucleus region, a cytoplasmic region and a cell region, namely a corresponding fourth region of interest. In the present invention, the size of the region is preferably 512×512. In addition, in fig. 3, in consideration of the layout of the drawing, the region-of-interest divided portion of the second preprocessed image is put into the feature extraction without substantial influence.
The feature extraction module 400 is configured to extract features of multiple preset categories of the third region of interest and the fourth region of interest respectively based on feature extraction methods of preset multiple preset categories, and construct a feature set.
In the present embodiment, different features are extracted for the third region of interest and the fourth region of interest.
Four features, which may also be referred to as imaging feature extraction, are extracted for the third region of interest, including first-order statistical features, morphological features, texture features, and wavelet transform features.
For the fourth region of interest, the extracted features may also be referred to as pathological features, including first-order statistical features, morphological features, and pathological image features constructed by means of mean, variance, median, decimal, and quartile values.
Based on the imaging features and the pathological features, namely combining the multi-mode magnetic resonance image feature set and the digital pathological image feature set, constructing a feature pool of the imaging features and the pathological features, and obtaining the combined feature set. In addition, in this embodiment, a plurality of feature extraction methods of preset categories are preset as methods corresponding to features extracted from the third region of interest and the fourth region of interest. For example, the first-order statistical feature corresponds to a first-order statistical feature extraction method, and the other similar matters are not described in detail herein.
The screening and sorting module 500 is configured to obtain a refined feature set from the feature set through a preset feature screening method and an importance sorting method.
In the embodiment, the univariate analysis method and the collinearity test method are adopted to eliminate the characteristics which do not accord with the statistical test hypothesis and redundancy, the information strategy fusion is realized, the characteristic screening is carried out, the characteristic importance ranking is obtained by adopting a mode of combining the statistical test method and the machine learning method, and the refined characteristic set is obtained after the ranking is completed.
The classification output module 600 is configured to obtain a classification result through a preset classification model based on the refined feature set.
In this embodiment, the classification model pre-trained by the machine learning method obtains an accurate classification result, that is, determines the benign and malignant, grading and grading types of the tumor, and if the classification result includes a therapeutic effect and a survival time after healing, if the classification result includes a multi-mode magnetic resonance image and a digital pathological image of the tumor portion to be treated.
Wherein the filter ranking module 500-the classification output module 600 corresponds to the portion of information fusion and modeling in fig. 4.
In addition, a refined feature extraction system based on the fusion of the magnetic resonance image and the digital pathology image can be constructed based on the image acquisition module 100-the screening and sorting module 500, so that the refined features after the fusion of the magnetic resonance image and the digital pathology image are extracted, and the problem that the classification accuracy is lower due to the fact that the extracted features of the classification model of the sample are inaccurate and comprehensive can be solved.
It should be noted that, in the sample classification system based on fusion of a magnetic resonance image and a digital pathology image provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the modules or steps in the foregoing embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
A sample classification method based on fusion of a magnetic resonance image and a digital pathology image according to a second embodiment of the present invention, as shown in fig. 4, includes the following steps:
step S100, a first image and a second image are acquired; the first image is a multi-mode magnetic resonance image, and the second image is a digital pathological image;
step S200, preprocessing the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
step S300, respectively acquiring a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image by a sketching method; performing expansion treatment on the first region of interest to obtain a third region of interest; sequentially performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest to obtain a fourth region of interest;
step S400, respectively extracting the characteristics of multiple preset categories of the third region of interest and the fourth region of interest based on a characteristic extraction method of preset multiple preset categories, and constructing a characteristic set;
step S500, obtaining a refined feature set through a preset feature screening method and an importance ranking method for the feature set;
and step S600, based on the refined feature set, obtaining a classification result through a preset classification model.
For training and testing the method of the invention, 981 real datasets with both multi-modal magnetic resonance images and digital pathology images were used in the embodiment for testing, of which 303 were used for training and 678 were used for testing the classification effect of the invention.
The present invention is used to compare existing classification methods (pure image method and pure pathology method), obtain the subject operating characteristic (ROC) curve and its Area Under Curve (AUC) of the classification method, and take the ROC curve and AUC as the measurement of classifier performance.
The classification ROC curves for both methods on the real experimental dataset are shown in fig. 2, respectively, where true positive rate refers to the percentage that was actually positive and correctly judged positive using the method, and false positive rate refers to the percentage that was actually negative and incorrectly judged positive according to the method. As shown in fig. 2, the ROC curve of the present method is higher in most of the threshold ranges than the other two methods, AUC values versus cases: the AUC value of the method was 0.868, the AUC value of the pure image method was 0.752, the AUC of the pure pathology method was 0.743, and the AUC of the method for randomly classifying samples was 0.5. The area under the curve (AUC) measures the overall classification performance, the greater the AUC value, the better the overall performance of the classification method. Therefore, the method has better effect than the other two methods.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working processes and related descriptions of the above-described method may refer to corresponding processes in the foregoing system embodiments, which are not described herein again.
A storage device according to a third embodiment of the present invention stores therein a plurality of programs adapted to be loaded by a processor and to implement the above-described sample classification method based on fusion of a magnetic resonance image with a digital pathology image.
A processing device according to a fourth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute each program; a storage device adapted to store a plurality of programs; the program is adapted to be loaded and executed by a processor to implement the above-described sample classification method based on fusion of magnetic resonance images with digital pathology images.
It can be clearly understood by those skilled in the art that the storage device, the specific working process of the processing device and the related description described above are not described conveniently and simply, and reference may be made to the corresponding process in the foregoing method example, which is not described herein.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the program(s) corresponding to the software modules, method steps, may be embodied in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different methods for each set application, but such implementation is not to be considered as beyond the scope of the present invention.
The terms "first," "second," and the like, are used for distinguishing between similar objects and not for describing a sequential or chronological order of setting.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
Claims (9)
1. The sample classification system based on the fusion of the magnetic resonance image and the digital pathological image is characterized by comprising an image acquisition module, a preprocessing module, a sketching module, a feature extraction module, a screening and sorting module and a classification output module;
the image acquisition module is configured to acquire a first image and a second image; the first image is a multi-mode magnetic resonance image, and the second image is a digital pathological image;
the preprocessing module is configured to preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image;
the sketching module is configured to respectively acquire a first region of interest and a second region of interest corresponding to the first preprocessed image and the second preprocessed image through a sketching method; performing expansion treatment on the first region of interest to obtain a third region of interest; sequentially performing rasterization processing, color channel separation processing and threshold segmentation processing on the second region of interest to obtain a fourth region of interest;
the feature extraction module is configured to respectively extract the features of multiple preset categories of the third region of interest and the fourth region of interest based on a feature extraction method of preset multiple preset categories, and construct a feature set;
the screening and sorting module is configured to acquire a refined feature set from the feature set through a preset feature screening method and an importance sorting method;
the classification output module is configured to obtain a classification result through a preset classification model based on the refined feature set.
2. The sample classification system based on fusion of magnetic resonance image and digital pathology image according to claim 1, wherein the preprocessing module performs preprocessing on the first image and the second image to obtain a first preprocessed image and a second preprocessed image, and the method is as follows:
based on the first image and the second image, performing image standardization processing by a Z-score standardization method to obtain a third image and a fourth image;
carrying out gray level distribution optimization treatment on the third image by a window width and level method to obtain a fifth image; performing color correction processing on the fourth image by a Macenko color correction method to obtain a sixth image;
and carrying out noise reduction processing by a super-resolution method based on a depth convolution neural network based on the fifth image and the sixth image to obtain a first preprocessed image and a second preprocessed image.
3. The sample classification system based on fusion of magnetic resonance image and digital pathology image according to claim 1, wherein the method of "expanding the first region of interest to obtain a third region of interest" in the sketching module is as follows: and performing expansion processing on the first region of interest through a filter with a preset N-by-N size to obtain a third region of interest.
4. The sample classification system based on fusion of a magnetic resonance image and a digital pathology image according to claim 1, wherein the method of "sequentially performing rasterization processing, color channel separation processing, and threshold segmentation processing on the second region of interest to obtain a fourth region of interest" in the sketching module is as follows:
dividing the second region of interest into a plurality of M x M sized regions;
obtaining dyeing channels of the divided areas by a color channel separation method;
based on each staining channel, a nuclear region, a cytoplasmic region, and a cell region are obtained by a threshold segmentation method, and a region composed of the nuclear region, the cytoplasmic region, and the cell region is taken as a fourth region of interest.
5. The sample classification system based on fusion of a magnetic resonance image and a digital pathology image according to claim 4, wherein the second region of interest is a region determined after the second pre-processed image is subjected to boundary review at a magnification of 2L, and a sketched region is obtained by a sketching method at a magnification of L.
6. The magnetic resonance image and digital pathology image fusion-based sample classification system according to claim 1, wherein the third region of interest comprises extracted features including: first order statistics, morphology, texture, wavelet transform.
7. The system for classifying samples based on fusion of magnetic resonance images with digital pathology images according to claim 1, characterized in that said fourth region of interest comprises extracted features comprising: first-order statistical features, morphological features and features constructed by means of mean, variance, median, decimal, quartile values of the morphological features.
8. The sample classification system based on fusion of magnetic resonance images and digital pathology images according to claim 1, wherein the classification output module obtains a refined feature set by a preset feature screening method and an importance ranking method, and the method is as follows: and screening all the characteristics in the characteristic set by a univariate analysis method and a collinearity test method, and sorting the importance of the characteristics by a statistical test method and a machine learning method after screening.
9. The sample classification system based on fusion of magnetic resonance images and digital pathology images according to claim 1, wherein the preset classification model is trained by a machine learning method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199840.4A CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199840.4A CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110969204A CN110969204A (en) | 2020-04-07 |
| CN110969204B true CN110969204B (en) | 2023-05-12 |
Family
ID=70032062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911199840.4A Active CN110969204B (en) | 2019-11-29 | 2019-11-29 | Sample classification system based on fusion of magnetic resonance image and digital pathology image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969204B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113643222A (en) * | 2020-04-23 | 2021-11-12 | 上海联影智能医疗科技有限公司 | Multi-modal image analysis method, computer device and storage medium |
| CN112288683A (en) * | 2020-06-30 | 2021-01-29 | 深圳市智影医疗科技有限公司 | Pulmonary tuberculosis judgment device and method based on multi-mode fusion |
| CN111507431B (en) * | 2020-06-30 | 2020-10-02 | 嘉兴太美医疗科技有限公司 | Medical image classification method, device, system and readable medium |
| CN111833332A (en) * | 2020-07-15 | 2020-10-27 | 中国医学科学院肿瘤医院深圳医院 | Generation method and identification method of energy spectrum CT identification model of bone metastasis tumor and bone island |
| CN112150565A (en) * | 2020-09-25 | 2020-12-29 | 呙明 | Medical image management system |
| CN112257769B (en) * | 2020-10-19 | 2022-10-21 | 中国科学院自动化研究所 | Multilayer nuclear magnetic image classification method and system based on reinforcement learning type brain reading |
| CN112419253B (en) * | 2020-11-16 | 2024-04-19 | 中山大学 | Digital pathology image analysis method, system, equipment and storage medium |
| CN114897796B (en) * | 2022-04-22 | 2023-06-30 | 深圳市铱硙医疗科技有限公司 | Method, device, equipment and medium for judging stability of atherosclerosis plaque |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281856A (en) * | 2014-10-14 | 2015-01-14 | 中国科学院深圳先进技术研究院 | Image preprocessing method and system for brain medical image classification |
| WO2015054666A1 (en) * | 2013-10-10 | 2015-04-16 | Board Of Regents, The University Of Texas System | Systems and methods for quantitative analysis of histopathology images using multi-classifier ensemble schemes |
| CN110321943A (en) * | 2019-06-25 | 2019-10-11 | 中国科学院自动化研究所 | CT image classification method, system, device based on semi-supervised deep learning |
-
2019
- 2019-11-29 CN CN201911199840.4A patent/CN110969204B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015054666A1 (en) * | 2013-10-10 | 2015-04-16 | Board Of Regents, The University Of Texas System | Systems and methods for quantitative analysis of histopathology images using multi-classifier ensemble schemes |
| CN104281856A (en) * | 2014-10-14 | 2015-01-14 | 中国科学院深圳先进技术研究院 | Image preprocessing method and system for brain medical image classification |
| CN110321943A (en) * | 2019-06-25 | 2019-10-11 | 中国科学院自动化研究所 | CT image classification method, system, device based on semi-supervised deep learning |
Non-Patent Citations (1)
| Title |
|---|
| Zhu Hancan 等."Multi-atlas label fusion with random local binary pattern features: Application to hippocampus segmentation".《Scientific reports》.2019,全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110969204A (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110969204B (en) | Sample classification system based on fusion of magnetic resonance image and digital pathology image | |
| CN107680678B (en) | Thyroid ultrasound image nodule diagnosis system based on multi-scale convolution neural network | |
| CN109886179B (en) | Image segmentation method and system of cervical cell smear based on Mask-RCNN | |
| CN111127466B (en) | Medical image detection method, device, equipment and storage medium | |
| CN109272492B (en) | Method and system for processing cytopathology smear | |
| KR102108050B1 (en) | Method for classifying breast cancer histology images through incremental boosting convolution networks and apparatus thereof | |
| CN108464840B (en) | Automatic detection method and system for breast lumps | |
| CN110376198B (en) | Cervical liquid-based cell slice quality detection system | |
| CN111986150B (en) | The method comprises the following steps of: digital number pathological image Interactive annotation refining method | |
| US7027627B2 (en) | Medical decision support system and method | |
| CN111178449A (en) | Liver cancer image classification method and device combining computer vision characteristics and imaging omics characteristics | |
| CN112164082A (en) | Method for segmenting multi-modal MR brain image based on 3D convolutional neural network | |
| CN109919254B (en) | Breast density classification method, system, readable storage medium and computer device | |
| CN110189293A (en) | Cell image processing method, device, storage medium and computer equipment | |
| Ikromjanov et al. | Whole slide image analysis and detection of prostate cancer using vision transformers | |
| CN112348059A (en) | Deep learning-based method and system for classifying multiple dyeing pathological images | |
| Kanwal et al. | Quantifying the effect of color processing on blood and damaged tissue detection in whole slide images | |
| CN112036298A (en) | Cell detection method based on double-segment block convolutional neural network | |
| CN116758336A (en) | Medical image intelligent analysis system based on artificial intelligence | |
| CN113205111B (en) | Identification method and device suitable for liver tumor and electronic equipment | |
| CN113052800B (en) | Alzheimer disease image analysis method and device | |
| CN109840564B (en) | Classification system based on ultrasound contrast image uniformity degree | |
| CN116959712A (en) | Lung adenocarcinoma prognosis method, system, equipment and storage medium based on pathological image | |
| CN114862799B (en) | Full-automatic brain volume segmentation method for FLAIR-MRI sequence | |
| CN109800820A (en) | A kind of classification method based on ultrasonic contrast image uniform degree |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |