CN114187475A - Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram - Google Patents
Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram Download PDFInfo
- Publication number
- CN114187475A CN114187475A CN202111511828.XA CN202111511828A CN114187475A CN 114187475 A CN114187475 A CN 114187475A CN 202111511828 A CN202111511828 A CN 202111511828A CN 114187475 A CN114187475 A CN 114187475A
- Authority
- CN
- China
- Prior art keywords
- cnn
- thermodynamic
- explaining
- ssp
- feature map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Animal Behavior & Ethology (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Physiology (AREA)
- Computational Linguistics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Neurology (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Fuzzy Systems (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- High Energy & Nuclear Physics (AREA)
- Neurosurgery (AREA)
- Radiology & Medical Imaging (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
A method for explaining a CNN classification result of multi-test complex fMRI data based on thermodynamic diagrams belongs to the field of biomedical signal processing. Inputting a single tested slice sample into a trained CNN model, and extracting a feature map output by the last pooling layer; outputting a gradient relative to the feature map through a back propagation calculation model; sending the feature map subjected to gradient weighting into a linear rectifying unit; and obtaining a thermodynamic diagram through upsampling, visualizing the decision basis of the model, and explaining the single tested classification result. And averaging the thermodynamic diagrams of the same type of test to obtain a group thermodynamic diagram for a plurality of tests, and explaining the classification result of the plurality of tests. The present invention calculates the thermodynamic diagrams of schizophrenic patients and healthy controls tested under two CNN models, respectively. SSP maps give more complete DMN regions and greater inter-group variation than SSM maps, improving the confidence of CNN classification.
Description
Technical Field
The invention belongs to the field of biomedical signal processing, and relates to a method for explaining classification results of multi-test complex number fMRI (functional magnetic resonance imaging) data CNN (volumetric neural network) based on thermodynamic diagrams.
Background
In recent years, deep learning has exhibited unique advantages in image classification. The CNN is widely applied to medical image classification by fully mining and utilizing the correlation between adjacent pixels of an input image. However, the CNN model has a "black box" attribute, and one cannot determine what factors the CNN prediction results are affected by, or what factors dominate the classification results. Therefore, interpretable studies of CNN models have become a current hotspot. Providing classification evidence for CNN is particularly important for medical image classification and computer-aided diagnosis.
fMRI data is often used for CNN classification of neuropsychiatric brain diseases such as schizophrenia due to its advantages of high safety, non-invasiveness, and spatial resolution up to a millimeter level. Considering that the signal-to-noise ratio of the fMRI observation data is extremely low and the noise is severe, most CNN inputs adopt the fMRI data after noise reduction processing or adopt space-time features extracted from the fMRI observation data. As a data-driven blind source separation algorithm, Independent Component Analysis (ICA) can extract a denoised spatial activation map of the brain from complex fMRI data and reflect changes in spatial activation due to changes in brain function, and thus, the ICA spatial activation map is an effective CNN input. For example, in the existing ICA-CNN classification framework (patent application No. 201910350137.2), ICA is first used to extract the Magnitude information of the complex Spatial activation map of the component of interest, i.e. the Spatial Source Magnitude (SSM) map, from the complex fMRI observation data, create a SSM map sample set, and then send into 2D-CNN to classify the schizophrenic patients from healthy control subjects. However, it has been shown that Phase information of a complex Spatial activation map, i.e., a Spatial Source Phase (SSP) map, is hundreds times more sensitive to Spatial differences between a schizophrenic patient group and a healthy control group than the SSM map. Therefore, SSP profiles have greater potential in the aided diagnosis of neuropsychiatric brain diseases. However, in the internal transmission and decision process of CNN, the SSP graph has unique advantages, and there is no interpretable method for decision according to whether the classification result can be interpreted or not.
Disclosure of Invention
Based on a trained CNN model and slice samples of a single tested three-dimensional SSP graph along the z-axis, the invention firstly extracts a feature graph output by the last pooling layer of the CNN, then outputs a gradient relative to the feature graph through a back propagation calculation model, and then sends the feature graph after gradient weighting into a Linear rectification Unit (RecU) so as to keep the feature which plays a forward role in the calculation of output categories, and finally obtains a thermodynamic diagram through up-sampling, thereby visualizing the decision basis of the model and explaining the classification result of the single tested. And averaging the thermodynamic diagrams of the same type of test to obtain a group thermodynamic diagram for a plurality of tests, and explaining the classification result of the plurality of tests.
The technical scheme adopted by the invention is as follows (see figure 1):
the first step is as follows: the two-dimensional SSP slice sample of test j is input into a trained 2D-CNN model. SSP slice representation asJ is the total number of samples, X × Y is the SSP slice size, X is 1, …, X, Y is 1, …, Y. The trained 2D-CNN model is denoted h (-).
The second step is that: and extracting a characteristic diagram of the 2D-CNN. Extracting the feature map output by the last pooling layer from the 2D-CNN model, and expressing the feature map asK is the number of channels, X 'X Y' is the size of the signature, X ', Y' are determined by the convolution kernel size of the corresponding layer, J is 1, …, J.
The third step: and outputting the prediction result of the SSP slice sample. Calculating the 2D-CNN model(j)(x, y) the probability of healthy control subjects and neuropsychiatric brain disease such as schizophrenia, i.e.:
The fourth step: a gradient-based weighting value is calculated. Calculating the gradient of each channel feature map output by the model output category corresponding to the last pooling layer through back propagation, and then performing global average pooling as follows:
The fifth step: a single trial thermodynamic diagram is generated. Feature maps of all channels outputting the last pooling layerUsing a correspondingWeighted and summed, and fed into the ReLU function to retain the characteristics that positively affect the computation of the output classes as follows:
to pairInterpolation was performed to the same sample size X Y as the input SSP slice to obtain a single-subject thermodynamic diagram, denoted by
And a sixth step: a multi-test-group thermodynamic diagram is generated. Averaging the thermodynamic diagrams of the same tested class with correct classification to obtain a group thermodynamic diagram, wherein the group thermodynamic diagram comprises the following steps:
wherein J' is the total number of homogeneous subjects.
The invention has the beneficial effects that: and visualizing the classification basis of the CNN model through thermodynamic diagrams, and performing post explanation on classification results. For example, after extracting interesting components of a Default Network (Default Mode Network, DMN) from 82 tested complex fMRI data, respectively establishing a sample set of an SSP map and an SSM map, training two 2D-CNN models with the same structure, respectively calculating thermodynamic diagrams of a schizophrenia patient and a health control subject under the two CNN models by using the thermodynamic diagram visualization method provided by the present invention, and performing single-test and multi-test analysis, wherein the results are respectively shown in fig. 2 and fig. 3. It was observed that the SSP plots depict more complete DMN regions than the SSM plots, and show the opposite trend, i.e., the healthy control tested DMN regions are large, while the schizophrenia patients DMN regions are small. For the SSM plot, healthy controls were tested without locating the DMN region, with schizophrenic patients located predominantly in the posterior cingulate cortical region and a small portion in the apical and inferior leaflet region. Therefore, the thermodynamic diagrams generated by SSP graphs have greater inter-group differences compared to SSM graphs. This explains the reason that the SSP graph has better classification result (the accuracy of the slice level is improved by 9.72%, and the accuracy of the tested level is improved by 15%) than the CNN classification result of the SSM graph, and improves the reliability of the CNN model classification.
Drawings
FIG. 1 is a schematic diagram of the present invention for generating a single trial thermodynamic diagram.
Figure 2 is a thermal map of healthy control subjects compared to single subjects with schizophrenia.
Figure 3 is a thermal map of healthy control subjects compared to multiple test groups of schizophrenic patients.
Detailed Description
The following describes embodiments of the present invention in detail with reference to the technical solutions.
A current J-82 complex resting fMRI data of subjects, including 42 schizophrenic patients and 40 healthy control subjects. Each test contained 146 whole brain scans, each of which had total brain data of voxels X × Y × Z53 × 63 × 46 153594, with 62336 voxels in the brain.
First, the SSP map of the interesting component is obtained. The model order N is set to 20, and the complex fMRI data of each test subject is separated by using a complex ebm (control Bound minimization) algorithm. Reference template according to DMN(selected from the analysis results given in the references "S.M. Smith, P.T. Fox et al, Correstance of the branched's functional architecture reduction activation and rest, Proceedings of the National Academy of Sciences of the United States of America, vol.106, No.31, pp.13040-13045,2009.) the complex DMN component of each test was extracted as the component of interest. Wherein the space activation mapTime courseBased onThe real part energy maximization calculates the rotation angle of the phase correction as follows:
in the formula, "Re {. The } represents taking a real part,then s is(j)The rotation correction in the complex plane is as follows:
where "corr (-) calculates the correlation coefficient of two vectors," -1 "indicates 180 degrees of rotation in the complex plane; to pairConstructing a binary maskThe following were used:
to represents (j)Phase values of the medium voxel v, v 1, …, 62336; using masking b(j)To pairs (j)Phase denoising is carried out to obtain:
in the formulaRepresenting a Hadamard product.The DMN space activation diagram after noise elimination is obtained; the above steps are repeated 10 times, and the best run results, expressed as "best run results", are taken from 10 DMN noise-canceling spatial activation maps using the method in "L.D. Kuang, Q.H. Lin et al, Model order effects on ICA of suppressing-state complex-valued fMRI data: application to schizohrenia, Journal of neural Methods vol.304, pp.24-38,2018GetThe phase of (d) gives a DMNSSP map.
Then, an SSP sample set is established. After filling the SSP map of DMN with zero-filling extrabrain voxels, it was expanded along the z-axis into a stack of 46 two-dimensional slices of size 53X 63, according to a reference templateAfter the 25 slices (z 12, …,36) with large information amount are retained, the absolute value of each slice is taken and normalized to obtain the sample set of the tested jIs composed ofTag (definition) ofIndicating that the section was from a healthy control subject,indicating from a schizophrenic patient); the model order N is from 30 to 140, one value is taken at intervals of 10, the steps are repeated for 12 times, and 82 tested augmentation sample sets are obtained
Next, a CNN model is trained. Randomly disorganizing the sequence of the test, and testing the test according to the following steps of 3: 1: 1, dividing a sample set into a training set, a verification set and a test set; A2D-CNN with a structure similar to that in patent application No. 201910350137.2 is built, and the 2D-CNN comprises two convolution layers, two pooling layers and two full-connection layers, except that the activation function after the two convolution layers is set to be tanh. The model was trained for 50 rounds with a batch size of 64. The model obtained from each round of training was evaluated with accuracy on the validation set. In 50 rounds, the model with the highest accuracy on the validation set was determined to be the final CNN model obtained from training. And (3) sending the test set into a CNN final model for prediction, and calculating the classification accuracy of the slice to be 77.85% and the classification accuracy of the tested slice to be 98%.
Based on the single tested SSP image slice sample and the trained CNN model, the steps of generating the thermodynamic diagram for explaining the CNN model by adopting the invention are as follows:
the first step is as follows: selecting 30 subjects with less noise (including 15 healthy control subjects and 15 schizophrenic patients) to generate a thermodynamic diagram, and explaining the classification results of single subjects and multiple subjects.
The second step is that: two-dimensional SSP slicing of test j (j 1, …,30)Inputting the trained 2D-CNN model h (·).
the fourth step: calculation of samples I according to equation (1)(j)CNN prediction probability p of (x, y)(j)Wherein the prediction class corresponds to a probability of
The fifth step: calculating the weight corresponding to the 16 channel feature maps of the last pooling layer by back propagation according to the formula (2)
And a sixth step: will be provided withAndis substituted by formula (3) to obtainThen carrying out bilinear interpolation to obtain a single tested thermodynamic diagram,
the seventh step: the correctly classified thermodynamic diagrams of the same test subjects were averaged according to equation (4) to obtain thermodynamic diagrams of healthy control group and schizophrenic patient group, which are shown as
Eighth step: obtaining an SSM map of the interested component. Repeating the above steps except that Obtaining an SSM image by using the amplitude, then establishing an SSM sample set in the same way as the SSP image sample set, training a 2D-CNN model with the same structure, and calculating the slice classification accuracy of 68.13% and the tested classification accuracy of 83%.
The ninth step: repeating the first to seventh steps, generating a single-subject, multi-subject thermodynamic diagram of the SSM diagram, and comparing the results with the results of the SSP diagram, the results are shown in fig. 2 and fig. 3.
Claims (1)
1. A method for explaining multi-test complex fMRI data CNN classification results based on thermodynamic diagrams comprises the steps of firstly inputting single test slice samples into a trained CNN model, extracting a feature diagram output by the last pooling layer, outputting a gradient relative to the feature diagram through a back propagation calculation model for weighting the feature diagram, processing the feature diagram through a linear rectification unit, and then performing up-sampling to obtain the thermodynamic diagrams for explaining the single test classification results; averaging the thermodynamic diagrams of the same type of testees to obtain a group thermodynamic diagram, and explaining a classification result of the testees; the method is characterized by comprising the following steps:
the first step is as follows: inputting the two-dimensional SSP slice sample of the tested j into a trained 2D-CNN model, and expressing the SSP slice asJ is the total number of subjects, X × Y is the SSP slice size, X ═ 1, …, X, Y ═ 1, …, Y, and the trained 2D-CNN model is denoted h (·);
the second step is that: extracting a feature map of the 2D-CNN, extracting a feature map output by the last pooling layer from the 2D-CNN model, and expressing the feature map asK is the number of channels, X 'X Y' is the size of the feature map, X ', Y' are determined by the convolution kernel size of the corresponding layer, J is 1, …, J;
the third step: outputting the prediction result of SSP slice sample, calculating 2D-CNN model I(j)(x, y) probability of being a healthy control subject and a neuropsychiatric brain disease patient, namely:
The fourth step: calculating a weighted value based on the gradient, calculating the gradient of each channel feature map output by the model output category corresponding to the last pooling layer through back propagation, and then performing global average pooling as follows:
The fifth step: generating a single-subject thermodynamic diagram, and outputting the feature maps of all channels of the last pooling layerUsing a correspondingWeighted and summed, and fed into the ReLU function to retain the characteristics that positively affect the computation of the output classes as follows:
to pairInterpolation was performed to the same sample size X Y as the input SSP slice to obtain a single-subject thermodynamic diagram, denoted by
And a sixth step: generating a plurality of tested group thermodynamic diagrams, and averaging the thermodynamic diagrams of the same tested group with correct classification to obtain the group thermodynamic diagrams, wherein the group thermodynamic diagrams are as follows:
wherein J' is the total number of homogeneous subjects.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111511828.XA CN114187475A (en) | 2021-12-06 | 2021-12-06 | Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111511828.XA CN114187475A (en) | 2021-12-06 | 2021-12-06 | Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114187475A true CN114187475A (en) | 2022-03-15 |
Family
ID=80543232
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111511828.XA Pending CN114187475A (en) | 2021-12-06 | 2021-12-06 | Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114187475A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115187821A (en) * | 2022-07-05 | 2022-10-14 | 阿波罗智能技术(北京)有限公司 | Method for verifying correctness before and after model conversion, related device and program product |
-
2021
- 2021-12-06 CN CN202111511828.XA patent/CN114187475A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115187821A (en) * | 2022-07-05 | 2022-10-14 | 阿波罗智能技术(北京)有限公司 | Method for verifying correctness before and after model conversion, related device and program product |
CN115187821B (en) * | 2022-07-05 | 2024-03-22 | 阿波罗智能技术(北京)有限公司 | Method, related device and program product for verifying correctness of model before and after conversion |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Zhang et al. | ANC: Attention network for COVID-19 explainable diagnosis based on convolutional block attention module | |
CN113040715B (en) | Human brain function network classification method based on convolutional neural network | |
Wang et al. | Texture analysis method based on fractional Fourier entropy and fitness-scaling adaptive genetic algorithm for detecting left-sided and right-sided sensorineural hearing loss | |
Dey et al. | Healthy and unhealthy rat hippocampus cells classification: A neural based automated system for Alzheimer disease classification | |
Fukushima et al. | MEG source reconstruction based on identification of directed source interactions on whole-brain anatomical networks | |
CN110797123B (en) | Graph convolution neural network evolution method of dynamic brain structure | |
CN110110776B (en) | Method for constructing resting state complex fMRI data ICA-CNN classification framework of patient and healthy person | |
CN109087298B (en) | Alzheimer's disease MRI image classification method | |
CN112085736B (en) | Kidney tumor segmentation method based on mixed-dimension convolution | |
Wang et al. | Optical pressure sensors based plantar image segmenting using an improved fully convolutional network | |
Lee et al. | Is intensity inhomogeneity correction useful for classification of breast cancer in sonograms using deep neural network? | |
Mehta et al. | Propagating uncertainty across cascaded medical imaging tasks for improved deep learning inference | |
CN114693933A (en) | Medical image segmentation device based on generation of confrontation network and multi-scale feature fusion | |
Naval Marimont et al. | Implicit field learning for unsupervised anomaly detection in medical images | |
Wang et al. | Detection of spectral signatures in multispectral MR images for classification | |
CN110782427A (en) | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution | |
CN114187475A (en) | Method for explaining CNN classification result of multi-test complex fMRI data based on thermodynamic diagram | |
Kherchouche et al. | Attention-guided neural network for early dementia detection using MRS data | |
Goutham et al. | Brain tumor classification using Efficientnet-B0 model | |
Rezaei et al. | Brain abnormality detection by deep convolutional neural network | |
CN108596900B (en) | Thyroid-associated ophthalmopathy medical image data processing device and method, computer-readable storage medium and terminal equipment | |
Pomiecko et al. | 3D convolutional neural network segmentation of white matter tract masks from MR diffusion anisotropy maps | |
Malik | Brain tumor image generations using Deep Convolutional Generative adversarial networks:(DCGAN) | |
Mazher et al. | Multi-disease, multi-view and multi-center right ventricular segmentation in cardiac MRI using efficient late-ensemble deep learning approach | |
CN111046918B (en) | ICA-CNN classified fMRI data space pre-smoothing and broadening method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |