CN115222688A - Medical image classification method based on graph network time sequence - Google Patents

Medical image classification method based on graph network time sequence Download PDF

Info

Publication number
CN115222688A
CN115222688A CN202210814372.2A CN202210814372A CN115222688A CN 115222688 A CN115222688 A CN 115222688A CN 202210814372 A CN202210814372 A CN 202210814372A CN 115222688 A CN115222688 A CN 115222688A
Authority
CN
China
Prior art keywords
graph
neural network
layer
time
convolution neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210814372.2A
Other languages
Chinese (zh)
Other versions
CN115222688B (en
Inventor
潘丹
骆根强
张怡聪
容华斌
曾安
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Polytechnic Normal University
Original Assignee
Guangdong Polytechnic Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Polytechnic Normal University filed Critical Guangdong Polytechnic Normal University
Priority to CN202210814372.2A priority Critical patent/CN115222688B/en
Publication of CN115222688A publication Critical patent/CN115222688A/en
Application granted granted Critical
Publication of CN115222688B publication Critical patent/CN115222688B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a medical image classification method based on a graph network time sequence, which comprises the following steps: acquiring an fMRI image sample; constructing a graph network time sequence capable of showing dynamic changes of functional connection among brain partitions based on a k-s verification method, and processing the fMRI image samples to obtain a graph network time sequence corresponding to each fMRI image sample; and constructing a graph convolution neural network-time domain convolution neural network model, training and verifying, and finally classifying the medical images by utilizing the graph convolution neural network-time domain convolution neural network model. The medical image classification method provided by the invention realizes the projection of the brain function network connection dynamic change rule; the method provides a graph convolution neural network-time domain convolution neural network model, is beneficial to the extraction of graph characteristics and the learning of change rules in a graph network time sequence, and effectively improves the classification capability of the model.

Description

Medical image classification method based on graph network time sequence
Technical Field
The invention relates to the technical field of computer analysis of medical images, in particular to a medical image classification method based on a graph network time sequence.
Background
With the development of modern medicine, medical images play an increasingly important role in the auxiliary diagnosis and treatment of diseases. A large number of studies indicate that many neuropsychiatric diseases (such as AD and schizophrenia) are related to topological changes of brain structures and functional networks, and recently, human brain connectivity (Human connectiome) is proposed to mainly study dynamic complex functional networks formed by brain connections on a wide space-time scale so as to better understand the pathological basis of the neuropsychiatric diseases of the brain and further help to understand the working mechanism in the brain. Among them, functional Magnetic Resonance Imaging (fMRI) has both high temporal resolution and spatial resolution, and provides an important means for studying the functions of the human brain, and has become a research hotspot and difficulty of human brain connectivity omics. However, at the same time, fMRI images themselves are susceptible to noise interference and have high data dimensionality, which causes great difficulty in data processing and analysis. Aiming at the characteristics of the fMRI image, more valuable information can be mined by utilizing a deep learning method and a data driving mode, and the process of manually processing and analyzing data is simplified, so that the burden of doctors and researchers is reduced.
In the medical image classification method based on fMRI, a brain function network is constructed by using fMRI based on brain connectivity, and classification is performed according to a topological structure and various network parameters in the brain function network. However, the method only constructs a brain function network for an individual brain by using the BOLD signal time sequence contained in an fMRI image, and does not utilize the associated information of the BOLD signal time sequence contained in the fMRI image on the spatial dimension to the maximum extent, so that the dynamic changes of the associated relationship among different brain areas along with the time change in the neurophysiological process cannot be reflected, and the changing trends may play a critical role in the classification of fMRI.
The prior art discloses a brain network classification method based on a atlas neural network. The method comprises the following steps: firstly, extracting BOLD signals of all brain areas from an fMRI image; secondly, constructing a brain map capable of reflecting the topological structure characteristics of functional connection between brain areas; thirdly, inputting the constructed brain network and the actual diagnosis label into a graph volume neural network for feature learning and model training. According to the method, a brain network is constructed through the fMRI image, feature learning and classification are carried out based on the brain network, important information hidden in the image can be ignored, and dynamic changes of correlation relations among different brain areas along with time changes in the neurophysiological process cannot be reflected.
The prior art discloses a training method and apparatus, a computer device and a storage medium for constructing a network model based on fMRI. The method comprises the following steps: sampling and preprocessing original fMRI image data; establishing a 3D-CNN + LSTM model; creating an fMRI image segment as a first training data set, and using the fMRI segment with the minimum loss value in the first training data set as a second training data set; and training the 4D-CNN model by applying a second test data set and outputting a classification result. The two convolution neural models adopted by the method can extract time and space information in the fMRI image, but the two models have more parameters, the input fMRI image has high dimensionality, only a short time segment can be selected as the model input, and long-time dynamic change information in the fMRI image cannot be acquired.
Disclosure of Invention
In order to solve at least one technical defect, the invention provides a medical image classification method based on a graph network time sequence, which can reflect the dynamic change rule of brain function network connection.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a medical image classification method based on a graph network time sequence comprises the following steps:
s1: collecting an original fMRI image, preprocessing and sampling to obtain an fMRI image sample;
s2: constructing a graph network time sequence capable of showing dynamic changes of functional connection among brain partitions based on a k-s (Kolmogorov-Smirnov) verification method, and processing the fMRI image samples to obtain a graph network time sequence corresponding to each fMRI image sample;
s3: constructing a graph convolution neural network-time domain convolution neural network model, and training and verifying the graph convolution neural network-time domain convolution neural network model by utilizing a graph network time sequence;
s4: and inputting the fMRI images to be classified into the graph convolution neural network-time domain convolution neural network model which completes training and verification, so as to realize the classification of the medical images.
In the scheme, a graph network time sequence capable of showing the dynamic change of the functional connection between the brain partitions is constructed through the fMRI image, so that the showing of the dynamic change rule of the brain functional network connection is realized; meanwhile, a graph convolution neural network-time domain convolution neural network model is provided, so that extraction of graph features and learning of change rules in a graph network time sequence are facilitated, and classification capability of the model is effectively improved.
In the step S1, the raw fMRI image is preprocessed by using DPARSF software.
In the image acquisition process, factors such as head movement, respiration, heartbeat and the like of a testee can generate noise, so that the imaging quality of an image is deteriorated, therefore, in the data analysis process, preprocessing is carried out firstly, the influence of irrelevant noise is reduced, the signal to noise ratio is improved, and the preprocessing process is realized by using DPARSF software.
In step S1, the process of sampling the preprocessed fMRI image specifically includes: assume a sampled time slice length of kThe calculation selects the start frame as
Figure 546219DEST_PATH_IMAGE001
Finally, sampling to obtain a sample segment of
Figure 523402DEST_PATH_IMAGE002
And repeating the steps to obtain a plurality of sample fragments to form the fMRI image sample.
Generally speaking, training a deep learning model from scratch requires a large number of fMRI image samples, and for fMRI images, it is often difficult to obtain a large number of fMRI image samples for model training; therefore, the scheme provides a method for increasing the number of samples by dividing the fMRI image samples into shorter segments, so that the fMRI image sample data is enhanced, the number of training samples is greatly increased, and the training effect of the model is improved.
In step S2, the fMRI image sample is composed of a plurality of time slices of each fMRI image, and for each time slice in one fMRI image, the process of obtaining the graph network time sequence corresponding to the fMRI image sample specifically includes:
s21: for a time slice, dividing a human brain into a plurality of interested areas according to a brain area division template; taking each interested area as a vertex to obtain a vertex set;
s22: taking the correlation among the vertexes of the vertex set as an edge, and checking the correlation between the vertexes as the strength of the edge based on a k-s verification method to obtain an edge set;
s23: constructing an undirected graph of the time slice according to the vertex set and the edge set;
s24: and reselecting a time slice, repeatedly executing the steps S21-S24 to obtain an undirected graph of each time slice in the fMRI image, and obtaining the graph network time sequence corresponding to the fMRI image sample according to all the undirected graphs.
Wherein, in the step S2, the vertex set is expressed as
Figure 795115DEST_PATH_IMAGE003
In which
Figure 386502DEST_PATH_IMAGE004
Represents the first
Figure 501089DEST_PATH_IMAGE005
A region of interest (ROI) is formed,
Figure 157329DEST_PATH_IMAGE006
is the number of regions of interest; edge set adjacency matrix
Figure 408182DEST_PATH_IMAGE007
It is shown that, among others,Nthe number of vertices is represented as a function of,
Figure 872268DEST_PATH_IMAGE008
is a vertex
Figure 474150DEST_PATH_IMAGE009
The strength of the middle edge; in particular, according to the region of interest
Figure 730819DEST_PATH_IMAGE005
And the region of interest
Figure 960813DEST_PATH_IMAGE010
Obtained by verifying the k-s verification method of the BOLD signalp-valueValue as vertex
Figure 972631DEST_PATH_IMAGE009
The intensity of the edge between, k-s, verification method can be used to verify whether the data in the two regions of interest obey the same distribution ifp-valueThe smaller the value, the smaller the correlation between the two regions of interest; the above-mentionedp-valueThe calculation process of the value is specifically as follows:
setting region of interest
Figure 468334DEST_PATH_IMAGE005
Has a BOLD signal of
Figure 482689DEST_PATH_IMAGE011
Region of interest
Figure 708134DEST_PATH_IMAGE010
Has a BOLD signal of
Figure 500641DEST_PATH_IMAGE012
Wherein
Figure 998487DEST_PATH_IMAGE013
Respectively, regions of interest
Figure 924855DEST_PATH_IMAGE005
And the region of interest
Figure 880172DEST_PATH_IMAGE010
The total number of the BOLD signals of the two interested areas is
Figure 233793DEST_PATH_IMAGE014
(ii) a The region of interest
Figure 655154DEST_PATH_IMAGE005
The BOLD signals are sorted from small to large and are renumbered
Figure 791737DEST_PATH_IMAGE015
The sorted BOLD signals:
Figure 726195DEST_PATH_IMAGE016
obtaining non-descending order interested region
Figure 640930DEST_PATH_IMAGE017
BOLD signal of (a):
Figure 332943DEST_PATH_IMAGE018
；
is provided with
Figure 866692DEST_PATH_IMAGE019
Is a region of interest
Figure 281755DEST_PATH_IMAGE005
Empirical distribution function of (2):
Figure 649283DEST_PATH_IMAGE020
wherein the content of the first and second substances,
Figure 422067DEST_PATH_IMAGE021
is a region of interest
Figure 352983DEST_PATH_IMAGE005
Is less than or equal to
Figure 262033DEST_PATH_IMAGE022
The number of BOLD signals; obtaining the region of interest by the same method
Figure 66041DEST_PATH_IMAGE023
Empirical distribution function of
Figure 441569DEST_PATH_IMAGE024
：
Figure 317121DEST_PATH_IMAGE025
Wherein the content of the first and second substances,
Figure 18361DEST_PATH_IMAGE026
is a region of interest
Figure 180220DEST_PATH_IMAGE027
Is less than or equal to
Figure 662017DEST_PATH_IMAGE028
The number of BOLD signals;
computing verification statistics for k-s verification methods
Figure 747785DEST_PATH_IMAGE029
：
Figure 257526DEST_PATH_IMAGE030
Figure 200074DEST_PATH_IMAGE031
Wherein the content of the first and second substances,
Figure 44534DEST_PATH_IMAGE032
is a region of interest
Figure 183260DEST_PATH_IMAGE005
Empirical distribution of BOLD signals
Figure 655829DEST_PATH_IMAGE033
Of interest
Figure 175804DEST_PATH_IMAGE027
Empirical distribution of BOLD signals
Figure 52099DEST_PATH_IMAGE034
The maximum value of the absolute value of the difference, and finally, the region of interest is calculated
Figure 541986DEST_PATH_IMAGE005
And a region of interest
Figure 72325DEST_PATH_IMAGE027
K-s verification of BOLD signalsp-value value
Figure 12468DEST_PATH_IMAGE035
：
Figure 424995DEST_PATH_IMAGE036
Where Z is the validation statistic and e is a natural constant.
Wherein, the step S3 specifically includes the following steps:
s31: respectively constructing a graph convolution neural network and a time domain convolution neural network, and forming the graph convolution neural network and the time domain convolution neural network into a graph convolution neural network-time domain convolution neural network model;
s32: taking one part of the graph network time sequence as a training set, and taking the rest part as a verification set;
s33: training a graph convolution neural network-time domain convolution neural network model by using a training set;
s34: in the training process, the graph convolution neural network-time domain convolution neural network model is verified through a verification set, and the parameters with the highest accuracy in the verification set are used as the parameters of the graph convolution neural network-time domain convolution neural network model to complete the training of the graph convolution neural network-time domain convolution neural network model;
in the training process, the graph characteristics of the graph network time sequence are extracted by the constructed graph convolution neural network, and the graph characteristics are input into the time domain convolution neural network to obtain a classification result.
In the step S2, extracting an average value and a standard deviation of BOLD signals of the region of interest as features of a vertex of the BOLD signals to obtain a vertex attribute matrix; in the step S3, the graph convolution neural network comprises a plurality of convolution pooling units, a full connection layer and a softmax classifier; the convolution pooling unit comprises a graph convolution layer, a self-attention graph pooling layer and a readout layer; setting graph network time sequence of graph convolution neural network input to contain vertex attribute matrix
Figure 656256DEST_PATH_IMAGE037
And adjacency matrix
Figure 995096DEST_PATH_IMAGE038
Wherein, in the process,
Figure 981506DEST_PATH_IMAGE039
is the number of the vertices,
Figure 22275DEST_PATH_IMAGE040
is the number of vertex attributes; the operation of the graph convolution layer is specifically as follows:
Figure 40915DEST_PATH_IMAGE041
wherein the content of the first and second substances,
Figure 608163DEST_PATH_IMAGE042
is that
Figure 375261DEST_PATH_IMAGE043
An order identity matrix;
Figure 27960DEST_PATH_IMAGE044
is a diagonal matrix, representing the degrees of each vertex,
Figure 348826DEST_PATH_IMAGE045
，
Figure 911526DEST_PATH_IMAGE046
representative matrix
Figure 239739DEST_PATH_IMAGE047
The elements of row i and column j,
Figure 238788DEST_PATH_IMAGE048
representative matrix
Figure 677859DEST_PATH_IMAGE049
The element of the ith row and ith column,
Figure 891803DEST_PATH_IMAGE050
is the first
Figure 17016DEST_PATH_IMAGE051
Node embedding of layer if the node of layer 0 is characterized by
Figure 644306DEST_PATH_IMAGE052
Then, then
Figure 496856DEST_PATH_IMAGE053
，
Figure 158781DEST_PATH_IMAGE054
Is a learnable weight parameter;
the self-attention-graph pooling layer needs to obtain the importance degree of each layer of nodes, called self-attention of the nodes, and then before ranking the attention score weightkIs reserved to formTop-KA node; first calculate the self-attention score
Figure 750169DEST_PATH_IMAGE055
Wherein N is the number of nodes:
Figure 474542DEST_PATH_IMAGE056
wherein
Figure 520996DEST_PATH_IMAGE057
Is a learnable self-attention weight; selecting in a node selection mode according to the self-attention scoreTop-KThe node reserves a part of the input graph network time sequence, and specifically comprises:
Figure 395017DEST_PATH_IMAGE058
wherein the content of the first and second substances,
Figure 908038DEST_PATH_IMAGE059
an index representing a reservation node;
Figure 244342DEST_PATH_IMAGE060
presentation selection
Figure 953541DEST_PATH_IMAGE061
Before ranking
Figure 58900DEST_PATH_IMAGE062
A node of (2); pooling ratio
Figure 946084DEST_PATH_IMAGE063
Representing the percentage of nodes to be retained, before deriving the self-attentiveness value
Figure 458099DEST_PATH_IMAGE064
Large node index, then Masking operation is performed:
Figure 580776DEST_PATH_IMAGE065
wherein the content of the first and second substances,
Figure 947167DEST_PATH_IMAGE066
indicating node embedding with reserved index mask,
Figure 864307DEST_PATH_IMAGE067
indicating the attention score corresponding to the retention node,
Figure 299836DEST_PATH_IMAGE068
meaning that the multiplication is performed in bits,
Figure 898308DEST_PATH_IMAGE069
an adjacency matrix representing the reserved nodes is shown,
Figure 243839DEST_PATH_IMAGE070
，
Figure 220629DEST_PATH_IMAGE071
a node embedding and adjacency matrix representing outputs from the attention pooling layer;
the readout layer aggregates the node features to form a representation of a fixed size to obtain a high-dimensional representation of the graph, and the output of the readout layer is specifically characterized by:
Figure 753241DEST_PATH_IMAGE072
wherein, the first and the second end of the pipe are connected with each other,Nthe number of the nodes is represented as,
Figure 358666DEST_PATH_IMAGE073
denotes the lLayer oneiEmbedding nodes of each node, | | represents splicing operation of the features, and the read-out layer is actually a global average pooling layer and a global maximum pooling layer to obtain splicing of the features;
in order to realize the reconstruction output of data, the forward propagation process of the full connection layer is as follows:
Figure 480075DEST_PATH_IMAGE074
Figure 4597DEST_PATH_IMAGE075
，
Figure 899872DEST_PATH_IMAGE076
are respectively the first
Figure 433621DEST_PATH_IMAGE077
The learnable weight matrix and the learnable bias for the fully connected one of the layers,
Figure 114263DEST_PATH_IMAGE078
and
Figure 481791DEST_PATH_IMAGE079
respectively representing the number of neurons of the l-th layer of full-junction layer and the number of neurons of the l + 1-th layer of full-junction layer, and finally obtaining a final classification result through a softmax classifier:
Figure 254575DEST_PATH_IMAGE080
wherein the content of the first and second substances,
Figure 185491DEST_PATH_IMAGE081
，
Figure 94541DEST_PATH_IMAGE082
is the number of neurons of the l-th fully-connected layer,
Figure 898549DEST_PATH_IMAGE083
is the number of categories; the graph convolution neural network obtains a plurality of graphs obtained by self-attention graph pooling layers, obtains high-dimensional feature representations of different hierarchical graphs through reading layers, adds the high-dimensional features to obtain a final high-dimensional feature representation, reconstructs the high-dimensional features through a full connection layer, uses the reconstructed features as input of a time domain convolution neural network, and finally obtains a classification result of an input graph through a softmax classifier.
In the step S3, an input layer of the time-domain convolutional neural network is connected to a full-connection layer of the graph convolutional neural network, processed by a plurality of TCN layers, and output by an output layer to the softmax classifier after being processed by an expansion layer, where each TCN layer converts the input dimension size into a dimension consistent with the output dimension size through a one-dimensional full-convolution structure, and the forward propagation process is as follows:
Figure 516218DEST_PATH_IMAGE084
sequence data is formed by splicing output vectors of assumed full connection layers
Figure 126191DEST_PATH_IMAGE085
Wherein
Figure 765114DEST_PATH_IMAGE086
Is the length of the time slice or slices,
Figure 989291DEST_PATH_IMAGE087
the number of neurons in the full junction layer; will be provided with
Figure 471088DEST_PATH_IMAGE088
Inputting the time slice into a TCN layer, outputting and expanding the time slice into a one-dimensional vector through an expansion layer after passing through a plurality of TCN layers, and finally classifying the time slice through a softmax classifier to obtain a classification result of the time slice.
Wherein, in the step S3, the TCN layer of the time-domain convolutional neural network is composed of a causal convolution and an expansion convolution, wherein:
is due toIn the convolution, the element of the output sequence depends only on the element preceding it in the input sequence, which is the previous layer at a certain time for the time series data
Figure 494539DEST_PATH_IMAGE089
Is dependent only on the next layer
Figure 378181DEST_PATH_IMAGE089
The values at and before time, namely:
Figure 274724DEST_PATH_IMAGE090
wherein the content of the first and second substances,
Figure 119183DEST_PATH_IMAGE091
the output at time T of the causal convolution is shown,
Figure 70959DEST_PATH_IMAGE092
a feature vector representing layer i time 1 to time T; the expansion convolution refers to performing convolution operation by using discontinuous neurons with the same size as a convolution kernel; the expansion convolution has a expansion coefficientdThe method is used for controlling the discontinuity degree of neurons participating in convolution operation, and the calculation formula of the dilation convolution is as follows:
Figure 464900DEST_PATH_IMAGE093
wherein the content of the first and second substances,ethe coefficient of expansion is expressed in terms of,
Figure 984874DEST_PATH_IMAGE094
which represents the size of the convolution kernel,
Figure 861170DEST_PATH_IMAGE095
weight of the i-th term of the convolution kernel wheneAt 1, the dilated convolution degenerates to the normal convolution, controlled byeSo as to enlarge the receptive field under the premise of unchanged calculated amount.
In the graph convolution neural network-time domain convolution neural network model constructed in the step S3, the loss function is composed of three parts, which are respectively node classification loss, time segment classification loss, and final classification loss, and the loss function is specifically expressed as:
Figure 616636DEST_PATH_IMAGE096
wherein, the first and the second end of the pipe are connected with each other,
Figure 84658DEST_PATH_IMAGE097
is the node classification loss of the jth node at the ith time point,
Figure 634588DEST_PATH_IMAGE098
，
Figure 499645DEST_PATH_IMAGE099
in this scheme, a self-attention pooling layer is applied in the graph convolution neural network, so that only the final for each graph is retainedTop-KAnd the loss function is also only calculatedTop-KClassification loss of nodes;
Figure 934168DEST_PATH_IMAGE100
is the firstiThe time slice classification of a time point is lost,
Figure 381330DEST_PATH_IMAGE098
，
Figure 993839DEST_PATH_IMAGE101
the number of time points is the classification loss of the graph convolution neural network;
Figure 159241DEST_PATH_IMAGE102
a classification loss, hyper-parameter, for the final time-domain convolutional neural network
Figure 928614DEST_PATH_IMAGE103
Classification losses for control nodes, time segments, andthe final classification loss has
Figure 886075DEST_PATH_IMAGE104
And is
Figure 777807DEST_PATH_IMAGE105
(ii) a All classification loss functions use cross-entropy loss functions, which are specifically expressed as:
Figure 305872DEST_PATH_IMAGE106
Figure 206832DEST_PATH_IMAGE107
represents the sample ofjThe true probability value of the seed class,
Figure 314072DEST_PATH_IMAGE108
representing the sample obtained from the modeljPredicted probability values for the species classes.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention provides a medical image classification method based on a graph network time sequence, which constructs the graph network time sequence capable of showing the dynamic change of functional connection between brain partitions through fMRI images, and realizes the showing of the dynamic change rule of the brain functional network connection; meanwhile, a graph convolution neural network-time domain convolution neural network model is provided, so that the extraction of graph characteristics and the learning of change rules in a graph network time sequence are facilitated, and the classification capability of the model is effectively improved.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a detailed schematic diagram of the graph convolutional neural network-time domain convolutional neural network model according to the present invention;
FIG. 3 is a detailed schematic diagram of the convolutional neural network of the present invention;
fig. 4 is a specific schematic diagram of the time domain convolutional neural network according to the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described with reference to the drawings and the embodiments.
Example 1
As shown in fig. 1, a medical image classification method based on graph network time series includes the following steps:
s1: collecting an original fMRI image, preprocessing and sampling to obtain an fMRI image sample;
s2: constructing a graph network time sequence capable of showing dynamic changes of functional connection among brain partitions based on a k-s verification method, and processing the fMRI image samples to obtain a graph network time sequence corresponding to each fMRI image sample;
s3: constructing a graph convolution neural network-time domain convolution neural network model, and training and verifying the graph convolution neural network-time domain convolution neural network model by utilizing a graph network time sequence;
s4: and inputting the fMRI image to be classified into the graph convolution neural network-time domain convolution neural network model which completes training and verification, so as to realize classification of the medical image.
In the specific implementation process, a graph network time sequence capable of showing the dynamic change of the functional connection between the brain partitions is constructed through the fMRI images, so that the showing of the dynamic change rule of the brain functional network connection is realized; meanwhile, a graph convolution neural network-time domain convolution neural network model is provided, so that the extraction of graph characteristics and the learning of change rules in a graph network time sequence are facilitated, and the classification capability of the model is effectively improved.
More specifically, in step S1, the raw fMRI image is preprocessed by DPARSF software.
In the image acquisition process, the testee moves, factors such as breathing, heartbeat, can produce the noise, leads to the imaging quality variation of image, consequently when data analysis, carries out the preliminary treatment earlier, reduces the influence of irrelevant noise, improves the SNR, and this scheme uses DPARSF software to realize the preliminary treatment process, specifically includes:
first removing the first 10 frames of data of each fMRI image sample to obtain a stable signal; second, each slice is time-corrected to ensure that the data on each slice corresponds to the same point in time. After the temporal correction, the spatial correction is continued, and each frame image of each subject is realigned with its average image and normalized spatially to MNI (Montreal Neurological Institute) space, thereby eliminating the inter-individual difference; all images were spatially smoothed using a 4 × 4mm 3 full-width half-height gaussian kernel; linear trend removal and low frequency filtering (0.01 Hz-0.08 Hz); covariate regression analysis, the interference factors eliminated include cerebrospinal fluid, white matter signals, and head movements.
More specifically, in step S1, the process of sampling the preprocessed fMRI image specifically includes: assuming a time slice length of k for the sample, the calculation selects a start frame of
Figure 517651DEST_PATH_IMAGE109
Finally, sampling to obtain a sample segment of
Figure 392066DEST_PATH_IMAGE110
And repeating the steps to obtain a plurality of sample fragments to form the fMRI image sample.
Generally speaking, training a deep learning model from scratch requires a large number of fMRI image samples, and for fMRI images, it is often difficult to obtain a large number of fMRI image samples for model training; therefore, the scheme provides a method for increasing the number of samples by segmenting the fMRI image samples into shorter segments, so that the fMRI image sample data is enhanced, the number of training samples is greatly increased, and the training effect of the model is improved.
More specifically, in step S2, the fMRI image sample is composed of a plurality of time slices of each fMRI image, and for each time slice in one fMRI image, the process of obtaining the graph network time series corresponding to the fMRI image sample specifically includes:
s21: for a time slice, dividing a human brain into a plurality of interested areas according to a brain area division template; taking each interested region as a vertex to obtain a vertex set
Figure 955771DEST_PATH_IMAGE111
；
S22: the correlation among all vertexes of the vertex set is used as an edge, the correlation size between the vertexes is checked based on a k-s verification method to be used as the strength of the edge, and the edge set is obtained
Figure 497611DEST_PATH_IMAGE112
；
S23: constructing the undirected graph of the time slice according to the vertex set and the edge set
Figure 668830DEST_PATH_IMAGE113
；
S24: reselecting a time slice, repeatedly executing the steps S21-S24 to obtain an undirected graph of each time slice in the fMRI image, and obtaining a graph network time sequence corresponding to the fMRI image sample according to all the undirected graphs
Figure 922219DEST_PATH_IMAGE114
. Wherein the content of the first and second substances,
Figure 899402DEST_PATH_IMAGE115
is the number of fMRI time points,
Figure 171114DEST_PATH_IMAGE116
represents the first
Figure 841130DEST_PATH_IMAGE117
A map constructed from time slices.
In a specific implementation, for each time slice in each fMRI image sample, the human brain is divided into N regions of interest according to a brain region division template, such as an AAL template, a Brainnetome template, and the like. In the scheme, an AAL template is adopted for division, the template divides a human brain into 116 interested areas, wherein 90 interested areas are brain areas, only 90 interested areas of the brain areas are selected in the scheme, and each interested area is used as a vertex to obtain a vertex set.
More specifically, in the step S2, the vertex set is represented as
Figure 877088DEST_PATH_IMAGE118
Wherein
Figure 798908DEST_PATH_IMAGE119
Represents the first
Figure 784181DEST_PATH_IMAGE120
A region of interest (ROI) is formed,
Figure 271705DEST_PATH_IMAGE121
is the number of regions of interest; edge set by adjacency matrix
Figure 545691DEST_PATH_IMAGE122
It is shown that, among others,Nthe number of vertices is represented as a function of,
Figure 864677DEST_PATH_IMAGE123
is a vertex
Figure 360249DEST_PATH_IMAGE124
The strength of the middle edge; in particular, according to the region of interest
Figure 44172DEST_PATH_IMAGE125
And the region of interest
Figure 133350DEST_PATH_IMAGE126
Obtained by verifying the k-s verification method of the BOLD signalp-valueValue as vertex
Figure 882126DEST_PATH_IMAGE124
The intensity of the edge between, k-s verification method can be used to verify whether the data in the two regions of interest obey the same distribution ifp-valueThe smaller the value, the smaller the correlation between the two regions of interest; the above-mentionedp-valueThe calculation process of the value is specifically as follows:
setting region of interest
Figure 841992DEST_PATH_IMAGE125
Has a BOLD signal of
Figure 962394DEST_PATH_IMAGE127
Region of interest
Figure 397924DEST_PATH_IMAGE126
Has a BOLD signal of
Figure 58712DEST_PATH_IMAGE128
Wherein
Figure 279609DEST_PATH_IMAGE129
Are respectively the region of interest
Figure 53137DEST_PATH_IMAGE125
And the region of interest
Figure 585749DEST_PATH_IMAGE130
The total number of BOLD signals of the two regions of interest is
Figure 191174DEST_PATH_IMAGE131
(ii) a The region of interest
Figure 391211DEST_PATH_IMAGE125
The BOLD signals are sorted from small to large and renumbered
Figure 837105DEST_PATH_IMAGE133
The sorted BOLD signals:
Figure 732380DEST_PATH_IMAGE016
obtaining non-descending order interested region
Figure 550DEST_PATH_IMAGE134
BOLD signal of (a):
Figure 477930DEST_PATH_IMAGE135
；
is provided with
Figure 48720DEST_PATH_IMAGE136
Is a region of interest
Figure 821504DEST_PATH_IMAGE137
Empirical distribution function of (2):
Figure 549157DEST_PATH_IMAGE138
wherein the content of the first and second substances,
Figure 599153DEST_PATH_IMAGE139
is a region of interest
Figure 465478DEST_PATH_IMAGE140
In (C) is less than or equal to
Figure 83147DEST_PATH_IMAGE141
The number of BOLD signals of (a); obtaining the region of interest by the same method
Figure 365224DEST_PATH_IMAGE142
Empirical distribution function of
Figure 394360DEST_PATH_IMAGE143
：
Figure 556220DEST_PATH_IMAGE144
Wherein the content of the first and second substances,
Figure 38017DEST_PATH_IMAGE145
is a region of interest
Figure 123785DEST_PATH_IMAGE146
Is less than or equal to
Figure 633526DEST_PATH_IMAGE147
The number of BOLD signals of (a);
computing verification statistics for k-s verification methods
Figure 841653DEST_PATH_IMAGE148
：
Figure 686112DEST_PATH_IMAGE149
Figure 559259DEST_PATH_IMAGE150
Wherein the content of the first and second substances,
Figure 562987DEST_PATH_IMAGE151
is a region of interest
Figure 817382DEST_PATH_IMAGE152
Empirical distribution of BOLD signals
Figure 273771DEST_PATH_IMAGE153
Of interest region
Figure 449144DEST_PATH_IMAGE154
Empirical distribution of BOLD signals
Figure 651587DEST_PATH_IMAGE155
The maximum value of the absolute value of the difference, and finally, the region of interest is calculated
Figure 467096DEST_PATH_IMAGE156
And a region of interest
Figure 332153DEST_PATH_IMAGE154
K-s verification of BOLD signalsp-value value
Figure 766676DEST_PATH_IMAGE157
：
Figure 213838DEST_PATH_IMAGE036
Where Z is the verification statistic and e is a natural constant.
Example 2
More specifically, on the basis of embodiment 1, a graph convolution neural network-time domain convolution neural network model is constructed in step S3, and in the model building process, the design mainly focuses on how to fuse the spatial dimension information and the time dimension information. In this embodiment, the graph features are first extracted by using a graph convolution network, and the graph features are input into a time domain convolution neural network, so as to obtain a final classification result. The step S3 specifically includes the following steps:
s31: respectively constructing a graph convolution neural network and a time domain convolution neural network, and forming the graph convolution neural network and the time domain convolution neural network into a graph convolution neural network-time domain convolution neural network model;
s32: taking one part of the graph network time sequence as a training set, and taking the rest part as a verification set;
s33: training a graph convolution neural network-time domain convolution neural network model by using a training set;
s34: in the training process, the graph convolution neural network-time domain convolution neural network model is verified through a verification set, and the parameters with the highest accuracy in the verification set are used as the parameters of the graph convolution neural network-time domain convolution neural network model to complete the training of the graph convolution neural network-time domain convolution neural network model;
in the training process, the graph characteristics of the graph network time sequence are extracted by the constructed graph convolution neural network, and the graph characteristics are input into the time domain convolution neural network to obtain a classification result.
More specifically, in the step S2, the BOLD signal of the region of interest is extractedThe average value and the standard deviation of the vertex attribute matrix are used as the characteristics of the vertex to obtain a vertex attribute matrix; in the step S3, the graph convolution neural network structure designed in this embodiment includes, as shown in fig. 3, a plurality of convolution pooling units, a full connection layer and a softmax classifier; the convolution pooling unit comprises a graph convolution layer, a self-attention graph pooling layer and a readout layer; setting graph network time sequence of graph convolution neural network input to contain vertex attribute matrix
Figure 357506DEST_PATH_IMAGE158
And adjacency matrix
Figure 663853DEST_PATH_IMAGE159
Wherein, in the step (A),
Figure 495543DEST_PATH_IMAGE160
is the number of the vertices,
Figure 453004DEST_PATH_IMAGE161
is the number of vertex attributes; the operation of the graph convolution layer is specifically as follows:
Figure 344736DEST_PATH_IMAGE162
wherein the content of the first and second substances,
Figure 935118DEST_PATH_IMAGE163
is that
Figure 193667DEST_PATH_IMAGE160
An order identity matrix;
Figure 615421DEST_PATH_IMAGE044
is a diagonal matrix, representing the degrees of each vertex,
Figure 615738DEST_PATH_IMAGE164
，
Figure 880366DEST_PATH_IMAGE165
representative matrix
Figure 319438DEST_PATH_IMAGE166
The elements of row i and column j,
Figure 736644DEST_PATH_IMAGE167
representative matrix
Figure 658595DEST_PATH_IMAGE168
The element of the ith row and ith column,
Figure 285885DEST_PATH_IMAGE169
is the first
Figure 138435DEST_PATH_IMAGE170
Node embedding of layer if the node of layer 0 is characterized by
Figure 534781DEST_PATH_IMAGE171
Then, then
Figure 126168DEST_PATH_IMAGE172
，
Figure 850542DEST_PATH_IMAGE173
Is a learnable weight parameter;
the self-attention-seeking pooling layer needs to obtain the degree of importance of each layer of nodes, called self-attention of the nodes, and then before ranking the attention score weightskIs reserved to formTop-KA node; first calculate the self-attention score
Figure 631416DEST_PATH_IMAGE174
Wherein N is the number of nodes:
Figure 567754DEST_PATH_IMAGE175
wherein
Figure 284038DEST_PATH_IMAGE057
Is a learnable self-attention weight; the above equation is ten-phase with the operation of the graph volume layerSimilarly, the graph convolution layer is obtained by embedding nodes in the next layer, and the above formula is obtained by self-attention scores of the nodes in the layer, and the nodes are selected by adopting a node selection mode according to the self-attention scoresTop-KThe node, which retains a part of the input graph network time sequence, specifically is:
Figure 885920DEST_PATH_IMAGE176
wherein the content of the first and second substances,
Figure 329540DEST_PATH_IMAGE177
an index representing a reservation node;
Figure 372582DEST_PATH_IMAGE178
presentation selection
Figure 384401DEST_PATH_IMAGE179
Before ranking
Figure 568520DEST_PATH_IMAGE062
The node of (2); pooling ratio
Figure 956776DEST_PATH_IMAGE180
Representing the percentage of nodes to be retained, before deriving the self-attention value
Figure 854325DEST_PATH_IMAGE181
Large node index, then Masking operation is performed:
Figure 161678DEST_PATH_IMAGE182
wherein the content of the first and second substances,
Figure 738153DEST_PATH_IMAGE183
indicating node embedding with reserved index mask,
Figure 274307DEST_PATH_IMAGE184
to representThe attention score corresponding to the node is retained,
Figure 774166DEST_PATH_IMAGE185
which means that the multiplication is performed in bits,
Figure 127787DEST_PATH_IMAGE186
an adjacency matrix representing the reserved nodes is shown,
Figure 801344DEST_PATH_IMAGE187
，
Figure 452775DEST_PATH_IMAGE188
a node embedding and adjacency matrix representing outputs from the attention pooling layer;
the readout layer aggregates the node features to form a fixed-size representation, resulting in a high-dimensional representation of the graph, and the readout layer outputs are specifically characterized by:
Figure 387232DEST_PATH_IMAGE189
wherein, the first and the second end of the pipe are connected with each other,Nthe number of the nodes is represented as,
Figure 787121DEST_PATH_IMAGE190
denotes the l th layeriEmbedding nodes of each node, | | represents splicing operation of the features, and the read-out layer is actually a global average pooling layer and a global maximum pooling layer to obtain splicing of the features;
in order to realize the reconstruction output of data, the forward propagation process of the full connection layer is as follows:
Figure 72609DEST_PATH_IMAGE191
Figure 763615DEST_PATH_IMAGE192
，
Figure 693525DEST_PATH_IMAGE193
are respectively the first
Figure 388949DEST_PATH_IMAGE194
The learnable weight matrix and the learnable bias for the fully connected one of the layers,
Figure 817525DEST_PATH_IMAGE195
and
Figure 764752DEST_PATH_IMAGE079
respectively represent
Figure 939382DEST_PATH_IMAGE194
And finally, obtaining a final classification result through a softmax classifier by the number of neurons of the layer full-junction layer and the number of neurons of the l +1 th layer full-junction layer:
Figure 163296DEST_PATH_IMAGE196
wherein the content of the first and second substances,
Figure 423376DEST_PATH_IMAGE197
，
Figure 705453DEST_PATH_IMAGE198
is the first
Figure 593644DEST_PATH_IMAGE194
The number of the neurons of the layer full-connection layer,
Figure 630870DEST_PATH_IMAGE199
is the number of categories; the graph convolution neural network obtains a plurality of graphs obtained by self-attention graph pooling layers, obtains high-dimensional feature representations of different hierarchical graphs through reading layers, adds the high-dimensional features to obtain a final high-dimensional feature representation, reconstructs the high-dimensional features through a full connection layer, uses the reconstructed features as input of a time domain convolution neural network, and finally obtains a classification result of an input graph through a softmax classifier.
More specifically, in the step S3, the time-domain convolutional neural network structure in this embodiment is as shown in fig. 4, an input layer of the time-domain convolutional neural network structure is connected to a fully connected layer of the graph convolutional neural network, and the time-domain convolutional (TCN) layer is processed by a plurality of time-domain convolutional (TCN) layers, and output by an output layer to a softmax classifier after being processed by an expansion layer, where each TCN layer transforms the dimension of its input to be consistent with the dimension of its output by a one-dimensional fully convolutional structure, and its forward propagation process is as follows:
Figure 50350DEST_PATH_IMAGE200
sequence data is formed by splicing output vectors of assumed full connection layers
Figure 90112DEST_PATH_IMAGE201
Wherein
Figure 708175DEST_PATH_IMAGE202
Is the length of the time slice or slices,
Figure 57248DEST_PATH_IMAGE203
the number of neurons in the full connection layer; will be provided with
Figure 947713DEST_PATH_IMAGE204
Inputting the time slice into a TCN layer, outputting and expanding the time slice into a one-dimensional vector through an expansion layer after passing through a plurality of TCN layers, and finally classifying the time slice through a softmax classifier to obtain a classification result of the time slice. In order to reduce the number of parameters of the model, the graph convolution neural network in the model of the present embodiment adopts a design of sharing weights.
More specifically, in the step S3, the TCN layer of the time domain convolutional neural network is composed of a causal convolution and an expansion convolution, wherein:
in causal convolution, the element of the output sequence depends only on the element before the element in the input sequence, future data cannot be seen, and the method is a strict sequence constraint model; the time sequence data is the previous layer at a certain moment
Figure 899488DEST_PATH_IMAGE205
Is only dependent onDepends on the next layer
Figure 513003DEST_PATH_IMAGE205
The values at and before time, namely:
Figure 892032DEST_PATH_IMAGE206
wherein the content of the first and second substances,
Figure 791765DEST_PATH_IMAGE091
the output representing the time T of the causal convolution,
Figure 422598DEST_PATH_IMAGE207
a feature vector representing layer i time 1 to time T; the expansion convolution refers to performing convolution operation by using a discontinuous neuron with the same size as a convolution kernel; the expansion convolution has a expansion coefficientdThe method is used for controlling the discontinuity degree of neurons participating in convolution operation, and the calculation formula of the dilation convolution is as follows:
Figure 15253DEST_PATH_IMAGE208
wherein the content of the first and second substances,dthe coefficient of expansion is expressed as a function of,
Figure 955396DEST_PATH_IMAGE209
which represents the size of the convolution kernel,
Figure 571185DEST_PATH_IMAGE210
weight of the i-th term of the convolution kernel wheneAt 1, the dilated convolution degenerates to the normal convolution by controleThereby enlarging the receptive field under the premise of unchanged calculated amount.
More specifically, in the graph convolution neural network-time domain convolution neural network model constructed in step S3, the loss function is composed of three parts, which are respectively node classification loss, time segment classification loss, and final classification loss, and the loss function is specifically expressed as:
Figure 864763DEST_PATH_IMAGE211
wherein the content of the first and second substances,
Figure 938024DEST_PATH_IMAGE212
is the node classification loss of the jth node at the ith time point,
Figure 924435DEST_PATH_IMAGE213
，
Figure 27520DEST_PATH_IMAGE214
in this scheme, a self-attention pooling layer is applied in the graph convolution neural network, so that only the final for each graph is retainedTop-KAnd the loss function is also only calculatedTop-KClassification loss of nodes;
Figure 983843DEST_PATH_IMAGE215
is the firstiThe time slice classification of a time point is lost,
Figure 551091DEST_PATH_IMAGE216
，
Figure 583769DEST_PATH_IMAGE217
the number of time points is the classification loss of the graph convolution neural network;
Figure 921953DEST_PATH_IMAGE218
a final classification loss, hyperparameter, for a time-domain convolutional neural network
Figure 557334DEST_PATH_IMAGE219
The influence of the classification loss of the control node, the classification loss of the time segment and the final classification loss respectively has
Figure 854454DEST_PATH_IMAGE220
And is
Figure 182667DEST_PATH_IMAGE221
(ii) a All classification loss functions use cross-entropy loss functions, which are specifically expressed as:
Figure 244033DEST_PATH_IMAGE222
Figure 558471DEST_PATH_IMAGE223
represents the sample ofjThe true probability value of the seed class,
Figure 100311DEST_PATH_IMAGE224
representing the sample obtained from the modeljPredicted probability values for the species classes.
In the specific implementation process, a loss function consisting of node classification loss, time segment classification loss and final classification loss is provided, so that the classification capability of each part of modules of the model and the classification capability of the final model are improved.
Example 3
More specifically, in step S3, the convolutional neural network-time domain convolutional neural network model may be tested, in the testing stage, the fMRI image is sampled in a sliding window manner, then all the sampling samples construct a convolutional network time sequence, the convolutional neural network-time domain convolutional neural network model provided in the present solution is input, and the obtained classification results of all the sampling samples are obtained in a simple voting manner to obtain the final classification result. In particular, assume a test fMRI image sample
Figure 22261DEST_PATH_IMAGE225
The length of the sampling segment is k frames, the sliding step length is m, and a sampling samples are finally obtained
Figure 524918DEST_PATH_IMAGE226
，
Figure 236522DEST_PATH_IMAGE227
，
Figure 819819DEST_PATH_IMAGE228
，
Figure 365201DEST_PATH_IMAGE229
In which
Figure 214208DEST_PATH_IMAGE230
. Inputting the prediction classification result into a model to obtain corresponding prediction classification results respectively
Figure 618251DEST_PATH_IMAGE231
Wherein
Figure 806787DEST_PATH_IMAGE232
The final simple voting results in classification
Figure 647704DEST_PATH_IMAGE233
。
Hereinafter, taking Alzheimer's Disease (AD) as an example, using fMRI image data from the american large Alzheimer's Disease public database ADNI (Alzheimer's Disease Neuroimaging Initiative), 250 fMRI image data (121 ADs, 129 control groups) from 60 subjects (25 ADs, 35 control groups) are collected in total, that is, one subject may have a plurality of fMRI image data, and the above-described data are inputted as experimental data of the present invention into the model in the present application to evaluate the effect of the model and compare performance differences. As described above, training data is input into the model, and the model is then tested for performance using the test data set. To reduce the impact of dataset partitioning on the experimental results, this example employed five cross-validation methods to evaluate the performance of the model. To avoid data leakage, the data set is partitioned according to the subjects, i.e., multiple fMRI images of one subject only appear in the training set or the test set at the same time.
1. Parameter setting
During training, the Batch _ size is 32, the epochs are 200, parameter updating is carried out by adopting an Adam gradient descent method, and the learning rate is0.001, the learning rate is exponentially reduced along with the change of time,
Figure 108641DEST_PATH_IMAGE234
. And dividing part of data in the training set as a verification set, wherein in the training process, the parameter with the highest accuracy of the model in the verification set is used as the final parameter of the model. Time window m =10 when tested.
2. Results of the experiment
Table 1 shows the effect of different sample lengths on the model results, which gave the best generalization performance for a sample frame length of 64.
TABLE 1
Sampling frame length Rate of accuracy Standard deviation of
16 0.68 0.08
32 0.62 0.16
48 0.69 0.07
64 0.72 0.10
Table 2 shows the loss function for a sample length of 64
Figure 693207DEST_PATH_IMAGE235
Influence on model results in different values:
TABLE 2
Figure 736249DEST_PATH_IMAGE236
Figure 374166DEST_PATH_IMAGE237
Figure 932186DEST_PATH_IMAGE238
 Rate of accuracy Standard deviation of
0 0 1 0.53 0.10
0 0.5 0.5 0.66 0.12
0.2 0.3 0.5 0.72 0.10
As can be seen from Table 2, the loss function designed by the present invention is effective compared to the loss function using only the final loss (
Figure 992546DEST_PATH_IMAGE239
) And loss of time slice and final loss (
Figure 342625DEST_PATH_IMAGE240
) The classification performance of the model trained by the loss function is improved to a certain extent. Finally, in order to verify the effectiveness of the method for constructing functional connections between the regions of interest based on the k-s verification method proposed in the present application, the traditional method for constructing functional connections based on pearson correlation is used as a comparison experiment, i.e. for each time point, the region of interest is constructediAnd a region of interestjThe connection strength of (2) is:
Figure 525344DEST_PATH_IMAGE241
wherein the content of the first and second substances,
Figure 711606DEST_PATH_IMAGE242
and
Figure 637974DEST_PATH_IMAGE243
representing a region of interest for which Pearson correlation analysis is to be performediAnd a region of interestjOf the BOLD signal，
Figure 137832DEST_PATH_IMAGE244
Respectively representing the interested regions at the t-th time pointiAnd a region of interestjAverage value of the BOLD signal of (1). In a graph network time sequence constructed by the method, the edge weight of each graph is the same, and finally the five-fold cross validation obtained by the method has the average accuracy rate of 60% and the standard deviation of 5%. Compared with the method, the method provided by the invention has the advantages that the precision is improved by about 12 percent, and better effect is obtained, which shows that the method for constructing the time sequence of the graph network provided by the invention is effective. The method can be interpreted as that the method for constructing the network time sequence of the map based on the k-s test can effectively reflect the dynamic change of the correlation relationship between the brain region functions presented along with the time change in the neurophysiologic process, and the traditional method based on the Pearson correlation is based on the construction of the brain function connection at all time points and cannot express the dynamic change mode.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A medical image classification method based on a graph network time sequence is characterized by comprising the following steps:
s1: acquiring an original fMRI image, preprocessing and sampling to obtain an fMRI image sample;
s2: constructing a graph network time sequence capable of showing dynamic changes of functional connection among brain partitions based on a k-s verification method, and processing the fMRI image samples to obtain a graph network time sequence corresponding to each fMRI image sample;
s3: constructing a graph convolution neural network-time domain convolution neural network model, and training and verifying the graph convolution neural network-time domain convolution neural network model by utilizing a graph network time sequence;
s4: and inputting the fMRI image to be classified into the graph convolution neural network-time domain convolution neural network model which completes training and verification, so as to realize classification of the medical image.
2. The method for classifying medical images based on graph network time series according to claim 1, wherein in step S1, raw fMRI images are preprocessed by DPARSF software.
3. The method for classifying medical images based on graph network time series according to claim 1, wherein in the step S1, the process of sampling the preprocessed fMRI image specifically comprises: assuming a time slice length of k for the sample, the calculation selects a start frame of
Figure DEST_PATH_IMAGE001
Finally, sampling to obtain a sample segment of
Figure 911160DEST_PATH_IMAGE002
And repeating the steps to obtain a plurality of sample fragments to form the fMRI image sample.
4. The method according to claim 1, wherein in step S2, the fMRI image samples are composed of a plurality of time slices of each fMRI image, and for each time slice in one fMRI image, the process of obtaining the atlas network time series corresponding to the fMRI image sample specifically comprises:
s21: for a time slice, dividing a human brain into a plurality of interested areas according to a brain area division template; taking each interested area as a vertex to obtain a vertex set;
s22: taking the correlation among the vertexes of the vertex set as an edge, and checking the correlation between the vertexes as the strength of the edge based on a k-s verification method to obtain an edge set;
s23: constructing an undirected graph of the time slice according to the vertex set and the edge set;
s24: and reselecting a time slice, repeatedly executing the steps S21-S24 to obtain an undirected graph of each time slice in the fMRI image, and obtaining the graph network time sequence corresponding to the fMRI image sample according to all the undirected graphs.
5. The method according to claim 4, wherein in step S2, the vertex set is represented as
Figure DEST_PATH_IMAGE003
Wherein
Figure 923240DEST_PATH_IMAGE004
Represents the first
Figure DEST_PATH_IMAGE005
The region of interest is determined by the area of interest,
Figure 296453DEST_PATH_IMAGE006
is the number of regions of interest; edge set by adjacency matrix
Figure DEST_PATH_IMAGE007
It is shown that, among others,Nthe number of vertices is represented as a function of,
Figure 783673DEST_PATH_IMAGE008
is a vertex
Figure DEST_PATH_IMAGE009
The strength of the middle edge; in particular, the BOLD signal according to the region of interest and the region of interest
Figure 394782DEST_PATH_IMAGE010
Obtained by verifying the k-s verification method of the BOLD signalp-valueValue as vertex
Figure 545141DEST_PATH_IMAGE009
The intensity of the edge between, k-s, verification method can be used to verify whether the data in the two regions of interest obey the same distribution ifp-valueThe smaller the value, the smaller the correlation between the two regions of interest; the describedp-valueThe calculation process of the value is specifically as follows:
setting region of interest
Figure DEST_PATH_IMAGE011
Has a BOLD signal of
Figure 608912DEST_PATH_IMAGE013
Region of interest
Figure 637173DEST_PATH_IMAGE010
Has a BOLD signal of
Figure 102790DEST_PATH_IMAGE015
Wherein
Figure 158470DEST_PATH_IMAGE016
Are respectively the region of interest
Figure 240696DEST_PATH_IMAGE018
And the region of interest
Figure 243287DEST_PATH_IMAGE019
The total number of BOLD signals of the two interested areas
Figure 593103DEST_PATH_IMAGE021
(ii) a The region of interest
Figure 288527DEST_PATH_IMAGE011
B of (A)The OLD signals are sorted from small to large and then renumbered
Figure 592469DEST_PATH_IMAGE023
The sorted BOLD signals:
Figure DEST_PATH_IMAGE024
obtaining non-descending order interested region
Figure 759271DEST_PATH_IMAGE010
BOLD signal of (a):
Figure 668321DEST_PATH_IMAGE025
；
is provided with
Figure DEST_PATH_IMAGE026
Is a region of interest
Figure 862542DEST_PATH_IMAGE011
Empirical distribution function of (2):
Figure 857043DEST_PATH_IMAGE027
wherein, the first and the second end of the pipe are connected with each other,
Figure 263753DEST_PATH_IMAGE028
is a region of interest
Figure 525845DEST_PATH_IMAGE011
Is less than or equal to
Figure 828650DEST_PATH_IMAGE029
The number of BOLD signals; obtaining the region of interest by the same method
Figure 107185DEST_PATH_IMAGE030
Empirical distribution function of
Figure 520849DEST_PATH_IMAGE031
：
Figure 138912DEST_PATH_IMAGE032
Wherein the content of the first and second substances,
Figure 409356DEST_PATH_IMAGE033
is a region of interest
Figure 614335DEST_PATH_IMAGE034
Is less than or equal to
Figure 566110DEST_PATH_IMAGE035
The number of BOLD signals of (a);
computing verification statistics for a k-s verification method
Figure 366576DEST_PATH_IMAGE036
：
Figure 745605DEST_PATH_IMAGE037
Figure 201994DEST_PATH_IMAGE038
Wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE039
is a region of interest
Figure 518312DEST_PATH_IMAGE040
Empirical distribution of BOLD signals
Figure 110968DEST_PATH_IMAGE041
Of interest region
Figure 723215DEST_PATH_IMAGE042
Empirical distribution of BOLD signals
Figure 666900DEST_PATH_IMAGE043
The maximum value of the absolute value of the difference, and finally, the region of interest is calculated
Figure 960478DEST_PATH_IMAGE040
And a region of interest
Figure 469957DEST_PATH_IMAGE042
K-s verification of BOLD signalsp-value value
Figure 426674DEST_PATH_IMAGE044
：
Figure DEST_PATH_IMAGE045
Where Z is the verification statistic and e is a natural constant.
6. The method according to claim 4, wherein the step S3 specifically comprises the following steps:
s31: respectively constructing a graph convolution neural network and a time domain convolution neural network, and forming the graph convolution neural network and the time domain convolution neural network into a graph convolution neural network-time domain convolution neural network model;
s32: taking one part of the graph network time sequence as a training set, and taking the rest part of the graph network time sequence as a verification set;
s33: training the graph convolution neural network-time domain convolution neural network model by using a training set;
s34: in the training process, the graph convolution neural network-time domain convolution neural network model is verified through a verification set, and the parameters with the highest accuracy in the verification set are used as the parameters of the graph convolution neural network-time domain convolution neural network model to complete the training of the graph convolution neural network-time domain convolution neural network model;
in the training process, the graph characteristics of the graph network time sequence are extracted by the constructed graph convolution neural network, and the graph characteristics are input into the time domain convolution neural network to obtain a classification result.
7. The method according to claim 6, wherein in step S2, the mean and standard deviation of BOLD signals of the region of interest are extracted as the features of its vertex to obtain a vertex attribute matrix; in the step S3, the graph convolution neural network comprises a plurality of convolution pooling units, a full connection layer and a softmax classifier; the convolution pooling unit comprises a graph convolution layer, a self-attention graph pooling layer and a readout layer; setting graph network time sequence of graph convolution neural network input to contain vertex attribute matrix
Figure 185551DEST_PATH_IMAGE046
And adjacency matrix
Figure 282820DEST_PATH_IMAGE047
Wherein, in the process,
Figure 850068DEST_PATH_IMAGE048
is the number of the vertices,
Figure 538538DEST_PATH_IMAGE049
is the number of vertex attributes; the operation of the graph convolution layer is specifically as follows:
Figure DEST_PATH_IMAGE050
wherein the content of the first and second substances,
Figure 17668DEST_PATH_IMAGE051
is that
Figure 653048DEST_PATH_IMAGE053
An order identity matrix;
Figure DEST_PATH_IMAGE054
is a diagonal matrix, representing the degrees of each vertex,
Figure 933857DEST_PATH_IMAGE055
，
Figure 497956DEST_PATH_IMAGE057
representative matrix
Figure 434688DEST_PATH_IMAGE059
The elements of row i and column j,
Figure DEST_PATH_IMAGE060
representative matrix
Figure 201655DEST_PATH_IMAGE061
The elements of row i and column i,
Figure DEST_PATH_IMAGE062
is the first
Figure DEST_PATH_IMAGE064
Node embedding of layer if node of layer 0 is characterized by
Figure 366664DEST_PATH_IMAGE065
Then, then
Figure DEST_PATH_IMAGE066
，
Figure 928095DEST_PATH_IMAGE067
Is a learnable weight parameter;
the self-attention-seeking pooling layer needs to obtain the degree of importance of each layer of nodes, called self-attention of the nodes, and then the self-attention of the nodes is given toAttention score weighting before rankingkIs reserved to formTop-KA node; first calculate the self-attention score
Figure DEST_PATH_IMAGE068
Wherein N is the number of nodes:
Figure 119168DEST_PATH_IMAGE069
wherein
Figure 361930DEST_PATH_IMAGE070
Is a learnable self-attention weight; selecting in a node selection mode according to the self-attention scoreTop- KThe node, which retains a part of the input graph network time sequence, specifically is:
Figure 492697DEST_PATH_IMAGE071
wherein, the first and the second end of the pipe are connected with each other,
Figure 959451DEST_PATH_IMAGE072
an index representing a reservation node;
Figure 605196DEST_PATH_IMAGE073
presentation selection
Figure 120491DEST_PATH_IMAGE074
Before ranking
Figure DEST_PATH_IMAGE075
A node of (2); pooling ratio
Figure 463354DEST_PATH_IMAGE076
Representing the percentage of nodes to be retained, before deriving the self-attention value
Figure DEST_PATH_IMAGE077
Large node index, then Masking operation is performed:
Figure 569850DEST_PATH_IMAGE078
wherein the content of the first and second substances,
Figure 968470DEST_PATH_IMAGE079
indicating node embedding with reserved index mask,
Figure 287456DEST_PATH_IMAGE080
indicating the attention score corresponding to the retention node,
Figure 956597DEST_PATH_IMAGE081
which means that the multiplication is performed in bits,
Figure 702836DEST_PATH_IMAGE082
an adjacency matrix representing the reserved nodes is shown,
Figure 526436DEST_PATH_IMAGE083
，
Figure 711430DEST_PATH_IMAGE084
a node embedding and adjacency matrix representing outputs from the attention pooling layer;
the readout layer aggregates the node features to form a representation of a fixed size to obtain a high-dimensional representation of the graph, and the output of the readout layer is specifically characterized by:
Figure 936875DEST_PATH_IMAGE085
wherein, the first and the second end of the pipe are connected with each other,Nthe number of the nodes is represented as,
Figure 854015DEST_PATH_IMAGE086
denotes the l th layeriEmbedding nodes of each node, wherein | | | represents the splicing operation of the features, and the read-out layer is actually a global average pooling layer and a global maximum pooling layer to obtain the splicing of the features;
in order to realize the reconstruction output of data, the forward propagation process of the full connection layer is as follows:
Figure 749200DEST_PATH_IMAGE087
Figure 675568DEST_PATH_IMAGE088
，
Figure DEST_PATH_IMAGE089
are respectively the first
Figure 817836DEST_PATH_IMAGE090
The learnable weight matrix and the learnable bias for the fully connected one of the layers,
Figure DEST_PATH_IMAGE091
and
Figure 968195DEST_PATH_IMAGE092
respectively represent
Figure 2272DEST_PATH_IMAGE090
And finally, obtaining a final classification result through a softmax classifier by the number of neurons of the layer full-junction layer and the number of neurons of the l +1 th layer full-junction layer:
Figure 529068DEST_PATH_IMAGE093
wherein the content of the first and second substances,
Figure 197947DEST_PATH_IMAGE094
，
Figure 988049DEST_PATH_IMAGE095
is the number of neurons of the l-th fully-connected layer,
Figure DEST_PATH_IMAGE096
is the number of categories; the graph convolution neural network obtains a plurality of graphs obtained by self-attention graph pooling layers, obtains high-dimensional feature representations of different hierarchical graphs through reading layers, adds the high-dimensional features to obtain a final high-dimensional feature representation, reconstructs the high-dimensional features through a full connection layer, uses the reconstructed features as input of a time domain convolution neural network, and finally obtains a classification result of an input graph through a softmax classifier.
8. The method for classifying medical images based on graph network time series according to claim 7, wherein in the step S3, the input layer of the time domain convolutional neural network is connected to the graph convolutional neural network full link layer, processed by a plurality of TCN layers, and output by the output layer to the softmax classifier after being processed by the expansion layer, wherein each TCN layer transforms the dimension size of its input into the same as the dimension size of its output by a one-dimensional full convolutional structure, and its forward propagation process is as follows:
Figure 335853DEST_PATH_IMAGE097
sequence data is formed by splicing output vectors of assumed full connection layers
Figure 899297DEST_PATH_IMAGE098
In which
Figure 953840DEST_PATH_IMAGE099
Is the length of the time slice or slices,
Figure 649264DEST_PATH_IMAGE100
is fully connectedThe number of neurons connected to the layer; will be provided with
Figure 953206DEST_PATH_IMAGE101
Inputting the time slice into a TCN layer, outputting and expanding the time slice into a one-dimensional vector through an expansion layer after passing through a plurality of TCN layers, and finally classifying the time slice through a softmax classifier to obtain a classification result of the time slice.
9. The method for classifying medical images based on graph network time series according to claim 8, wherein in said step S3, the TCN layer of the time domain convolution neural network is composed of causal convolution and dilation convolution, wherein:
in causal convolution, the element of the output sequence depends only on the element preceding it in the input sequence, which is one time earlier in the sequence for time series data
Figure 25067DEST_PATH_IMAGE102
Is dependent only on the next layer
Figure 934117DEST_PATH_IMAGE102
The values at and before time, namely:
Figure 98645DEST_PATH_IMAGE103
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE104
the output representing the time T of the causal convolution,
Figure 624304DEST_PATH_IMAGE105
a feature vector representing the layer l from time 1 to time T; the expansion convolution refers to performing convolution operation by using a discontinuous neuron with the same size as a convolution kernel; the expansion convolution has a expansion coefficientdThe method is used for controlling the discontinuity degree of neurons participating in convolution operation, and the calculation formula of the dilation convolution is as follows:
Figure DEST_PATH_IMAGE106
wherein the content of the first and second substances,dthe coefficient of expansion is expressed as a function of,
Figure 562173DEST_PATH_IMAGE107
which represents the size of the convolution kernel,
Figure 325730DEST_PATH_IMAGE108
weight of the i-th term of the convolution kernel wheneAt 1, the dilated convolution degenerates to the normal convolution, controlled byeSo as to enlarge the receptive field under the premise of unchanged calculated amount.
10. The method according to claim 9, wherein in the atlas network-temporal convolutional neural network model constructed in the step S3, the loss function is composed of three parts, which are node classification loss, time segment classification loss and final classification loss, and the loss function is specifically expressed as:
Figure 861491DEST_PATH_IMAGE109
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE110
is the node classification loss of the jth node at the ith time point,
Figure 671184DEST_PATH_IMAGE111
，
Figure DEST_PATH_IMAGE112
，
Figure 412744DEST_PATH_IMAGE113
is the loss of time slice classification at the ith time point,
Figure 30807DEST_PATH_IMAGE114
，
Figure 5979DEST_PATH_IMAGE115
the number of time points is the classification loss of the graph convolution neural network;
Figure 771809DEST_PATH_IMAGE116
a classification loss, hyper-parameter, for the final time-domain convolutional neural network
Figure 458006DEST_PATH_IMAGE117
The influence of the classification loss of the control node, the classification loss of the time segment and the final classification loss respectively has
Figure 461734DEST_PATH_IMAGE118
And is
Figure 637500DEST_PATH_IMAGE119
(ii) a All classification loss functions use cross-entropy loss functions, which are specifically expressed as:
Figure 828310DEST_PATH_IMAGE120
Figure 82312DEST_PATH_IMAGE121
represents the sample ofjThe true probability value of the seed class,
Figure DEST_PATH_IMAGE122
representing the sample obtained from the modeljPredicted probability values for the species classes.
CN202210814372.2A 2022-07-12 2022-07-12 Medical image classification method based on graph network time sequence Active CN115222688B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210814372.2A CN115222688B (en) 2022-07-12 2022-07-12 Medical image classification method based on graph network time sequence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210814372.2A CN115222688B (en) 2022-07-12 2022-07-12 Medical image classification method based on graph network time sequence

Publications (2)

Publication Number Publication Date
CN115222688A true CN115222688A (en) 2022-10-21
CN115222688B CN115222688B (en) 2023-01-10

Family

ID=83612470

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210814372.2A Active CN115222688B (en) 2022-07-12 2022-07-12 Medical image classification method based on graph network time sequence

Country Status (1)

Country Link
CN (1) CN115222688B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115909016A (en) * 2023-03-10 2023-04-04 同心智医科技(北京)有限公司 System, method, electronic device, and medium for analyzing fMRI image based on GCN
CN116030308A (en) * 2023-02-17 2023-04-28 齐鲁工业大学(山东省科学院) Multi-mode medical image classification method and system based on graph convolution neural network
CN117435995A (en) * 2023-12-20 2024-01-23 福建理工大学 Biological medicine classification method based on residual map network

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102855491A (en) * 2012-07-26 2013-01-02 中国科学院自动化研究所 Brain function magnetic resonance image classification method based on network centrality
CN108364006A (en) * 2018-01-17 2018-08-03 超凡影像科技股份有限公司 Medical Images Classification device and its construction method based on multi-mode deep learning
CN110720906A (en) * 2019-09-25 2020-01-24 上海联影智能医疗科技有限公司 Brain image processing method, computer device, and readable storage medium
CN111667459A (en) * 2020-04-30 2020-09-15 杭州深睿博联科技有限公司 Medical sign detection method, system, terminal and storage medium based on 3D variable convolution and time sequence feature fusion
WO2021001238A1 (en) * 2019-07-01 2021-01-07 Koninklijke Philips N.V. Fmri task settings with machine learning
CN112766332A (en) * 2021-01-08 2021-05-07 广东中科天机医疗装备有限公司 Medical image detection model training method, medical image detection method and device
CN113080847A (en) * 2021-03-17 2021-07-09 天津大学 Device for diagnosing mild cognitive impairment based on bidirectional long-short term memory model of graph
CN113592836A (en) * 2021-08-05 2021-11-02 东南大学 Deep multi-modal graph convolution brain graph classification method
CN114241240A (en) * 2021-12-15 2022-03-25 中国科学院深圳先进技术研究院 Method and device for classifying brain images, electronic equipment and storage medium
US20220122250A1 (en) * 2020-10-19 2022-04-21 Northwestern University Brain feature prediction using geometric deep learning on graph representations of medical image data

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102855491A (en) * 2012-07-26 2013-01-02 中国科学院自动化研究所 Brain function magnetic resonance image classification method based on network centrality
CN108364006A (en) * 2018-01-17 2018-08-03 超凡影像科技股份有限公司 Medical Images Classification device and its construction method based on multi-mode deep learning
WO2021001238A1 (en) * 2019-07-01 2021-01-07 Koninklijke Philips N.V. Fmri task settings with machine learning
CN110720906A (en) * 2019-09-25 2020-01-24 上海联影智能医疗科技有限公司 Brain image processing method, computer device, and readable storage medium
CN111667459A (en) * 2020-04-30 2020-09-15 杭州深睿博联科技有限公司 Medical sign detection method, system, terminal and storage medium based on 3D variable convolution and time sequence feature fusion
US20220122250A1 (en) * 2020-10-19 2022-04-21 Northwestern University Brain feature prediction using geometric deep learning on graph representations of medical image data
CN112766332A (en) * 2021-01-08 2021-05-07 广东中科天机医疗装备有限公司 Medical image detection model training method, medical image detection method and device
CN113080847A (en) * 2021-03-17 2021-07-09 天津大学 Device for diagnosing mild cognitive impairment based on bidirectional long-short term memory model of graph
CN113592836A (en) * 2021-08-05 2021-11-02 东南大学 Deep multi-modal graph convolution brain graph classification method
CN114241240A (en) * 2021-12-15 2022-03-25 中国科学院深圳先进技术研究院 Method and device for classifying brain images, electronic equipment and storage medium

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
AN ZENG等: "Discovery of Genetic Biomarkers for Alzheimer’s Disease Using Adaptive Convolutional Neural Networks Ensemble and Genome‑Wide Association Studies", 《INTERDISCIPLINARY SCIENCES: COMPUTATIONAL LIFE SCIENCES》 *
XIAOXIAO LI等: "BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis", 《MEDICAL IMAGE ANALYSIS》 *
唐朝生等: "医学图像深度学习技术:从卷积到图卷积的发展", 《中国图象图形学报》 *
曾安等: "基于3D卷积神经网络-感兴区域的阿尔茨海默症辅助诊断模型", 《生物医学工程研究》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116030308A (en) * 2023-02-17 2023-04-28 齐鲁工业大学(山东省科学院) Multi-mode medical image classification method and system based on graph convolution neural network
CN115909016A (en) * 2023-03-10 2023-04-04 同心智医科技(北京)有限公司 System, method, electronic device, and medium for analyzing fMRI image based on GCN
CN117435995A (en) * 2023-12-20 2024-01-23 福建理工大学 Biological medicine classification method based on residual map network
CN117435995B (en) * 2023-12-20 2024-03-19 福建理工大学 Biological medicine classification method based on residual map network

Also Published As

Publication number Publication date
CN115222688B (en) 2023-01-10

Similar Documents

Publication Publication Date Title
CN115222688B (en) Medical image classification method based on graph network time sequence
Niu et al. Multichannel deep attention neural networks for the classification of autism spectrum disorder using neuroimaging and personal characteristic data
CN113040715B (en) Human brain function network classification method based on convolutional neural network
Cui et al. Automatic sleep stage classification based on convolutional neural network and fine-grained segments
CN109242860B (en) Brain tumor image segmentation method based on deep learning and weight space integration
CN109345538A (en) A kind of Segmentation Method of Retinal Blood Vessels based on convolutional neural networks
Hong et al. Brain age prediction of children using routine brain MR images via deep learning
CN112037179B (en) Method, system and equipment for generating brain disease diagnosis model
Liu et al. An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders
Torres et al. Evaluation of interpretability for deep learning algorithms in EEG emotion recognition: A case study in autism
CN110148145A (en) A kind of image object area extracting method and application merging boundary information
Wang et al. Classification of structural MRI images in Adhd using 3D fractal dimension complexity map
Bayram et al. Deep learning methods for autism spectrum disorder diagnosis based on fMRI images
Toğaçar et al. Use of dominant activations obtained by processing OCT images with the CNNs and slime mold method in retinal disease detection
CN115272295A (en) Dynamic brain function network analysis method and system based on time domain-space domain combined state
Qiang et al. A deep learning method for autism spectrum disorder identification based on interactions of hierarchical brain networks
CN112036298A (en) Cell detection method based on double-segment block convolutional neural network
CN115474939A (en) Autism spectrum disorder recognition model based on deep expansion neural network
Kong et al. Data enhancement based on M2-Unet for liver segmentation in Computed Tomography
Jung et al. Inter-regional high-level relation learning from functional connectivity via self-supervision
Seshadri Ramana et al. Deep convolution neural networks learned image classification for early cancer detection using lightweight
Mareeswari et al. A survey: Early detection of Alzheimer’s disease using different techniques
CN115937590A (en) Skin disease image classification method with CNN and Transformer fused in parallel
Bonà et al. Vocal tract segmentation of dynamic speech MRI images based on deep learning for neurodegenerative disease application
Zhou et al. Spatial-Temporal Graph Convolutional Network for Insomnia Classification via Brain Functional Connectivity Imaging of rs-fMRI

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant