WO2023108712A1 - 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型 - Google Patents

结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型 Download PDF

Info

Publication number
WO2023108712A1
WO2023108712A1 PCT/CN2021/140017 CN2021140017W WO2023108712A1 WO 2023108712 A1 WO2023108712 A1 WO 2023108712A1 CN 2021140017 W CN2021140017 W CN 2021140017W WO 2023108712 A1 WO2023108712 A1 WO 2023108712A1
Authority
WO
WIPO (PCT)
Prior art keywords
brain
function
network
functional
structural
Prior art date
Application number
PCT/CN2021/140017
Other languages
English (en)
French (fr)
Inventor
王书强
丁陈
Original Assignee
深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳先进技术研究院 filed Critical 深圳先进技术研究院
Publication of WO2023108712A1 publication Critical patent/WO2023108712A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders

Definitions

  • the invention relates to the field of brain network models, in particular to a method for constructing a structure-function brain network bidirectional mapping model, a brain network bidirectional mapping model and a computer-readable storage medium.
  • Alzheimer's disease is a common irreversible dementia, and its incidence rate will increase significantly with age, which makes the research on Alzheimer's disease imminent.
  • the level of human brain exploration has been increasing year by year. Effective use of these imaging technologies can assist doctors in diagnosing and understanding the causes of diseases, and provide a favorable basis for subsequent treatment.
  • the object of the present invention is to provide a method for constructing a structure-function brain network bidirectional mapping model, a brain network bidirectional mapping model, and a computer-readable storage medium, which have the advantages of making the constructed structure-function brain network bidirectional mapping model helpful for revealing brain structure and function characteristics of complex relationships.
  • the specific method includes:
  • the feature preprocessing module is used to preprocess the brain structure modal image based on the brain structure modal image and brain function modal image of a specified type of patient to obtain the corresponding node features and edge Information brain structure network, preprocessing the brain function modal image to obtain a brain function network including corresponding node features and edge information;
  • the structural feature extraction module is used to extract features from the brain structure network based on the cyclic graph convolution algorithm, and obtain the structural features of the brain;
  • the functional feature extraction module is used to The graph convolution algorithm extracts the features of the brain functional network to obtain the functional characteristics of the brain;
  • the structural classifier module is used to diagnose the patient's condition based on the structural features of the brain to obtain a disease classification result based on the structural features, so as to realize the training of the structural feature extraction module;
  • the function classifier diagnoses the condition of the patient based on the functional characteristics of the brain to obtain a condition classification result based on the functional characteristics, so as to realize the training of the functional characteristic extraction module;
  • a structure-function bidirectional mapping network based on the structural and functional characteristics of the brain, using a structure generator module, a function generator module, a structure discriminator module and a function discriminator module to bidirectionally map the brain structure network and the brain function network ;
  • the constructed structural feature extraction module and functional feature extraction module, structural classifier module and functional classifier module and structure-function bidirectional mapping network are trained and studied .
  • the extraction of multimodal features and the collaborative training of brain network two-way mapping are realized, which is conducive to the successful learning of the potential feature representation of brain structure and function in the constructed brain network two-way mapping model. While assisting in the prediction of disease categories, it can help reveal the relationship between brain structure and function.
  • the specific method for constructing a structure-function bidirectional mapping network includes,
  • the structure generator Based on the structural features of the brain and the functional features of the brain, the structure generator generates the brain structural network G1 into the brain functional network G2', and the function generator generates the brain functional network G2 into the brain structural network G1';
  • the function discriminator judges the gap between the generated brain function network G2' and the real brain function network G2, and the structure discriminator judges the gap between the brain structure network G1' and the real brain structure network G1;
  • the function generator generates the brain structure network G1" from the brain function network G2', and the structure generator generates the brain function network G2" from the brain structure network G1';
  • the function discriminator judges the gap between the generated brain function network G2" and the real brain function network G2, and the structure discriminator judges the gap between the generated brain structure network G1" and the real brain structure network G1.
  • S101 and S102 includes,
  • Brain structure data fuzzification processing, brain structure real connection matrix and brain function real connection matrix are subjected to multiple convolution and activation function processing, so that the brain structure real connection matrix contains part of the information in the brain function real connection matrix, thus obtaining The real connection matrix of the brain structure after fuzzification; the real connection matrix of the brain structure and the generated connection matrix of the brain structure generated by the function generator are given to the structure discriminator;
  • Brain function data is fuzzy processed, and the real connection matrix of brain function and the real connection matrix of brain structure are subjected to multiple convolution and activation function processing, so that the real connection matrix of brain function contains part of the information in the real connection matrix of brain structure, thus obtaining The fuzzified real connection matrix of brain function; the fuzzy real connection matrix of brain function and the generated connection matrix of brain function generated by the structure generator are given to the function discriminator.
  • the activation function uses a ReLU activation function.
  • the brain structure modality images include DTI and sMRI modality images; the brain function modality images include fMRI and PET modality images.
  • the method for preprocessing the modal image of brain structure includes: preprocessing the DTI modal image of the brain to obtain fiber bundles of white matter, and mapping the fiber bundle image to a brain region division template, Divide the cerebral cortex and subcutaneous tissue into M brain regions, and combine these M brain regions to complete the construction of the DTI brain structure network; preprocess the sMRI modal images of the brain, and map the sMRI brain modal images to the division of brain regions Template, get the brain gray matter density of M brain regions, combine the brain gray matter density of these M brain regions to complete the construction of the sMRI brain structure network, and the density of fiber bundles between brain regions is the structural connection feature;
  • the method for preprocessing the brain function modal image includes: preprocessing the fMRI modal image of the brain, mapping the fMRI modal image image to a brain region division template to obtain a time series of M brain regions, and combining this The time series of M brain regions completes the construction of the fMRI brain function network; the brain PET is preprocessed, and the PET modality image image is mapped to the brain region division template to obtain markers in the M brain regions, including proteins, glucose, etc. Substance content, based on which the construction of the PET brain network is completed.
  • the node information N1 and edge connection information E1 of the DTI brain structure network and the node information N2 and edge connection information E2 of the sMRI brain structure network are regarded as a group of brain structures, and the node information N3 of the fMRI brain function network And the edge connection information E3 and the node information N4 and edge information E4 of the PET brain function network are used as a group of brain functions, and then two feature extraction modules are used to extract the structural features (N1', N3') and functional features of the brain in one-to-one correspondence (N2', N4').
  • a structure-function brain network two-way mapping model includes:
  • the feature preprocessing module based on the brain structure modality images and brain function modality images of patients of a specified type, preprocesses the brain structure modality images to obtain a brain structure network including corresponding node features and edge information, and performs the processing on the brain structure modality images.
  • Brain function modal images are preprocessed to obtain a brain function network including corresponding node features and side information;
  • the structural feature extraction module based on the circular graph convolution algorithm, extracts the features of the brain structural network to obtain the structural features of the brain;
  • the functional feature extraction module based on the circular graph convolution algorithm, extracts the features of the brain functional network to obtain the functional features of the brain;
  • Structural classifier module based on the structural features of the brain, diagnose the condition of the patient to obtain the result of disease classification based on the structural features, so as to realize the training of the structural feature extraction module;
  • Functional classifier module based on the functional features of the brain, diagnoses the patient's condition to obtain a condition classification result based on the functional features, so as to realize the training of the functional feature extraction module;
  • Structure-function two-way mapping network based on the structural and functional characteristics of the brain, using the included structure generator module, function generator module, structure discriminator module and function discriminator module to carry out bidirectional brain structure network and brain function network map.
  • the structure generator generates the brain structure network G1 into the brain function network G2', and the function generator generates the brain function network G2 into the brain structure network G1' Secondly, the function discriminator judges the gap between the generated brain function network G2' and the real brain function network G2, and the structure discriminator judges the gap between the brain structure network G1' and the real brain structure network G1; again, the The function generator generates the brain structure network G1" from the brain function network G2', and the structure generator generates the brain function network G2" from the brain structure network G1'; finally, the function discriminator judges the generated brain function network G2" and the real brain function network G2, the structure discriminator judges the gap between the generated brain structure network G1" and the real brain structure network G1.
  • the structure-function bidirectional mapping network also includes a structure balancer and a function balancer;
  • the structural balancer includes a convolution layer and an activation function, and performs multiple convolution and activation function processing on the real connection matrix of the brain structure and the real connection matrix of the brain function, so that the real connection matrix of the brain structure includes the real connection matrix of the brain function. Part of the information, so as to obtain the real connection matrix of the fuzzy brain structure;
  • the functional balancer includes a convolutional layer and an activation function, and performs multiple convolution and activation function processing on the real connection matrix of brain function and the real connection matrix of brain structure, so that the real connection matrix of brain function includes the real connection matrix of brain structure. Part of the information, so as to obtain the real connection matrix of brain function after fuzzification.
  • a computer program that can be loaded by a processor to execute any one of the above methods is stored.
  • Fig. 1 is a schematic diagram of the structure-function two-way mapping brain network model principle of one embodiment of the present invention
  • Fig. 2 is a schematic diagram of the network structure of non-Euclidean spatial feature extraction according to one embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a balancer according to one embodiment of the present invention.
  • a method for constructing a structure-function two-way mapping brain network model provided by the embodiment of the application of the present invention, the specific method includes,
  • the feature preprocessing module is used to preprocess the brain structure modal image based on the brain structure modal image and brain function modal image of the specified type of patients to obtain the brain structure including corresponding node features and edge information.
  • Network preprocessing the brain function modal image to obtain a brain function network including corresponding node features and edge information;
  • the structural feature extraction module is used to extract the features of the brain structure network based on the cyclic graph convolution algorithm to obtain the structural features of the brain;
  • the functional feature extraction module is used for the cyclic graph convolution algorithm. Extract the features of the brain functional network to obtain the functional characteristics of the brain;
  • the structural classifier module is used to diagnose the patient's condition based on the structural features of the brain to obtain a disease classification result based on the structural features, so as to realize the training of the structural feature extraction module;
  • functional classification The device diagnoses the condition of the patient based on the functional characteristics of the brain to obtain a condition classification result based on the functional characteristics, so as to realize the training of the functional characteristic extraction module;
  • Construct a structure-function bidirectional mapping network based on the structural and functional characteristics of the brain, use the structure generator module, function generator module, structure discriminator module and function discriminator module to perform bidirectional mapping on the brain structure network and brain function network to generate Structure-function bidirectional mapping network;
  • the constructed structural feature extraction module and functional feature extraction module, structural classifier module and functional classifier module and structure-function bidirectional mapping network are trained and studied , to obtain a structure-function bidirectional mapping brain network model based on multimodal data.
  • this application is based on the cyclic graph convolution algorithm, which not only considers the relationship within a single modality, but also considers the relationship between structure and function, so that all information within the modality can be fully utilized.
  • many articles at this stage use multimodal data, they often only discuss whether the structure can generate function, or whether the function can generate structure.
  • the model can simultaneously consider the mutual mapping relationship between structure and function.
  • the extraction of multi-modal features and the collaborative training of the bidirectional mapping network are conducive to the successful learning of the potential feature representation of brain structure and function by the constructed brain network model, while assisting in the prediction of disease categories, Can help uncover the relationship between brain structure and function. And based on this, it can be judged whether the brain disease is caused by abnormal brain connections in brain structure or function, or abnormal brain activity caused by inconsistency between structure and function.
  • Structure is the basis of function
  • function is the representation of structure
  • the relationship between structure and function plays a crucial role in revealing the organization principle of the brain.
  • the application of the present invention uses modal image data to construct brain structure and brain function network, and conducts auxiliary diagnosis and pathogenesis identification of brain diseases according to the relationship between the constructed brain structure and brain function network.
  • Alzheimer's disease can be divided into five stages, can be divided into following 5 classes according to the order of disease progression: normal person (NC ), salience memory concern (SMC), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and Alzheimer's disease (AD).
  • N subjects in each of the 5 stages are selected as subjects, and brain structure modal images and brain function modal images are collected for all 5*N subjects.
  • the multimodal recurrent graph convolutional network involved in the specific embodiments of the present application considers four modalities of each subject sample: sMRI modality image, DTI modality image, fMRI modality image, PET modality image.
  • DTI and sMRI modality images belong to brain structure modality images
  • fMRI and PET modality images belong to brain function modality images
  • four modality images of fMRI, DTI, sMRI and PET are collected as original data sets, and the original The data set is transformed into data.
  • the brain structure network and the brain function network are obtained by preprocessing respectively.
  • the method for preprocessing the brain structure modal image includes: preprocessing the DTI modal image of the brain to obtain the fiber bundle of the white matter, and mapping the fiber bundle image to the brain region division template, and dividing the cerebral cortex and subcutaneous tissue Divide into M brain regions, combine the M brain regions to complete the construction of the DTI brain structure network; preprocess the sMRI modal images of the brain, map the sMRI brain modal images to the brain region division template, and obtain M brain regions.
  • the density of brain gray matter in M brain regions is combined with the density of brain gray matter in these M brain regions to complete the construction of the sMRI brain structure network, and the density of fiber bundles between brain regions is the structural connection feature;
  • the method of preprocessing the brain function modal image includes preprocessing the fMRI modal image of the brain, mapping the fMRI modal image image to the brain region division template, obtaining the time series of M brain regions, and combining the M
  • the time series of brain regions completes the construction of the fMRI brain function network; preprocesses the brain PET, maps the PET modal image image to the brain region division template, and obtains the contents of substances including protein and glucose containing markers in M brain regions Based on this, the construction of the PET brain network is completed.
  • PANDA software is used to preprocess DTI modal images, and the cerebral cortex and subcutaneous tissue are divided into 90 brain regions based on the AAL template;
  • GRETNA software is used to preprocess fMRI modal images.
  • the fMRI image was mapped to the AAL template to obtain the time series of 90 brain regions;
  • CAT12 software was used to preprocess the sMRI modality image, and the sMRI modality image was mapped to the AAL template to obtain the midbrain gray matter density of 90 brain regions;
  • NetPET software was used The PET modality images are preprocessed, combined with the ALL template to obtain the contents of markers including protein, glucose and other substances in the M brain regions.
  • the scope of use of the preprocessing software in this application is not limited to PANDA, GRETNA, CAT12 and NetPET software, as long as the preprocessing process can be realized to obtain the corresponding brain network, it is not limited to the 90 brain regions of the ALL template, and other templates and Other quantitative brain regions, for example, 90 brain regions of the ALL1 template, 116 brain regions of the ALL2 template, or part of the 90 brain regions of the ALL1 template can also be used in some specific cases. region or part of the 116 brain regions of the ALL2 template.
  • the brain function network and brain structure network obtained after data conversion from the original data set include corresponding node information and edge information.
  • the node information N1 and edge connection information E1 of the DTI modal image were extracted using PANDA software; using Gretna The software extracts the time series of fMRI modal images, and uses the Pearson correlation coefficient to construct the node information N2 and edge connection information E2 of the fMRI modal images; uses CAT12 software to extract the gray matter density in the sMRI midbrain area, and constructs the nodes Information N3 and side information E3; use NetPET software to construct the node information N4 and side information E4 of the brain.
  • the features of the nodes are weighted and aggregated through the connection relationship between the nodes, and the dimension is reduced to a low-dimensional space, so as to realize the extraction of the features of the brain structure network and brain function network separately, and obtain the brain structural and functional features.
  • the non-Euclidean feature extraction network is used to transform different modal data (such as Fourier transform, Z domain transform, etc.), and the data is mapped to the corresponding potential space (Fourier space, Z domain space, etc.), and then convolve several kinds of data. Due to the existence of the circular convolution mechanism, the features in different modalities will aggregate information, and then reverse the convolution results. Transforming back to the original space, the features obtained in this way utilize the node information and edge information under all modal data. The process is described in detail below.
  • the brain network contains two parts, namely the node feature matrix and the adjacency matrix.
  • n 1 and n 2 both represent the length of a certain dimension
  • graph is a data structure in the field of mathematics, and brain network refers to the name of the brain when it is represented by graph data;
  • the degree matrix D only has values on the diagonal, which is the degree of the corresponding node, and the rest are 0;
  • D (i) represents D(:,:, i); I represents a unit tensor, and i represents a certain dimension;
  • a ⁇ R n1 ⁇ n2 ⁇ 3 and F ⁇ R n2 ⁇ n4 ⁇ 3 , and in U is the corresponding transform form (Fourier transform, Z transform, etc.), and U H is the inverse transform.
  • N1, E1 and N3, E3 are regarded as a group of brain structures
  • N2, E2 and N4, E4 are regarded as a group of brain functions
  • two feature extraction modules structural feature extraction module and functional feature extraction module
  • extraction module two feature extraction modules
  • the CNN algorithm can complete the feature extraction well, it gathers adjacent N*N nodes in space, and in time It collects the information of nodes in a certain period of time, but it ignores such a problem - adjacent nodes may have influence, but it is not necessarily the most important influence, but the degree of mutual influence between pixels far apart higher.
  • a simple and effective classifier which mainly includes a layer of cyclic graph Convolution module and MLP (Multilayer Perceptron) module, firstly give the node information and edge connection information to the cycle graph convolution module, and then give the further extracted information (90*1) to the MLP module, MLP can be used comprehensively
  • MLP Multilayer Perceptron
  • the four modules of 90*d (d is the length of a certain dimension of the feature matrix; the length is 90, indicating the number of brain regions; the width is d, indicating the length of the feature vector of a brain region) obtained in the feature extraction process
  • the state brain network data ((N1', E1), (N3', E3), (N2', E2) and (N4', E4)) use a layer of cyclic graph convolution to fuse the two feature information.
  • the softmax layer can calculate the probability of the output of the five categories (the above Alzheimer's disease is divided into the following five categories according to the order of deterioration of the disease).
  • the present invention chooses to use the generative confrontation network with two-way mapping, based on the structural and functional characteristics of the brain, using the structure generator module, function generator module, structure discriminator module and function discriminator
  • the module performs two-way mapping of brain structure network and brain function network.
  • the specific methods for constructing structure-function bidirectional mapping network include,
  • the structure generator Based on the structural characteristics and functional characteristics of the brain, the structure generator generates the brain structural network G1 into the brain functional network G2', and the function generator generates the brain functional network G2 into the brain structural network G1';
  • the function discriminator judges the gap between the generated brain function network G2' and the real brain function network G2, and the structure discriminator judges the gap between the brain structure network G1' and the real brain structure network G1;
  • the function generator generates the brain structure network G1" from the brain function network G2', and the structure generator generates the brain function network G2" from the brain structure network G1';
  • the function discriminator judges the gap between the generated brain function network G2" and the real brain function network G2, and the structure discriminator judges the gap between the generated brain structure network G1" and the real brain structure network G1.
  • the structure of the structural balancer shown in Figure 3 includes convolutional layers and activation functions. Based on the structure of the structural balancer, according to the performance difference between the structure generator and the structure discriminator in the previous cycle iteration process, the real connection matrix of the brain structure (i.e., the side information of the brain structure network) and the real connection matrix of the brain function (i.e.
  • the side information of the brain function network is processed by multiple convolutions and activation functions, so that the real connection matrix of the brain structure contains part of the information in the real connection matrix of the brain function, so as to obtain the fuzzy real connection matrix of the brain structure;
  • the real connection matrix of the brain structure after fuzzification and the generated connection matrix of the brain structure generated by the function generator are given to the structure discriminator, thereby reducing the gap between the generated results and the real results, thereby reducing the learning difficulty of the structure generator; at the same time Reducing the gap between the generated results and the real results can also slow down the rate at which the structure classifier fully learns the distribution of the real result data, so that there is a balance between the performance of the structure generator and the structure discriminator.
  • the structure of the functional balancer is the same as that of the structural balancer, including convolutional layers and activation functions.
  • the real connection matrix of brain function and the real connection matrix of brain structure are processed by multiple convolutions and activation functions, so that
  • the real connection matrix of brain function contains part of the information in the real connection matrix of brain structure, so as to obtain the real connection matrix of brain function after fuzzification; the brain function generated by the fuzzy real connection matrix of brain function and the structure generator
  • the connection matrix is given to the function discriminator, thereby reducing the gap between the generated results and the real results, thereby reducing the learning difficulty of the function generator; at the same time, reducing the gap between the generated results and the real results can also slow down the completeness of the function classifier.
  • the rate at which the true result data distribution is learned so that the performance of the feature generator and feature discriminator tends to be balanced.
  • the convolution kernel of the convolutional layer is 3*3, and the step size is 1.
  • the activation function can use the ReLU activation function to pass through 4 sets of convolutional layers and activation functions. Parameter adjustment needs to be obtained through experiments. You can control the remaining parameters unchanged, and then modify the current parameters. After multiple experiments, find the optimal number of groups of convolutional layers and activation functions among these parameters.
  • the model is trained.
  • the data of four modalities fMRI, DTI, sMRI, PET
  • sMRI and DTI are used as the brain structure network to extract the structural feature representation of the brain
  • fMRI and PET as a brain functional network
  • extracts the functional feature representation of the brain and gives the two feature representations one-to-one correspondence to the structural classifier module and the functional classifier module, and outputs the category of disease prediction.
  • K is the number of categories
  • p i is the output of the structural classifier module (or functional classifier module), that is, the probability of category i , this output value is calculated using softmax.
  • the loss function of the structural classifier module (or functional classifier module) is mainly used for backpropagation learning training for the evaluation of the structural feature extraction module (or functional feature extraction module).
  • the loss function algorithm of the structural classifier and the loss function algorithm of the functional classifier are the same, according to the loss function algorithm of the structural classifier given above, the function after replacing the functional classifier in parentheses with the structural classifier can be obtained
  • the loss function algorithm of the classifier, the contents in parentheses below have the same meaning.
  • X is the input of the model
  • Y is the generated result of the model
  • D is the abbreviation of the structure discriminator module (or function discriminator module)
  • ⁇ is a hyperparameter, which can be determined according to the structure generator module (or function generator module) and Adjust the poor performance of the structural discriminator module (or functional discriminator module),
  • the loss function of the structure generator module (or function generator module) in the bidirectional mapping network plus the loss function of the structure discriminator module (or function discriminator module) in the bidirectional mapping network This loss function is mainly used for back-propagation learning training for the evaluation of the structural feature extraction module (or functional feature extraction module), structural generator module (or functional generator module) and structural discriminator module (or functional discriminator module).
  • the difference between the results generated by the structure generator module (or function generator module) in the bidirectional mapping network and the real results This loss function is mainly used for backpropagation learning training for the evaluation of the structure feature extraction module, structure generator module and feature generator module.
  • Classification results of the structural classifier module (or functional classifier module) in the bidirectional mapping network It will be used as one of the evaluation criteria for the performance of the structure generator module (or function generator module); the loss function is mainly used for direction propagation learning training for the evaluation of the structure (or function) feature extraction module and the structure (or function) classifier.
  • the training samples are divided into 90% for training the model and 10% for testing the performance of the trained model.
  • the training process of the model can be qualitatively analyzed as follows: In the data-driven mode, with the continuous optimization of the bidirectional map network and the cyclic graph convolution, the structure discriminator module and the function discriminator module need to continuously update parameters to identify the structure generator module and The pseudo data distribution generated by the function generator module; the structure generator module and the function generator module need to continuously update the parameters through the results of the structure discriminator module and the function discriminator module, and accordingly make the generated distribution close to the real distribution; the structure classifier Modules and features Classifier modules are optimized so that their predicted data distributions are well discriminative between different classes.
  • the structural classifier module and functional classifier module can help doctors make auxiliary diagnosis.
  • the two-way mapping network can realize the mutual mapping between the structural brain network and the functional brain network, and based on this mechanism, find out the brain disease. , to explore whether there is a unified relationship between structure and function.
  • a structure-function brain network two-way mapping model includes:
  • the feature preprocessing module based on the brain structure modality images and brain function modality images of specified types of patients, preprocesses the brain structure modality images to obtain a brain structure network including corresponding node features and side information, and brain function modality images
  • the image is preprocessed to obtain a brain function network including corresponding node features and side information;
  • the structural feature extraction module based on the circular graph convolution algorithm, extracts the features of the brain structural network to obtain the structural features of the brain;
  • the functional feature extraction module based on the circular graph convolution algorithm, extracts the features of the brain functional network to obtain the functional features of the brain;
  • Structural classifier module based on the structural features of the brain, diagnose the condition of the patient to obtain the result of disease classification based on the structural features, so as to realize the training of the structural feature extraction module;
  • Functional classifier module based on the functional features of the brain, diagnoses the patient's condition to obtain a condition classification result based on the functional features, so as to realize the training of the functional feature extraction module;
  • Structure-function two-way mapping network based on the structural and functional characteristics of the brain, using the included structure generator module, function generator module, structure discriminator module and function discriminator module to carry out bidirectional brain structure network and brain function network map.
  • the corresponding abnormal brain connections were found. Based on these abnormal brain connections, construct a structural classifier and a functional classifier. If the final classification result is similar to the above results, it can be concluded that these brain connections do play an important role in the disease. Delete individual brain connections in turn to ensure that the remaining variables are the same, and conduct experiments again to prove the relationship between single brain connections and the disease.
  • a multi-modal image fusion auxiliary diagnosis system can be provided.
  • the first is optimal in the confrontation training.
  • DTI is mapped to the features of the latent space
  • the second cycle graph convolution module will learn the features mapped to the latent space by fMRI;
  • the classifier has learned the optimal parameters of multi-modal fusion, which can be relatively accurately input from the four
  • the modality data predicts the corresponding disease category and the corresponding brain connectivity model.
  • the trained generator module and classifier module are extracted and migrated to form an end-to-end multimodal image fusion auxiliary diagnosis system.
  • the generator module has learned the relationship between structure and function well, can map structure and function, and optimize the original brain atlas, which can not only improve the classification accuracy, but also assist researchers in exploring brain Deep coordination mechanisms between structure and function.
  • Alzheimer's disease is only a patient type selected in the embodiment of the present application, and the four modal images of DTI, fMRI, sMRI, and PET are used as examples to illustrate, but the scope of application of this application is not limited. It is limited to Alzheimer's disease and fMRI-DTI-sMRI-PET images, and it can also be other types of patients. As a patient type, 5 stages are selected here. During the test verification process, one of them can also be selected Several stages or not limited to these 5 stages.
  • the collected structural brain modal images are not limited to DTI and sMRI modal images, and can be one or several of them, or other modal images, as long as they belong to structural brain modal images; for
  • the collected functional brain modal images are not limited to fMRI and PET modal images, and can be one or more of them, or other modal images, as long as they belong to functional brain modal images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

本发明涉及一种结构-功能脑网络双向映射构建方法及脑网络双向映射模型，方法包括，构建特征预处理模块，得到脑结构网络和脑功能网络；构建结构特征提取模块和功能特征提取模块，得到大脑的结构特征和功能特征；构建结构分类器模块和功能分类器模块，得到基于结构特征的病情分类结果和基于功能特征的病情分类结果；构建结构-功能双向映射网络，对脑结构网络和脑功能网络进行双向映射；利用预处理后的所述脑结构网络和脑功能网络的数据集，对构建的结构特征提取模块和功能特征提取模块、结构分类器模块和功能分类器模块及结构-功能双向映射网络进行训练学习。使得构建的脑网络模型有助于揭示大脑结构和功能之间复杂关系。

Description

结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型 技术领域
本发明涉及脑网络模型领域，特别是涉及一种结构-功能脑网络双向映射模型构建方法、脑网络双向映射模型及计算机可读存储介质。
背景技术
阿尔茨海默症是一种常见的不可逆性痴呆症，其发病概率会随着年龄的增长而大幅提升，这使得对阿尔茨海默症的研究迫在眉睫。近年随着科技水平的不断发展，人类对大脑的探查水平逐年提升，有效利用这些成像技术可以辅助医生进行诊断和了解病情的产生原因，为后续治疗提供有利依据。
现阶段，一些学者利用脑结构数据对阿尔茨海默症进行辅助诊断，取得了较好的成果。目前利用图结构研究脑部疾病的研究中，某些技术中，利用使用多个图卷积模型融合了不同方向下的DTI特征，实现了单模态内的融合；某些技术中，不仅仅使用了结构和功能核磁共振成像数据，还增加了非影像数据，提高了MCI疾病的准确率，证明了先验知识在疾病诊断中起到重要作用；某些技术中，使用DTI和fMRI两种模态的数据构建了分类监督模型。然而，现阶段方案中，或只研究了结构或功能，忽略了结构和功能之间的相辅相成的关系，或仅仅只是考虑了结构和功能间的某一单向关系，而忽略了可能存在的复杂关系。
发明内容
本发明目是提供一种结构-功能脑网络双向映射模型构建方法、脑网络双向映射模型及计算机可读存储介质，具有使得构建的结构-功能脑网络双向映射模型有助于揭示大脑结构和功能之间复杂关系的特点。
根据本发明提供的一种结构-功能脑网络双向映射模型构建方法，具体方法包括，
构建特征预处理模块，所述特征预处理模块用于基于指定类型病患的脑结构模态影像和脑功能模态影像，对所述脑结构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对所述脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
构建结构特征提取模块和功能特征提取模块，所述结构特征提取模块用于基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；所述功能特征提取模块用于基于循环图卷积算法对脑功能网络进行特征提取，得到大脑的功能特征；
构建结构分类器模块和功能分类器模块，所述结构分类器模块用于基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；所述功能分类器基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
构建结构-功能双向映射网络，基于所述大脑的结构特征和功能特征，利用结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进行双向映射；
利用预处理后的所述脑结构网络和脑功能网络的数据集，对构建的结构特征提取模块和功能特征提取模块、结构分类器模块和功能分类器模块及结构-功能双向映射网络进行训练学习。
通过采用上述技术方案，在数据驱动模式下，实现多模态特征的提取和脑网络双向映射协同训练，有利于构建的脑网络双向映射模型成功地学习到大脑结构和功能的潜在特征表示，在辅助预测疾病类别的同时，可以帮助揭露大脑结构和功能之间的关系。
可选地，所述构建结构-功能双向映射网络的具体方法包括，
S101、基于所述大脑的结构特征和大脑的功能特征，结构生成器将脑结构网络G1生成脑功能网络G2’，功能生成器将脑功能网络G2生成脑结构网络G1’；
S102、功能判别器判断生成的脑功能网络G2’和真实的脑功能网络G2的差距，结构判别器判断脑结构网络G1’和真实的脑结构网络G1的差距；
S103、所述功能生成器将脑功能网络G2’生成脑结构网络G1”，所述结构生成器将脑结构网络G1’生成脑功能网络G2”；
S104、所述功能判别器判断生成的脑功能网络G2”和真实的脑功能网络G2的差距，所述结构判别器判断生成的脑结构网络G1”和真实的脑结构网络G1的差距。
可选地，所述S101和S102之间包括，
脑结构数据模糊化处理，将脑结构真实连接矩阵和脑功能真实连接矩阵进行多次卷积和激活函数处理，使得脑结构真实连接矩阵中包含有脑功能真实连接矩阵中的部分信息，从而得到模糊化后的脑结构真实连接矩阵；将此模糊化后的脑结构真实连接矩阵和功能生成器生成的脑结构生成连接矩阵给到结构判别器；
脑功能数据模糊化处理，将脑功能真实连接矩阵和脑结构真实连接矩阵进行多次卷积和激活函数处理，使得脑功能真实连接矩阵中包含有脑结构真实连接矩阵中的部分信息，从而 得到模糊化后的脑功能真实连接矩阵；将此模糊化后的脑功能真实连接矩阵和结构生成器生成的脑功能生成连接矩阵给到功能判别器。
可选地，所述激活函数采用ReLU激活函数。
可选地，所述脑结构模态影像包括DTI和sMRI模态影像；所述脑功能模态影像包括fMRI和PET模态影像。
可选地，对所述脑结构模态影像进行预处理的方法包括，对脑部DTI模态影像进行预处理，得到脑白质的纤维束，并将该纤维束图像映射到脑区划分模板，将大脑皮层及皮下组织划分为M个脑区，结合此M个脑区完成DTI脑结构网络的构建；对脑部sMRI模态影像进行预处理，将sMRI脑模态影像图像映射到脑区划分模板，得到M个脑区的脑灰质密度，结合此M个脑区的脑灰质密度完成sMRI脑结构网络的构建，脑区之间的纤维束的密度即为结构性连接特征；
对所述脑功能模态影像进行预处理的方法包括，对脑部fMRI模态影像进行预处理，将fMRI模态影像图像映射到脑区划分模板，得到M个脑区的时间序列，结合此M个脑区的时间序列完成fMRI脑功能网络的构建；对脑部PET进行预处理，将PET模态影像图像映射到脑区划分模板，得到M个脑区中含有标记的包括蛋白质、葡萄糖等物质含量，基于此完成PET脑网络的构建。
可选地，所述脑区划分模板为ALL模板，所述M＝90。
可选地，在所述结构特征提取模块和功能特征提取模块中，基于循环图卷积公式
Figure PCTCN2021140017-appb-000001
进行特征提取；其中，＊是循环图卷积运算符，ο运算形式见下，⊙是Hardmard乘积，
Figure PCTCN2021140017-appb-000002
是对g进行某种变换，同理，
Figure PCTCN2021140017-appb-000003
是对x进行相同的变换，而U T(...)是对其进行逆变换；
基于循环图卷积算法，将DTI脑结构网络的节点信息N1和边连接信息E1及sMRI脑结构网络的节点信息N2和边连接信息E2作为脑结构一组，将fMRI脑功能网络的节点信息N3和边连接信息E3及PET脑功能网络的节点信息N4和边信息E4作为脑功能一组，然后使用两个特征提取模块分别一一对应提取大脑的结构特征(N1’，N3’)和功能特征(N2’，N4’)。
根据本发明申请提供的一种结构-功能脑网络双向映射模型，包括，
特征预处理模块，基于指定类型病患的脑结构模态影像和脑功能模态影像，对所述脑结 构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对所述脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
结构特征提取模块，基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；
功能特征提取模块，基于循环图卷积算法，对脑功能网络进行特征提取，得到大脑的功能特征；
结构分类器模块，基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；
功能分类器模块，基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
结构-功能双向映射网络，基于所述大脑的结构特征和功能特征，利用包括的结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进行双向映射。
通过采用上述技术方案，有利于成功地学习到大脑结构和功能的潜在特征表示，在辅助预测疾病类别的同时，可以帮助揭露大脑结构和功能之间的关系。
可选地，基于所述大脑的结构特征和功能特征，首先，所述结构生成器将脑结构网络G1生成脑功能网络G2’，所述功能生成器将脑功能网络G2生成脑结构网络G1’；其次，所述功能判别器判断生成的脑功能网络G2’和真实的脑功能网络G2的差距，所述结构判别器判断脑结构网络G1’和真实的脑结构网络G1的差距；再次，所述功能生成器将脑功能网络G2’生成脑结构网络G1”，所述结构生成器将脑结构网络G1’生成脑功能网络G2”；最后，所述功能判别器判断生成的脑功能网络G2”和真实的脑功能网络G2的差距，所述结构判别器判断生成的脑结构网络G1”和真实的脑结构网络G1的差距。
可选地，所述结构-功能双向映射网络还包括结构平衡器和功能平衡器；
所述结构平衡器包括卷积层和激活函数，将脑结构真实连接矩阵和脑功能真实连接矩阵进行多次卷积和激活函数处理，使得脑结构真实连接矩阵中包含有脑功能真实连接矩阵中的部分信息，从而得到模糊化后的脑结构真实连接矩阵；
所述功能平衡器包括卷积层和激活函数，将脑功能真实连接矩阵和脑结构真实连接矩阵进行多次卷积和激活函数处理，使得脑功能真实连接矩阵中包含有脑结构真实连接矩阵中的部分信息，从而得到模糊化后的脑功能真实连接矩阵。
根据本发明提供的一种计算机可读存储介质，存储有能够被处理器加载并执行上述任一种方法的计算机程序。
通过采用上述技术方案，有利于实现上述脑网络模型的构建。
附图说明
图1是本发明其中一实施例的结构-功能双向映射脑网络模型原理示意图；
图2是本发明其中一实施例的非欧式空间特征提取的网络结构示意图；
[根据细则91更正 24.01.2022]
图3是本发明其中一实施例的平衡器结构示意图。
具体实施方式
以下结合附图对本发明作进一步详细说明。
本具体实施例仅仅是对本发明的解释，其并不是对本发明的限制，本领域技术人员在阅读完本说明书后可以根据需要对本实施例做出没有创造性贡献的修改，但只要在本发明的权利要求范围内都受到专利法的保护。
如图1所示，本发明申请实施例提供的一种结构-功能双向映射脑网络模型构建方法，具体方法包括，
构建特征预处理模块，特征预处理模块用于基于指定类型病患的脑结构模态影像和脑功能模态影像，对脑结构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
构建结构特征提取模块和功能特征提取模块，结构特征提取模块用于基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；功能特征提取模块用于基于循环图卷积算法对脑功能网络进行特征提取，得到大脑的功能特征；
构建结构分类器模块和功能分类器模块，结构分类器模块用于基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；功能分类器基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
构建结构-功能双向映射网络，基于大脑的结构特征和功能特征，利用结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进行双向映射，生成结构-功能双向映射网络；
利用预处理后的所述脑结构网络和脑功能网络的数据集，对构建的结构特征提取模块和功能特征提取模块、结构分类器模块和功能分类器模块及结构-功能双向映射网络进行训练学习，得到基于多模态数据的结构-功能双向映射脑网络模型。
一方面，本申请基于循环图卷积算法，不仅仅考虑了单一模态内的关系，还考虑了结构和功能之间的关系，如此可以充分地利用模态内的所有信息。另一方面，现阶段很多文章虽然利用了多模态数据，但是在讨论时往往只针对结构能否生成功能，或是功能能否生成结构。针对上述问题，基于本申请提出的结构-功能脑网络双向映射模型，使得模型可以同时考虑结构和功能之间相互映射关系。
因此，在数据驱动模式下，实现多模态特征的提取和双向映射网络协同训练，有利于构建的脑网络模型成功地学习到大脑结构和功能的潜在特征表示，在辅助预测疾病类别的同时，可以帮助揭露大脑结构和功能之间的关系。并基于此可以判断出该脑疾病是因为大脑结构或功能的哪些异常脑连接，或是由结构和功能协调不一致造成的异常脑活动。
结构是功能的基础，功能是结构的表征，结构和功能间的关系对揭露脑的组织原理有着至关重要的作用。本发明申请利用模态影像数据构建脑结构和脑功能网络，并根据构建的脑结构和脑功能网络间的关系对脑疾病进行辅助诊断和发病机制判明。
为了说明本发明的具体实施方案，以下以阿尔茨海默症(AD)为例进行具体说明，阿尔茨海默症可以分成五个阶段，按照病情恶化顺序可以分成下面5类：正常人(NC)、显著性记忆关注(SMC)、早期轻度认知障碍(EMCI)、晚期轻度认知障碍(LMCI)和阿尔茨海默症(AD)。在本实施例中，选择5个阶段各N位作为受试者，对该5＊N位受试者全部采集脑结构模态影像和脑功能模态影像。
本申请具体实施例所涉及的多模态循环图卷积网络考虑每个受试者样本的四种模态：sMRI模态影像、DTI模态影像和fMRI模态影像、PET模态影像。其中，DTI和sMRI模态影像属于脑结构模态影像，fMRI和PET模态影像属于脑功能模态影像；采集fMRI、DTI、sMRI和PET四种模态影像作为原始数据集，并对该原始数据集进行数据转化。
构建特征预处理模块过程中，基于采集的原始数据集，分别进行预处理得到脑结构网络和脑功能网络。
对脑结构模态影像进行预处理的方法包括，对脑部DTI模态影像进行预处理，得到脑白质的纤维束，并将该纤维束图像映射到脑区划分模板，将大脑皮层及皮下组织划分为M个脑区，结合此M个脑区完成DTI脑结构网络的构建；对脑部sMRI模态影像进行预处理，将 sMRI脑模态影像图像映射到脑区划分模板，得到M个脑区的脑灰质密度，结合此M个脑区的脑灰质密度完成sMRI脑结构网络的构建，脑区之间的纤维束的密度即为结构性连接特征；
对脑功能模态影像进行预处理的方法包括，对脑部fMRI模态影像进行预处理，将fMRI模态影像图像映射到脑区划分模板，得到M个脑区的时间序列，结合此M个脑区的时间序列完成fMRI脑功能网络的构建；对脑部PET进行预处理，将PET模态影像图像映射到脑区划分模板，得到M个脑区中含有标记的包括蛋白质、葡萄糖等物质含量，基于此完成PET脑网络的构建。
作为预处理的一种实施方式，采用PANDA软件对DTI模态影像进行预处理，基于AAL模板将大脑皮层及皮下组织划分为90个脑区；采用GRETNA软件对fMRI模态影像进行预处理，将fMRI图像映射到AAL模板得到90个脑区的时间序列；采用CAT12软件对sMRI模态影像进行预处理，将sMRI模态影像映射到AAL模板，得到90个脑区中脑灰质密度；采用NetPET软件对PET模态影像进行预处理，结合ALL模板得到M个脑区中含有标记的包括蛋白质、葡萄糖等物质含量。本申请的预处理软件使用范围不限于PANDA、GRETNA、CAT12和NetPET软件，只要能够实现预处理过程得到相应的脑网络即可，也不限于ALL模板的90个脑区，也可以采用其他模板和其他数量脑区划分，例如，可以采用ALL1模板的90个脑区，也可以采用ALL2模板的116个脑区，在某些特定情况下，也可以采用ALL1模板的90个脑区中的部分脑区或者ALL2模板的116个脑区中的部分脑区。
原始数据集经过数据转化后得到的脑功能网络和脑结构网络包括了相应的节点信息和边信息，具体地，使用PANDA软件提取了DTI模态影像的节点信息N1和边连接信息E1；使用Gretna软件提取了fMRI模态影像的时间序列，并使用皮尔森相关系数构建fMRI模态影像的节点信息N2和边连接信息E2；使用CAT12软件提取了sMRI中脑区内的灰质密度，并构建出节点信息N3和边信息E3；使用NetPET软件构建大脑的节点信息N4和边信息E4。接下来，基于循环图卷积算法，通过节点之间的连接关系对节点的特征进行加权聚合，并降维到低维空间，从而实现对脑结构网络和脑功能网络特征的分别提取，得到大脑的结构特征和功能特征。
结构特征和功能特征提取过程中，如图2所示，使用非欧式特征提取网络，将不同模态数据进行变换(如傅里叶变换、Z域变换等)，数据被映射到对应的潜在空间(傅里叶空间、Z域空间等)，而后对几种数据进行卷积，由于循环卷积机制的存在，不同模态下的特征会进行 信息的聚合，接着再将卷积后的结果反变换回原空间，这样得到的特征利用了所有模态数据下的节点信息和边信息。下面对该过程进行详细说明。
脑网络中包含了两部分，分别是节点特征矩阵和邻接矩阵。
Figure PCTCN2021140017-appb-000004
表示的是图的邻接矩阵(即边矩阵)；
Figure PCTCN2021140017-appb-000005
是实数，n 1和n 2均表示某一个维度的长度；图是一种数学领域数据结构，脑网络指的是大脑以图数据表示时的称呼；
Figure PCTCN2021140017-appb-000006
表示的是图的度矩阵，度矩阵D只有对角线上有值，为对应节点的度，其余为0；
Figure PCTCN2021140017-appb-000007
表示的是图的拉普拉斯矩阵，计算形式为
Figure PCTCN2021140017-appb-000008
其中D (i)表示的是D(：，：，i)；I表示单位张量，i表示某一维度；
Figure PCTCN2021140017-appb-000009
表示的是卷积滤波器，R表示某一个维度的长度。
Figure PCTCN2021140017-appb-000010
表示的是输入的特征矩阵。
利用公式
Figure PCTCN2021140017-appb-000011
进行特征提取；其中，＊是循环图卷积运算符，ο运算形式见下，⊙是Hardmard乘积，
Figure PCTCN2021140017-appb-000012
是对g进行某种变换，同理，
Figure PCTCN2021140017-appb-000013
是对x进行相同的变换，而U T(...)是对其进行逆变换。其中，U计算形式(傅里叶变换)为：
L dft＝DFT(L)
for i in 1，2，3 do
[u，ξ]＝EVD(L dft (i))
Figure PCTCN2021140017-appb-000014
end for
Figure PCTCN2021140017-appb-000015
ο运算形式为：
这里设A∈R n1×n2×3和F∈R n2×n4×3，而
Figure PCTCN2021140017-appb-000016
其中
Figure PCTCN2021140017-appb-000017
U是对应的变换形式(傅里叶变换、Z变换等)，而U H是逆变换。
基于循环图卷积算法，将N1、E1和N3、E3作为脑结构一组，将N2、E2和N4、E4作 为脑功能一组，然后使用两个特征提取模块(结构特征提取模块和功能特征提取模块)分别一一对应提取大脑的结构特征和功能特征(N1’，N3’)和功能特征(N2’，N4’)。
现阶段，一些学者利用脑结构数据对阿尔茨海默症进行辅助诊断，取得了较好的成果。相较于正常人(同龄)，患者的一些脑区会出现明显的萎缩，脑区的萎缩阻碍了某些功能的实现；一些学者测量了患者脑中的β-淀粉样蛋白，患者脑中的β-淀粉样蛋白要普遍高于正常人水平；还有一些学者利用脑功能数据去研究阿尔茨海默症，根据患者在静息状态下，某个时间段内大脑的活动情况来判断患者的病情。上述的几种方法大多数都是基于卷积方法(CNN)算法实现的，CNN算法虽然可以很好地完成特征的提取，但它是在空间上聚集相邻的N＊N个节点，在时间上汇集某个时间段内的节点的信息，它却忽略了这样的一个问题——相邻节点可能存在影响，但不一定是最重要的影响，反而相隔较远的像素点之间互相影响程度更高。
为此，现在越来越多的学者将图卷积应用到脑疾病的研究中。常见的方法是将其中一种模态数据作为图的边连接信息，将其它模态数据作为图的结点信息。相比于传统的图卷积，本发明申请方案使用的循环图卷积对其进行了维度上的扩展，将原有的矩阵计算形式扩展到了多维的张量计算。得到的好处是可以更加充分的利用到不同模态中的信息，此外因为循环卷积的机制，还可以尝试使用模态之间的关系，这样得到的信息将更加的丰富。
病情分类中，为了充分利用非欧式空间的特征信息，同时降低模型的运算复杂度、提高网络的协同训练效率，在本申请中我们选择了结构简单而有效的分类器，主要包括一层循环图卷积模块和MLP(多层感知机)模块，首先将节点信息和边连接信息给到循环图卷积模块，然后将进一步提取到的信息(90＊1)给到MLP模块，MLP可以综合使用这90个脑区的特征信息给出最终的分类结果。其提取特征的处理流程为：
将特征提取过程中得到的90＊d(d为特征矩阵的某一维度的长度；长是90，表示脑区的数量；宽是d，表示一个脑区的特征向量的长度)的4个模态的脑网络数据((N1’，E1)、(N3’，E3)、(N2’，E2)和(N4’，E4))使用一层循环图卷积融合两者特征信息。可以将4个90＊d的特征矩阵信息融合到一个90＊1的特征矩阵信息M3’；然后将M3’给到全连接层，全连接层对90个脑区的特征进行加权，再经过一个softmax层，就可以计算出5个类别(上述阿尔茨海默症按照病情恶化顺序分成下面5类)输出的概率大小。
为了探寻结构和功能之间的关系，本发明选择使用具有双向映射的生成对抗网络，基于大脑的结构特征和功能特征，利用结构生成器模块、功能生成器模块、结构判别器模块和功 能判别器模块对脑结构网络和脑功能网络进行双向映射。构建结构-功能双向映射网络的具体方法包括，
S101、基于大脑的结构特征和大脑的功能特征，结构生成器将脑结构网络G1生成脑功能网络G2’，功能生成器将脑功能网络G2生成脑结构网络G1’；
S102、功能判别器判断生成的脑功能网络G2’和真实的脑功能网络G2的差距，结构判别器判断脑结构网络G1’和真实的脑结构网络G1的差距；
S103、功能生成器将脑功能网络G2’生成脑结构网络G1”，结构生成器将脑结构网络G1’生成脑功能网络G2”；
S104、功能判别器判断生成的脑功能网络G2”和真实的脑功能网络G2的差距，结构判别器判断生成的脑结构网络G1”和真实的脑结构网络G1的差距。
从图1中可以看出，相比于复杂的生成器，判别器的结构和功能很简单，这样导致的结果是生成器和判别器的性能之间很难达到平衡。基于此，在S102的判别器前增加了平衡器，以减缓判别器在生成对抗过程中的学习速率。
如图3所示的结构平衡器的结构，包括卷积层和激活函数。基于结构平衡器的结构，根据前一个循环迭代过程中结构生成器和结构判别器之间的性能差，将脑结构真实连接矩阵(即脑结构网络的边信息)和脑功能真实连接矩阵(即脑功能网络的边信息)进行多次卷积和激活函数处理，使得脑结构真实连接矩阵中包含有脑功能真实连接矩阵中的部分信息，从而得到模糊化后的脑结构真实连接矩阵；将此模糊化后的脑结构真实连接矩阵和功能生成器生成的脑结构生成连接矩阵给到结构判别器，从而减小了生成结果和真实结果之间差距，进而降低了结构生成器的学习难度；同时减小生成结果和真实结果之间的差距还可以减缓结构分类器完全学习到真实结果数据分布的速率，从而使得结构生成器和结构判别器的性能之间趋于平衡。
功能平衡器的结构与结构平衡器的结构相同，包括卷积层和激活函数。基于功能平衡器的结构，根据前一个循环迭代过程中功能生成器和功能判别器之间的性能差，将脑功能真实连接矩阵和脑结构真实连接矩阵进行多次卷积和激活函数处理，使得脑功能真实连接矩阵中包含有脑结构真实连接矩阵中的部分信息，从而得到模糊化后的脑功能真实连接矩阵；将此模糊化后的脑功能真实连接矩阵和结构生成器生成的脑功能生成连接矩阵给到功能判别器，从而减小了生成结果和真实结果之间差距，进而降低了功能生成器的学习难度；同时减小生成结果和真实结果之间的差距还可以减缓功能分类器完全学习到真实结果数据分布的速率， 从而使得功能生成器和功能判别器的性能之间趋于平衡。
作为本发明申请的具体实施例，卷积层的卷积核为3＊3，步长为1，激活函数可以采用ReLU激活函数，经过4组卷积层和激活函数。参数调整需要通过实验尝试得到，可以控制其余参数不变，然后修改当前参数，多次实验而后在这些参数中找到最优的卷积层和激活函数的组数。
基于构建的模型框架，对模型进行训练，训练过程中，输入四种模态(fMRI，DTI，sMRI，PET)的数据，其中sMRI和DTI作为脑结构网络，提取大脑的结构特征表示，fMRI和PET作为脑功能网络，提取大脑的功能特征表示，将两种特征表示分别一一对应给到结构分类器模块和功能分类器模块，输出疾病预测的类别。
基于结构分类器模块(或功能分类器模块)的分类输出结果，和真实的分类结果进行比较，得到结构分类器模块(或功能分类器模块)的损失函数
Figure PCTCN2021140017-appb-000018
在这里，主要使用的是多分类实验。其中，K是种类数量；y i是类别为i的标签，y i＝1或y i＝0；p i是结构分类器模块(或功能分类器模块)的输出，也就是类别为i的概率，这个输出值就是用softmax计算出的。结构分类器模块(或功能分类器模块)的损失函数主要用于对结构特征提取模块(或功能特征提取模块)评估的反向传播学习训练。
由于结构分类器损失函数和功能分类器的损失函数算法相同，因此，根据以上给出的结构分类器的损失函数算法则可得出将小括号内的功能分类器替换为结构分类器后的功能分类器的损失函数算法，以下小括号内容代表的含义相同。
为了减缓结构分类器模块(或功能分类器模块)的学习速率，考虑结构生成器模块(或功能生成器模块)和结构判别器模块(或功能判别器模块)之间的性能差，得到结构平衡器(或功能平衡器)的损失函数
Figure PCTCN2021140017-appb-000019
其中，X是模型的输入，Y是模型的生成结果，D是结构判别器模块(或功能判别器模块)的简称，α是超参数，可以根据结构生成器模块(或功能生成器模块)和结构判别器模块(或功能判别器模块)的性能差进行调节,
Figure PCTCN2021140017-appb-000020
对于结构-功能双向映射网络的损失函数，其计算公式为
Figure PCTCN2021140017-appb-000021
主要包括下面几个部分：(1)双向映射网络中结构生成器模块(或功能生成器模块)的损失函数加双向映射网络中结构判别器模块(或功能判别器模块)的损失函数
Figure PCTCN2021140017-appb-000022
该损失函数主要用于对结构特征提取模块(或功能特征提取模块)、结构生成器模块(或功能生成器模块)和结构判别器模块(或功能判别器模块)评估的反向传播学习训练。(2)双向映射网络中结构生成器模块(或功能生成器模块)生成的结果和真实结果之间的差异
Figure PCTCN2021140017-appb-000023
该损失函数主要用于对结构特征提取模块、结构生成器模块和功能生成器模块评估的反向传播学习训练。(3)双向映射网路中生成器生成结果G2”与真实结果G2之间的差异
Figure PCTCN2021140017-appb-000024
该损失函数主要用于对功能特征提取模块、结构生成器模块和功能生成器模块评估的反向传播学习训练。(4)双向映射网络中结构生成器除了学习了G1到G2’的映射关系，同时在每次训练过程中还需要学习从G1到G1’的映射关系
Figure PCTCN2021140017-appb-000025
该损失函数主要用于对结构特征提取模块和结构生成器模块评估的方向传播学习训练。(5)双向映射网络中结构分类器模块(或功能分类器模块)的分类结果
Figure PCTCN2021140017-appb-000026
将作为结构生成器模块(或功能生成器模块)性能的评估标准之一；损失函数主要用于对结构(或功能)特征提取模块和结构(或功能)分类器评估的方向传播学习训练。
将训练样本划分为90％用于训练模型，10％用于测试训练完成模型的性能。模型的训练过程可做如下定性分析：在数据驱动模式下，随着双向映射网络和循环图卷积的不断优化，结构判别器模块和功能判别器模块需要不断更新参数以识别结构生成器模块和功能生成器模块产生的伪数据分布；结构生成器模块和功能生成器模块需要不断通过结构判别器模块和功能判别器模块的结果更新参数，并据此将生成分布向真实分布靠近；结构分类器模块和功能分类器模块优化使其预测的数据分布在不同类别之间具有很好的区分度。模型训练完成后，结构分类器模块和功能分类器模块可以帮助医生进行辅助诊断，同时，双向映射网络可以实现结构脑网络和功能脑网络之间的互相映射，并基于此机制查找该脑疾病下，探索结构和功能之间是否存在统一关系。
基于本发明申请方案进行结构和功能之间的互相映射，对生成结果测试了有效性，证明了结构生成器模块和功能生成器模块学习到的映射关系是有意义的。
有效性测试步骤见下：
1)使用训练出的结构生成器模块和功能生成器模块去验证生成结果的有效性，得到的分类结果相近甚至更好，说明结构生成器模块和功能生成器模块学习到了结构和功能之间的映射 关系，甚至于对原有的脑网络进行了优化。
2)进行错位学习。将不同患者的脑结构网络和脑功能网络进行组合，结构和功能之间无法完成互相映射，或者分类的效果不好，说明分类器模块和生成器模块都无法学习到结构和功能之间的关系。
为了研究结构和功能之间的关系，使用了统计学方法去研究患者中结构和功能键的关系，步骤如下：
统计了不同类别患者的结构和功能存在的脑连接的数量。不同类别的患者脑结构和功能连接数量有着较大的差异，而脑结构和功能连接数量的变化趋势也有着明显的差距。
比较了单个个体的脑结构和功能的脑网络。可以发现个体的结构和功能网络之间只有少数脑区之间有着明显的相似性，证明了脑结构和功能网络之间存在的不是简单的一一对应关系。
利用统计学方法研究了不同类别患者间结构和功能的变化趋势。不同类别的患者脑结构和功能网络分布并不是一致，利用该方法得到的异常结构、功能脑连接只是部分相同，还有一些脑连接在结构和功能上存在着较大的差异。这证明了结构和功能存在着一些复杂的关系而非简单的一一对应关系，一个功能可以需要多个脑区的协作完成。
根据本发明申请提供的一种结构-功能脑网络双向映射模型，包括，
特征预处理模块，基于指定类型病患的脑结构模态影像和脑功能模态影像，对脑结构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
结构特征提取模块，基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；
功能特征提取模块，基于循环图卷积算法，对脑功能网络进行特征提取，得到大脑的功能特征；
结构分类器模块，基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；
功能分类器模块，基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
结构-功能双向映射网络，基于所述大脑的结构特征和功能特征，利用包括的结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进 行双向映射。
基于本申请方案构建的结构-功能脑网络双向映射模型，找到了相应的异常脑连接。基于这些异常脑连接构建结构分类器和功能分类器，如果最终的分类结果与上面的结果相近，即可论断出这些脑连接确实与该种疾病有着重要的作用。依次删除单个脑连接，保证其余变量相同，再次进行实验，证明单个脑连接与该种疾病的关系。
基于上述脑网络模型可以提供一种多模态影像融合辅助诊断系统，通过对网络的迭代训练，第一在对抗训练中达到最优，此时第一循环图卷积模块将学习到由sMRI和DTI映射到潜在空间的特征，同时第二循环图卷积模块将学习到由fMRI映射到潜在空间的特征；分类器学习到了多模态融合的最优参数，可以相对准确地由输入的四种模态数据预测出相应的疾病类别以及对应的脑连接模型。将训练好的生成器模块和分类器模块进行提取和迁移，构成端到端的多模态影像融合辅助诊断系统。
辅助诊断系统应用时，采集四种影像模态数据sMRI、fMRI和DTI、PET；将多模态数据输入以训练好的多模态融合网络中，完成对患者认知疾病的分类诊断。生成器模块很好地学习到了结构和功能之间的关系，可以进行结构和功能之间的映射，并对原有脑图谱进行优化，这不仅仅可以提升分类精度，还可以辅助研究人员探索脑结构和功能之间的深层协调机制。
另外，需要说明的是，阿尔茨海默症只是本发明申请实施例选择的一种病患类型，以DTI、fMRI、sMRI、PET四种模态影像为例阐述，但本申请的应用范围不限于病种阿尔茨海默症和fMRI-DTI-sMRI-PET影像，也可以是其他类型的病患，作为一种病患类型，这里选择了5个阶段，试验验证过程中，也可以选择其中几个阶段或不限于该5个阶段。对于采集的结构脑模态影像，并不限于DTI和sMRI模态影像，可以是其中一种或几种模态影像，也可以是其他模态影像，只要属于结构脑模态影像即可；对于采集的功能脑模态影像，并不限于fMRI和PET模态影像，可以是其中一种或几种模态影像，也可以是其他模态影像，只要属于功能脑模态影像即可。
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本具体实施方式的实施例均为本发明的较佳实施例，并非依此限制本发明的保护范 围，故：凡依本发明的结构、形状、原理所做的等效变化，均应涵盖于本发明的保护范围之内。

Claims (10)

  1. 一种结构-功能脑网络双向映射模型构建方法，其特征在于，具体方法包括，
    构建特征预处理模块，所述特征预处理模块用于基于指定类型病患的脑结构模态影像和脑功能模态影像，对所述脑结构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对所述脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
    构建结构特征提取模块和功能特征提取模块，所述结构特征提取模块用于基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；所述功能特征提取模块用于基于循环图卷积算法，对脑功能网络进行特征提取，得到大脑的功能特征；
    构建结构分类器模块和功能分类器模块，所述结构分类器模块用于基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；所述功能分类器基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
    构建结构-功能双向映射网络，基于所述大脑的结构特征和功能特征，利用结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进行双向映射；
    利用预处理后的所述脑结构网络和脑功能网络的数据集，对构建的结构特征提取模块和功能特征提取模块、结构分类器模块和功能分类器模块及结构-功能双向映射网络进行训练学习。
  2. 根据权利要求1所述的结构-功能脑网络双向映射模型构建方法，其特征在于，所述构建结构-功能双向映射网络的具体方法包括，
    S101、基于所述大脑的结构特征和大脑的功能特征，所述结构生成器将脑结构网络G1生成脑功能网络G2’，所述功能生成器将脑功能网络G2生成脑结构网络G1’；
    S102、所述功能判别器判断生成的脑功能网络G2’和真实的脑功能网络G2的 差距，所述结构判别器判断脑结构网络G1’和真实的脑结构网络G1的差距；
    S103、所述功能生成器将脑功能网络G2’生成脑结构网络G1”，所述结构生成器将脑结构网络G1’生成脑功能网络G2”；
    S104、所述功能判别器判断生成的脑功能网络G2”和真实的脑功能网络G2的差距，所述结构判别器判断生成的脑结构网络G1”和真实的脑结构网络G1的差距。
  3. 根据权利要求2所述的结构-功能脑网络双向映射模型构建方法，其特征在于，所述S101和S102之间包括，
    脑结构数据模糊化处理，将脑结构真实连接矩阵和脑功能真实连接矩阵进行多次卷积和激活函数处理，使得脑结构真实连接矩阵中包含有脑功能真实连接矩阵中的部分信息，从而得到模糊化后的脑结构真实连接矩阵；将此模糊化后的脑结构真实连接矩阵和功能生成器生成的脑结构生成连接矩阵给到结构判别器；
    脑功能数据模糊化处理，将脑功能真实连接矩阵和脑结构真实连接矩阵进行多次卷积和激活函数处理，使得脑功能真实连接矩阵中包含有脑结构真实连接矩阵中的部分信息，从而得到模糊化后的脑功能真实连接矩阵；将此模糊化后的脑功能真实连接矩阵和结构生成器生成的脑功能生成连接矩阵给到功能判别器。
  4. 根据权利要求3所述的结构-功能脑网络双向映射模型构建方法，其特征在于，所述激活函数采用ReLU激活函数。
  5. 根据权利要求1所述的结构-功能脑网络双向映射模型构建方法，其特征在于，所述脑结构模态影像包括DTI和sMRI模态影像；所述脑功能模态影像包括fMRI和PET模态影像。
  6. 根据权利要求5所述的结构-功能脑网络双向映射模型构建方法，其特征在于，在所述结构特征提取模块和功能特征提取模块中，基于循环图卷积公式
    Figure PCTCN2021140017-appb-100001
    进行特征提取；其中，＊是循环图卷积运算符，ο运算形式见下，⊙是Hardmard乘积，
    Figure PCTCN2021140017-appb-100002
    是对g进行某种变换，同理，
    Figure PCTCN2021140017-appb-100003
    是对x进行相同的变换，而U T(...)是对其进行逆变换；
    基于循环图卷积算法，将DTI脑结构网络的节点信息N1和边连接信息E1及sMRI脑结构网络的节点信息N3和边连接信息E3作为脑结构一组，将fMRI脑功能网络的节点信息N2和边连接信息E2及PET脑功能网络的节点信息N4和边信息E4作为脑功能一组，然后使用两个特征提取模块分别一一对应提取大脑的结构特征(N1’，N3’)和功能特征(N2’，N4’)。
  7. 一种结构-功能脑网络双向映射模型，其特征在于，包括，
    特征预处理模块，基于指定类型病患的脑结构模态影像和脑功能模态影像，对所述脑结构模态影像进行预处理得到包括对应节点特征和边信息的脑结构网络，对所述脑功能模态影像进行预处理得到包括对应节点特征和边信息的脑功能网络；
    结构特征提取模块，基于循环图卷积算法，对脑结构网络进行特征提取，得到大脑的结构特征；
    功能特征提取模块，基于循环图卷积算法，对脑功能网络进行特征提取，得到大脑的功能特征；
    结构分类器模块，基于所述大脑的结构特征对患者进行病情诊断得到基于结构特征的病情分类结果，以实现对结构特征提取模块的训练；
    功能分类器模块，基于所述的大脑的功能特征对患者进行病情诊断得到基于功能特征的病情分类结果，以实现对功能特征提取模块的训练；
    结构-功能双向映射网络，基于所述大脑的结构特征和功能特征，利用包括的结构生成器模块、功能生成器模块、结构判别器模块和功能判别器模块对脑结构网络和脑功能网络进行双向映射。
  8. 根据权利要求7所述的结构-功能脑网络双向映射模型，其特征在于，
    基于所述大脑的结构特征和功能特征，首先，所述结构生成器将脑结构网络G1生成脑功能网络G2’，所述功能生成器将脑功能网络G2生成脑结构网络G1’；其次，所述功能判别器判断生成的脑功能网络G2’和真实的脑功能网络G2的差 距，所述结构判别器判断脑结构网络G1’和真实的脑结构网络G1的差距；再次，所述功能生成器将脑功能网络G2’生成脑结构网络G1”，所述结构生成器将脑结构网络G1’生成脑功能网络G2”；最后，所述功能判别器判断生成的脑功能网络G2”和真实的脑功能网络G2的差距，所述结构判别器判断生成的脑结构网络G1”和真实的脑结构网络G1的差距。
  9. 根据权利要求8所述的结构-功能脑网络双向映射模块，其特征在于，所述结构-功能双向映射网络还包括结构平衡器和功能平衡器；
    所述结构平衡器包括卷积层和激活函数，将脑结构真实连接矩阵和脑功能真实连接矩阵进行多次卷积和激活函数处理，使得脑结构真实连接矩阵中包含有脑功能真实连接矩阵中的部分信息，从而得到模糊化后的脑结构真实连接矩阵；
    所述功能平衡器包括卷积层和激活函数，将脑功能真实连接矩阵和脑结构真实连接矩阵进行多次卷积和激活函数处理，使得脑功能真实连接矩阵中包含有脑结构真实连接矩阵中的部分信息，从而得到模糊化后的脑功能真实连接矩阵。
  10. 一种计算机可读存储介质，其特征在于，存储有能够被处理器加载并执行如权利要求1至7中任一种方法的计算机程序。
PCT/CN2021/140017 2021-12-18 2021-12-21 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型 WO2023108712A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111556672.7 2021-12-18
CN202111556672.7A CN114242236A (zh) 2021-12-18 2021-12-18 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型

Publications (1)

Publication Number Publication Date
WO2023108712A1 true WO2023108712A1 (zh) 2023-06-22

Family

ID=80758539

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140017 WO2023108712A1 (zh) 2021-12-18 2021-12-21 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型

Country Status (2)

Country Link
CN (1) CN114242236A (zh)
WO (1) WO2023108712A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117036727A (zh) * 2023-10-09 2023-11-10 之江实验室 脑网络数据多层嵌入向量特征提取方法及装置
CN117079093A (zh) * 2023-10-18 2023-11-17 澄影科技(北京)有限公司 一种基于多模态影像数据的异动症预测方法、装置及设备
CN117350352A (zh) * 2023-12-06 2024-01-05 烟台大学 结构脑网络到功能连通性网络的学习方法、系统和设备
CN117372440A (zh) * 2023-12-08 2024-01-09 北京邮电大学 阿尔茨海默病风险预警模型训练方法、预警方法及设备
CN117934977A (zh) * 2024-03-22 2024-04-26 烟台大学 基于图连接和图节点的脑网络分类方法、系统和设备
CN117975041A (zh) * 2024-03-29 2024-05-03 烟台大学 基于图卷积神经网络的脑网络特征提取方法、系统和设备
CN117934977B (zh) * 2024-03-22 2024-05-31 烟台大学 基于图连接和图节点的脑网络分类方法、系统和设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114242236A (zh) * 2021-12-18 2022-03-25 深圳先进技术研究院 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型
CN115409843B (zh) * 2022-11-02 2023-04-07 长春理工大学 基于尺度均衡耦合卷积架构的脑神经影像特征提取方法
CN117408322B (zh) * 2023-12-15 2024-04-16 北京邮电大学 脑功能与脑结构的映射模型训练方法、装置及映射方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130231552A1 (en) * 2012-03-05 2013-09-05 Siemens Corporation Method and System for Diagnosis of Attention Deficit Hyperactivity Disorder from Magnetic Resonance Images
CN108345903A (zh) * 2018-01-25 2018-07-31 中南大学湘雅二医院 一种基于模态距离约束的多模态融合图像分类方法
CN113040715A (zh) * 2021-03-09 2021-06-29 北京工业大学 一种基于卷积神经网络的人脑功能网络分类方法
CN113571195A (zh) * 2021-07-20 2021-10-29 南京脑科医院 基于小脑功能连接特征的阿尔茨海默病早期的预测模型
CN113724880A (zh) * 2021-11-03 2021-11-30 深圳先进技术研究院 一种异常脑连接预测系统、方法、装置及可读存储介质
CN114242236A (zh) * 2021-12-18 2022-03-25 深圳先进技术研究院 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130231552A1 (en) * 2012-03-05 2013-09-05 Siemens Corporation Method and System for Diagnosis of Attention Deficit Hyperactivity Disorder from Magnetic Resonance Images
CN108345903A (zh) * 2018-01-25 2018-07-31 中南大学湘雅二医院 一种基于模态距离约束的多模态融合图像分类方法
CN113040715A (zh) * 2021-03-09 2021-06-29 北京工业大学 一种基于卷积神经网络的人脑功能网络分类方法
CN113571195A (zh) * 2021-07-20 2021-10-29 南京脑科医院 基于小脑功能连接特征的阿尔茨海默病早期的预测模型
CN113724880A (zh) * 2021-11-03 2021-11-30 深圳先进技术研究院 一种异常脑连接预测系统、方法、装置及可读存储介质
CN114242236A (zh) * 2021-12-18 2022-03-25 深圳先进技术研究院 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUANG, JIASHAUNG ET AL.: "Brain Network Analysis: Method and Application", JOURNAL OF DATA ACQUISITION AND PROCESSING, vol. 36, no. 4, 30 April 2021 (2021-04-30), XP009547127 *
WANG, ZHIQIONG ET AL.: "Computer-Aided Diagnosis of Mild Cognitive Impairment Based on Extreme Learning Machine", JOURNAL OF HARBIN ENGINEERING UNIVERSITY, vol. 42, no. 6, 30 June 2021 (2021-06-30), XP009547128 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117036727A (zh) * 2023-10-09 2023-11-10 之江实验室 脑网络数据多层嵌入向量特征提取方法及装置
CN117036727B (zh) * 2023-10-09 2024-01-05 之江实验室 脑网络数据多层嵌入向量特征提取方法及装置
CN117079093A (zh) * 2023-10-18 2023-11-17 澄影科技(北京)有限公司 一种基于多模态影像数据的异动症预测方法、装置及设备
CN117079093B (zh) * 2023-10-18 2023-12-12 澄影科技(北京)有限公司 一种基于多模态影像数据的异动症预测方法、装置及设备
CN117350352A (zh) * 2023-12-06 2024-01-05 烟台大学 结构脑网络到功能连通性网络的学习方法、系统和设备
CN117350352B (zh) * 2023-12-06 2024-02-23 烟台大学 结构脑网络到功能连通性网络的学习方法、系统和设备
CN117372440A (zh) * 2023-12-08 2024-01-09 北京邮电大学 阿尔茨海默病风险预警模型训练方法、预警方法及设备
CN117372440B (zh) * 2023-12-08 2024-03-19 北京邮电大学 阿尔茨海默病风险预警模型训练方法、预警方法及设备
CN117934977A (zh) * 2024-03-22 2024-04-26 烟台大学 基于图连接和图节点的脑网络分类方法、系统和设备
CN117934977B (zh) * 2024-03-22 2024-05-31 烟台大学 基于图连接和图节点的脑网络分类方法、系统和设备
CN117975041A (zh) * 2024-03-29 2024-05-03 烟台大学 基于图卷积神经网络的脑网络特征提取方法、系统和设备

Also Published As

Publication number Publication date
CN114242236A (zh) 2022-03-25

Similar Documents

Publication Publication Date Title
WO2023108712A1 (zh) 结构-功能脑网络双向映射模型构建方法及脑网络双向映射模型
CN109376751B (zh) 一种基于卷积神经网络的人脑功能网络分类方法
CN113040715B (zh) 一种基于卷积神经网络的人脑功能网络分类方法
WO2023077603A1 (zh) 一种异常脑连接预测系统、方法、装置及可读存储介质
CN113616184B (zh) 基于多模态磁共振图像的脑网络建模与个体预测方法
Chen et al. Iterative sparse and deep learning for accurate diagnosis of Alzheimer’s disease
Pan et al. Characterization multimodal connectivity of brain network by hypergraph GAN for Alzheimer’s disease analysis
WO2022147871A1 (zh) 影像驱动的脑图谱构建方法、装置、设备及存储介质
CN110889496B (zh) 一种基于对抗生成网络的人脑效应连接识别方法
CN113693563A (zh) 一种基于超图注意力网络的脑功能网络分类方法
CN113673244B (zh) 医疗文本处理方法、装置、计算机设备和存储介质
Zhu et al. Deep multi-modal discriminative and interpretability network for alzheimer’s disease diagnosis
CN115841607A (zh) 一种基于图注意力网络的脑网络结构和相似度的联合学习方法
CN115272295A (zh) 基于时域-空域联合状态的动态脑功能网络分析方法及系统
Feng et al. A review of methods for classification and recognition of ASD using fMRI data
WO2023108873A1 (zh) 一种脑网络和脑成瘾连接计算方法及装置
Singha et al. An Experimental Approach to Diagnose Covid-19 Using Optimized CNN.
Luo et al. Multi-view brain network analysis with cross-view missing network generation
CN117765530A (zh) 一种多模态脑网络分类方法、系统、电子设备及介质
CN112233805B (zh) 基于多图谱神经影像学数据进行生物标志物的挖掘方法
CN112863664A (zh) 基于多模态超图卷积神经网络的阿尔茨海默病分类方法
CN116759076A (zh) 一种基于医疗影像的无监督疾病诊断方法及系统
Ma et al. Semi-parametric Bayes regression with network-valued covariates
Liang et al. Multi-Scale Attention-Based Deep Neural Network for Brain Disease Diagnosis.
Vijaya et al. Classification of Pneumonia using InceptionNet, ResNet and CNN

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21967844

Country of ref document: EP

Kind code of ref document: A1