WO2022147885A1 - 影像驱动的脑图谱构建方法、装置、设备及存储介质 - Google Patents

影像驱动的脑图谱构建方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022147885A1
WO2022147885A1 PCT/CN2021/075798 CN2021075798W WO2022147885A1 WO 2022147885 A1 WO2022147885 A1 WO 2022147885A1 CN 2021075798 W CN2021075798 W CN 2021075798W WO 2022147885 A1 WO2022147885 A1 WO 2022147885A1
Authority
WO
WIPO (PCT)
Prior art keywords
hypergraph
matrix
node
feature matrix
node feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/075798
Other languages
English (en)
French (fr)
Inventor
王书强
潘俊任
申妍燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Priority to US17/764,755 priority Critical patent/US12159706B2/en
Publication of WO2022147885A1 publication Critical patent/WO2022147885A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the technical field of medical imaging, and in particular, to an image-driven brain atlas construction method, device, device and storage medium.
  • a structural connectivity matrix (including the density of fiber bundles between various brain regions of the brain, also known as a structural connectivity map) can be generated by collecting Diffusion Tensor Imaging (DTI) data, which can express the histological anatomy of the brain. information.
  • DTI Diffusion Tensor Imaging
  • rs-fMRI resting-state functional Magnetic Resonance Imaging
  • a node feature matrix (including the temporal feature sequence of each brain region, also known as a functional connectivity map) is generated, which can express Functional connectivity information between various brain regions.
  • Brain atlas is a very important reference in the process of diagnosing brain diseases.
  • the embodiments of the present application provide an image-driven brain atlas construction method, device, device, and storage medium, which can solve the problem that the current brain atlas can express less feature information.
  • an embodiment of the present application provides an image-driven brain atlas construction method, including: acquiring a node feature matrix, where the node feature matrix includes a time series of multiple nodes in the brain; Convert the hypergraph data structure to obtain the first hypergraph association matrix; input the first hypergraph association matrix and node feature matrix into the trained hypergraph transition matrix generator for processing, and output the first hypergraph transition matrix, and the first hypergraph transition matrix is obtained.
  • the graph transition matrix characterizes the constructed multimodal brain atlas.
  • a multimodal brain atlas can be constructed by transforming the node feature matrix into a hypergraph data structure and calculating the first hypergraph transition matrix.
  • the multimodal brain connectivity structures can express more feature information.
  • the accuracy of disease diagnosis can be improved.
  • performing hypergraph data structure transformation on the time series to obtain a first hypergraph association matrix including: using the Guth-Katz polynomial clustering algorithm to convert the node feature matrix into the first hypergraph data; using the KNN algorithm to convert the node feature matrix into the first hypergraph data; The feature matrix is converted into the second hypergraph data; the first hypergraph data and the second hypergraph data are fused to obtain the first hypergraph correlation matrix.
  • the Guth-Katz polynomial clustering algorithm to construct the first hypergraph data, and then fusing the first hypergraph data with the second hypergraph data to obtain the first hypergraph association matrix, it can be better preserved.
  • the topology information between nodes in the node characteristic matrix is obtained, and redundant interference information is effectively eliminated, and the accuracy of the first hypergraph association matrix is improved.
  • the processing of the node feature matrix by the hypergraph transition matrix generator includes: performing several iterative calculations based on the first hypergraph association matrix and the node feature matrix to obtain the target hyperedge feature matrix and the target node feature matrix;
  • the hypergraph association matrix and the target node feature matrix determine the node weight matrix;
  • the hyperedge weight matrix is determined according to the target hyperedge feature matrix;
  • the first hypergraph transition matrix is determined according to the hyperedge weight matrix and the node weight matrix.
  • the first hypergraph association matrix and the node feature matrix are obtained by several times of iterative calculations.
  • the complementary information and potential connections between different modal data can be fully mined, unnecessary parameters can be reduced, so as to avoid model overfitting and have strong generalization ability, which is more suitable for small sample learning.
  • the training sample preprocesses the training sample to obtain a node feature matrix sample and a structural connection matrix sample; perform hypergraph data structure transformation on the node feature matrix sample to obtain a second hypergraph association matrix ; Input the second hypergraph association matrix and node feature matrix samples into the initial hypergraph transition matrix generator for processing, and output the second hypergraph transition matrix; based on the random walk principle on the hypergraph, determine each corresponding training sample.
  • the node is based on the set of associated nodes N real (v) obtained based on the structural connection matrix samples and the set of associated nodes N fake (v) obtained based on the second hypergraph transition matrix; the discriminator is used to determine the N real (v) and N of each node. fake (v) Make a judgment to obtain the judgment result corresponding to the training sample; perform iterative training according to the judgment result corresponding to each training sample and the preset loss function to obtain a hypergraph transition matrix generator.
  • the random walk principle on the hypergraph is used to replace the kernel function transformation and the Lagrange multiplier method in the traditional training method, thereby avoiding the complicated calculation in the non-convex optimization problem and the inability of the loss function to stabilize to the global maximum.
  • the advantage of the convergence problem therefore, improves the robustness of the model as well as the training efficiency.
  • the discriminator uses the discriminator to judge N real (v) and N fake (v) of each node, and obtain the judgment result corresponding to the training sample, including: for each node, corresponding the node in the node feature matrix sample.
  • the time series is input into the preset multi-layer perceptron for processing, and the output features are obtained; the judgment result of each node in N real (v) and N fake (v) of the node and the node is calculated according to the following formula:
  • D(va , v b ) represents the judgment result that the node v a is the associated node of v b , represents the output feature of node v a , represents the output feature of node v b .
  • the loss function is:
  • the present application provides an image-driven brain atlas construction device, comprising: an acquisition unit for acquiring a node feature matrix, where the node feature matrix includes a time series of multiple nodes in the brain; a conversion unit for The feature matrix converts the hypergraph data structure to obtain the first hypergraph association matrix; the construction unit is used to input the first hypergraph association matrix and the node feature matrix into the trained hypergraph transition matrix generator for processing, and the output obtains the first hypergraph association matrix.
  • Hypergraph transition matrix the first hypergraph transition matrix to characterize the constructed multimodal brain atlas.
  • the acquisition unit is used to preprocess the training samples for each training sample in the training set to obtain the node feature matrix samples and the structural connection matrix samples; the conversion unit is also used to perform a hypergraph on the node feature matrix samples.
  • the data structure is converted to obtain a second hypergraph association matrix;
  • the construction device further includes a training unit, and the training unit is used for: inputting the second hypergraph association matrix and node feature matrix samples into the initial hypergraph transition matrix generator for processing, and the output obtains the first hypergraph The second hypergraph transition matrix; based on the random walk principle on the hypergraph, determine the associated node set N real (v) obtained based on the structural connection matrix sample for each node corresponding to the training sample and the association obtained based on the second hypergraph transition matrix Node set N fake (v); use the discriminator to judge N real (v) and N fake (v) of each node, and obtain the judgment result corresponding to the training sample; according to the judgment result and preset corresponding to each training sample
  • the loss function is iteratively trained
  • an embodiment of the present application provides a terminal device, including: a memory and a processor, where the memory is used to store a computer program; the processor is used to execute the first aspect or any implementation manner of the first aspect when the computer program is invoked the method described.
  • embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the method described in the first aspect or any implementation manner of the first aspect.
  • an embodiment of the present application provides a computer program product that, when the computer program product runs on a processor, causes the processor to execute the method described in any one of the foregoing first aspects.
  • FIG. 1 is a flowchart of an embodiment of an image-driven brain atlas construction method provided by the application
  • FIG. 2 is a network structure of a hypergraph conversion matrix generator provided by an embodiment of the present application.
  • Fig. 3 is a kind of dynamic hypergraph generation confrontation network (Dynamic Hyper Graph GAN, DHGGAN) provided by the embodiment of this application;
  • FIG. 4 is a schematic structural diagram of an image-driven brain atlas construction apparatus provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • brain atlases are mostly generated based on single-modal image data, which can express less feature information.
  • the structural connectivity matrix generated based on DTI data can express the structural connectivity relationship between brain regions, but cannot express the functional connectivity relationship between brain regions.
  • the node feature matrix generated based on rs-fMRI data can express the functional connectivity information between brain regions, but cannot express the structural connectivity relationship between brain regions.
  • the pathological characteristics are more complex, and it is impossible to accurately reflect disease information from single-modality imaging data. Therefore, based on the current brain atlas, effective disease information may not be extracted, resulting in misjudgment of disease diagnosis.
  • the present application provides an image-driven brain atlas construction method, which calculates a first hypergraph transition matrix by converting a node feature matrix into a hypergraph data structure.
  • the first hypergraph transition matrix is the constructed brain atlas.
  • a hypergraph is a data structure, which is usually expressed by a hypergraph association matrix.
  • edges become hyperedges, which can connect multiple nodes.
  • the hypergraph transition matrix is used to indicate the probability of each node in the hypergraph traveling to another node, that is, it can be understood that the hypergraph transition matrix is essentially used to describe the structural connection relationship between the various nodes in the hypergraph.
  • the node feature matrix includes the time series of multiple ROI voxel features of the brain.
  • the so-called time series is the blood oxygen saturation level data collected at d (d>1, d is an integer) consecutive sampling points by the ROI voxel feature.
  • the node feature matrix can also be called a functional connectivity matrix, which contains the functional connectivity information between each ROI voxel feature.
  • a node in the hypergraph is a ROI voxel feature.
  • the first hypergraph transition matrix calculated based on the node feature matrix can describe the structural connection relationship between multiple ROI voxel features, and has a certain similarity with the structural connection matrix generated based on the brain DTI data. That is to say, the first hypergraph transition matrix includes not only functional connection information between multiple ROI voxel features, but also structural connection relationships between multiple ROI voxel features.
  • a multimodal brain map can be constructed by transforming the node feature matrix into a hypergraph data structure and calculating the first hypergraph transition matrix.
  • the multimodal brain connectivity structures can express more feature information.
  • the execution subject of the method may be an image data acquisition device, such as positron emission computed tomography ( Positron Emission Computed Tomography, PET) equipment, Computed Tomography (Computed Tomography, CT) equipment, Magnetic Resonance Imaging (Magnetic Resonance Imaging, MRI) equipment and other terminal equipment. It can also be a control device of an image data acquisition device, a computer, a robot, a mobile terminal and other terminal devices. As shown in Figure 1, the method includes:
  • the node feature matrix may be generated based on image data of at least one modality.
  • the image data may include rs-fMRI images, or include MRI images and rs-fMRI images.
  • DPARSF software and GRETNA software can be used to preprocess the input MRI images and rs-fMRI images, and then the preprocessed data can be mapped to automatic anatomical labels (Anatomical Labels).
  • Automatic Labeling, AAL Automatic Labeling, AAL template to obtain time series of multiple ROI voxel features.
  • a ROI voxel feature is a brain region divided by the AAL template.
  • the ROI voxel features will be referred to as nodes.
  • the time series of each node is taken as a column vector to form a node feature matrix X V of multiple nodes, where X V is a real matrix of size
  • K-NN k-nearest neighbor classification
  • K-means clustering algorithm can be directly used to convert the hypergraph data structure to obtain the first hypergraph. Correlation matrix.
  • the node feature matrix may be first transformed into the first hypergraph data using the Guth-Katz polynomial clustering algorithm.
  • the Guth-Katz polynomial clustering algorithm is about to be used to cluster the time series in the node feature matrix, and each class obtained can be used as a hyperedge of the first hypergraph data.
  • Each node in the node feature matrix is a node in the first hypergraph data, thereby constructing the first hypergraph data.
  • the KNN algorithm is used to transform the node feature matrix into the second hypergraph data.
  • the first hypergraph data and the second hypergraph data are fused to obtain the first hypergraph correlation matrix.
  • the first hypergraph correlation matrix can be better preserved.
  • the node characteristic matrix is used for topology information between nodes, and redundant interference information is effectively eliminated, and the accuracy of the first hypergraph association matrix is improved.
  • a schematic diagram of the network structure of the hypergraph transition matrix generator provided by this application includes several block layers connected in series and an output layer.
  • each block layer includes a first sub-network layer, a first activation layer, a second sub-network layer, and a second activation layer.
  • the first sub-network layer is a d ⁇ d neural network, which receives the input node feature matrix X V for operation, and outputs the operation result to the first activation layer, and uses the sigmoid activation function to activate the operation result to obtain the hyperedge feature matrix.
  • X'E the expressions of the first sub-network layer and the first activation layer are:
  • WE and b E are the network parameters that can be learned in the first sub-network layer, WE is a d ⁇ d weight matrix, and b E is a d ⁇ 1 deviation vector.
  • X′ E is input into the second sub-network layer.
  • the second sub-network layer is a d ⁇ d neural network, which receives the input X′ E for operation, outputs the operation result to the second activation layer, and uses the sigmoid activation function to activate the operation result to obtain a new node feature matrix X' V .
  • W V and b V are network parameters that can be learned in the second sub-network layer, W V is a d ⁇ d weight matrix, and b V is a d ⁇ 1 bias vector.
  • the block layer where X' V is located can be output.
  • block1 receives X V input from the hypergraph transition matrix generator, and calculates a hyperedge feature matrix X′ E , and uses X′ E to calculate a new node feature matrix X′ V , and input X′ V into block2 .
  • block2 receives X' V , it uses X' V to calculate a new X' E , uses X' E to calculate a new node feature matrix X" V , and inputs X" V to block3.
  • block6 as the last layer will use the node characteristic matrix X′′′′ V output by block5 to calculate a new hyperedge characteristic matrix X′′ E , and then use X′′ E to calculate a new node characteristic matrix X′′′′ V , Then output X′′ E and X′′′′ V as the target hyperedge feature matrix and target node feature matrix, and input them to the output layer.
  • the number of block layers of the hypergraph transition matrix generator is not limited in this application, and can be set based on actual model performance. By setting several block layers and using the hypergraph edge neuron algorithm to alternately learn the hyperedge feature matrix and the node feature matrix, unnecessary parameters can be reduced, model overfitting can be avoided, and the generalization ability of the model can be enhanced.
  • WR is the learnable network parameter in the output layer
  • WR is a weight matrix of d ⁇
  • the expression is: in, and are the learnable network parameters in the output layer, is a d-dimensional vector, is an
  • the calculated hyperedge weight matrix W and node weight matrix R are substituted into the hypergraph transition probability formula to calculate the first hypergraph transition matrix P.
  • the expression is: Among them, D V is the vertex degree of freedom, and D E is the hyperedge degree of freedom.
  • the calculated first hypergraph transition matrix P indicates the probability of each node in A H swimming to another node, that is, the first hypergraph transition matrix is essentially used for Describe the structural connection relationship between each node in AH.
  • Each node in AH corresponds to each node (ie, ROI voxel feature) indicated by the node feature matrix. Therefore, the first hypergraph transition matrix P has a certain similarity with the structural connection matrix generated based on the brain DTI data. It can describe the structural connection relationship between multiple ROI voxel features.
  • the first hypergraph transition matrix P is calculated based on the node feature matrix X V , so the first hypergraph transition matrix P also includes functional connection information between multiple ROI voxel features.
  • the brain atlas is a multi-modal brain atlas constructed based on the node feature matrix.
  • the so-called multi-modality refers to having both functional connectivity information and structural connectivity information.
  • the brain atlas can be applied to the classification and discrimination tasks of any brain disease diagnosed based on medical images.
  • a classification network is trained for Alzheimer's disease (AD), for example, the classification network is a Support Vector Machine (SVM), which predicts the subjects from mild to mild by inputting a brain map. Probability of cognitive impairment developing into Alzheimer's disease. Compared with the existing brain atlas obtained based on single-modality image data, more disease information can be identified from the multi-modal brain atlas provided in this application, which can improve the SVM's diagnosis of Alzheimer's disease. prediction accuracy.
  • SVM Support Vector Machine
  • DHGGAN Dynamic Hyper Graph GAN
  • DHGGAN includes an initial hypergraph transition matrix generator, a random walk module, and a hypergraph random sampling discriminator as shown in Figure 2.
  • the training process can be entered as follows:
  • the preset training set includes a plurality of training samples, and each training sample includes at least two different modalities of the image data samples. By preprocessing image data samples of at least two different modalities, node connection matrix samples and structural connection matrix samples are obtained.
  • the training samples include rs-fMRI image samples, MRI image samples, and DTI image samples.
  • DPARSF software and GRETNA software to preprocess the input MRI image samples and rs-fMRI image samples, and then map the preprocessed data to the AAL template, the time series of multiple nodes v are obtained. Take the time series of each node v as a column vector, thus constituting the node feature matrix sample of the training sample.
  • the DTI image samples are input into PANDA software for preprocessing, and the data obtained after preprocessing is input into the AAL template to construct a structural connection matrix sample T between multiple nodes.
  • the specific processing procedure of the initial hypergraph transition matrix generator for the second hypergraph association matrix and the node feature matrix samples may refer to the above S103, which will not be repeated here.
  • the walking path from v 0 to vm is determined as (v 1 , v 2 , . . . vm -1 ).
  • v 0 ) of walking from v 0 to v m is determined as follows:
  • v j ) represents the probability of walking from node v j to node v i based on the structural connection matrix T, and the calculation formula of p T (v i
  • the walking path from v 0 to vm is (v 1 , v 2 , . . . vm -1 ).
  • v 0 ) of walking from v 0 to v m is (v 1 , v 2 , . . . vm -1 ).
  • v j ) represents the probability of walking from node v j to node v i based on the structural connection matrix P, and the calculation formula of p P (v i
  • n (n ⁇ 1, n is an integer) nodes with the largest walk probability are sampled based on the structural connection matrix T as associated nodes, denoted as N real (v).
  • N fake (v) nodes with the largest walking probability are sampled as associated nodes, denoted as N fake (v).
  • the hypergraph random sampling discriminator may include a multi-layer perceptron M, for each node v r , the corresponding time series in the node feature matrix sample X V As feature information, it is input into the multi-layer perceptual machine M for processing, and the output features are obtained.
  • the probability D( v a , v b ) of v a being an associated node of v b can be determined according to the following formula.
  • the calculated loss value L(G, D) does not meet the preset conditions, modify the network parameters in the decider and the hypergraph transition matrix generator, such as W E , b E , W V , b V , and WR of the output layer, Wait. Then select a new training sample, restart S2, and enter the next round of training until the model converges, that is, the calculated loss value L(G, D) meets the preset condition.
  • the preset condition can be that the loss value tends to level off.
  • the hypergraph transition matrix generator can generate a first hypergraph transition matrix that has a certain degree of similarity with the actual structural connection matrix based on the node feature matrix. So that when the first hypergraph transition matrix is used as a brain atlas, the brain atlas has both functional connection information, structural connection information, and complementary information between functional connection information and structural connection information, which can provide more feature information and improve follow-up. Disease diagnosis efficiency.
  • the random walk principle on the hypergraph is used to replace the kernel function transformation and the Lagrangian multiplier method in the traditional training method, thereby avoiding the complicated calculation in the non-convex optimization problem and the inability of the loss function to be stable globally.
  • the problem of optimal point convergence therefore, improves the robustness of the model as well as the training efficiency.
  • the embodiment of the present application provides an image-driven brain atlas construction device, and the device embodiment corresponds to the foregoing method embodiment.
  • the details in the foregoing method embodiments are described one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the content in the foregoing method embodiments.
  • FIG. 4 is a schematic structural diagram of an image-driven brain atlas construction apparatus provided by an embodiment of the present application.
  • the construction apparatus provided by this embodiment includes an acquisition unit 401 , a conversion unit 402 , and a construction unit 403 .
  • the obtaining unit 401 is configured to obtain a node feature matrix, where the node feature matrix includes a time series of multiple nodes of the brain.
  • the conversion unit 402 is configured to perform hypergraph data structure conversion on the node feature matrix to obtain a first hypergraph association matrix.
  • the construction unit 403 is configured to input the first hypergraph association matrix and the node feature matrix into the trained hypergraph transition matrix generator for processing, and output the first hypergraph transition matrix, the first hypergraph transition matrix Multimodal brain atlas constructed from matrix representations.
  • the obtaining unit 401 is further configured to preprocess the training samples for each training sample in the training set to obtain node feature matrix samples and structural connection matrix samples.
  • the conversion unit 402 is further configured to perform hypergraph data structure conversion on the node feature matrix samples to obtain a second hypergraph association matrix.
  • the construction device further includes a training unit 404, and the training unit 404 is used for:
  • the construction apparatus provided in this embodiment can execute the above method embodiments, and the implementation principle and technical effect thereof are similar, and details are not described herein again.
  • FIG. 5 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device provided by this embodiment includes: a memory 501 and a processor 502, where the memory 501 is used to store a computer program; the processor 502 is used for The methods described in the above method embodiments are executed when the computer program is invoked.
  • the terminal device provided in this embodiment may execute the foregoing method embodiments, and the implementation principle and technical effect thereof are similar, and details are not described herein again.
  • Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.
  • the embodiments of the present application further provide a computer program product, when the computer program product runs on a terminal device, the terminal device executes the method described in the above method embodiments.
  • the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program to instruct the relevant hardware.
  • the computer program can be stored in a computer-readable storage medium, and the computer program When executed by a processor, the steps of each of the above method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like.
  • the computer-readable storage medium may include at least: any entity or device capable of carrying the computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access Memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium.
  • computer readable media may not be electrical carrier signals and telecommunications signals.
  • the disclosed apparatus/device and method may be implemented in other manners.
  • the apparatus/equipment embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the term “if” may be contextually interpreted as “when” or “once” or “in response to determining” or “in response to detecting “.
  • the phrases “if it is determined” or “if the [described condition or event] is detected” may be interpreted, depending on the context, to mean “once it is determined” or “in response to the determination” or “once the [described condition or event] is detected. ]” or “in response to detection of the [described condition or event]”.
  • references in this specification to "one embodiment” or “some embodiments” and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically emphasized otherwise.
  • the terms “including”, “including”, “having” and their variants mean “including but not limited to” unless specifically emphasized otherwise.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

本申请提供一种影像驱动的脑图谱构建方法、装置、设备及存储介质，涉及医疗影像技术领域。该方法包括：获取节点特征矩阵，节点特征矩阵包括脑部的多个节点的时间序列；对节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵；将第一超图关联矩阵和节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，第一超图转移矩阵表征构建的多模态脑图谱。本申请提供的技术方案可以构建一个多模态脑图谱，该多模态脑连接结构能够表达更多的特征信息。当应用于脑疾病诊断过程时，可以提高疾病诊断的准确率。

Description

影像驱动的脑图谱构建方法、装置、设备及存储介质
本申请要求于2021年1月9日在中国专利局提交的、申请号为202110026745.5的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及医疗影像技术领域，尤其涉及一种影像驱动的脑图谱构建方法、装置、设备及存储介质。
背景技术
随着医疗影像技术发展，基于不同模态下所采集的影像数据可以生成不同的脑图谱。例如，可以通过采集弥散张量成像(Diffusion Tensor Imaging，DTI)数据生成结构连接矩阵(包含脑部的各个脑区之间纤维束密度，也成为结构连接图谱)能够表达脑部的组织学解剖结构信息。通过采集脑部静息态功能磁共振成像(resting-state functional Magnetic Resonance Imaging，rs-fMRI)数据生成节点特征矩阵(包括各个脑区的时间特征序列，也可以称为功能连接图谱)，能够表达各个脑区之间的功能连接信息。脑图谱在针对脑部疾病诊断的过程是非常重要的参考依据。
然而目前的脑图谱大多是基于单模态的影像数据生成的，所能表达的特征信息较少。因此，可能会导致无法从脑图谱中提取有效的疾病信息，从而造成疾病诊断的误判。
技术问题
有鉴于此，本申请实施例提供一种影像驱动的脑图谱构建方法、装置、设备及存储介质，能够解决目前脑图谱所能表达的特征信息较少的问题。
技术解决方案
为了实现上述目的，第一方面，本申请实施例提供一种影像驱动的脑图谱构建方法，包括：获取节点特征矩阵，节点特征矩阵包括脑部的多个节点的时间序列；对节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵；将第一超图关联矩阵和节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，第一超图转移矩阵表征构建的多模态脑图谱。
采用本申请提供的构建方法，通过将节点特征矩阵转化成超图数据结构，并计算第一超图转移矩阵，即可构建一个多模态脑图谱。相比于目前基于单模态的影像数据生成脑连接结构，该多模态脑连接结构能够表达更多的特征信息。当应用于脑疾病诊断过程时，可以提高疾病诊断的准确率。
可选的，对时间序列特进行超图数据结构转换，得到第一超图关联矩阵，包括：利用Guth-Katz多项式聚类算法将节点特征矩阵转换为第一超图数据；利用KNN算法将节点特征矩阵转换为第二超图数据；将第一超图数据和第二超图数据进行融合，得到第一超图关联矩阵。
基于该可选方式，通过利用Guth-Katz多项式聚类算法构建第一超图数据，然后将第一超图数据与第二超图数据融合来得到第一超图关联矩阵，能够更好的保留了节点特征矩阵种节点之间的拓扑信息，同时有效排除了冗余的干扰信息，提高了第一超图关联矩阵的精确性。
可选的，超图转移矩阵生成器对节点特征矩阵的处理包括：基于第一超图关联矩阵和节点特征矩阵进行若干次迭代计算，得到目标超边特征矩阵和目标节点特征矩阵；根据第一超图关联矩阵和目标节点特征矩阵确定节点权重矩阵；根据目标超边特征矩阵确定超边权重矩阵；根据超边权重矩阵和节点权重矩阵确定第一超图转移矩阵。
基于该可选方式，通过第一超图关联矩阵和节点特征矩阵进行若干次迭代计算，得到目标超边特征矩阵和目标节点特征矩阵。利用这种超图边神经元算法，能够充分挖掘不同模态数据之间的互补信息和潜在联系，减少不必要的参数，从而避免模型过拟合且泛化能力强，更加适合小样本学习。
可选的，针对训练集中的每个训练样本，对训练样本进行预处理，得到节点特征矩阵样本和结构连接矩阵样本；对节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵；将第二超图关联矩阵和节点特征矩阵样本输入初始超图转移矩阵生成器中处理，输出得到第二超图转移矩阵；基于超图上的随机游走原理，确定训练样本对应的每个节点基于结构连接矩阵样本得到的关联节点集合N real(v)和基于第二超图转移矩阵得到的关联节点集合N fake(v)；利用判别器对每个节点的N real(v)和N fake(v)进行判决，得到训练样本对应的判决结果；根据每个训练样本对应的判别结果和预设的损失函数进行迭代训练，得到超图转移矩阵生成器。
基于该可选方式，利用超图上的随机游走原理代替传统训练方法中核函数变换和拉格朗日乘子法，从而避免了非凸优化问题中的繁杂计算以及损失函数不能稳定向全局最优点收敛的问题，因此，提高了模型的鲁棒性以及训练效率。
可选的，利用判别器对每个节点的N real(v)和N fake(v)进行判决，得到训练样本对应的判决结果，包括：针对每个节点，将节点在节点特征矩阵样本中对应的时间序列输入预设 的多层感知机中处理，得到输出特征；根据如下公式计算节点与节点的N real(v)和N fake(v)中的每个节点的判决结果：
Figure PCTCN2021075798-appb-000001
其中，D(v a,v b)表示节点v a是v b的关联节点的判别结果，
Figure PCTCN2021075798-appb-000002
表示节点v a的输出特征，
Figure PCTCN2021075798-appb-000003
表示节点v b的输出特征。
可选的，损失函数为：
Figure PCTCN2021075798-appb-000004
其中，|V|表示节点数。
第二方面，本申请提供一种影像驱动的脑图谱构建装置，包括：获取单元，用于获取节点特征矩阵，节点特征矩阵包括脑部的多个节点的时间序列；转换单元，用于对节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵；构建单元，用于将第一超图关联矩阵和节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，第一超图转移矩阵表征构建的多模态脑图谱。
可选的，获取单元，用于针对训练集中的每个训练样本，对训练样本进行预处理，得到节点特征矩阵样本和结构连接矩阵样本；转换单元，还用于对节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵；构建装置还包括训练单元，训练单元用于：将第二超图关联矩阵和节点特征矩阵样本输入初始超图转移矩阵生成器中处理，输出得到第二超图转移矩阵；基于超图上的随机游走原理，确定训练样本对应的每个节点基于结构连接矩阵样本得到的关联节点集合N real(v)和基于第二超图转移矩阵得到的关联节点集合N fake(v)；利用判别器对每个节点的N real(v)和N fake(v)进行判决，得到训练样本对应的判决结果；根据每个训练样本对应的判别结果和预设的损失函数进行迭代训练，得到超图转移矩阵生成器。
第三方面，本申请实施例提供一种终端设备，包括：存储器和处理器，存储器用于存储计算机程序；处理器用于在调用计算机程序时执行如第一方面或第一方面的任一实施方式所述的方法。
第四方面，本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述第一方面或第一方面的任一实施方式所述的方法。
第五方面，本申请实施例提供一种计算机程序产品，当计算机程序产品在处理器上运行时，使得处理器执行上述第一方面中任一项所述的方法。
可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
附图说明
图1为本申请提供的一种影像驱动的脑图谱构建方法的一个实施例的流程图；
图2为本申请实施例提供的一种超图转换矩阵生成器的网络结构；
图3为本申请实施例提供的一种动态超图生成对抗网络(Dynamic Hyper Graph GAN，DHGGAN)；
图4为本申请实施例提供的一种影像驱动的脑图谱构建装置的结构示意图；
图5为本申请实施例提供的终端设备的结构示意图。
本发明的实施方式
目前，脑图谱大多是基于单模态的影像数据生成的，所能表达的特征信息较少。例如基于DTI数据生成的结构连接矩阵，能够表达脑部的各个脑区之间的结构连接关系，却无法表达各个脑区之间的功能连接关系。基于rs-fMRI数据生成的节点特征矩阵可以表达各个脑区之间的功能连接信息，却无法表达各个脑区之间的结构连接关系。而对于一些慢性神经性脑疾病，例如阿尔兹海默症，其病理特征较为复杂，无法从单模态的影像数据中准确反应疾病信息。因此，基于目前的脑图谱，可能无法提取有效的疾病信息，从而造成疾病诊断的误判。
针对这一问题，本申请提供一种影像驱动的脑图谱构建方法，通过将节点特征矩阵转化成超图数据结构，进而计算第一超图转移矩阵。该第一超图转移矩阵即为构建的脑图谱。
其中，超图(Hypergraph)是一种数据结构，通常采用超图关联矩阵来表达。在超图数据结构中边成为超边，可以连接多个节点。超图转移矩阵是用来指示超图中的每一个节点游走到另一个节点的概率，即可以理解为超图转移矩阵本质上用于描述超图中各个节点之间的结构连接关系。
而节点特征矩阵包括脑部的多个ROI体素特征的时间序列。所谓时间序列为ROI体素特征在d(d＞1，d为整数)个连续的采样点采集到的血氧饱和水平数据。节点特征矩阵也可以称为功能连接矩阵，包含各个ROI体素特征之间的功能连接信息。
那么，当将节点特征矩阵转换成超图数据格式后，超图中的一个节点即为一个ROI体素特征。基于节点特征矩阵计算得到的第一超图转移矩阵可以描述多个ROI体素特征之间的结构连接关系，与基于脑部DTI数据生成的结构连接矩阵具备一定的相似性。也就是说， 该第一超图转移矩阵不仅包含多个ROI体素特征之间的功能连接信息，包含多个ROI体素特征之间的结构连接关系。
因此，在申请中，通过将节点特征矩阵转化成超图数据结构，并计算第一超图转移矩阵，即可构建一个多模态脑图谱。相比于目前基于单模态的影像数据生成脑连接结构，该多模态脑连接结构能够表达更多的特征信息。当应用于脑疾病诊断过程时，可以提高疾病诊断的准确率。
下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。
如图1所示，为本申请提供的一种影像驱动的脑图谱构建方法的一个实施例的流程图，该方法的执行主体可以是影像数据采集设备，例如正电子发射型计算机断层显像(Positron Emission Computed Tomography，PET)设备、电子计算机断层扫描(Computed Tomography，CT)设备、磁共振成像(Magnetic Resonance Imaging，MRI)设备等终端设备。还可以是影像数据采集设备的控制设备、计算机、机器人、移动终端等终端设备。如图1所示，该方法包括：
S101，获取节点特征矩阵，节点特征矩阵包括脑部的多个节点的时间序列。
其中，节点特征矩阵可以是基于至少一种模态的影像数据生成的。示例性的，假设由MRI设备采集用于构建节点特征矩阵的图像，该影像数据可以包括rs-fMRI图像，或者包括MRI图像和rs-fMRI图像。
例如，采集到MRI图像和rs-fMRI图像后，可以先利用DPARSF软件以及GRETNA软件对输入的MRI图像和rs-fMRI图像进行预处理，然后将预处理后得到的数据映射到自动解剖标签(Anatomical Automatic Labeling，AAL)模板，得到多个ROI体素特征的时间序列。其中，一个ROI体素特征即为AAL模板所划分出来的一个脑区。下文中将把ROI体素特征称为节点。
得到多个节点的时间序列后，将每个节点的时间序列作为一个列向量，从而构成多个节点的节点特征矩阵X V，X V是一个大小为|V|×d的实数矩阵。其中，|V|表示节点数，d表示节点的时间序列长度。
S102，对节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵。
示例性的，在本申请中可以直接采用k近邻分类(k-nearest neighbor classification，K-NN)算法、K-means聚类算法等常用的方法进行超图数据结构的转换，得到第一超图关联矩阵。
在一个示例中，可以先利用Guth-Katz多项式聚类算法将所述节点特征矩阵转换为第 一超图数据。即将利用Guth-Katz多项式聚类算法对节点特征矩阵中的时间序列进行聚类，每得到一类，即可作为第一超图数据的一个超边。节点特征矩阵中每一个节点即为第一超图数据中的一个节点，从而构建第一超图数据。然后利用KNN算法将节点特征矩阵转换为第二超图数据。最后将第一超图数据和第二超图数据进行融合，得到第一超图关联矩阵。
在该示例中，通过利用Guth-Katz多项式聚类算法构建第一超图数据，然后将第一超图数据与第二超图数据融合来得到第一超图关联矩阵，能够更好的保留了节点特征矩阵种节点之间的拓扑信息，同时有效排除了冗余的干扰信息，提高了第一超图关联矩阵的精确性。
S103，将第一超图关联矩阵A H和节点特征矩阵X V输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，第一超图转移矩阵表征构建的多模态脑图谱。
如图2所示，为本申请提供的超图转移矩阵生成器的网络结构示意图，包括若干个串联的block层和一个输出层。
其中，每个block层包括第一子网络层、第一激活层、第二子网络层、第二激活层。第一子网络层为一个d×d神经网络，接收输入的节点特征矩阵X V进行运算，并将运算结果输出至第一激活层，利用sigmoid激活函数对运算结果进行激活，得到超边特征矩阵X′ E。具体的，第一子网络层和第一激活层的表达式为：
Figure PCTCN2021075798-appb-000005
其中，W E和b E为第一子网络层中可学习的网络参数，W E为一个d×d的权重矩阵，b E为一个d×1的偏差向量。
第一激活层得到超边特征矩阵X′ E后，将X′ E输入第二子网络层。第二子网络层为一个d×d神经网络，接收输入的X′ E进行运算，并将运算结果输出至第二激活层，利用sigmoid激活函数对运算结果进行激活，得到一个新的节点特征矩阵X′ V。具体的，第二子网络层和第二激活层的表达式为：X′ V＝σ(A HX′ EW V+b V)。
其中，W V和b V为第二子网络层中可学习的网络参数，W V为一个d×d的权重矩阵，b V为一个d×1的偏差向量。
第二激活层得到新的节点特征矩阵X′ V后，即可将X′ V输出所在的block层。
可以理解的是，若干个block层用于基于输入的第一超图关联矩阵和节点特征矩阵进行若干次迭代计算，得到目标超边特征矩阵和目标节点特征矩阵。
例如，图2中示出了6个block层。其中，block1接收由超图转移矩阵生成器外界输入的X V，计算得到一个超边特征矩阵X′ E，在利用X′ E计算一个新节点特征矩阵X′ V，并把X′ V输入block2。同样的，block2接收到X′ V后，利用X′ V计算一个新的X′ E，在利用X′ E计算一个新的节点特征矩阵X″ V，并把X″ V输入block3。依次迭代计算，block6作为最后一层将利用block5输出的节点特征矩阵X″″ V计算一个新的超边特征矩阵X″ E，然后利用X″ E计算一个新的节点特征矩阵X″″ V，然后输出X″ E和X″″ V作为目标超边特征矩阵和目标节点特征矩阵，并输入至输出层。
可以理解的是，超图转移矩阵生成器的block层数本申请不做限制，可以基于实际模型性能进行设置。通过设置若干层block层，利用超图边神经元算法交替学习超边特征矩阵和节点特征矩阵，可以减少不必要的参数，避免模型过拟合，增强模型的泛化能力。
输出层接收到目标超边特征矩阵X″ E和目标节点特征矩阵X″″ V后，即可根据第一超图关联矩阵A H和目标节点特征矩阵X″″ V确定节点权重矩阵R，表达式为：R＝(X″″ VW R)⊙A H。其中，W R为输出层中可学习的网络参数，W R为一个d×|E|的权重矩阵，其中E为超边的条数。
然后根据目标超边特征矩阵X″ E确定超边权重矩阵W，表达式为：
Figure PCTCN2021075798-appb-000006
其中，
Figure PCTCN2021075798-appb-000007
和
Figure PCTCN2021075798-appb-000008
均为输出层中可学习的网络参数，
Figure PCTCN2021075798-appb-000009
是d维向量，
Figure PCTCN2021075798-appb-000010
是|E|维向量。
之后，将计算出来的超边权重矩阵W和节点权重矩阵R，代入超图转移概率公式中计算得到第一超图转移矩阵P，表达式为：
Figure PCTCN2021075798-appb-000011
其中，D V为顶点自由度，D E为超边自由度。
对于第一超图关联矩阵A H来说，计算出来的第一超图转移矩阵P指示A H中的每一个节点游走到另一个节点的概率，即第一超图转移矩阵本质上用于描述A H中各个节点之间的结构连接关系。而A H中各个节点与节点特征矩阵所指示的各个节点(即ROI体素特征)对应，因此，第一超图转移矩阵P与基于脑部DTI数据生成的结构连接矩阵具备一定的相似性，能够描述多个ROI体素特征之间的结构连接关系。且，第一超图转移矩阵P是基于节点特征矩阵X V计算得来的，因此第一超图转移矩阵P还包含多个ROI体素特征之间的功能连接信息。那么，将第一超图转移矩阵P作为脑图谱时，该脑图谱即为基于节点特征 矩阵构建的一个多模态脑图谱，所谓多模态是指同时具备功能连接信息和结构连接信息。
得到多模态脑图谱后，即可将该脑图谱应用于与任何基于医学图像进行诊断的脑部疾病的分类及判别任务。
示例性的，针对阿尔茨海默症(Alzheimer’s disease,AD)训练有分类网络，例如，该分类网络是支持向量机(Support Vector Machine,SVM)，通过输入脑图谱来预测被试者从轻度认知障碍发展为阿尔茨海默病的概率。相比于使用现有的基于单模态的影像数据获取的脑图谱来说，从本申请提供的多模态脑图谱中可以识别出更多的疾病信息，能够提高SVM对阿尔茨海默病的预测准确度。
针对上述超图转移矩阵生成器，本申请还提供一种动态超图生成对抗网络(Dynamic Hyper Graph GAN，DHGGAN)来实现超图转移矩阵生成器的训练。如图3所示，DHGGAN包括如图2所示的初始超图转移矩阵生成器、随机游走模块和超图随机采样判别器。训练过程可以入下所示：
S1、多模态数据预处理。
预设的训练集中包括多个训练样本，每个训练样本包括至少两种不同模态的该影像数据样本。通过对至少两种不同模态的影像数据样本进行预处理，得到节点连接矩阵样本和结构连接矩阵样本。
示例性的，训练样本包括rs-fMRI图像样本、MRI图像样本和DTI图像样本。通过将先利用DPARSF软件以及GRETNA软件对输入的MRI图像样本和rs-fMRI图像样本进行预处理，然后将预处理后得到的数据映射到AAL模板，得到多个节点v的时间序列。将每个节点v的时间序列作为一个列向量，从而构成该训练样本的节点特征矩阵样本。
对于DTI图像样本，DTI图像样本输入PANDA软件进行预处理，并将预处理后得到的数据输入AAL模板中，构建多个节点之间结构连接矩阵样本T。
S2，对节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵。
S3，将第二超图关联矩阵和节点特征矩阵样本输入到初始超图转移矩阵生成器中处理，得到第二超超图转移矩阵。
初始超图转移矩阵生成器对第二超图关联矩阵和节点特征矩阵样本的具体处理过程可以参见上述S103，此处不再赘述。
S4，将第二超图转移矩阵P和结构连接矩阵样本T输入到随机游走模块，确定每个节点基于结构连接矩阵样本T得到的关联节点集合N real(v)和基于第二超图转移矩阵P得到的关联节点集合N fake(v)。
示例性的，以从v 0游走至v m为例。
基于结构连接矩阵T确定从v 0游走至v m的游走路径为(v 1，v 2，……v m-1)。按照如下公式计算从v 0游走至v m的概率G T(v m|v 0)：
Figure PCTCN2021075798-appb-000012
其中，p T(v i|v j)表示基于结构连接矩阵T，从节点v j游走至节点v i的概率，p T(v i|v j)的计算公式如下：
Figure PCTCN2021075798-appb-000013
基于第二超图转移矩阵P确定从v 0游走至v m的游走路径为(v 1，v 2，……v m-1)。按照如下公式计算从v 0游走至v m的概率G H(v m|v 0)：
Figure PCTCN2021075798-appb-000014
其中，p P(v i|v j)表示基于结构连接矩阵P，从节点v j游走至节点v i的概率，p P(v i|v j)的计算公式如下：
Figure PCTCN2021075798-appb-000015
对图中每个节点v r，按照上述随机游走概率的计算方式，基于结构连接矩阵T采样到n(n≥1，n为整数)个游走概率最大的节点作为关联节点，记作N real(v)。基于第二超图转移矩阵P采样到n个游走概率最大的节点作为关联节点，记作N fake(v)。
S5，将每个节点关联节点集合N real(v)和N fake(v)输入超图随机采样判别器，进行判决。
示例性的，超图随机采样判别器可以包括多层感知机M，对于每个节点v r，将节点特征矩阵样本X V中对应的时间序列
Figure PCTCN2021075798-appb-000016
作为特征信息输入多层感知机器M中处理，得到输出特征
Figure PCTCN2021075798-appb-000017
对于任意两个节点v a和v b，可以按照如下公式确定v a是v b的关联节点的概率 D(v a,v b)。
Figure PCTCN2021075798-appb-000018
对于每个节点v r，基于上述公式，计算该节点的关联节点集合N real(v)和N fake(v)中的每个节点是该节点的关联节点的概率，得到判决结果。
S6，基于超图随机采样判别器的判决结果，进行模型损失计算。
将判决结果代入以下损失函数，计算DHGGAN模型的损失值L(G,D)：
Figure PCTCN2021075798-appb-000019
如果计算得到的损失值L(G,D)不满足预设条件，则修改判决器和超图转移矩阵生成器中的网络参数，例如每个block层中的W E、b E、W V、b V，以及输出层的W R、
Figure PCTCN2021075798-appb-000020
等。然后选择新的训练样本，重新开始执行S2，进入下一轮训练，直至模型收敛，即计算得到的损失值L(G,D)满预预设条件。预设条件可以为损失值趋于平稳。
值得说明的是，基于模型收敛体条件，不同block层中的W E、b E、W V、b V最终可以被调整为不同或者相同的参数。且随着模型的收敛，第二超图转移矩阵P越来越相似于结构连接矩阵T。也就是说，训练完成后，超图转移矩阵生成器能够基于节点特征矩阵生成与实际的结构连接矩阵具备一定相似度的第一超图转移矩阵。以使得将第一超图转移矩阵作为脑图谱时，该脑图谱同时具备功能连接信息、结构连接信息以及功能连接信息与结构连接信息之间的互补信息，能够提供更多的特征信息，提高后续疾病诊断效率。
在本申请实施例中，利用超图上的随机游走原理代替传统训练方法中核函数变换和拉格朗日乘子法，从而避免了非凸优化问题中的繁杂计算以及损失函数不能稳定向全局最优点收敛的问题，因此，提高了模型的鲁棒性以及训练效率。
基于同一发明构思，作为对上述方法的实现，本申请实施例提供了一种影像驱动的脑图谱构建装置，该装置实施例与前述方法实施例对应，为便于阅读，本装置实施例不再对前述方法实施例中的细节内容进行逐一赘述，但应当明确，本实施例中的装置能够对应实现前述方法实施例中的全部内容。
图4为本申请实施例提供的影像驱动的脑图谱构建装置的结构示意图，如图4所示，本实施例提供的构建装置包括：获取单元401、转换单元402、构建单元403。
其中，获取单元401，用于获取节点特征矩阵，所述节点特征矩阵包括所述脑部的多个节点的时间序列。
转换单元402，用于对所述节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵。
构建单元403，用于将所述第一超图关联矩阵和所述节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，所述第一超图转移矩阵表征构建的多模态脑图谱。
可选的，所述获取单元401，还用于针对训练集中的每个训练样本，对所述训练样本进行预处理，得到节点特征矩阵样本和结构连接矩阵样本。
所述转换单元402，还用于对所述节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵。
所述构建装置还包括训练单元404，所述训练单元404用于：
将所述第二超图关联矩阵和所述节点特征矩阵样本输入初始超图转移矩阵生成器中处理，输出得到第二超图转移矩阵；基于超图上的随机游走原理，确定所述训练样本对应的每个节点基于所述结构连接矩阵样本得到的关联节点集合N real(v)和基于第二超图转移矩阵得到的关联节点集合N fake(v)；利用判别器对所述每个节点的N real(v)和N fake(v)进行判决，得到所述训练样本对应的判决结果；根据每个训练样本对应的判别结果和预设的损失函数进行迭代训练，得到所述超图转移矩阵生成器。
本实施例提供的构建装置可以执行上述方法实施例，其实现原理与技术效果类似，此处不再赘述。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
基于同一发明构思，本申请实施例还提供了一种终端设备。图5为本申请实施例提供 的终端设备的结构示意图，如图5所示，本实施例提供的终端设备包括：存储器501和处理器502，存储器501用于存储计算机程序；处理器502用于在调用计算机程序时执行上述方法实施例所述的方法。
本实施例提供的终端设备可以执行上述方法实施例，其实现原理与技术效果类似，此处不再赘述。
本申请实施例还提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述方法实施例所述的方法。
本申请实施例还提供一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备执行时实现上述方法实施例所述的方法。
上述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读存储介质至少可以包括：能够将计算机程序代码携带到拍照装置/终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区，根据立法和专利实践，计算机可读介质不可以是电载波信号和电信信号。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点， 所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
应当理解，当在本申请说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本申请说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外，在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (10)

  1. 一种影像驱动的脑图谱构建方法，其特征在于，包括：
    获取节点特征矩阵，所述节点特征矩阵包括所述脑部的多个节点的时间序列；
    对所述节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵；
    将所述第一超图关联矩阵和所述节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，所述第一超图转移矩阵表征构建的多模态脑图谱。
  2. 根据权利要求1所述的方法，其特征在于，所述对所述时间序列特进行超图数据结构转换，得到第一超图关联矩阵，包括：
    利用Guth-Katz多项式聚类算法将所述节点特征矩阵转换为第一超图数据；
    利用KNN算法将所述节点特征矩阵转换为第二超图数据；
    将所述第一超图数据和所述第二超图数据进行融合，得到所述第一超图关联矩阵。
  3. 根据权利要求1所述的方法，其特征在于，所述超图转移矩阵生成器对所述节点特征矩阵的处理包括：
    基于所述第一超图关联矩阵和所述节点特征矩阵进行若干次迭代计算，得到目标超边特征矩阵和目标节点特征矩阵；
    根据所述第一超图关联矩阵和所述目标节点特征矩阵确定节点权重矩阵；
    根据所述目标超边特征矩阵确定超边权重矩阵；
    根据所述超边权重矩阵和所述节点权重矩阵确定所述第一超图转移矩阵。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    针对训练集中的每个训练样本，对所述训练样本进行预处理，得到节点特征矩阵样本和结构连接矩阵样本；
    对所述节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵；
    将所述第二超图关联矩阵和所述节点特征矩阵样本输入初始超图转移矩阵生成器中处理，输出得到第二超图转移矩阵；
    基于超图上的随机游走原理，确定所述训练样本对应的每个节点基于所述结构连接矩阵样本得到的关联节点集合N real(v)和基于第二超图转移矩阵得到的关联节点集合N fake(v)；
    利用判别器对所述每个节点的N real(v)和N fake(v)进行判决，得到所述训练样本对应的判决结果；
    根据每个训练样本对应的判别结果和预设的损失函数进行迭代训练，得到所述超图转 移矩阵生成器。
  5. 根据权利要求4所述的方法，其特征在于，
    所述利用判别器对所述每个节点的N real(v)和N fake(v)进行判决，得到所述训练样本对应的判决结果，包括：
    针对所述每个节点，将所述节点在所述节点特征矩阵样本中对应的时间序列输入预设的多层感知机中处理，得到输出特征；
    根据如下公式计算所述节点与所述节点的N real(v)和N fake(v)中的每个节点的判决结果：
    Figure PCTCN2021075798-appb-100001
    其中，D(v a,v b)表示节点v a是v b的关联节点的判别结果，
    Figure PCTCN2021075798-appb-100002
    表示节点v a的输出特征，
    Figure PCTCN2021075798-appb-100003
    表示节点v b的输出特征。
  6. 根据权利要求5所述的方法，其特征在于，所述损失函数为：
    Figure PCTCN2021075798-appb-100004
    其中，|V|表示节点数。
  7. 一种影像驱动的脑图谱构建装置，其特征在于，包括：
    获取单元，用于获取节点特征矩阵，所述节点特征矩阵包括所述脑部的多个节点的时间序列；
    转换单元，用于对所述节点特征矩阵进行超图数据结构转换，得到第一超图关联矩阵；
    构建单元，用于将所述第一超图关联矩阵和所述节点特征矩阵输入已训练的超图转移矩阵生成器中处理，输出得到第一超图转移矩阵，所述第一超图转移矩阵表征构建的多模态脑图谱。
  8. 根据权利要求7所述的装置，其特征在于，
    所述获取单元，还用于针对训练集中的每个训练样本，对所述训练样本进行预处理，得到节点特征矩阵样本和结构连接矩阵样本；
    所述转换单元，还用于对所述节点特征矩阵样本进行超图数据结构转换，得到第二超图关联矩阵；
    所述构建装置还包括训练单元，所述训练单元用于：
    将所述第二超图关联矩阵和所述节点特征矩阵样本输入初始超图转移矩阵生成器中处理，输出得到第二超图转移矩阵；
    基于超图上的随机游走原理，确定所述训练样本对应的每个节点基于所述结构连接矩阵样本得到的关联节点集合N real(v)和基于第二超图转移矩阵得到的关联节点集合N fake(v)；
    利用判别器对所述每个节点的N real(v)和N fake(v)进行判决，得到所述训练样本对应的判决结果；
    根据每个训练样本对应的判别结果和预设的损失函数进行迭代训练，得到所述超图转移矩阵生成器。
  9. 一种终端设备，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序；所述处理器用于在调用所述计算机程序时执行如权利要求1-6任一项所述的方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-6任一项所述的方法。
PCT/CN2021/075798 2021-01-09 2021-02-07 影像驱动的脑图谱构建方法、装置、设备及存储介质 Ceased WO2022147885A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/764,755 US12159706B2 (en) 2021-01-09 2021-02-07 Image-driven brain atlas construction method, device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110026745.5 2021-01-09
CN202110026745.5A CN112951386B (zh) 2021-01-09 2021-01-09 影像驱动的脑图谱构建方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022147885A1 true WO2022147885A1 (zh) 2022-07-14

Family

ID=76235137

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/075798 Ceased WO2022147885A1 (zh) 2021-01-09 2021-02-07 影像驱动的脑图谱构建方法、装置、设备及存储介质

Country Status (3)

Country Link
US (1) US12159706B2 (zh)
CN (1) CN112951386B (zh)
WO (1) WO2022147885A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359837A (zh) * 2022-08-18 2022-11-18 京东方科技集团股份有限公司 药物与靶标的相互作用预测方法、装置及存储介质
CN116130089A (zh) * 2023-02-02 2023-05-16 湖南工商大学 基于超图神经网络的多模态抑郁症检测系统、装置及介质
CN116236158A (zh) * 2023-02-17 2023-06-09 浙江大学 一种基于头皮脑电信号的局灶癫痫发作识别方法和系统
CN116433964A (zh) * 2023-03-17 2023-07-14 南京航空航天大学 一种基于多数据中心的多模态脑网络特征分类方法
CN116597214A (zh) * 2023-05-18 2023-08-15 广东工业大学 基于多模态超图注意网络的阿尔茨海默症分类方法及系统
CN117407534A (zh) * 2023-10-23 2024-01-16 南京师范大学 一种基于激活扩散模型的带权认知图谱权重计算方法
CN118245950A (zh) * 2024-05-20 2024-06-25 银河航天(北京)网络技术有限公司 用于卫星的火工品状态确定方法、装置及计算机程序产品

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112951386B (zh) * 2021-01-09 2023-08-29 深圳先进技术研究院 影像驱动的脑图谱构建方法、装置、设备及存储介质
CN113254729B (zh) * 2021-06-29 2021-10-08 中国科学院自动化研究所 基于动态超图网络的多模态演化特征自动共形表示方法
CN113822757B (zh) * 2021-07-28 2024-10-18 同盾科技有限公司 基于超图的非正常交易预测方法、装置、设备和存储介质
CN113723485B (zh) * 2021-08-23 2023-06-06 天津大学 一种轻微肝性脑病脑影像超图处理方法
CN113724784B (zh) * 2021-09-08 2022-05-06 山东建筑大学 阿尔兹海默病半脑不对称的遗传结构绘制系统及方法
CN114359657B (zh) * 2021-12-14 2024-08-20 深圳先进技术研究院 脑图谱构建及神经环路检测的方法和相关产品
WO2023108418A1 (zh) * 2021-12-14 2023-06-22 深圳先进技术研究院 脑图谱构建及神经环路检测的方法和相关产品
CN118821838B (zh) * 2024-06-19 2025-10-10 浙江大学 数据与知识协同驱动的认知效度计算模型增强方法及系统
CN118864366B (zh) * 2024-06-28 2026-01-06 青岛大学 基于多模态微结构拓扑特征的脑疾病分类模型的构建方法和装置
CN119720910B (zh) * 2025-02-27 2025-07-29 中国科学院大学 与非图优化方法、装置、计算机设备和可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103093087A (zh) * 2013-01-05 2013-05-08 电子科技大学 一种基于多任务学习的多模态脑网络特征融合方法
CN104715260A (zh) * 2015-03-05 2015-06-17 中南大学 基于rls-elm的多模态融合图像分类方法
CN106203470A (zh) * 2016-06-22 2016-12-07 南京航空航天大学 一种基于超图的多模态特征选择及分类方法
CN107658018A (zh) * 2017-10-12 2018-02-02 太原理工大学 一种基于结构连接和功能连接的融合脑网络构建方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9563950B2 (en) * 2013-03-20 2017-02-07 Cornell University Methods and tools for analyzing brain images
US11640529B2 (en) * 2020-02-16 2023-05-02 International Business Machines Corporation Training a neural network to create an embedding for an unlabeled vertex in a hypergraph
CN112951386B (zh) * 2021-01-09 2023-08-29 深圳先进技术研究院 影像驱动的脑图谱构建方法、装置、设备及存储介质
US12056236B2 (en) * 2021-03-24 2024-08-06 International Business Machines Corporation Defending against adversarial queries in a data governance system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103093087A (zh) * 2013-01-05 2013-05-08 电子科技大学 一种基于多任务学习的多模态脑网络特征融合方法
CN104715260A (zh) * 2015-03-05 2015-06-17 中南大学 基于rls-elm的多模态融合图像分类方法
CN106203470A (zh) * 2016-06-22 2016-12-07 南京航空航天大学 一种基于超图的多模态特征选择及分类方法
CN107658018A (zh) * 2017-10-12 2018-02-02 太原理工大学 一种基于结构连接和功能连接的融合脑网络构建方法

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
SU LIFAN, GAO YUE, ZHAO XIBIN, WAN HAI, GU MING, SUN JIAGUANG: "Vertex-Weighted Hypergraph Learning for Multi-View Object Classification", PROCEEDINGS OF THE TWENTY-SIXTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 1 August 2017 (2017-08-01), California, US , pages 2779 - 2785, XP055949442, ISBN: 978-0-9992411-0-3, DOI: 10.24963/ijcai.2017/387 *
XIAO LI; WANG JUNQI; KASSANI PEYMAN H.; ZHANG YIPU; BAI YUNTONG; STEPHEN JULIA M.; WILSON TONY W.; CALHOUN VINCE D.; WANG YU-PING: "Multi-Hypergraph Learning-Based Brain Functional Connectivity Analysis in fMRI Data", IEEE TRANSACTIONS ON MEDICAL IMAGING, vol. 39, no. 5, 29 November 2019 (2019-11-29), USA, pages 1746 - 1758, XP011785815, ISSN: 0278-0062, DOI: 10.1109/TMI.2019.2957097 *
ZHANG ZIZHAO, LIN HAOJIE, GAO YUE: "Dynamic Hypergraph Structure Learning", PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 1 July 2018 (2018-07-01), California , pages 3162 - 3169, XP055949446, ISBN: 978-0-9992411-2-7, DOI: 10.24963/ijcai.2018/439 *
ZHU YINGYING; ZHU XIAOFENG; KIM MINJEONG; YAN JIN; KAUFER DANIEL; WU GUORONG: "Dynamic Hyper-Graph Inference Framework for Computer-Assisted Diagnosis of Neurodegenerative Diseases", IEEE TRANSACTIONS ON MEDICAL IMAGING, vol. 38, no. 2, 1 February 2019 (2019-02-01), USA, pages 608 - 616, XP011707515, ISSN: 0278-0062, DOI: 10.1109/TMI.2018.2868086 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359837A (zh) * 2022-08-18 2022-11-18 京东方科技集团股份有限公司 药物与靶标的相互作用预测方法、装置及存储介质
CN116130089A (zh) * 2023-02-02 2023-05-16 湖南工商大学 基于超图神经网络的多模态抑郁症检测系统、装置及介质
CN116130089B (zh) * 2023-02-02 2024-01-02 湖南工商大学 基于超图神经网络的多模态抑郁症检测系统、装置及介质
CN116236158A (zh) * 2023-02-17 2023-06-09 浙江大学 一种基于头皮脑电信号的局灶癫痫发作识别方法和系统
CN116433964A (zh) * 2023-03-17 2023-07-14 南京航空航天大学 一种基于多数据中心的多模态脑网络特征分类方法
CN116597214A (zh) * 2023-05-18 2023-08-15 广东工业大学 基于多模态超图注意网络的阿尔茨海默症分类方法及系统
CN116597214B (zh) * 2023-05-18 2025-12-26 广东工业大学 基于多模态超图注意网络的阿尔茨海默症分类方法及系统
CN117407534A (zh) * 2023-10-23 2024-01-16 南京师范大学 一种基于激活扩散模型的带权认知图谱权重计算方法
CN118245950A (zh) * 2024-05-20 2024-06-25 银河航天(北京)网络技术有限公司 用于卫星的火工品状态确定方法、装置及计算机程序产品

Also Published As

Publication number Publication date
US20240029866A1 (en) 2024-01-25
CN112951386A (zh) 2021-06-11
CN112951386B (zh) 2023-08-29
US12159706B2 (en) 2024-12-03

Similar Documents

Publication Publication Date Title
WO2022147885A1 (zh) 影像驱动的脑图谱构建方法、装置、设备及存储介质
US12288327B2 (en) Image-driven brain atlas construction method, apparatus, device and storage medium
CN112149717B (zh) 基于置信度加权的图神经网络训练方法及装置
CN110148142B (zh) 图像分割模型的训练方法、装置、设备和存储介质
CN110503654B (zh) 一种基于生成对抗网络的医学图像分割方法、系统及电子设备
Ayana et al. Multistage transfer learning for medical images
CN112102237A (zh) 基于半监督学习的脑部肿瘤识别模型的训练方法及装置
EP3975117A1 (en) Image segmentation method and apparatus, and training method and apparatus for image segmentation model
KR102177568B1 (ko) 레이블 있는 데이터 및 레이블 없는 데이터를 병용하는 준지도 강화 학습 방법 및 이를 이용한 장치
WO2021008328A1 (zh) 图像处理方法、装置、终端及存储介质
CN112102266A (zh) 基于注意力机制的脑梗死医学影像分类模型的训练方法
CN115330669A (zh) 预测解剖结构的疾病量化参数的计算机实现的方法、系统及存储介质
CN111340819A (zh) 图像分割方法、装置和存储介质
CN111709485B (zh) 医学影像处理方法、装置和计算机设备
CN111553892A (zh) 基于深度学习的肺结节分割计算方法、装置及系统
CN119169383B (zh) 一种基于ai的医疗图像识别方法
WO2024051655A1 (zh) 全视野组织学图像的处理方法、装置、介质和电子设备
CN116934754B (zh) 基于图神经网络的肝脏影像识别方法及装置
CN112488178B (zh) 网络模型的训练方法及装置、图像处理方法及装置、设备
CN120747650B (zh) 一种基于特征解耦与动态图构建的多模态脑网络分类方法
CN116739988A (zh) 一种基于多难度课程学习的深度学习脑出血分类方法
Hilal et al. Design of intelligent Alzheimer disease diagnosis model on CIoT environment
Priya et al. Advanced attention-based pre-trained transfer learning model for accurate brain tumor detection and classification from MRI images
CN111598870A (zh) 基于卷积神经网络端对端推理计算冠状动脉钙化比的方法
Chang et al. Deep learning from latent spatiotemporal information of the heart: Identifying advanced bioimaging markers from echocardiograms

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 17764755

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21916936

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21916936

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21916936

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.01.2024)