CN116543215A - Brain network classification method based on deep hash mutual learning - Google Patents

Brain network classification method based on deep hash mutual learning Download PDF

Info

Publication number
CN116543215A
CN116543215A CN202310522896.9A CN202310522896A CN116543215A CN 116543215 A CN116543215 A CN 116543215A CN 202310522896 A CN202310522896 A CN 202310522896A CN 116543215 A CN116543215 A CN 116543215A
Authority
CN
China
Prior art keywords
hash
brain
brain network
network
learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310522896.9A
Other languages
Chinese (zh)
Inventor
冀俊忠
张雅琴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN202310522896.9A priority Critical patent/CN116543215A/en
Publication of CN116543215A publication Critical patent/CN116543215A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a brain network classification method based on deep hash mutual learning, which comprises the following steps: data preprocessing and brain function network construction; dividing brain network data; individual feature extraction based on deep hash learning; group feature extraction based on deep hash learning; mutual learning based on hash codes; based on the classification of the hash code. The invention firstly considers the difference of phenotype labels in the brain network of the group, adopts the phenotype labels to construct a brain network relation diagram of the group, and provides a GCN-based deep hash learning model for extracting the group characteristics of the brain network; and in consideration of the relation between the individual characteristics and the group characteristics of the brain network, the distinguishing capability of the characteristics is enhanced by adopting a brain network classification method based on deep hash mutual learning through the mutual learning between the individual characteristics and the group characteristics. The method has better classification performance than other methods.

Description

Brain network classification method based on deep hash mutual learning
Technical Field
The invention relates to a brain network classification technology in the field of brain science, which mainly realizes interaction of individual brain network characteristics and group brain network characteristics through mutual learning between two deep hash learning models and designs a brain network classification method based on the deep hash mutual learning.
Background
In brain science, brain function network classification refers to a technology for automatically judging whether a subject suffers from a brain nerve disease or not by performing feature mining and analysis on human brain function network data, and provides an effective tool for understanding and early diagnosing pathogenesis of the brain disease, so that the brain function network classification has important research and application values. In recent years, since resting-state functional magnetic resonance (rs-fMRI) technology can reveal spontaneous functional activity rules of the brain, the technology is widely used for constructing brain functional networks. The constructed brain function network is made up of nodes corresponding to neurons, clusters of neurons, brain regions or regions of interest (Region ofInterest, ROI) in the brain and edges, which are generally defined as the connection relationship between pairs of nodes, also called functional connections (functional connectivity, FC), and the weight of an edge is FC intensity. The brain network is hereinafter referred to as the brain function network.
Currently, the most popular is a brain network classification method based on machine learning, mainly comprising a traditional machine learning method and a deep learning method. In the traditional machine learning method, the support vector machine (support vector machine, SVM) and the minimum absolute shrinkage and selection operator (least absolute shrinkage and selection operator, LASSO) are most widely used, but all belong to a shallow model, and when facing high-dimensional brain network data, classification performance is poor due to insufficient feature extraction capability. In recent years, deep learning methods have received increasing attention in brain network classification because richer, deeper features can be learned by layer-by-layer processing of the original brain network. In 2017, kawahara et al proposed a convolutional neural network (Convolutional Neural Network, CNN) framework named brain CNN that designed three convolutional filters edge-to-edge, edge-to-point, and point-to-point to extract the characteristics of the brain network. Experimental results show that brain network characteristics extracted from the BrainNetCNN framework can be used for predicting clinical neural development results. In 2018, heinfeld et al, a model based on deep neural networks (Deep Neural Networks, DNN) that identified autism (Autistic Spectrum Disorder, ASD) patients based on brain activation patterns. Experimental results show that the DNN model can improve the recognition accuracy of ASD and recognize brain regions which are most helpful for distinguishing ASD patients from healthy controls. In 2019 Ju et al proposed a framework based on stacked sparse self-coding (stacked sparse autoencoding, SSAE) method for distinguishing normal aging from mild cognitive impairment and significantly improved classification accuracy compared to traditional machine learning methods. In 2020, xing et al propose a new brain network classification method (adaptive multi-task CNN, AMTCNN) based on adaptive multi-task CNN, and have achieved a better effect in classification of ASD patients and health controls. 2021, ji et al propose a new convolution kernel with element-by-element weighting mechanism (CKEW) and a CNN classification framework based on CKEW. Experimental results show that the framework can more accurately classify brain networks and identify abnormal connection patterns related to brain diseases. In 2022, ji et al constructed semantic space using diagnostic and clinical phenotype tags, and combined with the close relationship of brain networks in semantic space, provided a brain function network classification method (brain network classification based on deep graph hashing learning, BNC-DGHL) based on deep map hash learning. Experimental results on three data sets show that the method can obtain better classification performance, which also shows the important auxiliary role of the phenotype label in brain network classification. However, existing methods only consider differences in phenotypic signatures between individual brain networks, ignoring differences in phenotypic signatures between population brain networks. In fact, in the classification task, the differences in phenotypic signatures between population brain networks are more relevant to the classification results. For example, the gender label of a single brain network under test alone cannot determine whether the test is ASD, but it is easily inferred from the gender label differences of a group of brain networks under test that men are more prone to ASD than women. That is, the phenotypic characteristics of the population brain network are more discriminative than the phenotypic characteristics of the individual brain network.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a brain network classification method based on deep hash mutual learning (Deep Hashing Mutual Learning, DHML), which mainly enhances the distinguishing capability of the features through mutual learning between individual features and group features. Specifically, DHML first uses a CNN-based deep hash learning model to extract individual features of a brain network and map to hash codes, wherein separable convolutional layers are designed to extract individual topological features of multiple levels; secondly, the DHML obtains an adjacency matrix of the brain network according to the similarity relation between the phenotype labels, and constructs a population relation diagram together with the individual characteristics; thirdly, the DHML adopts a GCN (Graph Convolutional Network) -based deep hash learning model to extract group characteristics of the brain network and map the group characteristics into hash codes; then, two deep hash learning models learn each other by minimizing the distribution difference between hash codes, thereby realizing interaction of individual features and group features; finally, the diagnostic tag of the new instance is predicted by calculating the degree of similarity between the hash codes.
The technical scheme adopted by the invention is a brain network classification method based on deep hash mutual learning, which comprises the following steps:
step 1, data preprocessing and brain function network construction.
rs-fMRI data of a subject are acquired, first, pretreatment is carried out by using a resting State functional magnetic resonance processing tool (Data Processing Assistant for Resting-State fMRI, DPARSF), and then, the construction of a brain functional network is carried out.
Step 1.1, preprocessing the original rs-fMRI data by using DPARSF.
(1) The first 5 time points of the original rs-fMRI time series are deleted.
(2) And (3) eliminating the time phase difference of interlayer scanning by adopting a linear interpolation method, and finishing layer time correction.
(3) And removing subject data with water flat head movement exceeding 2mm and rotary head movement exceeding 2 degrees, and completing head movement correction.
(4) The registration, smoothing and filtering of the rs-fMRI image is performed based on the T1 image.
And 1.2, constructing a brain network.
(1) The location of the brain region in the cerebral cortex is located according to the brain map as a node of the brain network.
(2) And calculating the Pearson correlation coefficient of the rs-fMRI time sequence among the nodes as the functional connection strength of the brain interval, namely the weight of the connection edge among the nodes of the brain area.
And 2, dividing brain network data.
Dividing brain network data set into training set X according to the proportion of 8:1:1 t Verification set X a And test set X e . Training setComprises N brain networks, and the corresponding label set is expressed as Y t Comprising a diagnostic tag set->And phenotype tagging set of->d represents a diagnostic signature and p represents a phenotypic signature. Similarly, verify set->Comprises M brain networks, and the corresponding tag set is expressed as +.>Test set->Comprises K brain networks, and the corresponding tag set is expressed as +.>
And 3, extracting individual features based on deep hash learning.
Deep hash learning model I based on CNN H The topological features of the individual brain network are extracted and mapped into hash codes. Specifically, five layers of CNNs were first designed for feature extraction, the extraction process being defined asThen a hash layer is designed for feature mapping, the mapping procedure is defined as +.>
And 3.1, extracting individual characteristics.
For the feature extraction part, separable CNNs are adopted to extract topological features of the individual brain network from the edge level, the node level and the graph level respectively. Specifically, L 0 Is an input layer that receives a brain network; l (L) 1 Is a separable Edge-to-Edge (E2E) layer comprising two processes of a lane-by-lane E2E (E2E, E2E-DW) and a point-by-point E2E (E2E, E2E-PW) for extracting Edge level topological features from an input brain network; l (L) 2 Is a separable Edge-to-Node (E2N) layer, comprising two processes of channel-by-channel E2N (E2N-DW) and point-by-point E2N (E2N-PW) for extracting Node-level features from Edge-level features; l (L) 3 Is a Node-to-Graph (N2G) layer for extracting Graph-level features from Node-level features; l (L) 4 Is a fully connected layer. The specific process is as follows.
(1) Definition L 0 -L 4 The characteristic extraction process of the layer CNN is as followsExtracting individual feature IF of brain network during training t The formula is:
wherein X is t For the training set, N brain networks are included.
(2) Definition acts on L 0 -L 4 Cross entropy loss function J of layers 1 The method comprises the following steps:
wherein W is c And b represents the weight and bias of the separable CNN respectively,and b q E b is the weight and bias of the Softmax layer, respectively, N is the training set X t Number of brain networks involved,/->Is X t Diagnostic tag set of->Is a Softmax function, IF t Is an individual feature of the brain network, ζ (IF t ) Is the predictive probability of an individual feature diagnostic tag.
And 3.2, mapping individual characteristics.
For the feature mapping part, a hash layer is designed, the extracted individual features are firstly mapped into feature vectors with the length of l through a hash function, wherein l is a predefined hash code length, binarization of the feature vectors is carried out, and finally hash code representations corresponding to the individual topological features are obtained. The specific process is as follows.
(1) Defining a feature mapping process of a hash layer asThe extracted individual features IF t Mapping into hash code IU through hash layer t The formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,and b h E b is the weight and bias of the hash layer, σ (·) is the sigmoid function, sgn (·) is the sign function, IU t ∈{0，1} N×l Is IF (IF) t Is represented by hash code of (2), N represents X t The number of brain networks contained in the system, i, is a predetermined hash code length, and each hash code is an l-dimensional vector with a value of 0 or 1. Training set X t IU of hash code set of (a) t Also known as a hash code bank.
(2) Definition of the similarity-preserving loss function J acting on Ha Xiceng 2 The method comprises the following steps:
wherein S.epsilon. {0,1 }) N×N Is a pair-wise similarity matrix constructed from diagnostic tags, N represents X t The number of brain networks included in the formula X t (i) With brain network X t (j) When the diagnostic tags of (a) are identical, i.e. S is then ij =1, otherwise s ij ＝0；iu t (i) Heiu (Heiu) t (j) Respectively brain network X t (i) And X t (j) Hash codes of individual features are represented.
And 3.3, constructing an objective function.
Loss function J by integrating feature extraction portions 1 And a loss function J of the feature mapping section 2 Deep hash learning model I based on CNN H Is defined as:
wherein W is c And b is model I respectively H Is included in the weight and bias of (1). The last term is a regularization term to avoid overfitting of the model during the training phase, lambda 1 ，λ 2 And lambda (lambda) r Is a super parameter.
And 4, extracting group characteristics based on deep hash learning.
Deep hash learning model G based on GCN H The topological features of the group brain network are extracted and mapped into hash codes. Specifically, the adjacency matrix A is first constructed according to the similarity of the phenotype labels, and then the individual brain network characteristics IF extracted from the step 3 are combined t Constructing a group relation diagram P, finally extracting group characteristics of the brain network by using GCN, mapping the group characteristics into hash codes through a hash layer, and defining the GCN extraction process as followsThe mapping procedure is defined as +.>
And 4.1, constructing an adjacency matrix A of the population brain network.
(1) The mahalanobis distance between the brain network phenotype label vectors is calculated as follows:
wherein d= { D ij |i，j=1..n } is mahalanobis distance, and can eliminate scale differences between different phenotypic signatures.Brain network x respectively t (i) And x t (j) Phenotype tag vector of C.epsilon.R N×N As covariance matrix, N represents X t Number of midbrain networks, C -1 Is the inverse of the covariance matrix.
(2) Defining an adjacency matrix of the population brain network:
s.t.α∈(0，1)
wherein a= { a ij I, j=1,..n } is the adjacency matrix, a ij =1 represents two brain networks x t (i) And x t (j) There is a connection between a ij =0 denotes brain network x t (i) And x t (j) Instead of neighboring nodes, α is used to control the number of connected edges between brain networks, D is the mahalanobis distance, and max (D) is the maximum value of D.
And 4.2, constructing a group brain network relation diagram P.
In a single brain network { x } t (1)，x t (2)，...，x t (N) } is a node, A is an adjacency matrix for judging whether edges exist between the nodes, IF t For initial attribute characteristics of the nodes, a group relationship graph P (A, IF t ) Wherein A is an adjacency matrix representing the structure of the graph, IF t Is a feature matrix representing the node properties.
And 4.3, extracting group characteristics.
For the feature extraction portion, GCN is adopted to extract the features of the group brain network. Specifically, L' 0 Is an input layer for receiving a population relationship graph P; l'. 1 -L′ 2 Is a graph roll overlay that aggregates features of neighboring nodes. The specific process is as follows.
(1) Definition L' 0 -L′ 2 The characteristic extraction process of the layer GCN is as followsExtracting group characteristics GF of brain network in training process t The formula is:
wherein X is t For the training set, N brain networks are included.
(2) Definition acts on L' 0 -L′ 2 Cross entropy loss function J of layers 3 The method comprises the following steps:
wherein W is g Is the weight of GCN, N is X t The number of the brain networks in the middle,is X t Diagnostic tag set of->As a Softmax function, GF t Is a group feature of brain network, +.>The predictive probability of the tag is diagnosed for the group feature.
And 4.4, mapping the group characteristics.
For the feature mapping part, a hash layer is designed to map the extracted group features into hash code representations with the length of l through a hash function. The specific process is as follows.
(1) Defining a feature mapping process of a hash layer asExtracting group characteristics GF t Mapping to hash codes GU through a hash layer t Formula (VI)Expressed as:
wherein σ (·) is a sigmoid function and sgn (·) is a sign function. GU (GU) t Is population characteristics GF t Each hash code is a vector of dimension l with a value of 0 or 1, l being a predetermined hash code length.
(2) Definition of the similarity-preserving loss function J acting on Ha Xiceng 4 The method comprises the following steps:
wherein S.epsilon. {0,1 }) N×N For pair-wise similarity matrix constructed from diagnostic tags s ij E S, N represents X t The number of brain networks included in the matrix, gu t (i) And gu t (j) Respectively brain network X t (i) And X t (j) Hash code representation of group features.
And 4.5, constructing an objective function.
Loss function J by integrating feature extraction portions 3 And a loss function J of the feature mapping section 4 GCN-based deep hash learning model G H Is defined as:
J(W g )＝λ 3 J 3 +λ 4 J 4
wherein W is g Is a model G H Weights, lambda 3 And lambda (lambda) 4 Is a super parameter.
And 5, mutual learning based on the hash codes.
In order to make two deep hash learning models I H And G H Co-learning and exploring features learned from each other during the training process,defining the maximum average difference as a loss function of the mutual learning process:
wherein IU t Is individual characteristic IF t Is a hash code representation of GU t Is population characteristics GF t Delta (·) is a gaussian kernel and eta is a standard deviation dependent parameter that controls the range of delta (·). Therefore, CNN-based deep hash learning model I H Is updated as:
wherein W is c And b is model I respectively H Is included in the weight and bias of (1). Lambda (lambda) 1 ，λ 2 ，λ m And lambda (lambda) r Is a super parameter.
GCN-based deep hash learning model G H Is updated as:
J(W g )＝λ 3 J 3 +λ 4 J 4 +λ′ m J m (14)
wherein W is g Is a model G H Weights, lambda 3 ，λ 4 And lambda' m Is a super parameter. The two models are jointly optimized by learning each other until the objective function converges.
And 6, classifying based on the hash codes.
Obtaining trained I through a mutual learning step H After modeling, use validation set X a To verify the classification performance of the model and further adjust the model parameters. The specific process is as follows.
(1) Obtaining a verification set X a Hash code of (1) indicates IU a Calculation formulaThe formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,is I H Feature extraction function of model, ++>Is I H Feature mapping functions of the model.
(2) Hash code IU of training set t (equation (3)) as a hash code library, IU is calculated a Is of the hash code iu a (i) With IU t The distance between all hash codes in IU t Finding the hash code with the smallest distance and taking the label as a prediction labelThe calculation formula is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing training set X t Is the real diagnosis label of the training set X t The number of brain networks involved, M is X a The number of brain networks involved,/->Is an XOR logical operation.
(3) When the loss function of the DHML method reaches convergence and the prediction accuracy of the verification set is not improved, the model and the hash code library at the moment are reserved and are used for testing the brain-collecting network X e Is a diagnostic tag class of (c).
Compared with the prior art, the invention has the following advantages:
(1) According to the method, the phenotype label difference in the population brain network is considered for the first time, the population brain network relation diagram is constructed by adopting the phenotype label, the population characteristics of the brain network are extracted by using the GCN-based deep hash learning model, and compared with other methods, the extracted characteristics comprise more abundant population difference information and have stronger distinguishing capability.
(2) The invention considers the relation between the individual characteristics and the group characteristics of the brain network, adopts the brain network classification method based on deep hash mutual learning to enhance the distinguishing capability of the characteristics through the mutual learning between the individual characteristics and the group characteristics, and has better classification performance compared with other methods.
Drawings
Fig. 1 is an overall framework of the DHML method.
Fig. 2 is a separable E2E convolution layer.
Fig. 3 is a separable E2N convolution layer.
Detailed Description
The invention will be further described with reference to the drawings and examples.
The overall framework of the brain network classification method based on deep hash mutual learning is shown in fig. 1.
And step 1, according to different brain map divisions, using O to represent the number of brain area nodes, and constructing a brain network to represent an O×O matrix.
Step 2, training set X t Verification set X a And test set X e Comprising N, M and K brain networks, N: M: K=8:1:1, respectively.
Step 3, individual feature extraction based on deep hash learning is shown as a module a) in fig. 1. For the feature extraction part, L 0 The input layer receives brain network, wherein O represents the number of brain areas, and the input brain network is represented as an O x O matrix; l (L) 1 Is a separable E2E layer, comprises two processes of E2E-DW and E2E-PW, and is used for extracting edge level topological features from an input brain network; as shown in fig. 2, the E2E-DW convolves the input matrix with the column convolution kernel o×1, then convolves the input matrix with the row convolution kernel 1×o, and finally adds and outputs the convolution result; E2E-PW willThe results of E2E-DW are convolved with 32 1X 1 convolution kernels; l (L) 2 Is a separable E2N layer, and comprises two processes of E2N-DW and E2N-PW for extracting node level features from edge level features; as shown in FIG. 3, E2N-DW will L 1 The layer result is firstly convolved and transposed with 32 O×1 column convolution kernels, then convolved with 32 1×O row convolution kernels, and finally the two convolution results are added; E2N-PW convolves the E2N-DW result with 64 1X 1 convolution kernels; l (L) 3 Is an N2G layer containing 128O x 1 convolution kernels for extracting graph-level features from node-level features; l (L) 4 Is a fully connected layer comprising 96 1 x 1 convolution kernels, resulting in individual features of the brain network. The feature mapping part comprises a hash layer for mapping the extracted brain network individual features into binary hash code representations.
And 4, extracting group characteristics based on deep hash learning as shown in a module b) in fig. 1. In a single brain network { x } t (1)，x t (2)，...，x t (N) } is a node, A is an adjacency matrix for judging whether edges exist between the nodes, IF t For initial attribute characteristics of nodes, a population relationship graph P (A, IF t ) Wherein A is an N adjacency matrix representing the graph structure, IF t Is an N X96 feature matrix representing node attributes, N is a training set X t The number of brain networks involved; the group relation diagram P firstly obtains a brain network of the brain network through GCN, and then obtains hash code representation of group characteristics through a hash layer.
Step 5, mutual learning based on hash codes is shown in block c) of fig. 1.
Step 6, sorting based on hash codes is shown in fig. 1 as block d). The results of the classification performance comparison with other algorithms are shown in table 1, taking the three brain atlas AAL, dosenbach and CC200 divisions of the ABIDE I dataset as an example.
Table 1 results of classification performance comparisons

Claims (2)

1. A brain network classification method based on deep hash mutual learning is characterized in that: the distinguishing capability of the characteristics is enhanced through mutual learning between the individual characteristics and the group characteristics; firstly, extracting individual features of a brain network by using a deep hash learning model based on CNN, mapping the individual features into hash codes, and designing separable convolution layers to extract multi-stage individual topological features; secondly, obtaining an adjacency matrix of the brain network according to the similarity relation between the phenotype labels, and constructing a population relation diagram together with the individual characteristics; thirdly, extracting group characteristics of the brain network by adopting a GCN-based deep hash learning model and mapping the group characteristics into hash codes; then two deep Hash learning models learn each other by minimizing the distribution difference between Hash codes, thereby realizing the interaction of individual features and group features; finally, the diagnostic tag of the new instance is predicted by calculating the degree of similarity between the hash codes.
2. The brain network classification method based on deep hash mutual learning according to claim 1, comprising the steps of:
step 1, data preprocessing and brain function network construction;
acquiring rs-fMRI data, preprocessing by using a resting-state functional magnetic resonance processing tool, and constructing a brain functional network; the specific method is as follows,
step 1.1, preprocessing original rs-fMRI data by using DPARSF;
(1) Deleting the first 5 time points of the original rs-fMRI time sequence;
(2) The linear interpolation method is adopted to eliminate the time phase difference of interlayer scanning, and layer time correction is completed;
(3) Removing subject data with water flat head movement exceeding 2mm and rotary head movement exceeding 2 degrees, and completing head movement correction;
(4) Registering, smoothing and filtering the rs-fMRI image based on the T1 image;
step 1.2, constructing a brain network;
(1) Positioning the position of a brain region in the cerebral cortex according to the brain map to serve as a node of a brain network;
(2) Calculating the Pearson correlation coefficient of the rs-fMRI time sequence among the nodes as the functional connection strength of the brain interval, namely the weight of the connecting edge among the nodes of the brain area;
step 2, dividing brain network data;
dividing the brain network data set into training sets X according to the proportion of 8:1:1 t Verification set X a And test set X e The method comprises the steps of carrying out a first treatment on the surface of the Training setComprises N brain networks, and the corresponding label set is expressed as Y t Comprising a diagnostic tag set->And phenotype tagging set of->d represents a diagnostic signature and p represents a phenotypic signature; similarly, verify set->Comprises M brain networks, and the corresponding tag set is expressed as +.>Test set->Comprises K brain networks, and the corresponding tag set is expressed as +.>
Step 3, individual feature extraction based on deep hash learning;
deep hash learning model I based on CNN H Extracting topological features of the individual brain network and mapping the topological features into hash codes; specifically, five layers of CNN are designed for feature extractionThe extraction process is defined asThen a hash layer is designed for feature mapping, the mapping procedure is defined as +.>
Step 3.1, extracting individual characteristics;
for the feature extraction part, adopting separable CNNs to extract topological features of the individual brain network from the edge level, the node level and the graph level respectively; l (L) 0 Is an input layer that receives a brain network; l (L) 1 Is a separable edge-to-edge layer, and comprises two processes of a channel-by-channel E2E and a point-by-point E2, and is used for extracting edge-level topological features from an input brain network; l (L) 2 Is a separable edge-to-node layer, comprising two processes of channel-by-channel E2N and point-by-point E2N, for extracting node-level features from edge-level features; l (L) 3 Is a node-to-layer for extracting graph-level features from node-level features; l (L) 4 Is a full connection layer; the specific process is as follows;
(1) Definition L 0 -L 4 The characteristic extraction process of the layer CNN is as followsExtracting individual feature IF of brain network during training t The formula is:
wherein X is t Is a training set comprising N brain networks;
(2) Definition acts on L 0 -L 4 Cross entropy loss function J of layers 1 The method comprises the following steps:
wherein W is c And b represents the weight and bias of the separable CNN respectively,and b q E b is the weight and bias of the Softmax layer, respectively, N is the training set X t Number of brain networks involved,/->Is X t Diagnostic tag set of->Is a Softmax function, IF t Is an individual feature of the brain network, ζ (IF t ) Is the predictive probability of the individual feature diagnostic tag;
step 3.2, mapping individual characteristics;
for the feature mapping part, designing a hash layer, firstly mapping the extracted individual features into feature vectors with the length of l through a hash function, wherein l is a predefined hash code length, and then binarizing the feature vectors to finally obtain hash code representations corresponding to the individual topological features; the specific process is as follows;
(1) Defining a feature mapping process of a hash layer asThe extracted individual features IF t Mapping into hash code IU through hash layer t The formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,and b h E b is the weight and bias of the hash layer, σ (·) is the sigmoid function, sgn (·) is the sign function, IU t ∈{0,1} N×l Is IF (IF) t Is represented by hash code of (2), N represents X t The number of brain networks contained in the data is l, the length of a hash code is preset, and each hash code is an l-dimensional vector with a value of 0 or 1; training set X t IU of hash code set of (a) t Also known as a hash code library;
(2) Definition of the similarity-preserving loss function J acting on Ha Xiceng 2 The method comprises the following steps:
wherein S.epsilon. {0,1 }) N×N Is a pair-wise similarity matrix constructed from diagnostic tags, N represents X t The number of brain networks included in the formula X t (i) With brain network X t (j) When the diagnostic tags of (a) are identical, i.e. S is then ij =1, otherwise s ij ＝0；iu t (i) Heiu (Heiu) t (j) Respectively brain network X t (i) And X t (j) Hash code representation of individual features;
step 3.3, constructing an objective function;
loss function J by integrating feature extraction portions 1 And a loss function J of the feature mapping section 2 Deep hash learning model I based on CNN H Is defined as:
wherein W is c And b is model I respectively H Weights and offsets of (2); the last term is a regularization term to avoid overfitting of the model during the training phase, lambda 1 ,λ 2 And lambda (lambda) r Is a super parameter;
step 4, extracting group characteristics based on deep hash learning;
deep hash learning model G based on GCN H Extracting topological features of the group brain network and mapping the topological features into hash codes; specifically, the adjacency matrix A is first constructed according to the similarity of the phenotype labels, and then the individual brain network characteristics IF extracted from the step 3 are combined t Constructing a group relation diagram P, finally extracting group characteristics of the brain network by using GCN, mapping the group characteristics into hash codes through a hash layer, and defining the GCN extraction process as followsThe mapping procedure is defined as +.>
Step 4.1, constructing an adjacency matrix A of a population brain network;
(1) The mahalanobis distance between the brain network phenotype label vectors is calculated as follows:
wherein d= { D ij I, j=1, …, N } is mahalanobis distance, which can eliminate scale differences between different phenotypic tags; brain network x respectively t (i) And x t (j) Phenotype tag vector of C.epsilon.R N×N As covariance matrix, N represents X t Number of midbrain networks, C -1 Is the inverse of the covariance matrix;
(2) Defining an adjacency matrix of the population brain network:
s.t.α∈(0,1)
wherein a= { a ij I, j=1, …, N } is the adjacency matrix, a ij =1 represents two brain networks x t (i) And x t (j) There is a connection between a ij =0 denotes brain network x t (i) And x t (j) Not adjacent nodes, wherein alpha is used for controlling the number of connecting edges between brain networks, D is the Mahalanobis distance, and max (D) is the maximum value of D;
step 4.2, constructing a group brain network relation diagram P;
in a single brain network { x } t (1),x t (2),…,x t (N) } is a node, A is an adjacency matrix for judging whether edges exist between the nodes, IF t For initial attribute characteristics of the nodes, a group relationship graph P (A, IF t ) Wherein A is an adjacency matrix representing the structure of the graph, IF t Is a feature matrix representing node attributes;
step 4.3, extracting group characteristics;
for the feature extraction part, adopting GCN to extract the features of the group brain network; specifically, L' 0 Is an input layer for receiving a population relationship graph P; l'. 1 -L' 2 Is a graph roll overlay for aggregating features of neighboring nodes; the specific process is as follows;
(1) Definition L' 0 -L' 2 The characteristic extraction process of the layer GCN is as followsExtracting group characteristics GF of brain network in training process t The formula is:
wherein X is t Is a training set comprising N brain networks;
(2) Definition acts on L' 0 -L' 2 Cross entropy loss function J of layers 3 The method comprises the following steps:
wherein W is g Is the weight of GCN, N is X t The number of the brain networks in the middle,is X t Diagnostic tag set of->As a Softmax function, GF t Is a group feature of brain network, +.>Predictive probability for a group feature diagnostic tag;
step 4.4, mapping group characteristics;
for the feature mapping part, a hash layer is designed to map the extracted group features into hash code representation with the length of l through a hash function; the specific process is as follows;
(1) Defining a feature mapping process of a hash layer asExtracting group characteristics GF t Mapping to hash codes GU through a hash layer t The formula is:
wherein, sigma (·) is a sigmoid function, sgn (·) is a sign function; GU (GU) t Is population characteristics GF t Each hash code is a l-dimensional vector with a value of 0 or 1, and l is a predetermined hash code length;
(2) Definition of the similarity-preserving loss function J acting on Ha Xiceng 4 The method comprises the following steps:
wherein S.epsilon. {0,1 }) N×N For pair-wise similarity matrix constructed from diagnostic tags s ij E S, N represents X t The number of brain networks included in the matrix, gu t (i) And gu t (j) Respectively brain network X t (i) And X t (j) Hash code representation of group features;
step 4.5, constructing an objective function;
loss function J by integrating feature extraction portions 3 And a loss function J of the feature mapping section 4 GCN-based deep hash learning model G H Is defined as:
J(W g )＝λ 3 J 3 +λ 4 J 4
wherein W is g Is a model G H Weights, lambda 3 And lambda (lambda) 4 Is a super parameter;
step 5, mutual learning based on hash codes;
in order to make two deep hash learning models I H And G H Co-learning and exploring each other's learned features during the training process, defining the maximum average difference as a loss function of the mutual learning process:
wherein IU t Is individual characteristic IF t Is a hash code representation of GU t Is population characteristics GF t Delta (·) is a gaussian kernel function and eta is a standard deviation-related parameter that controls the delta (·) range; therefore, CNN-based deep hash learning model I H Is updated as:
wherein W is c And b is model I respectively H Weights and offsets of (2); lambda (lambda) 1 ,λ 2 ,λ m And lambda (lambda) r Is a super parameter;
GCN-based deep hash learning model G H Is updated as:
J(W g )＝λ 3 J 3 +λ 4 J 4 +λ' m J m (14)
wherein W is g Is a model G H Weights, lambda 3 ,λ 4 And lambda' m Is a super parameter; the two models are jointly optimized through mutual learning until the objective function converges;
step 6, classifying based on the hash codes;
obtaining trained I through a mutual learning step H After modeling, use validation set X a Come testThe classification performance of the certificate model, and further adjusting model parameters; the specific process is as follows;
(1) Obtaining a verification set X a Hash code of (1) indicates IU a The calculation formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,is I H Feature extraction function of model, ++>Is I H A feature mapping function of the model;
(2) Hash code IU of training set t (equation (3)) as a hash code library, IU is calculated a Is of the hash code iu a (i) With IU t The distance between all hash codes in IU t Finding the hash code with the smallest distance and taking the label as a prediction labelThe calculation formula is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing training set X t Is the real diagnosis label of the training set X t The number of brain networks involved, M is X a The number of brain networks involved,/->Is an XOR logical operation;
(3) When the loss function of the DHML method reaches convergence and the prediction accuracy of the verification set is not improved, the model and the hash code library at the moment are reserved and are used for testing the brain-collecting network X e Is a diagnostic tag class of (c).
CN202310522896.9A 2023-05-10 2023-05-10 Brain network classification method based on deep hash mutual learning Pending CN116543215A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310522896.9A CN116543215A (en) 2023-05-10 2023-05-10 Brain network classification method based on deep hash mutual learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310522896.9A CN116543215A (en) 2023-05-10 2023-05-10 Brain network classification method based on deep hash mutual learning

Publications (1)

Publication Number Publication Date
CN116543215A true CN116543215A (en) 2023-08-04

Family

ID=87451948

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310522896.9A Pending CN116543215A (en) 2023-05-10 2023-05-10 Brain network classification method based on deep hash mutual learning

Country Status (1)

Country Link
CN (1) CN116543215A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116821774A (en) * 2023-08-28 2023-09-29 山东理工昊明新能源有限公司 Power generation fault diagnosis method based on artificial intelligence

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116821774A (en) * 2023-08-28 2023-09-29 山东理工昊明新能源有限公司 Power generation fault diagnosis method based on artificial intelligence
CN116821774B (en) * 2023-08-28 2023-11-28 山东理工职业学院 Power generation fault diagnosis method based on artificial intelligence

Similar Documents

Publication Publication Date Title
Li et al. Lightweight attention convolutional neural network for retinal vessel image segmentation
Zhang et al. Quantifying facial age by posterior of age comparisons
Wang et al. A unified framework for automatic wound segmentation and analysis with deep convolutional neural networks
CN111785329B (en) Single-cell RNA sequencing clustering method based on countermeasure automatic encoder
US20110135166A1 (en) Face Authentication Using Recognition-by-Parts, Boosting, and Transduction
JP2022551683A (en) Methods and systems for non-invasive genetic testing using artificial intelligence (AI) models
Abdulrahman et al. Comparative study for 8 computational intelligence algorithms for human identification
Fan et al. Accurate retinal vessel segmentation via octave convolution neural network
WO2015173435A1 (en) Method for predicting a phenotype from a genotype
Özbılge et al. Tomato disease recognition using a compact convolutional neural network
Liang et al. A convolutional neural network combined with prototype learning framework for brain functional network classification of autism spectrum disorder
CN116543215A (en) Brain network classification method based on deep hash mutual learning
Ahmadi et al. Eye state identification utilizing EEG signals: a combined method using self-organizing map and deep belief network
Ramaraj et al. Brain anomaly prediction with the intervention of fuzzy based clustering and optimization techniques for augmenting clinical diagnosis
CN115496953A (en) Brain network classification method based on space-time graph convolution
Han et al. Fundus retinal vessels image segmentation method based on improved U-Net
CN116259415A (en) Patient medicine taking compliance prediction method based on machine learning
Thangavel et al. EAD-DNN: Early Alzheimer's disease prediction using deep neural networks
Alotaibi et al. Stroke in-patients' transfer to the ICU using ensemble based model
Lohar et al. Automatic classification of autism spectrum disorder (ASD) from brain MR images based on feature optimization and machine learning
CN113241114A (en) LncRNA-protein interaction prediction method based on graph convolution neural network
CN116797817A (en) Autism disease prediction technology based on self-supervision graph convolution model
CN117195027A (en) Cluster weighted clustering integration method based on member selection
Duman et al. Ensemble the recent architectures of deep convolutional networks for skin diseases diagnosis
Adhikary et al. Feature assisted cervical cancer screening through DIC cell images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination