CN113762487B - Brain function network generation method and system based on neural architecture search and DBN network - Google Patents
Brain function network generation method and system based on neural architecture search and DBN network Download PDFInfo
- Publication number
- CN113762487B CN113762487B CN202110942242.2A CN202110942242A CN113762487B CN 113762487 B CN113762487 B CN 113762487B CN 202110942242 A CN202110942242 A CN 202110942242A CN 113762487 B CN113762487 B CN 113762487B
- Authority
- CN
- China
- Prior art keywords
- fmri
- dimensional
- dbn
- particles
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003925 brain function Effects 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000001537 neural effect Effects 0.000 title claims abstract description 23
- 238000002599 functional magnetic resonance imaging Methods 0.000 claims abstract description 120
- 239000002245 particle Substances 0.000 claims abstract description 81
- 239000011159 matrix material Substances 0.000 claims abstract description 53
- 238000007781 pre-processing Methods 0.000 claims abstract description 28
- 239000013598 vector Substances 0.000 claims abstract description 26
- 238000005457 optimization Methods 0.000 claims abstract description 7
- 210000002569 neuron Anatomy 0.000 claims description 34
- 210000004556 brain Anatomy 0.000 claims description 26
- 238000012549 training Methods 0.000 claims description 20
- 238000012937 correction Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 230000035772 mutation Effects 0.000 claims description 10
- 230000004886 head movement Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 210000001175 cerebrospinal fluid Anatomy 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 6
- 210000003625 skull Anatomy 0.000 claims description 6
- 210000004885 white matter Anatomy 0.000 claims description 6
- 230000015654 memory Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 abstract description 6
- 230000004044 response Effects 0.000 description 10
- 230000002068 genetic effect Effects 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 241000282414 Homo sapiens Species 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000000284 resting effect Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 230000007177 brain activity Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 229940008201 allegra Drugs 0.000 description 1
- 210000003710 cerebral cortex Anatomy 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- RWTNPBWLLIMQHL-UHFFFAOYSA-N fexofenadine Chemical compound C1=CC(C(C)(C(O)=O)C)=CC=C1C(O)CCCN1CCC(C(O)(C=2C=CC=CC=2)C=2C=CC=CC=2)CC1 RWTNPBWLLIMQHL-UHFFFAOYSA-N 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000004751 neurological system process Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
- A61B5/004—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
- A61B5/0042—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention belongs to the technical field of medical image and neural network intersection, and discloses a brain function network generation method and system based on neural architecture search and a DBN network, wherein the method comprises the following steps: step 1: acquiring natural normal form fMRI data, preprocessing to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix; step 2: generating M DBN networks and initializing the M DBN networks to obtain a solution space; inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles; step 3: and taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, acquiring a weight matrix of the global optimal particles, and acquiring a brain function network according to the weight matrix of the global optimal particles.
Description
Technical Field
The invention belongs to the technical field of medical image and neural network intersection, and particularly relates to a brain function network generation method and system based on neural architecture search and a DBN network.
Background
Functional magnetic resonance imaging (fMRI) has been widely used to identify functional brain networks. Most of the studies in existence are based on task paradigms, which, although they aim to participate in and isolate specific aspects of brain function (e.g. working memory or visual perception), are unclear whether and to what extent complex neural processes in real life can be revealed, since the behavior of the subject is unnatural. Meanwhile, when resting fMRI data are collected, a plurality of testees often generate micro sleep and head movements, which can generate individual variability of the data which should be the same, and influence the reliability of the collection result. The natural paradigm method based on the natural behavior of human beings overcomes the limitation of task paradigms, for example, a subject cannot easily sleep while watching a movie, the behavior of head movements is relatively reduced, and the design of the natural paradigm can greatly reveal the brain function activities in real life, because the method is mostly based on the natural behavior of human beings. However, the dynamics and complexity of natural paradigms make modeling their neural relevance difficult.
In previous studies, researchers have proposed a number of computational methods for reconstructing and characterizing brain function networks from fMRI data. There are common model driven methods-Generalized Linear Model (GLM), independent principal component analysis (ICA), sparse Dictionary Learning (SDL), etc. Although these methods can well construct meaningful brain function networks, because the models are based on shallow structures, the number of parameters is too small to characterize the hierarchical structure of the brain function network in its natural paradigm.
Recent studies have proposed modeling the spatiotemporal pattern of functional brain activity from fMRI data using various deep learning models and solving the shortcomings of shallow models, such as convolutional neural networks, convolutional Automatic Encoders (CAE), and Deep Belief Networks (DBNs), by their excellent data representation capabilities. Although these models exhibit excellent performance in extracting spatiotemporal features of multiscale fMRI data, there are still significant challenges in these deep learning models: due to the high dimensional nature of fMRI data and the high variety of training parameters, empirically designing neural network architectures is very time consuming and less reliable.
Disclosure of Invention
The invention aims to provide a brain map building method and system based on neural architecture search and a DBN network, which are used for solving the problems that task state and resting state fMRI data in the prior art cannot truly reduce natural brain activities, a traditional shallow model method cannot reveal a brain hierarchical structure, a deep learning model is time-consuming in manual adjustment, the reliability is low and the like.
In order to realize the tasks, the invention adopts the following technical scheme:
a brain function network generation method based on neural architecture search and DBN network comprises the following steps:
step 1: acquiring natural normal form fMRI data, preprocessing to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
step 2: generating M DBN networks and initializing the M DBN networks to obtain a solution space, wherein M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different;
inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks respectively to obtain N particles;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
step 3: and taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, acquiring a weight matrix of the global optimal particles, and acquiring a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature.
Further, step 1 comprises the following sub-steps:
step 1.1: acquiring natural normal form fMRI data;
step 1.2: preprocessing natural normal form fMRI data, wherein the preprocessing comprises skull removal, head movement correction, slice time correction, spatial smoothing, linear trend removal and band-pass filtering, so as to obtain a four-dimensional fMRI voxel image;
step 1.3: after eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, performing Mask processing after nonlinear registration of the four-dimensional fMRI voxel images on a standard MNI space to obtain one-dimensional fMRI vectors;
step 1.4: and connecting the time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix.
Further, step 2.1 uses equation 1 to mutate the number of neurons and the number of model layers of the selected DBN network:
wherein,is the number of neurons or model layers of the ith DBN network before mutation, i is E [1, N]，/>Is the number of neurons or model layer number of the ith DBN network after mutation, +.>Is the update rate of the previous search, +.>Is the update speed of the current search, w is E [0,1 ]]pBest, gBest is the individual extremum and global extremum, c 1 And c 2 Is a learning factor.
Further, w=0.1, c 1 ＝c 2 ＝2。
Further, the fitness function is a mean square error of the input and output of the particles.
A brain function network generation system based on neural architecture search and DBN network, the system comprises a processor and a memory for storing a plurality of functional modules capable of running on the processor, wherein the functional modules comprise an acquisition module, a preprocessing module, an optimal architecture search module and an output module;
the acquisition module is used for acquiring natural normal form fMRI data;
the preprocessing module is used for preprocessing natural normal form fMRI data to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
the optimal architecture searching module is used for generating M DBN networks and initializing the M DBN networks to obtain a solution space, M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different; inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks respectively to obtain N particles;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
the output module is used for taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, obtaining a weight matrix of the global optimal particles, and obtaining a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature. .
Further, the preprocessing module comprises the following sub-modules:
the first submodule is used for preprocessing natural-paradigm fMRI data, wherein the preprocessing comprises skull removal, head movement correction, slicing time correction, space smoothing, linear trend removal and band-pass filtering, and a four-dimensional fMRI voxel image is obtained;
the second submodule is used for eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, and performing Mask processing after the four-dimensional fMRI voxel images are non-linearly registered to a standard MNI space to obtain one-dimensional fMRI vectors;
the third submodule is used for connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix.
Further, step 2.1 uses equation 1 to mutate the number of neurons and the number of model layers of the selected DBN network:
wherein,is the number of neurons or model layers of the ith DBN network before mutation, i is E [1, N]，/>Is the number of neurons or model layer number of the ith DBN network after mutation, +.>Is the update rate of the previous search, +.>Is the update speed of the current search, w is E [0,1 ]]pBest, gBest is the individual extremum and global extremum, c 1 And c 2 Is a learning factor.
Further, w=0.1, c 1 ＝c 2 ＝2。
Further, the fitness function is a mean square error of the input and output of the particles.
Compared with the prior art, the invention has the following technical characteristics:
the invention discloses a deep belief network for realizing automatic parameter adjustment under a natural paradigm for the first time to identify a brain hierarchical structure. The hierarchical brain function network identification method of the neural architecture search and the spatial information deep belief network can find a feasible optimal solution of the spatial information deep belief network structure in acceptable time under the condition of limited computing resources, so that the hierarchical time response and the spatial distribution of the brain under a natural normal form are revealed.
Drawings
FIG. 1 is a flow chart of a model of a neural architecture search-based deep belief network modeling natural paradigm fMRI voxel image;
FIG. 2 is a corresponding 10 times neural architecture search result graph in an embodiment;
FIG. 3 is a graph of results of identifying brain function networks in an embodiment, each layer having 12 brain function networks;
FIG. 4 is a result diagram of identifying hierarchical temporal features, ISC per-layer network independent and group level results, in an embodiment;
FIG. 5 is a graph of brain spatial features versus natural range correspondence, top 5 spectra for each layer of ISC, ISC values for each brain function network, average and standard deviation for each layer of ISC noted;
FIG. 6 is a graph of results of verification of model identification layered spatial features, which is a spatial hierarchy attribute between brain function network layers measured by ISR, in an embodiment, (A) graph is a two-layer-to-one-layer ISR graph, and (B) graph is a three-layer-to-two-layer ISR graph;
FIG. 7 is a verification model identification hierarchy time feature result graph obtained by ISR between 3 hidden layers of a time hierarchy in an embodiment; a) The figure is a two-layer to one-layer ISR diagram, and (B) is a three-layer to two-layer ISR diagram.
Detailed Description
First, technical words appearing in the present invention are explained:
natural paradigm fMRI: a task mode is devised, such as moving a finger or the like over a continuous period of time, and then acquiring functional magnetic resonance images of the subject over a period of time during such task mode.
Resting state fMRI: nothing the subject is lying on the machine is done, then the functional magnetic resonance images acquired during this time.
DBN network: a deep belief network consisting of a visual layer and a plurality of hidden layers, each neuron between the layers having a connection, but no connection between neurons within the layers.
Brain function network: the cortical areas of different brain space locations are integrated by functional association to form a network model, such as a visual network, an auditory network, etc., and the different networks are responsible for processing different brain functions, as shown in fig. 3 of the present invention.
Standard MNI space: is a coordinate system established from a series of magnetic resonance images of a normal human brain. When the original fMRI images are acquired, the dimensions, origin and voxel sizes of the images are different, so that we will register the images to the same template using a standard MNI template.
Mask treatment: the method is a graph operation for partially or completely hiding the object or element, and can remove some bad values which do not accord with the statistical rule in the fMRI data which are initially collected.
In this embodiment, a brain function network generating method based on neural architecture searching and DBN network is disclosed, as shown in fig. 1, the method includes a neural architecture searching process, a process of representing natural stimulus level space-time characteristics by a deep belief network, and space-time characteristics corresponding to the output of a model;
the method comprises the following steps:
step 1: acquiring natural normal form fMRI data, preprocessing to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
step 2: generating M DBN networks and initializing the M DBN networks to obtain a solution space, wherein M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different;
inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks respectively to obtain N particles;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
step 3: and taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, acquiring a weight matrix of the global optimal particles, and acquiring a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature.
Each iteration records the current global optimal network structure, the original optimal network is replaced in the process of iteration, and after the maximum iteration number is reached, the local optimal particles with the minimum neural network loss are selected as global optimal particles.
And mapping the weight of each row of the global optimal particle weight matrix back to the 3D brain space to obtain a corresponding brain function network.
Specifically, step 1 includes the following sub-steps:
step 1.1: acquiring natural paradigm fMRI data of a plurality of subjects over a period of time by a 3T siemens Allegra scanner;
step 1.2: preprocessing natural-paradigm fMRI data by using an SPM12 tool package, wherein the preprocessing comprises skull removal, head motion correction, slice time correction, spatial smoothing, linear trend removal and band-pass filtering, so as to obtain a four-dimensional fMRI voxel image;
step 1.3: after eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, performing Mask processing after nonlinear registration of the four-dimensional fMRI voxel images on a standard MNI space to obtain one-dimensional fMRI vectors;
step 1.4: and connecting the one-dimensional fMRI vectors of all the testees into a two-dimensional matrix along the time points, and finally obtaining the two-dimensional fMRI matrix of the voxel level.
Specifically, m=20, that is, 20 DBN networks having different numbers of neurons and different numbers of layers of the neural network are randomly generated.
Specifically, step 2.1 uses formula 1 to mutate the number of neurons and the model layer number of the selected DBN network:
wherein,is the number of neurons or model layers of the ith DBN network before mutation, i is E [1, N]，/>Is the number of neurons or model layer number of the ith DBN network after mutation, +.>Is the update rate of the previous search, +.>Is the update speed of the current search, w is E [0,1 ]]pBest, gBest is the individual extremum and global extremum, c 1 And c 2 Is a learning factor.
Specifically, w=0.1, c 1 ＝c 2 ＝2。
Specifically, the fitness function is the mean square error of the input and output of the particles.
Specifically, due to the limitation of GPU memory and the high-dimensional nature of fMRI voxel images, the invention sets the search range of neuron number as [10,200], the search range of layer number as [2,10], and specific training parameters are as follows: the learning rate was 0.001, the weight sparsity was 0.01, epcoh was 20, and batch size was 10.
The embodiment also discloses a brain function network generation system based on neural architecture search and DBN network, which comprises a processor and a memory for storing a plurality of functional modules capable of running on the processor, wherein the functional modules comprise an acquisition module, a preprocessing module, an optimal architecture search module and an output module;
the acquisition module is used for acquiring natural normal form fMRI data;
the preprocessing module is used for preprocessing natural normal form fMRI data to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
the optimal architecture searching module is used for generating M DBN networks and initializing the M DBN networks to obtain a solution space, M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different; inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks respectively to obtain N particles;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
the output module is used for taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, obtaining a weight matrix of the global optimal particles, and obtaining a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature. .
Specifically, the preprocessing module comprises the following sub-modules:
the first submodule is used for preprocessing natural-paradigm fMRI data, wherein the preprocessing comprises skull removal, head movement correction, slicing time correction, space smoothing, linear trend removal and band-pass filtering, and a four-dimensional fMRI voxel image is obtained;
the second submodule is used for eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, and performing Mask processing after the four-dimensional fMRI voxel images are non-linearly registered to a standard MNI space to obtain one-dimensional fMRI vectors;
the third submodule is used for connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix.
Specifically, in step 3, an output matrix of globally optimal particles may also be obtained, where each row of the output matrix corresponds to a time response of each neuron in the network of globally optimal particles, and each row of the weight matrix represents a brain space feature learned by the neuron (Zhang Y, hu X et al). For deeper layers, this weight is interpreted using a linear combination method, i.e. w 3 ×w 2 ×w 1 Is the weight of the third layer, w 2 ×w 1 Is the weight of the second layer, w 1 The first layer weight is the weight corresponding to the corresponding linear relation, and the weight corresponding to the corresponding linear relation is the spatial feature corresponding to the feedback natural norm fMRI voxel image of each layer. In fig. 1 (B), the output of the hidden layer represents the time response. For the weight matrix, each row of the weight matrix is mapped back to the original 3D brain image space and quantized into a brain function network, as shown in fig. 1 (B).
Specifically, the present embodiment independently performs 10 times of neural architecture search processes, and the results show that in fig. 2, the number of layers and the number of neurons show high consistency and robustness, and in the results, it can be seen that the optimal model layer is 3, and the standard deviation of the results of the repeated neuron number search process is 5.49 (6% of the average neuron number). The 3-layer 91-node-per-layer DBN architecture was ultimately determined for characterizing the spatiotemporal features of natural-paradigm fMRI data.
Fig. 3 shows that the proposed method identifies natural paradigm-level brain function networks, all spatial brain networks are normalized and equal thresholded in order to achieve a fair comparison between brain function networks at different levels, at the first level, there are auditory networks, visual networks, etc. At the second level, there are default mode networks, auditory significance networks, and the like. In the third layer, there are vision-brain island network, inside vision-hearing network, occipital vision-hearing network, etc. The phenomenon that several lower layers of simple and local networks combine into a higher layer of complex and global networks suggests a hierarchy of brain function networks under natural stimulus.
The present example demonstrates the effectiveness of the method by ISC performance evaluation and genetic similarity evaluation:
to investigate the hierarchical organization of the time responses deduced by the method proposed in the present invention, inter-subject correlation (ISC) of individual time responses was further measured and compared, wherein ISC measures inter-subject consistency of each atomic time response among individuals, as shown in fig. 4, the lower layer time response showed lower inter-subject consistency, while the higher layer time response showed higher inter-subject consistency at individual and group levels, thus demonstrating the hierarchical organization of the time features under natural stimulus. Furthermore, fig. 5 shows the first five brain function networks with the highest ISC values at the group level in each layer. These functional neural networks are mostly associated with auditory, visual or vision-auditory networks, which are consistent with ISC maps developed by natural function magnetic resonance imaging studies, demonstrating the effectiveness of the proposed method in characterizing the time-sequential response of natural-paradigm fMRI voxel images.
Fig. 6 is a graph of genetic similarity of spatial features of the second and first layers and the third and second layers, and it can be seen that higher layers of brain networks exhibit higher genetic similarity and thus have higher similarity, indicating the layered structure of the brain under natural stimulus. FIG. 7 is a result of verifying genetic similarity of time responses derived between different layers, further verifying the superior ability of the present invention to be hierarchical at the time level. The genetic similarity between these associative networks and the temporal and spatial features at different layers quantitatively confirms the hierarchical organization of the spatial distribution and temporal features in the DBN model.
Claims (8)
1. A brain function network generation method based on neural architecture search and DBN network is characterized by comprising the following steps:
step 1: acquiring natural normal form fMRI data, preprocessing to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
step 2: generating M DBN networks and initializing the M DBN networks to obtain a solution space, wherein M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different;
inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks by adopting a formula 1 to obtain N particles;
wherein,is the number of neurons or model layers of the ith DBN network before mutation, i is E [1, N]，/>Is the number of neurons or model layer number of the ith DBN network after mutation, +.>Is the update rate of the previous search, +.>Is the update speed of the current search, w is E [0,1 ]]pBest, gBest is the individual extremum and global extremum, c 1 And c 2 Is a learning factor;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
step 3: and taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, acquiring a weight matrix of the global optimal particles, and acquiring a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature.
2. The brain function network generation method based on neural architecture search and DBN network according to claim 1, wherein step 1 comprises the following sub-steps:
step 1.1: acquiring natural normal form fMRI data;
step 1.2: preprocessing natural normal form fMRI data, wherein the preprocessing comprises skull removal, head movement correction, slice time correction, spatial smoothing, linear trend removal and band-pass filtering, so as to obtain a four-dimensional fMRI voxel image;
step 1.3: after eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, performing Mask processing after nonlinear registration of the four-dimensional fMRI voxel images on a standard MNI space to obtain one-dimensional fMRI vectors;
step 1.4: and connecting the time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix.
3. The brain function network generation method based on neural architecture search and DBN network according to claim 1, wherein w=0.1, c 1 ＝c 2 ＝2。
4. The brain function network generation method based on neural architecture search and DBN network according to claim 1, wherein said fitness function is a mean square error of particle input and output.
5. A brain function network generation system based on neural architecture search and DBN network, the system comprises a processor and a memory for storing a plurality of functional modules capable of running on the processor, wherein the functional modules comprise an acquisition module, a preprocessing module, an optimal architecture search module and an output module;
the acquisition module is used for acquiring natural normal form fMRI data;
the preprocessing module is used for preprocessing natural normal form fMRI data to obtain a four-dimensional fMRI voxel image, converting the four-dimensional fMRI voxel image into one-dimensional fMRI vectors, and connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix;
the optimal architecture searching module is used for generating M DBN networks and initializing the M DBN networks to obtain a solution space, M is a positive integer, and the neuron numbers and the model layer numbers of the M DBN networks are different; inputting a two-dimensional fMRI matrix into a solution space, and searching the solution space for multiple times by utilizing a structure optimization algorithm, wherein each search obtains a local optimal solution, and each local optimal solution contains N local optimal particles;
wherein, the arbitrary search comprises the following sub-steps:
step 2.1: randomly selecting N DBN networks, wherein N is less than M and is a positive integer, and mutating the number of neurons and the model layer number of the selected DBN networks by adopting a formula 1 to obtain N particles;
wherein,is the number of neurons or model layers of the ith DBN network before mutation, i is E [1, N]，/>Is the number of neurons or model layer number of the ith DBN network after mutation, +.>Is the update rate of the previous search, +.>Is the update speed of the current search, w is E [0,1 ]]pBest, gBest is the individual extremum and global extremum, c 1 And c 2 Is a learning factor;
step 2.2: respectively inputting the two-dimensional fMRI matrix into N particles for training, calculating the loss in the training process of each particle by adopting a fitness function, and taking the N particles after training as local optimal particles;
the output module is used for taking the local optimal particles with the minimum loss in all the local optimal solutions as global optimal particles, obtaining a weight matrix of the global optimal particles, and obtaining a brain function network according to the weight matrix of the global optimal particles, wherein the brain function network comprises a plurality of brain space features, and each row of the weight matrix of the global optimal particles corresponds to one brain space feature.
6. The brain function network generation system based on neural architecture search and DBN network according to claim 5, wherein the preprocessing module comprises the following sub-modules:
the first submodule is used for preprocessing natural-paradigm fMRI data, wherein the preprocessing comprises skull removal, head movement correction, slicing time correction, space smoothing, linear trend removal and band-pass filtering, and a four-dimensional fMRI voxel image is obtained;
the second submodule is used for eliminating the influence of white matter and cerebrospinal fluid signals in the four-dimensional fMRI voxel images, and performing Mask processing after the four-dimensional fMRI voxel images are non-linearly registered to a standard MNI space to obtain one-dimensional fMRI vectors;
the third submodule is used for connecting time points of all the one-dimensional fMRI vectors to obtain a two-dimensional fMRI matrix.
7. The neural architecture search and DBN network based brain function network generation system of claim 5, wherein w = 0.1, c 1 ＝c 2 ＝2。
8. The brain function network generation system based on neural architecture search and DBN network according to claim 5, wherein said fitness function is a mean square error of particle input and output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110942242.2A CN113762487B (en) | 2021-08-17 | 2021-08-17 | Brain function network generation method and system based on neural architecture search and DBN network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110942242.2A CN113762487B (en) | 2021-08-17 | 2021-08-17 | Brain function network generation method and system based on neural architecture search and DBN network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113762487A CN113762487A (en) | 2021-12-07 |
| CN113762487B true CN113762487B (en) | 2023-11-10 |
Family
ID=78790055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110942242.2A Active CN113762487B (en) | 2021-08-17 | 2021-08-17 | Brain function network generation method and system based on neural architecture search and DBN network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113762487B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355151A (en) * | 2016-08-30 | 2017-01-25 | 电子科技大学 | Recognition method, based on deep belief network, of three-dimensional SAR images |
| CN107077637A (en) * | 2014-03-24 | 2017-08-18 | 高通股份有限公司 | Differential coding in neutral net |
| CN109376751A (en) * | 2018-08-21 | 2019-02-22 | 北京工业大学 | A kind of human functional network's classification method based on convolutional neural networks |
| AU2020102977A4 (en) * | 2020-10-23 | 2020-12-24 | Bhima, Ravi Teja DR | A Deep learning technique to recognise brain activity by fMRI and DTI image fusion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110097185B (en) * | 2019-03-29 | 2021-03-23 | 北京大学 | Optimization model method based on generation of countermeasure network and application |
-
2021
- 2021-08-17 CN CN202110942242.2A patent/CN113762487B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107077637A (en) * | 2014-03-24 | 2017-08-18 | 高通股份有限公司 | Differential coding in neutral net |
| CN106355151A (en) * | 2016-08-30 | 2017-01-25 | 电子科技大学 | Recognition method, based on deep belief network, of three-dimensional SAR images |
| CN109376751A (en) * | 2018-08-21 | 2019-02-22 | 北京工业大学 | A kind of human functional network's classification method based on convolutional neural networks |
| AU2020102977A4 (en) * | 2020-10-23 | 2020-12-24 | Bhima, Ravi Teja DR | A Deep learning technique to recognise brain activity by fMRI and DTI image fusion |
Non-Patent Citations (2)
| Title |
|---|
| 一种改进粒子群算法优化BP神经网络实现核素识别方法;刘议聪;朱泓光;宋永强;;兵工自动化(第04期);全文 * |
| 基于神经认知计算模型的高分辨率遥感图像场景分类;刘扬;付征叶;郑逢斌;;系统工程与电子技术(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113762487A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Laplacian redecomposition for multimodal medical image fusion | |
| CN111310707B (en) | Bone-based graph annotation meaning network action recognition method and system | |
| Miao et al. | Local segmentation of images using an improved fuzzy C-means clustering algorithm based on self-adaptive dictionary learning | |
| US9454641B2 (en) | Systems and methods for modeling and processing functional magnetic resonance image data using full-brain vector auto-regressive model | |
| CN111476292A (en) | Small sample element learning training method for medical image classification processing artificial intelligence | |
| JP7373042B2 (en) | Brain function registration method based on graph model | |
| CN109389171B (en) | Medical image classification method based on multi-granularity convolution noise reduction automatic encoder technology | |
| Prasetio et al. | The facial stress recognition based on multi-histogram features and convolutional neural network | |
| Suganthe et al. | Diagnosis of Alzheimer's disease from brain magnetic resonance imaging images using deep learning algorithms | |
| CN113781640A (en) | Three-dimensional face reconstruction model establishing method based on weak supervised learning and application thereof | |
| CN110458802A (en) | Based on the projection normalized stereo image quality evaluation method of weight | |
| CN111862278B (en) | Animation obtaining method and device, electronic equipment and storage medium | |
| CN116110597B (en) | Digital twinning-based intelligent analysis method and device for patient disease categories | |
| CN114748053A (en) | fMRI high-dimensional time sequence-based signal classification method and device | |
| Zhou et al. | Intelligent recognition of medical motion image combining convolutional neural network with Internet of Things | |
| CN116051849A (en) | Brain network data feature extraction method and device | |
| CN111488934A (en) | Brain image data processing method, storage medium, computer device and apparatus | |
| Sharif et al. | Deep Learning: Convolutional Neural Networks for Medical Image Analysis-A Quick Review | |
| CN112288645B (en) | Skull face restoration model construction method and restoration method and system | |
| CN113762487B (en) | Brain function network generation method and system based on neural architecture search and DBN network | |
| Fujiwara et al. | Estimating image bases for visual image reconstruction from human brain activity | |
| CN111401209B (en) | Action recognition method based on deep learning | |
| CN117224082A (en) | Dynamic brain function network learning method based on graph learning optimization algorithm | |
| CN115909016B (en) | GCN-based fMRI image analysis system, method, electronic equipment and medium | |
| CN113077483B (en) | Method and system for predicting distribution of induced electric field based on brain tissue and depth regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |