CN115778323A - User rehabilitation level assessment system with multi-source data fusion - Google Patents
User rehabilitation level assessment system with multi-source data fusion Download PDFInfo
- Publication number
- CN115778323A CN115778323A CN202211453685.6A CN202211453685A CN115778323A CN 115778323 A CN115778323 A CN 115778323A CN 202211453685 A CN202211453685 A CN 202211453685A CN 115778323 A CN115778323 A CN 115778323A
- Authority
- CN
- China
- Prior art keywords
- data
- brain
- brain function
- taking
- phase coherence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title abstract description 10
- 230000003925 brain function Effects 0.000 claims abstract description 76
- 210000004556 brain Anatomy 0.000 claims abstract description 56
- 238000011156 evaluation Methods 0.000 claims abstract description 35
- 230000001364 causal effect Effects 0.000 claims abstract description 33
- 230000000284 resting effect Effects 0.000 claims abstract description 32
- 230000000694 effects Effects 0.000 claims abstract description 30
- 238000004070 electrodeposition Methods 0.000 claims abstract description 14
- 238000013507 mapping Methods 0.000 claims abstract description 13
- 238000010276 construction Methods 0.000 claims abstract description 8
- 230000006399 behavior Effects 0.000 claims description 38
- 210000001364 upper extremity Anatomy 0.000 claims description 25
- 230000033001 locomotion Effects 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000001133 acceleration Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 8
- 210000000337 motor cortex Anatomy 0.000 claims description 8
- 210000000869 occipital lobe Anatomy 0.000 claims description 8
- 210000002442 prefrontal cortex Anatomy 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 230000001766 physiological effect Effects 0.000 claims description 3
- 239000003814 drug Substances 0.000 abstract description 2
- 208000006011 Stroke Diseases 0.000 description 9
- 238000000034 method Methods 0.000 description 9
- 238000002474 experimental method Methods 0.000 description 8
- 238000012360 testing method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000011282 treatment Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- IBOFVQJTBBUKMU-UHFFFAOYSA-N 4,4'-methylene-bis-(2-chloroaniline) Chemical compound C1=C(Cl)C(N)=CC=C1CC1=CC=C(N)C(Cl)=C1 IBOFVQJTBBUKMU-UHFFFAOYSA-N 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 2
- 206010008190 Cerebrovascular accident Diseases 0.000 description 2
- 241001112258 Moca Species 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011369 optimal treatment Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006931 brain damage Effects 0.000 description 1
- 231100000874 brain damage Toxicity 0.000 description 1
- 208000029028 brain injury Diseases 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Landscapes
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention relates to the technical field of rehabilitation medicine, and provides a user rehabilitation level assessment system for multi-source data fusion, which comprises: a network construction module configured to: respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; an evaluation module configured to: and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network. And a reference index is provided for the subsequent establishment of a rehabilitation scheme from a multi-modal and multi-dimensional view.
Description
Technical Field
The invention belongs to the technical field of rehabilitation medicine, and particularly relates to a user rehabilitation level assessment system with multi-source data fusion.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Stroke is one of the leading causes of death and disability worldwide. Over 1 million people worldwide have experienced the effects of stroke, and the incidence of stroke, mortality, and prevalence has increased in people under 70 years of age over the past few decades. Because the apoplexy patient has symptoms of unclear expression, incoherent speaking and the like in speech expression, and the normal life of the patient is seriously influenced, an accurate and effective evaluation method for apoplexy is urgently needed.
At present, research means for stroke mainly comprises subjective evaluation based on scale evaluation and objective measurement based on physiological signals, a single scale can only explain the problems of cognition, memory and the like of a patient, and the physiological signals can reflect the activation degree and connection relation of brain regions corresponding to the capacities, but the methods have single dimensionality for data processing, are easily influenced by the behaviors of the patient and are difficult to achieve a good diagnosis effect. Therefore, there is a need for a more comprehensive and accurate method to help physicians understand the patient's level of rehabilitation to develop an optimal treatment regimen.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a user rehabilitation level evaluation system with multi-source data fusion.
In order to achieve the purpose, the invention adopts the following technical scheme:
the first aspect of the invention provides a multi-source data fused user rehabilitation level assessment system, which comprises:
a data acquisition module configured to: obtaining scale data, behavior data, and brain function data in a resting state and a task state of a subject;
a network construction module configured to: respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the intercalary granger causal relationship is the mean of the granger causal relationships of all channel pairs between two regions;
an evaluation module configured to: and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
Further, a coherency calculation module is included that is configured to: calculating wavelet phase coherence between every two channels of brain function data, generating a plurality of substitute signal pairs by using Fourier transform for the brain function data of the two channels respectively, calculating the mean value of wavelet phase coherence values between the substitute signal pairs and the double standard deviation of the wavelet phase coherence values between the substitute signal pairs, and judging whether to set the wavelet phase coherence value between the two channel pairs to be 0 or not by comparing the wavelet phase coherence value between the substitute signal pairs with the result of adding the double standard deviation to the mean value.
Further, a pre-processing module is included that is configured to: abnormal values, physiological activity noise and motion artifacts in brain function data are removed.
Further, the evaluation module is configured to: calculating a behavior index for evaluation of brain function of the subject based on the behavior data; wherein the behavior index comprises upper limb speed, standard deviation of upper limb speed, median upper limb speed, upper limb acceleration, standard deviation of upper limb acceleration and average motion angle of the left arm.
Further, the brain region includes a left prefrontal cortex, a right prefrontal cortex, a left motor cortex, a right motor cortex, a left occipital lobe, and a right occipital lobe.
Further, the behavior data is obtained by capturing the upper limb movement data of the subject during the brain function data acquisition process in the task state.
Further, the mapping model adopts a Keras deep learning framework comprising two branches; one branch is a multi-layer perceptron for handling numerical inputs; the other branch is a convolutional neural network, which operates on the adjacency matrix.
Further, the multi-layered perceptron includes a fully connected input layer with ReLU activation, a fully connected hidden layer, and a regression output.
A second aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
obtaining the comprehensive evaluation result of the brain function of the subject through a mapping model based on scale data, behavior data, a brain function connection network and a brain effect connection network
A third aspect of the present invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the program:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a user rehabilitation level evaluation system with multi-source data fusion, which aims at the problems that the traditional rehabilitation treatment of a stroke patient is restricted by the subjective clinical experience of a therapist at present and the rehabilitation progress of the patient is difficult to reflect in real time.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a structural diagram of a user rehabilitation level evaluation system with multi-source data fusion according to a first embodiment of the present invention;
FIG. 2 is a node location diagram according to a first embodiment of the present invention;
FIG. 3 is a diagram of a brain function connection network based on wavelet phase coherence according to a first embodiment of the present invention;
fig. 4 is a diagram of a glangel causal relationship-based brain effect connection network according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
Example one
The embodiment provides a multi-source data fusion user rehabilitation level evaluation system, as shown in fig. 1, including the following modules:
a data acquisition module configured to: scale data, behavioral data, and brain function data in resting and task states are obtained for the subject.
Before each time of collecting the brain function data of the stroke patient, the age, sex, height, weight and other information of the user are recorded in a quiet environment, and the brain damage state and various ability indexes of the patient are evaluated through Brunnstrom, MMSE, MOCA, fugl-Meyer and NIHSS scales.
Under the single task, the patient gathers brain function data through wearing near-infrared equipment, and the experiment is carried out after wearing the completion, and the experiment divide into two parts: the method comprises the steps of a rest state and a task state, wherein behavior data of a user are synchronously collected in the task state experiment process.
As shown in fig. 2, the acquisition channel positions of the near-infrared device on the human brain include 6 brain regions of the left prefrontal cortex (LPFC), the right prefrontal cortex (RPFC), the Left Motor Cortex (LMC), the Right Motor Cortex (RMC), the Left Occipital Lobe (LOL), and the Right Occipital Lobe (ROL), and the electrode positions are set according to international standard 10/10.
Wherein the near infrared device uses a nirsort portable near infrared brain function imaging system of danyang comet-invasive medical instruments ltd, a frequency of 10Hz, and wavelengths of 730 and 850nm used, for 18 channels, which are arranged above the left prefrontal cortex (LPFC), right prefrontal cortex (RPFC), left Motor Cortex (LMC), right Motor Cortex (RMC), left Occipital Lobe (LOL), and Right Occipital Lobe (ROL) of the brain following the international 10/10 electrode distribution system.
Specifically, the near infrared data are collected for 10 minutes in a quiet environment after the wearing of the test piece, the data are data of the rest state of the test piece, and the test piece is in a relaxed state of an eye-open sitting posture all the time; after resting for 10 minutes, the subject executes a task according to the requirement, the recorded near-infrared data is in a task state, the Kinect equipment is opened to capture upper limb movement data and acquire the data, and the data acquisition time is also 10 minutes; the behavior data acquisition uses Microsoft Kinect V2 equipment, kinect V2 uses infrared light to track multiple parts of a body in real time, at most 25 skeleton nodes are supported, and nodes numbered 1-11 and 20-24 are selected for upper limb data analysis. The data object types are provided in the form of a skeletal framework. Before the experiment began, each subject was instructed on the task rules to ensure that they learned the rules and were ready for testing, and that the subjects had a 10 minute rest time to ensure that the patients were in a relaxed state. The data is marked by the marking at the beginning and at the end of the job.
A pre-processing module configured to: and preprocessing the data acquired by the data acquisition module, including removing abnormal values, physiological activity noise and motion artifacts in the near infrared data.
The method comprises the steps of firstly intercepting near infrared and behavior data acquired in a task process according to a marking result, then eliminating abnormal values with absolute value threshold values exceeding 5 through moving average processing on the near infrared data, adopting an average value of 5 points before the abnormal points to replace the abnormal results during research, then removing motion artifacts from the near infrared data of each channel by using a time derivative method TDDR based on robust regression, and finally filtering signals by using a 6-order Butterworth filter, wherein the selected passband is 0.01-0.08.
A coherency calculation module configured to: calculating Wavelet Phase Coherence (WPCO) between every two channels of the data obtained by the preprocessing module, evaluating the relevance between the obtained data, and testing the significance of the WPCO by generating 100 substitute signals of the data of the two channels.
Wherein, the test mode is as follows: and calculating the sum of the average value and two times of standard deviation of the wavelet phase coherence value between the substitute signal pairs, if the value is smaller than the wavelet phase coherence value between the original data, considering the original data to be valid, and if not, setting the result between the channel pairs to be 0.
Wherein two near infrared devices collectThe calculation formula of the wavelet phase coherence of the data between channels is as follows: two time series data x 1 (t) and x 2 (t) after successive wavelet transforms, they are at a frequency f and time t n Corresponding instantaneous phase of time phi respectively 1 (f,t n ) And phi 2 (f,t n ) Calculating their instantaneous phase difference Δ φ (f, t) n )＝φ 1 (f,t n )-φ 2 (f,t n ) Then cos Δ φ (f, t) n ) And sin Δ φ (f, t) n ) Averaging in the time domain, i.e.
Where N is the total number of time points in the time series, and the value of the wavelet phase coherence of the two signals at the frequency f is obtained:
the wavelet phase coherence value (WPCO) between the two signals can be obtained by averaging the wavelet phase coherence values of the two signals under all frequencies f. The resulting wavelet phase coherence should be between 0-1, since the near infrared devices used above have 18 channels, the result here between the channels should be an 18 x 18 matrix with a diagonal element of 0.
For the wavelet phase coherence value between the two calculated channel data, the significance of the wavelet phase coherence value is tested by generating 100 substitute signals of the two channel data; the test mode is as follows: for original near-infrared data of two channels, 100 alternative signal pairs are generated by respectively using Fourier transform, the mean value of WPCO between the 100 alternative signal pairs and the two times standard deviation of the 100 WPCO results are calculated in the same way, the result obtained by adding the two times standard deviation to the mean value of the WPCO of the original channel data and the alternative signal results is compared, if the value is smaller than the wavelet phase coherence value between the original data, the original data is considered to be valid, and if not, the result between the channel pairs is set to be 0.
A network construction module configured to: respectively constructing a brain function connection network for a resting state and a task state by taking a brain area where an electrode position for collecting the brain function data is positioned as a node, taking connection among the nodes as an edge and taking phase coherence among the areas as a weight, and analyzing the synergistic effect among different brain areas of the brain, namely the neural activity and the connectivity of the brain function; wherein the inter-region phase coherence is the average of wavelet phase coherence values of all channel pairs between two regions.
Specifically, as shown in fig. 3, the brain region where the electrode position for acquiring the data is located is taken as a node, that is, there are 6 nodes in total, and each represents 6 brain regions, and the connection between the nodes is taken as an edge, and the wavelet phase coherence between the regions is calculated as a weight of the edge, for example, in a channel of nirsort, the brain region 1 includes four channels 13, 14, 17, and 18, and the brain region 2 includes four channels 11, 12, 15, and 16, so that the result of the wavelet phase coherence between the brain region 1 and the brain region 2 is the average of the wavelet phase coherence values of the 16 channel pairs (13-11, 13-12, 13-15, 13-16, 14-11, …, 18-16) obtained in step 5, thereby respectively constructing the brain function connection network in the resting state and the task state.
A network construction module configured to: and performing Glan's causal relationship calculation on the data obtained by the preprocessing module, and respectively constructing a brain effect connection network for a resting state and a task state by taking a brain region where the electrode position for acquiring the data is positioned as a node and the Glan's causal relationship among the regions, thereby analyzing the causal relationship of the interaction between the brain regions.
Specifically, as shown in fig. 4, glange causal relationship calculation is performed on the data obtained by the preprocessing module, and the hermitian causal relationship and the task-state brain effect connection networks are respectively constructed according to the glange causal relationship between the regions by using the hemmes software, and also by using the brain region where the electrode position for acquiring the data is located as a node, for example, after obtaining the glange causal relationship (GC) between the data, the effect connection result between the brain region 1 and the brain region 2 is the GC mean value of the 16 channel pairs (13 → 11, 13 → 12, 13 → 15, 13 → 16, 14 → 11, …,18 → 16) between the brain region 1 and the brain region 2 is the reverse GC mean value of the GC result between the 16 channel pairs (11 → 13, 11 → 14, 11 → 17, 11 → 18, 12 → 13, …,16 → 18).
An evaluation module configured to: and a doctor with abundant clinical experience judges a comprehensive evaluation result of the brain function of the subject under the task according to the subjective scale data acquired by the data acquisition module and the brain function connection and effect connection network multi-source data acquired by the network construction module.
Specifically, a json file generated by the Kinect device is read by using Python based on the behavior data obtained by the preprocessing module, the json file records the position information (namely, coordinates of different times) of each joint point, and each position has a corresponding time. The velocity of each point at the corresponding instant can be obtained by dividing the distance between the locations by the time. In addition, the overall upper limb velocity is the average value of all upper limb point velocities, and the standard deviation of the upper limb velocity, the median upper limb velocity (i.e. the velocity average value of the corresponding points), the overall upper limb acceleration, the standard deviation of the overall upper limb acceleration and the like can be obtained in the same way, and the calculation formula of other indexes such as the average motion angle of the left arm is that for three points E (x, y, z), S (x, y, z) and H (x, y, z),
and according to the calculated behavior data result, and by combining the subjective scale acquired by the data acquisition module and the multi-source data of the brain function connection and effect connection network acquired by the network construction module, seeking a comprehensive evaluation result of the brain function of the subject under the task judged by a doctor with abundant clinical experience.
After a period of time, the data acquisition task is carried out on the patient again, the data acquisition module, the preprocessing module, the coherence calculation module, the network construction module and the evaluation module are repeatedly executed, and the scale, the brain function connection network, the brain effect connection network, the behavior index and the comprehensive evaluation result judged by the doctor of the patient under the task are also recorded.
After multiple experiments, a mapping model is established by using a Keras deep learning framework of python and the comprehensive evaluation results judged by doctors and the multi-source data of the scales, the brain function connection networks, the brain effect connection networks and the behavior indexes obtained in all the experiments, and accordingly, the comprehensive evaluation results of the brain functions of the testees are directly obtained by collecting task data in subsequent treatment, and the rehabilitation level of the user is evaluated.
Specifically, after multiple experiments, with the increase of data, a mapping model can be established by using a Keras deep learning framework of python, the multi-source data of the scales, the brain function connection network, the brain effect connection network and the behavior indexes obtained under all the experiments and the comprehensive evaluation result judged by the doctor, for example, each scale is input as a time sequence on the result at different time, and when the used scales are brunstrom, MMSE, MOCA, fugl-Meyer and NIHSS, the input of the scale characteristics is the time sequence of 5 scales. For the brain function connection network and the brain effect connection network, the adjacency matrixes of the four networks in the resting state and the task state are respectively used as variable inputs, namely the input of the network characteristics is 4 adjacency matrix sequences in the time dimension. Similarly, each behavior index is input as an individual variable, and as 6 indexes of the overall upper limb velocity, the standard deviation of the upper limb velocity, the median upper limb velocity, the overall upper limb acceleration, the standard deviation of the overall upper limb acceleration, and the average motion angle of the left arm used in step 7, the input of the behavior characteristics is time-series data of the 6 behavior indexes. Therefore, the Keras deep learning framework has 11 time-series data and 4 adjacency matrix sequences as inputs, and outputs the comprehensive evaluation results given to each doctor. For the input of the multi-source data, two branches are established in Keras, wherein the first branch is a simple multilayer perceptron (MLP) used for processing numerical value input, the second branch is a convolution neural network used for operating a adjacency matrix, and then the branches are connected together to form a final multi-input Keras model. Wherein the MLP contains a fully-connected input layer with ReLU activation, a fully-connected hidden layer (also with ReLU), an optional regression output for linear activation; for the CNN module of the adjacency matrix, all the extracted features are combined into a one-dimensional feature vector, and a full connection layer with BatchNormalization and Dropout is added. The output results of the two branches are combined together, a complete connection layer is added finally, the comprehensive evaluation result given by a doctor each time is output, a brain function comprehensive evaluation model constructed by a scale, brain functions and behavior data is established by training the model, the comprehensive evaluation result of the brain functions of the testee can be directly obtained by acquiring task data in subsequent treatment, and the rehabilitation level of the user is evaluated.
The multi-source data fusion user rehabilitation level assessment system provided by the embodiment is used for solving the problems that the traditional rehabilitation treatment of the current stroke patient is restricted by the subjective clinical experience of a therapist and the rehabilitation progress of the patient is difficult to reflect in real time.
According to the user rehabilitation level evaluation system with multi-source data fusion, aiming at the problem that the rehabilitation progress of a patient is difficult to reflect in real time at present, the patient is subjected to rehabilitation evaluation by combining multi-source data such as the brain function, the behavior and the subjective scale of the user, and a doctor can be helped to know the rehabilitation level of the patient to make an optimal treatment scheme by a more comprehensive and accurate method.
Example two
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
EXAMPLE III
The embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the following steps:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for acquiring the brain function data is located as a node, taking connection among the nodes as a side and taking inter-area phase coherence as a weight, and respectively constructing a brain effect connection network for the resting state and the task state by taking inter-area Glan causal relation; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A multi-source data fused user rehabilitation level assessment system, comprising:
a data acquisition module configured to: obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
a network construction module configured to: respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
an evaluation module configured to: and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
2. The multi-source data-fused user rehabilitation level assessment system according to claim 1, further comprising a coherence calculation module configured to: calculating wavelet phase coherence between every two channels of brain function data, generating a plurality of substitute signal pairs by using Fourier transform for the brain function data of the two channels respectively, calculating the mean value of wavelet phase coherence values between the substitute signal pairs and the double standard deviation of the wavelet phase coherence values between the substitute signal pairs, and judging whether to set the wavelet phase coherence value between the two channel pairs to be 0 or not by comparing the wavelet phase coherence value between the substitute signal pairs with the result of adding the double standard deviation to the mean value.
3. The multi-source data-fused user rehabilitation level assessment system according to claim 1, further comprising a preprocessing module configured to: abnormal values, physiological activity noise and motion artifacts in brain function data are removed.
4. The multi-source data-fused user rehabilitation level assessment system according to claim 1, wherein said assessment module is configured to: calculating a behavior index for evaluation of brain function of the subject based on the behavior data; wherein the behavior index comprises upper limb speed, standard deviation of upper limb speed, median upper limb speed, upper limb acceleration, standard deviation of upper limb acceleration and average motion angle of the left arm.
5. The multi-source data fused user rehabilitation level assessment system according to claim 1, wherein the brain region comprises left prefrontal cortex, right prefrontal cortex, left motor cortex, right motor cortex, left occipital lobe and right occipital lobe.
6. The multi-source data fused user rehabilitation level assessment system according to claim 1, wherein the behavior data is upper limb movement data of the subject captured during brain function data acquisition in task state.
7. The system of claim 1, wherein the mapping model employs a Keras deep learning framework comprising two branches; one branch is a multi-layer perceptron for handling numerical inputs; the other branch is a convolutional neural network, which operates on the adjacency matrix.
8. The multi-source data-fused user rehabilitation level assessment system according to claim 7, wherein said multi-layered perceptron comprises a fully-connected input layer with ReLU activation, a fully-connected hidden layer and a regression output.
9. A computer-readable storage medium, on which a computer program is stored, which program, when executed by a processor, performs the steps of:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the intercalary granger causal relationship is the mean of the granger causal relationships of all channel pairs between two regions;
and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of:
obtaining scale data, behavior data, and brain function data of a subject in a resting state and a task state;
respectively constructing brain function connection networks for the resting state and the task state by taking a brain area where an electrode position for collecting the brain function data is located as a node, taking connection among the nodes as an edge and taking inter-area phase coherence as a weight, and respectively constructing brain effect connection networks for the resting state and the task state by taking inter-area Glan's causal relationship; the inter-region phase coherence is the average value of wavelet phase coherence of all channel pairs between two regions; the inter-region glancing causal relationship is the mean of the glancing causal relationships of all channel pairs between two regions;
and obtaining a comprehensive evaluation result of the brain function of the subject through a mapping model based on the scale data, the behavior data, the brain function connection network and the brain effect connection network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211453685.6A CN115778323B (en) | 2022-11-21 | 2022-11-21 | User rehabilitation level assessment system with multi-source data fusion function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211453685.6A CN115778323B (en) | 2022-11-21 | 2022-11-21 | User rehabilitation level assessment system with multi-source data fusion function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115778323A true CN115778323A (en) | 2023-03-14 |
| CN115778323B CN115778323B (en) | 2024-04-26 |
Family
ID=85439261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211453685.6A Active CN115778323B (en) | 2022-11-21 | 2022-11-21 | User rehabilitation level assessment system with multi-source data fusion function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115778323B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030941A (en) * | 2023-03-30 | 2023-04-28 | 北京科技大学 | Alzheimer's disease diagnosis method based on edge-centric effect connection network |
| CN116649916A (en) * | 2023-07-24 | 2023-08-29 | 北京大学 | Causal difference estimation method for brain region connection |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9107595B1 (en) * | 2014-09-29 | 2015-08-18 | The United States Of America As Represented By The Secretary Of The Army | Node excitation driving function measures for cerebral cortex network analysis of electroencephalograms |
| CN108685577A (en) * | 2018-06-12 | 2018-10-23 | 国家康复辅具研究中心 | A kind of brain function rehabilitation efficacy apparatus for evaluating and method |
| CN110251124A (en) * | 2019-07-19 | 2019-09-20 | 太原理工大学 | A kind of determination method and system of validity brain network |
| US20200222010A1 (en) * | 2016-04-22 | 2020-07-16 | Newton Howard | System and method for deep mind analysis |
| CN113558639A (en) * | 2021-07-05 | 2021-10-29 | 杭州电子科技大学 | Motor intention brain muscle network analysis method based on glange causal relation and graph theory |
| CN113947157A (en) * | 2021-10-26 | 2022-01-18 | 北京工业大学 | Dynamic brain effect connection network generation method based on hierarchical clustering and structural equation model |
| CN115067970A (en) * | 2022-07-01 | 2022-09-20 | 山东中科先进技术有限公司 | Rehabilitation effect evaluation method and system based on electroencephalogram and electromyogram signals |
| CN115349857A (en) * | 2022-07-18 | 2022-11-18 | 国家康复辅具研究中心 | Dynamic rehabilitation assessment method and system based on fNIRS brain function map |
-
2022
- 2022-11-21 CN CN202211453685.6A patent/CN115778323B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9107595B1 (en) * | 2014-09-29 | 2015-08-18 | The United States Of America As Represented By The Secretary Of The Army | Node excitation driving function measures for cerebral cortex network analysis of electroencephalograms |
| US20200222010A1 (en) * | 2016-04-22 | 2020-07-16 | Newton Howard | System and method for deep mind analysis |
| CN108685577A (en) * | 2018-06-12 | 2018-10-23 | 国家康复辅具研究中心 | A kind of brain function rehabilitation efficacy apparatus for evaluating and method |
| CN110251124A (en) * | 2019-07-19 | 2019-09-20 | 太原理工大学 | A kind of determination method and system of validity brain network |
| CN113558639A (en) * | 2021-07-05 | 2021-10-29 | 杭州电子科技大学 | Motor intention brain muscle network analysis method based on glange causal relation and graph theory |
| CN113947157A (en) * | 2021-10-26 | 2022-01-18 | 北京工业大学 | Dynamic brain effect connection network generation method based on hierarchical clustering and structural equation model |
| CN115067970A (en) * | 2022-07-01 | 2022-09-20 | 山东中科先进技术有限公司 | Rehabilitation effect evaluation method and system based on electroencephalogram and electromyogram signals |
| CN115349857A (en) * | 2022-07-18 | 2022-11-18 | 国家康复辅具研究中心 | Dynamic rehabilitation assessment method and system based on fNIRS brain function map |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116030941A (en) * | 2023-03-30 | 2023-04-28 | 北京科技大学 | Alzheimer's disease diagnosis method based on edge-centric effect connection network |
| CN116030941B (en) * | 2023-03-30 | 2023-08-25 | 北京科技大学 | Alzheimer's disease diagnosis method based on edge-centric effect connection network |
| CN116649916A (en) * | 2023-07-24 | 2023-08-29 | 北京大学 | Causal difference estimation method for brain region connection |
| CN116649916B (en) * | 2023-07-24 | 2023-10-20 | 北京大学 | Causal difference estimation method for brain region connection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115778323B (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wasimuddin et al. | Stages-based ECG signal analysis from traditional signal processing to machine learning approaches: A survey | |
| CN115778323B (en) | User rehabilitation level assessment system with multi-source data fusion function | |
| CN110420019B (en) | Deep regression heart rate estimation method for ballistocardiogram signals | |
| Wan et al. | Heartbeat classification algorithm based on one-dimensional convolution neural network | |
| CN109157231A (en) | Portable multi-channel Depression trend assessment system based on emotional distress task | |
| CN110619322A (en) | Multi-lead electrocardio abnormal signal identification method and system based on multi-flow convolution cyclic neural network | |
| US10719932B2 (en) | Identifying suspicious areas in ophthalmic data | |
| Obeidat et al. | A hybrid lightweight 1D CNN-LSTM architecture for automated ECG beat-wise classification. | |
| MacIsaac et al. | Fatigue estimation with a multivariable myoelectric mapping function | |
| CN112932486A (en) | Multi-mode-based university student emotional stress detection system and method | |
| Andrade et al. | An EEG brain-computer interface to classify motor imagery signals | |
| Pombo et al. | Big data reduction using RBFNN: A predictive model for ECG waveform for eHealth platform integration | |
| Legaria-Santiago et al. | Computer models evaluating hand tremors in Parkinson's disease patients | |
| Du et al. | Recognition of child congenital heart disease using electrocardiogram based on residual of residual network | |
| Dubey et al. | A neural network approach for ECG classification | |
| Khan et al. | Detection and classification of three-class initial dips using vector phase analysis with dual threshold circles: an fNIRS study | |
| de Melo | A brief review on electrocardiogram analysis and classification techniques with machine learning approaches | |
| Cai et al. | Associating endpoint accuracy and similarity of muscle synergies | |
| Almazroa et al. | An internet of things (iot) homecare management system using cardiac arrhythmia classification | |
| Tonekabonipour et al. | Comparison of neuro-fuzzy approaches with artificial neural networks for the detection of Ischemia in ECG signals | |
| Hosen et al. | Age classification based on EMG signal using Artificial Neural Network | |
| Pérez Valero | Automated Detection of Alzheimer's disease and other neurophysiological applications based on EEG | |
| Asanza et al. | Electrooculography Signals Classification for FPGA-based Human-Computer Interaction | |
| Sarin et al. | Analyzing and Comparing MLP, CNN, and LSTM for Classification of Heart Arrhythmia Using ECG Scans | |
| Movahed et al. | An Automated EEG-based mild cognitive impairment diagnosis framework using spectral and functional connectivity features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |