CN115358264A - Invasive brain-computer interface task related neural signal extraction method - Google Patents

Invasive brain-computer interface task related neural signal extraction method Download PDF

Info

Publication number
CN115358264A
CN115358264A CN202210973148.8A CN202210973148A CN115358264A CN 115358264 A CN115358264 A CN 115358264A CN 202210973148 A CN202210973148 A CN 202210973148A CN 115358264 A CN115358264 A CN 115358264A
Authority
CN
China
Prior art keywords
neural
loss function
neural signal
encoder
computer interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210973148.8A
Other languages
Chinese (zh)
Inventor
祁玉
李彦钢
王跃明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202210973148.8A priority Critical patent/CN115358264A/en
Publication of CN115358264A publication Critical patent/CN115358264A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Neurology (AREA)
  • Computational Linguistics (AREA)
  • Neurosurgery (AREA)
  • Dermatology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses an invasive brain-computer interface task related neural signal extraction method, which comprises the following steps: (1) Constructing an encoder h, converting an original neural signal x into a feature space, obtaining a neural representation z = h (x), and decoding z by using an affine function f to obtain a first predicted speed y of a target speed y 1 = f (z); (2) The construction decoder g generates a neural signal x from the neural representation z r = g (z); (3) Neural signal x to be generated r Re-sending to the encoder h and the affine function f to obtain a second predicted speed y 2 ＝f(h(x r ) ); (4) A priori generated neural network m is constructed,learning the prior neural representation z from the target velocity y p = m (y); (5) Calculating a loss function, and training an encoder, a decoder and a priori generating a neural network at the same time; (6) And inputting the original signal to be processed into the trained encoder and decoder to obtain the generated neural signal. The invention can obviously improve the decoding performance and the generation performance.

Description

Invasive brain-computer interface task related neural signal extraction method
Technical Field
The invention relates to the field of invasive action potential brain signal analysis, in particular to an invasive brain-computer interface task related neural signal extraction method.
Background
Understanding how the brain encodes and decodes task information is a fundamental goal of neuroscience and neuroengineering. While neural signals exhibit strong variability, repeated repetition of the same experimental conditions can result in significantly different neural activity. These changes can be caused by many factors, such as intrinsic noise of the nervous system, neural plasticity, instability of signal recordings, and simultaneous encoding of multiple task parameters by neurons, among others.
The significant variability of neural signals has a great impact on both neuro-mechanical studies and Brain-machine Interfaces (BMIs) applications. Since the true mission-related signal is unknown, current studies are based on study analysis of the original noisy signal, which may lead to unreliable conclusions. Therefore, the method has great significance for the fields of neuroscience and neuroengineering by extracting clean task related signals as far as possible.
For example, chinese patent publication No. CN103584851A discloses a multichannel neuron signal acquisition, regulation, and transmission device, and designs a device for multichannel parallel acquisition based on FPGA and synchronous stimulation according to the requirement of neuroelectrophysiology on neuron information analysis, which is suitable for multichannel neuron signal extraction for animals and provides a plurality of programmable stimulation waveforms.
Chinese patent publication No. CN111984118A discloses a method for decoding an electromyographic signal from an electroencephalogram signal based on a complex recurrent neural network, comprising: constructing electroencephalogram signal data and electromyogram signal data, and respectively preprocessing; respectively carrying out short-time Fourier transform on the preprocessed results; inputting the transformed result into a complex recurrent neural network for training and testing; and carrying out inverse short-time Fourier transform on the test result to obtain a decoding result.
Some methods are available to extract task-related neural signals, such as pi-VAE, PSID, VAE, LFADS, etc., but they can extract only a small portion of the neural signals. Currently, no method is focused on maximizing the method for extracting task-related signals.
Disclosure of Invention
The invention provides an invasive brain-computer interface task related neural signal extraction method, which can maximally extract task related signals and obviously improve decoding performance and generation performance compared with other methods.
The technical scheme of the invention is as follows:
an invasive brain-computer interface task related neural signal extraction method comprises the following steps:
(1) Constructing an encoder h, converting an original neural signal x into a feature space, obtaining a neural representation z = h (x), and decoding z by using an affine function f to obtain a first predicted speed y of a target speed y 1 ＝f(z)；
(2) The construction decoder g generates a neural signal x from the neural representation z r ＝g(z)；
(3) Neural signal x to be generated r Re-feeding the encoder h and the affine function f to obtain a second predicted speed y 2 ＝f(h(x r ))；
(4) Building a priori generation neural network m, and learning a priori neural representation z by a target speed y p ＝m(y)；
(5) Calculating a loss function, and training an encoder, a decoder and a priori generating a neural network at the same time;
(6) And inputting the original signal to be processed into the trained encoder and decoder to obtain the generated neural signal.
Further, in the invention, the encoder, the decoder and the priori generation neural network all adopt a multilayer perceptron.
In step (5), an end-to-end mode is adopted to carry out back propagation to train a coder, a decoder and a priori generation neural network.
The specific process of calculating the loss function is as follows:
(5-1) calculating the first predicted speed y 1 Loss function of target speed y
Figure BDA0003796959050000031
(5-2) computationally generating a neural signal x r Loss function from original neural signal x
Figure BDA0003796959050000032
(5-5) calculating the second predicted speed y 2 Loss function of target speed y
Figure BDA0003796959050000033
(5-6) calculating the prior neural representation z p KL divergence loss function from neural representation z
Figure BDA0003796959050000034
(5-5) weighting and combining the four loss functions to obtain the final loss
Figure BDA0003796959050000035
The formula is as follows:
Figure BDA0003796959050000036
in the step (5-1), the first predicted speed y is calculated 1 Loss function of target speed y
Figure BDA0003796959050000037
The mean square error loss function is adopted, and the formula is as follows:
Figure BDA0003796959050000038
where i represents the ith sample and N represents the total number of samples.
In the step (5-2), the neural signal x is calculated and generated r Loss function from the original neural signal x
Figure BDA0003796959050000039
The Poisson negative log-likelihood loss function is adopted, and the formula is as follows:
Figure BDA00037969590500000310
in the step (5-3), the second predicted speed y is calculated 2 Loss function of target speed y
Figure BDA00037969590500000311
Also, a mean square error loss function is used, and the formula is:
Figure BDA00037969590500000312
in step (5-4), a priori neural representation z is calculated p KL divergence loss function from neural representation z
Figure BDA00037969590500000313
The formula is as follows:
Figure BDA00037969590500000314
wherein q and p represent z and z, respectively p The distribution of (c).
Compared with the prior art, the invention has the following beneficial effects:
1. the method of the invention is improved on the basis of the variational self-encoder, decoding constraint is carried out on the neural representation z, and decoding constraint is carried out after the original signal is reconstructed (generated), so that the task related signal can be maximally extracted, and the method is beneficial to the research of the subsequent neural mechanism and the application of the neural engineering.
2. Compared with other methods, the method provided by the invention has the advantage that the decoding performance and the generation performance of the extracted neural signals are obviously improved.
Drawings
FIG. 1 is a schematic diagram of an experiment of decoding a motion signal in an embodiment of the invention;
FIG. 2 is an architecture diagram of an invasive brain-computer interface task related neural signal extraction method according to the present invention;
FIG. 3 is a graph comparing the decoding and resulting average performance of embodiments of the present invention and other methods on two days of test data.
Detailed Description
The invention will be described in further detail below with reference to the drawings and examples, which are intended to facilitate the understanding of the invention without limiting it in any way.
The present invention uses data from a monkey collected in the literature "Li, h., hao, y., zhang, s., wang, y., chen, w., & Zheng, x. (2017). Primer Knowledge of Target Direction and integrated motion Selection improvements index reading motion decoding. Behavioural Neurology,2017.
In the experiment, an adult male rhesus monkey is trained, and the monkey controls a large rocker through an arm to complete the task of avoiding an obstacle and moving a cursor to a target point. The monkey is fixed to be sat on the chair, and the display is put 50 centimetres in front of the chair, and the monkey is trained to use the arm to control the cursor that the rocker moved on the two-dimensional plane and avoids barrier (rectangle strip) from initial position (small circle) and reachs target position (great circle), and the system can give the monkey as reward by automatic feedwater this moment, specifically as shown in fig. 1.
The experiment used a 96-channel array of microelectrodes (10 × 10 matrix arrangement, 4.2 × 4.2 mm) implanted in the dorsal anterior motor cortex region (PMd) of the monkey. The neural signals obtained by the array are transmitted to the Cerebus system, the analog waveforms of the neural signals are amplified and then filtered using a Butterworth filter band-pass filter (0.3 Hz to 7.5 kHz), followed by digitization (16-bit resolution and 30kHz sampling rate) and Butterworth high-pass filtering (250 Hz). Finally, acquiring an action potential pulse signal by using a threshold method (minus 4.5 times of baseline root mean square).
In the present invention, with the two-day data test, the pulse signal is subjected to binning processing with a time window of 100ms using MUA (multi unit activity) data.
The method frame diagram of the invention is shown in fig. 2, and an invasive brain-computer interface task related neural signal extraction method comprises the following steps:
step S1, constructing an Encoder (Encoder) h, converting an original neural signal x into a feature space to obtain a neural representation z = h (x), and decoding by using an affine function f z Obtaining a first predicted speed y of the target speed y 1 ＝f(z)。
Step S2, constructing a Decoder (Decoder) g to generate a neural signal x from the neural representation z r ＝g(z)。
S3, the generated neural signals are sent to the encoder h and the affine function f again to obtain a second predicted speed y 2 ＝f(h(x r ))。
S4, constructing a priori generated neural network m, and learning a priori neural representation z from the target speed y p ＝m(y)。
And S5, calculating a loss function, and reversely propagating the training model (the encoder, the decoder and the prior generated neural network) in an end-to-end mode.
S501, calculating the predicted speed y 1 Loss function of target speed y
Figure BDA0003796959050000051
Figure BDA0003796959050000052
Where i represents the ith sample and N represents the total number of samples.
S502, calculating and generating a neural signal x r Loss function from the original neural signal x
Figure BDA0003796959050000061
Figure BDA0003796959050000062
S503, calculating the predicted speed y 2 Loss function of target speed y
Figure BDA0003796959050000063
Figure BDA0003796959050000064
S504, calculating the prior neural representation z p KL divergence loss function from neural representation z
Figure BDA0003796959050000065
Figure BDA0003796959050000066
Wherein q and p represent z and z, respectively p Distribution of (2).
S505, the four loss functions are combined in a weighting mode to obtain the final loss
Figure BDA0003796959050000067
The formula is as follows:
Figure BDA0003796959050000068
in the invention, the encoder, the decoder and the priori generation neural network can adopt a multilayer perceptron.
In order to verify the feasibility of the method for extracting the task related signals in the motion area, the performance of the model is tested in a monkey motion obstacle avoidance paradigm experiment as a special case. In the experiment, an invasive electrode array is adopted to capture signals of a front motor core (PMD) of the back side of the monkey brain, and the signals are collected into a multi-channel discrete pulse sequence.
The method uses two-day data, the extracted task related signals are decoded by an artificial neural network ANN, and the predicted result adopts a decision coefficient R 2 To measure the decoding effect. The similarity degree between the generated neural signal and the original neural signal is also an important index, and the coefficient R is also used for determining 2 To measure.
The experimental result is shown in fig. 3, and the experimental result shows that the decoding and generating capacity of the neural signal extracted by the method is obviously superior to that of the methods such as VAE, LFADS, pi-VAE, PSID and the like.
By utilizing the method, the relevant neural signals of the task can be extracted to the maximum extent, the research of subsequent neural mechanisms and the application of neural engineering are facilitated, and the decoding and generating performance is obviously superior to that of other methods.
The technical solutions and advantages of the present invention have been described in detail with reference to the above embodiments, it should be understood that the above embodiments are only specific examples of the present invention and should not be construed as limiting the present invention, and any modifications, additions and equivalents made within the scope of the principles of the present invention should be included in the scope of the present invention.

Claims (8)

1. An invasive brain-computer interface task related neural signal extraction method is characterized by comprising the following steps:
(1) Constructing an encoder h, converting an original neural signal x into a feature space, obtaining a neural representation z = h (x), and decoding z by using an affine function f to obtain a first predicted speed y of a target speed y 1 ＝f(z)；
(2) The construction decoder g generates a neural signal x from the neural representation z r ＝g(z)；
(3) Neural signal x to be generated r Re-sending to the encoder h and the affine function f to obtain a second predicted speed y 2 ＝f(h(x r ))；
(4) Constructing a priori generated neural network m, and learning a priori neural representation z by a target speed y p ＝m(y)；
(5) Calculating a loss function, and training an encoder, a decoder and a priori generation neural network at the same time;
(6) And inputting the original signal to be processed into the trained encoder and decoder to obtain the generated neural signal.
2. The invasive brain-computer interface task related neural signal extraction method of claim 1, wherein the encoder, the decoder and the a priori generated neural network all use a multi-layer perceptron.
3. The invasive brain-computer interface task related neural signal extraction method of claim 1, wherein in step (5), end-to-end back propagation is employed to train the encoder, decoder and a priori neural network.
4. The invasive brain-computer interface task related neural signal extraction method according to claim 1, wherein in the step (5), the specific process of calculating the loss function is:
(5-1) calculating the first predicted speed y 1 Loss function of target speed y
Figure FDA0003796959040000011
(5-2) computationally generating a neural signal x r Loss function from the original neural signal x
Figure FDA0003796959040000012
(5-3) calculating the second predicted speed y 2 Loss function of target speed y
Figure FDA0003796959040000013
(5-4) calculating the prior neural representation z p KL divergence loss function from neural representation z
Figure FDA0003796959040000021
(5-5) weighting and combining the four loss functions to obtain the final loss
Figure FDA0003796959040000022
5. The invasive brain-computer interface task related neural signal extraction method of claim 4, wherein the method comprisesIn step (5-1), the first predicted speed y is calculated 1 Loss function of target speed y
Figure FDA0003796959040000023
The mean square error loss function is adopted, and the formula is as follows:
Figure FDA0003796959040000024
where i represents the ith sample and N represents the total number of samples.
6. The invasive brain-computer interface task related neural signal extraction method of claim 4, wherein in the step (5-2), the neural signal x is generated by calculation r Loss function from original neural signal x
Figure FDA0003796959040000025
The Poisson negative log-likelihood loss function is adopted, and the formula is as follows:
Figure FDA0003796959040000026
7. the invasive brain-computer interface task related neural signal extraction method of claim 4, wherein in step (5-3), the second predicted speed y is calculated 2 Loss function of target speed y
Figure FDA0003796959040000027
The mean square error loss function is adopted, and the formula is as follows:
Figure FDA0003796959040000028
8. according to claim 4The invasive brain-computer interface task related neural signal extraction method is characterized in that in the step (5-4), the prior neural representation z is calculated p KL divergence loss function from neural representation z
Figure FDA0003796959040000029
The formula is as follows:
Figure FDA00037969590400000210
wherein q and p represent z and z, respectively p The distribution of (c).
CN202210973148.8A 2022-08-15 2022-08-15 Invasive brain-computer interface task related neural signal extraction method Pending CN115358264A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210973148.8A CN115358264A (en) 2022-08-15 2022-08-15 Invasive brain-computer interface task related neural signal extraction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210973148.8A CN115358264A (en) 2022-08-15 2022-08-15 Invasive brain-computer interface task related neural signal extraction method

Publications (1)

Publication Number Publication Date
CN115358264A true CN115358264A (en) 2022-11-18

Family

ID=84033314

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210973148.8A Pending CN115358264A (en) 2022-08-15 2022-08-15 Invasive brain-computer interface task related neural signal extraction method

Country Status (1)

Country Link
CN (1) CN115358264A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116382465A (en) * 2023-02-17 2023-07-04 中国科学院自动化研究所 Optical brain-computer interface system and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116382465A (en) * 2023-02-17 2023-07-04 中国科学院自动化研究所 Optical brain-computer interface system and method
CN116382465B (en) * 2023-02-17 2024-02-13 中国科学院自动化研究所 Optical brain-computer interface system and method

Similar Documents

Publication Publication Date Title
CN109389059B (en) P300 detection method based on CNN-LSTM network
CN108304917B (en) P300 signal detection method based on LSTM network
CN110969108B (en) Limb action recognition method based on autonomic motor imagery electroencephalogram
Yang et al. A hardware-efficient scalable spike sorting neural signal processor module for implantable high-channel-count brain machine interfaces
CN102940490B (en) Method for extracting motor imagery electroencephalogram signal feature based on non-linear dynamics
RU2415642C1 (en) Method of classification of electroencephalographic signals in interface brain-computer
CN101515200B (en) Target selecting method based on transient visual evoked electroencephalogram
Masse et al. Non-causal spike filtering improves decoding of movement intention for intracortical BCIs
CN110946576A (en) Visual evoked potential emotion recognition method based on width learning
CN113589937B (en) Invasive brain-computer interface decoding method based on twin network kernel regression
DE60018783T2 (en) METHOD FOR DETECTING HEAD MOVEMENT OR HEAD MUSCLE ACTIVITY AND FOR GENERATING AN OUTPUT SIGNAL
CN114533086B (en) Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation
CN113017645B (en) P300 signal detection method based on void convolutional neural network
CN115358264A (en) Invasive brain-computer interface task related neural signal extraction method
CN116058800A (en) Automatic sleep stage system based on deep neural network and brain-computer interface
CN105956547A (en) Decomposition method based on array surface electromyogram signal smoothing
CN105892676A (en) Human-machine interaction device, system and method of vascular intervention operation wire feeder
Soon et al. Speech recognition using facial sEMG
CN108509869A (en) Feature set based on OpenBCI optimizes on-line training method
CN111772669B (en) Elbow joint contraction muscle force estimation method based on adaptive long-time and short-time memory network
Ahmed et al. Effective hybrid method for the detection and rejection of electrooculogram (EOG) and power line noise artefacts from electroencephalogram (EEG) mixtures
CN111543983A (en) Electroencephalogram signal channel selection method based on neural network
CN114098768A (en) Cross-individual surface electromyographic signal gesture recognition method based on dynamic threshold and easy TL
Mo et al. A Bi-LSTM based network with attention mechanism for EEG visual classification
Chen et al. Detecting movement-related EEG change by wavelet decomposition-based neural networks trained with single thumb movement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination