CN113688952A - Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection - Google Patents

Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection Download PDF

Info

Publication number
CN113688952A
CN113688952A CN202111237839.3A CN202111237839A CN113688952A CN 113688952 A CN113688952 A CN 113688952A CN 202111237839 A CN202111237839 A CN 202111237839A CN 113688952 A CN113688952 A CN 113688952A
Authority
CN
China
Prior art keywords
decoding
data
decoded
electroencephalogram
channel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111237839.3A
Other languages
Chinese (zh)
Other versions
CN113688952B (en
Inventor
王佳星
侯增广
王卫群
梁旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN202111237839.3A priority Critical patent/CN113688952B/en
Publication of CN113688952A publication Critical patent/CN113688952A/en
Application granted granted Critical
Publication of CN113688952B publication Critical patent/CN113688952B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08Feature extraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Probability & Statistics with Applications (AREA)
  • Human Computer Interaction (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention provides a brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection, which comprises the following steps: acquiring electroencephalogram data to be decoded; inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded; the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data. The method is used for solving the defect of low decoding efficiency caused by the fact that multi-channel is used for decoding the electroencephalogram data in the prior art, realizes conversion and selection of the electroencephalogram data decoding channel to be decoded through the decoding model, and improves the decoding efficiency on the premise of not reducing or even improving the decoding precision.

Description

Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection
Technical Field
The invention relates to the technical field of brain-computer interface decoding, in particular to a brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection.
Background
The brain-computer interface is a direct connection path established between a human or animal brain (or a culture of brain cells) and an external device, namely, the brain intention is directly decoded from an electroencephalogram signal, and then the external device is controlled. Brain-computer interface rehabilitation training plays an important role in the functional rehabilitation of patients with nerve injury caused by stroke, spinal cord injury and the like, and the method is widely applied to the fields of nerve rehabilitation and motor assistance. Specifically, the rehabilitation is realized by collecting electroencephalogram information related to the active motor intention of the nerve injury patient, analyzing the electroencephalogram information, controlling the rehabilitation training equipment based on the analysis result (related to the active motor intention), and performing limb motor function training of the nerve injury patient. Compared with the traditional rehabilitation method and the robot-assisted rehabilitation method, the rehabilitation training based on the brain-computer interface technology is that the nerve injury patient actively participates in rehabilitation training control, and the generation of nerve plasticity is promoted by improving the participation degree of nerves, so that the rehabilitation training effect is effectively improved.
One of the main challenges facing brain-computer interface rehabilitation training is the low efficiency of electroencephalogram decoding. Specifically, the amount of input data is a key factor that affects the decoding efficiency of the brain-computer interface. However, to ensure decoding accuracy, a typical brain-computer interface system typically contains 32 to 128 brain electrical channels. Although the decoding accuracy can be improved to a certain extent by using a plurality of channels, the high brain electrical data volume and the high computing resource requirement caused by the use of the channels cause the decoding efficiency of the brain-computer interface to be low, and limit the deployment platform of the brain-computer interface system, namely the brain-computer interface system is difficult to deploy on some devices with insufficient computing power.
Disclosure of Invention
The invention provides a brain-computer interface decoding accelerating method and system based on self-adaptive electroencephalogram channel selection, which are used for solving the defect of low decoding efficiency caused by the fact that electroencephalogram data are decoded by multiple channels in the prior art, realizing the conversion and selection of electroencephalogram data decoding channels to be decoded through a decoding model, and improving the decoding efficiency on the premise of not reducing or even improving the decoding precision.
The invention provides a brain-computer interface decoding acceleration method based on self-adaptive electroencephalogram channel selection, which comprises the following steps:
acquiring electroencephalogram data to be decoded;
inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
According to the brain-computer interface decoding acceleration method based on the self-adaptive electroencephalogram channel selection, the electroencephalogram data to be decoded is input into a decoding model, and a decoding result of performing the intended decoding on the electroencephalogram data to be decoded is output, and the method specifically comprises the following steps:
compressing channel data based on input electroencephalogram data to be decoded to obtain the minimum channel data of the electroencephalogram data to be decoded;
extracting the strategy features related to the optimal channel number based on the minimum channel data to obtain the strategy features related to the optimal channel number;
based on the strategy characteristics, performing decision probability calculation on the electroencephalogram data to be decoded to obtain decision probability values of the electroencephalogram data to be decoded, wherein the number of the optimal channels is selected;
obtaining the number of the selected optimal channels based on the decision probability value;
and converting the electroencephalogram data to be decoded into a data format corresponding to the selected optimal channel number based on the selected optimal channel number to obtain a decoding result of performing intended decoding on the electroencephalogram data to be decoded in the data format corresponding to the optimal channel number.
The invention also provides a brain-computer interface decoding acceleration system based on self-adaptive electroencephalogram channel selection, which comprises the following components:
the acquisition module is used for acquiring electroencephalogram data to be decoded;
the execution module is used for inputting the electroencephalogram data to be decoded into a decoding model and outputting a decoding result of the intended decoding of the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
The invention also provides a method for training the decoding model, which specifically comprises the following steps:
inputting electroencephalogram data samples to be decoded;
performing channel data compression on the electroencephalogram data sample to be decoded through a plurality of candidate conversion matrixes constructed according to a preset channel conversion rule to form a plurality of optimal channel data candidates;
arranging the optimal channel data candidates in an ascending order of the number of channels to construct an optimal channel data candidate library;
obtaining a strategy characteristic related to an optimal channel data candidate with the least number of channels in the optimal channel data candidate library and the optimal channel number candidate, and then obtaining a decision probability value for selecting a plurality of optimal channel number candidates according to the strategy characteristic;
selecting the optimal channel number of the electroencephalogram data samples to be decoded in the optimal channel number candidates based on the decision probability value by utilizing an argmax function;
converting the electroencephalogram data sample to be decoded into a data format corresponding to the optimal channel number, and then performing intent decoding;
calculating the decoding loss of the electroencephalogram data sample to be decoded by using a pre-constructed loss function, and judging whether the decoding loss meets a preset loss standard;
if so, taking the candidate transformation matrix as an optimal transformation matrix, and obtaining a trained decoding model;
and if not, updating the candidate conversion matrix, and returning to perform channel data compression on the electroencephalogram data sample to be decoded through the updated candidate conversion matrix again.
According to the method for training the decoding model, the preset channel conversion rule specifically includes: a difference rule, a mean rule, and a selective activation rule; wherein,
the difference rule realizes channel data compression by calculating the difference of the electroencephalogram data collected by the corresponding electrodes of the brain areas on the two sides;
the average rule is used for realizing channel data compression by averaging the electroencephalogram data of adjacent channels;
the selective activation rule removes channels corresponding to the electroencephalogram data irrelevant to the tasks to be classified, and channel data compression is achieved.
According to the training method of the decoding model, the loss function is the weighted sum of a decoding precision loss function and a decoding efficiency loss function; wherein,
the decoding precision loss function is a cross entropy loss function which is constructed on a ShallowConvNet classification model based on the electroencephalogram data sample to be decoded and a real label;
the decoding efficiency loss function is a loss function constructed by averaging the floating point operation times of the electroencephalogram data samples to be decoded, which are input into the decoding model, on a ShallowConvNet classification model.
According to the method for training the decoding model, the updating the candidate transformation matrix specifically includes:
and approximating the argmax function based on the decision probability value by using Gumbel-Softmax, and updating the candidate conversion matrix according to an approximation result.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that when the processor executes the program, the steps of the method for accelerating the decoding of the brain-computer interface based on the adaptive electroencephalogram channel selection or the method for training the decoding model are realized.
The present invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method for training the decoding model according to any one of the methods for accelerating the decoding of the brain-computer interface based on adaptive electroencephalogram channel selection.
The invention also provides a computer program product, which comprises a computer program, and the computer program is used for realizing the steps of the decoding model training method of the brain-computer interface decoding acceleration method based on the self-adaptive electroencephalogram channel selection when being executed by a processor.
The brain-computer interface decoding accelerating method and system based on the self-adaptive electroencephalogram channel selection, provided by the invention, have the advantages that electroencephalogram data to be decoded are input into a decoding model, the electroencephalogram data to be decoded are converted into minimum channel data through the decoding model, the strategy characteristics are extracted, the optimal channel number is selected through the strategy characteristics to obtain the optimal channel data, the electroencephalogram data to be decoded are subjected to the intended decoding through the optimal channel data, the optimal channel data obtained through the decoding model is the decoding channel data selected from the minimum channel data obtained by compressing the channel data of the electroencephalogram data to be decoded, the defects of high electroencephalogram data volume and high computing resource requirements caused by multi-channel decoding are avoided, and the decoding efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a brain-computer interface decoding acceleration method based on adaptive electroencephalogram channel selection according to the present invention;
FIG. 2 is a schematic structural diagram of a brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection according to the present invention;
FIG. 3 is a flow chart of a decoding model training process provided by the present invention;
FIG. 4 is a schematic diagram of the construction of a decoding model provided by the present invention;
FIG. 5 is an exemplary diagram of an interpolation rule provided by the present invention;
FIG. 6 is an exemplary diagram of the averaging rule provided by the present invention;
FIG. 7 is an exemplary diagram of selective activation rules provided by the present invention;
FIG. 8 is a graph of brain decoding efficiency and decoding accuracy generated by an example of a decoding model according to the present invention;
FIG. 9 is a graph comparing the performance of 10 of data sources BCI competition IV dataset 2a (BCIC IV 2a) that were tried to decode using the decoding model provided by the present invention with decoding using the existing ShallowConvNet model;
fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The information obtained from high density brain electrical channels is generally highly redundant (cross-correlated) and the key brain electrical channels involved in identifying different types of brain intentions are also different. For example, for a brain-computer interface paradigm based on steady-state visual evoked, information collected by brain electrical channels distributed on occipital lobe is crucial for intent decoding, while for a motor imagery brain-computer interface paradigm, efficient and accurate decoding may be achieved only by CZ, C3, and C4 channels distributed on motor cortex. Therefore, the electroencephalogram data decoded by multiple channels is not completely necessary, and based on the method, the invention provides a brain-computer interface decoding acceleration method based on self-adaptive electroencephalogram channel selection and based on self-adaptive electroencephalogram channel selection from the perspective of compressing input data quantity. Meanwhile, for the convenience of understanding, the terms mentioned in the technical scheme of the invention are explained first, namely, the number of the channels refers to the number of the channels; the "optimal channel number" refers to the number of optimal channels, that is, the number of optimal channels selected by the policy features and required for decoding the electroencephalogram data to be decoded, such as 6 channels, 9 channels, and the like mentioned in the following embodiments of the present invention; "channel data" refers to the specific data content of a channel; the "optimal channel data" refers to the final channel data for the intended decoding, which is obtained by acting on the electroencephalogram data to be decoded, based on the optimal conversion matrix learned by the policy network.
The following describes, with reference to fig. 1, a brain-computer interface decoding acceleration method based on adaptive electroencephalogram channel selection according to the present invention, as shown in fig. 1, the method includes the following steps:
101. acquiring electroencephalogram data to be decoded;
102. inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
it should be noted that the decoding model is obtained by training based on the electroencephalogram data sample to be decoded and the decoding result corresponding to the sample, when the decoding model is used for channel selection, the optimal channel data is selected from the electroencephalogram data to be decoded by the strategy characteristics obtained by performing characteristic extraction on the minimum channel data converted from the electroencephalogram data to be decoded, therefore, the electroencephalogram data to be decoded can be compressed firstly after being input into the decoding model, thereby removing redundant or other irrelevant channel data, reducing the channel data volume, then, the optimal channel number is selected by utilizing the strategy characteristics extracted from the compressed data to obtain the optimal channel data, the decoding efficiency is improved, after different electroencephalogram data to be decoded are input into the decoding model, the required optimal channel data can be selected in a self-adaptive mode to improve the decoding efficiency.
In an embodiment of the present invention, the inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing an intended decoding on the electroencephalogram data to be decoded specifically includes:
compressing channel data based on input electroencephalogram data to be decoded to obtain the minimum channel data of the electroencephalogram data to be decoded;
extracting the strategy features related to the optimal channel number based on the minimum channel data to obtain the strategy features related to the optimal channel number;
based on the strategy characteristics, performing decision probability calculation on the electroencephalogram data to be decoded to obtain decision probability values of the electroencephalogram data to be decoded, wherein the number of the optimal channels is selected;
obtaining the number of the selected optimal channels based on the decision probability value;
and converting the electroencephalogram data to be decoded into a data format corresponding to the selected optimal channel number based on the selected optimal channel number to obtain a decoding result of performing intended decoding on the electroencephalogram data to be decoded in the data format corresponding to the optimal channel number.
It should be noted that, when the decoding model is applied to obtain the optimal channel data of the electroencephalogram data to be decoded, the decoding model compresses the channel data of the electroencephalogram data to be decoded into the minimum channel data first, namely, redundant and useless equal channel data are removed, the reduction of the initial input data volume is realized, then the strategic characteristic extraction related to the optimal channel number is carried out on the minimum channel data, and the decision probability calculation related to the optimal channel number in the electroencephalogram data to be decoded is carried out according to the extracted strategy characteristics, and finally, selecting the optimal channel number suitable for decoding the electroencephalogram data to be decoded from the electroencephalogram data to be decoded according to the decision probability value so as to obtain the optimal channel data for decoding the electroencephalogram data to be decoded. The strategy features are extracted through the minimum channel data, the strategy features of all channels in the electroencephalogram data to be decoded can be comprehensively covered by using less channel data, the calculated amount is obviously reduced in the extraction process of the strategy features, and the decoding rate is effectively improved.
The following describes a brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection according to the present invention with reference to fig. 2, and the brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection described below and the brain-computer interface decoding acceleration method based on adaptive electroencephalogram channel selection described above may be referred to in correspondence.
As shown in fig. 2, the brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection provided by the present invention includes an obtaining module 210 and an executing module 220; wherein,
the obtaining module 210 is configured to obtain electroencephalogram data to be decoded;
the execution module 220 is configured to input the electroencephalogram data to be decoded into a decoding model, so as to obtain an optimal channel for decoding the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
It should be noted that, the brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection of the present invention utilizes a decoding model to select channel data, and specifically, selects optimal channel data from electroencephalogram data to be decoded by performing feature extraction on minimum channel data into which electroencephalogram data to be decoded is converted, so that the electroencephalogram data to be decoded can be compressed first after being input into the decoding model, thereby removing redundant or other irrelevant channel data, reducing channel data amount, and then selecting optimal channel data from the compressed channel data, thereby improving decoding efficiency, that is, after different electroencephalogram data to be decoded are input into the decoding model, the optimal channel data can be adaptively selected to improve decoding efficiency.
In a preferred embodiment, the executing module 220 performs compression processing on channel data based on the input electroencephalogram data to be decoded to obtain the minimum channel data of the electroencephalogram data to be decoded;
extracting the strategy features related to the optimal channel number based on the minimum channel data to obtain the strategy features related to the optimal channel number;
based on the strategy characteristics, performing decision probability calculation on the electroencephalogram data to be decoded to obtain decision probability values of the electroencephalogram data to be decoded, wherein the number of the optimal channels is selected;
obtaining the number of the selected optimal channels based on the decision probability value;
and converting the electroencephalogram data to be decoded into a data format corresponding to the selected optimal channel number based on the selected optimal channel number to obtain a decoding result of performing intended decoding on the electroencephalogram data to be decoded in the data format corresponding to the optimal channel number.
The brain-computer interface decoding acceleration system based on the self-adaptive electroencephalogram channel selection is used for signing the brain-computer interface decoding acceleration method based on the self-adaptive electroencephalogram channel selection in each embodiment. The specific method and flow for realizing the corresponding functions of each module included in the brain-computer interface decoding acceleration system based on the adaptive electroencephalogram channel selection are detailed in the embodiment of the brain-computer interface decoding acceleration method based on the adaptive electroencephalogram channel selection, and are not described in detail herein.
The brain-computer interface decoding acceleration system based on the self-adaptive electroencephalogram channel selection is used for the brain-computer interface decoding acceleration method based on the self-adaptive electroencephalogram channel selection in each embodiment. Therefore, the description and definition of the brain-computer interface decoding acceleration method based on adaptive brain electrical channel selection in the foregoing embodiments can be used for understanding the execution modules in the embodiments of the present invention.
The following describes a decoding model training method according to the present invention with reference to fig. 3 to fig. 9, and it can be understood that the decoding model trained by the method can be applied to the aforementioned brain-computer interface decoding acceleration method based on adaptive electroencephalogram channel selection or brain-computer interface decoding acceleration system based on adaptive electroencephalogram channel selection; as shown in fig. 3, the method comprises the steps of:
301. inputting electroencephalogram data samples to be decoded;
302. performing channel data compression on the electroencephalogram data sample to be decoded through a plurality of candidate conversion matrixes constructed according to a preset channel conversion rule to form a plurality of optimal channel data candidates;
303. arranging the optimal channel data candidates in an ascending order of the number of channels to construct an optimal channel data candidate library;
304. obtaining a strategy characteristic related to an optimal channel data candidate with the least number of channels in the optimal channel data candidate library and the optimal channel number candidate, and then obtaining a decision probability value for selecting a plurality of optimal channel number candidates according to the strategy characteristic;
305. selecting the optimal channel number of the electroencephalogram data samples to be decoded in the optimal channel number candidates based on the decision probability value by utilizing an argmax function;
306. converting the electroencephalogram data sample to be decoded into a data format corresponding to the optimal channel number, and then performing intent decoding;
307. calculating the decoding loss of the electroencephalogram data sample to be decoded by using a pre-constructed loss function, and judging whether the decoding loss meets a preset loss standard; if yes, go to 308; if not, skipping to 309;
308. taking the candidate transformation matrix as an optimal transformation matrix and obtaining a trained decoding model;
309. and updating the candidate conversion matrix, returning to compress the channel data of the electroencephalogram data sample to be decoded through the updated candidate conversion matrix again to form a new optimal channel data candidate, and returning to 303.
It should be noted that, with reference to the schematic diagram of the decoding model shown in fig. 4, the decoding model is obtained by training through continuously selecting the optimal decoding channel data of the electroencephalogram data sample to be decoded, where the electroencephalogram data sample to be decoded is the electroencephalogram data to be decoded in a full channel.
Specifically, the original input electroencephalogram data sample to be decoded is defined as
Figure 654036DEST_PATH_IMAGE001
I.e. each trial runXIncludedNThe utility model is characterized by comprising an electroencephalogram channel,Ta time series of samples. Through the formula 1, the candidate conversion matrix compresses the channel data of the electroencephalogram data sample to be decoded to form a plurality of optimal channel data candidates, and the optimal channel data candidate library is defined by the ascending arrangement of the channel number as:
Figure 940660DEST_PATH_IMAGE002
formula 1;
wherein,
Figure 798895DEST_PATH_IMAGE003
Figure 381448DEST_PATH_IMAGE004
further, in order to reduce the amount of computation of the policy network layer, the channel data form of the optimal channel data candidate with the least number of channels in the optimal channel data candidate library
Figure 590713DEST_PATH_IMAGE005
Is input into a policy network layer for extracting policy features, and the calculation method is as follows:
Figure 415449DEST_PATH_IMAGE006
formula 2;
wherein,PMthe representative strategy feature extraction model is mainly composed of two layers, wherein the first layer is a time domain convolution layer, and the second layer is a space domain convolution layer.
The extracted strategic characteristics are output differently through the full connection layerBehavioral decision probability, i.e. the probability of each optimal channel number candidate in the optimal channel data candidate library being selectedP
Figure 393770DEST_PATH_IMAGE007
Formula 3;
in the training process of the conventional strategy network layer, discrete behaviors need to be acquired through argmax operationANamely, the channel sample label corresponding to the optimal channel number candidate:
Figure 114601DEST_PATH_IMAGE008
Figure 811162DEST_PATH_IMAGE009
Figure 672545DEST_PATH_IMAGE010
formula 4;
wherein,irepresenting an optimal channel number candidate index;
Figure 770951DEST_PATH_IMAGE011
representing a Gumbel distribution;
Figure 662684DEST_PATH_IMAGE012
representing a uniform distribution.
Selecting the optimal channel number through a strategy network layer, performing intended decoding on the electroencephalogram data sample to be decoded in a data format corresponding to the optimal channel number, and finally calculating the decoding loss of the electroencephalogram data sample to be decoded by using a decoding loss function so as to continuously train and optimize the decoding model through the loss function.
Further, at the beginning of the training of the decoding model, a plurality of candidate transformation matrices constructed according to the preset channel transformation rules are needed first, so as to facilitate the first channel data compression of the electroencephalogram data to be decoded, it can be understood that the candidate transformation matrices constructed for the first time are more in line with the electroencephalogram data decoding rules, and the optimal transformation matrix is more easily obtained, so the preset channel transformation rules for compressing the electroencephalogram data sample to be decoded in the present application are based on the prior knowledge in the field of electroencephalogram signal decoding, that is, in another embodiment of the present invention, the preset channel transformation rules specifically include: a difference rule, a mean rule, and a selective activation rule; wherein,
the difference rule realizes channel data compression by calculating the difference of the electroencephalogram data collected by the corresponding electrodes of the brain areas on the two sides;
the average rule is used for realizing channel data compression by averaging the electroencephalogram data of adjacent channels;
the selective activation rule removes channels corresponding to the electroencephalogram data irrelevant to the tasks to be classified, and channel data compression is achieved.
It should be noted that brain activities, especially brain signals excited by motor or motor imagery, generally exhibit the phenomena of enhancement of amplitude of brain signals in ipsilateral brain areas of limbs of the motor (motor imagery) and suppression of brain signals in contralateral brain areas. Therefore, the compression of channel data can be realized by calculating the difference of the electroencephalogram signals collected by the corresponding electrodes of the brain areas at the two sides, namely the difference rule; in consideration of the similarity and redundancy of adjacent electroencephalogram information, channel data compression can be realized on the basis of the average principle of adjacent channel signals, namely the mean rule; meanwhile, the electroencephalogram signals of some brain areas are possibly unrelated to the task to be classified, and the channel data can be directly removed at this time, so that the compression of the channel data is realized by selectively activating the rules.
Specifically, for example, as shown in fig. 5 to 7, an example of compressing channel data using the channel conversion rule of the present invention by taking 22 electroencephalograms as an example is illustrated, wherein fig. 5, 6, and 7 are diagrams of compressing 22 channels into 12 channels, 9 channels, and 6 channels using the difference rule, the mean rule, and the selective activation rule, respectively.
Further, constructing a candidate transformation matrix according to a preset channel transformation rule
Figure 846540DEST_PATH_IMAGE013
For example, in the above fig. 7, the selective activation rule is used to compress 22 channels into 6 channels, and the corresponding transformation matrix T6 is shown in equation 5:
Figure 13079DEST_PATH_IMAGE014
equation 5;
through the following formula 6, the electroencephalogram data to be decodedXCan be converted intoN i Data format of each input channel:
Figure 201878DEST_PATH_IMAGE015
equation 6;
wherein,X*the representation comprises*The electroencephalogram data to be decoded of each input channel.
It can be understood that, the construction of the loss function can ensure that the optimal number of channels obtained by the trained decoding model can meet the decoding requirement, and the most important requirements for decoding are decoding precision and decoding efficiency, however, when the requirements of the user on the decoding precision and the decoding efficiency are different, if the construction of the corresponding decoding model can be realized according to different requirements of the user, the brain-computer interface decoding acceleration method based on the adaptive electroencephalogram channel selection of the present invention can be more flexibly used, and therefore, in another embodiment of the present invention, the loss function is the weighted sum of the decoding precision loss function and the decoding efficiency loss function; wherein,
the decoding precision loss function is a cross entropy loss function which is constructed on a ShallowConvNet classification model based on the electroencephalogram data sample to be decoded and a real label;
the decoding efficiency loss function is a loss function constructed by averaging the floating point operation times of the electroencephalogram data samples to be decoded, which are input into the decoding model, on a ShallowConvNet classification model.
It should be noted that the ShallowConvNet classification model itself has an efficient electroencephalogram signal decoding speed, and therefore, the ShallowConvNet is adopted in the method of the present invention to decode electroencephalogram data.
More specifically, the decoding precision loss function and the decoding efficiency loss function are both constructed on a ShallowConvNet classification model, and the precision loss function is as follows:
Figure 795670DEST_PATH_IMAGE016
equation 7;
wherein,
Figure 201243DEST_PATH_IMAGE017
representing the original input electroencephalogram data sample and a behavior label output by the decision network layer;
Figure 905894DEST_PATH_IMAGE018
representing the ShallowConvNet classification model,
Figure 713313DEST_PATH_IMAGE019
representing model parameters.
For different inputs, the number of channels of the input data of the classification model is different based on the behavior decision of the policy network layer, so that different inputs are causedGFLOPsIn the present invention, for training dataGFLOPsAveraging is performed as a function of the final decoding loss.
Figure 478007DEST_PATH_IMAGE020
Equation 8;
the loss function that is finally constructed is a weighted sum of the decoding accuracy loss and the decoding efficiency loss:
Figure 892849DEST_PATH_IMAGE021
equation 9;
wherein,
Figure 401191DEST_PATH_IMAGE022
representing weight coefficients for trading off decoding efficiency against decoding accuracy, i.e.
Figure 797537DEST_PATH_IMAGE022
The larger the decoding efficiency.
It is to be understood that, since the argmax operation is not differentiable and cannot be trained using the conventional gradient back propagation algorithm, in another embodiment of the present invention, the updating the candidate transformation matrix specifically includes:
and approximating the argmax function based on the decision probability value by using Gumbel-Softmax, and updating the candidate conversion matrix according to an approximation result.
It should be noted that Gumbel-Softmax is used to approximate argmax, as shown in the following equation 10:
Figure 733132DEST_PATH_IMAGE023
Figure 113298DEST_PATH_IMAGE024
Figure 159751DEST_PATH_IMAGE025
equation 10.
Next, taking the data source BCI composition IV dataset 2a (BCIC IV 2a) as an example, and combining the 22-derived electroencephalogram provided above as an example, the example of channel compression using the channel conversion rule of the present invention uses 10 subjects to test the weight coefficients
Figure 443227DEST_PATH_IMAGE022
The performance of the decoding model was tested when different values were taken, and the results are shown in table 1:
TABLE 1 weighting coefficients
Figure 284145DEST_PATH_IMAGE022
Comparison table of decoding model performance when different values are taken
Figure 417186DEST_PATH_IMAGE026
The performance of the decoding model corresponding to Table 1 is shown in FIG. 8, where the asterisks are different
Figure 1751DEST_PATH_IMAGE022
And the solid line is a fitting curve of the decoding precision and the calculated amount.
Further, comparing the performance of decoding by using 10 decoding models provided by the invention in data source BCI competition IV dataset 2a (BCIC IV 2a) and the prior shallowconvet classification model with the optimal decoding speed, the result is shown in fig. 9, it can be seen that, compared with shallowconvet, the decoding model provided by the invention can reduce 35% of the calculation amount on the premise that the precision is not reduced (even slightly improved), so the brain-computer interface decoding acceleration method based on adaptive brain electrical channel selection has obvious advantages.
Fig. 10 illustrates a physical structure diagram of an electronic device, and as shown in fig. 10, the electronic device may include: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform an adaptive brain-computer interface decoding acceleration method based on adaptive brain electrical channel selection, the method comprising:
101. acquiring electroencephalogram data to be decoded;
102. inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention further provides a computer program product, where the computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute a method for accelerating decoding of a brain-computer interface based on adaptive brain electrical channel selection provided by the above methods, where the method includes:
101. acquiring electroencephalogram data to be decoded;
102. inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for accelerating decoding of a brain-computer interface based on adaptive brain electrical channel selection provided by the above methods, the method comprising:
101. acquiring electroencephalogram data to be decoded;
102. inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A brain-computer interface decoding acceleration method based on self-adaptive electroencephalogram channel selection is characterized by comprising the following steps:
acquiring electroencephalogram data to be decoded;
inputting the electroencephalogram data to be decoded into a decoding model, and outputting a decoding result of performing intended decoding on the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
2. The brain-computer interface decoding acceleration method based on adaptive electroencephalogram channel selection according to claim 1, wherein the inputting of the electroencephalogram data to be decoded into a decoding model and the outputting of a decoding result of the intended decoding of the electroencephalogram data to be decoded specifically comprises:
compressing channel data based on input electroencephalogram data to be decoded to obtain the minimum channel data of the electroencephalogram data to be decoded;
extracting the strategy features related to the optimal channel number based on the minimum channel data to obtain the strategy features related to the optimal channel number;
based on the strategy characteristics, performing decision probability calculation on the electroencephalogram data to be decoded to obtain decision probability values of the electroencephalogram data to be decoded, wherein the number of the optimal channels is selected;
obtaining the number of the selected optimal channels based on the decision probability value;
and converting the electroencephalogram data to be decoded into a data format corresponding to the selected optimal channel number based on the selected optimal channel number to obtain a decoding result of performing intended decoding on the electroencephalogram data to be decoded in the data format corresponding to the optimal channel number.
3. A brain-computer interface decoding acceleration system based on self-adaptive electroencephalogram channel selection is characterized by comprising:
the acquisition module is used for acquiring electroencephalogram data to be decoded;
the execution module is used for inputting the electroencephalogram data to be decoded into a decoding model and outputting a decoding result of the intended decoding of the electroencephalogram data to be decoded;
the decoding model is used for carrying out feature extraction on the minimum channel data compressed by the electroencephalogram data to be decoded to obtain strategy features, selecting the optimal channel number according to the strategy features to obtain the optimal channel data, and then carrying out intent decoding on the electroencephalogram data to be decoded through the optimal channel data.
4. A method for training a decoding model is characterized by specifically comprising the following steps:
inputting electroencephalogram data samples to be decoded;
performing channel data compression on the electroencephalogram data sample to be decoded through a plurality of candidate conversion matrixes constructed according to a preset channel conversion rule to form a plurality of optimal channel data candidates;
arranging the optimal channel data candidates in an ascending order of the number of channels to construct an optimal channel data candidate library;
obtaining a strategy characteristic related to an optimal channel data candidate with the least number of channels in the optimal channel data candidate library and the optimal channel number candidate, and then obtaining a decision probability value for selecting a plurality of optimal channel number candidates according to the strategy characteristic;
selecting the optimal channel number of the electroencephalogram data samples to be decoded in the optimal channel number candidates based on the decision probability value by utilizing an argmax function;
converting the electroencephalogram data sample to be decoded into a data format corresponding to the optimal channel number, and then performing intent decoding;
calculating the decoding loss of the electroencephalogram data sample to be decoded by using a pre-constructed loss function, and judging whether the decoding loss meets a preset loss standard;
if so, taking the candidate transformation matrix as an optimal transformation matrix, and obtaining a trained decoding model;
and if not, updating the candidate conversion matrix, and returning to perform channel data compression on the electroencephalogram data sample to be decoded through the updated candidate conversion matrix again.
5. The method for training a decoding model according to claim 4, wherein the preset channel conversion rule specifically includes: a difference rule, a mean rule, and a selective activation rule; wherein,
the difference rule realizes channel data compression by calculating the difference of the electroencephalogram data collected by the corresponding electrodes of the brain areas on the two sides;
the average rule is used for realizing channel data compression by averaging the electroencephalogram data of adjacent channels;
the selective activation rule removes channels corresponding to the electroencephalogram data irrelevant to the tasks to be classified, and channel data compression is achieved.
6. The method of claim 4, wherein the loss function is a weighted sum of a decoding precision loss function and a decoding efficiency loss function; wherein,
the decoding precision loss function is a cross entropy loss function which is constructed on a ShallowConvNet classification model based on the electroencephalogram data sample to be decoded and a real label;
the decoding efficiency loss function is a loss function constructed by averaging the floating point operation times of the electroencephalogram data samples to be decoded, which are input into the decoding model, on a ShallowConvNet classification model.
7. The method for training a decoding model according to claim 4, wherein the updating the candidate transformation matrix specifically includes:
and approximating the argmax function based on the decision probability value by using Gumbel-Softmax, and updating the candidate conversion matrix according to an approximation result.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for accelerating decoding of a brain-computer interface based on adaptive brain electrical channel selection according to claim 1 or 2 or the method for training a decoding model according to any one of claims 4 to 7.
9. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the method for accelerating decoding of a brain-computer interface based on adaptive brain electrical channel selection according to claim 1 or 2, or the method for training a decoding model according to any one of claims 4 to 7.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method for accelerating the decoding of a brain-computer interface based on adaptive brain electrical channel selection according to claim 1 or 2 or the method for training a decoding model according to any one of claims 4 to 7.
CN202111237839.3A 2021-10-25 2021-10-25 Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection Active CN113688952B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111237839.3A CN113688952B (en) 2021-10-25 2021-10-25 Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111237839.3A CN113688952B (en) 2021-10-25 2021-10-25 Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection

Publications (2)

Publication Number Publication Date
CN113688952A true CN113688952A (en) 2021-11-23
CN113688952B CN113688952B (en) 2022-01-25

Family

ID=78587789

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111237839.3A Active CN113688952B (en) 2021-10-25 2021-10-25 Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection

Country Status (1)

Country Link
CN (1) CN113688952B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015164300A1 (en) * 2014-04-21 2015-10-29 The General Hospital Corporation Biomedical system variably configured based on estimation of information content of input signals
CN108805953A (en) * 2018-06-15 2018-11-13 郑州布恩科技有限公司 A kind of simple image method for reconstructing based on LFP phase properties and k nearest neighbor algorithm
CN112001306A (en) * 2020-08-21 2020-11-27 西安交通大学 Electroencephalogram signal decoding method for generating neural network based on deep convolution countermeasure
CN112861625A (en) * 2021-01-05 2021-05-28 深圳技术大学 Method for determining stacking denoising autoencoder model
CN113486794A (en) * 2021-07-06 2021-10-08 燕山大学 Motor imagery electroencephalogram signal classification method based on hybrid model

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015164300A1 (en) * 2014-04-21 2015-10-29 The General Hospital Corporation Biomedical system variably configured based on estimation of information content of input signals
CN108805953A (en) * 2018-06-15 2018-11-13 郑州布恩科技有限公司 A kind of simple image method for reconstructing based on LFP phase properties and k nearest neighbor algorithm
CN112001306A (en) * 2020-08-21 2020-11-27 西安交通大学 Electroencephalogram signal decoding method for generating neural network based on deep convolution countermeasure
CN112861625A (en) * 2021-01-05 2021-05-28 深圳技术大学 Method for determining stacking denoising autoencoder model
CN113486794A (en) * 2021-07-06 2021-10-08 燕山大学 Motor imagery electroencephalogram signal classification method based on hybrid model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JIAXING WANG等: ""Toward Improving Engagement in Neural Rehabilitation: Attention Enhancement Based on Brain–Computer Interface and Audiovisual Feedback"", 《IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS》 *
任士鑫等: ""基于改进共空间模式与视觉反馈的闭环脑机接口"", 《机械工程学报》 *

Also Published As

Publication number Publication date
CN113688952B (en) 2022-01-25

Similar Documents

Publication Publication Date Title
CN109165556B (en) Identity recognition method based on GRNN
Liu et al. Energy efficient telemonitoring of physiological signals via compressed sensing: A fast algorithm and power consumption evaluation
CN108363493B (en) User characteristic model establishing method and system based on brain-computer interface and storage medium
CN108960299B (en) Method for identifying multi-class motor imagery electroencephalogram signals
CN114224360B (en) EEG signal processing method, equipment and storage medium based on improved EMD-ICA
CN104771163A (en) Electroencephalogram feature extraction method based on CSP and R-CSP algorithms
Wang et al. ECG signal denoising based on deep factor analysis
CN111008693A (en) Network model construction method, system and medium based on data compression
CN114533086A (en) Motor imagery electroencephalogram decoding method based on spatial domain characteristic time-frequency transformation
CN115919330A (en) EEG Emotional State Classification Method Based on Multi-level SE Attention and Graph Convolution
CN118211268A (en) Heterogeneous federal learning privacy protection method and system based on diffusion model
CN118350051B (en) Privacy improving method, system, device and medium based on federal learning
CN115969392A (en) Cross-period brainprint recognition method based on tensor frequency space attention domain adaptive network
CN117574059A (en) High-resolution brain-electrical-signal deep neural network compression method and brain-computer interface system
Ai et al. Flexible coding scheme for robotic arm control driven by motor imagery decoding
CN113688952B (en) Brain-computer interface decoding acceleration method and system based on self-adaptive electroencephalogram channel selection
CN116796223A (en) Electroencephalogram signal decoding method and device, electronic equipment and storage medium
CN117455757A (en) Image processing method, device, equipment and storage medium
CN114721514B (en) Geometric model selection intention distinguishing method and system based on electroencephalogram signals
CN111797991A (en) Deep network model compression system, method and device
CN111210009A (en) Information entropy-based multi-model adaptive deep neural network filter grafting method, device and system and storage medium
CN115982558B (en) Electroencephalogram movement intention classification model building method and application thereof
CN118114146B (en) Brain wave optimizing classifying and identifying system and method based on group intelligent algorithm
CN112149724B (en) Electroencephalogram data feature extraction method based on intra-class compactness
CN117731303A (en) Emotion recognition method and device based on electroencephalogram three-dimensional filtering characteristics and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant