WO2019144776A1 - 一种基于非对称脑电特征的脑-机接口系统编解码方法 - Google Patents

一种基于非对称脑电特征的脑-机接口系统编解码方法 Download PDF

Info

Publication number
WO2019144776A1
WO2019144776A1 PCT/CN2018/125927 CN2018125927W WO2019144776A1 WO 2019144776 A1 WO2019144776 A1 WO 2019144776A1 CN 2018125927 W CN2018125927 W CN 2018125927W WO 2019144776 A1 WO2019144776 A1 WO 2019144776A1
Authority
WO
WIPO (PCT)
Prior art keywords
brain
multiple access
division multiple
coding
asymmetric
Prior art date
Application number
PCT/CN2018/125927
Other languages
English (en)
French (fr)
Inventor
许敏鹏
明东
肖晓琳
Original Assignee
天津大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201810065388.1A external-priority patent/CN108469896A/zh
Priority claimed from CN201810065848.0A external-priority patent/CN108470182B/zh
Application filed by 天津大学 filed Critical 天津大学
Priority to US16/616,834 priority Critical patent/US11221672B2/en
Publication of WO2019144776A1 publication Critical patent/WO2019144776A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]

Definitions

  • the invention relates to the technical field of brain-computer interface systems, in particular to a brain-computer interface system codec method based on asymmetric brain electrical characteristics.
  • BCI Brain-Computer Interface
  • the traditional SSVEP-BCI stimulation area occupies the user's viewing angle range to 4°, calculated according to the retinal-cortical map.
  • the single stimuli activate the primary visual cortex area up to 1300 mm 2
  • the traditional P300-speller target stimulating activation area is 160 mm 2 , and the performance of both is greatly reduced due to the reduction of cortical activation area. Therefore, the traditional visual stimuli-induced BCI
  • the system has the problems of large stimulation area, high intensity and occupying many cognitive resources, which is not conducive to users' long-term multi-task operation, which limits the further development and application of BCI technology.
  • Asymmetric Visual Evoked Potential (aVEP) is a typical representative of asymmetric EEG features.
  • aVEP can be induced by unilateral stimuli that appear in the surrounding field of view.
  • P300-speller based on P300 features in Event-Related Potential (ERP) and SSVEP-BCI based on Steady-State Visual Evoked Potential (SSVEP) are induced by widely used visual stimuli.
  • EEP Event-Related Potential
  • SSVEP-BCI Steady-State Visual Evoked Potential
  • the brain-computer interface system, its related technology has been developed more stable and mature.
  • Traditional filtering methods usually filter out specific band frequencies, such as low-pass filtering, high-pass filtering, band-pass filtering, notching, etc. .
  • EEG signals have nonlinear and non-stationary characteristics.
  • the object of the present invention is to overcome the shortcomings of the above background art, and to provide a brain-computer interface coding and decoding method based on asymmetric electroencephalographic features, wherein the coding is based on the asymmetric characteristics of brain electrophysiological activity response to stimuli.
  • the coding is based on the asymmetric characteristics of brain electrophysiological activity response to stimuli.
  • the decoding is based on the feature classification method of discriminant mode spatial filtering and template matching principle. Based on the existing template matching CCA classification strategy, the DSP spatial filtering method is introduced, and different decoding templates are constructed according to the coding strategies of different stimulation paradigms. Improve the signal-to-noise ratio of the EEG signal to improve the classification and recognition efficiency of the signal characteristics.
  • Step one constructing an induced stimulation module in a brain-computer interface system
  • Step two the induced stimulation module sends a mixed coded visual stimulus to the tester according to the requirement to induce a specific brain electrical signal
  • Step 3 The acquisition module forms data information by amplifying and filtering the EEG signal
  • step four the decoding module converts the data information into an instruction set output.
  • the hybrid coding generated by the induced stimulation module includes at least any two combinations of coding in spatial division multiple access coding, code division multiple access coding, frequency division multiple access, and phase division multiple access.
  • the hybrid coding generated by the induced stimulation module includes spatial division multiple access coding, code division multiple access coding, frequency division multiple access, and phase division multiple access.
  • the present invention also provides the following technical solutions to be implemented:
  • Step one establishing an EEG signal data set including a training set X k and a test sample Y through a brain-computer interface system;
  • Step 2 performing frequency domain filtering and downsampling data processing on the sample Y of the EEG signal data concentration test
  • Step 3 based on the Fisher linear discriminant criterion, calculating the training set X k in the EEG signal module to obtain a spatial projection matrix W;
  • Step 4 Performing DSP spatial filtering on the EEG signal data set training set X k and the test sample Y according to the following formulas (5) and (6) And W T Y feature vectors;
  • Step five according to And the W T Y eigenvectors are constructed by CCA spatial filtering using the following formula (8) to construct projection matrices U k and V k ;
  • Step six by obtaining the feature vector W T Y, projection matrices U k and V k perform template matching to generate feature vector ⁇ k according to the following formula (9);
  • step 7 the eigenvectors ⁇ k are identified by different classifier models and output.
  • the present invention has the advantages of:
  • the coding design in the present invention is a novel brain-computer interface system coding method based on asymmetric brain electrical characteristics, which utilizes the spatial asymmetry of the brain response to the stimulus, thereby performing spatial, code, frequency and phase hybrid coding on the stimulation command.
  • the different parameters of the coding paradigm method can be adjusted to adapt to different user requirements and different application scenarios, and the paradigm is not limited to the visually induced system illustrated in the technical solution, and can also be applied to different brains such as auditory induction and somatosensory induction.
  • the method can effectively expand the number of instruction sets of the brain-computer interface system, which helps to further improve the brain-computer interface technology and promote the conversion of the technology to the application results.
  • the invention has been applied to a brain-computer interface system based on asymmetric brain electrical feature control, and designs and implements a BCI-speller offline and online brain-computer interface system experiment with an instruction set of 32; it is expected to obtain considerable social and economic benefits.
  • the decoding design of the present invention is a novel brain-computer interface system decoding method based on asymmetric EEG features, which is used for classification and identification of asymmetric EEG features, which can effectively improve the signal-to-noise ratio of the identification signal and improve the classification accuracy rate.
  • the experimental results of the above-mentioned 32-instruction brain-computer interface system show that the average classification accuracy rate of asymmetric EEG features is 17.88% higher than that of the traditional classification method after using the identification method of the present invention, which proves that the brain can be further improved by using this method- Machine interface technology to promote the conversion of this technology to application results; a wide range of applications.
  • FIG. 1 is a schematic structural view of a brain-computer interface system according to the present invention.
  • FIG. 2 is a schematic diagram showing the division of field of view and stimulation in the present invention.
  • FIG. 3 is a flow chart of a method for identifying an asymmetric brain electrical feature in the present invention.
  • Lateralization of the brain is an important field in cognitive neuroscience. Due to the lateral effects of brain structure and function, there are also asymmetry in the characteristics of EEG induced by different stimuli, such as asymmetric visual evoked potentials induced by visual stimulation. It reflects the asymmetry of neuronal activity between the two hemispheres of the brain.
  • the gist of the present invention is to propose an asymmetric EEG feature-inducing paradigm applied to a brain-computer interface system, and use the space-side dominant characteristic of the brain to stimulate the response to code division multiple access (CDMA).
  • CDMA code division multiple access
  • SDMA Space Division Multiple Access
  • FDMA Frequency Division Multiple Access
  • PDMA Phase Division Multiple Access
  • the number of instruction sets can be further studied to obtain a perfect brain-computer interface system, which is expected to obtain considerable social and economic benefits.
  • the invention can be used in the fields of rehabilitation of disabled persons, electronic entertainment, industrial control, etc., and further research can obtain a perfect brain-computer interface system, which is expected to obtain considerable social and economic benefits.
  • the following is the implementation process of the present invention:
  • Step one constructing an induced stimulation module in a brain-computer interface system
  • Step two the induced stimulation module sends a mixed coded visual stimulus to the tester according to the requirement to induce a specific brain electrical signal
  • Step 3 The acquisition module forms data information by amplifying and filtering the EEG signal
  • step four the decoding module converts the data information into an instruction set output.
  • the hybrid coding generated by the induced stimulation module includes at least any two combinations of coding in spatial division multiple access coding, code division multiple access coding, frequency division multiple access, and phase division multiple access.
  • the hybrid coding generated by the induced stimulation module includes spatial division multiple access coding, code division multiple access coding, frequency division multiple access, and phase division multiple access.
  • the space division multiple access (Fig. 1): using spatial information coding, up/down/left/right respectively represent four kinds of spatial information, spatial position increases, and encoded information increases.
  • the code division multiple access based on the space division multiple access strategy, digitally encodes up/down/left/right four information, which can be coded as “0”, “1”, “2” and “3”, and the spatial information is increased and encoded. Can continue to increment (Table 1 only shows the encoding of the left/right two spatial position information);
  • the frequency division multiple access different frequencies are induced at different times of one stimulation, for example, a single 100 ms stimulation continuously induces a background EEG frequency of 10 Hz; the phase division multiple access: the difference of the stimulation start time changes the phase of the stimulation .
  • the hybrid coding generated by the induced stimulation module includes spatial division multiple access coding, code division multiple access coding, frequency division multiple access, and phase division multiple access.
  • FIG. 1 is a schematic structural diagram of a brain-computer interface system of the paradigm coding application of the present invention.
  • the system includes a liquid crystal display stimulation interface, an EEG acquisition system such as an EEG electrode and an EEG amplifier, and a computer processing platform.
  • the system uses the paradigm of the present invention to stimulate the induction, and uses the EEG digital acquisition system produced by NeuroScan to collect the EEG signals, and the signals are amplified and filtered by the EEG amplifier, and then input into a computer for correlation calculation, and finally the EEG signals are decoded and converted into The BCI instruction is output. Stimulus presentation and data processing analysis are all based on the Matlab platform.
  • the user sits on a chair at a certain distance from the stimulation interface, and the line of sight is at the center of the stimulation interface, as shown by the "+" position in Figure 2, in different spatial positions (up, down, left, right, etc.)
  • the stimuli appearing at different locations induce different EEG signature signals, and the spatial position characteristics of these EEG signature signals are used to encode the spatial division multiple access coding strategy.
  • the visual stimulus-induced asymmetric VEP feature the stimulus appears in the user's left and right fields of view (Fig. 2), and avEP brain electrical characteristic signal is induced in the corresponding spatial position of the subject's brain, that is, the stimulus is on the left.
  • avEP brain electrical characteristic signal is induced in the corresponding spatial position of the subject's brain, that is, the stimulus is on the left.
  • the side appears it will induce more obvious VEP features in the right occipital region of the brain, while the stimulation on the right side will induce more obvious VEP features in the left occipital region of the brain
  • the shape and area of the stimulus can be adjusted according to different needs.
  • the stimulus shown in Figure 2 is a white dot with a diameter of 2 mm.
  • the code is defined as 0, when the stimulus is on the right.
  • its code is defined as 1.
  • the code division multiple access coding strategy is added according to the order of change of the stimulus at different times, thereby forming the space and code hybrid coding strategy proposed by the present invention. Taking the encoding of 4 characters "AB" as an example, "left/right" space division multiple access encoding, "0/1" code division multiple access two-digit encoding can be completed, as shown in Table 1.
  • Table 1 shows the space and code hybrid coding strategy
  • the difference in stimulation time will change the frequency characteristics induced by continuous stimulation (ie frequency division multiple access coding). For example, a single 100ms stimulation continuously induces a background EEG frequency of 10Hz, and a single 50ms stimulation continuously induces a background EEG frequency of 20Hz. And the difference in the start time of the stimulus changes the phase of the stimulus (ie, phase division multiple access coding), taking two frequencies (10/20 Hz), two phase (0°/90°) encoding as an example, for 8 characters.
  • the "A ⁇ H" coding is shown in Table 2.
  • the "0/1" code division multiple access one bit code can be completed.
  • Table 2 shows the mixed coding strategy for space, code, frequency and phase
  • a brain-computer interface system decoding method based on asymmetric brain electrical characteristics includes the following processes:
  • Step 2102 selecting from the EEG signal data set Test samples for frequency domain filtering and downsampling data processing;
  • step 4104 the training set X k and the test sample Y of the EEG signal data are obtained by performing DSP spatial filtering according to the following formula.
  • the DSP algorithm obtains a projection matrix W to make the two types of characteristic signals have greater separability after being projected.
  • the matrix W can be regarded as a spatial filter.
  • W spatial filtering can filter the common mode signals between the two types of signals, and the CCA algorithm can be used to calculate the DSP spatial filtering by constructing two projection matrices U k and V k .
  • the correlation between the CCA spatial filters U k and V k is calculated by the following formula (8).
  • Step five 105 according to And W T Y eigenvectors use the following formula for CCA spatial filtering to construct projection matrices U k and V k
  • ⁇ [ ⁇ ] represents a mathematical expectation.
  • Canonical correlation analysis is a statistical analysis method that measures the linear correlation between two multidimensional variables. Different from the use of straight lines to fit sample points in linear regression, CCA treats multidimensional feature vectors as a whole, and uses mathematical methods to find a set of optimal solutions, so that the two entities have the greatest associated weight, that is, the formula (8) The calculated value is the largest, which is the purpose of the typical correlation analysis.
  • Step six 106 by obtaining a feature vector W T Y, projection matrices U k and V k are template-matched to generate feature vector ⁇ k according to the following formula;
  • the template is constructed from the training set data. According to the different stimulation methods, the template construction can also be adjusted accordingly. Taking the classification of the asymmetric EEG characteristic signal as an example, the vector ⁇ k shown in the formula (9) Indicates the similarity between the training template and the test sample signal Y.
  • corr(*) denotes the Pearson correlation coefficient and dist(*) denotes the Euclidean distance. If ⁇ k1 , ⁇ k2 , ⁇ k3 , ⁇ k4 and ⁇ k5 are larger, it means Y and The greater the correlation between them. Connecting ⁇ k* eigenvector ⁇ k
  • step 7107 the feature vector ⁇ k is identified by using different classifier models and output.
  • ⁇ k different classifier models of different pattern recognition algorithms such as Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) are established.
  • LDA Linear Discriminant Analysis
  • SVM Support Vector Machine
  • the test sample Y is sent after preprocessing and feature extraction.
  • the classifier performs pattern recognition to predict the category of the sample and output the result, as shown in Figure 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Dermatology (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Neurosurgery (AREA)
  • Neurology (AREA)
  • Psychiatry (AREA)
  • Psychology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

一种基于非对称脑电特征的脑-机接口系统编解码方法，在脑-机接口系统中包括刺激诱发模块、脑电信号采集模块和由训练集X k和测试样本Y的构成的脑电数据模块以及脑电信号解码模块；所述诱发刺激模块根据需求向测试者发出混合编码视觉刺激以诱发对应的脑电信号；采集模块通过对脑电信号放大、滤波处理后形成数据信息，构成脑电数据模块；所述解码模块将脑电数据解码并转换成指令集输出；该方法根据大脑电生理活动对刺激响应的不对称特性，结合空分多址、码分多址、频分多址和相分多址等混合编码策略，有效扩展系统指令集个数，同时提出的解码方法可以提高脑电信号自身信噪比从而提高信号特征的分类识别效率。

Description

一种基于非对称脑电特征的脑-机接口系统编解码方法 技术领域
本发明涉及脑-机接口系统技术领域，具体涉及一种基于非对称脑电特征的脑-机接口系统编解码方法。
背景技术
脑-机接口(Brain-Computer Interface,BCI)是一个将中枢神经系统活动直接转化为人工输出的系统，它能够替代、修复、增强、补充或者改善中枢神经系统的正常输出，从而改善中枢神经系统与内外环境之间的交互作用。通过采集和分析不同刺激下受试者的脑电信号，再使用一定的工程技术手段建立起人脑与计算机或其它电子设备之间的交流和控制通道。BCI技术实现了一种全新的信息交互与控制方式，可以为残疾人尤其是那些基本肢体运动功能受损但思维正常的患者提供一种与外界进行信息交流和控制的途径，使他们无需进行语言或肢体动作即可同外界交流或操纵外界设备。为此，BCI技术也越来越受到重视。
在脑-机接口系统的研究中，利用视觉刺激诱发的范式应用较为普遍，其中基于事件相关电位(Event-Related Potential,ERP)中P300特征的P300-speller和基于稳态视觉诱发电位(Steady-State Visual Evoked Potential,SSVEP)的SSVEP-BCI应用最为广泛，其相关技术已经发展得较为稳定和成熟。由于脑电信号具有非线性及非平稳性的特征，从繁杂的背景脑电中提取微弱的脑电信号特征是BCI系统的重要技术之一，为了克服脑电背景噪声，传统BCI范式通常采用强刺激诱发，调用大量相关神经元激活以诱发幅值更大、特征更明显的脑电特征，如传统SSVEP-BCI的刺激面积占据用户视角范围至4°以外，依视网膜-皮层图计算，该范式下单个刺激激活初级视觉皮层区域面积达1300mm 2，传统P300-speller的靶刺激激活面积达160mm 2，且二者的性能会因皮层激活面积的减少而大大降低，故传统基于视觉刺激诱发的BCI系统存在刺激面积大、强度高、占用认知资源多的问题，不利于用户进行长时间多任务的操作，限制了BCI技术的进一步发展和应用。非对称视觉诱发电位(asymmetric Visual Evoked Potential,aVEP)作为非对称脑电特征的典型代表，属于极微弱偏侧化视觉诱发电位，幅值通常低于1μ V。根据大脑视觉刺激响应的空间对侧优势特性，aVEP可由出现在周围视野的单侧刺激诱发产生。
此外，基于事件相关电位(Event-Related Potential,ERP)中P300特征的P300-speller和基于稳态视觉诱发电位(Steady-State Visual Evoked Potential,SSVEP)的SSVEP-BCI是应用较广泛的视觉刺激诱发的脑-机接口系统，其相关技术已经发展得较为稳定和成熟。对于实时数据采集系统，为了消除干扰信号，通常需要对采集到的数据进行数字滤波，传统滤波方法通常将特定波段频率滤除，如：低通滤波、高通滤波、带通滤波、陷波等等。脑电信号具有非线性及非平稳性的特征，在脑-机接口系统的研究中，如何对采集到的脑电信号进行处理分析，从繁杂的背景脑电中提取微弱的脑电信号特征并对不同特征进行分类识别是决定BCI系统性能的关键性因素，由于脑电信号存在频率特性，因此滤波手段也常用于脑电信号的处理分析中，通常滤波频段会根据不同脑电特征进行调整。滤波后，对传统脑电信号进行分类识别方法有线性判别分析(Linear Discriminant Analysis,LDA)，共空间模式(Common Spatial Pattern,CSP)，支持向量机(Support Vector Machine,SVM)，典型相关分析(Canonical Correlation Analysis,CCA)等方法。这些方法均包含空间滤波的思想，即在高维空间选择一个或几个分类平面，将其向量作为空间滤波器对信号进行空间滤波，目的是将高维信号降至低维，便于对其进行分类。典型相关分析算法目前被普遍应用于SSVEP-BCI系统中，且有研究对该算法做了进一步改进，即在脑电信息处理过程中应用模板匹配原则引入了受试者自身信号，提升了系统的识别正确率和信息传输速率，为将BCI技术进一步向应用成果转化奠定了有力基础。
发明内容
本发明的目的在于克服上述背景技术存在的缺陷，提供一种基于非对称脑电特征的脑-机接口系统编解码方法，该方法中的编码是根据大脑电生理活动对刺激响应的不对称特性，结合空分多址、码分多址、频分多址和相分多址等编码策略，有效扩展系统指令集个数，为脑-机接口系统的发展提供了新方向，同时，该解码方法是结合判别模式空间滤波及模板匹配原则的特征分类方法，在现有的模板匹配CCA分类策略的基础之上，引入DSP空间滤波方法，并根据不同刺激范式的编码策略构建不同解码模板，以提高脑电信号自身信噪比从而提高信号特征的分类识别效率。
本发明的技术方案:
一种基于非对称脑电特征的脑-机接口系统编码方法，包括如下步骤：
步骤一，在脑-机接口系统中构建诱发刺激模块；
步骤二，所述诱发刺激模块根据需求向测试者发出混合编码视觉刺激以诱发特定的脑电信号；
步骤三，采集模块通过对脑电信号放大、滤波处理后形成数据信息；
步骤四，所述解码模块将数据信息转换成指令集输出。
所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址中至少任意两种编码组合。
所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址。
为了解决技术问题，本发明还提供如下技术方案予以实施：
一种基于非对称脑电特征的脑-机接口系统解码方法，包括如下步骤：
步骤一，通过脑-机接口系统建立包括训练集X k和测试样本Y的脑电信号数据集；
步骤二，对脑电信号数据集中测试样本Y进行频域滤波和降采样数据处理；
步骤三，基于Fisher线性判别准则，对脑电信号模块中训练集X k进行计算得到空间投影矩阵W；
步骤四，对脑电信号数据集中训练集X k和测试样本Y按照如下公式(5)和(6)进行DSP空间滤波获得
Figure PCTCN2018125927-appb-000001
和W TY特征向量；
Figure PCTCN2018125927-appb-000002
Figure PCTCN2018125927-appb-000003
步骤五，根据
Figure PCTCN2018125927-appb-000004
和W TY特征向量采用如下公式(8)进行CCA空间滤波构建投影矩阵U k和V k；
Figure PCTCN2018125927-appb-000005
步骤六，通过获得特征向量
Figure PCTCN2018125927-appb-000006
W TY、投影矩阵U k和V k按照如下公式(9)进行模板匹配生成特征向量ρ k；
Figure PCTCN2018125927-appb-000007
步骤七，采用不同分类器模型对特征向量ρ k进行识别后输出。
所述训练集
Figure PCTCN2018125927-appb-000008
k表示两类特征，即k＝1，2；所述测试样本
Figure PCTCN2018125927-appb-000009
其中N c表示采集脑电的通道数，N t表示截取信号长度，N s表示训练集样本个数。
与现有技术相比，本发明具有的优点：
1、本发明中编码设计是基于非对称脑电特征的新型脑-机接口系统编码方法，利用了大脑对刺激响应的空间不对称性从而对刺激指令进行空、码、频、相混合编码，所采用编码范式方法的不同参数可调以适应不同用户需求及不同应用场景，且该范式不局限于技术方案中举例说明的视觉诱发系统，还可以应用于听觉诱发、体感诱发等不同的脑-机接口系统；同时，利用该方法能有效扩展脑-机接口系统的指令集个数，有助于进一步完善脑-机接口技术，促进该技术向应用成果转化。本发明已应用于基于非对称脑电特征控制的脑-机接口系统，设计实施了指令集为32的BCI-speller离线和在线脑-机接口系统实验；有望获得可观的社会效益和经济效益。
2、本发明中解码设计是基于非对称脑电特征的新型脑-机接口系统解码方法，用于非对称脑电特征的分类识别，可以有效提升识别信号的信噪比并提升分类正确率。上述32指令集的脑-机接口系统实验结果表明，利用本发明的识别方法后，非对称脑电特征的平均分类正确率较传统分类方法提高了17.88％，证明利用该方法能进一步完善脑-机接口技术，促进该技术向应用成果转化；应用范围广泛。
附图说明
图1为本发明中脑-机接口系统结构示意图。
图2为本发明中视野划分及刺激呈现示意图。
图3为本发明中针对非对称脑电特征的识别方法流程图。
具体实施方式
下面通过具体实施例和附图对本发明作进一步的说明。本发明的实施例是为了更好地使本领域的技术人员更好地理解本发明，并不对本发明作任何的限制。
大脑偏侧化是认知神经科学中的一个重要领域，由于大脑结构和功能的偏侧化效应，不同刺激诱发的脑电特征也存在不对称性，如视觉刺激诱发的非对称视觉诱发电位，其反映了大脑两半球之间神经元活动的不对称性。
本发明的主旨是提出一种应用于脑-机接口系统的非对称脑电特征诱发范式，利用大脑对刺激响应的空间对侧占优特性对指令进行码分多址(Code Division Multiple Access，CDMA)与空分多址(Space Division Multiple Access，SDMA)混合编码，同时引入频分多址(Frequency Division Multiple Access，FDMA)和相分多址(Phase Division Multiple Access，PDMA)编码策略，有效扩展系统指令集个数，进一步研究可以得到完善的脑-机接口系统，有望获得可观的社会效益和经济效益。该项发明可以用于残疾人康复、电子娱乐、工业控制等领域，进一步研究可以得到完善的脑-机接口系统，有望获得可观的社会效益和经济效益。如下是本发明实现过程：
一种基于非对称脑电特征的脑-机接口系统编码方法，包括如下步骤：
步骤一，在脑-机接口系统中构建诱发刺激模块；
步骤二，所述诱发刺激模块根据需求向测试者发出混合编码视觉刺激以诱发特定的脑电信号；
步骤三，采集模块通过对脑电信号放大、滤波处理后形成数据信息；
步骤四，所述解码模块将数据信息转换成指令集输出。
所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址中至少任意两种编码组合。
所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址。
所述空分多址(图1)：利用空间信息编码，上/下/左/右分别代表四种空间信息，空间位置增加，编码的信息增加
所述码分多址：基于空分多址策略，将上/下/左/右四种信息进行数字编码，可编码为“0”“1”“2”“3”，空间信息增加，编码可继续递增(表1只表示了左/右两种空间位置信息的编码)；
所述频分多址：一次刺激的不同时间会诱发不同频率，如单次100ms的刺激持续诱发10Hz的背景脑电频率；所述相分多址：刺激起始时刻的不同会改变刺激的相位。
如表2所示，所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址。所述混合编码组合策略包括：所述空分多址编为“左/右”码；所述码分多址编码为“0/1”码分多址两位编码；所述频分多址编码为“12Hz(单次刺激时间83.33ms的)、13(单次刺激时间76.92ms的)、14(单次刺激时间71.43ms的)、15(单次刺激时间66.67ms的)、16(单次刺激时间62.5ms的)”五种频率和所述相分多址编码“0°/90°”编码，即可实现2x2x5x2＝40的混合编码策略。
以诱发视觉非对称脑电特征为例，图1为本发明范式编码应用的脑-机接口系统结构示意图。该系统包括液晶显示器刺激界面、脑电电极和脑电放大器等脑电采集系统以及计算机处理平台等部分。该系统应用本发明范式进行刺激诱发，采用NeuroScan公司生产的脑电数字采集系统采集脑电信号，将信号经过脑电放大器放大、滤波后输入计算机进行相关计算，最终将脑电信号解码后转化为BCI指令进行输出。刺激呈现及数据处理分析均基于Matlab平台完成。
使用该系统时，用户坐在距刺激界面一定距离的椅子上，视线注视在刺激界面的中心位置，如图2中“+”位置所示，在不同空间位置(上、下、左、右等不同位置)出现的刺激会诱发不同的脑电特征信号，利用这些脑电特征信号的空间位置特性进行编码即形成空分多址编码策略。以视觉刺激诱发非对称VEP特征为例，刺激在用户的左、右视野内出现(如图2)，并在该受试者大脑的对应空间位置诱发出aVEP脑电特征信号，即刺激在左侧出现时会在大脑的右侧枕区诱发出更加明显的VEP特征，而刺激在右侧出现则会在大脑的左侧枕区诱发出更加明显的VEP特征。
刺激的形状、面积等参数可根据不同需求进行调整，以图2所示刺激为直径2mm的白色圆点为例，当刺激在左侧视野内产生时定义其编码为0，当刺激在右侧视野内产生时定义其编码为1，利用“0/1”二进制编码思想，根据刺激在不同时刻的变化顺序加入码分多址编码策略，进而形成本发明提出的空、码混合编码策略。以编码4个字符“AB”为例， “左/右”空分多址编码，“0/1”码分多址两位编码即可完成，如表1所示。
表1空、码混合编码策略示意
Figure PCTCN2018125927-appb-000010
一次刺激时间的不同会改变持续刺激诱发的频率特性(即频分多址编码)，如单次100ms的刺激持续诱发10Hz的背景脑电频率，单次50ms的刺激持续诱发20Hz的背景脑电频率，且刺激起始时刻的不同会改变刺激的相位(即相分多址编码)，以两种频率(10/20Hz)，两种相位(0°/90°)编码为例，对8个字符“A～H”编码示意如表2所示，“0/1”码分多址一位编码即可完成。
表2空、码、频、相混合编码策略示意
Figure PCTCN2018125927-appb-000011
对比表1和表2可以看出，加入频、相编码策略后可有效扩展指令集，通过增加码分多址 编码位数亦可扩展编码的指令集个数，且一次刺激持续时间及其占空比、刺激序列重复轮次、每轮序列间隔时间等参数均可结合实际情况进行调节。再对采集得到的脑电信号进行解码，从而定位出用户所注视的字符。以设计实现40指令集的刺激范式为例，通过“左/右”空分多址编码，“0/1”码分多址两位编码，“12Hz(单次刺激时间83.33ms的)、13(单次刺激时间76.92ms的)、14(单次刺激时间71.43ms的)、15(单次刺激时间66.67ms的)、16(单次刺激时间62.5ms的)”五种频率的频分多址编码。表2列举了，两种空间位置，两位数字码，两个频率和两个相位的编码示意，利用该种方式可编码40个字符，若增加空间位置，或者频率，或者相位等任意一个参数，都可增加编写字符的个数。
本发明中一种基于非对称脑电特征的脑-机接口系统解码方法，包括如下过程：
图1也可以表示为本发明算法应用的包含32指令集的脑-机接口系统结构示意图。该系统包括液晶显示器刺激界面、脑电电极和脑电放大器等脑电采集系统以及计算机处理平台等部分。该系统应用视觉刺激范式编码的诱发两类非对称脑电特征，采用NeuroScan公司生产的脑电数字采集系统采集脑电信号，将信号经过脑电放大器放大、滤波后输入计算机，应用本发明算法对两类脑电特征进行分类，最终将脑电信号解码后转化为BCI指令进行输出。刺激呈现及数据处理分析均基于Matlab平台完成。
计算两类非对称脑电特征信号的信噪比(signal-to-noise rate,SNR)分别为-17.98dB和-14.90dB，SNR定义为信号能量与噪声能量之比，其计算公式(11)为：
Figure PCTCN2018125927-appb-000012
其中AMP i表示第i个试次时间窗内的信号的平均幅值，N表示试次数量。
本发明算法应用的BCI系统对12名受试者进行了测试。实验结果表明，在应用本发明算法后，12名被试的平均分类正确率提升17.88％，有显著性提升(配对T检验结果为：t 11＝-8.91，p＜0.01)，且两类特征信号的信噪比经DSP空间滤波后分别提升至-9.71dB和-8.68dB。
如图3所示，本发明提供一种基于非对称脑电特征脑-机接口进行识别解码的方法；步骤一101，通过脑-机接口系统建立包括训练集X k和测试样本Y的脑电信数据集：
假定
Figure PCTCN2018125927-appb-000013
为训练集，k表示两类特征，即k＝1，2，
Figure PCTCN2018125927-appb-000014
为测试样本， 其中N c表示采集脑电的通道数，N t表示截取信号长度，N s表示训练集样本个数。训练集和测试集都在时间尺度上进行了零均值处理，即每一个时间点的数值s t都减去时间窗[t 1，t 2]内的时间平均值
Figure PCTCN2018125927-appb-000015
如公式(1)所示:
Figure PCTCN2018125927-appb-000016
对训练集所有样本求均值得到类别k的模板信号，由
Figure PCTCN2018125927-appb-000017
表示。两类模板之间的协方差矩阵
Figure PCTCN2018125927-appb-000018
表示为：
Figure PCTCN2018125927-appb-000019
两类信号X 1和X 2的方差分别表示为：
Figure PCTCN2018125927-appb-000020
Figure PCTCN2018125927-appb-000021
步骤二102，从脑电信号数据集中选
Figure PCTCN2018125927-appb-000022
测试样本进行频域滤波和降采样数据处理；
步骤三103，基于Fisher线性判别准则，对脑电信号模块中训练集X k进行计算得到空间投影矩阵W；
步骤四104，对脑电信号数据集中训练集X k和测试样本Y按照如下公式进行DSP空间滤波获得
Figure PCTCN2018125927-appb-000023
特征向量
基于Fisher线性判别准则，DSP算法求得一个投影矩阵W使两类特征信号投射之后具有更大的可分性，该矩阵W可被当做空间滤波器，其求解方法为：
Figure PCTCN2018125927-appb-000024
S B＝Σ 11+Σ 22-Σ 22-Σ 21     (6)
S W＝σ 1 2+σ 2 2    (7)
其中λ i是矩阵W中第i列的特征向量，N W表示被挑选出的空间滤波器个数。经W空间滤波 可以滤除两类信号之间的共模信号，而应用CCA算法可以通过构造两个投影矩阵U k和V k来计算DSP空间滤波后
Figure PCTCN2018125927-appb-000025
之间的相关性，CCA空间滤波器U k和V k由下述公式(8)计算得到。
步骤五105，根据
Figure PCTCN2018125927-appb-000026
和W TY特征向量采用如下公式进行CCA空间滤波构建投影矩阵U k和V k
Figure PCTCN2018125927-appb-000027
其中，ε[·]表示数学期望。典型相关分析是衡量两个多维变量之间的线性相关关系的统计分析方法。区别于在线性回归中利用直线来拟合样本点，CCA是将多维特征向量都看作一个整体，利用数学方法寻求一组最优解，使得两个整体之间有最大关联的权重，即令公式(8)计算得到的数值最大，这就是典型相关分析的目的。
步骤六106，通过获得特征向量
Figure PCTCN2018125927-appb-000028
W TY、投影矩阵U k和V k按照如下公式进行模板匹配生成特征向量ρ k；
在模板匹配过程中，由训练集数据构建模板，根据刺激方式的不同，模板构建也可进行相应调整，以对非对称脑电特征信号的分类为例，公式(9)所示的向量ρ k表示训练模板和测试样本信号Y之间的相似性。
Figure PCTCN2018125927-appb-000029
其中corr(*)表示皮尔森相关系数，dist(*)表示欧几里德距离。若ρ k1,ρ k2,ρ k3,ρ k4和ρ k5越大，则表示Y和
Figure PCTCN2018125927-appb-000030
之间的相关性越大。连接ρ k*特征向量ρ k
步骤七107，采用不同分类器模型对特征向量ρ k进行识别后输出。
根据特征向量ρ k建立线性判别分析(Linear Discriminant Analysis,LDA)、支持向量机(Support Vector Machine,SVM)等不同模式识别算法的不同分类器模型，测试样本Y经预处理和特征提取后送入分类器进行模式识别，进而预测该样本的类别并输出结果，如图1所示。
应当理解的是，这里所讨论的实施方案及实例只是为了说明，对本领域技术人员来说，可以加以改进或变换，而所有这些改进和变换都应属于本发明所附权利要求的保护范围。

Claims (5)

  1. 一种基于非对称脑电特征的脑-机接口系统编码方法，其特征在于，包括如下步骤：
    步骤一，在脑-机接口系统中构建诱发刺激模块；
    步骤二，所述诱发刺激模块根据需求向测试者发出混合编码视觉刺激以诱发对应的脑电信号；
    步骤三，采集模块通过对脑电信号放大、滤波处理后形成数据信息；
    步骤四，所述解码模块将数据信息转换成指令集输出。
  2. 根据权利要求1所述的一种基于非对称脑电特征的脑-机接口系统编码方法，其特征在于，所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址中至少任意两种编码组合。
  3. 根据权利要求2所述的一种基于非对称脑电特征的脑-机接口系统编码方法，其特征在于，所述诱发刺激模块生成的混合编码包括空分多址编码、码分多址编码、频分多址和相分多址。
  4. 一种基于非对称脑电特征的脑-机接口系统解码方法，其特征在于，包括如下步骤：
    步骤一，通过脑-机接口系统建立包括训练集X k和测试样本Y的脑电信号数据集
    步骤二，对脑电信号数据集中测试样本Y进行频域滤波和降采样数据处理；
    步骤三，基于Fisher线性判别准则，对脑电信号模块中训练集X k进行计算得到空间投影矩阵W；
    步骤四，对脑电信号数据集中训练集X k和测试样本Y按照如下公式(5)和(6)进行DSP空间滤波获得
    Figure PCTCN2018125927-appb-100001
    和W TY特征向量；
    Figure PCTCN2018125927-appb-100002
    Figure PCTCN2018125927-appb-100003
    步骤五，根据
    Figure PCTCN2018125927-appb-100004
    和W TY特征向量采用如下公式(8)进行CCA空间滤波构建投影矩阵U k和V k；
    Figure PCTCN2018125927-appb-100005
    步骤六，通过获得特征向量
    Figure PCTCN2018125927-appb-100006
    W TY、投影矩阵U k和V k按照如下公式(9)进行模板匹配生成特征向量ρ k；
    Figure PCTCN2018125927-appb-100007
    步骤七，采用不同分类器模型对特征向量ρ k进行识别后输出。
  5. 根据权利要求4所述的一种基于非对称脑电特征的脑-机接口系统解码方法，其特征在于：所述训练集
    Figure PCTCN2018125927-appb-100008
    k表示两类特征，即k＝1，2；所述测试样本
    Figure PCTCN2018125927-appb-100009
    其中N c表示采集脑电的通道数，N t表示截取信号长度，N s表示训练集样本个数。
PCT/CN2018/125927 2018-01-23 2018-12-30 一种基于非对称脑电特征的脑-机接口系统编解码方法 WO2019144776A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/616,834 US11221672B2 (en) 2018-01-23 2018-12-30 Asymmetric EEG-based coding and decoding method for brain-computer interfaces

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201810065388.1A CN108469896A (zh) 2018-01-23 2018-01-23 一种基于非对称脑电特征的脑-机接口混合编码范式方法
CN201810065848.0A CN108470182B (zh) 2018-01-23 2018-01-23 一种用于非对称脑电特征增强与识别的脑-机接口方法
CN201810065388.1 2018-01-23
CN201810065848.0 2018-01-23

Publications (1)

Publication Number Publication Date
WO2019144776A1 true WO2019144776A1 (zh) 2019-08-01

Family

ID=67394548

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/125927 WO2019144776A1 (zh) 2018-01-23 2018-12-30 一种基于非对称脑电特征的脑-机接口系统编解码方法

Country Status (2)

Country Link
US (1) US11221672B2 (zh)
WO (1) WO2019144776A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112861629A (zh) * 2021-01-07 2021-05-28 天津大学 一种多窗口判别典型模式匹配方法及脑-机接口应用
CN113143291A (zh) * 2021-05-11 2021-07-23 燕山大学 一种快速序列视觉呈现下的脑电特征提取方法
CN113672082A (zh) * 2021-07-12 2021-11-19 华南理工大学 一种多通道高共模抑制比低功耗脑电采集的脑机接口系统
CN113705732A (zh) * 2021-09-26 2021-11-26 华东理工大学 基于通用模型的减少p300训练时间的方法及装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113805694A (zh) * 2021-08-26 2021-12-17 上海大学 一种基于脑机接口与计算机视觉的辅助抓取系统及方法
CN114145756B (zh) * 2021-12-15 2023-05-05 电子科技大学中山学院 协作机器人控制方法、装置及计算机可读存储介质
CN115836864B (zh) * 2022-09-30 2024-08-09 中国人民解放军国防科技大学 一种基于多脑bci的目标增强判读方法和装置
CN116738215A (zh) * 2023-08-11 2023-09-12 之江实验室 一种基于稳态视觉诱发电位的脑电身份识别方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1778272A (zh) * 2004-11-22 2006-05-31 中国科学院心理研究所 便携式事件相关电位仪
CN103150017A (zh) * 2013-03-05 2013-06-12 天津大学 基于空间、时间和频率联合编码的脑-机接口通讯方法
CN105824418A (zh) * 2016-03-17 2016-08-03 天津大学 一种基于非对称视觉诱发电位的脑-机接口通讯系统
CN108469896A (zh) * 2018-01-23 2018-08-31 天津大学 一种基于非对称脑电特征的脑-机接口混合编码范式方法
CN108470182A (zh) * 2018-01-23 2018-08-31 天津大学 一种用于非对称脑电特征增强与识别的脑-机接口方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2779842C (en) * 2009-11-04 2021-06-22 Arizona Board Of Regents For And On Behalf Of Arizona State University Devices and methods for modulating brain activity
CA3187490A1 (en) * 2013-03-15 2014-09-18 Interaxon Inc. Wearable computing apparatus and method
AT515038B1 (de) * 2013-10-21 2015-12-15 Guger Christoph Dipl Ing Dr Techn Verfahren zur Quantifizierung der Wahrnehmungsfähigkeit einer Person
PL409241A1 (pl) * 2014-08-22 2016-02-29 Maciej Rudziński Sposób i system przetwarzania sygnału EEG
EP3184043A1 (en) * 2015-12-22 2017-06-28 IMEC vzw Sensor, system and holder arrangement for biosignal activity measurement
WO2019094953A1 (en) * 2017-11-13 2019-05-16 Neurable Inc. Brain-computer interface with adaptations for high-speed, accurate, and intuitive user interactions

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1778272A (zh) * 2004-11-22 2006-05-31 中国科学院心理研究所 便携式事件相关电位仪
CN103150017A (zh) * 2013-03-05 2013-06-12 天津大学 基于空间、时间和频率联合编码的脑-机接口通讯方法
CN105824418A (zh) * 2016-03-17 2016-08-03 天津大学 一种基于非对称视觉诱发电位的脑-机接口通讯系统
CN108469896A (zh) * 2018-01-23 2018-08-31 天津大学 一种基于非对称脑电特征的脑-机接口混合编码范式方法
CN108470182A (zh) * 2018-01-23 2018-08-31 天津大学 一种用于非对称脑电特征增强与识别的脑-机接口方法

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112861629A (zh) * 2021-01-07 2021-05-28 天津大学 一种多窗口判别典型模式匹配方法及脑-机接口应用
CN113143291A (zh) * 2021-05-11 2021-07-23 燕山大学 一种快速序列视觉呈现下的脑电特征提取方法
CN113672082A (zh) * 2021-07-12 2021-11-19 华南理工大学 一种多通道高共模抑制比低功耗脑电采集的脑机接口系统
CN113705732A (zh) * 2021-09-26 2021-11-26 华东理工大学 基于通用模型的减少p300训练时间的方法及装置
CN113705732B (zh) * 2021-09-26 2024-02-02 华东理工大学 基于通用模型的减少p300训练时间的方法及装置

Also Published As

Publication number Publication date
US20210173482A1 (en) 2021-06-10
US11221672B2 (en) 2022-01-11

Similar Documents

Publication Publication Date Title
WO2019144776A1 (zh) 一种基于非对称脑电特征的脑-机接口系统编解码方法
Zhang et al. Predicting inter-session performance of SMR-based brain–computer interface using the spectral entropy of resting-state EEG
CN109784023B (zh) 基于深度学习的稳态视觉诱发脑电身份识别方法及系统
CN105956624B (zh) 基于空时频优化特征稀疏表示的运动想象脑电分类方法
Kelly et al. Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication
Kachenoura et al. ICA: a potential tool for BCI systems
Lee et al. A brain-wave-actuated small robot car using ensemble empirical mode decomposition-based approach
Ong et al. Power spectral density analysis for human EEG-based biometric identification
Li et al. Decoding EEG in cognitive tasks with time-frequency and connectivity masks
CN111580643B (zh) 一种基于稳态不对称视觉诱发电位的脑-机接口方法
Zhang et al. Clustering linear discriminant analysis for MEG-based brain computer interfaces
CN111184509A (zh) 一种基于传递熵的情绪诱导脑电信号分类方法
Zhao et al. Individual identification based on code-modulated visual-evoked potentials
CN101828921A (zh) 基于视觉诱发脑电vep的身份识别方法
Dan et al. An identification system based on portable EEG acquisition equipment
CN108470182B (zh) 一种用于非对称脑电特征增强与识别的脑-机接口方法
CN112137616B (zh) 一种多感觉脑-体联合刺激的意识检测装置
Kaewwit et al. High accuracy EEG biometrics identification using ICA and AR model
CN101703405A (zh) 基于小波分析和Welch算法的身份识别方法
CN109009098A (zh) 一种运动想象状态下的脑电信号特征识别方法
Zheng et al. A power spectrum pattern difference-based time-frequency sub-band selection method for MI-EEG classification
CN112861629B (zh) 一种多窗口判别典型模式匹配方法及脑-机接口应用
CN106843509B (zh) 一种脑机接口系统
Wang et al. Optimized preprocessing and tiny ml for attention state classification
Wei et al. Mild cognitive impairment classification convolutional neural network with attention mechanism

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18902593

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10/11/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18902593

Country of ref document: EP

Kind code of ref document: A1