WO2024093796A1 - 一种基于双脑耦合特征的脑机接口控制方法及系统 - Google Patents

一种基于双脑耦合特征的脑机接口控制方法及系统 Download PDF

Info

Publication number
WO2024093796A1
WO2024093796A1 PCT/CN2023/126874 CN2023126874W WO2024093796A1 WO 2024093796 A1 WO2024093796 A1 WO 2024093796A1 CN 2023126874 W CN2023126874 W CN 2023126874W WO 2024093796 A1 WO2024093796 A1 WO 2024093796A1
Authority
WO
WIPO (PCT)
Prior art keywords
brain
dual
eeg
feature matrix
computer interface
Prior art date
Application number
PCT/CN2023/126874
Other languages
English (en)
French (fr)
Inventor
李翀
季林红
贾天宇
孙晶尧
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Publication of WO2024093796A1 publication Critical patent/WO2024093796A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Definitions

  • the present application relates to the field of brain-computer interface control technology, and in particular to a brain-computer interface control method and system based on dual-brain coupling characteristics.
  • Brain-computer interface technology decodes the human body's intentions to control external devices through thoughts, so that the completion of tasks no longer depends on the limbs of the human body, greatly expanding the human's motor control ability and range.
  • single-person brain-computer interface control technology is difficult to complete the execution of complex tasks, so it requires the cooperation of two or more people to complete the control of complex tasks. Therefore, the degree of cooperation between two or more people directly affects the decoding accuracy and control precision of the brain-computer interface.
  • the improvement of the degree of cooperation in cooperative tasks can be reflected in the synchronization of the EEG signals of both parties, which is also called dual-brain coupling.
  • This application proposes a brain-computer interface control method based on dual-brain coupling characteristics, and proposes to construct a dual-person brain-computer interface control algorithm model based on the feature of synchronization of dual-person EEG characteristics, so as to promote the improvement of the accuracy of brain-computer interface control based on dual-brain coupling characteristics.
  • Another object of the present application is to propose a brain-computer interface control system based on dual-brain coupling characteristics.
  • the present application proposes a brain-computer interface control method based on dual-brain coupling characteristics, including:
  • the method before at least two subjects collaborate to perform a brain-computer interface motion control task, the method further includes: obtaining a preset motion instruction.
  • the dual-brain coupling feature extraction is performed based on the lead pair EEG data, and the data is transformed based on the extracted features to obtain a feature matrix, including: obtaining a weight vector of the brain region where the lead pair EEG data is located based on the difference information of the brain region functions, and calculating the instantaneous phase of the lead pair EEG signal through Hilbert transform, and weighting the instantaneous phase according to the weight vector to obtain a weighted instantaneous phase; calculating the correlation coefficient of the lead pair based on the weighted instantaneous phase and a preset formula, and standardizing the correlation coefficient using Fisher’s Z transform to obtain a dual-brain coupling feature matrix; numerically sorting the correlation values of the lead pairs in the dual-brain coupling feature matrix, and obtaining the feature matrix based on the numerical sorting results.
  • the numerical sorting is performed according to the correlation values of the lead pairs in the dual-brain coupling feature matrix, and the feature matrix is obtained based on the numerical sorting result, including: obtaining multiple lead pairs in the dual-brain coupling feature matrix; arranging the preset number of lead pairs with the largest correlation values in the multiple lead pairs from large to small to obtain the first dimension feature; using the matrix row and column position number value corresponding to the first dimension feature in the dual-brain coupling feature matrix as the second dimension information, and obtaining the feature matrix based on the first dimension feature and the second dimension information.
  • the expression for calculating the correlation coefficient of the EEG training data lead pair based on the weighted instantaneous phase and the preset formula is:
  • ⁇ w is the correlation coefficient of any pair of leads between the two brains, ⁇ w and are the weighted instantaneous phases of the EEG signals of any pair of leads between the two brains.
  • a brain-computer interface control system based on dual-brain coupling characteristics comprising:
  • a collection and coordination module for responding to at least two subjects cooperating to perform a brain-computer interface motion control task, to divide the synchronously collected lead-pair EEG data of at least two subjects into EEG training data and EEG test data; wherein the motion control task includes at least one motion pattern;
  • the feature transformation module is used to extract dual-brain coupling features based on the EEG training data, and to The row data is transformed to obtain the feature matrix;
  • a classification test module used for inputting the feature matrix into a classifier model for training to obtain a classification model, classifying the movement pattern of the EEG test data based on the classification model to obtain a pattern classification result, judging the movement pattern completion of the pattern classification result, and obtaining a test classification accuracy rate according to the completion judgment result;
  • the collaborative control module is used to determine the effectiveness of collaborative brain-computer interface control based on the test classification accuracy.
  • the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned brain-computer interface control method based on dual-brain coupling characteristics when executing the program.
  • the present application also provides a non-transitory computer-readable storage medium, which includes a computer program.
  • the computer program When executed by the processor, it implements the above-mentioned brain-computer interface control method based on dual-brain coupling characteristics.
  • the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned brain-computer interface control method based on dual-brain coupling characteristics.
  • the brain-computer interface control method and system based on dual-brain coupling characteristics of the embodiments of the present application realize the control of the brain-computer interface by decoding the collaborative cooperation intention, thereby promoting the improvement of the control accuracy of the brain-computer interface based on dual-brain coupling characteristics.
  • FIG1 is a flow chart of a brain-computer interface control method based on dual-brain coupling characteristics according to an embodiment of the present application
  • FIG2 is a schematic diagram of data collection according to an embodiment of the present application.
  • FIG3 is a visualization diagram of a dual-brain coupling feature matrix according to an embodiment of the present application.
  • FIG4 is a schematic diagram showing a comparison of decoding accuracy results in a collaborative scenario with other non-collaborative scenarios according to an embodiment of the present application
  • FIG5 is a schematic diagram of the structure of a brain-computer interface control system based on dual-brain coupling characteristics according to an embodiment of the present application.
  • FIG6 is a schematic diagram of a physical structure of an electronic device provided according to an embodiment of the present application.
  • FIG1 is a flow chart of a brain-computer interface control method based on dual-brain coupling characteristics according to an embodiment of the present application.
  • the method includes but is not limited to the following steps:
  • this step requires at least two subjects to collaboratively complete a brain-computer interface motion control task, and synchronously collect n-lead EEG data of at least two subjects for subsequent extraction of dual-brain coupling features.
  • Part of the n-lead EEG data is used as EEG training data for training the subsequent model, and part is used as EEG test data for testing the model.
  • 12 pairs of friends who have known each other for one year can be selected as subjects for the embodiment experiment of this application.
  • Each pair of subjects collaborates to complete a brain-computer interface control task. Eye contact and physical contact must be maintained during the experiment.
  • the control task requires the subjects to cooperate with the brain-computer interface to control the robot arm to move precisely along a given trajectory after receiving the initial instructions.
  • the initial instructions can be visual, auditory and other forms of prompts provided to the subjects through external devices.
  • 31 leads of EEG data of each pair of subjects are collected during the experiment for extraction of dual-brain coupling features.
  • a weight vector W of the brain region where each lead is located is defined, and effective information of brain region activities closely related to the motor control task function in step S1 is highlighted;
  • Fisher's Z transform is used to convert the correlation coefficient of any lead pair between the two brains calculated by formula (1) into Standardization is performed to ensure that it conforms to the normal distribution, and the weighted correlation feature matrix C (dual-brain coupling feature matrix) of the paired lead pairs between the two brains is obtained.
  • C is visualized and plotted as shown in FIG3 (FIG3 only plots the first 10% of the dual-brain coupling lead pairs with the largest correlation values);
  • the first 9 values with the largest correlation values in the 31*31 lead pairs in the weighted correlation feature matrix C are arranged from large to small to form the first dimension feature, and the corresponding matrix row and column position number values in C are used as the second dimension information, and these two dimensions of information form the feature matrix X.
  • X is input as a feature into the support vector machine model (a classifier model) for training, and a classification model is obtained by training, and the EEG test data collected in step S1 is used for model classification test, and the test classification accuracy is output. It can be obtained from the output result, as shown in Figure 4, that its classification accuracy in the collaborative scenario is significantly higher than the model recognition accuracy without considering the collaborative factor. For example, the subject will be told what task to do this time through the early instruction prompts, sound prompts, visual prompts, etc., and the completion degree is judged as whether the task is completed.
  • a two-person brain-computer interface control algorithm model is constructed to promote the improvement of the control accuracy of the brain-computer interface based on the dual-brain coupling feature.
  • the robot arm is controlled to move precisely according to a given trajectory, such as controlling the robot arm to pick up objects, or assisting patients in rehabilitation training, etc.
  • the control of the brain-computer interface is achieved by decoding the collaborative cooperation intention, thereby promoting the improvement of the control accuracy of the brain-computer interface based on dual-brain coupling characteristics.
  • a brain-computer interface control system 10 based on dual-brain coupling characteristics is also provided in the embodiment of the present application.
  • the system 10 includes: an acquisition coordination module 100 , a feature transformation module 200 , a classification test module 300 and a collaborative control module 400 .
  • the acquisition coordination module 100 is used for dividing the lead pair EEG data of at least two subjects synchronously collected into EEG training data and EEG test data in response to at least two subjects collaboratively performing a motion control task of a brain-computer interface; wherein the motion control task includes at least one motion mode;
  • a feature transformation module 200 is used to extract dual-brain coupling features based on EEG training data, and to perform data transformation based on the extracted features to obtain a feature matrix;
  • the classification test module 300 is used to input the feature matrix into the classifier model for training to obtain a classification model, classify the movement pattern of the EEG test data based on the classification model to obtain a pattern classification result, judge the movement pattern completion of the pattern classification result, and obtain the test classification accuracy according to the completion judgment result;
  • the collaborative control module 400 is used to determine the effectiveness of implementing collaborative brain-computer interface control based on the test classification accuracy.
  • system 10 further includes:
  • the command transmission module is used to obtain preset motion commands.
  • the feature transformation module 200 includes:
  • the first transformation subunit is used to obtain the weight vector of the brain region where the EEG training data is located based on the difference information of the brain region function, and to obtain the instantaneous phase of the EEG training data by Hilbert transformation calculation, and to weight the instantaneous phase according to the weight vector to obtain the weighted instantaneous phase;
  • the second transformation subunit is used to calculate the correlation coefficient of the EEG training data lead pair based on the weighted instantaneous phase and the preset formula, and use Fisher’s Z transformation to standardize the correlation coefficient to obtain the dual-brain coupling feature matrix;
  • the sorting output subunit is used to numerically sort the lead pairs in the dual-brain coupling feature matrix according to the correlation values, and obtain the feature matrix based on the numerical sorting results.
  • the above-mentioned sorting output subunit is used to:
  • the matrix row and column position number values corresponding to the first-dimensional features in the dual-brain coupling feature matrix are used as the second-dimensional information, and the feature matrix is obtained based on the first-dimensional features and the second-dimensional information.
  • ⁇ w is the correlation coefficient of any pair of leads between the two brains, ⁇ w and are the weighted instantaneous phases of the EEG signals of any pair of leads between the two brains.
  • FIG6 is a schematic diagram of the physical structure of an electronic device provided according to an embodiment of the present application.
  • the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640.
  • the processor 610 may call the logic instructions in the memory 630 to execute a brain-computer interface control method based on dual-brain coupling characteristics, and the method includes:
  • the logic instructions in the above-mentioned memory 630 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art.
  • the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc.
  • An embodiment of the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the brain-computer interface control method based on dual-brain coupling characteristics provided in the above embodiment when executing the computer program.
  • An embodiment of the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon.
  • the computer program is executed by a processor, the brain-computer interface control method based on the dual-brain coupling feature provided in the above embodiment is implemented.
  • the processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.
  • magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.
  • optical storage such as CD, DVD, BD, HVD, etc.
  • semiconductor storage such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)
  • An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the brain-computer interface control method based on the dual-brain coupling feature provided in the above embodiment.
  • the control of the brain-computer interface is achieved by decoding the collaborative cooperation intention, thereby promoting the improvement of the control accuracy of the brain-computer interface based on the dual-brain coupling feature.
  • first and second are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality” is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Psychiatry (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Physiology (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Fuzzy Systems (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Psychology (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

一种基于双脑耦合特征的脑机接口控制方法包括：响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；运动控制任务包括至少一种运动模式；基于脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；将特征矩阵输入分类器模型中进行训练得到分类模型，基于分类模型对脑电测试数据进行运动模式的分类得到模式分类结果，对模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；基于测试分类准确率，判断协同脑机接口控制的有效性。

Description

一种基于双脑耦合特征的脑机接口控制方法及系统
相关申请的交叉引用
本申请基于申请号为202211375933X、申请日为2022年11月04日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及脑机接口控制技术领域，尤其涉及一种基于双脑耦合特征的脑机接口控制方法及系统。
背景技术
脑机接口技术通过解码人体的意图实现通过意念控制外部设备，使得任务的完成不再依赖于人身体的四肢，极大的拓展了人的运动控制能力和范围。但是，随着外部设备控制复杂性的提升，单人脑机接口控制技术难以完成复杂任务的执行，因此需要双人或多人的协同配合共同完成复杂任务的控制。因此，双人或多人的协同配合程度直接影响了脑机接口的解码准确率和控制精确性。合作任务中协同配合程度的提升可反映在双方的脑电信号的同步化，该同步化又被称为双脑耦合。2018年Goldstein等人证明了双脑耦合特征与人痛觉感知的敏感性之间具有相关性，2021年Reinero等人证明了大脑间的同步耦合特征可预测团队合作的最终工作绩效，并且合作者们大脑间的同步化会带来更好的任务完成表现。从上述双脑耦合特征与行为学的相关关系研究得出，基于双脑耦合特征构建意图解码模型的方法将进一步提升双人协同配合的意图解码准确率，提高对复杂任务的脑机接口控制效率。
已有的基于双脑的特征信息挖掘和解码的方法，如专利CN202110373684，是将双人各自的脑电信息分别作为特征输入，实现特征信息的融合和解码。不足之处在于忽略了双人在进行同一任务情况下双脑信息的协同耦合特征，缺少对双人协同配合下脑电特征同步化这一特征的挖掘和利用，限制了脑机接口控制准确率的提升。
发明内容
本申请提出了一种基于双脑耦合特征的脑机接口控制方法，提出基于双人脑电特征同步化这一特征构建双人脑机接口控制算法模型，促进基于双脑耦合特征的脑机接口控制准确率的提升。
本申请的另一个目的在于提出一种基于双脑耦合特征的脑机接口控制系统。
为达上述目的，本申请一方面提出了一种基于双脑耦合特征的脑机接口控制方法，包括：
响应于至少两个被试协同进行脑机接口的运动控制任务，同步采集至少两个被试的导联对脑电数据；其中，所述运动控制任务包括至少一种运动模式；
基于所述导联对脑电数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；
将所述特征矩阵输入分类器模型中进行训练得到分类模型，基于所述分类模型对所述脑电测试数据进行所述运动模式的分类得到模式分类结果，对所述模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
基于所述测试分类准确率，判断实现协同脑机接口控制的有效性。
根据本申请实施例的基于双脑耦合特征的脑机接口控制方法还可以具有以下附加技术特征：
进一步地，在本申请的一个实施例中，所述在至少两个被试协同进行脑机接口的运动控制任务之前，所述方法，还包括：获取预设的运动指令。
进一步地，在本申请的一个实施例中，所述基于导联对脑电数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵，包括：基于脑区功能的差异信息获取导联对脑电数据所在脑区的权重向量，以及通过Hilbert变换计算得到导联对脑电信号的瞬时相位，并根据所述权重向量对所述瞬时相位进行赋权得到赋权瞬时相位；基于所述赋权瞬时相位和预设公式计算导联对的相关系数，利用Fisher’s Z变换将所述相关系数进行标准化操作，得到双脑耦合特征矩阵；根据所述双脑耦合特征矩阵中导联对的相关度值进行数值排序，基于数值排序结果得到所述特征矩阵。
进一步地，在本申请的一个实施例中，所述根据双脑耦合特征矩阵中导联对的相关度值进行数值排序，基于数值排序结果得到所述特征矩阵，包括：：获取双脑耦合特征矩阵中的多个导联对；将所述多个导联对中相关度值最大的预设数量数值由大到小进行排列，得到第一维特征；将所述第一维特征对应的在所述双脑耦合特征矩阵中的矩阵行列位置编号值作为第二维信息，基于所述第一维特征和所述第二维信息得到特征矩阵。
进一步地，在本申请的一个实施例中，所述基于赋权瞬时相位和预设公式计算脑电训练数据导联对的相关系数的表达式为：
其中，为两脑间任意导联对的相关系数，φw和分别为两脑间任意导联对的脑电信号的赋权瞬时相位。
为达到上述目的，本申请另一方面提出了一种基于双脑耦合特征的脑机接口控制系统，包括：
采集协同模块，用于响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，所述运动控制任务包括至少一种运动模式；
特征变换模块，用于基于所述脑电训练数据进行双脑耦合特征提取，基于提取的特征进 行数据变换得到特征矩阵；
分类测试模块，用于将所述特征矩阵输入分类器模型中进行训练得到分类模型，基于所述分类模型对所述脑电测试数据进行运动模式的分类得到模式分类结果，对所述模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
协同控制模块，用于基于所述测试分类准确率，判断协同脑机接口控制的有效性。
本申请还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述基于双脑耦合特征的脑机接口控制方法。
本申请还提供一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质包括计算机程序，所述计算机程序被所述处理器执行时实现上述基于双脑耦合特征的脑机接口控制方法。
本申请还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述基于双脑耦合特征的脑机接口控制方法。
本申请实施例的基于双脑耦合特征的脑机接口控制方法及系统，通过对协同配合意图的解码实现对脑机接口的控制，促进基于双脑耦合特征的脑机接口控制准确率的提升。
附图说明
图1为根据本申请实施例的基于双脑耦合特征的脑机接口控制方法的流程图；
图2为根据本申请实施例的数据采集示意图；
图3为根据本申请实施例的双脑耦合特征矩阵可视化图；
图4为根据本申请实施例的协同配合场景下的解码准确率与其他非协同下的准确率结果比较示意图；
图5为根据本申请实施例的基于双脑耦合特征的脑机接口控制系统结构示意图。
图6为根据本申请实施例提供的一种电子设备的实体结构示意图。
具体实施方式
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
下面参照附图描述根据本申请实施例提出的基于双脑耦合特征的脑机接口控制方法及系统。
图1是本申请一个实施例的基于双脑耦合特征的脑机接口控制方法的流程图。
如图1所示，该方法包括但不限于以下步骤：
S1，响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，运动控制任务包括至少一种运动模式。
可以理解的是，该步骤是需要至少两个被试协同完成一项脑机接口运动控制任务，同步采集至少两个被试的n个导联的脑电数据，用于后续的双脑耦合特征的提取。将n个导联的脑电数据一部分作为脑电训练数据对后续模型的训练，一部分作为脑电测试数据用于模型的测试。
作为一种示例，如图2所示，可以选取12对相识了1年的朋友作为本申请的实施例实验的被试，每对被试协同完成一项脑机接口控制任务，实验中须保持眼神交流和肢体接触，该控制任务要求受试者在接收到初始的指令后，通过脑机接口协同配合控制机械臂按照给定轨迹精准运动，初始的指令可以为通过外界设备向被试者提供的视觉、听觉等多种形式的提示。在一些实施例中，该实验过程中采集每对被试的31个导联的脑电数据，用于双脑耦合特征的提取。
S2，基于脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵。
在一些实施例中，依据被试者脑区功能的差异，定义各导联所在脑区的权重向量W，将与步骤S1中运动控制任务功能密切相关的脑区活动的有效信息凸显出来；
通过Hilbert变换计算得到两脑间任意导联对的脑电信号的瞬时相位分别为φ和并依据权重向量W，对φ和分别进行赋权得到φw和
基于计算所得到的瞬时相位φw和按下式计算两脑间任意导联对的相关系数
采用Fisher’s Z变换将式(1)计算得到的两脑间任意导联对的相关系数进行标准化以保证其符合正态分布，得到两脑间配对导联对带权的相关特征矩阵C(双脑耦合特征矩阵)，将C进行可视化绘图如图3所示(图3中仅画出了相关度值最大的前10％的双脑耦合导联对)；
将带权的相关特征矩阵C中31*31个导联对中相关度值最大的前9个数值由大到小排列组成第一维特征，将其对应的在C中的矩阵行列位置编号值作为第二维信息，将这两维信息组成特征矩阵X。
S3，将特征矩阵输入分类器模型中进行训练得到分类模型，基于分类模型对脑电测试数据进行运动模式的分类得到模式分类结果，对模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
S4，基于测试分类准确率，判断实现协同脑机接口控制的有效性。。
可以理解的是，将X作为特征输入到支持向量机模型(一种分类器模型)中进行训练，训练得到分类模型，并用步骤S1中采集到的脑电测试数据进行模型分类测试，输出测试分类准确率。由输出结果可得，如图4所示，在协同配合场景下其分类准确率明显高于未考虑协同配合因素的模型识别准确率。例如，通过前期的指令提示会告诉受试者这次要做什么任务，声音提示、视觉提示等，判断完成度为是否完成该任务，基于双人脑电特征同步化这一特征构建双人脑机接口控制算法模型，促进基于双脑耦合特征的脑机接口控制准确率的提升。通过脑机接口协同配合控制机械臂按照给定轨迹精准运动，如控制机械臂拿取物品、或者辅助患者进行康复训练等等。
根据本申请实施例的基于双脑耦合特征的脑机接口控制方法，通过对协同配合意图的解码实现对脑机接口的控制，促进基于双脑耦合特征的脑机接口控制准确率的提升。
为了实现上述实施例，如图5所示，本申请的实施例中还提供了基于双脑耦合特征的脑机接口控制系统10，该系统10包括：采集协同模块100、特征变换模块200、分类测试模块300和协同控制模块400。
采集协同模块100，用于响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，运动控制任务包括至少一种运动模式；
特征变换模块200，用于基于脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；
分类测试模块300，用于将特征矩阵输入分类器模型中进行训练得到分类模型，基于分类模型对脑电测试数据进行运动模式的分类得到模式分类结果，对模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
协同控制模块400，用于基于测试分类准确率，判断实现协同脑机接口控制的有效性。
进一步的，系统10，还包括：
指令传输模块，用于获取预设的运动指令。
进一步的，上述特征变换模块200，包括：
第一变换子单元，用于基于脑区功能的差异信息获取脑电训练数据所在脑区的权重向量，以及通过Hilbert变换计算得到脑电训练数据的瞬时相位，并根据权重向量对瞬时相位进行赋权得到赋权瞬时相位；
第二变换子单元，用于基于赋权瞬时相位和预设公式计算脑电训练数据导联对的相关系数，利用Fisher’s Z变换将相关系数进行标准化操作，得到双脑耦合特征矩阵；
排序输出子单元，用于根据双脑耦合特征矩阵中导联对的相关度值进行数值排序，基于数值排序结果得到特征矩阵。
进一步的，上述排序输出子单元，用于：
获取双脑耦合特征矩阵中脑电训练数据的多个导联对；
将多个导联对中相关度值最大的预设数量数值由大到小进行排列，得到第一维特征；
将第一维特征对应的在双脑耦合特征矩阵中的矩阵行列位置编号值作为第二维信息，基于第一维特征和第二维信息得到特征矩阵。
进一步的，相关系数的表达式为：
其中，为两脑间任意导联对的相关系数，φw和分别为两脑间任意导联对的脑电信号的赋权瞬时相位。
图6为根据本申请实施例提供的一种电子设备的实体结构示意图，如图6所示，电子设备可以包括：处理器(processor)610、通信接口(Communications Interface)620、存储器(memory)630和通信总线640，其中，处理器610，通信接口620，存储器630通过通信总线640完成相互间的通信。处理器610可以调用存储器630中的逻辑指令，以执行基于双脑耦合特征的脑机接口控制方法，该方法包括：
响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，所述运动控制任务包括至少一种运动模式；
基于所述脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；
将所述特征矩阵输入分类器模型中进行训练得到分类模型，基于所述分类模型对所述脑电测试数据进行运动模式的分类得到模式分类结果，对所述模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
基于所述测试分类准确率，判断协同脑机接口控制的有效性。
此外，上述的存储器630中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
本申请的实施例还提供一种电子设备，所述电子设备包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述计算机程序时实现上述实施例提供的基于双脑耦合特征的脑机接口控制方法。
本申请的实施例还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述实施例提供的基于双脑耦合特征的脑机接口控制方法。
所述处理器可读存储介质可以是处理器能够存取的任何可用介质或数据存储设备，包括但不限于磁性存储器(例如软盘、硬盘、磁带、磁光盘(MO)等)、光学存储器(例如CD、DVD、BD、HVD等)、以及半导体存储器(例如ROM、EPROM、EEPROM、非易失性存储器(NAND FLASH)、固态硬盘(SSD))等。
本申请的实施例还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述实施例提供的基于双脑耦合特征的脑机接口控制方法。
根据本申请实施例的基于双脑耦合特征的脑机接口控制系统，通过对协同配合意图的解码实现对脑机接口的控制，促进基于双脑耦合特征的脑机接口控制准确率的提升。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (13)

  1. 一种基于双脑耦合特征的脑机接口控制方法，包括以下步骤：
    响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，所述运动控制任务包括至少一种运动模式；
    基于所述脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；
    将所述特征矩阵输入分类器模型中进行训练得到分类模型，基于所述分类模型对所述脑电测试数据进行运动模式的分类得到模式分类结果，对所述模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
    基于所述测试分类准确率，判断协同脑机接口控制的有效性。
  2. 根据权利要求1所述的方法，其中，所述在至少两个被试协同进行脑机接口的运动控制任务之前，所述方法，还包括：
    获取预设的运动指令。
  3. 根据权利要求1所述的方法，其中，所述基于脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵，包括：
    基于脑区功能的差异信息获取脑电训练数据所在脑区的权重向量，以及通过Hilbert变换计算得到脑电训练数据的瞬时相位，并根据所述权重向量对所述瞬时相位进行赋权得到赋权瞬时相位；
    基于所述赋权瞬时相位和预设公式计算脑电训练数据导联对的相关系数，利用Fisher’s Z变换将所述相关系数进行标准化操作，得到双脑耦合特征矩阵；
    根据所述双脑耦合特征矩阵中导联对的相关度值进行数值排序，基于数值排序结果得到所述特征矩阵。
  4. 根据权利要求3所述的方法，其中，所述根据双脑耦合特征矩阵中脑电训练数据导联对的相关度值进行数值排序，基于数值排序结果得到所述特征矩阵，包括：
    获取双脑耦合特征矩阵中脑电训练数据的多个导联对；
    将所述多个导联对中相关度值最大的预设数量数值由大到小进行排列，得到第一维特征；
    将所述第一维特征对应的在所述双脑耦合特征矩阵中的矩阵行列位置编号值作为第二维信息，基于所述第一维特征和所述第二维信息得到特征矩阵。
  5. 根据权利要求3所述的方法，其中，所述基于赋权瞬时相位和预设公式计算脑电训练数据导联对的相关系数的表达式为：
    其中，为两脑间任意导联对的相关系数，φw和分别为两脑间任意导联对的脑电信号的赋权瞬时相位。
  6. 一种基于双脑耦合特征的脑机接口控制系统，包括：
    采集协同模块，用于响应于至少两个被试协同进行脑机接口的运动控制任务，将同步采集至少两个被试的导联对脑电数据分为脑电训练数据和脑电测试数据；其中，所述运动控制任务包括至少一种运动模式；
    特征变换模块，用于基于所述脑电训练数据进行双脑耦合特征提取，基于提取的特征进行数据变换得到特征矩阵；
    分类测试模块，用于将所述特征矩阵输入分类器模型中进行训练得到分类模型，基于所述分类模型对所述脑电测试数据进行运动模式的分类得到模式分类结果，对所述模式分类结果的运动模式完成度进行判断，根据完成度判断结果得到测试分类准确率；
    协同控制模块，用于基于所述测试分类准确率，判断协同脑机接口控制的有效性。
  7. 根据权利要求6所述的系统，其中，所述系统，还包括：
    指令传输模块，用于获取预设的运动指令。
  8. 根据权利要求6所述的系统，其中，所述特征变换模块，包括：
    第一变换子单元，用于基于脑区功能的差异信息获取脑电训练数据所在脑区的权重向量，以及通过Hilbert变换计算得到脑电训练数据的瞬时相位，并根据所述权重向量对所述瞬时相位进行赋权得到赋权瞬时相位；
    第二变换子单元，用于基于所述赋权瞬时相位和预设公式计算脑电训练数据导联对的相关系数，利用Fisher’s Z变换将所述相关系数进行标准化操作，得到双脑耦合特征矩阵；
    排序输出子单元，用于根据所述双脑耦合特征矩阵中导联对的相关度值进行数值排序，基于数值排序结果得到所述特征矩阵。
  9. 根据权利要求8所述的系统，其中，所排序输出子单元，还用于：
    获取双脑耦合特征矩阵中脑电训练数据的多个导联对；
    将所述多个导联对中相关度值最大的预设数量数值由大到小进行排列，得到第一维特征；
    将所述第一维特征对应的在所述双脑耦合特征矩阵中的矩阵行列位置编号值作为第二维信息，基于所述第一维特征和所述第二维信息得到特征矩阵。
  10. 根据权利要求8所述的系统，其中，所述相关系数的表达式为：
    其中，为两脑间任意导联对的相关系数，φw和分别为两脑间任意导联对的脑电信号的赋权瞬时相位。
  11. 一种电子设备，所述电子设备包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述计算机程序时实现权利要求1至5中任一项所述的基于双脑耦合特征的脑机接口控制方法。
  12. 一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质包括计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至5中任一项所述的基于双脑耦合特征的脑机接口控制方法。
  13. 一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如权利要求1至5中任一项所述的基于双脑耦合特征的脑机接口控制方法。
PCT/CN2023/126874 2022-11-04 2023-10-26 一种基于双脑耦合特征的脑机接口控制方法及系统 WO2024093796A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211375933.X 2022-11-04
CN202211375933.XA CN115421597B (zh) 2022-11-04 2022-11-04 一种基于双脑耦合特征的脑机接口控制方法及系统

Publications (1)

Publication Number Publication Date
WO2024093796A1 true WO2024093796A1 (zh) 2024-05-10

Family

ID=84208142

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/126874 WO2024093796A1 (zh) 2022-11-04 2023-10-26 一种基于双脑耦合特征的脑机接口控制方法及系统

Country Status (2)

Country Link
CN (1) CN115421597B (zh)
WO (1) WO2024093796A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115421597B (zh) * 2022-11-04 2023-01-13 清华大学 一种基于双脑耦合特征的脑机接口控制方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108363493A (zh) * 2018-03-20 2018-08-03 山东建筑大学 基于脑机接口的用户特征模型建立方法、系统及存储介质
CN112465059A (zh) * 2020-12-07 2021-03-09 杭州电子科技大学 基于跨脑融合决策的多人运动想象识别方法及脑机系统
CN113191395A (zh) * 2021-04-07 2021-07-30 杭州电子科技大学 一种基于双脑的多层次信息融合的目标检测方法
CN113867533A (zh) * 2021-09-30 2021-12-31 中国人民解放军战略支援部队信息工程大学 多脑协同脑机接口系统及基于该系统实现的视频目标检测方法
CN115421597A (zh) * 2022-11-04 2022-12-02 清华大学 一种基于双脑耦合特征的脑机接口控制方法及系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112433617B (zh) * 2020-12-11 2022-06-14 中国人民解放军国防科技大学 一种两人协同的p300-bci目标决策系统及方法
CN114469090A (zh) * 2021-12-31 2022-05-13 杭州电子科技大学 基于跨脑耦合关系计算的脑电情感识别方法及脑机系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108363493A (zh) * 2018-03-20 2018-08-03 山东建筑大学 基于脑机接口的用户特征模型建立方法、系统及存储介质
CN112465059A (zh) * 2020-12-07 2021-03-09 杭州电子科技大学 基于跨脑融合决策的多人运动想象识别方法及脑机系统
CN113191395A (zh) * 2021-04-07 2021-07-30 杭州电子科技大学 一种基于双脑的多层次信息融合的目标检测方法
CN113867533A (zh) * 2021-09-30 2021-12-31 中国人民解放军战略支援部队信息工程大学 多脑协同脑机接口系统及基于该系统实现的视频目标检测方法
CN115421597A (zh) * 2022-11-04 2022-12-02 清华大学 一种基于双脑耦合特征的脑机接口控制方法及系统

Also Published As

Publication number Publication date
CN115421597A (zh) 2022-12-02
CN115421597B (zh) 2023-01-13

Similar Documents

Publication Publication Date Title
ALEnezi A method of skin disease detection using image processing and machine learning
Ambekar et al. Disease risk prediction by using convolutional neural network
WO2024093796A1 (zh) 一种基于双脑耦合特征的脑机接口控制方法及系统
US20200337580A1 (en) Time series data learning and analysis method using artificial intelligence
Alharbi et al. Using genetic algorithm and ELM neural networks for feature extraction and classification of type 2-diabetes mellitus
JP7191443B2 (ja) 機械学習に基づくターゲットオブジェクト属性予測方法、関連機器及びコンピュータプログラム
JP7320856B2 (ja) 生体画像診断システム、生体画像診断方法、及びこれを行うための端末
JP7032747B1 (ja) 心電図解析支援装置、プログラム、心電図解析支援方法、及び心電図解析支援システム
Athif et al. WaveCSP: a robust motor imagery classifier for consumer EEG devices
CN111785385A (zh) 疾病分类方法、装置、设备和存储介质
Amin et al. Personalized health monitoring using predictive analytics
CN113488200A (zh) 智能问诊的方法、装置、计算机设备及存储介质
WO2022179645A2 (zh) 心电图分析方法、装置、电子设备和存储介质
CN114783580B (zh) 一种医疗数据质量评估方法及系统
CN113541958B (zh) 基于区块链的银行业务办理方法及装置
CN111785372A (zh) 基于关联规则的协同过滤疾病预测系统及其电子设备
US20170235913A1 (en) Image data ingestion applciation of a medical imaging data processing and retrieval system
Kampouraki et al. e-Doctor: a web based support vector machine for automatic medical diagnosis
Mirabet-Herranz et al. New insights on weight estimation from face images
Marcos et al. Emotional AI in Healthcare: a pilot architecture proposal to merge emotion recognition tools
CN113705595A (zh) 异常细胞转移程度的预测方法、装置和存储介质
CN114548262B (zh) 一种情感计算中多模态生理信号的特征级融合方法
CN113723519B (zh) 基于对比学习的心电数据处理方法、装置及存储介质
CN110853012B (zh) 获得心脏参数的方法、装置及计算机存储介质
Albi et al. Diabetes complication prediction using deep learning-based analytics

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23884728

Country of ref document: EP

Kind code of ref document: A1