WO2022042122A1 - 脑电信号的分类方法、分类模型的训练方法、装置及介质 - Google Patents

脑电信号的分类方法、分类模型的训练方法、装置及介质 Download PDF

Info

Publication number
WO2022042122A1
WO2022042122A1 PCT/CN2021/106614 CN2021106614W WO2022042122A1 WO 2022042122 A1 WO2022042122 A1 WO 2022042122A1 CN 2021106614 W CN2021106614 W CN 2021106614W WO 2022042122 A1 WO2022042122 A1 WO 2022042122A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
signal
distribution
domain
eeg signal
Prior art date
Application number
PCT/CN2021/106614
Other languages
English (en)
French (fr)
Inventor
柳露艳
洪晓林
马锴
郑冶枫
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP21859962.9A priority Critical patent/EP4101371A4/en
Publication of WO2022042122A1 publication Critical patent/WO2022042122A1/zh
Priority to US17/969,177 priority patent/US20230054751A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/0006—Exoskeletons, i.e. resembling a human figure
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A61B5/374—Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/04—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs motor-driven
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/10—Parts, details or accessories
    • A61G5/1051—Arrangements for steering
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55—Controlling game characters or game objects based on the game progress
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
    • A63F13/67—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/006—Controls for manipulators by means of a wireless system for controlling one or several manipulators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2505/00—Evaluating, monitoring or diagnosing in the context of a particular type of medical care
    • A61B2505/09—Rehabilitation or training
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G2203/00—General characteristics of devices
    • A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G2203/00—General characteristics of devices
    • A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
    • A61G2203/18—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering by patient's head, eyes, facial muscles or voice
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/10—Parts, details or accessories
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H2003/005—Appliances for aiding patients or disabled persons to walk about with knee, leg or stump rests
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H2003/007—Appliances for aiding patients or disabled persons to walk about secured to the patient, e.g. with belts
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/50—Control means thereof
    • A61H2201/5007—Control means thereof computer controlled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/50—Control means thereof
    • A61H2201/5023—Interfaces to the user
    • A61H2201/5041—Interfaces to the user control is restricted to certain individuals
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1012—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8082—Virtual reality

Definitions

  • the present application relates to the field of transfer learning, and in particular, to a method for classifying EEG signals, a method, device and medium for training a classification model.
  • EEG signals are electrical signals generated by each neuron during brain activity. Through EEG signals, the type of motor imagery corresponding to the EEG signals can be identified, that is, the body movements realized by the brain through "ideas".
  • EEG signals can be applied in the medical field, such as in the medical and health service cloud platform created by combining medical technology and using "cloud computing”. Medical staff can use EEG signals to check the patient's lesion area.
  • the brain-computer interface (Brain Computer Interface, BCI) is used to connect the EEG signal acquisition device with the computer device (external device), and the motor imagery represented by the EEG signal output by the brain-computer interface is identified through the external device (such as computer device). type to achieve direct brain control over objects. Since the EEG signals of different individuals are quite different, it is necessary to train a separate EEG signal classification model for each individual's EEG signal, so as to ensure that the relevant model can correctly identify the type of motor imagery represented by the EEG signal.
  • the EEG signal classification model can only identify the EEG signals used in the model training, which makes the use scenarios of the EEG signal classification model relatively limited and not universal.
  • the embodiments of the present application provide an EEG signal classification method, a classification model training method, device and medium.
  • the technical solution is as follows.
  • a method for classifying EEG signals is provided, which is applied to computer equipment, and the method includes:
  • the difference distribution ratio is obtained, and the difference distribution ratio is used to represent the influence of different types of difference distribution on the distribution of the signal feature and the source domain feature on the feature domain, and the source domain feature is corresponding to the source domain EEG signal.
  • the source domain EEG signal is the sample EEG signal used by the EEG signal classification model in the training process;
  • a training method for an EEG signal classification model is provided, which is applied to a computer device, and the method includes:
  • the difference distribution ratio is used to characterize the influence of different types of difference distributions on the distribution of the source domain feature and the target domain feature on the feature domain;
  • the aligned target domain features are classified, and the EEG signal classification model is trained according to the classification result to obtain a trained EEG signal classification model.
  • a device for classifying electroencephalogram signals comprising:
  • a first acquisition module used for acquiring EEG signals
  • a first feature extraction module configured to perform feature extraction on the EEG signal to obtain signal features corresponding to the EEG signal
  • the first obtaining module is used to obtain the difference distribution ratio, and the difference distribution ratio is used to represent the influence of different types of difference distributions on the distribution of the signal feature and the source domain feature on the feature domain, and the source domain
  • the feature is the feature corresponding to the source domain EEG signal; illustratively, the source domain EEG signal is the sample EEG signal used in the training process of the EEG signal classification model;
  • a first processing module configured to align the signal feature with the source domain feature according to the difference distribution ratio to obtain the aligned signal feature
  • a classification module configured to classify the aligned signal features to obtain a motor imagery type corresponding to the EEG signal.
  • an apparatus for training an EEG signal classification model comprising:
  • the second acquisition module is used to acquire the source domain EEG signal and the target domain EEG signal;
  • the second feature extraction module is configured to perform feature extraction on the source domain EEG signal and the target domain EEG signal, and obtain the source domain feature corresponding to the source domain EEG signal and the target domain EEG signal corresponding to the target domain features;
  • the second acquisition module is used to acquire the difference distribution ratio, and the difference distribution ratio is used to represent the influence of different types of difference distributions on the distribution of the source domain feature and the target domain feature on the feature domain;
  • a second processing module configured to align the source domain feature and the target domain feature on the feature domain according to the difference distribution ratio to obtain the aligned target domain feature
  • the training module is used for classifying the aligned target domain features, and training the EEG signal classification model according to the classification result to obtain a trained EEG signal classification model.
  • a computer device includes a processor and a memory, and the memory stores at least one instruction, at least a piece of program, code set or instruction set, the at least one instruction , The at least one piece of program, the code set or the instruction set is loaded and executed by the processor to implement the method for classifying EEG signals and the method for training an EEG signal classification model according to the above aspects.
  • a computer-readable storage medium wherein at least one instruction, at least one piece of program, code set or instruction set is stored in the readable storage medium, the at least one instruction, the at least one A piece of program, the code set or the instruction set is loaded and executed by the processor to implement the method for classifying EEG signals and the method for training an EEG signal classification model as described in the above aspects.
  • a computer program product or computer program comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method for classifying EEG signals and the EEG as described in the above aspects Training methods for signal classification models.
  • the signal features of the EEG signal are aligned with the source domain features, thereby ensuring that the signal features of the EEG signal tend to be close to the feature distribution.
  • Source domain features the EEG signal classification model trained based on the source domain features can transfer the classification method to the signal features corresponding to the EEG signal, and improve the accuracy of the EEG signal classification model to identify the motor imagery type corresponding to the EEG signal.
  • the EEG signal classification model can identify various types of EEG signals, which is universal.
  • FIG. 1 is a frame diagram of a computer system provided by an exemplary embodiment of the present application.
  • FIG. 2 is a flowchart of a method for classifying EEG signals provided by an exemplary embodiment of the present application
  • FIG. 3 is a schematic structural diagram of an EEG signal classification model provided by an exemplary embodiment of the present application.
  • FIG. 4 is a flowchart of a method for classifying EEG signals provided by another exemplary embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an EEG signal classification model provided by another exemplary embodiment of the present application.
  • FIG. 6 is a flowchart of a training method for an EEG signal classification model provided by an exemplary embodiment of the present application
  • FIG. 7 is a framework diagram of a training method for an EEG signal model provided by an exemplary embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an EEG signal classification model provided by another exemplary embodiment of the present application.
  • FIG. 9 is a structural block diagram of an apparatus for classifying EEG signals provided by an exemplary embodiment of the present application.
  • FIG. 10 is a structural block diagram of an apparatus for training an EEG signal classification model provided by an exemplary embodiment of the present application
  • FIG. 11 is a schematic diagram of an apparatus structure of a server provided by an exemplary embodiment of the present application.
  • Electroencephalogram refers to the electrical signals produced by each neuron during brain activity, and is the overall reflection of the electrophysiological activity of brain nerve cells on the surface of the cerebral cortex or scalp.
  • the EEG signal is obtained by collecting the biological voltage of the brain through an invasive or non-invasive brain-computer interface device, and the graph drawn by the EEG signal is an EEG.
  • the embodiments of the present application are described by taking the acquisition of the biological voltage of the brain through a non-invasive brain-computer interface device as an example.
  • MI Motor Imagery
  • BCI Brain Computer Interface
  • the BCI system combined with the electric wheelchair controls the movement direction of the electric wheelchair by collecting the user's EEG signals, helping users with physical disabilities to move freely.
  • Domain Adaptation Learning refers to a machine learning theory that can solve the inconsistency of the probability distribution of training samples and test samples.
  • DAL Domain Adaptation Learning
  • the training samples (source domain) and the test samples (target domain) come from the same probability distribution, and corresponding models and discriminant criteria are constructed to predict the samples to be tested.
  • the test sample and the training sample do not meet the condition that the probability distribution is the same, and the inconsistency of the probability distribution of the source domain and the target domain can be solved by domain adaptive learning.
  • domain adaptive learning the source domain refers to the existing knowledge or learned knowledge of the model; the target domain refers to the knowledge to be learned by the model.
  • Medical Cloud It refers to the use of "cloud computing” to create a cloud platform for medical and health services based on new technologies such as cloud computing, mobile technology, multimedia, 4G communications, big data, and the Internet of Things, combined with medical technology. It has realized the sharing of medical resources and the expansion of medical scope. Because of the combination of cloud computing technology, medical cloud improves the efficiency of medical institutions and facilitates residents to seek medical treatment. For example, the hospital's appointment registration, electronic medical records, medical insurance, etc. are all products of the combination of cloud computing and the medical field. The medical cloud also has the advantages of data security, information sharing, dynamic expansion, and overall layout.
  • the method for classifying EEG signals provided in the embodiments of the present application can be combined with the above-mentioned cloud platform for medical and health services, and the medical staff uploads the EEG signals of various patients and the motor imagery types corresponding to the EEG signals to the cloud platform for other Medical staff refer to it during diagnosis and treatment.
  • Cloud Gaming Also known as Gaming on Demand, it is an online gaming technology based on cloud computing technology.
  • Cloud gaming technology enables light-end devices (Thin Clients) with relatively limited graphics processing and data computing capabilities to run high-quality games.
  • the game is not run on the player's game terminal, but in the cloud server, and the cloud server renders the game scene into a video and audio stream, and transmits it to the player's game terminal through the network.
  • the player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input instructions and send them to the cloud server.
  • the method for classifying EEG signals provided by the embodiments of the present application can be combined with cloud games, and by identifying the type of motor imagery corresponding to the EEG signals, game characters in the cloud game are controlled to perform activities.
  • the EEG signal classification method provided in this embodiment of the present application can be applied to the following scenarios.
  • the EEG signal classification method provided by the embodiments of the present application can be applied to BCI systems of some medical devices or rehabilitation robots.
  • the EEG signal of a patient with hemiplegia or stroke is input into the BCI system of the exoskeleton robot, and the BCI system of the exoskeleton robot can classify the EEG signal of the patient according to the EEG signal classification method provided in the embodiment of the present application. Identify and determine the type of motor imagery to which the EEG signal belongs, so as to drive the exoskeleton robot to help patients carry out active rehabilitation activities.
  • the EEG signal of the user with mobility impairment is input into the BCI system of the electric wheelchair, and the BCI system of the electric wheelchair identifies the EEG signal of the user with mobility impairment according to the EEG signal classification method provided in the embodiment of the present application. , to determine the type of motor imagery to which the EEG signal belongs, so as to control the movement of the electric wheelchair according to the type of motor imagery, helping users with disabilities to travel freely.
  • the EEG signal classification method provided by the embodiment of the present application can be applied to the background server on the game application side.
  • a dynamic domain adaptive model is built in the background server.
  • the user's EEG signal determines the type of motor imagery to which the user's EEG signal belongs, so as to drive the virtual characters in the game to perform various activities in the virtual environment according to the type of motor imagery.
  • the user can control the virtual character in the virtual reality application through "ideas" by wearing a head-mounted device with an EEG signal acquisition function.
  • the above only takes two application scenarios as examples for description.
  • the method provided in this embodiment of the present application can also be applied to other scenarios where objects need to be controlled through EEG signals (for example, scenarios in which the cause of a patient is analyzed through EEG signals, etc.).
  • the application embodiments do not limit specific application scenarios.
  • the EEG signal classification method and the EEG signal classification model training method provided in the embodiments of the present application can be applied to computer equipment with data processing capabilities.
  • the EEG signal classification method and the EEG signal classification model training method provided in the embodiments of the present application may be applied to a personal computer, a workstation or a server, that is, a personal computer, workstation or server may be used. Realize the classification of EEG signals and train the EEG signal classification model.
  • the computer device for executing the method for classifying EEG signals is the first computer device
  • the computer device for executing the training method for the EEG signal classification model is the second computer device
  • the first computer device and the second computer device can be For the same device, it can also be a different device.
  • the trained EEG signal classification model it can be implemented as a part of the application program and installed in the terminal, so that when the terminal receives the EEG signal, it can identify the motor imagery type corresponding to the EEG signal;
  • the trained EEG signal classification model is set in the background server of the application program, so that the terminal with the application program can realize the function of identifying the motor imagery type corresponding to the EEG signal by means of the background server.
  • FIG. 1 shows a schematic diagram of a computer system provided by an exemplary embodiment of the present application.
  • the computer system 100 includes a terminal 110 and a server 120, wherein data communication is performed between the terminal 110 and the server 120 through a communication network, optionally, the communication network may be a wired network or a wireless network, and the communication network may be a local area network , at least one of a metropolitan area network and a wide area network.
  • the communication network may be a wired network or a wireless network
  • the communication network may be a local area network , at least one of a metropolitan area network and a wide area network.
  • An application program supporting the EEG signal recognition function is installed in the terminal 110, and the application program may be a virtual reality application (Virtual Reality, VR), an augmented reality application (Augmented Reality, AR), a medical program, a game application, etc., This embodiment of the present application does not limit this.
  • a virtual reality application Virtual Reality, VR
  • an augmented reality application Augmented Reality, AR
  • a medical program a game application, etc.
  • the terminal 110 may be a mobile terminal such as a smart phone, a smart watch, a tablet computer, a laptop portable computer, an intelligent robot, etc., or a terminal such as a desktop computer, a projection computer, or an exoskeleton robot or an electric wheelchair. , head-mounted displays and other terminals.
  • a mobile terminal such as a smart phone, a smart watch, a tablet computer, a laptop portable computer, an intelligent robot, etc.
  • a terminal such as a desktop computer, a projection computer, or an exoskeleton robot or an electric wheelchair.
  • head-mounted displays and other terminals does not limit the type of the terminal.
  • the server 120 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.
  • the server 120 is a background server of the application program in the terminal 110 .
  • a game application program is running in the terminal 110 , and the user controls the virtual character 111 in the game application program through the EEG signal 11 .
  • the user wears an EEG signal collection device that collects EEG signals, such as an EEG signal collection helmet.
  • the EEG signal acquisition device is an invasive or non-invasive signal acquisition device.
  • the EEG signal collection device is connected to the terminal 110.
  • the terminal records the user's EEG signal 11 and sends the user's EEG signal 11 to the server.
  • an EEG signal classification model 10 is constructed in the server 120, and the EEG signal classification model 10 is a trained machine learning model.
  • the EEG signal classification model 10 When the EEG signal classification model 10 receives the user's EEG signal 11, it extracts the signal feature 12 of the EEG signal, and combines the source domain EEG signal feature 13 of the source domain EEG signal used by the EEG signal classification model 10 during training to convert the
  • the signal feature 12 is aligned with the source domain feature 13, so that the signal feature 12 is close to the source domain feature 13 in the feature domain, and the aligned signal feature 14 is obtained, so that the EEG signal classification model 10 is migrated according to the domain adaptive learning method , so that the EEG signal classification model 10 is suitable for identifying the current EEG signal 11 .
  • the aligned signal features 14 are input into the classifier 15 of the EEG signal classification model 10 , and the motor imagery type 16 corresponding to the EEG signal 11 is output.
  • the motor imagery type 16 includes at least one of moving forward, moving backward, lying down, crouching, shooting, throwing, and driving a virtual vehicle.
  • the server 120 generates a control instruction according to the type of motor imagery, and sends the control instruction to the terminal 110 , and the terminal 110 controls the virtual character 111 to perform activities corresponding to the type of motor imagery according to the control instruction.
  • the user wears an EEG signal collection helmet
  • the user imagines and controls the virtual character 111 to run forward
  • the EEG signal collection helmet sends the collected EEG signal 11 to the terminal 110
  • the terminal 110 sends the EEG signal 11 Sent to the server 120
  • the EEG signal classification model 10 identifies the motor imagery type of the EEG signal 11 as running forward
  • the server 120 generates a forward running control instruction according to the motor imagery type of the forward running, and sends the control instruction
  • a picture of the virtual character 111 running forward is displayed on the terminal 110 , so that the user can control the virtual character 111 through the EEG signal 11 .
  • the EEG signal collection device is connected to the server 120 , the EEG signal classification model is built in the server 120 , and the EEG signal collection device directly inputs the collected EEG signal 11 into the server 120 .
  • the EEG signal classification model is constructed in the terminal 110 , and the collected EEG signal 11 is received and identified by the terminal 110 .
  • the above embodiment only takes the game application in the terminal as an example.
  • the above-mentioned EEG signal classification model can also be built in the driving system of the robot with rehabilitation function, and in the electric motor.
  • the types of motor imagery corresponding to the EEG signals are different, which are not limited in this embodiment of the present application.
  • FIG. 2 shows a flowchart of a method for classifying EEG signals provided by an exemplary embodiment of the present application. This embodiment is described by taking the method for the server 120 in the computer system 100 as shown in FIG. 1 as an example, and the method includes the following steps.
  • Step 201 acquiring an EEG signal.
  • EEG signal refers to the electrical signal generated by each neuron during brain activity, which is the overall reflection of the electrophysiological activity of brain nerve cells on the surface of the cerebral cortex or scalp.
  • the EEG signal can be obtained by an EEG signal acquisition device.
  • the acquisition methods of EEG signals include implantable and non-invasive. Potential information of a single neuron or local neurocortex; non-implantable is a non-invasive method of acquiring information on neural activity by attaching detection electrodes to the brain to collect EEG signals, schematically, the subject (i.e. The person whose EEG signal is collected) collects the EEG signal by wearing an EEG signal collection device, such as an EEG signal collection helmet, or attaching electrodes to the subject's head.
  • an EEG signal collection device such as an EEG signal collection helmet
  • EEG Electroencephalography
  • FMRI Functional Magnetic Resonance Imaging
  • NIRS Near Infra-Red
  • NIRS Near Infra-Red
  • EMG Magnetoencephalography
  • the EEG signal collection device there is a communication connection between the EEG signal collection device and the server, and the EEG signal collection device sends the collected EEG signals to the server; in other embodiments, the EEG signal collection device communicates with the user
  • the terminal used has a communication connection, the terminal has a communication connection with the server, the terminal records and stores the collected EEG signal, and the terminal sends the EEG signal to the server; in other embodiments, the EEG signal acquisition device and the user are connected.
  • the terminal used has a communication connection, the terminal records and stores the collected EEG signals, and the terminal identifies the EEG signals. This embodiment of the present application does not limit this.
  • Step 202 perform feature extraction on the EEG signal to obtain signal features corresponding to the EEG signal.
  • an EEG signal classification model is constructed in the server 120 , wherein the EEG signal classification model includes a feature extraction model 21 , and the signal feature 12 corresponding to the EEG signal 11 is extracted through the feature extraction model 21 .
  • the EEG signal needs to be preprocessed to remove artifacts and interference in the EEG signal and reduce the influence of noise on the EEG signal.
  • the signal features 12 that are difficult to be observed and detected in the input EEG signal 11 are extracted by the feature extraction model 21 , so that the EEG signal classification model can accurately classify the signal features 12 .
  • the signal features 12 of the EEG signal include: time domain features and spatial domain features. Temporal domain features refer to changes in EEG signals in time, and spatial domain features refer to changes in EEG signals in different brain regions.
  • the EEG signal features 12 also include frequency domain features.
  • the frequency domain feature refers to the change in frequency of the EEG signal.
  • the signal features of the EEG signal include time domain features and spatial domain features.
  • Step 203 Obtain the difference distribution ratio, which is used to represent the influence of different types of difference distribution on the distribution of the signal feature and the source domain feature on the feature domain, and the source domain feature is the feature corresponding to the source domain EEG signal.
  • the source domain feature is the signal feature corresponding to the source domain EEG signal
  • the source domain EEG signal is the sample EEG signal used in the training process of the EEG signal classification model.
  • the EEG signal is the EEG signal Test EEG signals used by the classification model during testing (or during use).
  • the distribution of EEG signals of different subjects in the same time domain and spatial domain is different, the same subject is in different states or emotions, in the same time domain or
  • the distribution in the spatial domain is different, that is, different EEG signals will have different distribution in the characteristic domain.
  • the physical state and emotional state of subject 1 and subject 2 are similar, subject 1 produces a peak in the EEG signal from the 4th second to the 6th second, and the subject 2 is in the 6th second.
  • the EEG signal peaks during the 8th second; for another example, subject 1 is a young person, and the EEG signal is always in an active state, and subject 2 is an elderly person, and the EEG signal is always in an inactive state.
  • the difference distribution ratio is used to characterize the influence of different types of difference distributions on the distribution of signal features and source domain features on the feature domain.
  • the embodiments of the present application take the difference distribution types including the conditional distribution difference and the marginal distribution difference as an example for description.
  • the difference distribution ratio includes the first distribution ratio corresponding to the marginal distribution difference and the second distribution ratio corresponding to the conditional distribution difference.
  • EEG signal 1 is the source domain EEG signal
  • EEG signal 2 is the test EEG signal input to the EEG signal classification model.
  • the electrical signal 2 forms a conditional distribution difference in the characteristic domain; if the EEG signal 1 and the EEG signal 2 have a large difference, then the EEG signal 1 and the EEG signal 2 form a marginal distribution difference in the characteristic domain.
  • Step 204 align the signal features with the source domain features according to the difference distribution ratio to obtain the aligned signal features.
  • Alignment is to make the distribution of two different EEG signals in the feature domain tend to be the same.
  • the EEG signal used in the training of the EEG signal classification model is the source domain EEG signal.
  • the EEG signal classification model can identify the same EEG signal as the source domain EEG signal, and the EEG signal classification model uses the test during testing. There is a difference between the EEG signal and the source domain signal. Inputting the test EEG signal into the EEG signal classification model may cause the EEG signal classification model to misclassify the test EEG signal.
  • the distribution of the signal features on the feature domain approaches the distribution of the source domain features on the feature domain, so that the EEG signal classification model can correctly classify the test EEG signals .
  • the EEG signal classification model includes a conditional discriminator 22 and an edge discriminator 23.
  • the conditional discriminator 22 is used to classify the signal feature 12 of the EEG signal and the source domain feature according to the second distribution ratio corresponding to the conditional distribution difference Aligning on the feature domain
  • the edge discriminator 23 is used to align the signal feature 12 of the EEG signal and the source domain feature on the feature domain according to the first distribution ratio corresponding to the edge distribution difference.
  • the conditional discriminator 22 and the edge discriminator 23 simultaneously align the signal feature 12 and the source domain feature on the feature domain according to their respective distribution ratios.
  • Step 205 Classify the aligned signal features to obtain a motor imagery type corresponding to the EEG signal.
  • the EEG signal classification model uses the transfer learning method to transfer the classification method of the source domain features to the signal features, so that the EEG signal classification model can EEG signals for accurate classification.
  • the EEG signal classification model includes a classifier 15, which is used to classify the signal features 12 of the EEG signal, and combines the conditional discriminator 22 and the edge discriminator 23 to comprehensively output the predicted probability of motor imagery types 18.
  • Motor imagery refers to the thinking process in which people use the brain to imagine normal movement of limbs, that is, imagine that their limbs are moving, but the limbs are not actually moving. According to different application scenarios, common types of motor imagery include at least one of bimanual movement, bipedal movement, and tongue movement (swallowing movement).
  • the motor imagery type is used to indicate the imagined limb movements of the EEG signal.
  • the method provided in this embodiment dynamically adjusts the distribution of the signal features corresponding to the EEG signal in the feature domain according to the difference distribution ratio, so that the signal features of the EEG signal are aligned with the features of the source domain, thereby ensuring that the brain
  • the signal features of the electrical signals are close to the source domain features in the feature distribution.
  • the EEG signal classification model trained based on the source domain features can transfer the classification method to the signal features corresponding to the EEG signals, which improves the recognition of the EEG signal classification model.
  • the accuracy of the motor imagery type corresponding to the EEG signal enables the EEG signal classification model to identify various types of EEG signals, which is universal.
  • FIG. 4 shows a flowchart of a method for classifying EEG signals provided by another exemplary embodiment of the present application. This embodiment is described by taking the method for the server 120 in the computer system 100 as shown in FIG. 1 as an example, and the method includes the following steps.
  • Step 401 acquiring an EEG signal.
  • EEG electrodes and 3 OMG electrodes were attached to the subject's head, the signal sampling rate was 250 hertz (Hz), and the acquisition included four types of motion Visualize types of EEG signals (left hand, right hand, feet and tongue).
  • the number of electrodes of the above-mentioned EEG electrodes and ophthalmic electrodes is a number for schematic illustration, and different embodiments may have different values.
  • the collected EEG signals are preprocessed, and the EEG signals in the time interval corresponding to motor imagery are intercepted from the EEG signals.
  • a signal is used as an example to illustrate. It should be noted that the first 2 seconds is the preparation stage for data acquisition. At this time, the acquisition device may collect some noise signals or non-EEG signals that interfere with the EEG signal. Therefore, from the second to the sixth second After cutting.
  • EEG signals in the 22 EEG channels are filtered using a Butterworth filter with a third-order band-pass filter (the band-pass range is 0 to 38 Hz), and some biological artifacts and biological artifacts in the EEG signals are removed again. Noise (eg eyeball artifact, muscle artifact, cardiac artifact, etc.). Standardize the filtered EEG signals. Illustratively, the exponentially weighted moving average method is used for the standardization, and the weight parameter is set to 0.999. ).
  • Step 402 Perform feature extraction on the EEG signal to obtain signal features corresponding to the EEG signal.
  • the EEG signal classification model includes a feature extraction model 21, and the feature extraction model 21 includes a Temporal Convolution 211, a Spatial Convolution 212, and a Batch Normalization layer. , BN) 213, Square 214, Average Pooling 215 and Dropout 216.
  • the EEG signal collected in step 401 has a dimension of 1000 dimensions (250 ⁇ 4), and the size of each EEG signal is 1000 ⁇ 22.
  • the temporal convolution layer 211 is called to perform feature extraction on the EEG signal to obtain the first signal feature corresponding to the EEG signal, and the temporal convolution layer 211 is used to perform a convolution operation on the input EEG signal along the time dimension.
  • the spatial convolution layer 212 is called to perform feature extraction on the first signal to obtain the second signal feature corresponding to the EEG signal, and the spatial convolution layer 212 performs a convolution operation on the input first signal feature along the spatial dimension.
  • the batch normalization layer 213 is called to perform feature extraction on the second signal feature, and the third signal feature corresponding to the EEG signal is obtained.
  • the square activation layer 214 is called to perform feature extraction on the third signal feature to obtain the fourth signal feature corresponding to the EEG signal.
  • the average pooling layer 215 is called to perform feature extraction on the fourth signal feature, and the fifth signal feature corresponding to the EEG signal is obtained.
  • the discarding layer 216 is called to perform feature extraction on the fifth signal feature to obtain the sixth signal feature corresponding to the EEG signal, and the sixth signal feature is determined as the feature corresponding to the EEG signal, that is, the signal feature finally output by the feature extraction model 21 .
  • Table 1 shows the parameters of each layered structure in the EEG signal classification model and the size of the output signal of each layered structure.
  • the discarding layer refers to that the activation value of a neuron stops working with a certain probability (a discarded node) when the EEG signal classification model propagates forward.
  • the drop rate refers to the ratio of dropped nodes to total nodes in the drop layer.
  • Step 403 Obtain the difference distribution ratio, which is used to represent the influence of different types of difference distribution on the distribution of the signal feature and the source domain feature on the feature domain, and the source domain feature is the feature corresponding to the source domain EEG signal.
  • the source domain feature is the signal feature corresponding to the source domain EEG signal
  • the source domain EEG signal is the sample EEG signal used in the training process of the EEG signal classification model.
  • the input EEG signal is a test EEG signal used by the EEG signal classification model in the testing process (or the use process).
  • the difference distribution ratio includes the first distribution ratio corresponding to the marginal distribution difference and the second distribution ratio corresponding to the conditional distribution difference.
  • Marginal distribution refers to the existence of random variables (X, Y), which can describe the distribution of events within a certain range by random variable X or random variable Y. This distribution is usually called the marginal distribution of random variable X (Marginal Distribution).
  • Conditional distribution refers to the existence of a set of random variables. When the values of some of the random variables are determined, the distribution of the rest of the random variables is a conditional distribution.
  • EEG signal 1 is the source domain EEG signal
  • EEG signal 2 is the test EEG signal input to the EEG signal classification model.
  • the electrical signal 2 forms a conditional distribution difference in the characteristic domain; if the EEG signal 1 and the EEG signal 2 have a large difference, then the EEG signal 1 and the EEG signal 2 form a marginal distribution difference in the characteristic domain.
  • step 403 can be implemented as the following steps.
  • Step 4031 obtain the first distribution distance and the second distribution distance between the signal feature and the source domain feature, the first distribution distance is used to characterize the edge distribution difference between the signal feature and the source domain feature, and the second distribution distance is used to characterize Conditional distribution differences between signal features and source domain features.
  • the first distribution distance is calculated according to the classification accuracy of the edge discriminator, and the second distribution distance is calculated according to the classification accuracy of the conditional discriminator. See the implementation of steps 4041 to 4043 .
  • Step 4032 Obtain the number of types of motor imagery types.
  • the embodiments of the present application are described with the number of types of motor imagery being four, that is, the types of motor imagery include left-hand movement, right-hand movement, foot movement, and tongue movement. It can be understood that the types of motor imagery are divided according to actual EEG signals, and are not limited to the above four types of motor imagery.
  • step 4031 and step 4032 can be executed simultaneously, step 4031 can be executed before step 4032 , and step 4032 can be executed before step 4031 .
  • Step 4033 Obtain the first distribution ratio corresponding to the edge distribution difference and the second distribution ratio corresponding to the conditional distribution difference according to the first distribution distance, the second distribution distance, and the number of types.
  • Steps 4031 to 4032 are used to calculate the first distribution ratio and the second distribution ratio, wherein the calculation formula of the first distribution ratio is the following formula.
  • the second distribution ratio is Indicates the importance of the conditional distribution to the distribution difference between the source domain feature and the signal feature in the feature domain.
  • the first distribution ratio can be obtained by using the second distribution ratio.
  • Step 404 Obtain the marginal distribution difference and the conditional distribution difference between the source domain feature and the signal feature on the feature domain.
  • step 404 may be replaced by the following steps.
  • Step 4041 obtain the first classification accuracy of the edge discriminator and the second classification accuracy of the conditional discriminator, the edge discriminator is used to determine the domain signal to which the EEG signal belongs, and the conditional discriminator is used to determine the different types of EEG signals belong to.
  • the domain signal includes at least one of the source domain EEG signal and the input EEG signal.
  • the edge discriminator 23 is used to output the predicted probability of the domain signal to which the EEG signal belongs, that is, to output the predicted probability that the EEG signal belongs to the source domain EEG signal and the predicted probability of the target domain EEG signal.
  • the source domain EEG signal is the sample EEG signal used by the EEG signal classification model in the training process
  • the target domain EEG signal is the test EEG signal of the EEG signal classification model during the testing process.
  • the predicted probability that the output EEG signal of the edge discriminator 23 belongs to the source domain EEG signal is 0.3
  • the predicted probability that the output EEG signal belongs to the target domain EEG signal is 0.7.
  • the prediction result is that the input EEG signal belongs to the target domain EEG signal, which is consistent with the type of the input EEG signal, and the prediction result output by the edge discriminator 23 is correct, thus recording the first classification accuracy of the edge discriminator 23 .
  • the conditional discriminator 22 is used to output the predicted probability of domain signals to which different types of EEG signals belong, and different types of EEG signals refer to EEG signals corresponding to different subjects, or EEG signals of different brain regions of the same subject Signal. For example, the predicted probability that the first type of EEG signal output by the conditional discriminator 22 belongs to the source domain signal is 0.19, and the predicted probability that the output first type of EEG signal belongs to the target domain signal is 0.82.
  • the prediction result is that the input EEG signal belongs to the target domain EEG signal, which is consistent with the type of the input EEG signal, and the prediction result output by the conditional discriminator 22 is correct, thus recording the second classification accuracy of the conditional discriminator 22 .
  • Step 4042 Obtain the first distribution distance between the signal feature and the source domain feature according to the first classification accuracy rate, and obtain the second distribution distance between the signal feature and the source domain feature according to the second classification accuracy rate.
  • the evaluation index adopts the distance evaluation index (A-distance) to measure the distribution difference between the source domain and the target domain.
  • the first distribution distance and the second distribution distance are calculated by the following formula.
  • Step 4043 Determine the first distribution distance as the marginal distribution difference, and determine the second distribution distance as the conditional distribution difference.
  • the first distribution distance Determined as the marginal distribution difference the second distribution distance Determined as conditional distribution differences.
  • Step 405 Reduce the edge distribution difference according to the first distribution scale, and reduce the conditional distribution difference according to the second distribution scale.
  • the conditional distribution difference and the edge distribution difference are simultaneously reduced, and the edge distribution difference and the conditional distribution difference are adjusted according to the first distribution ratio and the second distribution ratio respectively.
  • Step 406 according to the reduced edge distribution difference and the conditional distribution difference, obtain the signal feature after the reduced distribution difference.
  • conditional distribution and edge distribution of the signal feature on the feature domain are aligned with the source domain feature respectively, and the aligned signal feature is obtained, and the distribution difference between the signal feature and the source domain feature is reduced.
  • Step 407 Determine the signal feature after reducing the distribution difference as the aligned signal feature.
  • the distribution of the signal features on the feature domain is close to the distribution of the source domain features on the same feature domain.
  • Step 408 Classify the aligned signal features to obtain a motor imagery type corresponding to the EEG signal.
  • step 408 may be replaced by the following steps.
  • Step 4081 Invoke the classifier to process the aligned signal features to obtain the predicted probability of the motor phenomenon type corresponding to the EEG signal, and the motor imagery type includes at least one of the movement of the hands, the movement of the feet and the movement of the tongue.
  • the signal features corresponding to the EEG signals output by the feature extraction model 21 are respectively input to the classifier 15 , the conditional discriminator 22 and the edge discriminator 23 .
  • the classifier 15 includes a convolutional neural network (C-Conv or CNN) and a logistic regression layer (Softmax).
  • the convolutional layer performs convolution processing on the signal features, outputs an intermediate vector, and inputs the intermediate vector into the logistic regression. layer, which outputs the predicted probability of the motor imagery type corresponding to the EEG signal. For example, the classifier 15 outputs the predicted probability of left hand movement as 0.2, the predicted probability of right hand movement as 0.7, the predicted probability of double foot movement as 0, and the predicted probability of tongue movement as 0.1.
  • the feature vector output by the classifier 15 is input to the gradient reversal layer 217 (Gradient Reversal Layer, GRL), which is further calculated by the gradient reversal layer 217 to obtain another feature vector and input to the conditional discriminator 22 and edge discriminator 23 respectively.
  • GRL Gradient Reversal Layer
  • the gradient inversion layer 217 is used to multiply the error passed to this layer by a negative number (- ⁇ ), so that the training objectives of the network layers before and after the gradient inversion layer are opposite, so as to achieve a confrontational effect.
  • Step 4082 call the conditional discriminator to process the aligned signal features to obtain the predicted probability of the domain signal to which different types of EEG signals belong, and the domain signal includes at least one of the source domain EEG signal and the input EEG signal .
  • the conditional discriminator 22 includes a deep convolutional layer (Deep Convolutional Neural Network, D-Conv or DCNN) and a logistic regression layer.
  • the deep convolutional layer convolves the signal features, outputs an intermediate vector, and converts the intermediate
  • the vector is input to the logistic regression layer, which outputs the predicted probability that different types of EEG signals belong to the domain signal.
  • the types of EEG signals can be divided according to different types of subjects, or divided according to different regions of the brain, or divided according to the mood (or state) of the subject.
  • the predicted probability that the EEG signal of the first type output by the conditional discriminator 22 belongs to the source domain EEG signal is 0.7
  • the predicted probability that the output EEG signal of the first type belongs to the target domain EEG signal is 0.3
  • the EEG signal belongs to the source domain EEG signal.
  • the first type of EEG is divided according to the EEG signals produced by different regions of the brain.
  • Step 4083 calling the edge discriminator to process the aligned signal features to obtain the predicted probability of the domain signal to which the EEG signal belongs.
  • the edge discriminator 23 includes a depth convolution layer and a logistic regression layer.
  • the depth convolution layer performs convolution processing on the signal features, outputs an intermediate vector, inputs the intermediate vector into the logistic regression layer, and outputs the EEG signal to which it belongs.
  • the predicted probability of the domain signal is that the edge discriminator 23 only determines the domain signal to which the EEG signal belongs.
  • step 4081, step 4082 and step 4083 are executed simultaneously.
  • Step 4084 Obtain the motor imagery type corresponding to the EEG signal according to the predicted probability of the motor imagery type corresponding to the EEG signal, the predicted probability of the domain signal to which the EEG signal of different types belongs, and the predicted probability of the domain signal to which the EEG signal belongs. .
  • the conditional discriminator 22 and the edge discriminator 23 determine the respective distribution ratios.
  • the output results of the classifier 15, the conditional discriminator 22 and the edge discriminator 23 are integrated to obtain the motor imaginary type.
  • the method of this embodiment dynamically adjusts the distribution of the signal features corresponding to the EEG signals in the feature domain according to the difference distribution ratio, so that the signal features of the EEG signals are aligned with the source domain features, thereby ensuring that the EEG signal features are aligned with the source domain features.
  • the signal features of the signal are close to the source domain features in the feature distribution.
  • the EEG signal classification model trained based on the source domain features can transfer the classification method to the signal features corresponding to the EEG signal, which improves the recognition of the brain by the EEG signal classification model.
  • the accuracy of the motor imagery type corresponding to the electrical signal enables the EEG signal classification model to identify various types of EEG signals, which is universal.
  • the signal features of the input EEG signal classification model and the source domain features can be accurately approached to the source domain features. , so as to ensure that the EEG signal classification model can accurately obtain the aligned signal features.
  • the EEG signal classification model is made according to the first The distribution distance and the second distribution distance define marginal distribution differences and conditional distribution differences.
  • the edge distribution difference and the conditional distribution difference between the signal feature and the source domain feature are represented by the first distribution distance and the second distribution distance between the signal feature and the source domain feature, respectively, so that the edge distribution can be accurately calculated in combination with the number of types of motor imagery types.
  • the motor imagery classification model corresponding to the EEG signal is synthesized according to the predicted probability output by the three, so that the EEG signal classification model can identify different types of The EEG signal can improve the accuracy of the motor imagery type output by the EEG signal classification model.
  • the signal features of the EEG signal are time-invariant, so that the subsequent EEG signal classification model can output accurate motor imagery types.
  • FIG. 6 shows a flowchart of a training method for an EEG signal classification model provided by an exemplary embodiment of the present application. This embodiment is described by taking the method for the server 120 in the computer system 100 as shown in FIG. 1 as an example, and the method includes the following steps.
  • Step 601 acquiring the source domain EEG signal and the target domain EEG signal.
  • the source domain EEG signal is the sample EEG signal used by the EEG signal classification model in the training process
  • the target domain EEG signal is the test EEG signal used by the EEG signal classification model in the testing process.
  • the EEG signal classification model is trained using a public data set, and the public data set includes a training set and a test set.
  • the public competition data The BCI Competition IV Dataset 2a Motor Imagery Public Dataset, the dataset includes 9 subjects, in which the EEG data of each subject is passed through 22 EEG electrodes and 3 EEGs The electrodes recorded with a signal sampling rate of 205 Hz and included four types of motor imagery (left-hand movement, right-hand movement, foot movement, and tongue movement).
  • the entire training process of the EEG signal classification model is the same as the use process, including inputting the EEG signal 701 , preprocessing the EEG signal 702 , and introducing the distribution difference dynamic evaluation mechanism 703 into the EEG signal classification model 704 , and output the prediction result 705 .
  • Preprocessing 702 includes bandpass filtering 7021 and signal normalization 7022.
  • the EEG signal classification model is also named dynamic domain adaptation model.
  • step 401 For the preprocessing process of the EEG signal, refer to the implementation of step 401, and details are not repeated here.
  • Step 602 perform feature extraction on the source domain EEG signal and the target domain EEG signal to obtain the source domain feature corresponding to the source domain EEG signal and the target domain feature corresponding to the target domain EEG signal.
  • the source domain EEG signal 101 and the target domain EEG signal 102 are input into the feature extraction model 21 in the EEG signal classification model 704 , and the source domain corresponding to the source domain signal is output through each layer of the feature extraction model 21 .
  • the domain feature and the target domain feature corresponding to the target domain EEG signal Refer to the implementation of step 402, which is not repeated here.
  • step 603 the difference distribution ratio is obtained, and the difference distribution ratio is used to represent the influence of different types of difference distribution on the distribution of the source domain feature and the target domain feature on the feature domain.
  • the difference distribution ratio includes the first distribution ratio corresponding to the marginal distribution difference and the second distribution ratio corresponding to the conditional distribution difference.
  • the target domain features and source domain features have different distributions on the feature domain
  • a dynamic evaluation mechanism of distribution differences is introduced to the source domain features and target domain features. Differences in marginal distributions and differences in conditional distributions are evaluated. According to the distribution ratios corresponding to marginal distribution differences and conditional distribution differences, the distribution of differences with greater influence is dynamically adjusted.
  • Step 604 align the source domain feature and the target domain feature on the feature domain according to the difference distribution ratio to obtain the aligned target domain feature.
  • Step 604 may be replaced with the following steps.
  • Step 6041 Obtain the marginal distribution difference and the conditional distribution difference between the source domain feature and the target domain feature on the feature domain.
  • the edge discriminator is used to determine the domain signal to which the EEG signal belongs
  • the conditional discriminator is used to determine the domain signal to which the EEG signal belongs according to the type of motor imagery.
  • the domain signal includes at least one of the source domain EEG signal and the target domain EEG signal; the first distribution distance between the target domain feature and the source domain feature is obtained according to the first classification accuracy, and the second classification accuracy Obtain the second distribution distance between the target domain feature and the source domain feature; determine the first distribution distance as the marginal distribution difference, and determine the second distribution distance as the conditional distribution difference.
  • Step 6042 Reduce the marginal distribution difference according to the first distribution scale, and reduce the conditional distribution difference according to the second distribution scale.
  • Step 6043 according to the reduced edge distribution difference and the conditional distribution difference, obtain the target domain feature after the reduced distribution difference.
  • Step 6044 Determine the target domain feature after reducing the distribution difference as the aligned target domain feature.
  • steps 6041 to 6043 refer to the implementation of steps 404 to 407, which will not be repeated here.
  • Step 605 classify the aligned target domain features, train the EEG signal classification model according to the classification result, and obtain a trained EEG signal classification model.
  • Step 605 may be replaced with the following steps.
  • Step 6051 Invoke the classifier, the edge discriminator and the conditional discriminator in the EEG signal classification model to process the aligned target domain features respectively to obtain the predicted probability of the motor imagery type corresponding to the target domain EEG signal.
  • the classifier 15, the edge discriminator 23 and the conditional discriminator 22 simultaneously process the source domain features and the target domain features output by the feature extraction model according to the distribution difference dynamic evaluation mechanism.
  • the distributions are similar in the feature domain, so the EEG signal classification model can transfer the classification method of the source domain features to the target domain features, so that the EEG signal classification model can obtain the motor imagery type corresponding to the target domain EEG signals.
  • the feature vector output by the classifier 15 is input to the gradient reversal layer 217 (Gradient Reversal Layer, GRL), which is further calculated by the gradient reversal layer 217 to obtain another feature vector and input to the conditional discriminator 22 and edge discriminator 23 respectively.
  • GRL Gradient Reversal Layer
  • the gradient inversion layer 217 is used to multiply the error passed to this layer by a negative number (- ⁇ ), so that the training objectives of the network layers before and after the gradient inversion layer are opposite, so as to achieve a confrontational effect.
  • Step 6052 Calculate the result error of the EEG signal classification model according to the predicted probability and the real label of the motor imagery type corresponding to the EEG signal.
  • the result error includes the error corresponding to the classifier, the error corresponding to the conditional discriminator, and the error of the edge discriminator.
  • the embodiment of the present application takes the error of the error function calculation result as an example, and the calculation methods of the three types of errors are described below.
  • L c ( ⁇ f , ⁇ c ) represents the first loss function corresponding to the classifier, represents the expected value of the source domain feature, represents the source domain EEG signal, represents the predicted probability of the source domain EEG signal, Ds represents the source domain, L() represents the cross-entropy loss function, Represents the true label of the source domain EEG signal, and " ⁇ " represents the belonging relationship.
  • the above formula represents the expectation of calculating the classification loss of the source domain EEG signal by extracting the source domain EEG signal and the real label corresponding to the source domain EEG signal from the source domain EEG signal data set.
  • the second loss function corresponding to the conditional discriminator is calculated according to the source domain conditional feature map corresponding to the source domain feature output by the conditional discriminator and the target domain conditional feature map corresponding to the target domain feature.
  • the formula is the following formula.
  • D c represents the c-th type of conditional discriminator, that is, the source-domain EEG signals and target-domain EEG signals belonging to the c-th type are input into the conditional discriminator for classification
  • D s represents the source domain
  • D t represents the target domain
  • " ⁇ " represents the belonging relation
  • the calculation method of the conditional feature map is as follows.
  • the conditional feature map g is the following matrix.
  • the third loss function corresponding to the edge discriminator is calculated.
  • the formula is the following formula.
  • the total result error is the following error.
  • L c represents the first loss function of the classifier
  • ⁇ represents the parameters to be learned of the EEG signal classification model
  • L g represents the third loss function corresponding to the edge discriminator
  • L l represents the second loss corresponding to the conditional discriminator function
  • C represents the number of types of motor imagery types.
  • Step 6053 using the error back propagation algorithm to train the EEG signal classification model according to the result error to obtain the trained EEG signal classification model.
  • the Error Back Propagation Algorithm refers to backpropagating the output error layer by layer through the hidden layer to the input layer in some form, and apportioning the error to all units of each layer, so as to obtain the error of each layer.
  • the error signal is the basis for correcting the unit weights.
  • the machine learning model consists of two processes: forward propagation and error back propagation. During forward propagation, the input sample is passed in from the input layer, and after being processed layer by layer in each hidden layer, it is transmitted to the output layer. If the actual output of the output layer does not match the expected output, it is transferred to the error back-propagation stage.
  • the embodiment of the present application solves the parameters of the neural network model based on the gradient descent method of Stochastic Gradient Descent (SGD), and uses the Xavier initialization method to initialize the model parameters.
  • SGD Stochastic Gradient Descent
  • the EEG signals and corresponding labels of each subject are sent to the network for learning, and the model is optimized through error back propagation.
  • the method provided in this embodiment dynamically adjusts the distribution of the target domain features on the feature domain according to the difference distribution ratio, so that the target domain features are aligned with the source domain features, so as to train the brain based on the aligned target domain features.
  • the electrical signal classification model enables the EEG signal classification model to transfer the classification learning method to the target domain features, so that the trained EEG signal classification model can accurately output the motor imagery type corresponding to the EEG signal, and identify various types of brains. Electrical signals are universal.
  • the target domain features can be accurately transferred to the source domain.
  • the features are close to each other, so as to ensure that the EEG classification model can accurately obtain the aligned target domain features.
  • the EEG signal classification model is made according to the first and second distribution distances.
  • the first distribution distance and the second distribution distance define marginal distribution differences and conditional distribution differences.
  • the aligned target domain is processed by the classifier, edge discriminator and conditional discriminator, and the error between the real label of the motor phenomenon type corresponding to the EEG signal and the predicted probability output by the EEG signal classification model is used to classify the EEG signal.
  • the classification model is trained to ensure that the EEG signal classification model can accelerate the convergence and shorten the training time of the model.
  • the result error of the EEG signal classification model is accurately calculated, so that the EEG signal classification model is based on accurate
  • the result error is used for training, which improves the convergence speed of the model and shortens the training time of the model.
  • FIG. 9 shows a structural block diagram of an EEG signal classification apparatus provided by an exemplary embodiment of the present application, and the apparatus can be implemented as all or a part of a computer device.
  • the device includes the following parts:
  • a first acquisition module 910 configured to acquire EEG signals
  • the first feature extraction module 920 is configured to perform feature extraction on the EEG signal to obtain signal features corresponding to the EEG signal;
  • the first obtaining module 910 is used to obtain the difference distribution ratio, and the difference distribution ratio is used to characterize the influence of different types of difference distribution on the distribution of the signal feature and the source domain feature on the feature domain, and the source domain feature is the source domain brain.
  • the first processing module 930 is used for aligning the signal feature and the source domain feature according to the difference distribution ratio to obtain the aligned signal feature;
  • the classification module 940 is configured to classify the aligned signal features to obtain the motor imagery type corresponding to the EEG signal.
  • the difference distribution ratio includes a first distribution ratio corresponding to the marginal distribution difference and a second distribution ratio corresponding to the conditional distribution difference;
  • the first obtaining module 910 is used to obtain the edge distribution difference and the conditional distribution difference between the source domain feature and the signal feature on the feature domain;
  • the first processing module 930 is configured to reduce the edge distribution difference according to the first distribution ratio, and reduce the conditional distribution difference according to the second distribution ratio; obtain the signal characteristics after reducing the distribution difference according to the reduced edge distribution difference and the conditional distribution difference ; Determine the signal feature after reducing the distribution difference as the signal feature after alignment.
  • the first obtaining module 910 is configured to obtain the first classification accuracy of the edge discriminator and the second classification accuracy of the conditional discriminator, and the edge discriminator is used to determine the Domain signal, the conditional discriminator is used to determine the domain signal to which different types of EEG signals belong, and the domain signal includes at least one of the source domain EEG signal and the input EEG signal; the signal features and source are obtained according to the first classification accuracy The first distribution distance between the domain features, and the second distribution distance between the signal feature and the source domain feature is obtained according to the second classification accuracy; the first distribution distance is determined as the edge distribution difference, and the second distribution distance is determined is the conditional distribution difference.
  • the difference distribution ratio includes a first distribution ratio corresponding to the marginal distribution difference and a second distribution ratio corresponding to the conditional distribution difference;
  • the first obtaining module 910 is used to obtain the first distribution distance and the second distribution distance between the signal feature and the source domain feature, and the first distribution distance is used to characterize the edge distribution difference between the signal feature and the source domain feature,
  • the second distribution distance is used to characterize the conditional distribution difference between the signal feature and the source domain feature; the number of types of motor imagery types is obtained; the first distribution corresponding to the edge distribution difference is obtained according to the first distribution distance, the second distribution distance and the number of types The proportion of the second distribution corresponding to the difference between the proportion and the conditional distribution.
  • the classification module 940 includes a classifier 9401, a conditional discriminator 9402 and an edge discriminator 9403;
  • the classifier 9401 is used to process the aligned signal features, and obtain the predicted probability of the motor imagery type corresponding to the EEG signal; A sort of;
  • the conditional discriminator 9402 is used to process the aligned signal features to obtain predicted probabilities of domain signals to which different types of EEG signals belong.
  • the domain signals include at least one of the source domain EEG signals and the input EEG signals. A sort of;
  • the edge discriminator 9403 is used to process the aligned signal features to obtain the predicted probability of the domain signal to which the EEG signal belongs;
  • the classification module 940 is used to obtain the EEG signal according to the predicted probability of the motor imagery type corresponding to the EEG signal, the predicted probability of the domain signal to which the EEG signal of different types belongs, and the predicted probability of the domain signal to which the EEG signal belongs.
  • the corresponding type of motor imagery is used to obtain the EEG signal according to the predicted probability of the motor imagery type corresponding to the EEG signal, the predicted probability of the domain signal to which the EEG signal of different types belongs, and the predicted probability of the domain signal to which the EEG signal belongs.
  • the corresponding type of motor imagery is used to obtain the EEG signal according to the predicted probability of the motor imagery type corresponding to the EEG signal, the predicted probability of the domain signal to which the EEG signal of different types belongs.
  • the first feature extraction module 920 includes a temporal convolution layer 9201, a spatial convolution layer 9202, a batch normalization layer 9203, a square activation layer 9204, an average pooling layer 9205 and a dropout layer 9206;
  • the temporal convolution layer 9201 is used to perform feature extraction on the EEG signal to obtain the first signal feature corresponding to the EEG signal;
  • the spatial convolution layer 9202 is used to perform feature extraction on the first signal feature to obtain the second signal feature corresponding to the EEG signal;
  • the batch normalization layer 9203 is used to perform feature extraction on the second signal feature to obtain the third signal feature corresponding to the EEG signal;
  • the square activation layer 9204 is used to perform feature extraction on the third signal feature to obtain the fourth signal feature corresponding to the EEG signal;
  • the average pooling layer 9205 is used to perform feature extraction on the fourth signal feature to obtain the fifth signal feature corresponding to the EEG signal;
  • the discarding layer 9206 is configured to perform feature extraction on the fifth signal feature to obtain the sixth signal feature corresponding to the EEG signal, and determine the sixth signal feature as the signal feature corresponding to the EEG signal.
  • the device provided in this embodiment dynamically adjusts the distribution of the signal features corresponding to the EEG signals in the feature domain according to the difference distribution ratio, so that the signal features of the EEG signals are aligned with the source domain features, thereby ensuring that the brain
  • the signal features of the electrical signals are close to the source domain features in the feature distribution.
  • the EEG signal classification model trained based on the source domain features can transfer the classification method to the signal features corresponding to the EEG signals, which improves the recognition of the EEG signal classification model.
  • the accuracy of the motor imagery type corresponding to the EEG signal enables the EEG signal classification model to identify various types of EEG signals, which is universal.
  • the signal features of the input EEG signal classification model and the source domain features can be accurately approached to the source domain features. , so as to ensure that the EEG signal classification model can accurately obtain the aligned signal features.
  • the EEG signal classification model is made according to the first The distribution distance and the second distribution distance define marginal distribution differences and conditional distribution differences.
  • the edge distribution difference and the conditional distribution difference between the signal feature and the source domain feature are represented by the first distribution distance and the second distribution distance between the signal feature and the source domain feature, respectively, so that the edge distribution can be accurately calculated in combination with the number of types of motor imagery types.
  • the motor imagery classification model corresponding to the EEG signal is synthesized according to the predicted probability output by the three, so that the EEG signal classification model can identify different types of The EEG signal can improve the accuracy of the motor imagery type output by the EEG signal classification model.
  • the signal features of the EEG signal are time-invariant, so that the subsequent EEG signal classification model can output accurate motor imagery types.
  • the apparatus for classifying EEG signals provided by the above-mentioned embodiments is only illustrated by the division of the above-mentioned functional modules.
  • the internal structure is divided into different functional modules to complete all or part of the functions described above.
  • the EEG signal classification apparatus provided in the above embodiments and the EEG signal classification method embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.
  • FIG. 10 shows a structural block diagram of an apparatus for training an EEG signal classification model provided by an exemplary embodiment of the present application, and the apparatus may be implemented as all or a part of a computer device.
  • the device includes the following parts:
  • the second acquisition module 1010 is configured to acquire the source domain EEG signal and the target domain EEG signal;
  • the second feature extraction module 1020 is configured to perform feature extraction on the source domain EEG signal and the target domain EEG signal to obtain the source domain feature corresponding to the source domain EEG signal and the target domain feature corresponding to the target domain EEG signal;
  • the second obtaining module 1010 is used to obtain the difference distribution ratio, and the difference distribution ratio is used to represent the influence of different types of difference distributions on the distribution of the source domain feature and the target domain feature on the feature domain;
  • the second processing module 1030 is configured to align the source domain feature and the target domain feature on the feature domain according to the difference distribution ratio to obtain the aligned target domain feature;
  • the training module 1040 is configured to classify the aligned target domain features, train the EEG signal classification model according to the classification result, and obtain the trained EEG signal classification model.
  • the difference distribution ratio includes a first distribution ratio corresponding to the marginal distribution difference and a second distribution ratio corresponding to the conditional distribution difference;
  • the second obtaining module 1010 is used to obtain the marginal distribution difference and conditional distribution difference between the source domain feature and the target domain feature on the feature domain;
  • the second processing module 1030 is configured to reduce the edge distribution difference according to the first distribution ratio, and reduce the conditional distribution difference according to the second distribution ratio; obtain the signal characteristics after reducing the distribution difference according to the reduced edge distribution difference and the conditional distribution difference ; Determine the target domain feature after reducing the distribution difference as the aligned target domain feature.
  • the second obtaining module 1010 is configured to obtain the first classification accuracy of the edge discriminator and the second classification accuracy of the conditional discriminator, and the edge discriminator is used to determine the Domain signal, the conditional discriminator is used to determine the domain signal to which the EEG signal belongs according to the type of motor imagery, and the domain signal includes at least one of the source domain EEG signal and the target domain EEG signal; the target domain feature is obtained according to the first classification accuracy rate The first distribution distance from the source domain feature, and the second distribution distance between the target domain feature and the source domain feature is obtained according to the second classification accuracy; the first distribution distance is determined as the edge distribution difference, and the second distribution distance is Distances are determined as conditional distribution differences.
  • the apparatus includes an EEG signal classification model 1050;
  • the EEG signal classification model 1050 is used to call the classifier, the edge discriminator and the conditional discriminator to process the aligned target domain features respectively, to obtain the predicted probability of the motor imagery type corresponding to the target domain EEG signal;
  • the training module 1040 is used to calculate the result error of the EEG signal classification model according to the predicted probability and the real label of the motor imagery type corresponding to the EEG signal; according to the result error, use the error back propagation algorithm to train the EEG signal classification model, and obtain: A trained EEG classification model.
  • the training module 1040 is configured to calculate the first loss function corresponding to the classifier according to the predicted probability and the true label; according to the source domain conditional feature map corresponding to the source domain feature output by the conditional discriminator and The target domain conditional feature map corresponding to the target domain feature is used to calculate the second loss function corresponding to the conditional discriminator; the edge is calculated according to the source domain feature map corresponding to the source domain feature output by the edge discriminator and the target domain feature map corresponding to the target domain feature.
  • the third loss function corresponding to the discriminator; the result error of the electrical signal classification model is calculated according to the first loss function, the second loss function and the third loss function.
  • the device provided in this embodiment dynamically adjusts the distribution of the target domain features on the feature domain according to the difference distribution ratio, so that the target domain features are aligned with the source domain features, thereby training the brain based on the aligned target domain features.
  • the electrical signal classification model enables the EEG signal classification model to transfer the classification learning method to the target domain features, so that the trained EEG signal classification model can accurately output the motor imagery type corresponding to the EEG signal, and identify various types of brains. Electrical signals are universal.
  • the target domain features can be accurately transferred to the source domain.
  • the features are close to each other, so as to ensure that the EEG classification model can accurately obtain the aligned target domain features.
  • the EEG signal classification model is made according to the first and second distribution distances.
  • the first distribution distance and the second distribution distance define marginal distribution differences and conditional distribution differences.
  • the aligned target domain is processed by the classifier, edge discriminator and conditional discriminator, and the error between the real label of the motor phenomenon type corresponding to the EEG signal and the predicted probability output by the EEG signal classification model is used to classify the EEG signal.
  • the classification model is trained to ensure that the EEG signal classification model can accelerate the convergence and shorten the training time of the model.
  • the result error of the EEG signal classification model is accurately calculated, so that the EEG signal classification model is based on accurate
  • the result error is used for training, which improves the convergence speed of the model and shortens the training time of the model.
  • the training device for the EEG signal classification model provided by the above-mentioned embodiment is only illustrated by the division of the above-mentioned functional modules. That is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • the apparatus for training an EEG signal classification model provided by the above embodiments and the embodiment of the training method for an EEG signal classification model belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
  • FIG. 11 shows a schematic structural diagram of a server provided by an exemplary embodiment of the present application.
  • the server may be server 120 in computer system 100 as shown in FIG. 1 .
  • the server 1100 includes a central processing unit (CPU, Central Processing Unit) 1101, a system memory 1104 including a random access memory (RAM, Random Access Memory) 1102 and a read only memory (ROM, Read Only Memory) 1103, and a connection system memory 1104 And the system bus 1105 of the central processing unit 1101.
  • the server 1100 also includes a basic input/output system (I/O system, Input Output System) 1106, which facilitates the transfer of information between various devices within the computer, and a large database for storing the operating system 1113, application programs 1114, and other program modules 1115.
  • the basic input/output system 1106 includes a display 1108 for displaying information and input devices 1109 such as a mouse, keyboard, etc., for user input of information. Both the display 1108 and the input device 1109 are connected to the central processing unit 1101 through the input and output controller 1110 connected to the system bus 1105 .
  • the basic input/output system 1106 may also include an input output controller 1110 for receiving and processing input from a number of other devices such as a keyboard, mouse, or electronic stylus. Similarly, input output controller 1110 also provides output to a display screen, printer, or other type of output device.
  • Mass storage device 1107 is connected to central processing unit 1101 through a mass storage controller (not shown) connected to system bus 1105 .
  • Mass storage device 1107 and its associated computer-readable media provide non-volatile storage for server 1100 . That is, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read Only Memory (CD-ROM) drive.
  • a computer-readable medium such as a hard disk or a Compact Disc Read Only Memory (CD-ROM) drive.
  • Computer-readable media can include computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory, or other solid-state storage Its technology, CD-ROM, Digital Versatile Disc (DVD, Digital Versatile Disc) or Solid State Drives (SSD, Solid State Drives), other optical storage, tape cartridges, magnetic tape, disk storage or other magnetic storage devices.
  • EPROM Erasable Programmable Read Only Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • flash memory or other solid-state storage Its technology, CD-ROM, Digital Versatile Disc (DVD, Digital Versatile Disc) or Solid
  • the random access memory may include a resistive random access memory (ReRAM, Resistance Random Access Memory) and a dynamic random access memory (DRAM, Dynamic Random Access Memory).
  • ReRAM resistive random access memory
  • DRAM Dynamic Random Access Memory
  • the computer storage medium is not limited to the above-mentioned types.
  • the system memory 1104 and the mass storage device 1107 described above may be collectively referred to as memory.
  • the server 1100 may also be operated by connecting to a remote computer on the network through a network such as the Internet. That is, the server 1100 can be connected to the network 1112 through the network interface unit 1111 connected to the system bus 1105, or can also use the network interface unit 1111 to connect to other types of networks or remote computer systems (not shown).
  • the above-mentioned memory also includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.
  • a computer device in an optional embodiment, includes a processor and a memory, and the memory stores at least one instruction, at least one program, code set or instruction set, at least one instruction, at least one program , a code set or an instruction set is loaded and executed by the processor to implement at least one of the above-mentioned method for classifying an electroencephalographic signal and a method for training an electroencephalographic signal classification model.
  • a computer-readable storage medium stores at least one instruction, at least one piece of program, code set or instruction set, at least one instruction, at least one piece of program, code set or The set of instructions is loaded and executed by the processor to implement at least one of the method for classifying electroencephalogram signals and the method for training an electroencephalogram signal classification model as described above.
  • the computer-readable storage medium may include: Read Only Memory (ROM, Read Only Memory), Random Access Memory (RAM, Random Access Memory), Solid State Drive (SSD, Solid State Drives), or an optical disc.
  • the random access memory may include a resistive random access memory (ReRAM, Resistance Random Access Memory) and a dynamic random access memory (DRAM, Dynamic Random Access Memory).
  • ReRAM resistive random access memory
  • DRAM Dynamic Random Access Memory
  • Embodiments of the present application also provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method for classifying EEG signals and the EEG as described in the above aspects At least one of the training methods of the signal classification model.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Psychiatry (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Epidemiology (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Psychology (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Primary Health Care (AREA)
  • General Physics & Mathematics (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Rehabilitation Therapy (AREA)
  • Physical Education & Sports Medicine (AREA)

Abstract

一种脑电信号的分类方法、分类模型的训练方法、装置及介质，涉及迁移学习领域。方法包括：获取脑电信号（201）；对脑电信号进行特征提取，得到脑电信号对应的信号特征（202）；获取差异分布比例，差异分布比例用于表征不同类型的差异分布对信号特征和源域特征在特征域上的分布产生的影响，源域特征是源域脑电信号对应的特征（203）；根据差异分布比例将信号特征与源域特征进行对齐，得到对齐后的信号特征（204）；对对齐后的信号特征进行分类，得到脑电信号对应的运动想象类型（205）。通过差异分布比例动态地调整脑电信号对应的信号特征在特征域上的分布，使得脑电信号分类模型基于迁移学习的思想识别多种类型的脑电信号。

Description

脑电信号的分类方法、分类模型的训练方法、装置及介质
本申请要求于2020年08月26日提交的申请号为202010867943.X、发明名称为“脑电信号的分类方法、分类模型的训练方法、装置及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及迁移学习领域，特别涉及一种脑电信号的分类方法、分类模型的训练方法、装置及介质。
背景技术
脑电信号是大脑活动时各个神经元产生的电信号，通过脑电信号可识别出脑电信号对应的运动想象类型，即识别出大脑通过“意念”实现的肢体运动。
脑电信号可应用于医疗领域，如应用于结合医疗技术，使用“云计算”来创建的医疗健康服务云平台，医护人员通过脑电信号检查患者的病灶区域。通常利用脑机接口(Brain Computer Interface，BCI)将脑电信号的采集设备与计算机设备(外部设备)进行连接，通过外部设备(如计算机设备)识别脑机接口输出的脑电信号代表的运动想象类型，以实现大脑对物体的直接控制。由于不同的个体的脑电信号存在较大差异，需要针对每个个体的脑电信号单独训练一个脑电信号分类模型，从而保证相关模型能够正确识别出脑电信号代表的运动想象类型。
上述技术方案中，脑电信号分类模型只能针对模型训练时使用的脑电信号进行识别，使得脑电信号分类模型的使用场景较为局限，不具有普适性。
发明内容
本申请实施例提供了一种脑电信号的分类方法、分类模型的训练方法、装置及介质。所述技术方案为如下方案。
根据本申请的一方面，提供了一种脑电信号的分类方法，应用于计算机设备中，所述方法包括：
获取脑电信号；
对所述脑电信号进行特征提取，得到所述脑电信号对应的信号特征；
获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述信号特征和源域特征在特征域上的分布产生的影响，所述源域特征是源域脑电信号对应的特征；示意性的，源域脑电信号是脑电信号分类模型在训练的过程中使用的样本脑电信号；
根据所述差异分布比例将所述信号特征与所述源域特征进行对齐，得到对齐后的信号特征；
对所述对齐后的信号特征进行分类，得到所述脑电信号对应的运动想象类型。
根据本申请的另一方面，提供了一种脑电信号分类模型的训练方法，应用于计算机设备中，所述方法包括：
获取源域脑电信号和目标域脑电信号；
对所述源域脑电信号和所述目标域脑电信号进行特征提取，得到所述源域脑电信号对应的源域特征和所述目标域脑电信号对应的目标域特征；
获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述源域特征和所述目标域特征在特征域上的分布产生的影响；
根据所述差异分布比例将所述源域特征和所述目标域特征在所述特征域上对齐，得到对齐后的目标域特征；
对所述对齐后的目标域特征进行分类，根据分类结果对所述脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
根据本申请的另一方面，提供了一种脑电信号的分类装置，所述装置包括：
第一获取模块，用于获取脑电信号；
第一特征提取模块，用于对所述脑电信号进行特征提取，得到所述脑电信号对应的信号特征；
所述第一获取模块，用于获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述信号特征和源域特征在特征域上的分布产生的影响，所述源域特征是源域脑电信号对应的特征；示意性的，源域脑电信号是脑电信号分类模型在训练的过程中使用的样本脑电信号；
第一处理模块，用于根据所述差异分布比例将所述信号特征与所述源域特征进行对齐，得到对齐后的信号特征；
分类模块，用于对所述对齐后的信号特征进行分类，得到所述脑电信号对应的运动想象类型。
根据本申请的另一方面，提供了一种脑电信号分类模型的训练装置，所述装置包括：
第二获取模块，用于获取源域脑电信号和目标域脑电信号；
第二特征提取模块，用于对所述源域脑电信号和所述目标域脑电信号进行特征提取，得到所述源域脑电信号对应的源域特征和所述目标域脑电信号对应的目标域特征；
所述第二获取模块，用于获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述源域特征和所述目标域特征在特征域上的分布产生的影响；
第二处理模块，用于根据所述差异分布比例将所述源域特征和所述目标域特征在所述特征域上对齐，得到对齐后的目标域特征；
训练模块，用于对所述对齐后的目标域特征进行分类，根据分类结果对所述脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
根据本申请的另一方面，提供了一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如上述方面所述的脑电信号的分类方法和脑电信号分类模型的训练方法。
根据本申请的另一方面，提供了一种计算机可读存储介质，所述可读存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如上述方面所述的脑电信号的分类方法和脑电信号分类模型的训练方法。
根据本申请的另一方面，提供了一种计算机程序产品或计算机程序，所述计算机程序产品或计算机程序包括计算机指令，所述计算机指令存储在计算机可读存储介质中。计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令，所述处理器执行所述计算机指令，使得所述计算机设备执行如上方面所述的脑电信号的分类方法和脑电信号分类模型的训练方法。
本申请实施例提供的技术方案带来的有益效果至少包括如下效果。
通过根据差异分布比例动态地调整脑电信号对应的信号特征在特征域上的分布，使得脑电信号的信号特征与源域特征对齐，从而保证脑电信号的信号特征在特征分布上趋近于源域特征，基于源域特征训练的脑电信号分类模型能够将分类方法迁移至脑电信号对应的信号特征上，提高了脑电信号分类模型识别脑电信号对应的运动想象类型的准确率，使得脑电信号分类模型能够识别多种类型的脑电信号，具有普适性。
附图说明
图1是本申请一个示例性实施例提供的计算机系统的框架图；
图2是本申请一个示例性实施例提供的脑电信号的分类方法的流程图；
图3是本申请一个示例性实施例提供的脑电信号分类模型的结构示意图；
图4是本申请另一个示例性实施例提供的脑电信号的分类方法的流程图；
图5是本申请另一个示例性实施例提供的脑电信号分类模型的结构示意图；
图6是本申请一个示例性实施例提供的脑电信号分类模型的训练方法的流程图；
图7是本申请一个示例性实施例提供的脑电信号模型的训练方法的框架图；
图8是本申请另一个示例性实施例提供的脑电信号分类模型的结构示意图；
图9是本申请一个示例性实施例提供的脑电信号的分类装置的结构框图；
图10是本申请一个示例性实施例提供的脑电信号分类模型的训练装置的结构框图；
图11是本申请一个示例性实施例提供的服务器的装置结构示意图。
具体实施方式
首先，对本申请实施例涉及的名词进行介绍。
脑电信号(Electroencephalogram，EEG)：是指大脑活动时各个神经元产生的电信号，是脑神经细胞的电生理活动在大脑皮层或头皮表面的总体反映。通过侵入式或非侵入式脑机接口设备采集脑部的生物电压得到脑电信号，将脑电信号绘制成的曲线图为脑电图。通过分析脑电信号，检测识别不同脑部区域的激活效果来判断用户意图，进而实现人脑与外部设备之间的通信与控制。本申请实施例以通过非侵入式脑机接口设备采集脑部的生物电压为例进行说明。
运动想象(Motor Imagery，MI)：是指人凭借大脑想象肢体进行正常运动的思维过程，通过对运动过程反复进行想象或预演，实现通过“意念控制”肢体运动，是一种自发性脑电信号。常见的运动想象部位为：左手、右手、双脚和舌头。
脑机接口(Brain Computer Interface，BCI)：是指人或动物的脑部与外部设备交换信息时建立的连接通路。比如，电动轮椅结合的BCI系统，通过采集用户的脑电信号来控制电动轮椅的移动方向，帮助肢体不便的用户自由活动。
域自适应学习(Domain Adaptation Learning，DAL)：是指一种能够解决训练样本和测试样本概率分布不一致的机器学习理论。在传统的机器学习算法中，通常假设训练样本(源域)和测试样本(目标域)来自同一概率分布，构建相应的模型和判别准则对待测试的样本进行预测。但在一些场景下，测试样本与训练样本不满足概率分布相同这一条件，通过域自适应学习可解决源域和目标域概率分布不一致的情况。在域自适应学习中，源域(Source Domain)是指模型已有的知识或已学习到的知识；目标域(Target Domain)是指模型待学习的知识。
医疗云(Medical Cloud)：是指在云计算、移动技术、多媒体、4G通信、大数据、以及物联网等新技术基础上，结合医疗技术，使用“云计算”来创建医疗健康服务云平台，实现了医疗资源的共享和医疗范围的扩大。因为云计算技术的运用于结合，医疗云提高医疗机构的效率，方便居民就医。像现在医院的预约挂号、电子病历、医保等都是云计算与医疗领域结合的产物，医疗云还具有数据安全、信息共享、动态扩展、布局全局的优势。本申请实施例中提供的脑电信号的分类方法可与上述医疗健康服务云平台结合，医护人员将多种患者的脑电信号以及脑电信号对应的运动想象类型上传至该云平台，供其他医护人员在诊疗时参考。
云游戏(Cloud Gaming)：又可称为游戏点播(Gaming on Demand)，是一种以云计算技术为基础的在线游戏技术。云游戏技术使图形处理与数据运算能力相对有限的轻端设备(Thin Client)能运行高品质游戏。在云游戏场景下，游戏并不在玩家游戏终端，而是在云端服务器中运行，并由云端服务器将游戏场景渲染为视频音频流，通过网络传输给玩家游戏终端。玩家游戏终端无需拥有强大的图形运算与数据处理能力，仅需拥有基本的流媒体播放能力与获取玩家输入指令并发送给云端服务器的能力即可。本申请实施例提供的脑电信号的分类方法可与云游戏结合，通过识别脑电信号对应的运动想象类型，控制云游戏中的游戏角色进行活动。
本申请实施例提供的脑电信号的分类方法可以应用于如下场景。
一、智能医疗。
在该应用场景下，采用本申请实施例提供的脑电信号的分类方法可应用于一些医疗器械或复健机器人的BCI系统。
比如，将患有偏瘫、脑卒中的患者的脑电信号输入至外骨骼机器人的BCI系统中，外骨 骼机器人的BCI系统根据本申请实施例提供的脑电信号的分类方法对患者的脑电信号进行识别，确定脑电信号所属的运动想象类型，从而驱动外骨骼机器人帮助患者进行主动式的复健活动。
又如，将行动不便的用户的脑电信号输入至电动轮椅的BCI系统中，电动轮椅的BCI系统根据本申请实施例提供的脑电信号的分类方法对行动不便的用户的脑电信号进行识别，确定脑电信号所属的运动想象类型，从而根据运动想象类型控制电动轮椅移动，帮助行动不便的用户自由出行。
二、游戏应用中控制虚拟角色。
在该应用场景下，采用本申请实施例提供的脑电信号的分类方法可应用于游戏应用侧的后台服务器中。后台服务器中构建有动态域自适应模型，通过获取用户的脑电信号，判断用户的脑电信号所属的运动想象类型，从而根据运动想象类型驱动游戏中的虚拟角色在虚拟环境中进行各类活动。比如，在虚拟现实应用中，用户通过佩戴具有脑电信号采集功能的头戴设备，使得用户能够通过“意念”控制虚拟现实应用中的虚拟角色。
上述仅以两种应用场景为例进行说明，本申请实施例提供的方法还可以应用于其他需要通过脑电信号控制物体的场景(比如，通过脑电信号分析患者病因的场景等等)，本申请实施例并不对具体应用场景进行限定。
本申请实施例提供的脑电信号的分类方法和脑电信号分类模型的训练方法可以应用于具有数据处理能力的计算机设备中。在一种可能的实施方式中，本申请实施例提供的脑电信号的分类方法和脑电信号分类模型的训练方法可以应用于个人计算机、工作站或服务器中，即可以通过个人计算机、工作站或服务器实现对脑电信号进行分类和训练脑电信号分类模型。假设用于执行脑电信号的分类方法的计算机设备为第一计算机设备，用于执行脑电信号分类模型的训练方法的计算机设备为第二计算机设备，则第一计算机设备和第二计算机设备可以为同一个设备，也可以是不同的设备。
而对于训练后的脑电信号分类模型，其可以实现成为应用程序的一部分，并被安装在终端中，使终端在接收到脑电信号时，识别脑电信号对应的运动想象类型；或者，该训练后的脑电信号分类模型设置在应用程序的后台服务器中，以便安装有应用程序的终端借助后台服务器实现识别脑电信号对应的运动想象类型的功能。
图1示出了本申请一个示例性实施例提供的计算机系统的示意图。该计算机系统100包括终端110和服务器120，其中，终端110与服务器120之间通过通信网络进行数据通信，可选地，通信网络可以是有线网络也可以是无线网络，且该通信网络可以是局域网、城域网以及广域网中的至少一种。
终端110中安装有支持脑电信号识别功能的应用程序，该应用程序可以是虚拟现实应用程序(Virtual Reality，VR)、增强现实应用程序(Augmented Reality，AR)、医疗程序、游戏应用程序等，本申请实施例对此不作限定。
可选的，终端110可以是智能手机、智能手表、平板电脑、膝上便携式笔记本电脑、智能机器人等移动终端，也可以是台式电脑、投影式电脑等终端，还可以是外骨骼机器人、电动轮椅、头戴式显示器等终端。本申请实施例对终端的类型不做限定。
服务器120可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。在一种可能的实施方式中，服务器120是终端110中应用程序的后台服务器。
如图1所示，在本实施例中，终端110中运行有游戏应用程序，用户通过脑电信号11控制游戏应用程序中的虚拟角色111。示意性的，用户佩戴有采集脑电信号的脑电信号采集设备，如脑电信号采集头盔。该脑电信号采集设备是侵入式或非侵入式的信号采集设备。示意性的，脑电信号采集设备与终端110连接，当脑电信号采集设备采集到用户的脑电信号11时， 终端记录用户的脑电信号11，并将用户的脑电信号11发送至服务器120中，服务器120中构建有脑电信号分类模型10，该脑电信号分类模型10是经过训练的机器学习模型。
脑电信号分类模型10在接收到用户的脑电信号11时，提取脑电信号的信号特征12，结合脑电信号分类模型10在训练时使用的源域脑电信号的源域特征13，将信号特征12与源域特征13进行对齐，使得信号特征12与源域特征13在特征域上接近，得到对齐后的信号特征14，从而使得脑电信号分类模型10根据域自适应学习方法进行迁移，使得脑电信号分类模型10适用于识别当前的脑电信号11。将对齐后的信号特征14输入至脑电信号分类模型10的分类器15中，输出脑电信号11对应的运动想象类型16。示意性的，运动想象类型16包括向前移动、向后移动、趴下、蹲下、射击、投掷和驾驶虚拟载具中的至少一种。
服务器120根据运动想象类型生成控制指令，并将控制指令发送至终端110，终端110根据控制指令控制虚拟角色111进行与运动想象类型对应的活动。
在一个示例中，用户佩戴有脑电信号采集头盔，用户想象控制虚拟角色111向前奔跑，脑电信号采集头盔将采集到的脑电信号11发送至终端110中，终端110将脑电信号11发送至服务器120中，脑电信号分类模型10识别脑电信号11的运动想象类型为向前奔跑，服务器120根据向前奔跑的运动想象类型生成向前奔跑的控制指令，并将该控制指令发送至终端110，终端110上显示有虚拟角色111向前奔跑的画面，从而实现用户通过脑电信号11控制虚拟角色111。
在一些实施例中，脑电信号采集设备与服务器120相连，脑电信号分类模型构建在服务器120中，脑电信号采集设备将采集到的脑电信号11直接输入至服务器120中。在另一些实施例中，脑电信号分类模型构建在终端110中，由终端110接收采集到的脑电信号11以及对脑电信号11进行识别。
可以理解的是，上述实施例仅以终端中的游戏应用程序为例，在实际使用过程中，上述脑电信号分类模型还可以构建在具有复健功能的机器人的驱动系统中，以及构建在电动轮椅的驱动系统中，针对不同的应用场景，脑电信号对应的运动想象类型不同，本申请实施例对此不加以限定。
为了方便表述，下述各个实施例以脑电信号的分类方法和脑电信号分类模型的训练方法由服务器执行为例进行说明，但这两个方法也可以由终端执行，也即这两个方法可以由计算机设备执行。
图2示出了本申请一个示例性实施例提供的脑电信号的分类方法的流程图。本实施例以该方法用于如图1所示的计算机系统100中的服务器120为例进行说明，该方法包括如下步骤。
步骤201，获取脑电信号。
脑电信号是指大脑活动时各个神经元产生的电信号，是脑神经细胞的电生理活动在大脑皮层或头皮表面的总体反映。脑电信号可通过脑电信号采集设备获得。
按照脑电信号采集设备进行分类，脑电信号的获取方式包括植入式(Invasive)和非植入式(Non-invasive)，植入式需要通过手术将微电极植入人体的神经皮层来收集单个神经元或局部神经皮层的电位信息；而非植入式是一种无创的神经活动信息获取方式，通过在大脑上贴附检测电极来采集脑电信号，示意性的，受试者(即被采集脑电信号的人)通过佩戴脑电信号采集设备采集脑电信号，比如，脑电信号采集头盔，或者，在受试者的头部贴附电极。
根据不同的脑电信号采集方式，生成不同类型的脑电信号图，比如，脑电图(Electroencephalography，EEG)、功能磁共振成像(Function Magnetic Resonance Imaging，FMRI)、近红外光谱(Near Infra-Red Spectroscopy，NIRS)和脑磁图(Magneto Encephalography，EMG)中的至少一种。
在一些实施例中，脑电信号采集设备与服务器之间存在通讯连接，脑电信号采集设备将采集到的脑电信号发送至服务器中；在另一些实施例中，脑电信号采集设备与用户使用的终 端存在通讯连接，终端与服务器存在通讯连接，终端记录并存储采集到的脑电信号，由终端将脑电信号发送至服务器中；在另一些实施例中，脑电信号采集设备与用户使用的终端存在通讯连接，终端记录并存储采集到的脑电信号，终端对脑电信号进行识别。本申请实施例对此不加以限定。
步骤202，对脑电信号进行特征提取，得到脑电信号对应的信号特征。
如图3所示，在服务器120中构建有脑电信号分类模型，其中，脑电信号分类模型包括特征提取模型21，通过特征提取模型21提取脑电信号11对应的信号特征12。在一些实施例中，在脑电信号11输入至特征提取模型前，还需要对脑电信号进行预处理，去除脑电信号中的伪迹和干扰，降低噪声对脑电信号的影响。
通过特征提取模型21将输入的脑电信号11中不易被观察和检测的信号特征12提取出来，使得脑电信号分类模型能够对信号特征12进行准确分类。脑电信号的信号特征12包括：时域特征和空间域特征。时域特征是指脑电信号在时间上的变化，空间域特征是指不同脑部区域中脑电信号的变化。在一些实施例中，脑电信号特征12还包括频域特征。频域特征是指脑电信号在频率上的变化。本申请实施例以脑电信号的信号特征包括时域特征和空间域特征进行说明。
步骤203，获取差异分布比例，差异分布比例用于表征不同类型的差异分布对信号特征和源域特征在特征域上的分布产生的影响，源域特征是源域脑电信号对应的特征。
源域特征是源域脑电信号对应的信号特征，源域脑电信号是脑电信号分类模型在训练的过程中使用的样本脑电信号，在本实施例中，脑电信号是脑电信号分类模型在测试过程中(或使用过程中)使用的测试脑电信号。
由于不同的受试者之间存在较大的差异，不同受试者的脑电信号在同一时域和空间域上的分布不同，同一受试者在不同状态或情绪下，在同一时域或空间域上分布不同，即不同的脑电信号在特征域上会产生分布差异。比如，受试者1和受试者2的身体状态和情绪状态等方面接近，受试者1在第4秒至第6秒的时间内脑电信号产生峰值，受试者2在第6秒至第8秒的时间内脑电信号产生峰值；又比如，受试者1是年轻人，脑电信号始终处于活跃状态，受试者2是老年人，脑电信号始终处于非活跃状态。
差异分布比例用于表征不同类型的差异分布对信号特征和源域特征在特征域上的分布产生的影响。本申请实施例以差异分布类型包括条件分布差异和边缘分布差异为例进行说明。差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。比如，脑电信号1为源域脑电信号，脑电信号2为输入脑电信号分类模型的测试脑电信号，若脑电信号1和脑电信号2较为接近，则脑电信号1和脑电信号2在特征域上形成条件分布差异；若脑电信号1和脑电信号2相差较大，则脑电信号1和脑电信号2在特征域上形成边缘分布差异。
步骤204，根据差异分布比例将信号特征与源域特征进行对齐，得到对齐后的信号特征。
对齐是令两种存在差异的脑电信号在特征域上的分布趋于相同。脑电信号分类模型训练时使用的脑电信号为源域脑电信号，脑电信号分类模型能够识别与源域脑电信号相同的脑电信号，而脑电信号分类模型在测试时使用的测试脑电信号与源域信号存在差异，将测试脑电信号输入至脑电信号分类模型中，可能会产生脑电信号分类模型对测试脑电信号进行误分类的情况。当信号特征和源域特征在特征域上对齐时，信号特征在特征域上的分布趋近于源域特征在特征域上的分布，从而使得脑电信号分类模型对测试脑电信号进行正确分类。
如图3所示，脑电信号分类模型包括条件判别器22和边缘判别器23，条件判别器22用于根据条件分布差异对应的第二分布比例将脑电信号的信号特征12和源域特征在特征域上对齐，边缘判别器23用于根据边缘分布差异对应的第一分布比例将脑电信号的信号特征12和源域特征在特征域上对齐。可选地，条件判别器22和边缘判别器23根据各自的分布比例同时将信号特征12和源域特征在特征域上对齐。
步骤205，对对齐后的信号特征进行分类，得到脑电信号对应的运动想象类型。
由于脑电信号的信号特征与源域特征在特征域上的分布相近，脑电信号分类模型利用迁移学习的方式将对源域特征的分类方法迁移至信号特征，使得脑电信号分类模型能够对脑电信号进行准确分类。
如图3所示，脑电信号分类模型包括分类器15，分类器15用于对脑电信号的信号特征12进行分类，结合条件判别器22和边缘判别器23综合输出运动想象类型的预测概率18。运动想象是指人凭借大脑想象肢体进行正常运动的思维过程，即想象自己的肢体正在运动，但肢体实际上并未运动。根据不同的应用场景，常见的运动想象类型包括双手运动、双脚运动和舌头运动(吞咽运动)中的至少一种。
示意性的，运动想象类型用于指示脑电信号想象的肢体运动。
综上所述，本实施例提供的方法，通过根据差异分布比例动态地调整脑电信号对应的信号特征在特征域上的分布，使得脑电信号的信号特征与源域特征对齐，从而保证脑电信号的信号特征在特征分布上趋近于源域特征，基于源域特征训练的脑电信号分类模型能够将分类方法迁移至脑电信号对应的信号特征上，提高了脑电信号分类模型识别脑电信号对应的运动想象类型的准确率，使得脑电信号分类模型能够识别多种类型的脑电信号，具有普适性。
图4示出了本申请另一个示例性实施例提供的脑电信号的分类方法的流程图。本实施例以该方法用于如图1所示的计算机系统100中的服务器120为例进行说明，该方法包括如下步骤。
步骤401，获取脑电信号。
示意性的，在受试者的头部贴附22个脑电电极和3个眼电电极(用于记录眼部活动的电极)，信号采样率为250赫兹(Hz)，采集包括四类运动想象类型的脑电信号(左手、右手、双脚和舌头)。上述脑电电极和眼电电极的电极数量为示意性说明的数量，不同实施例可以有不同的取值。
对采集到的脑电信号进行预处理，从脑电信号中截取运动想象对应的时间区间的脑电信号，示意性的，本申请实施例以截取第2秒至第6秒之间的脑电信号为例进行说明。需要说明的是，前2秒的时间是采集数据的准备阶段，此时采集设备可能会采集到一些干扰脑电信号的噪声信号，或者非脑电信号，因此，从第2秒至第6秒后进行截取。
然后从截取后的脑电信号中去除3个眼电通道，保留22个脑电通道。使用带通滤波为三阶的巴特沃斯滤波器对22个脑电通道中的脑电信号进行滤波处理，(带通范围是0至38Hz)，再次去除一些脑电信号中的生物伪迹和噪声(如眼球伪迹、肌肉伪迹、心脏伪迹等)。对滤波后的脑电信号进行标准化处理，示意性的，标准化处理选用指数加权移动平均法，权重参数设置为0.999，或者，标准化处理选用均值方差标准化操作和共空间模式算法(Common Spatial Patterns，CSP)。
步骤402，对脑电信号进行特征提取，得到脑电信号对应的信号特征。
如图5所示，脑电信号分类模型包括特征提取模型21，特征提取模型21包括时间卷积层(Temporal Convolution)211、空间卷积层(Spatial Convolution)212、批量归一化层(Batch Normalization，BN)213、平方激活层(Square)214、平均池化层(Average Pooling)215和丢弃层(Dropout)216。
示例性的，由步骤401采集到的脑电信号是维度为1000维(250×4)，每个脑电信号的尺寸为1000×22。
示例性的，调用时间卷积层211对脑电信号进行特征提取，得到脑电信号对应的第一信号特征，时间卷积层211用于沿着时间维度对输入的脑电信号进行卷积操作。调用空间卷积层212对第一信号进行特征提取，得到脑电信号对应的第二信号特征，空间卷积层212沿着空间维度对输入的第一信号特征进行卷积操作。调用批量归一化层213对第二信号特征进行特征提取，得到脑电信号对应的第三信号特征。调用平方激活层214对第三信号特征进行特征提取，得到脑电信号对应的第四信号特征。调用平均池化层215对第四信号特征进行特征 提取，得到脑电信号对应的第五信号特征。调用丢弃层216对第五信号特征进行特征提取，得到脑电信号对应的第六信号特征，将第六信号特征确定为脑电信号对应的特征，即特征提取模型21最终输出的信号特征。
以表一表示脑电信号分类模型中各层状结构的参数以及各层状结构输出信号的尺寸。
表一
Figure PCTCN2021106614-appb-000001
其中，丢弃层是指脑电信号分类模型在前向传播时，令某个神经元的激活值以一定的概率停止工作(被丢弃的节点)。丢弃率是指在丢弃层中被丢弃的节点与总节点的比值。
步骤403，获取差异分布比例，差异分布比例用于表征不同类型的差异分布对信号特征和源域特征在特征域上的分布产生的影响，源域特征是源域脑电信号对应的特征。
源域特征是源域脑电信号对应的信号特征，源域脑电信号是脑电信号分类模型在训练的过程中使用的样本脑电信号。在本实施例中，输入的脑电信号是脑电信号分类模型在测试过程中(或使用过程中)使用的测试脑电信号。
差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。边缘分布是指存在随机变量(X，Y)，能够用随机变量X或随机变量Y描述事件在某个范围内的分布，这个分布通常称为随机变量X的边缘分布(Marginal Distribution)。条件分布是指存在一组随机变量，当其中某些随机变量的取值确定时，其余随机变量的分布即为条件分布(Conditional Distribution)。比如，脑电信号1为源域脑电信号，脑电信号2为输入脑电信号分类模型的测试脑电信号，若脑电信号1和脑电信号2较为接近，则脑电信号1和脑电信号2在特征域上形成条件分布差异；若脑电信号1和脑电信号2相差较大，则脑电信号1和脑电信号2在特征域上形成边缘分布差异。
示例性的，步骤403可实现成为如下步骤。
步骤4031，获取信号特征与源域特征之间的第一分布距离和第二分布距离，第一分布距离用于表征信号特征与源域特征之间的边缘分布差异，第二分布距离用于表征信号特征与源域特征之间的条件分布差异。
第一分布距离是根据边缘判别器的分类准确率计算得到的，第二分布距离是根据条件判别器的分类准确率计算得到的。参见步骤4041至步骤4043的实施方式。
步骤4032，获取运动想象类型的类型数量。
本申请实施例以运动想象类型的类型数量为四进行说明，即运动想象类型包括左手运动、右手运动、双脚运动和舌头运动。可以理解的是，运动想象类型是根据实际脑电信号进行划分的，不局限于上述四种运动想象类型。
可以理解的是，步骤4031和步骤4032可同时执行，步骤4031可先于步骤4032之前执行，步骤4032可先于步骤4031之前执行。
步骤4033，根据第一分布距离、第二分布距离和类型数量得到边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。
步骤4031至步骤4032用于计算第一分布比例和第二分布比例，其中第一分布比例的计算公式为如下公式。
Figure PCTCN2021106614-appb-000002
其中，
Figure PCTCN2021106614-appb-000003
表示源域特征与信号特征之间的第一分布距离，
Figure PCTCN2021106614-appb-000004
表示源域特征与信号特征之间的第二分布距离，C表示运动想象类型的类型数，
Figure PCTCN2021106614-appb-000005
表示边缘分布对源域特征和信号特征在特征域上产生分布差异时的重要性。
由此可知，第二分布比例为
Figure PCTCN2021106614-appb-000006
表示条件分布对源域特征和信号特征在特征域上产生分布差异时的重要性。
可以理解的是，可通过计算条件分布差异对应的第二分布比例，通过第二分布比例得到第一分布比例。
步骤404，获取源域特征和信号特征在特征域上的边缘分布差异和条件分布差异。
示意性的，步骤404可替换为如下步骤。
步骤4041，获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，边缘判别器用于确定脑电信号所属的域信号，条件判别器用于确定不同类型的脑电信号所属的域信号，域信号包括源域脑电信号和输入的脑电信号中的至少一种。
边缘判别器23用于输出脑电信号所属的域信号的预测概率，即输出脑电信号属于源域脑电信号的预测概率和目标域脑电信号的预测概率。源域脑电信号是脑电信号分类模型在训练过程中使用的样本脑电信号，目标域脑电信号是脑电信号分类模型在测试过程中的测试脑电信号。比如，边缘判别器23输出脑电信号属于源域脑电信号的预测概率为0.3，输出脑电信号属于目标域脑电信号的预测概率为0.7。预测结果为输入的脑电信号属于目标域脑电信号，与输入脑电信号的类型相符，边缘判别器23输出的预测结果是正确的，从而记录边缘判别器23的第一分类准确率。
条件判别器22用于输出不同类型的脑电信号所属的域信号的预测概率，不同类型的脑电信号是指不同受试者对应的脑电信号，或同一受试者大脑不同区域的脑电信号。比如，条件判别器22输出第一类型的脑电信号属于源域信号的预测概率为0.19，输出第一类型的脑电信号属于目标域信号的预测概率为0.82。预测结果为输入的脑电信号属于目标域脑电信号，与输入脑电信号的类型相符，条件判别器22输出的预测结果是正确的，从而记录条件判别器22的第二分类准确率。
步骤4042，根据第一分类准确率获取信号特征与源域特征之间的第一分布距离，以及根据第二分类准确率获取信号特征与源域特征之间的第二分布距离。
为了评估边缘分布和条件分布对跨域差异性的影响，引入分布差异动态评估机制对两种分布的差异进行评价，评价指标采用距离评估指标(A-distance)衡量源域和目标域的分布差异，通过如下公式计算第一分布距离和第二分布距离。
Figure PCTCN2021106614-appb-000007
Figure PCTCN2021106614-appb-000008
其中，
Figure PCTCN2021106614-appb-000009
表示源域特征与信号特征之间的第一分布距离，
Figure PCTCN2021106614-appb-000010
表示源域特征与信号特征之间的第二分布距离，
Figure PCTCN2021106614-appb-000011
表示边缘判别器对应的第一分类准确率，
Figure PCTCN2021106614-appb-000012
表示条件判别器对应的第二分类准确率。
步骤4043，将第一分布距离确定为边缘分布差异，以及将第二分布距离确定为条件分布 差异。
将第一分布距离
Figure PCTCN2021106614-appb-000013
确定为边缘分布差异，将第二分布距离
Figure PCTCN2021106614-appb-000014
确定为条件分布差异。
步骤405，根据第一分布比例缩小边缘分布差异，以及根据第二分布比例缩小条件分布差异。
信号特征和源域特征在特征域上对齐时，同时缩小条件分布差异和边缘分布差异，按照第一分布比例和第二分布比例去分别调整边缘分布差异和条件分布差异。
步骤406，根据缩小的边缘分布差异和条件分布差异，得到缩小分布差异后的信号特征。
信号特征在特征域上的条件分布和边缘分布分别与源域特征对齐，得到对齐后的信号特征，此信号特征与源域特征之间的分布差异缩小。
步骤407，将缩小分布差异后的信号特征确定为对齐后的信号特征。
此时的信号特征在特征域上的分布趋近于源域特征在同一特征域上的分布。
步骤408，对对齐后的信号特征进行分类，得到脑电信号对应的运动想象类型。
示意性的，步骤408可替换为如下步骤。
步骤4081，调用分类器对对齐后的信号特征进行处理，得到脑电信号对应的运动现象类型的预测概率，运动想象类型包括双手运动、双脚运动和舌头运动中的至少一种。
如图5所示，特征提取模型21输出的脑电信号对应的信号特征分别输入至分类器15、条件判别器22和边缘判别器23中。其中，分类器15包括卷积层(Convolutional Neural Network，C-Conv或CNN)和逻辑回归层(Softmax)，卷积层对信号特征进行卷积处理，输出中间向量，将中间向量输入至逻辑回归层，输出脑电信号对应的运动想象类型的预测概率。比如，分类器15输出左手运动的预测概率为0.2，右手运动的预测概率为0.7，双脚运动的预测概率为0，舌头运动的预测概率为0.1。
分类器15输出的特征向量输入至梯度反转层217(Gradient Reversal Layer，GRL)，由梯度反转层217进行进一步计算，得到另一特征向量分别输入至条件判别器22和边缘判别器23。梯度反转层217用于将传到本层的误差与一个负数相乘(-λ)，使得梯度反转层前后的网络层的训练目标相反，以实现对抗的效果。
步骤4082，调用条件判别器对对齐后的信号特征进行处理，得到不同类型的脑电信号所属的域信号的预测概率，域信号包括源域脑电信号和输入的脑电信号中的至少一种。
如图5所示，条件判别器22包括深度卷积层(Deep Convolutional Neural Network，D-Conv或DCNN)和逻辑归回层，深度卷积层对信号特征进行卷积处理，输出中间向量，将中间向量输入至逻辑回归层，输出不同类型的脑电信号属于域信号的预测概率。脑电信号的类型可根据不同类型的受试者进行划分，或者，根据大脑的不同区域进行划分，或者，根据受试者的情绪(或状态)进行划分。比如，条件判别器22输出第一类型的脑电信号属于源域脑电信号的预测概率为0.7，输出第一类型的脑电信号属于目标域脑电信号的预测概率为0.3，则第一类型的脑电信号属于源域脑电信号。第一类型的脑电信号是根据大脑的不同区域产生的脑电信号进行划分的。
步骤4083，调用边缘判别器对对齐后的信号特征进行处理，得到脑电信号所属的域信号的预测概率。
如图5所示，边缘判别器23包括深度卷积层和逻辑回归层，深度卷积层对信号特征进行卷积处理，输出中间向量，将中间向量输入至逻辑回归层，输出脑电信号所属的域信号的预测概率。边缘判别器23与条件判别器22不同的是，边缘判别器23只判断脑电信号所属的域信号。
需要说明的是，步骤4081、步骤4082和步骤4083是同时执行的。
步骤4084，根据脑电信号对应的运动想象类型的预测概率、不同类型的脑电信号所属的域信号的预测概率和脑电信号所属的域信号的预测概率，得到脑电信号对应的运动想象类型。
由于引入了分布差异动态评估机制对两种分布的差异进行评估，优先对齐具有显著影响 的分布，条件判别器22和边缘判别器23确定各自的分布比例，因此，在脑电信号分类模型输出运动想象类型时，综合分类器15、条件判别器22和边缘判别器23的输出结果综合得到运动想象类型。
综上所述，本实施例的方法，通过根据差异分布比例动态地调整脑电信号对应的信号特征在特征域上的分布，使得脑电信号的信号特征与源域特征对齐，从而保证脑电信号的信号特征在特征分布上趋近于源域特征，基于源域特征训练的脑电信号分类模型能够将分类方法迁移至脑电信号对应的信号特征上，提高了脑电信号分类模型识别脑电信号对应的运动想象类型的准确率，使得脑电信号分类模型能够识别多种类型的脑电信号，具有普适性。
通过根据边缘分布差异对应第一分布比例和条件分布差异对应的第二比例，动态调整输入脑电信号分类模型的信号特征与源域特征之间的分布差异，使得信号特征准确向源域特征靠近，从而保证脑电信号分类模型能够准确得到对齐后的信号特征。
通过根据边缘判别器的第一分类准确率和条件判别器的第二分类准确率确定源域特征和信号特征之间的第一分布距离和第二分布距离，使得脑电信号分类模型根据第一分布距离和第二分布距离明确边缘分布差异和条件分布差异。
通过信号特征与源域特征之间的第一分布距离和第二分布距离分别表征信号特征和源域特征之间的边缘分布差异和条件分布差异，从而结合运动想象类型的类型数量准确计算边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。
通过调用分类器、边缘判别器和条件判别器对对齐后的信号特征进行处理，根据三者输出的预测概率综合得到脑电信号对应的运动想象分类模型，使得脑电信号分类模型能够识别不同类型的脑电信号，提高脑电信号分类模型输出运动想象类型的准确率。
通过调用特征提取模型中的各个层次结构输出脑电信号的信号特征，使得脑电信号的信号特征具有时间不变性，使得后续脑电信号分类模型输出准确的运动想象类型。
下面对脑电信号分类模型的训练方法进行说明。
图6示出了本申请一个示例性实施例提供的脑电信号分类模型的训练方法的流程图。本实施例以该方法用于如图1所示的计算机系统100中的服务器120为例进行说明，该方法包括如下步骤。
步骤601，获取源域脑电信号和目标域脑电信号。
源域脑电信号是脑电信号分类模型在训练的过程中使用的样本脑电信号，目标域脑电信号是脑电信号分类模型在测试的过程中使用的测试脑电信号。示意性的，使用公开的数据集训练脑电信号分类模型，公开的数据集包括训练集和测试集。比如，使用公开的比赛数据The BCI Competition IV Dataset 2a运动想象公开数据集，该数据集包括9个受试者，其中每个受试者的脑电数据通过22个脑电电极和3个眼电电极进行记录，信号采样率为205Hz，包括四类运动想象类型(左手运动、右手运动、双脚运动和舌头运动)。
如图7所示，脑电信号分类模型的整个训练流程与使用流程相同，包括输入脑电信号701，对脑电信号进行预处理702，将分布差异动态评估机制703引入脑电信号分类模型704，输出预测结果705。预处理702包括带通滤波7021和信号标准化7022。在一些实施例中，脑电信号分类模型又被命名为动态域自适应模型。
脑电信号的预处理过程参见步骤401的实施方式，此处不再赘述。
步骤602，对源域脑电信号和目标域脑电信号进行特征提取，得到源域脑电信号对应的源域特征和目标域脑电信号对应的目标域特征。
如图8所示，将源域脑电信号101和目标域脑电信号102输入至脑电信号分类模型704中的特征提取模型21，经过特征提取模型21的各个层输出源域信号对应的源域特征和目标域脑电信号对应的目标域特征。参见步骤402的实施方式，此处不再赘述。
步骤603，获取差异分布比例，差异分布比例用于表征不同类型的差异分布对源域特征和目标域特征在特征域上的分布产生的影响。
差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。
由于目标域特征和源域特征在特征域上的分布不同，因此，为了评估不同类型的差异分布对源域特征和目标域特征的影响，引入分布差异动态评估机制对源域特征和目标域特征在边缘分布差异和条件分布差异进行评估。根据边缘分布差异和条件分布差异各自对应的分布比例动态调整影响较大的差异分布。
步骤604，根据差异分布比例将源域特征和目标域特征在特征域上对齐，得到对齐后的目标域特征。
步骤604可替换为如下步骤。
步骤6041，获取源域特征和目标域特征在特征域上的边缘分布差异和条件分布差异。
获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，边缘判别器用于确定脑电信号所属的域信号，条件判别器用于根据运动想象类型确定脑电信号所属的域信号，域信号包括源域脑电信号和目标域脑电信号中的至少一种；根据第一分类准确率获取目标域特征与源域特征之间的第一分布距离，以及根据第二分类准确率获取目标域特征与源域特征之间的第二分布距离；将第一分布距离确定为边缘分布差异，将第二分布距离确定为条件分布差异。
参见步骤4041至步骤4043的实施方式，此处不再赘述。
步骤6042，根据第一分布比例缩小边缘分布差异，以及根据第二分布比例缩小条件分布差异。
步骤6043，根据缩小的边缘分布差异和条件分布差异，得到缩小分布差异后的目标域特征。
步骤6044，将缩小分布差异后的目标域特征确定为对齐后的目标域特征。
步骤6041至步骤6043的实施方式参见步骤404至步骤407的实施方式，此处不赘述。
步骤605，对对齐后的目标域特征进行分类，根据分类结果对脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
步骤605可替换为如下步骤。
步骤6051，调用脑电信号分类模型中的分类器、边缘判别器和条件判别器分别对对齐后的目标域特征进行处理，得到目标域脑电信号对应的运动想象类型的预测概率。
如图8所示，分类器15、边缘判别器23和条件判别器22根据分布差异动态评估机制同时对特征提取模型输出的源域特征和目标域特征进行处理，由于源域特征和目标域特征在特征域上分布相近，因此脑电信号分类模型可将对源域特征进行的分类方法迁移至目标域特征，使得脑电信号分类模型得到目标域脑电信号对应的运动想象类型。
分类器15输出的特征向量输入至梯度反转层217(Gradient Reversal Layer，GRL)，由梯度反转层217进行进一步计算，得到另一特征向量分别输入至条件判别器22和边缘判别器23。梯度反转层217用于将传到本层的误差与一个负数相乘(-λ)，使得梯度反转层前后的网络层的训练目标相反，以实现对抗的效果。
步骤6052，根据预测概率与脑电信号对应的运动想象类型的真实标签计算脑电信号分类模型的结果误差。
结果误差包括分类器对应的误差、条件判别器对应的误差和边缘判别器的误差，本申请实施例以误差函数计算结果误差为例，下面分别对三类误差的计算方式进行说明。
1、分类器对应的误差。
根据预测概率和真实标签计算分类器对应的第一损失函数。公式为如下公式。
Figure PCTCN2021106614-appb-000015
其中，L c(θ f，θ c)表示分类器对应的第一损失函数，
Figure PCTCN2021106614-appb-000016
表示源域特征的期望值，
Figure PCTCN2021106614-appb-000017
表示源域脑电信号，
Figure PCTCN2021106614-appb-000018
表示源域脑电信号的预测概率，D s表示源域，L()表示交叉熵损失函数，
Figure PCTCN2021106614-appb-000019
表示源域脑电信号的真实标签，“～”表示属于关系。上述公式表示从源域脑电信号数据集中取出源域脑电信号和源域脑电信号对应的真实标签，计算源域脑电信号的分类损失的期望。
2、条件判别器对应的误差。
根据条件判别器输出的源域特征对应的源域条件特征图和目标域特征对应的目标域条件特征图，计算条件判别器对应的第二损失函数。公式为如下公式。
Figure PCTCN2021106614-appb-000020
其中，
Figure PCTCN2021106614-appb-000021
表示源域条件特征图，
Figure PCTCN2021106614-appb-000022
表示目标域条件特征图，D c表示第c类条件判别器，即将属于第c类源域脑电信号和目标域脑电信号输入至条件判别器中进行分类，D s表示源域，D t表示目标域，
Figure PCTCN2021106614-appb-000023
表示源域特征的期望值，
Figure PCTCN2021106614-appb-000024
表示目标域特征的期望值，“～”表示属于关系。
条件特征图的计算方式为如下方式。
设分类器的预测概率为f＝[f 1，f 2，f 3...f d]，特征图p＝[p 1，p 2，p 3...p d]，则条件特征图g为如下矩阵。
Figure PCTCN2021106614-appb-000025
3、边缘判别器对应的误差。
根据边缘判别器输出的源域特征对应的源域特征图和目标域特征对应的目标域特征图，计算边缘判别器对应的第三损失函数。公式为如下公式。
Figure PCTCN2021106614-appb-000026
其中，
Figure PCTCN2021106614-appb-000027
表示源域特征图，
Figure PCTCN2021106614-appb-000028
表示目标域特征图，G(x)表示特征提取模型，D m表示边缘判别器，D s表示源域，D t表示目标域，
Figure PCTCN2021106614-appb-000029
表示源域特征的期望值，
Figure PCTCN2021106614-appb-000030
表示目标域特征的期望值，“～”表示属于关系。
综上所述，总的结果误差为如下误差。
Figure PCTCN2021106614-appb-000031
其中，L c表示分类器的第第一损失函数，α表示脑电信号分类模型的待学习参数，L g表示边缘判别器对应的第三损失函数，L l表示条件判别器对应的第二损失函数，
Figure PCTCN2021106614-appb-000032
表示边缘分布对源域特征和信号特征在特征域上产生分布差异时的重要性，C表示运动想象类型的类型数量。
步骤6053，根据结果误差利用误差反向传播算法训练脑电信号分类模型，得到训练后的脑电信号分类模型。
误差反向传播算法(Error BackPropagation Algorithm，BP算法)是指将输出误差以某种形式通过隐藏层向输入层逐层反传，并将误差分摊给各层的所有单元，从而获得各层的误差 信号，此误差信号即为修正单元权值的依据。机器学习模型在学习的过程中由正向传播和误差反向传播两个过程组成，正向传播时，输入样本从输入层传入，经过各隐藏层逐层处理后，传向输出层，若输出层的实际输出与期望输出不符，则转入误差反向传播阶段。
本申请实施例基于随机梯度下降(Stochastic Gradient Descent，SGD)的梯度下降法求解神经网络模型的参数，采用泽维尔(Xavier)初始化方法来初始化模型参数。在求解过程中，把每个受试者的脑电信号及相应标签送进网络中学习，并通过误差反向传播完成模型优化。
综上所述，本实施例提供的方法，通过根据差异分布比例动态地调整目标域特征在特征域上的分布，使得目标域特征与源域特征对齐，从而基于对齐后的目标域特征训练脑电信号分类模型，使得脑电信号分类模型能够将分类学习方法迁移至目标域特征，使得训练后的脑电信号分类模型能够准确输出脑电信号对应的运动想象类型，并且识别多种类型的脑电信号，具有普适性。
通过根据边缘分布差异对应第一分布比例和条件分布差异对应的第二比例，动态调整输入脑电信号分类模型的目标域特征与源域特征之间的分布差异，使得目标域特征准确向源域特征靠近，从而保证脑电信号分类模型能够准确得到对齐后的目标域特征。
通过根据边缘判别器的第一分类准确率和条件判别器的第二分类准确率确定源域特征和目标域特征之间的第一分布距离和第二分布距离，使得脑电信号分类模型根据第一分布距离和第二分布距离明确边缘分布差异和条件分布差异。
通过分类器、边缘判别器和条件判别器对对齐后的目标域进行处理，利用脑电信号对应的运动现象类型的真实标签和脑电信号分类模型输出的预测概率之间的误差对脑电信号分类模型进行训练，从而保证脑电信号分类模型可以加快收敛，缩短模型的训练时间。
根据分类器对应的第一损失函数、边缘判别器对应的第二损失函数和条件判别器对应的第三损失函数准确计算出脑电信号分类模型的结果误差，从而使得脑电信号分类模型基于准确的结果误差进行训练，提高模型的收敛速度，缩短模型的训练时间。
图9示出了本申请一个示例性实施例提供的脑电信号的分类装置的结构框图，该装置可以实现成为计算机设备的全部或一部分。该装置包括如下部分：
第一获取模块910，用于获取脑电信号；
第一特征提取模块920，用于对脑电信号进行特征提取，得到脑电信号对应的信号特征；
所述第一获取模块910，用于获取差异分布比例，差异分布比例用于表征不同类型的差异分布对信号特征和源域特征在特征域上的分布产生的影响，源域特征是源域脑电信号对应的特征；
第一处理模块930，用于根据差异分布比例将信号特征与源域特征进行对齐，得到对齐后的信号特征；
分类模块940，用于对对齐后的信号特征进行分类，得到脑电信号对应的运动想象类型。
在一个可选的实施例中，差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
所述第一获取模块910，用于获取源域特征和信号特征在特征域上的边缘分布差异和条件分布差异；
所述第一处理模块930，用于根据第一分布比例缩小边缘分布差异，以及根据第二分布比例缩小条件分布差异；根据缩小的边缘分布差异和条件分布差异，得到缩小分布差异后的信号特征；将缩小分布差异后的信号特征确定为对齐后的信号特征。
在一个可选的实施例中，所述第一获取模块910，用于获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，边缘判别器用于确定脑电信号所属的域信号，条件判别器用于确定不同类型的脑电信号所属的域信号，域信号包括源域脑电信号和输入的脑电信号中的至少一种；根据第一分类准确率获取信号特征与源域特征之间的第一分布距离，以及根据第二分类准确率获取信号特征与源域特征之间的第二分布距离；将第一分布距离确定为边 缘分布差异，以及将第二分布距离确定为条件分布差异。
在一个可选的实施例中，差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
所述第一获取模块910，用于获取信号特征与源域特征之间的第一分布距离和第二分布距离，第一分布距离用于表征信号特征与源域特征之间的边缘分布差异，第二分布距离用于表征信号特征与源域特征之间的条件分布差异；获取运动想象类型的类型数量；根据第一分布距离、第二分布距离和类型数量得到边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。
在一个可选的实施例中，所述分类模块940包括分类器9401、条件判别器9402和边缘判别器9403；
所述分类器9401，用于对对齐后的信号特征进行处理，得到脑电信号对应的运动想象类型的预测概率；示意性的，运动想象类型包括双手运动、双脚运动和舌头运动中的至少一种；
所述条件判别器9402，用于对对齐后的信号特征进行处理，得到不同类型的脑电信号所属的域信号的预测概率，域信号包括源域脑电信号和输入的脑电信号中的至少一种；
所述边缘判别器9403，用于对对齐后的信号特征进行处理，得到脑电信号所属的域信号的预测概率；
所述分类模块940，用于根据脑电信号对应的运动想象类型的预测概率、不同类型的脑电信号所属的域信号的预测概率和脑电信号所属的域信号的预测概率，得到脑电信号对应的运动想象类型。
在一个可选的实施例中，所述第一特征提取模块920包括时间卷积层9201、空间卷积层9202、批量归一化层9203、平方激活层9204、平均池化层9205和丢弃层9206；
所述时间卷积层9201，用于对脑电信号进行特征提取，得到脑电信号对应的第一信号特征；
所述空间卷积层9202，用于对第一信号特征进行特征提取，得到脑电信号对应的第二信号特征；
所述批量归一化层9203，用于对第二信号特征进行特征提取，得到脑电信号对应的第三信号特征；
所述平方激活层9204，用于对第三信号特征进行特征提取，得到脑电信号对应的第四信号特征；
所述平均池化层9205，用于对第四信号特征进行特征提取，得到脑电信号对应的第五信号特征；
所述丢弃层9206，用于对第五信号特征进行特征提取，得到脑电信号对应的第六信号特征，将第六信号特征确定为脑电信号对应的信号特征。
综上所述，本实施例提供的装置，通过根据差异分布比例动态地调整脑电信号对应的信号特征在特征域上的分布，使得脑电信号的信号特征与源域特征对齐，从而保证脑电信号的信号特征在特征分布上趋近于源域特征，基于源域特征训练的脑电信号分类模型能够将分类方法迁移至脑电信号对应的信号特征上，提高了脑电信号分类模型识别脑电信号对应的运动想象类型的准确率，使得脑电信号分类模型能够识别多种类型的脑电信号，具有普适性。
通过根据边缘分布差异对应第一分布比例和条件分布差异对应的第二比例，动态调整输入脑电信号分类模型的信号特征与源域特征之间的分布差异，使得信号特征准确向源域特征靠近，从而保证脑电信号分类模型能够准确得到对齐后的信号特征。
通过根据边缘判别器的第一分类准确率和条件判别器的第二分类准确率确定源域特征和信号特征之间的第一分布距离和第二分布距离，使得脑电信号分类模型根据第一分布距离和第二分布距离明确边缘分布差异和条件分布差异。
通过信号特征与源域特征之间的第一分布距离和第二分布距离分别表征信号特征和源域特征之间的边缘分布差异和条件分布差异，从而结合运动想象类型的类型数量准确计算边缘 分布差异对应的第一分布比例和条件分布差异对应的第二分布比例。
通过调用分类器、边缘判别器和条件判别器对对齐后的信号特征进行处理，根据三者输出的预测概率综合得到脑电信号对应的运动想象分类模型，使得脑电信号分类模型能够识别不同类型的脑电信号，提高脑电信号分类模型输出运动想象类型的准确率。
通过调用特征提取模型中的各个层次结构输出脑电信号的信号特征，使得脑电信号的信号特征具有时间不变性，使得后续脑电信号分类模型输出准确的运动想象类型。
需要说明的是：上述实施例提供的脑电信号的分类装置，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的脑电信号的分类装置与脑电信号的分类方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图10示出了本申请一个示例性实施例提供的脑电信号分类模型的训练装置的结构框图，该装置可以实现成为计算机设备的全部或一部分。该装置包括如下部分：
第二获取模块1010，用于获取源域脑电信号和目标域脑电信号；
第二特征提取模块1020，用于对源域脑电信号和目标域脑电信号进行特征提取，得到源域脑电信号对应的源域特征和目标域脑电信号对应的目标域特征；
所述第二获取模块1010，用于获取差异分布比例，差异分布比例用于表征不同类型的差异分布对源域特征和目标域特征在特征域上的分布产生的影响；
第二处理模块1030，用于根据差异分布比例将源域特征和目标域特征在特征域上对齐，得到对齐后的目标域特征；
训练模块1040，用于对对齐后的目标域特征进行分类，根据分类结果对脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
在一个可选的实施例中，差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
所述第二获取模块1010，用于获取源域特征和目标域特征在特征域上的边缘分布差异和条件分布差异；
所述第二处理模块1030，用于根据第一分布比例缩小边缘分布差异，以及根据第二分布比例缩小条件分布差异；根据缩小的边缘分布差异和条件分布差异，得到缩小分布差异后的信号特征；将缩小分布差异后的目标域特征确定为对齐后的目标域特征。
在一个可选的实施例中，所述第二获取模块1010，用于获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，边缘判别器用于确定脑电信号所属的域信号，条件判别器用于根据运动想象类型确定脑电信号所属的域信号，域信号包括源域脑电信号和目标域脑电信号中的至少一种；根据第一分类准确率获取目标域特征与源域特征之间的第一分布距离，以及根据第二分类准确率获取目标域特征与源域特征之间的第二分布距离；将第一分布距离确定为边缘分布差异，将第二分布距离确定为条件分布差异。
在一个可选的实施例中，该装置包括脑电信号分类模型1050；
所述脑电信号分类模型1050，用于调用分类器、边缘判别器和条件判别器分别对对齐后的目标域特征进行处理，得到目标域脑电信号对应的运动想象类型的预测概率；
所述训练模块1040，用于根据预测概率与脑电信号对应的运动想象类型的真实标签计算脑电信号分类模型的结果误差；根据结果误差利用误差反向传播算法训练脑电信号分类模型，得到训练后的脑电信号分类模型。
在一个可选的实施例中，所述训练模块1040，用于根据预测概率和真实标签计算分类器对应的第一损失函数；根据条件判别器输出的源域特征对应的源域条件特征图和目标域特征对应的目标域条件特征图，计算条件判别器对应的第二损失函数；根据边缘判别器输出的源域特征对应的源域特征图和目标域特征对应的目标域特征图，计算边缘判别器对应的第三损 失函数；根据第一损失函数、第二损失函数和第三损失函数计算电信号分类模型的结果误差。
综上所述，本实施例提供的装置，通过根据差异分布比例动态地调整目标域特征在特征域上的分布，使得目标域特征与源域特征对齐，从而基于对齐后的目标域特征训练脑电信号分类模型，使得脑电信号分类模型能够将分类学习方法迁移至目标域特征，使得训练后的脑电信号分类模型能够准确输出脑电信号对应的运动想象类型，并且识别多种类型的脑电信号，具有普适性。
通过根据边缘分布差异对应第一分布比例和条件分布差异对应的第二比例，动态调整输入脑电信号分类模型的目标域特征与源域特征之间的分布差异，使得目标域特征准确向源域特征靠近，从而保证脑电信号分类模型能够准确得到对齐后的目标域特征。
通过根据边缘判别器的第一分类准确率和条件判别器的第二分类准确率确定源域特征和目标域特征之间的第一分布距离和第二分布距离，使得脑电信号分类模型根据第一分布距离和第二分布距离明确边缘分布差异和条件分布差异。
通过分类器、边缘判别器和条件判别器对对齐后的目标域进行处理，利用脑电信号对应的运动现象类型的真实标签和脑电信号分类模型输出的预测概率之间的误差对脑电信号分类模型进行训练，从而保证脑电信号分类模型可以加快收敛，缩短模型的训练时间。
根据分类器对应的第一损失函数、边缘判别器对应的第二损失函数和条件判别器对应的第三损失函数准确计算出脑电信号分类模型的结果误差，从而使得脑电信号分类模型基于准确的结果误差进行训练，提高模型的收敛速度，缩短模型的训练时间。
需要说明的是：上述实施例提供的脑电信号分类模型的训练装置，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的脑电信号分类模型的训练装置与脑电信号分类模型的训练方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图11示出了本申请一个示例性实施例提供的服务器的结构示意图。该服务器可以如图1所示的计算机系统100中的服务器120。
服务器1100包括中央处理单元(CPU，Central Processing Unit)1101、包括随机存取存储器(RAM，Random Access Memory)1102和只读存储器(ROM，Read Only Memory)1103的系统存储器1104，以及连接系统存储器1104和中央处理单元1101的系统总线1105。服务器1100还包括帮助计算机内的各个器件之间传输信息的基本输入/输出系统(I/O系统，Input Output System)1106，和用于存储操作系统1113、应用程序1114和其他程序模块1115的大容量存储设备1107。
基本输入/输出系统1106包括有用于显示信息的显示器1108和用于用户输入信息的诸如鼠标、键盘之类的输入设备1109。其中显示器1108和输入设备1109都通过连接到系统总线1105的输入输出控制器1110连接到中央处理单元1101。基本输入/输出系统1106还可以包括输入输出控制器1110以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入输出控制器1110还提供输出到显示屏、打印机或其他类型的输出设备。
大容量存储设备1107通过连接到系统总线1105的大容量存储控制器(未示出)连接到中央处理单元1101。大容量存储设备1107及其相关联的计算机可读介质为服务器1100提供非易失性存储。也就是说，大容量存储设备1107可以包括诸如硬盘或者紧凑型光盘只读存储器(CD-ROM，Compact Disc Read Only Memory)驱动器之类的计算机可读介质(未示出)。
计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、可擦除可编程只读存储器(EPROM，Erasable Programmable Read Only Memory)、带电可擦可编程只读存储器(EEPROM，Electrically Erasable Programmable Read Only Memory)、闪存或其他固态存 储其技术，CD-ROM、数字通用光盘(DVD，Digital Versatile Disc)或固态硬盘(SSD，Solid State Drives)、其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。其中，随机存取记忆体可以包括电阻式随机存取记忆体(ReRAM，Resistance Random Access Memory)和动态随机存取存储器(DRAM，Dynamic Random Access Memory)。当然，本领域技术人员可知计算机存储介质不局限于上述几种。上述的系统存储器1104和大容量存储设备1107可以统称为存储器。
根据本申请的各种实施例，服务器1100还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即服务器1100可以通过连接在系统总线1105上的网络接口单元1111连接到网络1112，或者说，也可以使用网络接口单元1111来连接到其他类型的网络或远程计算机系统(未示出)。
上述存储器还包括一个或者一个以上的程序，一个或者一个以上程序存储于存储器中，被配置由CPU执行。
在一个可选的实施例中，提供了一种计算机设备，该计算机设备包括处理器和存储器，存储器中存储有至少一条指令、至少一段程序、代码集或指令集，至少一条指令、至少一段程序、代码集或指令集由处理器加载并执行以实现如上所述的脑电信号的分类方法和脑电信号分类模型的训练方法中的至少一种。
在一个可选的实施例中，提供了一种计算机可读存储介质，该存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，至少一条指令、至少一段程序、代码集或指令集由处理器加载并执行以实现如上所述的脑电信号的分类方法和脑电信号分类模型的训练方法中的至少一种。
可选地，该计算机可读存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、固态硬盘(SSD，Solid State Drives)或光盘等。其中，随机存取记忆体可以包括电阻式随机存取记忆体(ReRAM，Resistance Random Access Memory)和动态随机存取存储器(DRAM，Dynamic Random Access Memory)。上述本申请实施例序号仅为了描述，不代表实施例的优劣。
本申请实施例还提供了一种计算机程序产品或计算机程序，所述计算机程序产品或计算机程序包括计算机指令，所述计算机指令存储在计算机可读存储介质中。计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令，所述处理器执行所述计算机指令，使得所述计算机设备执行如上方面所述的脑电信号的分类方法和脑电信号分类模型的训练方法中的至少一种。

Claims (15)

  1. 一种脑电信号的分类方法，应用于计算机设备中，所述方法包括：
    获取脑电信号；
    对所述脑电信号进行特征提取，得到所述脑电信号对应的信号特征；
    获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述信号特征和源域特征在特征域上的分布产生的影响，所述源域特征是源域脑电信号对应的特征；
    根据所述差异分布比例将所述信号特征与所述源域特征进行对齐，得到对齐后的信号特征；
    对所述对齐后的信号特征进行分类，得到所述脑电信号对应的运动想象类型。
  2. 根据权利要求1所述的方法，所述差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
    所述根据所述差异分布比例将所述信号特征与所述源域特征进行对齐，得到对齐后的信号特征，包括：
    获取所述源域特征和所述信号特征在所述特征域上的所述边缘分布差异和所述条件分布差异；
    根据所述第一分布比例缩小所述边缘分布差异，以及根据所述第二分布比例缩小所述条件分布差异；
    根据缩小的所述边缘分布差异和所述条件分布差异，得到缩小分布差异后的信号特征；
    将所述缩小分布差异后的信号特征，确定为所述对齐后的信号特征。
  3. 根据权利要求2所述的方法，所述获取所述源域特征和所述信号特征在所述特征域上的边缘分布差异和条件分布差异，包括：
    获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，所述边缘判别器用于确定所述脑电信号所属的域信号，所述条件判别器用于确定不同类型的脑电信号所属的域信号，所述域信号包括所述源域脑电信号和输入的脑电信号中的至少一种；
    根据所述第一分类准确率获取所述信号特征与所述源域特征之间的第一分布距离，以及根据所述第二分类准确率获取所述信号特征与所述源域特征之间的第二分布距离；
    将所述第一分布距离确定为所述边缘分布差异，以及将所述第二分布距离确定为所述条件分布差异。
  4. 根据权利要求1至3任一所述的方法，所述差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
    所述获取差异分布比例，包括：
    获取所述信号特征与所述源域特征之间的第一分布距离和第二分布距离，所述第一分布距离用于表征所述信号特征与所述源域特征之间的所述边缘分布差异，所述第二分布距离用于表征所述信号特征与所述源域特征之间的所述条件分布差异；
    获取所述运动想象类型的类型数量；
    根据所述第一分布距离、所述第二分布距离和所述类型数量得到所述边缘分布差异对应的所述第一分布比例和所述条件分布差异对应的所述第二分布比例。
  5. 根据权利要求1至3任一所述的方法，所述对所述对齐后的信号特征进行分类，得到所述脑电信号对应的运动想象类型，包括：
    调用分类器对所述对齐后的信号特征进行处理，得到所述脑电信号对应的运动想象类型的预测概率；
    调用条件判别器对所述对齐后的信号特征进行处理，得到不同类型的脑电信号所属的域信号的预测概率，所述域信号包括所述源域脑电信号和输入的脑电信号中的至少一种；
    调用边缘判别器对所述对齐后的信号特征进行处理，得到所述脑电信号所属的域信号的 预测概率；
    根据所述脑电信号对应的运动想象类型的预测概率、所述不同类型的脑电信号所属的域信号的预测概率和所述脑电信号所属的域信号的预测概率，得到所述脑电信号对应的运动想象类型。
  6. 根据权利要求1至3任一所述的方法，所述对所述脑电信号进行特征提取，得到所述脑电信号对应的信号特征，包括：
    调用时间卷积层对所述脑电信号进行特征提取，得到所述脑电信号对应的第一信号特征；
    调用空间卷积层对所述第一信号特征进行特征提取，得到所述脑电信号对应的第二信号特征；
    调用批量归一化层对所述第二信号特征进行特征提取，得到所述脑电信号对应的第三信号特征；
    调用平方激活层对所述第三信号特征进行特征提取，得到所述脑电信号对应的第四信号特征；
    调用平均池化层对所述第四信号特征进行特征提取，得到所述脑电信号对应的第五信号特征；
    调用丢弃层对所述第五信号特征进行特征提取，得到所述脑电信号对应的第六信号特征，将所述第六信号特征确定为所述脑电信号对应的信号特征。
  7. 一种脑电信号分类模型的训练方法，所述方法包括：
    获取源域脑电信号和目标域脑电信号；
    对所述源域脑电信号和所述目标域脑电信号进行特征提取，得到所述源域脑电信号对应的源域特征和所述目标域脑电信号对应的目标域特征；
    获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述源域特征和所述目标域特征在特征域上的分布产生的影响；
    根据所述差异分布比例将所述源域特征和所述目标域特征在所述特征域上对齐，得到对齐后的目标域特征；
    对所述对齐后的目标域特征进行分类，根据分类结果对所述脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
  8. 根据权利要求7所述的方法，所述差异分布比例包括边缘分布差异对应的第一分布比例和条件分布差异对应的第二分布比例；
    所述根据所述差异分布比例将所述源域特征和所述目标域特征在所述特征域上对齐，得到对齐后的目标域特征，包括：
    获取所述源域特征和所述目标域特征在所述特征域上的边缘分布差异和条件分布差异；
    根据所述第一分布比例缩小所述边缘分布差异，以及根据所述第二分布比例缩小所述条件分布差异；
    根据缩小的所述边缘分布差异和所述条件分布差异，得到缩小分布差异后的目标域特征；
    将所述缩小分布差异后的目标域特征确定为所述对齐后的目标域特征。
  9. 根据权利要求8所述的方法，所述获取所述源域特征和所述目标域特征在所述特征域上的边缘分布差异和条件分布差异，包括：
    获取边缘判别器的第一分类准确率和条件判别器的第二分类准确率，所述边缘判别器用于确定所述脑电信号所属的域信号，所述条件判别器用于确定不同类型的脑电信号所属的域信号，所述域信号包括所述源域脑电信号和所述目标域脑电信号中的至少一种；
    根据所述第一分类准确率获取所述目标域特征与所述源域特征之间的第一分布距离，以及根据所述第二分类准确率获取所述目标域特征与所述源域特征之间的第二分布距离；
    将所述第一分布距离确定为所述边缘分布差异，将所述第二分布距离确定为所述条件分布差异。
  10. 根据权利要求7至9任一所述的方法，所述对所述对齐后的目标域特征进行分类，根据分类结果对所述脑电信号分类模型进行训练，得到训练后的脑电信号分类模型，包括：
    调用所述脑电信号分类模型中的分类器、边缘判别器和条件判别器分别对所述对齐后的目标域特征进行处理，得到所述目标域脑电信号对应的运动想象类型的预测概率；
    根据所述预测概率与所述脑电信号对应的运动想象类型的真实标签，计算所述脑电信号分类模型的结果误差；
    根据所述结果误差利用误差反向传播算法训练所述脑电信号分类模型，得到所述训练后的脑电信号分类模型。
  11. 根据权利要求10所述的方法，所述根据所述预测概率与所述脑电信号对应的运动想象类型的真实标签，计算所述脑电信号分类模型的结果误差，包括：
    根据所述预测概率和所述真实标签计算所述分类器对应的第一损失函数；
    根据所述条件判别器输出的所述源域特征对应的源域条件特征图和所述目标域特征对应的目标域条件特征图，计算所述条件判别器对应的第二损失函数；
    根据所述边缘判别器输出的所述源域特征对应的源域特征图和所述目标域特征对应的目标域特征图，计算所述边缘判别器对应的第三损失函数；
    根据所述第一损失函数、所述第二损失函数和所述第三损失函数计算所述脑电信号分类模型的结果误差。
  12. 一种脑电信号的分类装置，所述装置包括：
    第一获取模块，用于获取脑电信号；
    第一特征提取模块，用于对所述脑电信号进行特征提取，得到所述脑电信号对应的信号特征；
    所述第一获取模块，用于获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述信号特征和源域特征在特征域上的分布产生的影响，所述源域特征是源域脑电信号对应的特征；
    第一处理模块，用于根据所述差异分布比例将所述信号特征与所述源域特征进行对齐，得到对齐后的信号特征；
    分类模块，用于对所述对齐后的信号特征进行分类，得到所述脑电信号对应的运动想象类型。
  13. 一种脑电信号分类模型的训练装置，所述装置包括：
    第二获取模块，用于获取源域脑电信号和目标域脑电信号；
    第二特征提取模块，用于对所述源域脑电信号和所述目标域脑电信号进行特征提取，得到所述源域脑电信号对应的源域特征和所述目标域脑电信号对应的目标域特征；
    所述第二获取模块，用于获取差异分布比例，所述差异分布比例用于表征不同类型的差异分布对所述源域特征和所述目标域特征在特征域上的分布产生的影响；
    第二处理模块，用于根据所述差异分布比例将所述源域特征和所述目标域特征在所述特征域上对齐，得到对齐后的目标域特征；
    训练模块，用于对所述对齐后的目标域特征进行分类，根据分类结果对所述脑电信号分类模型进行训练，得到训练后的脑电信号分类模型。
  14. 一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一段程序，所述至少一段程序由所述处理器加载并执行以实现如权利要求1至6任一所述的脑电信号的分类方法以及7至11任一所述的脑电信号分类模型的训练方法。
  15. 一种计算机可读存储介质，所述可读存储介质中存储有至少一段程序，所述至少一段程序由处理器加载并执行以实现如权利要求1至6任一所述的脑电信号的分类方法以及7至11任一所述的脑电信号分类模型的训练方法。
PCT/CN2021/106614 2020-08-26 2021-07-15 脑电信号的分类方法、分类模型的训练方法、装置及介质 WO2022042122A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21859962.9A EP4101371A4 (en) 2020-08-26 2021-07-15 METHOD AND DEVICE FOR CLASSIFICATION OF ELECTROENCEPHALOGRAM SIGNALS, METHOD AND DEVICE FOR TRAINING A MODEL FOR CLASSIFICATION OF ELECTROENCEPHALOGRAM SIGNALS, AND MEDIUM
US17/969,177 US20230054751A1 (en) 2020-08-26 2022-10-19 Method and apparatus for classifying electroencephalogram signal, method and apparatus for training classification model, and electronic device and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010867943.X 2020-08-26
CN202010867943.XA CN111728609B (zh) 2020-08-26 2020-08-26 脑电信号的分类方法、分类模型的训练方法、装置及介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/969,177 Continuation US20230054751A1 (en) 2020-08-26 2022-10-19 Method and apparatus for classifying electroencephalogram signal, method and apparatus for training classification model, and electronic device and computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2022042122A1 true WO2022042122A1 (zh) 2022-03-03

Family

ID=72658890

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/106614 WO2022042122A1 (zh) 2020-08-26 2021-07-15 脑电信号的分类方法、分类模型的训练方法、装置及介质

Country Status (4)

Country Link
US (1) US20230054751A1 (zh)
EP (1) EP4101371A4 (zh)
CN (1) CN111728609B (zh)
WO (1) WO2022042122A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114692697A (zh) * 2022-04-22 2022-07-01 南通大学 一种头皮脑电信号的在线多特征空间迁移识别方法
CN116889411A (zh) * 2023-07-21 2023-10-17 北京交通大学 自动驾驶安全员脑电信息语义分析方法及系统
WO2024192259A1 (en) * 2023-03-14 2024-09-19 The Regents Of The University Of California Systems and methods for brain-machine interface shared autonomy

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111728609B (zh) * 2020-08-26 2020-11-17 腾讯科技（深圳）有限公司 脑电信号的分类方法、分类模型的训练方法、装置及介质
CN112690793B (zh) * 2020-12-28 2023-05-16 中国人民解放军战略支援部队信息工程大学 情绪脑电迁移模型训练方法、系统及情绪识别方法和设备
CN113693613B (zh) * 2021-02-26 2024-05-24 腾讯科技（深圳）有限公司 脑电信号分类方法、装置、计算机设备及存储介质
CN113031766B (zh) * 2021-03-15 2022-09-23 哈尔滨工业大学 一种通过脑电解码汉语发音的方法
CN113392733B (zh) * 2021-05-31 2022-06-21 杭州电子科技大学 基于标签对齐的多源域自适应跨被试eeg认知状态评估方法
CN113688673B (zh) * 2021-07-15 2023-05-30 电子科技大学 在线场景下心电信号的跨用户情感识别方法
CN113925509B (zh) * 2021-09-09 2024-01-23 杭州回车电子科技有限公司 基于脑电信号的注意力值计算方法、装置和电子装置
CN113842152B (zh) * 2021-09-28 2023-05-30 西交利物浦大学 脑电信号分类网络训练方法、分类方法、设备及存储介质
CN114103974B (zh) * 2021-11-01 2022-07-29 北京理工大学 一种面向车辆连续控制的脑机接口方法
CN115836864B (zh) * 2022-09-30 2024-08-09 中国人民解放军国防科技大学 一种基于多脑bci的目标增强判读方法和装置
CN116070796B (zh) * 2023-03-29 2023-06-23 中国科学技术大学 柴油车排放等级评估方法及系统
CN116996722B (zh) * 2023-06-29 2024-06-04 广州慧思软件科技有限公司 一种5g网络环境下虚拟同步课堂教学系统及其工作方法
CN117034078B (zh) * 2023-08-07 2024-09-20 上海韶脑传感技术有限公司 脑电信号解码方法、装置、上位机及计算机可读存储介质
CN117390542A (zh) * 2023-10-13 2024-01-12 上海韶脑传感技术有限公司 一种基于域泛化的跨被试运动想象脑电建模方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110071965A1 (en) * 2009-09-24 2011-03-24 Yahoo! Inc. System and method for cross domain learning for data augmentation
CN104523269A (zh) * 2015-01-15 2015-04-22 江南大学 一种面向癫痫脑电信号迁移环境的自适应识别方法
US20190073605A1 (en) * 2017-02-27 2019-03-07 Andrew Jay KELLER Systems and methods for real-time neural command classification and task execution
CN111544856A (zh) * 2020-04-30 2020-08-18 天津大学 基于新型迁移学习模型的脑-肌电智能全肢体康复方法
CN111584029A (zh) * 2020-04-30 2020-08-25 天津大学 基于判别式对抗网络的脑电自适应模型及在康复中的应用
CN111728609A (zh) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 脑电信号的分类方法、分类模型的训练方法、装置及介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104523268B (zh) * 2015-01-15 2017-02-22 江南大学 一种具备迁移学习能力的脑电信号识别模糊系统方法
US9984283B2 (en) * 2015-02-14 2018-05-29 The Trustees Of The University Of Pennsylvania Methods, systems, and computer readable media for automated detection of abnormalities in medical images
CN107273853B (zh) * 2017-06-16 2019-05-21 中国地质大学（武汉） 一种基于类心和协方差对齐的遥感图像迁移学习方法
WO2020023989A1 (en) * 2018-08-03 2020-02-06 Rehabswift Pty Ltd Stroke rehabilitation method and system using a brain-computer interface (bci)
CN111444747B (zh) * 2019-01-17 2023-09-29 复旦大学 一种基于迁移学习和空洞卷积的癫痫状态识别方法
CN110503082B (zh) * 2019-08-30 2024-03-12 腾讯科技（深圳）有限公司 一种基于深度学习的模型训练方法以及相关装置
CN110531861B (zh) * 2019-09-06 2021-11-19 腾讯科技（深圳）有限公司 运动想象脑电信号的处理方法和装置及存储介质
CN111104509B (zh) * 2019-11-08 2023-06-16 武汉理工大学 一种基于概率分布自适应的实体关系分类方法
CN110851783B (zh) * 2019-11-12 2021-10-08 华中科技大学 一种用于脑机接口校准的异构标签空间迁移学习方法
CN111444859A (zh) * 2020-03-30 2020-07-24 哈尔滨工程大学 一种无监督跨域人脸表情识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110071965A1 (en) * 2009-09-24 2011-03-24 Yahoo! Inc. System and method for cross domain learning for data augmentation
CN104523269A (zh) * 2015-01-15 2015-04-22 江南大学 一种面向癫痫脑电信号迁移环境的自适应识别方法
US20190073605A1 (en) * 2017-02-27 2019-03-07 Andrew Jay KELLER Systems and methods for real-time neural command classification and task execution
CN111544856A (zh) * 2020-04-30 2020-08-18 天津大学 基于新型迁移学习模型的脑-肌电智能全肢体康复方法
CN111584029A (zh) * 2020-04-30 2020-08-25 天津大学 基于判别式对抗网络的脑电自适应模型及在康复中的应用
CN111728609A (zh) * 2020-08-26 2020-10-02 腾讯科技（深圳）有限公司 脑电信号的分类方法、分类模型的训练方法、装置及介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4101371A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114692697A (zh) * 2022-04-22 2022-07-01 南通大学 一种头皮脑电信号的在线多特征空间迁移识别方法
WO2024192259A1 (en) * 2023-03-14 2024-09-19 The Regents Of The University Of California Systems and methods for brain-machine interface shared autonomy
CN116889411A (zh) * 2023-07-21 2023-10-17 北京交通大学 自动驾驶安全员脑电信息语义分析方法及系统

Also Published As

Publication number Publication date
US20230054751A1 (en) 2023-02-23
EP4101371A4 (en) 2023-08-02
CN111728609B (zh) 2020-11-17
EP4101371A1 (en) 2022-12-14
CN111728609A (zh) 2020-10-02

Similar Documents

Publication Publication Date Title
WO2022042122A1 (zh) 脑电信号的分类方法、分类模型的训练方法、装置及介质
Nakisa et al. Long short term memory hyperparameter optimization for a neural network based emotion recognition framework
Kim et al. Deep physiological affect network for the recognition of human emotions
KR102221264B1 (ko) 인간 감정 인식을 위한 딥 생리적 정서 네트워크를 이용한 인간 감정 추정 방법 및 그 시스템
Lin et al. EEG-based learning system for online motion sickness level estimation in a dynamic vehicle environment
CN109157231A (zh) 基于情绪刺激任务的便携式多通道抑郁倾向评估系统
Zhuang et al. Real-time emotion recognition system with multiple physiological signals
Qu et al. Bio-physiological-signals-based vr cybersickness detection
Mellouk et al. CNN-LSTM for automatic emotion recognition using contactless photoplythesmographic signals
Shi et al. Convolutional neural network for freezing of gait detection leveraging the continuous wavelet transform on lower extremities wearable sensors data
CN117064388A (zh) 基于情绪识别实现精神障碍评估分析的系统
Theerthagiri Stress emotion recognition with discrepancy reduction using transfer learning
Dar et al. YAAD: young adult’s affective data using wearable ECG and GSR sensors
Sharma et al. Novel eeg based schizophrenia detection with iomt framework for smart healthcare
Ahamad System architecture for brain-computer interface based on machine learning and internet of things
Wang et al. Diagnosis and intervention for children with autism spectrum disorder: a survey
Irshad et al. Wearable-based human flow experience recognition enhanced by transfer learning methods using emotion data
Lian et al. An Ear Wearable Device System for Facial Emotion Recognition Disorders
Ilyas et al. Deep transfer learning in human–robot interaction for cognitive and physical rehabilitation purposes
Petrenko Breathmonitor: AI Sleep Apnea Mobile Detector
Shermadurai et al. Deep learning framework for classification of mental stress from multimodal datasets
Ward et al. Dilated temporal convolutional neural network architecture with independent component layer for human activity recognition
Sweeney-Fanelli et al. ECG-based automated emotion recognition using temporal convolution neural networks
Majdi et al. Direct lingam and visibility graphs for analyzing brain connectivity in BCI
Li et al. The early warning research on nursing care of stroke patients with intelligent wearable devices under COVID-19

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21859962

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021859962

Country of ref document: EP

Effective date: 20220906

NENP Non-entry into the national phase

Ref country code: DE