CN113220120B - Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation - Google Patents
Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation Download PDFInfo
- Publication number
- CN113220120B CN113220120B CN202110457656.6A CN202110457656A CN113220120B CN 113220120 B CN113220120 B CN 113220120B CN 202110457656 A CN202110457656 A CN 202110457656A CN 113220120 B CN113220120 B CN 113220120B
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram signal
- test
- trial
- electroencephalogram
- category
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Abstract
The invention provides a self-adaptive motor imagery brain-computer interface training method fused with subjective and objective assessment, which comprises the following steps: acquiring a multi-test off-line electroencephalogram signal, and obtaining a trained source classifier based on the multi-test off-line electroencephalogram signal and an initial classifier; receiving the single-test electroencephalogram signal and the subjective screening information, and reserving or rejecting the single-test electroencephalogram signal based on the subjective screening information; if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selectively reserving or removing the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal; and obtaining a trained target classifier based on the single-test electroencephalogram signal and the source classifier. According to the invention, the electroencephalogram signals of single test times sent when the attention of a user is not concentrated can be removed through the subjective screening information, the balance of the electroencephalogram signals is ensured, and the classification accuracy of brain-computer interfaces is improved.
Description
Technical Field
The application relates to the field of electroencephalogram recognition, in particular to a self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation.
Background
The development of Brain Computer Interface (BCI) technology has provided a channel for serious dyskinesia sufferers to communicate with the outside world for control and also provides a method for enhancing the control ability of a person to interact with the outside world, and BCI converts human mind into external control commands by decoding Brain activities, which are generally represented by Brain electrical signals. Compared with the brain-computer interface based on evoked potential, the brain-computer interface based on motor imagery has the advantages of being independent of external stimulation and simple to operate.
At present, brain-computer interfaces based on motor imagery cause brain-computer signals to generate non-stationarity due to cognitive factors such as attention, workload and fatigue of users, and further have the problem of low classification accuracy.
Therefore, the prior art is in need of improvement.
Disclosure of Invention
The invention aims to solve the technical problem that the electroencephalogram signals are unstable due to the cognitive factors such as attention, workload and fatigue of a user, and further the classification accuracy of a brain-computer interface is low. The user sends subjective screening information after sending each single-test electroencephalogram signal, the single-test electroencephalogram signals sent when the attention of the user is not concentrated can be removed through the subjective screening information, the stability of the electroencephalogram signals is guaranteed, and the classification accuracy of the brain-computer interface is improved.
In a first aspect, an embodiment of the present invention provides a method for training a brain-computer interface of adaptive motor imagery in combination with subjective and objective assessment, including:
acquiring a multi-test off-line electroencephalogram signal generated by a user according to a multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signal to obtain a trained source classifier;
receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information;
if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal;
and modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier.
As a further improved technical scheme, the training of the initial classifier based on the off-line electroencephalogram signals with multiple test times to obtain a trained source classifier specifically includes:
preprocessing the multi-trial off-line electroencephalogram signals to obtain multi-trial multi-frequency-band off-line electroencephalogram signals;
extracting a plurality of feature matrixes corresponding to the multi-trial multiband off-line electroencephalogram signal by adopting a common space mode algorithm, wherein each feature matrix comprises elements corresponding to a plurality of features respectively;
determining a plurality of characteristic column vectors based on the plurality of characteristic matrixes, determining mutual information corresponding to each characteristic column vector based on the characteristic column vector and a motor imagery category label vector corresponding to the multi-trial off-line electroencephalogram signal, and determining a plurality of target characteristics in the plurality of characteristics according to the plurality of mutual information;
and determining a plurality of target feature matrixes based on the plurality of target features and the plurality of feature matrixes, and training the initial classifier based on the plurality of target feature matrixes to obtain a trained source classifier.
As a further improved technical scheme, the subjective screening information includes selection or deletion; the determining to reserve or reject the single-test electroencephalogram signal based on the subjective screening information specifically comprises the following steps:
when the subjective screening information is selected, reserving the single-test electroencephalogram signal;
and when the subjective screening information is deleted, rejecting the single-test electroencephalogram signal.
As a further improvement, the category includes correct or incorrect; the selecting, reserving or rejecting the single-trial electroencephalogram signal based on the category and the multi-trial offline electroencephalograms specifically comprises the following steps:
when the category is correct, retaining the single-trial electroencephalogram signal;
when the category is wrong, determining a motor imagery category corresponding to the single-test-time electroencephalogram signal based on the multi-test-time offline electroencephalogram signal;
if the motor imagery category is consistent with the prompt category, the single-trial electroencephalogram signal is reserved, and if the motor imagery category is inconsistent with the prompt category, the single-trial electroencephalogram signal is removed.
As a further improved technical scheme, the off-line electroencephalogram signals with multiple test times comprise a plurality of off-line electroencephalogram signals, and each off-line electroencephalogram signal has a motor imagery category respectively corresponding to each off-line electroencephalogram signal; the determining of the motor imagery category corresponding to the single-test-time electroencephalogram signal based on the multi-test-time offline electroencephalogram signal specifically comprises the following steps:
for each off-line electroencephalogram signal, calculating the relative entropy between the off-line electroencephalogram signal and the single-test electroencephalogram signal;
dividing all the calculated relative entropies into a first relative entropy set and a second relative entropy set based on the motor imagery categories respectively corresponding to each off-line electroencephalogram signal;
calculating the sum of the relative entropies in the first relative entropy set to obtain a first sum value, and calculating the sum of the relative entropies in the second relative entropy set to obtain a second sum value;
and determining a target sum value in the first sum value and the second sum value, and taking the operation imagination category corresponding to the target sum value as the motor imagination category corresponding to the single-trial electroencephalogram signal.
As a further improved technical solution, the calculating the relative entropy between the off-line electroencephalogram signal and the single-test electroencephalogram signal specifically includes:
performing dimensionality reduction processing on the single-test electroencephalogram signal to obtain a single-test low-dimensional electroencephalogram signal;
performing dimensionality reduction processing on the offline electroencephalogram signal to obtain a low-dimensional offline electroencephalogram signal corresponding to the offline electroencephalogram signal;
and calculating the relative entropy between the low-dimensional off-line electroencephalogram signal and the single-test-time low-dimensional electroencephalogram signal.
As a further improved technical scheme, the task feedback game comprises a chessboard, target blocks, a plurality of barrier blocks and moving blocks, wherein the target blocks, the barrier blocks and the moving blocks are arranged on the chessboard; the task feedback game is used for enabling the user to move to the target block under the condition of avoiding the plurality of obstacle blocks according to the prompt type, and the prompt type is left-hand imagination or right-hand imagination.
In a second aspect, the present invention also provides a brain-computer interface method, including:
collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multi-channel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multi-channel electroencephalogram control signal, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
In a third aspect, the present invention provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a multi-test off-line electroencephalogram signal generated by a user according to a multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signal to obtain a trained source classifier;
receiving single-test electroencephalogram signals sent by the user based on preset prompt categories of task feedback games and subjective screening information corresponding to the single-test electroencephalogram signals, and determining to reserve or eliminate the single-test electroencephalogram signals based on the subjective screening information;
if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal;
modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier;
or, collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multi-channel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multi-channel electroencephalogram control signal, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring a multi-test off-line electroencephalogram signal generated by a user according to a multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signal to obtain a trained source classifier;
receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information;
if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal;
modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier;
obtaining a trained target classifier based on the single-trial electroencephalograms and the source classifier;
or collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multichannel electroencephalogram control signals into the target classifier to obtain a classification result corresponding to the multichannel electroencephalogram control signals, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method integrating subjective and objective evaluation;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
Compared with the prior art, the embodiment of the invention has the following advantages:
according to the embodiment of the invention, multi-test off-line electroencephalogram signals generated by a user according to multi-test motor imagery are collected, and an initial classifier is trained on the basis of the multi-test off-line electroencephalogram signals to obtain a trained source classifier; receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information; if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal; and modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier. According to the method and the device, the user sends the subjective screening information after sending each single-test electroencephalogram signal, the single-test electroencephalogram signals sent when the attention of the user is not concentrated can be removed through the subjective screening information, the stability of the electroencephalogram signals is guaranteed, and the classification accuracy of the brain-computer interface is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a training method of an adaptive motor imagery brain-computer interface with fusion of subjective and objective assessment according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a source classifier obtained by training an initial classifier through multi-trial off-line electroencephalogram signals in the embodiment of the present invention;
FIG. 3 is a schematic diagram of a task feedback game in an embodiment of the present invention;
FIG. 4 is a schematic diagram of a task feedback game including an information prompt box according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of a task feedback game in the embodiment of the present invention when acquiring single-trial electroencephalogram signals;
FIG. 6 is a schematic diagram illustrating subjective screening and objective screening of a single-trial electroencephalogram signal according to an embodiment of the present invention;
fig. 7 is a schematic flowchart of a control method of a brain-computer interface according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The inventor finds that the development of Brain Computer Interface (BCI) technology provides a channel for a serious dyskinesia person to communicate with the outside world for control, and also provides a method for enhancing the control ability of a person to interact with the outside world, and the BCI converts the thinking of the person into an external control command by decoding Brain activities generally represented by Brain electrical signals. Compared with the brain-computer interface based on evoked potential, the brain-computer interface based on motor imagery has the advantages of being independent of external stimulation and simple to operate.
Research shows that users who finish motor imagery tasks usually feel fatigue, and the fatigue can cause electroencephalogram signals of the same user to deviate in different time periods, so that the electroencephalogram signals are unstable. The reasons for generating non-stationarity of the electroencephalogram signals also include cognitive factors such as inattention and the like. The non-stationarity of the brain electrical signals can cause low classification accuracy of brain-computer interfaces.
In order to solve the problems, in the invention, a multi-trial off-line electroencephalogram signal generated by a user through motor imagery according to a multi-trial motor imagery category is collected, and an initial classifier is trained on the basis of the multi-trial off-line electroencephalogram signal to obtain a trained source classifier; receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information; if the single-trial-time electroencephalogram signal is reserved, determining the category corresponding to the single-trial-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-trial-time electroencephalogram signal based on the category and the multi-trial-time off-line electroencephalogram signal; and modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier. According to the method and the device, the user sends the subjective screening information after sending each single-test electroencephalogram signal, the single-test electroencephalogram signals sent when the attention of the user is not concentrated can be removed through the subjective screening information, the stability of the electroencephalogram signals is guaranteed, and the classification accuracy of the brain-computer interface is improved.
The adaptive motor imagery brain-computer interface training method fusing subjective and objective assessment can be applied to a brain-computer interface system, the brain-computer interface system at least comprises electroencephalogram data acquisition equipment, a processor and a storage medium, functions realized by the method can be realized by calling a program code by the processor, and the program code can be stored in the storage medium.
Various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a method for training an adaptive motor imagery brain-computer interface with fusion of subjective and objective evaluations in an embodiment of the present invention is shown, including the following steps:
s1, collecting multi-test off-line electroencephalogram signals generated by a user according to the multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signals to obtain a trained source classifier.
In the embodiment of the invention, the off-line electroencephalogram signals of multiple trials are acquired through electroencephalogram data acquisition equipment in a brain-computer interface system. The electroencephalogram data acquisition equipment comprises an amplifier and an electroencephalogram cap, the amplifier can use an actiCHamp amplifier produced by Brainproduct company, the electroencephalogram cap can use an actiCAP electroencephalogram cap, the actiCAP electroencephalogram cap is provided with 64 electrodes, and electroencephalogram signals of 64 channels can be acquired.
In the embodiment of the invention, a user carries out a two-classification motor imagery task in a brain-computer interface system so that the brain-computer interface system acquires off-line electroencephalogram signals of multiple test times, the application imagery task comprises multiple test times, each test time brain-computer interface system shows a motor imagery category, and the user makes an operation imagery according to the motor imagery categories so that the brain-computer interface system acquires the off-line electroencephalogram signals of the test times. The motor imagery categories include a left hand motor imagery category and a right hand motor imagery category. In one trial, the user performs left-hand imagination to generate the off-line brain electrical signals for the trial in the left-hand motor imagination category shown by the brain-computer interface system. After the user finishes the motor imagery tasks of a plurality of trials, the electroencephalogram data acquisition equipment acquires the off-line electroencephalogram signals of the plurality of trials. And each off-line electroencephalogram signal of the trial time has the corresponding motor imagery category.
In the embodiment of the invention, the brain-computer interface system further comprises a processor, the motor imagery task runs based on a processor calling program, and the brain-computer interface system further comprises a display, and the motor imagery category given by the motor imagery task is displayed on the display.
In the embodiment of the present invention, referring to fig. 2, the training of the initial classifier based on the off-line multi-trial electroencephalogram signal to obtain the trained source classifier specifically includes:
and S11, preprocessing the multi-test off-line electroencephalogram signals to obtain multi-test multi-frequency-band off-line electroencephalogram signals.
In the embodiment of the invention, noise and signals irrelevant to the motor imagery part may exist in the multi-test off-line electroencephalogram signals, and the multi-test off-line electroencephalogram signals need to be preprocessed. The preprocessing comprises channel positioning, filtering, segmentation, baseline correction, re-referencing, independent component analysis and finite impulse response filtering.
The channel positioning is to distribute coordinate information to each channel of the multi-trial off-line electroencephalogram signal so as to analyze data, such as drawing a electroencephalogram topographic map; the filtering is to filter the frequency band data irrelevant to the motor imagery in the multi-trial off-line electroencephalogram signals; the segmentation is to find out data corresponding to each motor imagery time in the off-line electroencephalogram signal of multiple trials, for example, the off-line electroencephalogram signal data is represented as 64 × 800000 before the segmentation, and the off-line electroencephalogram signal data is represented as 64 × 6000 × 100 after the segmentation. The motor imagery category corresponding to each trial time in the multi-trial-time off-line electroencephalogram signals after the segmentation processing is a left-hand motor imagery category or a right-hand motor imagery category, that is, the off-line electroencephalogram signals irrelevant to the motor imagery are removed through the segmentation processing.
The baseline correction can remove the noise of the off-line electroencephalogram signal; re-referencing similar baseline correction, again to remove noise; independent component analysis may better identify and remove eye movement and other noise. The off-line electroencephalogram signals of multiple test times are subjected to channel positioning, filtering, segmentation, baseline correction, re-referencing and independent component analysis to obtain the pre-processing training signals of multiple test times.
Filtering each frequency band of the multi-trial preprocessing training signal by using a plurality of groups of Finite Impulse Response (FIR) filters with overlapped frequency bands, for example, filtering the frequency bands of the multi-trial preprocessing training signal at 8-12Hz,10-14Hz,12-16Hz and … … to obtain a multi-trial multiband off-line electroencephalogram signal; the motor imagery category of each tested off-line electroencephalogram signal in the multi-test multi-band off-line electroencephalogram signal is a left-hand motor imagery category or a right-hand motor imagery category.
S12, extracting a plurality of feature matrixes corresponding to the multi-trial multiband off-line electroencephalogram signal by adopting a common space mode algorithm, wherein each feature matrix comprises elements corresponding to a plurality of features respectively.
In the embodiment of the invention, a Common Spatial Pattern (CSP) algorithm is used for constructing a Spatial filter, and the multi-trial multiband off-line electroencephalogram signal is subjected to Spatial filtering to obtain a plurality of characteristic matrixes of overlapped frequency bands.
Specifically, the co-space mode algorithm adopts a supervised method to create a space filter, maximizes one class of variance and minimizes the other class of variance, and adopts a mode of diagonalizing two classes of task covariance matrixes simultaneously to obtain the eigenvector with the maximum distinguishable degree. The process of processing the multi-trial multiband off-line electroencephalogram signal by adopting a common space mode algorithm to obtain a plurality of characteristic matrixes is as follows:
if the motor imagery category of the multi-band offline electroencephalogram signal of the ith test is left-hand imagery, determining a corresponding first offline electroencephalogram signal matrix E according to the multi-band offline electroencephalogram signal of the ith test 1,i If the motor imagery category of the ith tested multiband off-line electroencephalogram signal is the right hand imagery, determining a corresponding second off-line electroencephalogram signal matrix E according to the ith tested multiband off-line electroencephalogram signal 2,i 。E 1,i And E 2,i The dimensions of (a) are nb multiplied by nt, nb is the number of electroencephalogram channels, and nt is the number of samples collected by each channel.
And when the motor imagery category of the multiband offline electroencephalogram signal of the ith test is left-hand imagery, calculating a first covariance matrix of the ith test corresponding to the multiband offline electroencephalogram signal of the ith test according to the formula (1).
Wherein, C 1,i Is the first covariance matrix of the ith trial, E 1,i Is the first off-line brain electrical signal matrix of the ith trial,
is E 1,i The transpose of (a) is performed,
is that
The trace of (c).
And when the motor imagery category of the multiband off-line electroencephalogram signal of the ith test is right hand imagery, calculating a second covariance matrix of the ith test corresponding to the second off-line electroencephalogram signal according to the formula (2).
Wherein, C 2,i Is the second covariance matrix of the ith trial, E 2,i Is the second off-line brain electrical signal matrix of the ith trial,
is E 2,i The method (2) is implemented by the following steps,
is that
The trace of (c).
Determining a first average covariance matrix according to the first covariance matrices of all trials of which the motor imagery category is left-hand imagery in the multi-trial multiband offline electroencephalogram signal, wherein the first average covariance matrix is shown in a formula (3); and determining a second average covariance matrix according to the second covariance matrices of all the trials of which the motor imagery class is the right hand imagery in the multi-trial multiband off-line electroencephalogram signal, wherein the second average covariance matrix is shown in a formula (4).
Wherein the content of the first and second substances,
is a first mean covariance matrix, n 1 Is the number of all trials for which the motor imagery category is left-hand imagery,
is a second mean covariance matrix, n 2 Is the number of all trials for which the motor imagery category is the right hand imagery.
A hybrid spatial covariance matrix is calculated from the first and second mean covariance matrices as shown in equation (5).
Where C is the mixed spatial covariance matrix.
And (4) performing eigenvalue decomposition on the mixed space covariance matrix to obtain a mixed space eigenvector, as shown in formula (6).
C＝UλU T (6)
Where U is the hybrid spatial feature vector, U T Is the transpose of U, and λ represents the sum of the covariance matrices of the mixtureAnd carrying out line eigenvalue decomposition to obtain a diagonal array of eigenvalues.
The whitening matrix is determined from the diagonal matrix of eigenvalues and the mixed spatial eigenvector, as shown in equation (7).
Where P is the whitening matrix.
A first covariance matrix corresponding to all trials with the motor imagery class being left-hand imagery is determined as shown in equation (8), and a second covariance matrix corresponding to all trials with the motor imagery class being right-hand imagery is determined as shown in equation (9).
Wherein, C 1 Is a first covariance matrix corresponding to all trials with the motion imagery class of left-hand imagery, C 2 The motor imagery class is the second covariance matrix corresponding to all trials of the right hand imagery.
Applying whitening matrix to C 1 And C 2 And respectively obtaining a first transformation matrix and a second transformation matrix, as shown in formula (10) and formula (11).
S 1 ＝PC 1 P T (10)
S 2 ＝PC 2 P T (11)
Wherein S is 1 Is a first transformation matrix, S 2 Is the second transformation matrix.
S 1 With common eigenvectors and a first transformed diagonal matrix lambda 1 And transforming the eigenvector matrix B, S 2 With common eigenvectors and a second transformed diagonal matrix λ 2 And a transformed eigenvector matrix B. Such asFormula (12) and formula (13).
S 1 ＝Bλ 1 B T (12)
S 2 ＝Bλ 2 B T (13)
Wherein λ is 1 +λ 2 I is an identity matrix. S 1 The feature vector corresponding to the maximum feature value of (1) makes S 2 There is a minimum eigenvalue. The projection matrix W may be obtained using the transformed eigenvector matrix B and the whitening matrix P, as shown in equation (14).
W＝B T P (14)
For the multi-trial multi-band off-line electroencephalogram signal, the feature extraction method is shown in formula (15).
Wherein X i Is the off-line electroencephalogram signal of the ith trial, f i Is the extracted feature matrix, VAR (Z) i ) Represents a pair Z i And (6) calculating the variance.
In the embodiment of the invention, the multi-trial off-line electroencephalogram signal comprises a plurality of off-line electroencephalogram signals, the multi-trial off-line multiband off-line electroencephalogram signal comprises a plurality of multiband off-line electroencephalogram signals, the off-line electroencephalogram signals of a plurality of frequency bands extract characteristic matrixes of a plurality of frequency bands, and each characteristic matrix comprises elements corresponding to a plurality of characteristics respectively.
S13, determining a plurality of characteristic column vectors based on the characteristic matrixes, determining mutual information corresponding to the characteristic column vectors for each characteristic column vector based on the characteristic column vector and the motor imagery category label vector corresponding to the multi-trial off-line electroencephalogram signals, and determining a plurality of target characteristics in the plurality of characteristics according to the mutual information.
In the embodiment of the invention, each feature matrix comprises a plurality of elements, each element belongs to different features, and the elements belonging to the same feature in the plurality of feature matrices form a feature column vector so as to obtain a plurality of feature column vectors. That is, each feature column vector includes an element in each feature matrix, each feature column vector has its corresponding feature, and the features corresponding to any two feature column vectors are different from each other.
The motor imagery category label vector comprises a motor imagery category label corresponding to each tested off-line electroencephalogram signal, and the motor imagery category label is used for reflecting a left hand motor imagery category or a right hand motor imagery category; for example, the motor imagery category corresponding to the offline electroencephalogram of one trial time is a left-hand motor imagery category, a "1" may be used as the motor imagery category tag of the offline electroencephalogram of the trial time, the motor imagery category corresponding to the offline electroencephalogram of one trial time is a right-hand motor imagery category, and a "2" may be used as the motor imagery category tag of the offline electroencephalogram of the trial time. The number of motor imagery class labels included in the motor imagery class label vector is the same as the number of elements included in the feature column vector.
And for each characteristic column vector, determining mutual information according to the characteristic column vector and the motion imagery category label vector, and determining a plurality of target characteristics in a plurality of characteristics corresponding to the characteristic column vectors through the mutual information. The mutual information is used for representing the degree of association between the characteristic column vector and the motor imagery class label vector. Mutual information is calculated according to equation (16).
Wherein I (X, Y) is mutual information between the feature column vector X and the motor imagery class label vector Y, p (X, Y) is a joint probability density function of the feature column vector X and the motor imagery class label vector Y, p (X) is a probability density function of the feature column vector X, and p (Y) is a probability density function of the motor imagery class label vector Y.
The mutual information is in a numerical value form, so that the mutual information has a magnitude relation, all the obtained mutual information is arranged according to a sequence from large to small, a plurality of mutual information arranged in the front row are selected, a plurality of characteristic column vectors corresponding to the selected plurality of mutual information are determined, and a plurality of characteristics corresponding to the plurality of characteristic column vectors are used as a plurality of target characteristics. The number of target features may be set as desired.
S14, determining a plurality of target feature matrixes based on the plurality of target features and the plurality of feature matrixes, and training the initial classifier based on the plurality of target feature matrixes to obtain a trained source classifier.
In the embodiment of the present invention, for each feature matrix, the elements corresponding to the plurality of target features are determined in all the elements included in the feature matrix, and the target feature matrix corresponding to the feature matrix is composed of the elements corresponding to the plurality of target features, that is, only the elements corresponding to the plurality of target features are included in the target feature matrix. The initial classifier may be a Support Vector Machine (SVM) classifier based on a radial basis kernel function, and is trained through a plurality of target feature matrices to obtain a trained source classifier.
S2, receiving the single-test electroencephalogram signals sent by the user based on the preset prompt category of the task feedback game and the subjective screening information corresponding to the single-test electroencephalogram signals, and determining to reserve or reject the single-test electroencephalogram signals based on the subjective screening information.
In an embodiment of the present invention, as shown in fig. 3, the task feedback game includes a checkerboard, and a target block, a plurality of obstacle blocks, and a moving block that are arranged on the checkerboard, and the task feedback game is configured to enable the user to move to the target block while avoiding the plurality of obstacle blocks according to the prompt type, where the prompt type is a left-hand imagination or a right-hand imagination. The task feedback game may be constructed by Qt.
In the embodiment of the present invention, in the task feedback game, the user needs to manipulate the moving block by the motor imagery and travel to the target block without touching the obstacle block. The prompt categories may be Left and Right displayed to the Right of the interface, with the prompt categories giving prompts for Left-hand and Right-hand imagination. When the user performs the left-hand imagination, the moving block moves downwards to the left by one grid. When the user imagines the right hand, the moving block moves one lattice towards the lower right. The prompt will be given in terms of the position of the movement block relative to the obstacle and the board boundary, and will not prompt the user to bump into the movement block or move out of the boundary. If the system fails to give a correct instruction, the moving block is fast to hit the obstacle or goes out of the chessboard boundary, the interface selection dialog box pops up, and the user can select to reset or stop the experiment. After the imagination of the single trial run of the user is finished, the interface pops up a dialog box to prompt the user whether the user carries out the correct imagination in the current trial run.
In the embodiment of the invention, the prompt category is popped up in the task feedback game, and the user performs motor imagery according to the prompt category so that the brain-computer interface system acquires the single-test electroencephalogram signal. And calling a program through a processor of the brain-computer interface system to run the task feedback game, and displaying the prompt category on a display of the brain-computer interface system.
In the embodiment of the invention, after the trained source classifier is obtained, the task feedback game is carried out, the user carries out the motor imagery according to the prompt type sent by the task feedback game, and the user can send the subjective screening information every time the user finishes the motor imagery, wherein the subjective screening information comprises selection or deletion. When the user thinks that there are bad conditions such as vagus, anxiety and interference from the outside in the motor imagery, the subjective screening information including deletion can be sent, and when the user thinks that the motor imagery state is good, the subjective screening information including selection can be sent. When the subjective screening information is selected, reserving the single-test electroencephalogram signal; and when the subjective screening information is deleted, rejecting the single-test electroencephalogram signal.
In one implementation, the brain-computer interface system may display an information prompt box after receiving the single-trial electroencephalogram signal, where the information prompt box is shown in fig. 4. The information prompt box is used for inquiring whether the trial data is reserved or not for the user, and the brain-computer interface system receives subjective screening information sent by the user after displaying the information prompt box.
In one implementation, referring to FIG. 5, the re-played mission feedback game may be referred to as an on-line feedback test, 0-2s of which the user relaxes to observe the current travel position. 2-4s, a prompt category appears on the right side of the interface, and a user prepares what imagination to perform in advance according to the prompt category. 4-10s, the user focuses attention and performs corresponding motor imagery according to the prompt type. And 10, the system pops up an information prompt box to prompt the user to carry out subjective screening. And 10-14s, relaxing the thinking of the user, observing feedback contents of the interface, and evaluating self imagination skills.
And S3, if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or reject the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal.
In the embodiment of the invention, the category comprises correct or wrong, the reserved single-test electroencephalogram signal is input into a source classifier, the motor imagery result corresponding to the single-test electroencephalogram signal is obtained through the source classifier, the motor imagery result comprises left hand imagery or right hand imagery, if the motor imagery result corresponding to the single-test electroencephalogram signal is different from the motor imagery corresponding to the prompt category, the category is judged to be wrong, and if the motor imagery result corresponding to the single-test electroencephalogram signal is the same as the motor imagery corresponding to the prompt category, the category is judged to be correct.
For example, the prompting category is left-hand imagination, and the motor imagination result corresponding to the single-trial electroencephalogram signal is left-hand imagination, so that the category is correct.
In the embodiment of the invention, if the category is wrong, objective screening is carried out to further judge whether the single-test electroencephalogram signal is reserved, and if the category is correct, objective screening is not required to be carried out, and the single-test electroencephalogram signal is directly reserved.
Specifically, step S3 includes:
s31, when the category is correct, reserving the single-test electroencephalogram signal;
and S32, when the category is wrong, determining the motor imagery category corresponding to the single-test-time electroencephalogram signal based on the multi-test-time offline electroencephalogram signal.
In the embodiment of the present invention, when the category is an error, it needs to be determined whether the user motor imagery is an error or the source classifier itself has a problem causing a classification error. The off-line electroencephalogram signals with the multiple test times comprise a plurality of off-line electroencephalogram signals, each off-line electroencephalogram signal has a motor imagery class corresponding to each off-line electroencephalogram signal, and then the motor imagery class corresponding to the electroencephalogram signal with the single test time is determined according to the motor imagery class corresponding to each off-line electroencephalogram signal.
Specifically, step S32 includes:
s321, for each off-line electroencephalogram signal, calculating the relative entropy between the off-line electroencephalogram signal and the single-test electroencephalogram signal.
In the embodiment of the invention, the single-test-time electroencephalogram signal is subjected to dimensionality reduction processing to obtain a single-test-time low-dimensional electroencephalogram signal; performing dimension reduction processing on the off-line electroencephalogram signal to obtain a low-dimensional off-line electroencephalogram signal corresponding to the off-line electroencephalogram signal; specifically, a Principal Component Analysis (PCA) algorithm may be used to perform the dimensionality reduction processing on the single-trial electroencephalogram signal, and similarly, the PCA algorithm may be used to perform the dimensionality reduction processing on the offline electroencephalogram signal.
In the embodiment of the invention, the relative entropy between the low-dimensional off-line electroencephalogram signal and the single-trial low-dimensional electroencephalogram signal is calculated. Relative entropy measures the difference between two probability distributions in the same event space, and when two random distributions are the same, their relative entropy is zero, and when the difference between two random distributions increases, their relative entropy also increases. Relative entropy may thus be used to determine a similarity between the single trial brain signal and each of the plurality of offline brain signals. And calculating the relative entropy between one off-line electroencephalogram signal and the single-test electroencephalogram signal through a formula (17).
Wherein, P (x) is a single-trial low-dimensional electroencephalogram signal, Q (x) is a low-dimensional off-line electroencephalogram signal, and D (P | | Q) is a relative entropy between an off-line electroencephalogram signal and the single-trial electroencephalogram signal.
And S322, dividing all the calculated relative entropies into a first relative entropy set and a second relative entropy set based on the motor imagery categories respectively corresponding to each off-line electroencephalogram signal.
In an embodiment of the present invention, the motor imagery categories include a left-hand motor imagery category and a right-hand motor imagery category, and in an implementation manner, the relative entropy calculated based on the offline electroencephalogram signals corresponding to the left-hand motor imagery category may be divided into a first relative entropy set, and the relative entropy calculated based on the offline electroencephalogram signals corresponding to the right-hand motor imagery category may be divided into a second relative entropy set. That is to say, the motor imagery categories corresponding to all relative entropies in the first relative entropy set are the same, the motor imagery categories corresponding to all relative entropies in the second relative entropy set are the same, and the motor imagery categories corresponding to the first relative entropy set are different from the motor imagery categories corresponding to the second relative entropy set.
S323, calculating the sum of the relative entropies in the first relative entropy set to obtain a first sum value, and calculating the sum of the relative entropies in the second relative entropy set to obtain a second sum value;
s324, determining a target sum value in the first sum value and the second sum value, and taking the operation imagination category corresponding to the target sum value as the motor imagination category corresponding to the single-trial electroencephalogram signal.
In the embodiment of the present invention, the target sum value is the smaller sum value of the first sum value and the second sum value, that is, the smaller sum value of the first sum value and the second sum value is taken as the target sum value. The operation imagination category corresponding to the target sum value is a motor imagination category corresponding to the relative entropy set (the first relative entropy set or the second relative entropy set) corresponding to the target sum value. If the target sum value is the first sum value, the motor imagery class corresponding to the first relative entropy is the operation imagery class corresponding to the target sum value, namely the motor imagery class corresponding to the single-trial electroencephalogram signal; and if the target sum value is the second sum value, the motor imagery class corresponding to the second relative entropy is the operation imagery class corresponding to the target sum value, namely the motor imagery class corresponding to the single-trial electroencephalogram signal.
And S33, if the motor imagery category corresponding to the single-test electroencephalogram signal is consistent with the prompt category, retaining the single-test electroencephalogram signal, and if the motor imagery category corresponding to the single-test electroencephalogram signal is inconsistent with the prompt category, rejecting the single-test electroencephalogram signal.
In the embodiment of the invention, if the motor imagery category corresponding to the single-test electroencephalogram signal is consistent with the prompt category, the fact that the single-test electroencephalogram signal category is wrong is probably because the classifier needs to be improved, and the single-test electroencephalogram signal needs to be reserved; if the motor imagery category corresponding to the single-test electroencephalogram signal is consistent with the prompt category, the problem that the user is not aware of the single-test electroencephalogram signal possibly is shown, and the single-test electroencephalogram signal does not obtain correct motor imagery, the single-test electroencephalogram signal needs to be removed.
In the embodiment of the invention, every time a single-test electroencephalogram signal is received, the brain-computer interface system moves the moving block according to the single-test electroencephalogram signal, and if the moving condition of the moving block is different from the motor imagery of the user, the user can adjust the imagery mode. For example, the user performs the left-hand imagery in the cueing category by imagining "waving the left hand", but the brain-computer interface system recognizes this as right-hand imagery, the user may change to imagining "grasping the left hand" to perform the left-hand imagery.
For ease of illustration, referring to fig. 6, the process of retaining or rejecting individual electroencephalograms by subjective screening and objective screening includes:
if the single-test electroencephalogram signal passes through subjective screening, determining the category of the single-test electroencephalogram signal through a source classifier, and if the category of the single-test electroencephalogram signal is correct, retaining the single-test electroencephalogram signal;
if the category of the single-test electroencephalogram signal is wrong, determining a motor imagery category corresponding to the single-test electroencephalogram signal based on the multi-test offline electroencephalogram signal;
if the motor imagery category corresponding to the single-test electroencephalogram signal is consistent with the prompt category, the single-test electroencephalogram signal is reserved;
if the motor imagery category corresponding to the single-test electroencephalogram signal is inconsistent with the prompt category, rejecting the single-test electroencephalogram signal;
and if the single-test electroencephalogram signal does not pass the subjective screening, rejecting the single-test electroencephalogram signal.
S4, modifying the parameters of the source classifier based on the single-test electroencephalogram signal, and continuing to execute the step of receiving the single-test electroencephalogram signal sent by the user based on the prompting category of the preset task feedback game until a preset training condition is met, so as to obtain the trained target classifier.
In the embodiment of the invention, the single-test electroencephalogram signals obtained through the steps of S2 and S3 are the single-test electroencephalogram signals which are reserved after subjective screening and objective screening. And modifying the parameters of the source classifier by reserving the obtained single-trial electroencephalogram signal, and finishing one training by modifying the parameters of the source classifier once, wherein the preset training condition can be that the training times reach preset times or the source classifier is converged.
In the embodiment of the invention, through subjective screening information, single-test electroencephalograms sent by a user when the user is interfered, not concentrated in attention or tired can be removed, the retained single-test electroencephalograms are all electroencephalograms with high quality, objective screening is carried out on the single-test electroencephalograms subjected to subjective screening, and correct and reliable data can be screened out through the objective screening. Therefore, the reserved single-test electroencephalogram signal is motor imagery made by the user when the user is attentive, and the single-test electroencephalogram signal is accurate and effective and high in quality.
Compared with the classifier based on the single-test electroencephalogram signal training source which is not subjectively screened and objectively screened, the classifier based on the single-test electroencephalogram signal training source which is subjectively screened and objectively screened has smaller calculation amount, can shorten the training time, and eliminates invalid single-test electroencephalogram signals through the subjective screening and the objective screening, thereby avoiding the pollution of the classifier, and obtaining the target classifier through training with higher precision and stronger stability.
Based on the above-mentioned adaptive motor imagery brain-computer interface training method fusing subjective and objective assessment, an embodiment of the present invention further provides a brain-computer interface method, see fig. 7, including:
m1, collecting multi-channel electroencephalogram control signals sent by a user.
M2, inputting the multichannel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multichannel electroencephalogram control signal, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation.
M3, converting the classification result into an operation command, and executing the operation corresponding to the operation command.
In the embodiment of the invention, the classification result corresponding to the multi-channel electroencephalogram control signal is determined through the target classifier, and the operation instruction corresponding to the classification result is executed.
In the embodiment of the invention, in the training method of the self-adaptive motor imagery brain-computer interface fused with subjective and objective evaluation, invalid single-test-time electroencephalogram signals are removed through subjective screening and objective screening, a target classifier obtained through training is higher in precision and higher in stability, and a classification result obtained through the target classifier is more accurate.
The embodiment of the invention also provides computer equipment which can be a terminal, and the internal structure of the computer equipment is shown in figure 8. The computer device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an adaptive motor imagery brain-computer interface training method or a brain-computer interface method that incorporates subjective and objective assessment. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that fig. 8 is a block diagram of only a portion of the structure associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The embodiment of the present invention further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:
acquiring a multi-test off-line electroencephalogram signal generated by a user according to a multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signal to obtain a trained source classifier;
receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information;
if the single-test-time electroencephalogram signal is reserved, determining the category corresponding to the single-test-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal;
modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier;
or, collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multi-channel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multi-channel electroencephalogram control signal, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:
acquiring a multi-test off-line electroencephalogram signal generated by a user according to a multi-test motor imagery, and training an initial classifier based on the multi-test off-line electroencephalogram signal to obtain a trained source classifier;
receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information;
if the single-trial-time electroencephalogram signal is reserved, determining the category corresponding to the single-trial-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-trial-time electroencephalogram signal based on the category and the multi-trial-time off-line electroencephalogram signal;
modifying the parameters of the source classifier based on the single-trial electroencephalogram signal, and continuously executing the step of receiving the single-trial electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier;
or collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multi-channel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multi-channel electroencephalogram control signal, wherein the target classifier is the target classifier in the self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
Claims (7)
1. A self-adaptive motor imagery brain-computer interface training method fused with subjective and objective assessment is characterized by comprising the following steps:
acquiring a multi-trial off-line electroencephalogram signal generated by a user according to a multi-trial motor imagery, and training an initial classifier based on the multi-trial off-line electroencephalogram signal to obtain a trained source classifier;
receiving a single-test electroencephalogram signal sent by the user based on a preset prompt category of a task feedback game and subjective screening information corresponding to the single-test electroencephalogram signal, and determining to reserve or eliminate the single-test electroencephalogram signal based on the subjective screening information;
if the single-trial-time electroencephalogram signal is reserved, determining the category corresponding to the single-trial-time electroencephalogram signal based on the source classifier and the prompt category, and selecting to reserve or eliminate the single-trial-time electroencephalogram signal based on the category and the multi-trial-time off-line electroencephalogram signal;
modifying the parameters of the source classifier based on the single-trial-time electroencephalogram signal, and continuously executing the step of receiving the single-trial-time electroencephalogram signal sent by the user based on the prompt category of the preset task feedback game until a preset training condition is met, so as to obtain a trained target classifier;
the subjective screening information comprises selection or deletion; the determining to reserve or reject the single-test electroencephalogram signal based on the subjective screening information specifically comprises the following steps:
when the subjective screening information is selected, reserving the single-test electroencephalogram signal;
when the subjective screening information is deleted, the single-test electroencephalogram signal is removed;
the categories include correct or incorrect; the selecting, reserving or rejecting the single-test-time electroencephalogram signal based on the category and the multi-test-time offline electroencephalogram signal specifically comprises:
when the category is correct, the single-trial electroencephalogram signal is reserved;
when the category is wrong, determining a motor imagery category corresponding to the single-test-time electroencephalogram signal based on the multi-test-time offline electroencephalogram signal;
if the motor imagery category corresponding to the single-trial electroencephalogram signal is consistent with the prompting category, retaining the single-trial electroencephalogram signal, and if the motor imagery category corresponding to the single-trial electroencephalogram signal is inconsistent with the prompting category, rejecting the single-trial electroencephalogram signal;
the off-line electroencephalogram signals with multiple test times comprise a plurality of off-line electroencephalogram signals, and each off-line electroencephalogram signal has a motor imagery category which corresponds to each off-line electroencephalogram signal; the determining of the motor imagery category corresponding to the single-test-time electroencephalogram signal based on the multi-test-time offline electroencephalogram signal specifically comprises the following steps:
for each off-line electroencephalogram signal, calculating the relative entropy between the off-line electroencephalogram signal and the single-test electroencephalogram signal;
dividing all the calculated relative entropies into a first relative entropy set and a second relative entropy set based on the motor imagery categories respectively corresponding to each off-line electroencephalogram signal;
calculating the sum of the relative entropies in the first relative entropy set to obtain a first sum value, and calculating the sum of the relative entropies in the second relative entropy set to obtain a second sum value;
and determining a target sum value in the first sum value and the second sum value, and taking the operation imagination category corresponding to the target sum value as the motor imagination category corresponding to the single-trial electroencephalogram signal.
2. The method for training the adaptive motor imagery brain-computer interface fused with subjective and objective assessment according to claim 1, wherein the training of the initial classifier based on the multi-trial off-line electroencephalogram signal to obtain the trained source classifier specifically includes:
preprocessing the multi-trial off-line electroencephalogram signals to obtain multi-trial multi-frequency-band off-line electroencephalogram signals;
extracting a plurality of feature matrixes corresponding to the multi-trial multiband off-line electroencephalogram signal by adopting a common space mode algorithm, wherein each feature matrix comprises elements corresponding to a plurality of features respectively;
determining a plurality of characteristic column vectors based on the plurality of characteristic matrixes, determining mutual information corresponding to each characteristic column vector based on the characteristic column vector and a motor imagery category label vector corresponding to the multi-trial off-line electroencephalogram signal, and determining a plurality of target characteristics in the plurality of characteristics according to the plurality of mutual information;
and determining a plurality of target feature matrixes based on the plurality of target features and the plurality of feature matrixes, and training the initial classifier based on the plurality of target feature matrixes to obtain a trained source classifier.
3. The method for training the adaptive motor imagery brain-computer interface fused with subjective and objective assessment according to claim 1, wherein the calculating the relative entropy between the off-line electroencephalogram signal and the single-trial electroencephalogram signal specifically includes:
performing dimensionality reduction processing on the single-trial electroencephalogram signal to obtain a single-trial low-dimensional electroencephalogram signal;
performing dimension reduction processing on the off-line electroencephalogram signal to obtain a low-dimensional off-line electroencephalogram signal corresponding to the off-line electroencephalogram signal;
and calculating the relative entropy between the low-dimensional off-line electroencephalogram signal and the single-test-time low-dimensional electroencephalogram signal.
4. The method for training an adaptive motor imagery brain-computer interface fused with subjective and objective assessment according to claim 1, wherein the task feedback game includes a checkerboard, target blocks, a plurality of obstacle blocks and moving blocks arranged on the checkerboard; the task feedback game is used for enabling the user to move to the target block under the condition of avoiding the plurality of obstacle blocks according to prompt information, and the prompt information is left hand imagination or right hand imagination.
5. A brain-computer interface method, comprising:
collecting a multi-channel electroencephalogram control signal sent by a user;
inputting the multi-channel electroencephalogram control signal into the target classifier to obtain a classification result corresponding to the multi-channel electroencephalogram control signal, wherein the target classifier is the target classifier according to any one of claims 1 to 4;
and converting the classification result into an operation command, and executing the operation corresponding to the operation command.
6. A brain-computer interface system comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the steps of the adaptive motor imagery brain-computer interface training method incorporating subjective and objective assessment according to any one of claims 1 to 4 or the brain-computer interface method according to claim 5.
7. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the adaptive motor imagery brain-computer interface training method with fused subjective and objective assessment according to any one of claims 1 to 4 or the steps of claim 5 based on the brain-computer interface method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110457656.6A CN113220120B (en) | 2021-04-27 | 2021-04-27 | Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110457656.6A CN113220120B (en) | 2021-04-27 | 2021-04-27 | Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113220120A CN113220120A (en) | 2021-08-06 |
| CN113220120B true CN113220120B (en) | 2022-08-30 |
Family
ID=77089692
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110457656.6A Active CN113220120B (en) | 2021-04-27 | 2021-04-27 | Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113220120B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114366101B (en) * | 2021-12-31 | 2024-05-03 | 西安臻泰智能科技有限公司 | Motor imagery electroencephalogram signal classification method, device, equipment and storage medium |
| CN116522210B (en) * | 2023-07-03 | 2023-09-01 | 中国医学科学院生物医学工程研究所 | Motor imagery electroencephalogram signal classification method based on brain network difference analysis |
| CN117643475B (en) * | 2024-01-30 | 2024-04-16 | 南京信息工程大学 | Feature extraction method based on KL divergence |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106502405A (en) * | 2016-10-24 | 2017-03-15 | 天津大学 | Based on the compound limbs Imaginary Movement multi-mode brain-computer interface method and system of synchronicity |
| WO2017084416A1 (en) * | 2015-11-17 | 2017-05-26 | 天津大学 | Feedback system based on motor imagery brain-computer interface |
| CN109992113A (en) * | 2019-04-09 | 2019-07-09 | 燕山大学 | A kind of MI-BCI system and its control method induced based on more scenes |
| CN110390272A (en) * | 2019-06-30 | 2019-10-29 | 天津大学 | A kind of EEG signal feature dimension reduction method based on weighted principal component analyzing |
| CN111310656A (en) * | 2020-02-13 | 2020-06-19 | 燕山大学 | Single motor imagery electroencephalogram signal identification method based on multi-linear principal component analysis |
| CN112465059A (en) * | 2020-12-07 | 2021-03-09 | 杭州电子科技大学 | Multi-person motor imagery identification method based on cross-brain fusion decision and brain-computer system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11402905B2 (en) * | 2018-01-09 | 2022-08-02 | Holland Bloorview Kids Rehabilitation Hospital | EEG brain-computer interface platform and process for detection of changes to mental state |
-
2021
- 2021-04-27 CN CN202110457656.6A patent/CN113220120B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084416A1 (en) * | 2015-11-17 | 2017-05-26 | 天津大学 | Feedback system based on motor imagery brain-computer interface |
| CN106502405A (en) * | 2016-10-24 | 2017-03-15 | 天津大学 | Based on the compound limbs Imaginary Movement multi-mode brain-computer interface method and system of synchronicity |
| CN109992113A (en) * | 2019-04-09 | 2019-07-09 | 燕山大学 | A kind of MI-BCI system and its control method induced based on more scenes |
| CN110390272A (en) * | 2019-06-30 | 2019-10-29 | 天津大学 | A kind of EEG signal feature dimension reduction method based on weighted principal component analyzing |
| CN111310656A (en) * | 2020-02-13 | 2020-06-19 | 燕山大学 | Single motor imagery electroencephalogram signal identification method based on multi-linear principal component analysis |
| CN112465059A (en) * | 2020-12-07 | 2021-03-09 | 杭州电子科技大学 | Multi-person motor imagery identification method based on cross-brain fusion decision and brain-computer system |
Non-Patent Citations (1)
| Title |
|---|
| 注意定向功能评估及其与危险驾驶的关系;颜莉蓉;《汽车工程》;20210425;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113220120A (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113220120B (en) | Self-adaptive motor imagery brain-computer interface training method fusing subjective and objective evaluation | |
| Ko et al. | Multi-scale neural network for EEG representation learning in BCI | |
| Delorme et al. | EEGLAB, SIFT, NFT, BCILAB, and ERICA: new tools for advanced EEG processing | |
| KR102143900B1 (en) | System for providing subject-independent brain-computer interface and method thereof | |
| US8880163B2 (en) | Method and a system for classifying neural signals, and a method of selecting electrodes for direct neural control | |
| Xu et al. | High accuracy classification of EEG signal | |
| CN111265212A (en) | Motor imagery electroencephalogram signal classification method and closed-loop training test interaction system | |
| Singh et al. | Small sample motor imagery classification using regularized Riemannian features | |
| CN111317468A (en) | Electroencephalogram signal classification method and device, computer equipment and storage medium | |
| CN104503592A (en) | Method and device for determining head gestures | |
| CN111680620A (en) | Human-computer interaction intention identification method based on D-S evidence theory | |
| Coyle | Neural network based auto association and time-series prediction for biosignal processing in brain-computer interfaces | |
| CN113191395B (en) | Target detection method based on multi-level information fusion of double brains | |
| US11717163B2 (en) | Wearable device, signal processing method and device | |
| Das et al. | An effect-size based channel selection algorithm for mental task classification in brain computer interface | |
| KR102443961B1 (en) | Apparatus and method for motor imagery classification using eeg | |
| CN116541751B (en) | Electroencephalogram signal classification method based on brain function connection network characteristics | |
| Dai et al. | Electrode channel selection based on backtracking search optimization in motor imagery brain–computer interfaces | |
| Lan et al. | Estimating cognitive state using EEG signals | |
| Forney et al. | Echo state networks for modeling and classification of EEG signals in mental-task brain-computer interfaces | |
| Wei et al. | The backtracking search optimization algorithm for frequency band and time segment selection in motor imagery-based brain–computer interfaces | |
| Shoker et al. | Distinguishing between left and right finger movement from EEG using SVM | |
| Hossain et al. | Calibration time reduction using subjective features selection based transfer learning for multiclass BCI | |
| KR102594818B1 (en) | Eigenface Electro-Encephalographic signal classification method using covariance de-centering and EEG signal classification device using the same | |
| Garakani et al. | Control of a 2-DOF robotic arm using a P300-based brain-computer interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |