CN115429289A - Brain-computer interface training data amplification method, device, medium and electronic equipment - Google Patents
Brain-computer interface training data amplification method, device, medium and electronic equipment Download PDFInfo
- Publication number
- CN115429289A CN115429289A CN202211071579.1A CN202211071579A CN115429289A CN 115429289 A CN115429289 A CN 115429289A CN 202211071579 A CN202211071579 A CN 202211071579A CN 115429289 A CN115429289 A CN 115429289A
- Authority
- CN
- China
- Prior art keywords
- signal
- subset
- matrix
- electroencephalogram
- brain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003321 amplification Effects 0.000 title claims abstract description 116
- 238000003199 nucleic acid amplification method Methods 0.000 title claims abstract description 116
- 238000012549 training Methods 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 64
- 239000011159 matrix material Substances 0.000 claims abstract description 226
- 238000012360 testing method Methods 0.000 claims abstract description 35
- 210000004556 brain Anatomy 0.000 claims abstract description 21
- 230000006870 function Effects 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 14
- 238000002360 preparation method Methods 0.000 claims description 7
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000013145 classification model Methods 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 230000000638 stimulation Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000001054 cortical effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002582 magnetoencephalography Methods 0.000 description 2
- 238000003333 near-infrared imaging Methods 0.000 description 2
- 210000000653 nervous system Anatomy 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000000578 peripheral nerve Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- UFHFLCQGNIYNRP-UHFFFAOYSA-N Hydrogen Chemical compound [H][H] UFHFLCQGNIYNRP-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 229910052739 hydrogen Inorganic materials 0.000 description 1
- 239000001257 hydrogen Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/372—Analysis of electroencephalograms
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Psychology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Psychiatry (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The application provides a brain-computer interface training data amplification method, a brain-computer interface training data amplification device, a brain-computer interface training data amplification medium and electronic equipment. The brain-computer interface training data amplification method comprises the following steps: acquiring an original signal subset included in an electroencephalogram signal set to be processed; wherein, when acquiring an original signal subset, the following operations are executed for the acquired original signal subset to obtain an amplified signal subset for constituting an amplified electroencephalogram signal set: determining the mean value of the brain electrical signal unit corresponding to the acquired original signal subset based on the acquired original signal subset; obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram signal unit average value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset. The method can improve the recognition performance of the brain-computer interface under the condition of small samples.
Description
Technical Field
The present application relates to the field of brain-computer interface technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for amplifying brain-computer interface training data.
Background
The brain-computer interface (BCI) provides a direct communication path between the brain and the external environment without depending on peripheral nerves, and can replace, repair, enhance, supplement or improve normal nervous system functions. Typical brain-computer interface systems decode brain information by detecting specific neural activity and translate it into machine instructions that can be output, thereby enabling direct expression of brain intent.
Compared with brain information acquisition modes such as functional near-infrared imaging, magnetoencephalography and cortical electroencephalogram, electroencephalogram (EEG) signals have the advantages of being non-invasive, low in price and high in time resolution, and therefore the EEG signals are widely applied to the field of brain-computer interfaces. However, limited by the variability of the electroencephalogram signals, such as across-modality, across-individual, across-time, and the like, the current brain-computer interface decoding algorithm mostly adopts an individual calibration mode, that is, before each use of the system, the user is required to re-acquire the training signals to construct a classification model conforming to the characteristics of the current electroencephalogram.
The brain-computer interface recognition scheme of the related art obtains EEG signal data for calibration through experimental acquisition. This approach is time consuming, less comfortable, and requires a high investment in human costs, making it difficult to obtain enough calibration data to train the classification model. The problem of insufficient data in the electroencephalogram signal decoding process easily causes low accuracy of electroencephalogram signal identification, and blocks the brain-computer interface system from being applied to practical application.
Disclosure of Invention
The embodiment of the application provides a brain-computer interface training data amplification method, a brain-computer interface training data amplification device, a brain-computer interface training data amplification medium and electronic equipment, and aims to solve the problem of insufficient data in the existing electroencephalogram signal decoding process, improve the recognition performance of a brain-computer interface under the condition of a small sample and further reduce the calibration burden of a system.
In a first aspect, an embodiment of the present application provides a method for amplifying brain-computer interface training data, including:
acquiring an original signal subset included in an electroencephalogram signal set to be processed;
wherein, each time one of the original signal subsets is acquired, the following operations are performed on the acquired original signal subsets to obtain an amplified signal subset used for constituting an amplified electroencephalogram signal set:
determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental test times;
obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value;
obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix;
and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
The brain-computer interface training data amplification method provided by the embodiment of the application can be used for reconstructing the electroencephalogram signals by acquiring the original signal subsets contained in the electroencephalogram signal sets to be processed and based on the reference matrix, the random noise signal sets and the average value of the electroencephalogram signal units corresponding to the acquired original signal subsets, so that the amplification signals conforming to the electroencephalogram characteristics are acquired, efficient data amplification of the electroencephalogram signal sets to be processed in the brain-computer interface training data is realized, the recognition performance of the brain-computer interfaces under the condition of small samples is improved, and the calibration burden of the system is reduced.
In one possible implementation, the determining, based on the acquired subset of original signals, a mean value of a brain electrical signal unit corresponding to the acquired subset of original signals includes:
summing the electroencephalogram signal units included in the acquired original signal subset to obtain a first electroencephalogram signal corresponding to the acquired original signal subset;
and carrying out quotient calculation on the first electroencephalogram signal and the experimental trial times to obtain an electroencephalogram signal unit average value corresponding to the acquired original signal subset.
In the brain-computer interface training data amplification method provided by this embodiment, the electroencephalogram signal units included in the acquired original signal subset are summed up to obtain a first electroencephalogram signal corresponding to the acquired original signal subset; and carrying out quotient calculation on the first electroencephalogram signal and the experimental trial times to obtain an electroencephalogram signal unit average value corresponding to the acquired original signal subset. The method can effectively reduce irrelevant background noise by averaging the original signal subsets.
In one possible implementation, the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
In the method provided in this embodiment, the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit. The method solves the source aliasing matrix by using the least square method, has simple calculation and easy realization, can reduce the calculated amount in the brain-computer interface training data amplification process, and improves the calibration efficiency of the brain-computer interface.
In a possible implementation manner, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularized norm of the source aliasing matrix; and the second deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
In the method provided by this embodiment, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularization norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the EEG unit average value. According to the method, the regularized norm is added into the target function, so that the source aliasing matrix can be restrained, parameters are thinned, the stability of numerical values in the process of solving the source aliasing matrix is enhanced, and the effectiveness of amplification data obtained by performing data amplification on the electroencephalogram signal set to be processed in brain-computer interface training data is improved.
In a possible implementation manner, the obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix includes:
and performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.
In the method provided by this embodiment, the target source aliasing matrix and the reference matrix are subjected to matrix multiplication to obtain the reconstructed source signal. According to the method, matrix multiplication is carried out on the target source aliasing matrix and the reference matrix to obtain a reconstructed source signal, the source signal related to the current training task can be reconstructed, the source signal can be used in the subsequent signal amplification process, and efficient data amplification can be carried out on an electroencephalogram signal set to be processed in brain-computer interface training data.
In a possible implementation manner, the obtaining, based on the reconstructed source signal, a random noise signal set, a preset number of amplification tests, and the acquired original signal subset, the amplified signal subset corresponding to the acquired original signal subset includes:
generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set comprises the number of random noise signal units which is the number of times of the amplification test;
acquiring the random noise signal units from the random noise signal set one by one, and executing a first operation when acquiring one random noise signal unit to obtain an amplification signal unit corresponding to the acquired original signal subset; the first operation comprises summing the acquired random noise signal unit with the reconstructed source signal;
and adding each obtained amplification signal unit serving as a new electroencephalogram signal unit into the obtained original signal subset to obtain the amplification signal subset corresponding to the obtained original signal subset.
In the method provided in this embodiment, a random noise signal set is generated based on the amplification test times and a preset rule; summing the acquired random noise signal unit and the reconstruction source signal to obtain an amplification signal unit; and adding each obtained amplification signal unit serving as a new electroencephalogram signal unit into the obtained original signal subset to obtain the amplification signal subset corresponding to the obtained original signal subset.
In one possible implementation, the method further includes:
training an electroencephalogram recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram recognition model;
acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
The method provided by this embodiment further includes: training an electroencephalogram recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram recognition model; acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result. The method realizes the efficient data amplification of the electroencephalogram signal set to be processed in the brain-computer interface training data, and trains the electroencephalogram signal recognition model according to the amplified electroencephalogram signal set obtained by data amplification, so that the recognition performance of the brain-computer interface under the condition of a small sample can be improved, and the calibration burden of the system is reduced.
In a second aspect, an embodiment of the present application provides a brain-computer interface training data amplification apparatus, including:
the data preparation module is used for acquiring an original signal subset included in the electroencephalogram signal set to be processed;
a data amplification module, configured to, when the data preparation module acquires each original signal subset, perform the following operations on the acquired original signal subset to obtain an amplified signal subset used to form an amplified electroencephalogram signal set:
determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram units with the number of experimental trial times; obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
In a possible implementation manner, the data amplification module is specifically configured to:
summing the electroencephalogram signal units included in the acquired original signal subset to obtain a first electroencephalogram signal corresponding to the acquired original signal subset;
and obtaining the average value of the electroencephalogram signal units corresponding to the acquired original signal subset by taking the quotient of the first electroencephalogram signal and the experimental trial times.
In one possible implementation, the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
In a possible implementation manner, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularized norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the EEG unit average value.
In one possible implementation, the data amplification module is specifically configured to:
and performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.
In one possible implementation, the data amplification module is specifically configured to:
generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set comprises the number of random noise signal units which is the number of times of the amplification test;
acquiring the random noise signal units from the random noise signal set one by one, and executing a first operation every time one random noise signal unit is acquired so as to obtain an amplification signal unit corresponding to the acquired original signal subset; the first operation comprises summing the acquired random noise signal unit with the reconstructed source signal;
and adding each obtained amplification signal unit serving as a new electroencephalogram signal unit into the obtained original signal subset to obtain the amplification signal subset corresponding to the obtained original signal subset.
In one possible implementation, the apparatus further includes:
the electroencephalogram signal recognition module is used for training an electroencephalogram signal recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram signal recognition model; acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
In a third aspect, this application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the method of any one of the first aspect.
In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program that is executable on the processor, and when the computer program is executed by the processor, the electronic device implements the method of any one of the first aspects.
In a fifth aspect, embodiments of the present application provide a computer program product, which includes computer instructions stored in a computer-readable storage medium; when the processor of the computer device reads the computer instructions from the computer readable storage medium, the processor executes the computer instructions, causing the computer device to perform the steps of the method of any of the first aspects.
The technical effects brought by any one of the implementation manners of the second aspect to the fifth aspect may be referred to the technical effects brought by the implementation manner of the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a method for amplifying brain-computer interface training data according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of determining an average value of an electroencephalogram signal unit according to a brain-computer interface training data amplification method provided in an embodiment of the present application;
fig. 3 is a schematic flowchart of obtaining an amplified signal subset according to an embodiment of the present disclosure;
fig. 4 is a schematic flowchart of another method for amplifying brain-computer interface training data according to an embodiment of the present application;
fig. 5 is a block diagram illustrating a structure of a brain-computer interface training data amplification apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of another structure of an apparatus for augmenting training data of a brain-computer interface according to an embodiment of the present application;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The brain-computer interface (BCI) provides a direct communication path between the brain and the external environment without depending on peripheral nerves, and can replace, repair, enhance, supplement or improve normal nervous system functions. Typical brain-computer interface systems decode brain information by detecting specific neural activity and translate it into machine instructions that can be output, thereby enabling direct expression of brain intent.
Compared with brain information acquisition modes such as functional near-infrared imaging, magnetoencephalography and cortical electroencephalogram, electroencephalogram (EEG) signals have the advantages of being non-invasive, low in price and high in time resolution, and therefore the EEG signals are widely applied to the field of brain-computer interfaces. However, limited by the variability of the electroencephalogram signals, such as across-modality, across-individual, across-time, and the like, the current brain-computer interface decoding algorithm mostly adopts an individual calibration mode, that is, before each use of the system, the user is required to re-acquire the training signals to construct a classification model conforming to the characteristics of the current electroencephalogram.
The brain-computer interface recognition scheme of the related art obtains EEG signal data for calibration through experimental acquisition. This approach is time consuming, less comfortable, and requires a high investment in human costs, making it difficult to obtain enough calibration data to train the classification model. The problem of insufficient data in the electroencephalogram signal decoding process easily causes low accuracy of electroencephalogram signal identification, and blocks the brain-computer interface system from being applied to practical application.
Based on this, embodiments of the present application provide a brain-computer interface training data amplification method, apparatus, medium, and electronic device. The brain-computer interface training data amplification method comprises the following steps: acquiring an original signal subset included in a brain electrical signal set to be processed; wherein, each time one original signal subset is acquired, the following operations are executed for the acquired original signal subset to obtain an amplified signal subset used for forming an amplified electroencephalogram signal set: determining the average value of electroencephalogram signal units corresponding to the acquired original signal subsets on the basis of the acquired original signal subsets, wherein any original signal subset comprises electroencephalogram signal units with the number of experimental trials; obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset. According to the brain-computer interface training data amplification method, the original signal subsets included in the brain-computer signal set to be processed are obtained, the brain-computer signal is reconstructed on the basis of the reference matrix, the random noise signal set and the mean value of the brain-computer signal units corresponding to the obtained original signal subsets, and the amplification signals conforming to the characteristics of the brain-computer are obtained, so that the brain-computer signal set to be processed in the brain-computer interface training data is efficiently subjected to data amplification, the recognition performance of a brain-computer interface under the condition of a small sample can be improved, and the calibration burden of a system is reduced.
In order to make the objects, technical solutions and advantages of the embodiments of the present application more apparent, the present application will be described in further detail with reference to the accompanying drawings. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
The following further explains the brain-computer interface training data amplification method provided in the embodiments of the present application. The brain-computer interface training data amplification method provided by the application is shown in fig. 1 and comprises the following steps:
step S101, obtaining an original signal subset included in the electroencephalogram signal set to be processed.
Wherein, each time one original signal subset is acquired, the following steps are executed for the acquired original signal subset to obtain an amplified signal subset for forming an amplified electroencephalogram signal set.
In specific implementation, the electroencephalogram signal set to be processed can be a preprocessed signal of an input brain-computer interface; the original signal subset included in the electroencephalogram signal set to be processed can be a category of training data determined from the preprocessed signals of the input brain-computer interface.
The input preprocessed signal is expressed as a four-dimensional tensor
Wherein, N c Number of leads representing electroencephalogram signals, N s Number of sampling points, N, representing an electroencephalogram signal t Representing the number of experimental trials, representing the number of trials collected in the repeated experiments under the same category, N f Represents the total number of categories of the video,
representing a set of real numbers. Obtaining the original signal subset included in the electroencephalogram signal set to be processed, which can be from four-dimensional tensor
In (1), the training signal under the nth class is determined and expressed as
The following electroencephalogram signal set to be processed is
The subset of the original signal obtained is
The description is given for the sake of example.
Step S102, determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental trials.
In an alternative embodiment, the process of determining the mean of the brain electrical signal units corresponding to the acquired subset of raw signals based on the acquired subset of raw signals, as shown in FIG. 2, may include the following steps:
step S201, the electroencephalogram signal units included in the acquired original signal subset are summed to obtain a first electroencephalogram signal corresponding to the acquired original signal subset.
Illustratively, based on the acquired subset of the original signal
Determining and acquiring a subset of original signals
The corresponding EEG signal unit average value process can be to obtain the original signal subset
Comprises an electroencephalogram signal unit
Summing to obtain a pair of original signal subsetsFirst brain electrical signal
And S202, quoting the first electroencephalogram signal and the experimental trial times to obtain an electroencephalogram signal unit average value corresponding to the acquired original signal subset.
Illustratively, a first brain electrical signal is
And number of experimental trials N t Obtaining the original signal subset obtained by quotient finding
Corresponding EEG signal unit mean value
In an embodiment of the application, a training signal in the nth class is determined
Thereafter, the average value of the EEG signal units in the nth class can be calculated according to the following formula
Wherein, N t For the number of experimental trials, the number of trials collected for repeated experiments under the same category is characterized.
S103, obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix that satisfies a minimum value for a preset objective function.
In an embodiment of the present application, the predetermined reference matrix may be a known frequency f in the nth category n Sine and cosine sequences of
As shown in the following formula:
wherein, N h Representing the harmonic order contained in the reference matrix;
F s representing the sampling rate of the signal.
In specific implementation, the reference matrix Y is based on the preset reference matrix Y n And the mean value of the EEG signal unit
Obtaining a target source aliasing matrix
Target source aliasing matrix
Is a source aliasing matrix that satisfies the minimum for the preset objective function.
In an embodiment of the application, the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on the source aliasing matrix and the reference matrix and then performing difference on the mean value of the electroencephalogram signal unit.
Illustratively, the first bias matrix is formed by combining the source aliasing matrix Φ with the reference matrix Y n Performing matrix multiplication to obtain phi Y n And then the obtained phi Y is n Mean value of electroencephalogram signal unit
Obtained by making a difference. The objective function is a Frobenius norm of the first deviation matrix.
In this embodiment, the source aliasing matrix may be estimated according to the following equation
And determining a source aliasing matrix which enables a preset target function to take the minimum value so as to obtain the target source aliasing matrix.
Wherein the argmin function is used to search for a variable value that minimizes the objective function;
‖‖ F a Frobenius norm representing a matrix;
It should be noted that, the embodiments of the present application only take sine and cosine sequences as an example to reference matrix Y n For illustrative purposes, the construction of the reference matrix, including but not limited to the above form, should be determined according to different categories of electroencephalogram characteristics, and the present application is not limited in particular.
In an embodiment of the application, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularized norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on the source aliasing matrix and the reference matrix and then performing difference on the mean value of the electroencephalogram signal unit.
Illustratively, the second deviation matrix is formed by combining the source aliasing matrix Φ with the reference matrix Y n Performing matrix multiplication to obtain phi Y n Then the obtained phi Y is processed n Mean value of electroencephalogram signal unit
Obtained by making a difference. TargetThe function is the Frobenius norm of the second deviation matrix and is obtained by summing the regularized norm of the source aliasing matrix.
In this embodiment, assume the second bias matrix is
The regularized norm is the L1 norm, and the source aliasing matrix can be estimated according to the following formula
And determining a source aliasing matrix which meets the condition that the preset target function takes the minimum value to obtain a target source aliasing matrix.
Wherein the argmin function is used to search for a variable value that minimizes the objective function;
‖‖ F represents the Frobenius norm of the matrix;
‖‖ 1 represents the L1 norm of the matrix;
It should be noted that, in the embodiment of the present application, the regularization norm included in the objective function is only explained by taking the L1 norm as an example, but the construction of the objective function includes, but is not limited to, the above form, and should be determined according to characteristics of different classes of electroencephalogram signals, and the present application is not limited specifically. For example, in some other embodiments, the regularization norm may also be an L2 regularization norm.
And step S104, obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix.
In an optional embodiment, the reconstructed source signal is obtained based on the target source aliasing matrix and the reference matrix, specifically, the reconstructed source signal is obtained by performing matrix multiplication on the target source aliasing matrix and the reference matrix.
Illustratively, based on the determined reference matrix Y n And target source aliasing matrix
Calculating a reconstructed source signal by the following formula
Wherein the content of the first and second substances,
Y n is a reference matrix;
Step S105, obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
In an alternative embodiment, the process of obtaining an amplified signal subset corresponding to the acquired original signal subset based on the reconstructed source signal, the random noise signal set, the preset number of amplification tests and the acquired original signal subset may be implemented by the following steps as shown in fig. 3:
step S301, generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set contains the number of random noise signal units as the number of amplification tests.
In some embodiments of the present application, the preset rule may be a rule for generating a random noise signal set satisfying a multivariate gaussian distribution with a mean value of 0 and a covariance of Σ, where the random noise signal set includes a number of random noise signal units as the number of amplification tests.
Illustratively, the random noise signal set is generated based on the amplification test times and a preset rule, and the random noise signal set can be based on the amplification test times N a And a preset rule for generating a random noise signal set
Wherein N is k A multivariate gaussian distribution with mean 0 and covariance Σ is satisfied.
Step S302, random noise signal units are acquired from a random noise signal set one by one, and when each random noise signal unit is acquired, a first operation is executed to obtain an amplification signal unit corresponding to the acquired original signal subset; a first operation includes summing the acquired random noise signal unit with a reconstructed source signal.
Illustratively, from a random noise signal set
In-line random noise signal acquisition unit
Each time a random noise signal unit N is acquired k Performing a first operation to obtain a subset X of the original signals to be acquired n A corresponding amplification signal unit
A first operation comprising a random noise signal unit N to be acquired k And reconstructing the source signal S n And (6) summing.
I.e. the amplified signal units corresponding to the acquired subsets of the original signal
Can be determined by the following formula:
wherein, the first and the second end of the pipe are connected with each other,
represents the kth amplified signal under the nth class,
S n represents the reconstructed source signal in the nth class,
N a representing the number of amplification tests.
Step S303, adding each obtained amplification signal unit as a new electroencephalogram signal unit into the obtained original signal subset to obtain an amplification signal subset corresponding to the obtained original signal subset.
Illustratively, each amplified signal unit obtained
Adding to the acquired subset of original signals as a new EEG signal unit
Deriving and acquiring a subset of the original signal
Corresponding amplified signal subsets
By means of the EEG signal set to be processed
Containing N f Original signal subset of individual classes
The above-mentioned signal amplification is performed one by category,can obtain the amplification signal subsets corresponding to each original signal subset, and further obtain the electroencephalogram signal set to be processed
Corresponding amplified EEG signal set
Embodiments of the present application also provide another method for augmenting brain-computer interface training data. As shown in fig. 4, the method comprises the following steps:
step S401, obtaining an original signal subset included in the electroencephalogram signal set to be processed.
Wherein, every time an original signal subset is acquired, the following operations of steps S402-S405 are performed on the acquired original signal subset to obtain an amplified signal subset for constituting an amplified electroencephalogram signal set.
In specific implementation, the electroencephalogram signal set to be processed can be a preprocessed signal of an input brain-computer interface; the original signal subset included in the electroencephalogram signal set to be processed can be a category of training data determined from the preprocessed signals of the input brain-computer interface.
The input preprocessed signal is expressed as a four-dimensional tensor
Wherein, N c Number of leads representing electroencephalogram signals, N s Number of sampling points, N, representing the electroencephalogram signal t Representing the number of experimental trials, representing the number of trials collected in the repeated experiment in the same category, N f Represents the total number of categories of the video,
representing a set of real numbers. Obtaining the original signal subset included in the electroencephalogram signal set to be processed, which can be from four-dimensional tensor
In (1), the training signal under the nth class is determined and expressed as
The following electroencephalogram signal set to be processed is
The subset of the original signal obtained is
The description is given by way of example. Each time a subset X of the original signal is acquired n For the original signal subset X acquired n The following operations of steps S402-S405 are performed to obtain and acquire the original signal subset X n Corresponding amplified signal subsets
Thereby obtaining an electroencephalogram signal set to be processed
Corresponding amplified EEG signal set
Step S402, determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental trials.
Step S403, obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix that satisfies the minimum value of a preset objective function.
And S404, obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix.
Step S405, obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
And S406, training an electroencephalogram signal recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram signal recognition model.
Illustratively, based on an amplified electroencephalogram signal set
And training the brain electrical signal recognition model of the brain-computer interface to obtain the trained brain electrical signal recognition model.
Step S407, acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
The method of the embodiment realizes efficient data amplification of the electroencephalogram signal set to be processed in the brain-computer interface training data, and carries out electroencephalogram signal recognition model training according to the amplified electroencephalogram signal set obtained by data amplification, so that the recognition performance of the brain-computer interface under the condition of a small sample can be improved, and the calibration burden of the system is reduced.
In one embodiment of the present application, the brain-computer interface is a frequency-phase encoded brain-computer interface. As shown in Table 1, each command of the frequency-phase coding brain-computer interface is coded by flicker stimulation with different frequencies and different phases, and the induced brain electrical signal comprises a fundamental wave and a plurality of harmonic components with corresponding frequencies.
TABLE 1
In the embodiment, a Filter Bank Task Related Component Analysis (FBTRCA) algorithm is applied to classify and identify the electroencephalogram signals, wherein the filter bank analysis requires that the signals are filtered according to different sub-bands, so that harmonic information in the electroencephalogram signals is fully utilized. The brain-computer interface training data amplification method of the frequency-phase coding brain-computer interface comprises the following steps:
and A01, acquiring an original signal subset included in the electroencephalogram signal set to be processed.
Wherein, every time an original signal subset is acquired, the following operations of steps A02-A05 are executed aiming at the acquired original signal subset to obtain an amplified signal subset used for forming an amplified electroencephalogram signal set.
In specific implementation, the electroencephalogram signal of the frequency-phase coding brain-computer interface is firstly subjected to filter bank decomposition to obtain training signals corresponding to a plurality of sub-bands, and the training signal corresponding to each sub-band is an electroencephalogram signal set to be processed.
Illustratively, the preprocessed electroencephalogram signal input in this embodiment includes 9 leads of Pz, PO5, PO3, POz, PO4, PO6, O1, oz, and O2, and the signal sampling rate is F s At 250Hz, the signal duration was 1s, and 5 trials were collected as training data for each event. Thus, the electroencephalogram signal can be expressed as a four-dimensional tensor
Wherein N is c Is 9,N s Is 250,N t Is 5,N f Is a group of 40, and has the advantages of,
representing a set of real numbers.
Performing filter bank decomposition on the EEG signal chi', assuming the number N of filter sub-bands b The upper limit cut-off frequency of the band-pass filter is set to be 5 Hz, and the lower limit cut-off frequency of the band-pass filter is respectively 8Hz,16Hz,24Hz,32Hz and 40Hz. After decomposition, training signals under different sub-bands are obtained
As the electroencephalogram signal set to be processed. Selecting training signals χ 'of each sub-band one by one' m Each time selecting oneSubband training signal χ' m Acquiring a selected training signal χ' m Original signal subset of inclusion
I.e. the mth subband, the nth class of training signals. Each time a subset of the original signal is acquired
For acquired original signal subsets
The following operations of steps A02-A05 are performed to obtain an amplified signal subset for constructing an amplified electroencephalogram signal set.
Step A02, determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental trials.
Illustratively, a selected training signal χ 'is acquired' m Original signal subset of inclusion
Then, the mth sub-band is determined as the current sub-band, and N is added t The data of the test times are superposed and averaged to obtain the average value of the brain electrical signal units in the nth class and the mth sub-band
A03, obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix that satisfies a minimum value for a preset objective function.
In specific implementation, different types of electroencephalogram signals in the embodiment contain fundamental waves and harmonic components of different stimulation frequencies, so that the reference matrix can be constructed by sine and cosine sequences of corresponding frequencies. Since filter bank decomposition is applied, the range of harmonic orders can be based onFrequency of stimulation f n Lower cut-off frequency with different sub-bands
Collectively, the reference matrix is given by:
wherein N is he The maximum harmonic number, here set to 5;
N hs is the minimum harmonic number
Is the smallest integer of (a).
According to a reference matrix
And the average value of the electroencephalogram signal units obtained in the step A02
Solving an objective function to obtain an objective source aliasing matrix
Target source aliasing matrix
May be obtained by the following formula:
wherein the argmin function is used to search for a variable value that minimizes the objective function;
‖‖ F represents the Frobenius norm of the matrix;
And A04, obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix.
Illustratively, in terms of a reference matrix
And target source aliasing matrix
Computing a reconstructed source signal in an nth class, an mth subband
Reconstructing a source signal
Can be determined by the following formula:
and A05, obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
In particular, the source signal is reconstructed
Adding random noise to obtain and obtain original signal subset
Corresponding to N a Each amplified signal unit is used as a new oneThe EEG signal unit of (A) is added to the acquired original signal subset
Deriving and acquiring a subset of original signals
Corresponding amplified signal subsets
Wherein m =1,2, \ 8230and N b 。
And A06, training an electroencephalogram signal recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram signal recognition model.
Illustratively, new training data γ at different sub-bands is utilized m And solving model parameters of the FBTRCA algorithm, such as a spatial filter and the like, and using the model parameters for the classification and identification of a subsequent brain-computer interface system.
And A07, acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
In another embodiment of the present application, the brain-computer interface is a frequency-space encoded brain-computer interface. As shown in Table 2, each command of the frequency-space coding brain-computer interface is jointly coded by two different target frequencies at different spatial positions, the induced electroencephalogram signal simultaneously comprises fundamental waves and harmonic wave components at two frequencies, and the different frequencies have obvious spatial distribution differences.
TABLE 2
In this embodiment, an EEG-net neural network algorithm is applied to perform classification and identification on electroencephalogram signals. The brain-computer interface training data amplification method of the frequency-space coding brain-computer interface comprises the following steps:
and B01, acquiring an original signal subset included in the electroencephalogram signal set to be processed.
Wherein, every time an original signal subset is acquired, the following operations of steps B02-B05 are executed aiming at the acquired original signal subset to obtain an amplified signal subset used for forming an amplified electroencephalogram signal set.
Illustratively, the preprocessed post-electroencephalogram signal input in this embodiment includes 21 leads Pz, P1, P2, P3, P4, P5, P6, P7, P8, PO3, PO4, PO5, PO6, PO7, PO8, CB1, CB2, POz, O1, oz and O2, and the signal sampling rate is F s At 250Hz, the signal duration was 0.5s, and 20 trials were collected as training data for each event. Thus, the electroencephalogram signal can be expressed as a four-dimensional tensor
Wherein N is c Is 21; n is a radical of s Is 250; n is a radical of hydrogen t Is 20; n is a radical of f To 15, characterize the total number of classes;
representing a set of real numbers. And taking the four-dimensional tensor x' as an electroencephalogram signal set to be processed. Acquiring original signal subsets x included in the electroencephalogram signal set x to be processed one by one " n Wherein each original signal subset χ " n Is a class of training signals. Each time a subset χ of the original signal is acquired " n For the acquired original signal subset χ " n The following operations of steps B02-B05 are performed to obtain an amplified signal subset for constituting an amplified brain electrical signal set.
B02, determining the electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental trials.
Illustratively, the original signal subset χ for the nth class of acquisition " n Is a reaction of N t Carrying out superposition averaging on the data of the test times to obtain the electroencephalogram unit of the nth categoryMean value of
B03, obtaining a target source aliasing matrix based on a preset reference matrix and the EEG unit average value; the target source aliasing matrix is a source aliasing matrix that satisfies a minimum value for a preset objective function.
In specific implementation, different types of electroencephalogram signals are induced by two flicker stimuli at different spatial positions together, so that the reference matrix can be constructed by sine and cosine sequences containing all frequencies at the same time. Let the left and right frequencies under the nth category be f n,1 And f n,2 Then reference matrix
Can be represented by the following formulae (1) and (2):
wherein N is h Representing the harmonic orders contained in the reference matrix.
According to a reference matrix Y n "average sum of electroencephalogram signal units
Solving an objective function to obtain an objective source aliasing matrix
Target source aliasing matrix
May be obtained by the following formula:
wherein the argmin function is used to search for a variable value that minimizes the objective function;
‖‖ F represents the Frobenius norm of the matrix;
‖‖ 1 represents the L1 norm of the matrix;
Y n "is a reference matrix;
And step B04, obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix.
Illustratively, in terms of a reference matrix Y n "sum target source aliasing matrix
Computing the n-th class of reconstructed source signals S " n . Reconstructing a source signal S " n Can be determined by the following formula:
and B05, obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
In particular, the source signal S is reconstructed " n Adding random noise to obtain and obtain original signal subset chi' n Corresponding to N a Each amplified signal unit is used as a new electroencephalogram signalNumber element added to the acquired original signal subset χ " n Obtaining the original signal subset χ' obtained and acquired " n Corresponding amplified signal subsets
And step B06, training an electroencephalogram signal recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram signal recognition model.
Illustratively, the model parameters of the EEG-net network are solved using the amplified signal subset γ "for classification recognition of the subsequent brain-computer interface system.
And B07, acquiring the electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
The brain-computer interface training data amplification method provided by the embodiment can obtain the electroencephalogram signal subset included in the electroencephalogram signal set to be processed, and reconstruct the electroencephalogram signal based on the reference matrix, the random noise signal set and the average value of the electroencephalogram signal unit corresponding to the obtained original signal subset, so as to obtain the amplification signal conforming to the electroencephalogram characteristics, thereby realizing the efficient data amplification of the electroencephalogram signal set to be processed in the brain-computer interface training data, improving the recognition performance of the brain-computer interface under the condition of a small sample, and reducing the calibration burden of the system.
Based on the same inventive concept, the embodiment of the application also provides a brain-computer interface training data amplification device. Because the device is a device corresponding to the brain-computer interface training data amplification method in the embodiment of the application, and the principle of the device for solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.
Fig. 5 is a schematic structural diagram of a brain-computer interface training data amplification device according to an embodiment of the present application, where the brain-computer interface training data amplification device, as shown in fig. 5, includes: a data preparation module 501 and a data amplification module 502.
The data preparation module 501 is configured to obtain an original signal subset included in a electroencephalogram signal set to be processed;
a data amplification module 502, configured to, when the data preparation module acquires each original signal subset, perform the following operations on the acquired original signal subset to obtain an amplified signal subset used to form an amplified electroencephalogram signal set:
determining the mean value of the brain electrical signal unit corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram units with the number of experimental trial times; obtaining a target source aliasing matrix based on a preset reference matrix and an electroencephalogram unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
In one possible implementation, the data amplification module is specifically configured to:
summing the electroencephalogram signal units included in the acquired original signal subset to obtain a first electroencephalogram signal corresponding to the acquired original signal subset;
and (4) carrying out quotient calculation on the first electroencephalogram signal and the experimental test times to obtain an electroencephalogram signal unit average value corresponding to the acquired original signal subset.
In one possible implementation, the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on the source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
In a possible implementation manner, the objective function is obtained by summing the Frobenius norm of the second deviation matrix and the regularized norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on the source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
In a possible implementation manner, obtaining a reconstructed source signal based on a target source aliasing matrix and a reference matrix includes:
and performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain a reconstructed source signal.
In one possible implementation manner, obtaining an amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times, and the obtained original signal subset includes:
generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set comprises the number of random noise signal units as the amplification test times;
acquiring random noise signal units one by one from a random noise signal set, and executing a first operation when acquiring one random noise signal unit to obtain an amplification signal unit corresponding to the acquired original signal subset; a first operation comprising summing the acquired random noise signal unit with a reconstructed source signal;
and adding each obtained amplification signal unit serving as a new electroencephalogram signal unit into the obtained original signal subset to obtain an amplification signal subset corresponding to the obtained original signal subset.
In one possible implementation, as shown in fig. 6, the apparatus further includes:
the electroencephalogram signal recognition module 601 is used for training an electroencephalogram signal recognition model of a brain-computer interface based on the amplified electroencephalogram signal set to obtain a trained electroencephalogram signal recognition model; acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
The electronic equipment is based on the same inventive concept as the method embodiment, and the embodiment of the application also provides the electronic equipment. The electronic device may be used for brain-computer interface training data augmentation. In one embodiment, the electronic device may be a server or other electronic device. In this embodiment, the electronic device may be configured as shown in fig. 7, and includes a memory 701, a communication module 703 and one or more processors 702.
A memory 701 for storing a computer program executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area, where the program storage area may store an operating system, programs required for running an instant messaging function, and the like; the storage data area can store various instant messaging information, operation instruction sets and the like.
The memory 701 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory (non-volatile memory) such as, but not limited to, a read-only memory (rom), a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD), or any other medium which can be used to carry or store desired program code in the form of instructions or data structures and which can be accessed by a computer 701. Memory 701 may be a combination of the above.
The processor 702 may include one or more Central Processing Units (CPUs), or be a digital processing unit, etc. The processor 702 is configured to implement the above-mentioned brain-computer interface training data amplification method when calling the computer program stored in the memory 701.
The communication module 703 is used for communicating with a terminal device or other servers.
The embodiment of the present application does not limit the specific connection medium among the memory 701, the communication module 703 and the processor 702. In the embodiment of the present application, the memory 701 and the processor 702 are connected by a bus 704 in fig. 7, the bus 704 is represented by a thick line in fig. 7, and the connection manner between other components is merely illustrative and is not limited. The bus 704 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 7, but that does not indicate only one bus or one type of bus.
The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used for implementing the brain-computer interface training data amplification method according to any embodiment of the present application.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to make the computer device execute the brain-computer interface training data amplification method in the above embodiment. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.
Claims (10)
1. A brain-computer interface training data amplification method is characterized by comprising the following steps:
acquiring an original signal subset included in an electroencephalogram signal set to be processed;
wherein, each time one of the original signal subsets is acquired, the following operations are performed on the acquired original signal subsets to obtain an amplified signal subset used for constituting an amplified electroencephalogram signal set:
determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram signal units with the number of experimental test times;
obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value;
obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix;
and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
2. The method of claim 1, wherein said determining, based on said acquired subset of raw signals, a brain electrical signal unit mean corresponding to said acquired subset of raw signals comprises:
summing the electroencephalogram signal units included in the acquired original signal subset to obtain a first electroencephalogram signal corresponding to the acquired original signal subset;
and obtaining the average value of the electroencephalogram signal units corresponding to the acquired original signal subset by taking the quotient of the first electroencephalogram signal and the experimental trial times.
3. The method of claim 1, wherein the objective function is a Frobenius norm of the first deviation matrix; the first deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the average value of the electroencephalogram signal unit.
4. The method of claim 1, wherein the objective function is obtained by summing a Frobenius norm of the second bias matrix with a regularized norm of the source aliasing matrix; the second deviation matrix is obtained by performing matrix multiplication on a source aliasing matrix and the reference matrix and then performing difference on the EEG unit average value.
5. The method of claim 1, wherein deriving a reconstructed source signal based on the target source aliasing matrix and the reference matrix comprises:
and performing matrix multiplication on the target source aliasing matrix and the reference matrix to obtain the reconstructed source signal.
6. The method of claim 1, wherein obtaining the subset of amplified signals corresponding to the obtained subset of original signals based on the reconstructed source signal, a set of random noise signals, a preset number of amplification tests, and the obtained subset of original signals comprises:
generating a random noise signal set based on the amplification test times and a preset rule; the random noise signal set comprises the number of random noise signal units which is the number of times of the amplification test;
acquiring the random noise signal units from the random noise signal set one by one, and executing a first operation when acquiring one random noise signal unit to obtain an amplification signal unit corresponding to the acquired original signal subset; the first operation comprises summing the acquired random noise signal unit with the reconstructed source signal;
and adding each obtained amplification signal unit as a new electroencephalogram signal unit into the obtained original signal subset to obtain the amplification signal subset corresponding to the obtained original signal subset.
7. The method of claim 1, further comprising:
training an electroencephalogram recognition model of the brain-computer interface based on the amplified electroencephalogram signal set to obtain the trained electroencephalogram recognition model;
acquiring an electroencephalogram signal to be recognized, and recognizing the electroencephalogram signal to be recognized by using the trained electroencephalogram signal recognition model to obtain an electroencephalogram signal recognition result.
8. An apparatus for augmenting brain-computer interface training data, comprising:
the data preparation module is used for acquiring an original signal subset included in the electroencephalogram signal set to be processed;
a data amplification module, configured to, when the data preparation module acquires each original signal subset, perform the following operations on the acquired original signal subset to obtain an amplified signal subset used to form an amplified electroencephalogram signal set:
determining an electroencephalogram signal unit average value corresponding to the acquired original signal subset based on the acquired original signal subset; any original signal subset comprises electroencephalogram units with the number of experimental trial times; obtaining a target source aliasing matrix based on a preset reference matrix and the electroencephalogram signal unit average value; the target source aliasing matrix is a source aliasing matrix which enables a preset target function to take the minimum value; obtaining a reconstructed source signal based on the target source aliasing matrix and the reference matrix; and obtaining the amplification signal subset corresponding to the obtained original signal subset based on the reconstructed source signal, the random noise signal set, the preset amplification test times and the obtained original signal subset.
9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium; the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.
10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, the computer program, when executed by the processor, implementing the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211071579.1A CN115429289B (en) | 2022-09-01 | 2022-09-01 | Brain-computer interface training data amplification method, device, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211071579.1A CN115429289B (en) | 2022-09-01 | 2022-09-01 | Brain-computer interface training data amplification method, device, medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115429289A true CN115429289A (en) | 2022-12-06 |
| CN115429289B CN115429289B (en) | 2024-05-31 |
Family
ID=84247481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211071579.1A Active CN115429289B (en) | 2022-09-01 | 2022-09-01 | Brain-computer interface training data amplification method, device, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115429289B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116491960A (en) * | 2023-06-28 | 2023-07-28 | 南昌大学第一附属医院 | Brain transient monitoring device, electronic device, and storage medium |
| CN116595456A (en) * | 2023-06-06 | 2023-08-15 | 之江实验室 | Data screening and model training method and device based on brain-computer interface |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040181375A1 (en) * | 2002-08-23 | 2004-09-16 | Harold Szu | Nonlinear blind demixing of single pixel underlying radiation sources and digital spectrum local thermometer |
| CN103211597A (en) * | 2013-04-27 | 2013-07-24 | 上海海事大学 | Resting brain function connected region detecting method based on affine clustering |
| US20140369537A1 (en) * | 2013-06-14 | 2014-12-18 | Oticon A/S | Hearing assistance device with brain computer interface |
| CN105551041A (en) * | 2015-12-15 | 2016-05-04 | 中国科学院深圳先进技术研究院 | Universal blood vessel segmentation method and system |
| CN106599801A (en) * | 2016-11-26 | 2017-04-26 | 施志刚 | Face recognition method based on intra-class average maximum likelihood cooperative expressions |
| CN109299751A (en) * | 2018-11-26 | 2019-02-01 | 南开大学 | The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data |
| WO2019217622A1 (en) * | 2018-05-09 | 2019-11-14 | Biosig Technologies, Inc. | Systems and methods to display signals based on a signal characteristic |
| CN110909164A (en) * | 2019-11-22 | 2020-03-24 | 科大国创软件股份有限公司 | Text enhancement semantic classification method and system based on convolutional neural network |
| CN111449644A (en) * | 2020-03-19 | 2020-07-28 | 复旦大学 | Bioelectricity signal classification method based on time-frequency transformation and data enhancement technology |
| CN112237431A (en) * | 2020-09-08 | 2021-01-19 | 浙江大学山东工业技术研究院 | Electrocardio parameter calculation method based on deep learning |
| CN112257521A (en) * | 2020-09-30 | 2021-01-22 | 中国人民解放军军事科学院国防科技创新研究院 | CNN underwater acoustic signal target identification method based on data enhancement and time-frequency separation |
| CN112270255A (en) * | 2020-10-27 | 2021-01-26 | 广州大学 | Electroencephalogram signal identification method and device, electronic equipment and storage medium |
| CN112370017A (en) * | 2020-11-09 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Training method and device of electroencephalogram classification model and electronic equipment |
| CN112545532A (en) * | 2020-11-26 | 2021-03-26 | 中国人民解放军战略支援部队信息工程大学 | Data enhancement method and system for classification and identification of electroencephalogram signals |
| CN113269048A (en) * | 2021-04-29 | 2021-08-17 | 北京工业大学 | Motor imagery electroencephalogram signal classification method based on deep learning and mixed noise data enhancement |
| CN113509188A (en) * | 2021-04-20 | 2021-10-19 | 天津大学 | Method and device for amplifying electroencephalogram signal, electronic device and storage medium |
| CN114371784A (en) * | 2022-01-14 | 2022-04-19 | 天津大学 | Brain-computer interface decoding method for steady-state visual evoked potential |
| US20220117538A1 (en) * | 2020-10-21 | 2022-04-21 | Pacesetter, Inc. | Methods and systems to confirm device classified arrhythmias utilizing machine learning models |
| CN114492501A (en) * | 2021-12-13 | 2022-05-13 | 重庆邮电大学 | Electroencephalogram signal sample expansion method, medium and system based on improved SMOTE algorithm |
-
2022
- 2022-09-01 CN CN202211071579.1A patent/CN115429289B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040181375A1 (en) * | 2002-08-23 | 2004-09-16 | Harold Szu | Nonlinear blind demixing of single pixel underlying radiation sources and digital spectrum local thermometer |
| CN103211597A (en) * | 2013-04-27 | 2013-07-24 | 上海海事大学 | Resting brain function connected region detecting method based on affine clustering |
| US20140369537A1 (en) * | 2013-06-14 | 2014-12-18 | Oticon A/S | Hearing assistance device with brain computer interface |
| CN105551041A (en) * | 2015-12-15 | 2016-05-04 | 中国科学院深圳先进技术研究院 | Universal blood vessel segmentation method and system |
| CN106599801A (en) * | 2016-11-26 | 2017-04-26 | 施志刚 | Face recognition method based on intra-class average maximum likelihood cooperative expressions |
| WO2019217622A1 (en) * | 2018-05-09 | 2019-11-14 | Biosig Technologies, Inc. | Systems and methods to display signals based on a signal characteristic |
| CN109299751A (en) * | 2018-11-26 | 2019-02-01 | 南开大学 | The SSVEP brain electricity classification method of convolutional Neural model based on the enhancing of EMD data |
| CN110909164A (en) * | 2019-11-22 | 2020-03-24 | 科大国创软件股份有限公司 | Text enhancement semantic classification method and system based on convolutional neural network |
| CN111449644A (en) * | 2020-03-19 | 2020-07-28 | 复旦大学 | Bioelectricity signal classification method based on time-frequency transformation and data enhancement technology |
| CN112237431A (en) * | 2020-09-08 | 2021-01-19 | 浙江大学山东工业技术研究院 | Electrocardio parameter calculation method based on deep learning |
| CN112257521A (en) * | 2020-09-30 | 2021-01-22 | 中国人民解放军军事科学院国防科技创新研究院 | CNN underwater acoustic signal target identification method based on data enhancement and time-frequency separation |
| US20220117538A1 (en) * | 2020-10-21 | 2022-04-21 | Pacesetter, Inc. | Methods and systems to confirm device classified arrhythmias utilizing machine learning models |
| CN112270255A (en) * | 2020-10-27 | 2021-01-26 | 广州大学 | Electroencephalogram signal identification method and device, electronic equipment and storage medium |
| CN112370017A (en) * | 2020-11-09 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Training method and device of electroencephalogram classification model and electronic equipment |
| CN112545532A (en) * | 2020-11-26 | 2021-03-26 | 中国人民解放军战略支援部队信息工程大学 | Data enhancement method and system for classification and identification of electroencephalogram signals |
| CN113509188A (en) * | 2021-04-20 | 2021-10-19 | 天津大学 | Method and device for amplifying electroencephalogram signal, electronic device and storage medium |
| CN113269048A (en) * | 2021-04-29 | 2021-08-17 | 北京工业大学 | Motor imagery electroencephalogram signal classification method based on deep learning and mixed noise data enhancement |
| CN114492501A (en) * | 2021-12-13 | 2022-05-13 | 重庆邮电大学 | Electroencephalogram signal sample expansion method, medium and system based on improved SMOTE algorithm |
| CN114371784A (en) * | 2022-01-14 | 2022-04-19 | 天津大学 | Brain-computer interface decoding method for steady-state visual evoked potential |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595456A (en) * | 2023-06-06 | 2023-08-15 | 之江实验室 | Data screening and model training method and device based on brain-computer interface |
| CN116595456B (en) * | 2023-06-06 | 2023-09-29 | 之江实验室 | Data screening and model training method and device based on brain-computer interface |
| CN116491960A (en) * | 2023-06-28 | 2023-07-28 | 南昌大学第一附属医院 | Brain transient monitoring device, electronic device, and storage medium |
| CN116491960B (en) * | 2023-06-28 | 2023-09-19 | 南昌大学第一附属医院 | Brain transient monitoring device, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115429289B (en) | 2024-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115429289A (en) | Brain-computer interface training data amplification method, device, medium and electronic equipment | |
| Chen et al. | Removal of muscle artifacts from the EEG: A review and recommendations | |
| Akhtar et al. | Employing spatially constrained ICA and wavelet denoising, for automatic removal of artifacts from multichannel EEG data | |
| Leite et al. | Deep convolutional autoencoder for EEG noise filtering | |
| Yazdani et al. | Classification of EEG signals using Dempster Shafer theory and a k-nearest neighbor classifier | |
| Goh et al. | Evolutionary big optimization (BigOpt) of signals | |
| Baali et al. | A transform-based feature extraction approach for motor imagery tasks classification | |
| Hassanpour et al. | Time domain signal enhancement based on an optimized singular vector denoising algorithm | |
| Alyasseri et al. | An efficient optimization technique of eeg decomposition for user authentication system | |
| CN112515685A (en) | Multi-channel electroencephalogram signal channel selection method based on time-frequency co-fusion | |
| Alyasseri et al. | EEG signal denoising using hybridizing method between wavelet transform with genetic algorithm | |
| CN111671420B (en) | Method for extracting features from resting state electroencephalogram data and terminal equipment | |
| CN115211869A (en) | EEG (electroencephalogram) signal denoising method, device and system based on empirical mode decomposition and Kalman filtering | |
| Mishra et al. | Noise removal in EEG signals using SWT–ICA combinational approach | |
| Maher et al. | Hybrid EEG-fNIRS brain-computer interface based on the non-linear features extraction and stacking ensemble learning | |
| Ai et al. | Flexible coding scheme for robotic arm control driven by motor imagery decoding | |
| Kumar et al. | SPECTRA: a tool for enhanced brain wave signal recognition | |
| Guan et al. | Multiclass Motor Imagery Recognition of Single Joint in Upper Limb Based on NSGA‐II OVO TWSVM | |
| Řondík et al. | Comparison of various approaches for P3 component detection using basic methods for signal processing | |
| CN114569140A (en) | Spindle wave extraction method, system, computer device, storage medium and program product | |
| CN114757236A (en) | Electroencephalogram signal denoising optimization method and system based on TQWT and SVMD | |
| Zhang et al. | Attention-Based Multiscale Spatial-Temporal Convolutional Network for Motor Imagery EEG Decoding | |
| Saha et al. | Data adaptive filtering approach to improve the classification accuracy of motor imagery for BCI | |
| Gupta et al. | A three phase approach for mental task classification using EEG | |
| Agarwal et al. | Wavelet based approaches in optimization theory and practice |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |