CN116785085A - Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals - Google Patents
Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals Download PDFInfo
- Publication number
- CN116785085A CN116785085A CN202310650168.6A CN202310650168A CN116785085A CN 116785085 A CN116785085 A CN 116785085A CN 202310650168 A CN202310650168 A CN 202310650168A CN 116785085 A CN116785085 A CN 116785085A
- Authority
- CN
- China
- Prior art keywords
- signals
- mode
- module
- training
- wheelchair
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 50
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 210000004556 brain Anatomy 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims description 63
- 238000000034 method Methods 0.000 claims description 38
- 230000033001 locomotion Effects 0.000 claims description 27
- 238000004422 calculation algorithm Methods 0.000 claims description 26
- 238000005457 optimization Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 21
- 230000002068 genetic effect Effects 0.000 claims description 17
- 230000004927 fusion Effects 0.000 claims description 16
- 238000012706 support-vector machine Methods 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000013507 mapping Methods 0.000 claims description 9
- 230000035772 mutation Effects 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 238000005381 potential energy Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 238000002790 cross-validation Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 230000026058 directional locomotion Effects 0.000 claims description 4
- 238000012880 independent component analysis Methods 0.000 claims description 3
- 238000003909 pattern recognition Methods 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 3
- 108090000623 proteins and genes Proteins 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 2
- 230000001020 rhythmical effect Effects 0.000 claims description 2
- 230000003867 tiredness Effects 0.000 abstract description 3
- 208000016255 tiredness Diseases 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000001537 neural effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 206010033799 Paralysis Diseases 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000033764 rhythmic process Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000763 evoking effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000013011 mating Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000037023 motor activity Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000000578 peripheral nerve Anatomy 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000002269 spontaneous effect Effects 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/372—Analysis of electroencephalograms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
- A61B5/397—Analysis of electromyograms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
- A61G5/1051—Arrangements for steering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/20—Displays or monitors
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/30—General characteristics of devices characterised by sensor means
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Surgery (AREA)
- Artificial Intelligence (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Psychiatry (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Psychology (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Physiology (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Genetics & Genomics (AREA)
- Fuzzy Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals, which relates to the technical field of brain-computer control and comprises the following components: the system comprises a data acquisition module, a system mode selection module, a signal processing module and a decision module; the data acquisition module is used for acquiring physiological signals of a wheelchair user and sending the physiological signals to the system mode selection module; the system mode selection module is used for selecting a processing mode of the physiological signal; the signal processing module is used for preprocessing physiological signals and extracting signal characteristics for recognition; the signal processing module sends the identification result to the decision module; the decision module is used for fusing and converting the instruction information into a final wheelchair control instruction and outputting the control instruction to the wheelchair. The invention can improve the operation accuracy of the control system, reduce the tiredness of a user when the user executes the motor imagery, improve the user experience and the application performance of the wheelchair control system, and expand the user range of the brain-controlled wheelchair.
Description
Technical Field
The invention relates to the technical field of brain-computer control, in particular to a wheelchair control system based on motor imagery brain electrical signals and surface electromyographic signals.
Background
Brain-computer interface (Brain-Computer Interface, BCI) technology provides a method for replacing natural communication and control by measuring and processing neural activity signals of the Brain, and does not depend on common neural information communication channels such as peripheral nerves and muscles, and establishes direct communication between the Brain and external equipment. The BCI system recognizes brain intention by analyzing and processing an Electroencephalogram (EEG) signal, and communicates or controls external devices according to the final recognition result. Because of the technical features and advantages of the BCI system, the initial research objective was to aid paralyzed and disabled persons. The control or man-machine interaction of the robot is realized through the BCI technology, and the life quality of the aged, paralyzed and disabled can be improved.
At present, domestic and foreign scholars mainly study three EEG signals: event-related potentials (Event-RelatedPotential, ERP), motor Imagery (MI), and Steady-state visual evoked potentials (Steady-StateVisual Evoke Potential, SSVEP). Visual stimulus-based BCI robot control systems, such as ERP-BCI and SSVEP-BCI, increase the user's tiredness during actual operation. While the BCI system based on visual stimuli is more pronounced in EEG signal characteristics, resolution is higher. However, the user needs to continuously watch the stimulation interface, and the experience is poor from the viewpoint of use of the user. MI-BCI is induced by imagining motor activity to evoke brain neural activity, without external stimulus induction. Although the exercise imagination is a skill which can be mastered through learning and training, and has the problems of less brain mode identification categories and the like, the exercise imagination has the advantages of spontaneous brain electricity, no dependence on external stimulus, obvious induced brain electricity phenomenon and the like. From the aspects of practical application and man-machine interaction, MI-BCI is more suitable for being applied to robot control. Currently, research in the field of motor imagery brain-computer interfaces at home and abroad mostly focuses on the left/right hand two-class identification problem, and the four-class identification problem on the left/right hand, both feet and tongue. However, due to the factors of nonlinearity and non-stationarity of the electroencephalogram signal, large individual and age difference, strong randomness and the like, the accuracy of electroencephalogram signal decoding identification is low.
Disclosure of Invention
According to the problems of poor use experience, low recognition rate and poor application performance of the traditional electroencephalogram wheelchair control system, the wheelchair control system based on the motor imagery electroencephalogram signals and the surface electromyogram signals is provided. The invention mainly adopts the brain-computer interface control system composed of the brain-computer signal and the surface electromyographic signal, and the multi-sensor mode recognition result outputs the control instruction through decision fusion, thereby improving the operation accuracy of the control system, reducing the tiredness of a user when executing motor imagery, and improving the user experience and the application performance of the wheelchair control system.
The invention adopts the following technical means:
a wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals, comprising: the system comprises a data acquisition module, a system mode selection module, a signal processing module and a decision module;
the data acquisition module is used for acquiring physiological signals of a wheelchair user and sending the physiological signals to the system mode selection module; the physiological signals comprise motor imagery electroencephalogram signals and gesture surface electromyogram signals;
the system mode selection module is used for selecting a processing mode of the physiological signals, the processing mode comprises a training mode and a working mode, the training mode generates a classifier model according to personal data of a user, and the system mode selection module sends the physiological signals to the signal processing module in the working mode;
the signal processing module is used for preprocessing physiological signals, extracting signal characteristics and identifying the signal characteristics, and training and classifying the signal characteristics by using a support vector machine algorithm to obtain an identification result; the signal processing module sends the identification result to the decision module;
the decision module is used for fusing and converting the instruction information into a final wheelchair control instruction and outputting the control instruction to the wheelchair.
Further, the signal processing module comprises an electroencephalogram signal processing module and a surface electromyogram signal processing module, wherein the electroencephalogram signal processing module processes motor imagery electroencephalogram signals, and the surface electromyogram signal processing module processes hand surface electromyogram signals.
Further, buttons of the training mode and the working mode are arranged on an operation interface, and a basic parameter setting button, a directional movement state prompting lamp button, a central circular movement potential energy prompting lamp button and a four-section movement speed prompting lamp button are further arranged on the operation interface.
Further, in the working mode, the physiological signals collected by the data collection module are subjected to the signal processing module, the decision fusion module sends control instructions to the wheelchair, and corresponding prompts are displayed on the control panel.
Further, the electroencephalogram signal processing module processes the motor imagery electroencephalogram signal, and comprises the following steps:
signal pretreatment: removing ocular artifacts from the motor imagery electroencephalogram signals through re-referencing and independent component analysis;
extracting airspace characteristics: performing band-pass filtering on the motor imagery electroencephalogram signals subjected to signal preprocessing for three times, namely 8-12Hz,12-30Hz and 8-30Hz, to obtain three sections of signal data; respectively constructing a spatial filter for the three signal data through a co-space mode and carrying out spatial feature extraction and fusion;
identifying and classifying: and using a support vector machine as a classifier to obtain the identification classification result of the electroencephalogram signals.
Further, the surface electromyographic signal processing module processes the characteristic surface electromyographic signal, and the processing method comprises the following steps:
preprocessing signals; carrying out 50Hz notch filtering treatment on the signal data of each channel;
extracting time domain features; the time domain features comprise a mean value, a root mean square, a skewness, a waveform factor, a peak factor, a pulse factor and a margin factor of the signals, and feature values of all channels are fused to form feature vectors;
identifying and classifying; and using a support vector machine as a classifier to obtain the identification classification result of the electroencephalogram signals.
Further, the identifying and classifying of the signal processing module comprises the following steps:
training and classifying by using a Support Vector Machine (SVM) algorithm to obtain a recognition result, wherein the selected kernel function is a Gaussian kernel, and the formula is as follows:
wherein: x is x i Represents the ith sample, x j Represents the jth sample, σ is the Gaussian kernel bandwidth, and σ>0；
The classifier decision mode is selected as OvR mode to process four classification problems, namely 4 SVM classifiers are constructed, and final recognition results are output through voting.
Further, the training mode includes the steps of:
finishing the training mode parameter setting in the basic parameters, and storing the parameter setting after finishing the setting, wherein the training mode parameter setting comprises training times and single training time;
clicking a training mode button to start training data acquisition;
the training task button can be selected independently, the central circular movement potential energy prompting lamp is clicked to start training task data acquisition, and the central circular movement potential energy prompting lamp is turned off after acquisition is finished; after all the single training tasks are finished, all the button indicator lamps are turned off, and the next training period is started.
Further, in the training mode, the feature selection optimization function module based on the genetic algorithm is selectively started to improve the performance of the model, and at the moment, the electroencephalogram signal processing module uses a filter bank co-space mode method based on rhythm features, and the method comprises the following steps:
the calculation of the fitness function and the pattern recognition accuracy of the feature matrix are mapped, and the relation is as follows:
Fit(f(x))＝Accuracy i
wherein Accuracy i A ten-fold cross validation result under the input of the feature matrix of the ith feature combination is obtained;
the individual genotype codes are binary codes;
determining genetic algorithm related parameters, wherein the genetic algorithm related parameters comprise population scale, crossover probability and mutation probability;
reading a feature matrix and a label vector which are obtained after the electroencephalogram data set is processed and taking the feature matrix and the label vector as a preparation part in a feature selection optimization process;
the feature selection optimization process based on the genetic algorithm is started.
Further, the genetic algorithm-based feature selection optimization process comprises the following steps:
constructing a primary population, generating an initial group of a individuals, calculating the fitness value of each individual genotype in the group, constructing roulette, and selecting high-quality individuals in the roulette to be added into the initial population;
after meeting a preset population scale b, crossing individuals in the population with larger probability to improve the searching efficiency and the optimizing effect, and carrying out pairwise partial mapping crossing on the basis of crossing probability Pc;
carrying out single-point mutation operation based on mutation probability Pv on single-point genes of each individual genotype in the population;
calculating fitness values of all individuals in the new population, recording genotypes and fitness values of the highest-quality individuals in the new population, replacing the old population by the new population, and starting the next iterative evolution; and after the iterative evolution is completed, a final feature selection optimization result is obtained.
Compared with the prior art, the invention has the following advantages:
the invention discloses a wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals, which adopts the electroencephalogram signals and the surface electromyogram signals to form a mixed brain-computer interface control system, and a multi-sensor mode recognition result outputs a control instruction through decision fusion, so that the operation accuracy of the control system is improved, the tired feeling of a user when the user executes motor imagery is reduced, and the user experience feeling and the wheelchair control system application performance are improved. Compared with a deep learning method, the MI electroencephalogram signal decoding method based on traditional machine learning has the advantages that model training time is short, a customized exclusive classifier model can be generated according to personal physiological signal data of a user, and differences between individuals and ages can be avoided to a certain extent.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort to a person skilled in the art.
FIG. 1 is a general frame diagram of the system of the present invention.
FIG. 2 is a schematic diagram of an operation interface according to the present invention.
FIG. 3 is a diagram of the decision map of the present invention.
FIG. 4 is a flow chart of the system operation under the decision fusion of the present invention.
FIG. 5 is a training mode workflow diagram of the present invention.
Fig. 6 is a specific flowchart of the filter bank co-spatial mode based on the rhythmic features of the present invention.
FIG. 7 is a diagram of a feature selection optimization process based on a genetic algorithm in accordance with the present invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other. The invention will be described in detail below with reference to the drawings in connection with embodiments.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
The relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise. Meanwhile, it should be clear that the dimensions of the respective parts shown in the drawings are not drawn in actual scale for convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
The invention provides a wheelchair control system based on motor imagery and surface electromyographic signals, as shown in figure 1, comprising:
the data acquisition module is used for acquiring four types of motor imagery electroencephalogram data (left hand, right hand, tongue and feet) and gesture surface electromyogram data (relaxation gesture, hand opening gesture, digital 2 gesture and fist making gesture) of a user;
and the operation interface module is used for selecting a system operation mode, setting basic parameters and prompting the current wheelchair operation state.
The system mode selection module is used for selecting the acquired data processing mode, the training mode can be customized according to personal data of a user to generate a classifier model, the working mode can be used for carrying out normal working operation of the system according to an input signal by utilizing the existing classifier model, and a wheelchair control signal is sent.
The electroencephalogram signal processing module is used for preprocessing signals, extracting co-space mode characteristics of the signals, training and classifying by using an SVM algorithm, and obtaining recognition results;
the surface electromyographic signal processing module is used for preprocessing signals, extracting time domain characteristics of the signals, training and classifying by using an SVM algorithm, and obtaining recognition results;
the SVM kernel function selects a Gaussian kernel, the formula is as above, where x is i Represents the ith sample, x j Represents the jth sample, σ is the Gaussian kernel bandwidth, and σ>0。
The decision mode of the classifier is selected as OvR mode (one pair of other modes), the problems in the invention are dealt with, and four classification problems are processed, namely 4 SVM classifiers are constructed, and final recognition results are output through voting.
And the decision fusion control module performs decision fusion on the modal result through construction of a decision layer and converts the modal result into a final wheelchair control instruction to be output.
Further, the data acquisition module comprises the following parts:
the electroencephalogram data acquisition comprises amplifier parameter setting, lead electrode setting and sampling frequency setting of electroencephalogram data acquisition equipment;
the surface electromyographic signal data acquisition comprises parameter setting of surface electromyographic signal acquisition equipment.
Further, the operation interface module in the system comprises basic parameter setting, wheelchair state display and an operation interface:
the working mode selection comprises a working mode and a training mode.
Basic parameter setting, including basic parameter setting of electroencephalogram signal acquisition equipment and surface electromyogram signal acquisition equipment, and relevant parameter setting of a training mode;
the wheelchair status display and operation interface is used for daily operation application of the wheelchair control system, as shown in fig. 2, and comprises three functional buttons including a training mode, a working mode and basic parameter settings, four triangular directional movement status indicator lights, a central circular movement potential energy indicator light and four-section movement speed indicator lights. Each indicator light click is invalid in the working mode, only a prompt function is provided, and the indicator light click is valid in the training mode, and is used for selecting the input of the training data state label and providing the prompt function.
Further, the system mode selection module comprises a training mode and a working mode, the default starting state is the working mode after the normal system is started, the system normally operates in the mode, the data of the reading acquisition equipment passes through the signal processing module, the decision fusion module sends a control instruction to the wheelchair, and the control panel is provided with a corresponding prompt. In the training mode, corresponding prompting buttons can be selected according to the training program and parameter setting according to the operation prompts to start training data input, and after all training data are input, the system processes the data set, trains the classifier model and stores the classifier model, as shown in fig. 5.
Under the training mode, firstly, the training mode parameter setting is completed in basic parameters, wherein the training mode parameter setting comprises training times and single training time (the minimum time is 10 s), and the parameter setting is saved after the setting is completed; returning to the main interface, clicking a training mode button, and starting training data acquisition; in each single training, training task buttons (namely a triangular directional movement state prompting lamp and a four-section movement speed prompting lamp) can be selected independently, and the central circular movement potential energy prompting lamp is clicked to start training task data acquisition, and is extinguished after the acquisition is finished; after all the single training tasks are finished, all the button indicator lamps are turned off, and the next training period is started.
In the training mode, the feature selection optimization function module based on the genetic algorithm can be selectively started so as to improve the model performance. When this function is enabled, the electroencephalogram signal processing module uses a filter bank co-space mode method (RF-FBCSP) based on the rhythm characteristics, and the algorithm flow is as shown in fig. 6.
The feature selection optimization process based on the genetic algorithm is shown in fig. 7. The characteristic selection optimization process is equivalent to solving the problem of maximization of an objective function, and the objective function result is the identification accuracy of a certain characteristic matrix in terms of specific problems; therefore, the calculation of the fitness function and the pattern recognition accuracy of the feature matrix are mapped, and the relation is as follows:
Fit(f(x))＝Accuracy i
wherein Accuracy i And inputting a ten-fold cross validation result for the feature matrix of the ith feature combination.
Due to the adaptability function mapping relation and the characteristic of the data feature matrix, the individual genotype codes are binary codes, so that subsequent genetic operation is facilitated, and the risk of Hamming cliffs is avoided when the feature selection optimization problem is solved. The characteristic selection optimization process is started by determining related parameters of a genetic algorithm, wherein the related parameters comprise population scale, cross probability, variation probability and the like; and then, reading a feature matrix and a label vector which are obtained by processing the electroencephalogram data set, wherein the feature matrix and the label vector are prepared in the feature selection optimization process. After the preparation work is finished, the evolution process of the genetic algorithm is started, as shown in fig. 7, first, the construction of a primary population is carried out, an initial group of a individuals is generated, the fitness value of each individual genotype in the group is calculated, roulette is constructed, and high-quality individuals in the roulette are selected to be added into the initial population. After the preset population scale b is met, the individuals in the population are crossed with larger probability for improving the searching efficiency and the optimizing effect, and the cross probability Pc is used for carrying out the cross of the partial mapping of the groups. After the cross operation is finished, single-point mutation operation based on mutation probability Pv is carried out on single-point genes of each individual genotype in the population; after the cross mutation operation is finished, calculating the fitness value of all individuals in the new population, recording the genotype and fitness value of the highest-quality individuals in the new population, replacing the old population by the new population, and starting the next iterative evolution; and after the iterative evolution is completed, a final feature selection optimization result is obtained.
When the genetic algorithm is used for feature selection optimization, related control parameters are also required to be set, so that the calculation speed and the calculation effect of feature selection optimization are improved, and the specific parameters are set as follows: the initial panel size was set to 5, the population size was set to 400, the mating probability was 60%, the variation probability was 2%, and the number of iterations was 500.
Furthermore, the electroencephalogram signal processing module comprises three parts of signal preprocessing, airspace feature extraction, feature selection optimization and identification classification. The multichannel electroencephalogram signal is subjected to pretreatment of removing ocular artifacts through re-referencing and independent component analysis; and three-section signal data are obtained by carrying out three-time band-pass filtering (8-12 Hz,12-30Hz and 8-30 Hz) on the pretreated signals; respectively constructing spatial filters for the three sections of signals through a co-spatial mode and carrying out spatial domain feature extraction and fusion, wherein the spatial filter set construction of the co-spatial mode adopts a one-to-one mode (OvO); and using a support vector machine as a classifier to obtain the identification classification result of the electroencephalogram signals.
Further, the surface electromyographic signal processing module comprises signal preprocessing, time domain feature extraction and identification classification. The signal preprocessing is to carry out 50Hz notch filtering processing on the signal data of each channel; the time domain features comprise the mean value, root mean square, skewness, waveform factors, peak factors, pulse factors and margin factors of signals, and feature values of all channels are fused to form feature vectors; and using a support vector machine as a classifier to obtain the identification classification result of the electroencephalogram signals.
Further, the decision module comprises a decision layer and wheelchair control instruction output. The decision layer mainly fuses the identification and classification result received by the parallel channels and the contained instruction information and converts the identification and classification result and the contained instruction information into a final wheelchair control instruction by constructing decision layer rules, and the execution accuracy and the application performance of the wheelchair are improved by decision fusion. The decision layer is composed of decision mapping and decision fusion, the decision mapping is the mapping association of a physiological signal identification result of user behavior and a control instruction, as shown in fig. 3, wherein an electroencephalogram signal when a user executes motor imagery behavior is processed by a processing module to obtain an identification result of user imagery specific limb movement, and the identification result is mapped with the movement mode selection of a wheelchair, and the specific mapping relation is: imagine a tongue movement-advance mode, imagine a bipedal movement-retreat mode, imagine a left-hand movement-left turn mode, and imagine a right-hand movement-right turn mode; and the surface electromyographic signals when the user executes the gesture actions are processed by the processing module to obtain the recognition results of the gesture actions executed by the user, and the specific mapping relation between the surface electromyographic signals and the speed mode of the wheelchair is as follows: relaxation posture-wheelchair stop, hand-opening gesture-wheelchair low-speed operation, digital 2 gesture-wheelchair medium-speed operation and fist-making gesture-wheelchair high-speed operation.
And the decision fusion refers to the fusion of two types of control instructions, and under the framework of a decision layer, the decision is made by acquiring the recognition result of the signal data of the parallel channel according to the current wheelchair state. The specific workflow of the decision layer is shown in fig. 4, the motion mode defaults to the forward mode after the system is started, and the speed selection defaults to the stop state; reading the identification result of the surface electromyographic signal processing module in real time, carrying out fusion decision according to the identification result and the current motion state of the wheelchair, and stopping the wheelchair from moving immediately if the identification result is in a relaxation posture-stop state; if the identification result is one of low, medium and high-speed motions, the wheelchair executes the motions in the current motion mode and the speed selection mode when the wheelchair is in the current stop state, otherwise, the wheelchair keeps the current motion state. When the wheelchair is kept in a stopped state for more than 3s, firstly, the wheelchair is kept in the stopped state, the timer 1 is reset, and the counter 2 counts; then, a counter 2 is used for judging whether the forced adjustment of the motion mode is needed to be advanced; if the count of the counter 2 is less than 2 times, the brain electrical signals with the duration of 2s are read and processed, and the movement mode is changed according to the identification result; if the count of the counter 2 reaches 2 times, the result of the 2 times of movement mode switching is not the requirement mode of the user, and the movement mode is forcedly adjusted to the forward mode. After the motion mode is modified, the surface electromyographic signal processing and recognition result is read, and if the gesture recognition result still keeps the stop state for 3s, the next motion mode selection is started. The forced adjustment triggering threshold value of the counter 2 is set, so that the motion state is adjusted to be the most commonly used forward state, and the practical application performance is improved.
The invention is based on the ten-time ten-fold cross validation of the training dataset of the fourth brain-computer interface large race 2a dataset. As shown in Table 1, compared with other algorithms, the method has the technical advantages that compared with a large algorithm model under a deep learning framework, the method has lower time cost for model establishment and model parameter calculation, can realize the establishment of a customized model by using personal data of users, and avoids the problem of larger physiological signal data difference among users to a certain extent.
Table 1 motor imagery electroencephalogram data ten-fold cross-validation results
The classification results of the methods based on the fourth brain-computer interface large race 2a dataset are shown in Table 2, wherein the FBCSP belongs to the traditional machine learning method, and the ATCNet and the HDNN-TL belong to the deep learning method. Compared with the FBCSP, the RF-FBCSP method studied in the text has better verification effect; the kappa value result of the HDNN-TL shows the advantages of the deep learning algorithm, the model performance is good, and the model is complex. The recognition accuracy of ATCNet under the condition of considering the variability among the test is inferior to the method studied herein, and the recognition accuracy under the condition of not considering the variability among the test is higher, reaching 85.38%. The reason for the greater variability in these two cases may be affected by the size of the data set and the capacity of the model. In the case of small sample data sets, the RF-FBCSP method studied herein has certain advantages, which can meet the need to generate proprietary models using user personal data, reducing the effects of inter-subject variability to some extent.
Table 2 classification results for each method based on the fourth brain-computer interface major 2a dataset
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (10)
1. Wheelchair control system based on motor imagery brain electrical signal and surface electromyographic signal, characterized by comprising: the system comprises a data acquisition module, a system mode selection module, a signal processing module and a decision module;
the data acquisition module is used for acquiring physiological signals of a wheelchair user and sending the physiological signals to the system mode selection module; the physiological signals comprise motor imagery electroencephalogram signals and gesture surface electromyogram signals;
the system mode selection module is used for selecting a processing mode of the physiological signals, the processing mode comprises a training mode and a working mode, the training mode generates a classifier model according to personal data of a user, and the system mode selection module sends the physiological signals to the signal processing module in the working mode;
the signal processing module is used for preprocessing physiological signals, extracting signal characteristics and identifying the signal characteristics, and training and classifying the signal characteristics by using a support vector machine algorithm to obtain an identification result; the signal processing module sends the identification result to the decision module;
the decision module is used for fusing and converting the instruction information into a final wheelchair control instruction and outputting the control instruction to the wheelchair.
2. The wheelchair control system of claim 1 wherein the signal processing module comprises an electroencephalogram signal processing module that processes motor imagery electroencephalogram signals and a surface electromyogram signal processing module that processes gesture surface electromyogram signals.
3. The wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals according to claim 1, wherein buttons of the training mode and the working mode are arranged on an operation interface, and a basic parameter setting button, a directional movement state prompting lamp button, a central circular movement potential energy prompting lamp button and a four-section movement speed prompting lamp button are further arranged on the operation interface.
4. The wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals according to claim 1, wherein in the working mode, physiological signals collected by the data collection module are processed by the signal processing module, a control instruction is sent to the wheelchair after the decision fusion module, and corresponding prompts are displayed on the control panel.
5. The wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals according to claim 2, wherein the processing of motor imagery electroencephalogram signals by the electroencephalogram signal processing module comprises the steps of:
signal pretreatment: removing ocular artifacts from the motor imagery electroencephalogram signals through re-referencing and independent component analysis;
extracting airspace characteristics: performing band-pass filtering on the motor imagery electroencephalogram signals subjected to signal preprocessing for three times, namely 8-12Hz,12-30Hz and 8-30Hz, to obtain three sections of signal data; respectively constructing a spatial filter for the three signal data through a co-space mode and carrying out spatial feature extraction and fusion;
identifying and classifying: and using a support vector machine as a classifier to obtain the identification classification result of the electroencephalogram signals.
6. The wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals according to claim 2, wherein the surface electromyogram signal processing module processes the hand surface electromyogram signals comprising the steps of:
preprocessing signals; carrying out 50Hz notch filtering treatment on the signal data of each channel;
extracting time domain features; the time domain features comprise a mean value, a root mean square, a skewness, a waveform factor, a peak factor, a pulse factor and a margin factor of the signals, and feature values of all channels are fused to form feature vectors;
identifying and classifying; and using a support vector machine as a classifier to obtain the recognition classification result of the surface electromyographic signals.
7. The motor imagery electroencephalogram and surface electromyogram signal based wheelchair control system of claim 6, wherein the identification classification of the signal processing module comprises the steps of:
training and classifying by using an SVM algorithm to obtain an identification result, wherein the selected kernel function is a Gaussian kernel, and the formula is as follows:
wherein: x is x i Represents the ith sample, x j Represents the jth sample, σ is the Gaussian kernel bandwidth, and σ>0；
The classifier decision mode is selected as OvR mode to process four classification problems, namely 4 SVM classifiers are constructed, and final recognition results are output through voting.
8. Wheelchair control system based on motor imagery electroencephalogram and surface electromyogram signals according to claim 1, wherein the training mode comprises the steps of:
finishing the training mode parameter setting in the basic parameters, and storing the parameter setting after finishing the setting, wherein the training mode parameter setting comprises training times and single training time;
clicking a training mode button to start training data acquisition;
the training task button can be selected independently, the central circular movement potential energy prompting lamp is clicked to start training task data acquisition, and the central circular movement potential energy prompting lamp is turned off after acquisition is finished; after all the single training tasks are finished, all the button indicator lamps are turned off, and the next training period is started.
9. The motor imagery electroencephalogram and surface electromyogram signal based wheelchair control system of claim 8, wherein in the training mode, a genetic algorithm based feature selection optimization function module is selectively enabled to promote model performance, and wherein the electroencephalogram signal processing module uses a rhythmic feature based filter bank co-spatial mode method comprising:
the calculation of the fitness function and the pattern recognition accuracy of the feature matrix are mapped, and the relation is as follows:
Fit(f(x))＝Accuracy i
wherein Accuracy i A ten-fold cross validation result under the input of the feature matrix of the ith feature combination is obtained;
the individual genotype codes are binary codes;
determining genetic algorithm related parameters, wherein the genetic algorithm related parameters comprise population scale, crossover probability and mutation probability;
reading a feature matrix and a label vector which are obtained after the electroencephalogram data set is processed and taking the feature matrix and the label vector as a preparation part in a feature selection optimization process;
the feature selection optimization process based on the genetic algorithm is started.
10. The motor imagery electroencephalogram and surface electromyogram signal based wheelchair control system of claim 9, wherein the genetic algorithm based feature selection optimization process comprises the steps of:
constructing a primary population, generating an initial group of a individuals, calculating the fitness value of each individual genotype in the group, constructing roulette, and selecting high-quality individuals in the roulette to be added into the initial population;
after meeting a preset population scale b, crossing individuals in the population with larger probability to improve the searching efficiency and the optimizing effect, and carrying out pairwise partial mapping crossing on the basis of crossing probability Pc;
carrying out single-point mutation operation based on mutation probability Pv on single-point genes of each individual genotype in the population;
calculating fitness values of all individuals in the new population, recording genotypes and fitness values of the highest-quality individuals in the new population, replacing the old population by the new population, and starting the next iterative evolution; and after the iterative evolution is completed, a final feature selection optimization result is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310650168.6A CN116785085A (en) | 2023-06-02 | 2023-06-02 | Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310650168.6A CN116785085A (en) | 2023-06-02 | 2023-06-02 | Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116785085A true CN116785085A (en) | 2023-09-22 |
Family
ID=88045598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310650168.6A Pending CN116785085A (en) | 2023-06-02 | 2023-06-02 | Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116785085A (en) |
-
2023
- 2023-06-02 CN CN202310650168.6A patent/CN116785085A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017084416A1 (en) | Feedback system based on motor imagery brain-computer interface | |
| CN103793058B (en) | A kind of active brain-computer interactive system Mental imagery classification of task method and device | |
| CN103699216B (en) | A kind of based on Mental imagery and the E-mail communication system of vision attention mixing brain-computer interface and method | |
| CN111553295B (en) | Multi-mode emotion recognition method based on self-attention mechanism | |
| He et al. | EEG-and EOG-based asynchronous hybrid BCI: a system integrating a speller, a web browser, an e-mail client, and a file explorer | |
| CN107981997B (en) | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention | |
| CN101464728B (en) | Human-machine interaction method with vision movement related neural signal as carrier | |
| CN104758130B (en) | A kind of intelligent nursing device and method based on brain-computer interface | |
| CN103699226A (en) | Tri-modal serial brain-computer interface method based on multi-information fusion | |
| CN105411580B (en) | A kind of brain control wheelchair system based on tactile auditory evoked potential | |
| CN103955269A (en) | Intelligent glass brain-computer interface method based on virtual real environment | |
| CN103699217A (en) | Two-dimensional cursor motion control system and method based on motor imagery and steady-state visual evoked potential | |
| CN101201696A (en) | Chinese input BCI system based on P300 brain electric potential | |
| CN104978035A (en) | Brain computer interface system evoking P300 based on somatosensory electrical stimulation and implementation method thereof | |
| CN113208593A (en) | Multi-modal physiological signal emotion classification method based on correlation dynamic fusion | |
| Li et al. | EEG signal classification method based on feature priority analysis and CNN | |
| CN108523883A (en) | A kind of continuous Mental imagery identifying system of left and right index finger based on actual act modeling | |
| CN112328072A (en) | Multi-mode character input system and method based on electroencephalogram and electrooculogram | |
| CN106774851A (en) | Sense of touch finger motion rehabilitation system and method based on brain-computer interface | |
| CN109009098A (en) | A kind of EEG signals characteristic recognition method under Mental imagery state | |
| CN114145745B (en) | Graph-based multitasking self-supervision emotion recognition method | |
| CN108491792B (en) | Office scene human-computer interaction behavior recognition method based on electro-oculogram signals | |
| CN110688013A (en) | English keyboard spelling system and method based on SSVEP | |
| CN116785085A (en) | Wheelchair control system based on motor imagery electroencephalogram signals and surface electromyogram signals | |
| Han et al. | Confidence-aware subject-to-subject transfer learning for brain-computer interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |