CN107961120B - Intelligent wheelchair control system based on electroencephalogram control - Google Patents

Intelligent wheelchair control system based on electroencephalogram control Download PDF

Info

Publication number
CN107961120B
CN107961120B CN201711294688.9A CN201711294688A CN107961120B CN 107961120 B CN107961120 B CN 107961120B CN 201711294688 A CN201711294688 A CN 201711294688A CN 107961120 B CN107961120 B CN 107961120B
Authority
CN
China
Prior art keywords
electroencephalogram
module
control
wheelchair
feature extraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201711294688.9A
Other languages
Chinese (zh)
Other versions
CN107961120A (en
Inventor
柳建光
胡豪
杜会盈
胡怡芳
毛洪川
韩洁
冯少康
刘当
李德峰
胡争争
马留洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CETC 27 Research Institute
Original Assignee
CETC 27 Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CETC 27 Research Institute filed Critical CETC 27 Research Institute
Priority to CN201711294688.9A priority Critical patent/CN107961120B/en
Publication of CN107961120A publication Critical patent/CN107961120A/en
Application granted granted Critical
Publication of CN107961120B publication Critical patent/CN107961120B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
    • A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
    • A61G5/10—Parts, details or accessories
    • A61G5/1051—Arrangements for steering
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7225—Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Abstract

The invention provides an intelligent wheelchair control system based on electroencephalogram control, which comprises an electroencephalogram acquisition module, a preprocessing module, a feature extraction module, a classification and identification module, an equipment control module and a data center carrier, wherein the electroencephalogram acquisition module is used for acquiring images of a user; the electroencephalogram acquisition module is used for acquiring electroencephalogram signals; the preprocessing module is used for preprocessing the electroencephalogram signals; the feature extraction module is used for extracting electroencephalogram features, and the feature extraction module comprises motor imagery feature extraction and alertness feature extraction; the classification identification module is used for classifying and identifying brain-computer interface tasks; the equipment control module converts the EEG signal identification result code into a control instruction; the data center carrier is a carrier platform of electroencephalogram data and a processing algorithm. The invention discloses an intelligent wheelchair control system based on electroencephalogram control, which can realize the real-time adjustment of the speed and the state of a wheelchair according to the state of a user while the electroencephalogram control of the wheelchair is carried out, and provides a safe and reliable application for the user.

Description

Intelligent wheelchair control system based on electroencephalogram control
Technical Field
The invention relates to the technical field of brain-computer interface (BCI), in particular to an intelligent wheelchair control system based on electroencephalogram control.
Background
A Brain-Computer interface (BCI) is a system for directly controlling an execution device without depending on peripheral nerves and muscles of the Brain, and controls an external device by decoding a Brain-conscious signal. BCI has wide application prospect in the fields of medical rehabilitation, life entertainment and military operation, and has become a cross-research hotspot of brain science, neurophysiology, signal analysis, control science and computer science.
at present, BCI application research in vulnerable group assisted life is still in a starting stage, currently, research on a single brain-computer interface is relatively common and mature, but only normal application scenes of users are usually considered when a practical system is designed, the state of a user is not considered, and the state of a controlled device cannot be automatically controlled according to the state of the user, for example, the real-time and automatic speed change control requirements of a wheelchair are not considered, the Switzerland Jos del R.Mill n and the like utilize a motion imagination brain-computer interface to successfully extract 2 instructions for event-related synchronization/desynchronization (RED/RES) to be applied to left and right control of the wheelchair, but because the system adopts a single-mode brain-computer interface, a plurality of freedom control signals cannot be provided, and the functions of adding, decelerating, starting and stopping are not high, the system practicability and safety are not high, the brain-computer interface-computer control system and the brain-computer signal processing method thereof disclosed in patent CN200810053558.0 utilize α wave interruption phenomenon to design a set of an operating system based on the alpha wave brain-computer interface, and realize front, rear, left and right movement control functions, but cannot realize speed regulation functions.
Disclosure of Invention
Aiming at the situation, in order to overcome the defects of the existing brain-controlled wheelchair control technology, the invention provides an intelligent wheelchair control system based on electroencephalogram control, which can realize self-adaptive speed regulation according to the self state of a user.
The device comprises an electroencephalogram acquisition module, a preprocessing module, a feature extraction module, a classification and identification module, an equipment control module and a data center carrier; the electroencephalogram acquisition module is used for acquiring electroencephalogram signals; the preprocessing module is used for preprocessing the electroencephalogram signals; the feature extraction module is used for extracting electroencephalogram features, and the feature extraction module comprises motor imagery feature extraction and alertness feature extraction; the classification identification module is used for classifying and identifying brain-computer interface tasks, judging the category and the alertness level of the motor imagery tasks and outputting control instructions; the equipment control module converts the EEG signal recognition result code into a control instruction for realizing the action of the wheelchair.
According to the invention, by extracting the electroencephalogram characteristics and the alertness characteristics of the motor imagery, the classification and identification module judges the category and the alertness grade of the motor imagery task and outputs a control instruction; finally, in order to realize the mechanical control of the brain-computer interface instead of the wheelchair control lever, the equipment control module is designed to enable the electric wheelchair to have a communication function, namely, the wheelchair can move forwards, turn left, turn right and stop operating at different speeds by receiving the instruction of the brain-computer interface.
Drawings
FIG. 1 is a diagram of BCI signal analysis.
Fig. 2 is a system diagram of an electroencephalogram control intelligent wheelchair.
Detailed Description
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
An intelligent wheelchair control system based on electroencephalogram control comprises an electroencephalogram acquisition module, a preprocessing module, a feature extraction module, a classification and identification module, an equipment control module and a calculation center carrier; the electroencephalogram acquisition module is used for acquiring electroencephalogram signals; the preprocessing module is used for preprocessing the electroencephalogram signals; the feature extraction module is used for extracting electroencephalogram features, and the feature extraction module comprises motor imagery feature extraction and alertness feature extraction; the classification identification module is used for classifying and identifying brain-computer interface tasks; the equipment control module converts the EEG signal identification result code into a control instruction; the data center carrier is a carrier platform of electroencephalogram data and a processing algorithm, wherein an auxiliary program is used for managing the cooperative operation among all the modules.
The electroencephalogram acquisition module comprises an electrode cap and an electroencephalogram acquisition instrument, the electrode cap is used for acquiring electroencephalogram signals, and the electroencephalogram acquisition instrument is used for amplifying, filtering and carrying out analog-to-digital conversion on the acquired electroencephalogram signals; the control module comprises a communication unit and a controller, and outputs two paths of analog voltage signals according to instructions of the multi-mode brain-computer interface for controlling the movement of the electric wheelchair.
The algorithms of the event-associated potential electroencephalogram signal processing module and the motor imagery electroencephalogram signal processing module are provided by programming of a vehicle-mounted notebook. The motor imagery electroencephalogram signal processing algorithm flow is as follows: CAR filtering → Mu rhythm frequency band extraction → CSP feature extraction → SVM classification. The alertness testing algorithm flow comprises the following steps: low-pass filtering → P300 amplitude feature → SVM classification.
The system firstly collects electroencephalograms through an electroencephalogram collection module, wherein the electroencephalogram collection uses Neuroscan electroencephalogram collection equipment, and 50Hz notch treatment is set;
the preprocessing module carries out 8-30Hz band-pass filtering, artifact removal and baseline correction on the electroencephalogram information;
the motor imagery feature extraction uses a Common Space Pattern (CSP) method to extract the electroencephalogram feature quantities under the left hand, the right hand and the idle state. The processing steps are as follows:
(1) calculating an average covariance matrix of the 3 types of electroencephalogram signals;
(2) approximate joint diagonalization of the covariance matrix of the 3 types of electroencephalogram signals;
(3) selecting a proper spatial filter by adopting the maximum mutual information;
(4) and filtering the electroencephalogram signals by utilizing a spatial filter.
Extracting 4 spatial filters from each type of electroencephalogram signals, and taking the energy mean value of the electroencephalogram signals after filtering as a characteristic;
the alertness feature extraction uses the algorithm as follows:
(1) for α wave brain rhythm fa(t) Fourier transform to obtain Fa(t)；
(2) F is to bea(t) squaring to obtain the energy E of the brain electrical characteristicsa；
(3) according to the above mode, the energy of beta wave, delta wave and theta wave is obtained as Eβ,Eδ,Eθ
(4) An alertness index model is designed by comprehensively analyzing energy changes of a plurality of rhythms
Figure BDA0001500068230000031
(5) according to experiments, the consciousness of a human subject is clear when α waves and β waves are dominant, and the consciousness of the human subject is fuzzy when delta waves and theta waves are dominant;
the classification and identification module adopts a Support Vector Machine (SVM) as a classifier, a penalty factor C and a kernel function g are main parameters influencing the performance of the SVM, C influences the data distribution after space transformation, and the parameter g determines the convergence speed and the popularization capability of the SVM.
An equipment control module: the equipment control module controls the direction and the rotating speed of the wheelchair. The three states of the motor imagery are left-handed motion (a), right-handed motion (B) and the idle state (C) which respectively control the left turning, the right turning and the straight going of the wheelchair, the alertness is used for controlling the rotating speed of the wheelchair, the alertness is divided into three levels ABC, wherein the level A represents that the wheelchair is controlled to normally move, the level B represents that the wheelchair is controlled to slowly move, and the level C represents that the wheelchair is forbidden to move; studies have shown that when a 60 minute training experiment is performed on a subject, the electroencephalogram data is equally divided into 6 segments in time (less than 10 minutes for one segment), and are recorded as segment 1, segment 2, segment 3, segment 4, segment 5, and segment 6. When a plurality of groups of alertness experiments are carried out, the alertness curves show a graph law along with the change of time. Thus, two thresholds w 1-1.5 and w 2-3.5 are set to classify the alertness of a subject into three classes ABC. Wherein F is 0-1.5A, A is awake state, F is 1.5-3.5B, B is doze state, F is more than 3.5C, and C is asleep state.
And (3) equipment control strategy: normal left turn (a, a), normal right turn (a, B), normal straight (a, C), slow left turn (B, a), slow right turn (B, B), slow straight (B, C), no motion (C, a) (C, B) (C, C);
calculating a central carrier module: the module is used for carrying out rapid operation processing on the acquired data and then outputting processing result information to the equipment control module.

Claims (4)

1. An intelligent wheelchair control system based on electroencephalogram control is characterized by comprising an electroencephalogram acquisition module, a preprocessing module, a feature extraction module, a classification and identification module and an equipment control module; the electroencephalogram acquisition module is used for acquiring electroencephalogram signals; the preprocessing module is used for preprocessing the electroencephalogram signals; the feature extraction module is used for extracting electroencephalogram features, and the feature extraction module comprises motor imagery feature extraction and alertness feature extraction; the classification identification module is used for classifying and identifying brain-computer interface tasks, judging the category and the alertness level of the motor imagery tasks and outputting control instructions; the equipment control module converts the EEG signal recognition result code into a control instruction for realizing the action of the wheelchair;
the motor imagery feature extraction uses a common space mode method to extract left-hand, right-hand and electroencephalogram signal feature quantities in an idle state, and the processing steps are as follows:
(1) calculating an average covariance matrix of the 3 types of electroencephalogram signals;
(2) approximate joint diagonalization of the covariance matrix of the 3 types of electroencephalogram signals;
(3) selecting a proper spatial filter by adopting the maximum mutual information;
(4) filtering the electroencephalogram signals by using a spatial filter;
extracting 4 spatial filters from each type of electroencephalogram signals, and taking the energy mean value of the electroencephalogram signals after filtering as a characteristic;
the alertness feature extraction uses the algorithm as follows:
(1) for α wave brain rhythm fa(t) Fourier transform to obtain Fa(t)；
(2) F is to bea(t) squaring to obtain the energy E of the brain electrical characteristicsa；
(3) according to the above mode, the energy of beta wave, delta wave and theta wave is obtained as Eβ,Eδ,Eθ
(4) An alertness index model is designed by comprehensively analyzing energy changes of a plurality of rhythms
Figure FDA0002129443050000011
(5) according to experiments, the consciousness of a human subject is clear when α waves and β waves are dominant, and the consciousness of the human subject is fuzzy when delta waves and theta waves are dominant;
an equipment control module: the device control module controls the direction and the rotating speed of the wheelchair, wherein three states of motion imagination, namely left-hand motion (a), right-hand motion (B) and an idle state (C), respectively control the left-hand rotation, right-hand rotation and straight movement of the wheelchair, the alertness is used for controlling the rotating speed of the wheelchair, and the alertness is divided into three levels ABC, wherein the level A represents that the wheelchair is controlled to normally move, the level B represents that the wheelchair is controlled to slowly move, and the level C represents that the wheelchair is forbidden to move; research shows that when a subject is subjected to a 60-minute training experiment, electroencephalogram data are equally divided into 6 segments according to time, wherein each 10 minutes is one segment and is recorded as a time segment 1, a time segment 2, a time segment 3, a time segment 4, a time segment 5 and a time segment 6; when a plurality of groups of alertness experiments are carried out, the alertness changes along the time, so that two thresholds w 1-1.5 and w 2-3.5 are set for dividing the alertness of a subject into three grades ABC; wherein F is A at 0-1.5, A is awake state, F is B at 1.5-3.5, B is doze state, F is C at 3.5 above, C is asleep state;
and (3) equipment control strategy: normal left turn (a, a), normal right turn (a, B), normal straight (a, C), slow left turn (B, a), slow right turn (B, B), slow straight (B, C), no motion (C, a) (C, B) (C, C).
2. The brain electric control-based intelligent wheelchair control system of claim 1, further comprising a data center carrier, wherein the data center carrier is a carrier platform for brain electric data and processing algorithms, and the auxiliary programs are used for managing the cooperative operation among the modules.
3. The brain electricity control-based intelligent wheelchair control system as claimed in claim 1, wherein the brain electricity collection module comprises an electrode cap and a brain electricity collection instrument, the electrode cap is used for collecting brain electricity signals, and the brain electricity collection instrument is used for carrying out amplification, filtering and analog-to-digital conversion processing on the collected brain electricity signals.
4. The brain electric control-based intelligent wheelchair control system as claimed in claim 1, wherein the device control module comprises a communication unit and a controller, and the control module outputs two paths of analog voltage signals according to instructions of the multi-modal brain-computer interface for controlling the movement of the electric wheelchair.
CN201711294688.9A 2017-12-08 2017-12-08 Intelligent wheelchair control system based on electroencephalogram control Expired - Fee Related CN107961120B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711294688.9A CN107961120B (en) 2017-12-08 2017-12-08 Intelligent wheelchair control system based on electroencephalogram control

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711294688.9A CN107961120B (en) 2017-12-08 2017-12-08 Intelligent wheelchair control system based on electroencephalogram control

Publications (2)

Publication Number Publication Date
CN107961120A CN107961120A (en) 2018-04-27
CN107961120B true CN107961120B (en) 2020-06-02

Family

ID=61998426

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711294688.9A Expired - Fee Related CN107961120B (en) 2017-12-08 2017-12-08 Intelligent wheelchair control system based on electroencephalogram control

Country Status (1)

Country Link
CN (1) CN107961120B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108919947B (en) * 2018-06-20 2021-01-29 北京航空航天大学 Brain-computer interface system and method realized through visual evoked potential
CN109464239A (en) * 2019-01-09 2019-03-15 浙江强脑科技有限公司 Intelligent wheel chair based on E.E.G control
CN111522445A (en) * 2020-04-27 2020-08-11 兰州交通大学 Intelligent control method
CN112245131A (en) * 2020-09-03 2021-01-22 深圳睿瀚医疗科技有限公司 Wheelchair control system and method based on facial expression electroencephalogram signal driving
CN114176920A (en) * 2021-12-20 2022-03-15 曲阜师范大学 Intelligent wheelchair based on electroencephalogram control

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101897640A (en) * 2010-08-10 2010-12-01 北京师范大学 Novel movement imagery electroencephalogram control-based intelligent wheelchair system
CN205144580U (en) * 2015-11-05 2016-04-13 西南交通大学 Wearable driver EEG signal mining head area

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6496724B1 (en) * 1998-12-31 2002-12-17 Advanced Brain Monitoring, Inc. Method for the quantification of human alertness
JP4687935B2 (en) * 2000-03-13 2011-05-25 国立大学法人北海道大学 Personal adaptive biological signal driven equipment control system
TWI257214B (en) * 2004-12-10 2006-06-21 Univ Nat Chiao Tung Brainwave-controlled embedded Internet robot agent architecture
CN100536766C (en) * 2008-01-31 2009-09-09 杨杰 Method and apparatus for monitoring and awakening fatigue doze
CN101571748A (en) * 2009-06-04 2009-11-04 浙江大学 Brain-computer interactive system based on reinforced realization
CN101987017A (en) * 2010-11-18 2011-03-23 上海交通大学 Electroencephalo-graph (EEG) signal identification and detection method for measuring alertness of driver
CN102309380A (en) * 2011-09-13 2012-01-11 华南理工大学 Intelligent wheelchair based on multimode brain-machine interface
CN106691474A (en) * 2016-11-25 2017-05-24 中原电子技术研究所（中国电子科技集团公司第二十七研究所） Brain electrical signal and physiological signal fused fatigue detection system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101897640A (en) * 2010-08-10 2010-12-01 北京师范大学 Novel movement imagery electroencephalogram control-based intelligent wheelchair system
CN205144580U (en) * 2015-11-05 2016-04-13 西南交通大学 Wearable driver EEG signal mining head area

Also Published As

Publication number Publication date
CN107961120A (en) 2018-04-27

Similar Documents

Publication Publication Date Title
CN107961120B (en) Intelligent wheelchair control system based on electroencephalogram control
Tang et al. Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network
Zeng et al. A lightGBM-based EEG analysis method for driver mental states classification
CN111091074B (en) Motor imagery electroencephalogram signal classification method of optimal region co-space mode
Sun et al. On-line EEG classification for brain-computer interface based on CSP and SVM
CN102331782A (en) Automatic vehicle controlling method of multi-mode brain-computer interface
Zhuang et al. Ensemble learning based brain–computer interface system for ground vehicle control
Bozhkov et al. EEG-based subject independent affective computing models
Wang et al. Decoding single-hand and both-hand movement directions from noninvasive neural signals
Chen et al. An EEG-based brain-computer interface for automatic sleep stage classification
Choi et al. Learning-based instantaneous drowsiness detection using wired and wireless electroencephalography
Zhuang et al. Motion control of a four-wheel-independent-drive electric vehicle by motor imagery EEG based BCI system
Geng et al. A novel design of 4-class BCI using two binary classifiers and parallel mental tasks
Liu et al. Identification of anisomerous motor imagery EEG signals based on complex algorithms
Li et al. Classification of imaginary movements in ECoG
Wang et al. Optimized preprocessing and tiny ml for attention state classification
Belwafi et al. Online adaptive filters to classify left and right hand motor imagery
Zhang et al. Recognition method for multi-class motor imagery EEG based on channel frequency selection
Chum et al. Optimal EEG feature selection by genetic algorithm for classification of imagination of hand movement
CN114358090A (en) Motor imagery electroencephalogram signal classification method based on PSD and CSP
Kotoky et al. Bispectrum analysis of EEG for motor imagery classification
CN107669416B (en) Wheelchair system based on continuous-light motor imagination nerve decoding and control method
Ming et al. ICA-SVM combination algorithm for identification of motor imagery potentials
CN113705324A (en) Multi-class motor imagery electroencephalogram signal feature extraction and classification method based on decision tree and CSP-SVM
Guan et al. Motor imagery recognition of upper limb single joint based on bci technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200602

Termination date: 20201208