CN114652532A - Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection - Google Patents
Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection Download PDFInfo
- Publication number
- CN114652532A CN114652532A CN202210155156.1A CN202210155156A CN114652532A CN 114652532 A CN114652532 A CN 114652532A CN 202210155156 A CN202210155156 A CN 202210155156A CN 114652532 A CN114652532 A CN 114652532A
- Authority
- CN
- China
- Prior art keywords
- wheelchair
- module
- user
- data
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
- A61G5/1051—Arrangements for steering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/168—Evaluating attention deficit, hyperactivity
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/377—Electroencephalography [EEG] using evoked responses
- A61B5/378—Visual stimuli
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/22—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering for automatically guiding movable devices, e.g. stretchers or wheelchairs in a hospital
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/30—General characteristics of devices characterised by sensor means
- A61G2203/42—General characteristics of devices characterised by sensor means for inclination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention discloses a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, which comprises: the operation interface module is used for controlling and operating the wheelchair by a user; the data input module is used for acquiring the electroencephalogram signals of visual stimulation of a user and preprocessing the data; the attention detection module is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not; the intelligent wheelchair control module is used for detecting the running state of the wheelchair, performing danger early warning and automatic danger avoidance, positioning the wheelchair, realizing the functions of uploading the position of the wheelchair in real time and calling for help, detecting the running state and performing early warning and calling for help when the abnormal running state of the wheelchair is detected; and the instruction judgment module is used for judging and outputting the instruction result. The wheelchair can improve the accuracy of instruction execution and lower probability of misoperation and execution in the using process of the wheelchair, and meanwhile, various functions capable of ensuring the safety of users are added according to the characteristics of patients who mostly have inconvenient actions, so that the safety and the use experience are improved.
Description
Technical Field
The invention relates to the technical field of brain-computer interface technology and artificial intelligence, in particular to a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection.
Background
For patients with mobility impairment, the wheelchair is the best travel tool, and the traditional electric wheelchair needs limb operation, such as limb control, etc., so the traditional wheelchair has poor operability and high operation difficulty for people who have lost working ability. With the increasing aging trend and the increase of disabled people, people of the group are concerned about the difficulty of inconvenient movement, many people of the group cannot normally operate the wheelchair, and some patients with diseases which have serious influence on physical activities even cannot use the wheelchair.
Brain-Computer Interface (BCI) is a system for connecting information from Brain nerves to a Computer or other external devices because it can directly analyze Brain electrical signals without external muscle action, because Brain Interface technology can help handicapped people with dyskinesia to interact with external operations. The wheelchair (brain-controlled wheelchair) combined with the brain interface technology can well solve the defects of the traditional wheelchair, and the brain-controlled wheelchair directly reads the electroencephalogram signals of the user as the control signals, so the operation process can be realized without the body of the user to react, and even a paralyzed person can use the brain-controlled wheelchair as long as the thinking of the paralyzed person is normal. The existing brain-controlled wheelchair generally uses a single-mode brain-controlled wheelchair, the operability of the brain-controlled wheelchair can be greatly influenced by the problems of low stability of input signals, high training difficulty and the like, and the single-mode brain-controlled wheelchair has the defects of accuracy, response time and interference resistance. In addition, the accuracy of the brain control on the wheelchair is only emphasized in the existing brain control wheelchair, and the method for controlling the wheelchair and the emergency reaction of a user in case of emergency are not considered to have certain delay, so that how to improve the driving safety of the wheelchair by using other modes while ensuring the accuracy of the instruction of the brain control wheelchair is also a problem to be solved when the brain control wheelchair is widely used in the future.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, and aims to improve the accuracy of instruction identification issued by a user through the attention detection and perfect other functions of a brain-controlled wheelchair which mainly uses injured and sick patients as main application groups, including position real-time supply, necessary help calling, emergency obstacle avoidance and danger early warning, wheelchair driving state detection and the like.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection comprises:
the operation interface module is used for controlling and operating the wheelchair by a user, and different functional keys of the operation interface can have different visual stimuli to the user;
the data input module is used for acquiring SSVEP electroencephalogram signal data of visual stimulation by a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not and increasing the accuracy rate of instruction issuing;
the intelligent wheelchair control module detects the running state of the wheelchair by using a radar, carries out danger early warning and automatic danger avoidance, positions the wheelchair by using a satellite positioning and communication module, realizes the functions of uploading the position of the wheelchair in real time and calling for help, detects the running state by using the running state detection module, and carries out early warning and calling for help when the abnormal running state of the wheelchair is detected;
and the instruction judgment module is used for training and classifying by using an SVM algorithm and is used for judging and outputting an instruction result.
Further, the interface operation module comprises the following parts:
a. setting basic parameters, including setting basic parameters connected with an electroencephalogram acquisition instrument and selecting an operation mode, including a training mode before use and a use mode after successful training, wherein a user needs to train firstly when using the electroencephalogram acquisition instrument for the first time, and automatically stores a model after training so as to be directly used at a later period if the model meets requirements;
b. the function control of the wheelchair comprises the control of driving and stopping the wheelchair in the front, back, left and right directions by using keys, the wheelchair reminds a user when encountering an obstacle and inquires whether the user needs to continue driving, the expression form of the keys on a computer display screen is five square blocks which continuously flicker with respective set frequencies, the user can generate different SSVEP (steady state visual evoked potential) electroencephalograms when watching the square blocks with different flicker frequencies, and the flicker frequencies of the five square blocks are respectively set to be 8.2HZ, 9HZ, 10HZ, 11HZ and 12 HZ.
Further, the data input module comprises a data acquisition module and a data preprocessing module, wherein:
the data acquisition module acquires SSVEP electroencephalogram data generated when a user is stimulated by the visual sense of the operation interface, the data is in a cnt format, and the training data comprises a data set of each visual sense stimulation activity of the user;
the data preprocessing module carries out time-frequency conversion on data, filters the acquired data by using a ten-order finite impulse response digital filter, and then carries out standardized processing on the filtered data according to the rows, and the standardized processing method comprises the following steps:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, Newdata is new data, Rowdata is original data, Edata is a data mean value, and Sdata is a standard deviation; wherein the preprocessing of the data is performed in Matlab.
Further, the attention detection module is used for detecting the attention state of the user when using the wheelchair, and specifically performs the following operations:
1) constructing an attention normal form model of a user, wherein the attention normal form model comprises two situations of user attention concentration when visual stimulation exists and user attention non-concentration when visual stimulation exists, constructing a first support vector machine on the basis of data when attention is concentrated, constructing a second support vector machine on the basis of data when attention is not concentrated, combining the two support vector machines to construct a Support Vector Machine (SVM) model of the user attention normal form, setting an SVM kernel function type as a linear type, and setting a kernel function expression as follows:
K(x,y)＝x*y
in the formula, K (x, y) is a kernel function, and x and y are sample data;
2) the electroencephalogram signal data of the user are classified through the built Support Vector Machine (SVM) model of the user attention paradigm, whether the electroencephalogram instructions of the user belong to visual stimulation received when the attention is focused or not is judged, and instruction false triggering caused by the visual stimulation received when the attention is not focused is avoided.
Further, intelligence wheelchair control module includes early warning and automatic danger module, satellite positioning module and communication module and the state detection module that traveles, wherein:
the early warning and automatic danger avoiding module uses four radar modules, namely a front radar module, a rear radar module, a left radar module, a right radar module and a left radar module to detect the driving state of the wheelchair in real time, and stops the wheelchair in time when fixed obstacles are detected around and the distance between the fixed obstacles and the obstacles is smaller than a threshold value; when detecting that a moving object approaches, the emergency hedge method mainly comprises the following steps:
a. when a fixed obstacle is detected in the advancing direction of the wheelchair, real-time distance measurement is carried out on the obstacle, when the distance is larger than a set first threshold value, the wheelchair is controlled to follow the instruction judgment result of a user, when the distance between the wheelchair and the obstacle is smaller than the set first threshold value, the wheelchair is decelerated, an operation interface can prompt the user whether to advance towards the obstacle or not, and when the distance is smaller than a set second threshold value, the wheelchair is automatically stopped;
b. when the moving object is detected to approach the wheelchair, the distance measurement and the speed measurement are carried out on the moving object, and the wheelchair automatically and emergently avoids danger when the moving object collides with the wheelchair and the distance is small;
the satellite positioning module and the communication module are used for positioning the wheelchair by using a GPS and BDS dual mode, and can upload the position information of the wheelchair through GPRS, so that the position of the wheelchair can be inquired by the family of a user in real time; the user can call for help through one key of the operation interface, and the help calling content can send the position information of the wheelchair to a prestored mobile phone of an emergency contact through the GPRS and short message functions;
the driving state detection module is used for detecting the driving state of the wheelchair by using the gyroscopes, and analyzing the angles of the wheelchair in the horizontal direction and the vertical direction by using the two sets of gyroscopes respectively, so as to detect the road condition of the wheelchair and give an early warning in time when the wheelchair turns over.
Further, the instruction judgment module is used for constructing an instruction judgment model and analyzing results, and specifically executes the following operations:
a. a Support Vector Machine (SVM) model is established by SSVEP electroencephalogram signals collected during user training, the SVM model is set to be a classic C-SVC model, a kernel function type of the SVM model is set to be a Gaussian kernel function, a punishment coefficient of the C-SVC model is set to be 1, and a kernel function expression is as follows:
in the formula, x and y are sample data, gamma is a unique hyperparameter of a Gaussian kernel function, | | x-y | | | represents the norm of a vector, and K (x, y) is the kernel function of the SVM;
b. after the real-time electroencephalogram data of a user are subjected to data input module pretreatment, data prediction is carried out according to the built SVM model, the predicted value of each classification condition is obtained, and the instruction corresponding to the largest predicted value is judged as the final instruction of the user.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the system of the invention does not only analyze the electroencephalogram signals generated by visual stimulation of the user, and simultaneously assists in judging the current state of the user through attention detection, thereby preventing the user from reaching wrong instructions or triggering wrong instructions by mistake due to inattention and the like, ensuring the whole system to be more reliable, and simultaneously converting the user's idea into actual actions better.
2. The system of the invention focuses more on better experience of users when in actual use, and considers that the used group is mainly sick and injured patients, the social action ability of the system is generally weaker, the system is difficult to call for help like normal people when in danger and special situations, and the brain control wheelchair has certain delay in response speed and can cause some dangerous situations. Therefore, the system of the invention is added with a plurality of sensors on the basis of the brain-controlled wheelchair, can realize the functions of danger detection, uploading of wheelchair state information and real-time position, emergency obstacle avoidance and the like, and pays more attention to the safety of the wheelchair on the basis of assisting the brain-controlled wheelchair.
3. The system can be used in the medical field, reduces the workload of medical staff and helps the disabled people to better live.
Drawings
FIG. 1 is a diagram illustrating the relationship between modules of the system of the present invention.
FIG. 2 is a block diagram of a wheelchair operator interface of the present invention.
FIG. 3 is a logic diagram of the structure of the intelligent wheelchair control module of the system of the present invention.
FIG. 4 is a block diagram of a system brain electrical data processing flow of the present invention.
Detailed Description
The present invention will be further described with reference to the following specific examples.
In the multifunctional brain-controlled wheelchair system based on SSVEP and attention detection provided by the embodiment, the operation interface is a wheelchair operation interface which is developed on QT by using C + + language and runs on Windows equipment, the training and classification of the algorithm model are compiled in Matlab, Matlab is called in QT background, the output result of Matlab is fed back to the background, and the background finally sends the instruction result to the wheelchair control panel through Bluetooth to realize the control of the wheelchair. The relationship between the modules of the system is shown in fig. 1, which includes:
the operation interface module is used for controlling and operating the wheelchair by a user, and different functional keys of the operation interface can have different visual stimuli to the user;
the data input module is used for acquiring SSVEP electroencephalogram signal data of visual stimulation by a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not and increasing the accuracy rate of instruction issuing;
the intelligent wheelchair control module detects the running state of the wheelchair by using a radar, carries out danger early warning and automatic danger avoidance, positions the wheelchair by using a satellite positioning and communication module, realizes the functions of uploading the position of the wheelchair in real time and calling for help, detects the running state by using the running state detection module, and carries out early warning and calling for help when the abnormal running state of the wheelchair is detected;
an instruction judgment module: and training and classifying by using an SVM algorithm, and judging and outputting the instruction result.
The interface operation module mainly comprises basic parameter setting and function control of the wheelchair, and is shown in figure 2, wherein:
the basic parameter setting comprises basic parameter setting connected with the electroencephalogram acquisition instrument, including port numbers, IP addresses and basic setting of label tags, including communication protocols and port numbers. In addition, the operation interface is also provided with a mode setting, the mode setting is a training mode when the user uses the operation interface for the first time, the model is stored after the model meeting the requirements is trained, and the operation interface can be directly set as a working model in the later use.
The function control of the wheelchair comprises keys for driving the wheelchair front and back, left and right and stopping, reminding a user when encountering an obstacle and performing a symptom on whether to continue driving again, wherein the expression form of the function keys on a computer display screen is five flashing blocks. The frequency of each scintillation block is 8.2HZ, 9HZ, 10HZ, 11HZ, 12 HZ. The refresh rate of the computer display screen is 60 HZ. The function of the scintillation block is to provide external stimulation for human eyes through the scintillation in the form, and further induce the human brain to generate SSVEP signals.
The data input module comprises a data acquisition module and a data preprocessing module, as shown in fig. 4, wherein:
the data acquisition module gathers the user and to the amazing multiunit training data of operation interface vision, and operation interface begins to carry out the data acquisition of training when clicking the training button of operation interface, and it is 2s long for every button training, and rest 5s is once after gathering 5 buttons, needs to gather 10 group data altogether, and this data can be used for training the model. When the use key of the operation interface is clicked, the system can acquire electroencephalogram data of a user in real time, the electroencephalogram data are intercepted in a time period of 2s and serve as instruction data, and the data are in a cnt format.
The data preprocessing module firstly carries out time-frequency conversion on data, namely filtering the acquired data by using a ten-order finite impulse response digital filter, and then carrying out feature extraction on the original data according to channels, wherein the feature extraction method is to subtract a data mean value from the filtered data and divide the data mean value by a standard value of the data, and the method is represented as follows:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, Newdata is new data, Rowdata is original data, Edata is a data mean value, and Sdata is a standard deviation. Wherein the preprocessing of the data is performed in Matlab.
The attention detection module is used for detecting the attention state of a user when the user uses the wheelchair, and specifically executes the following operations:
the method comprises the steps of constructing an attention normal model of a user, mainly constructing a model in two states of concentrating the attention of the user when visual stimulation exists and not concentrating the attention of the user when visual stimulation exists, collecting electroencephalogram data of the user when the two states exist, processing the collected electroencephalogram data to obtain an electroencephalogram feature vector, and constructing a Support Vector Machine (SVM) model by utilizing the electroencephalogram feature vector. The method comprises the steps of establishing a first support vector machine by using electroencephalogram feature vectors in an attention concentration state when visual stimulation exists, establishing a second support vector machine by using electroencephalogram feature vectors in an attention non-concentration state when visual stimulation exists, combining and establishing the support vector machines in the two states to obtain a Support Vector Machine (SVM) model corresponding to a specific user, and judging the concentration degree of the attention of the specific user through the model.
Because SSVEP signal generation requires the use of continuous frequency stimulation for the user, false triggering without visual stimulation may not be considered. And collecting the data of two states of attention concentration and non-concentration with visual stimulation to construct a model, wherein an SVM algorithm is adopted as a construction mode. Setting an SVM model as a C-SVC model, setting an SVM kernel function type as a linear type, and setting a kernel function expression as follows:
K(x,y)＝x*y
wherein K (x, y) is a kernel function, and x, y is sample data.
And calculating the data serving as the instruction by using the attention normal form model, and using the calculated result as an index for indicating whether the user pays the instruction by focusing attention.
The structural logic of the intelligent wheelchair control module is shown in fig. 3, and the intelligent wheelchair control module comprises an early warning and automatic risk avoiding module, a satellite positioning module, a communication module and a driving state detection module, wherein:
the early warning and automatic danger avoiding module uses four radar modules, namely a front radar module, a rear radar module, a left radar module, a right radar module and a mobile radar module to detect the running state of the wheelchair in real time, the radar module is HB100, the anti-interference capability of the radar of the model is strong, when the obstacle around the wheelchair is detected, whether the obstacle is fixed or the obstacle is moved is firstly judged, when the obstacle is fixed, if the obstacle is the advancing direction of the wheelchair, the distance is tested in real time, when the distance is smaller than a threshold value 1, an operation interface prompts a user whether to continue to advance towards the current direction, if the distance is selected, the user continues to advance and decelerate, and when the distance is smaller than a threshold value 2, the user stops advancing. And when the obstacle is judged to move, ranging is carried out on the moving obstacle, and emergency danger avoidance is started when the obstacle is judged to possibly collide with the current wheelchair. Threshold 1 was set to 1.5m and threshold 2 was set to 60 cm.
The wheelchair is located by the aid of a GPS and BDS dual-mode through the satellite positioning module and the communication module, the MC20 is used by the satellite module, the SIM900A is used by the communication module, the position of the wheelchair can be uploaded by the aid of GPRS after the data card is inserted, and families of a user can inquire the position of the wheelchair in real time. The user can also call for help by one key through the operation interface, and the help calling content can send the position information of the wheelchair to a prestored mobile phone of an emergency contact through the functions of GPRS and short message.
The driving state detection module detects the driving state j of the wheelchair by using a gyroscope MPU9050 model, two groups of gyroscopes are used together to analyze the angles of the wheelchair in the horizontal direction and the vertical direction respectively, when the change of the horizontal angle and the change of the vertical angle are changed rapidly and have large change range, whether the wheelchair turns over can be detected, if the wheelchair turns over, the satellite communication and positioning module can give an early warning in time, and help seeking information is sent to a prestored mobile phone number. And also comprises the detection of the running road surface.
The instruction judgment module is used for training and classifying the electroencephalogram instructions of the user and finally calculating the instructions selected by the user, and the instruction judgment module specifically comprises the following steps:
when a user is trained, a Support Vector Machine (SVM) model is established through the collected SSVEP electroencephalogram signals, 90% of samples in training samples are used as a training set during training, 10% of samples are used as a test set to carry out result testing on the background, each sample is used as the test set in sequence, the rest samples are used as the training set to carry out cycle testing, and when the test result passes, the model is stored as a qualified model. The method comprises the following steps of setting a support SVM model as a classical C-SVC model, setting a kernel function type of the SVM model as a Gaussian kernel function, setting a punishment coefficient of the C-SVC model as 1, and setting a kernel function expression as follows:
in the formula, x and y are sample data, γ is a unique hyper-parameter of a gaussian kernel, and is set to be 0.001, | | x-y | | | represents a norm of a vector, and K (x, y) is a kernel of the SVM.
When a user starts to control the wheelchair, the acquired data is preprocessed and then calculated by using the SVM model of the SSVEP, wherein the SVM model of the SSVEP can obtain a matrix containing probability estimation values of each category through calculation, and the label corresponding to the maximum value in the matrix is the final result of instruction judgment.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that the changes in the shape and principle of the present invention should be covered within the protection scope of the present invention.
Claims (6)
1. Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection is characterized by comprising:
the operation interface module is used for controlling and operating the wheelchair by a user, and different functional keys of the operation interface can have different visual stimuli to the user;
the data input module is used for acquiring SSVEP electroencephalogram signal data of visual stimulation by a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not and increasing the instruction issuing accuracy;
the intelligent wheelchair control module detects the running state of the wheelchair by using a radar, carries out danger early warning and automatic danger avoidance, positions the wheelchair by using a satellite positioning and communication module, realizes the functions of uploading the position of the wheelchair in real time and calling for help, detects the running state by using the running state detection module, and carries out early warning and calling for help when the abnormal running state of the wheelchair is detected;
and the instruction judgment module is used for training and classifying by using an SVM algorithm and is used for judging and outputting an instruction result.
2. The SSVEP and attention detection-based multifunctional brain-controlled wheelchair system of claim 1, wherein: the interface operation module comprises the following parts:
a. setting basic parameters, including setting basic parameters connected with an electroencephalogram acquisition instrument and selecting an operation mode, including a training mode before use and a use mode after successful training, wherein a user needs to train firstly when using the electroencephalogram acquisition instrument for the first time, and automatically stores a model after training so as to be directly used at a later period if the model meets requirements;
b. the function control of the wheelchair comprises the control of driving and stopping the wheelchair in the front, back, left and right directions by using keys, the wheelchair reminds a user when encountering an obstacle and inquires whether the user needs to continue driving, the expression form of the keys on a computer display screen is five square blocks which continuously flicker with respective set frequencies, the user can generate different SSVEP (steady state visual evoked potential) electroencephalograms when watching the square blocks with different flicker frequencies, and the flicker frequencies of the five square blocks are respectively set to be 8.2HZ, 9HZ, 10HZ, 11HZ and 12 HZ.
3. The SSVEP and attention detection-based multifunctional brain-controlled wheelchair system of claim 1, wherein: the data input module comprises a data acquisition module and a data preprocessing module, wherein:
the data acquisition module acquires SSVEP electroencephalogram data generated when a user is stimulated by the visual sense of the operation interface, the data is in a cnt format, and the training data comprises a data set of each visual sense stimulation activity of the user;
the data preprocessing module carries out time-frequency conversion on data, filters the acquired data by using a ten-order finite impulse response digital filter, and then carries out standardized processing on the filtered data according to the rows, and the standardized processing method comprises the following steps:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, Newdata is new data, Rowdata is original data, Edata is a data mean value, and Sdata is a standard deviation; wherein the preprocessing of the data is performed in Matlab.
4. The SSVEP and attention detection-based multifunctional brain-controlled wheelchair system of claim 1, wherein: the attention detection module is used for detecting the attention state of a user when the user uses the wheelchair, and specifically executes the following operations:
1) constructing an attention normal form model of a user, wherein the attention normal form model comprises two situations of user attention concentration when visual stimulation exists and user attention non-concentration when visual stimulation exists, constructing a first support vector machine on the basis of data when attention is concentrated, constructing a second support vector machine on the basis of data when attention is not concentrated, combining the two support vector machines to construct a Support Vector Machine (SVM) model of the user attention normal form, setting an SVM kernel function type as a linear type, and setting a kernel function expression as follows:
K(x,y)＝x*y
in the formula, K (x, y) is a kernel function, and x and y are sample data;
2) the electroencephalogram signal data of the user are classified through the built Support Vector Machine (SVM) model of the user attention paradigm, whether the electroencephalogram instructions of the user belong to visual stimulation received when the attention is focused or not is judged, and instruction false triggering caused by the visual stimulation received when the attention is not focused is avoided.
5. The SSVEP and attention detection-based multifunctional brain-controlled wheelchair system of claim 1, wherein: intelligence wheelchair control module includes early warning and automatic danger module, satellite positioning module and communication module and the state detection module that traveles, wherein:
the early warning and automatic danger avoiding module uses four radar modules, namely a front radar module, a rear radar module, a left radar module, a right radar module and a left radar module to detect the driving state of the wheelchair in real time, and stops the wheelchair in time when fixed obstacles are detected around and the distance between the fixed obstacles and the obstacles is smaller than a threshold value; when detecting that a moving object approaches, the emergency hedge method mainly comprises the following steps:
a. when a fixed obstacle is detected in the advancing direction of the wheelchair, real-time distance measurement is carried out on the obstacle, when the distance is larger than a set first threshold value, the wheelchair is controlled to follow the instruction judgment result of a user, when the distance between the wheelchair and the obstacle is smaller than the set first threshold value, the wheelchair is decelerated, an operation interface can prompt the user whether to advance towards the obstacle or not, and when the distance is smaller than a set second threshold value, the wheelchair is automatically stopped;
b. when the moving object is detected to approach the wheelchair, the distance measurement and the speed measurement are carried out on the moving object, and the wheelchair automatically and emergently avoids danger when the moving object collides with the wheelchair and the distance is small;
the satellite positioning module and the communication module are used for positioning the wheelchair by using a GPS and BDS dual mode, and can upload the position information of the wheelchair through GPRS, so that the position of the wheelchair can be inquired by the family of a user in real time; the user can call for help through one key of the operation interface, and the help calling content can send the position information of the wheelchair to a pre-stored mobile phone of an emergency contact through the GPRS and short message functions;
the driving state detection module is used for detecting the driving state of the wheelchair by using the gyroscopes, and analyzing the angles of the wheelchair in the horizontal direction and the vertical direction by using the two sets of gyroscopes respectively, so as to detect the road condition of the wheelchair and give an early warning in time when the wheelchair turns over.
6. The SSVEP and attention detection-based multifunctional brain-controlled wheelchair system of claim 1, wherein: the instruction judgment module is used for constructing an instruction judgment model and analyzing results, and specifically executes the following operations:
a. a Support Vector Machine (SVM) model is established by SSVEP electroencephalogram signals collected during user training, the SVM model is set to be a classic C-SVC model, a kernel function type of the SVM model is set to be a Gaussian kernel function, a punishment coefficient of the C-SVC model is set to be 1, and a kernel function expression is as follows:
in the formula, x and y are sample data, gamma is a unique super parameter of a Gaussian kernel function, | x-y | represents the norm of a vector, and K (x, y) is the kernel function of the SVM;
b. after the real-time electroencephalogram data of a user are subjected to data input module pretreatment, data prediction is carried out according to the built SVM model, the predicted value of each classification condition is obtained, and the instruction corresponding to the largest predicted value is judged as the final instruction of the user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210155156.1A CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210155156.1A CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114652532A true CN114652532A (en) | 2022-06-24 |
| CN114652532B CN114652532B (en) | 2023-07-18 |
Family
ID=82027861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210155156.1A Active CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114652532B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116631150A (en) * | 2023-05-12 | 2023-08-22 | 小舟科技有限公司 | Brain-controlled wheelchair anti-collision early warning method and device, equipment and storage medium |
| CN116617011A (en) * | 2023-07-21 | 2023-08-22 | 小舟科技有限公司 | Wheelchair control method, device, terminal and medium based on physiological signals |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976115A (en) * | 2010-10-15 | 2011-02-16 | 华南理工大学 | Motor imagery and P300 electroencephalographic potential-based functional key selection method |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| WO2012035739A1 (en) * | 2010-09-13 | 2012-03-22 | パナソニック株式会社 | Boarded mobile body and method for controlling boarded mobile body |
| CN102799274A (en) * | 2012-07-17 | 2012-11-28 | 华南理工大学 | Method of asynchronous brain switch based on steady state visual evoked potentials |
| CN103263324A (en) * | 2013-05-06 | 2013-08-28 | 西安电子科技大学 | Intelligent wheelchair system based on SSVEP (steady-state visual evoked potential) |
| WO2015192610A1 (en) * | 2014-06-17 | 2015-12-23 | 华南理工大学 | Intelligent wheel chair control method based on brain computer interface and automatic driving technology |
| US20160370774A1 (en) * | 2015-06-17 | 2016-12-22 | Universite du Sud - Toulon - Var Universite du Sud - Toulon - Var | Process for controlling a mobile device |
| RU2627075C1 (en) * | 2016-10-28 | 2017-08-03 | Ассоциация "Некоммерческое партнерство "Центр развития делового и культурного сотрудничества "Эксперт" | Neuro computer system for selecting commands based on brain activity registration |
| CN107981997A (en) * | 2017-11-23 | 2018-05-04 | 郑州布恩科技有限公司 | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention |
| CN112070141A (en) * | 2020-09-01 | 2020-12-11 | 燕山大学 | SSVEP asynchronous classification method fused with attention detection |
| CN112116422A (en) * | 2020-09-10 | 2020-12-22 | 湖南工商大学 | Online shopping system and method based on brain-computer interface |
| CN112370259A (en) * | 2020-11-16 | 2021-02-19 | 吉林大学 | Control system of brain-controlled wheelchair based on SSVEP |
| CN113208634A (en) * | 2021-04-07 | 2021-08-06 | 北京脑陆科技有限公司 | Attention detection method and system based on EEG brain waves |
-
2022
- 2022-02-21 CN CN202210155156.1A patent/CN114652532B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012035739A1 (en) * | 2010-09-13 | 2012-03-22 | パナソニック株式会社 | Boarded mobile body and method for controlling boarded mobile body |
| CN101976115A (en) * | 2010-10-15 | 2011-02-16 | 华南理工大学 | Motor imagery and P300 electroencephalographic potential-based functional key selection method |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN102799274A (en) * | 2012-07-17 | 2012-11-28 | 华南理工大学 | Method of asynchronous brain switch based on steady state visual evoked potentials |
| CN103263324A (en) * | 2013-05-06 | 2013-08-28 | 西安电子科技大学 | Intelligent wheelchair system based on SSVEP (steady-state visual evoked potential) |
| WO2015192610A1 (en) * | 2014-06-17 | 2015-12-23 | 华南理工大学 | Intelligent wheel chair control method based on brain computer interface and automatic driving technology |
| US20160370774A1 (en) * | 2015-06-17 | 2016-12-22 | Universite du Sud - Toulon - Var Universite du Sud - Toulon - Var | Process for controlling a mobile device |
| RU2627075C1 (en) * | 2016-10-28 | 2017-08-03 | Ассоциация "Некоммерческое партнерство "Центр развития делового и культурного сотрудничества "Эксперт" | Neuro computer system for selecting commands based on brain activity registration |
| CN108604124A (en) * | 2016-10-28 | 2018-09-28 | 商业与文化合作专家发展中心（非商业合伙协会） | The neurocomputer system of select command is recorded based on cerebration |
| CN107981997A (en) * | 2017-11-23 | 2018-05-04 | 郑州布恩科技有限公司 | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention |
| CN112070141A (en) * | 2020-09-01 | 2020-12-11 | 燕山大学 | SSVEP asynchronous classification method fused with attention detection |
| CN112116422A (en) * | 2020-09-10 | 2020-12-22 | 湖南工商大学 | Online shopping system and method based on brain-computer interface |
| CN112370259A (en) * | 2020-11-16 | 2021-02-19 | 吉林大学 | Control system of brain-controlled wheelchair based on SSVEP |
| CN113208634A (en) * | 2021-04-07 | 2021-08-06 | 北京脑陆科技有限公司 | Attention detection method and system based on EEG brain waves |
Non-Patent Citations (1)
| Title |
|---|
| 刘明;王康宁;陈小刚;王瑶;王慧泉;蒲江波;谢小波;王金海;徐圣普;: "基于Matlab环境的脑控轮椅搭建与实验验证", 北京生物医学工程, no. 02, pages 84 - 91 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116631150A (en) * | 2023-05-12 | 2023-08-22 | 小舟科技有限公司 | Brain-controlled wheelchair anti-collision early warning method and device, equipment and storage medium |
| CN116631150B (en) * | 2023-05-12 | 2024-01-23 | 小舟科技有限公司 | Brain-controlled wheelchair anti-collision early warning method and device, equipment and storage medium |
| CN116617011A (en) * | 2023-07-21 | 2023-08-22 | 小舟科技有限公司 | Wheelchair control method, device, terminal and medium based on physiological signals |
| CN116617011B (en) * | 2023-07-21 | 2023-09-15 | 小舟科技有限公司 | Wheelchair control method, device, terminal and medium based on physiological signals |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114652532B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105078449B (en) | Senile dementia monitor system based on health service robot | |
| CN105078445B (en) | Senior health and fitness's service system based on health service robot | |
| CN114652532A (en) | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection | |
| CN103699226A (en) | Tri-modal serial brain-computer interface method based on multi-information fusion | |
| CN107981997B (en) | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention | |
| CN110584898B (en) | Brain-controlled wheelchair automatic obstacle avoidance method based on multiple sensors | |
| CN110013249A (en) | A kind of Portable adjustable wears seizure monitoring instrument | |
| CN103083014A (en) | Method controlling vehicle by electroencephalogram and intelligent vehicle using method | |
| CN106491251A (en) | One kind is based on non-intrusion type brain-computer interface robotic arm control system and its control method | |
| US11442536B2 (en) | EOG-based method and apparatus for asynchronous character input | |
| Taher et al. | EEG control of an electric wheelchair for disabled persons | |
| CN109714419A (en) | A kind of vehicle-mounted health detection system and its method | |
| CN113156861A (en) | Intelligent wheelchair control system | |
| CN113359991A (en) | Intelligent brain-controlled mechanical arm auxiliary feeding system and method for disabled people | |
| CN112936259B (en) | Man-machine cooperation method suitable for underwater robot | |
| CN115712350A (en) | Blink communication expression system based on myoelectricity detection and control method | |
| JPH02112098A (en) | Information selection presenting device | |
| Shao et al. | Design of intentional eye-blink signal acquisition and control system based on pressure sensor | |
| Ngo et al. | Control of a Smart Electric Wheelchair Based on EEG Signal and Graphical User Interface for Disabled People | |
| CN112230767B (en) | Wheelchair control method based on SSMVEP-ERP-OSR hybrid brain-computer interface drive | |
| CN114237388B (en) | Brain-computer interface method based on multi-mode signal identification | |
| CN110315541B (en) | Computer control system for brain electricity and eyes of multi-freedom-degree mechanical arm | |
| CN220327451U (en) | Steering wheel capable of monitoring health state | |
| De Venuto et al. | User-centered ambient assisted living: brain environment interface | |
| CN112633312B (en) | Automatic optimization method based on SSMVEP-ERP-OSR mixed brain-computer interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |