CN114652532B - Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection - Google Patents
Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection Download PDFInfo
- Publication number
- CN114652532B CN114652532B CN202210155156.1A CN202210155156A CN114652532B CN 114652532 B CN114652532 B CN 114652532B CN 202210155156 A CN202210155156 A CN 202210155156A CN 114652532 B CN114652532 B CN 114652532B
- Authority
- CN
- China
- Prior art keywords
- wheelchair
- user
- module
- data
- instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
- A61G5/1051—Arrangements for steering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/168—Evaluating attention deficit, hyperactivity
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/377—Electroencephalography [EEG] using evoked responses
- A61B5/378—Visual stimuli
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/22—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering for automatically guiding movable devices, e.g. stretchers or wheelchairs in a hospital
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/30—General characteristics of devices characterised by sensor means
- A61G2203/42—General characteristics of devices characterised by sensor means for inclination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Artificial Intelligence (AREA)
- Theoretical Computer Science (AREA)
- Biomedical Technology (AREA)
- Psychiatry (AREA)
- Biophysics (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- General Engineering & Computer Science (AREA)
- Pathology (AREA)
- Data Mining & Analysis (AREA)
- Psychology (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Developmental Disabilities (AREA)
- Hospice & Palliative Care (AREA)
- Mathematical Physics (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Fuzzy Systems (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Educational Technology (AREA)
- Child & Adolescent Psychology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
Abstract
The invention discloses a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, which comprises: the operation interface module is used for controlling and operating the wheelchair by a user; the data input module is used for collecting brain electrical signals stimulated by a user to view and preprocessing data; the attention detection module is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction; the intelligent wheelchair control module is used for detecting the running state of the wheelchair, carrying out danger early warning and automatic danger avoiding, positioning the wheelchair, realizing the functions of uploading the position of the wheelchair in real time and calling for help, detecting the running state, and carrying out early warning and calling for help when the abnormal running state of the wheelchair is detected; and the instruction judging module is used for judging and outputting an instruction result. The invention can improve the accuracy of instruction execution and the probability of lower misoperation and execution in the using process of the wheelchair, and simultaneously adds various functions capable of guaranteeing the safety of users aiming at the characteristics of most patients with inconvenient actions, thereby improving the safety and the using experience.
Description
Technical Field
The invention relates to the technical field of brain-computer interface technology and artificial intelligence, in particular to a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection.
Background
For patients with mobility impairment, the wheelchair is the best travel tool, and the traditional electric wheelchair needs limb operation such as limb control and the like, so that the traditional wheelchair has poor operability and high operation difficulty for people who have lost working capacity. With the increasing trend of aging and the increasing number of disabled people, people with such people are also increasingly concerned about the difficulty in mobility, many of them cannot operate the wheelchair normally, and some patients with diseases that have serious influence on physical activities cannot use the wheelchair.
The Brain interface BCI (Brain-Computer Interface) is a system for connecting information from Brain nerves to a computer or other external device, because it can directly analyze Brain electrical signals without external muscle actions, because Brain interface technology can help disabled people with movement disorder interact with external operations. The wheelchair (brain-controlled wheelchair) combined with the brain interface technology can well solve the defects of the traditional wheelchair, and because the brain-controlled wheelchair directly reads the brain electrical signals of a user as control signals, the operation process can be realized without the body of the user to react, even if a person with physical paralysis only needs to think normally, the person can use the brain-controlled wheelchair. At present, the existing brain-controlled wheelchair generally uses a single-mode brain-controlled wheelchair, the problems of low stability of input signals, high training difficulty and the like of the brain-controlled wheelchair can greatly influence the operability of the brain-controlled wheelchair, and the brain-controlled wheelchair using the single mode has the defects of accuracy, reaction time and anti-interference, and as visual stimulus on an operation interface of a brain-computer interface always exists, a user can not always keep a concentrated state in using the wheelchair, wrong instructions are easily generated when the attention is not concentrated, and the problem of how to reduce the number of false triggering instructions is also a urgent problem for disabled people without mobility. The existing brain-controlled wheelchair only pays attention to the accuracy of brain control on the wheelchair, and the fact that the brain-controlled wheelchair method has certain delay with the emergency response of a user when an emergency situation is met is not considered, so that how to improve the running safety of the wheelchair by using other modes while ensuring the instruction accuracy of the brain-controlled wheelchair is also a problem which needs to be solved when the brain-controlled wheelchair is widely used in the future.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art, provides a multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, aims to improve the accuracy of instruction identification issued by a user through attention detection, and improves other functions of the brain-controlled wheelchair taking injured patients as main application groups, including position real-time provision, necessary help calling, emergency obstacle avoidance and danger early warning, wheelchair driving state detection and the like.
In order to achieve the above purpose, the technical scheme provided by the invention is as follows: multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, comprising:
the operation interface module is used for controlling and operating the wheelchair by a user, and different function keys of the operation interface can have different visual stimuli for the user;
the data input module is used for collecting SSVEP brain electrical signal data of visual stimulus of a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not, so that the accuracy rate of instruction issuing is increased;
the intelligent wheelchair control module is used for detecting the running state of the wheelchair by using a radar, carrying out danger early warning and automatic danger avoidance, positioning the wheelchair by using a satellite positioning and communication module, realizing the functions of uploading the position of the wheelchair in real time and calling for help, detecting the running state by using the running state detection module, and carrying out early warning and calling for help when the abnormal running state of the wheelchair is detected;
the instruction judging module is used for training and classifying by using an SVM algorithm and is used for judging and outputting instruction results.
Further, the interface operation module includes the following parts:
a. the method comprises the steps of setting basic parameters, including basic parameter setting and operation mode selection connected with an electroencephalogram acquisition instrument, including a training mode before use and a use mode after successful training, wherein a user needs to train for the first time, and the model is automatically stored after training according with requirements so as to be directly used in the later period;
b. the functional control of the wheelchair comprises the control of using keys to drive and stop the wheelchair in the front, back, left and right directions, reminding a user and inquiring whether the user needs to continue driving when encountering an obstacle, wherein the representation form of the keys on a computer display screen is five square blocks which continuously flash at the set frequency, the user can generate different SSVEP electroencephalograms when watching the square blocks with different flashing frequencies, and the flashing frequencies of the five square blocks are respectively 8.2HZ,9HZ,10HZ,11HZ and 12HZ.
Further, the data input module comprises a data acquisition module and a data preprocessing module, wherein:
the data acquisition module acquires SSVEP electroencephalogram data generated when a user is stimulated by the vision of the operation interface, the data are in cnt format, and the training data comprise data sets of each visual stimulation activity of the user;
the data preprocessing module performs time-frequency conversion on data, filters the acquired data by using a ten-order finite impulse response digital filter, and performs standardized processing on the filtered data according to a line, wherein the standardized processing method comprises the following steps of:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, newdata is new data, rowdata is original data, edata is a data average value, and Sdata is a standard deviation; wherein the preprocessing of the data is performed in Matlab.
Further, the attention detection module is used for detecting the attention state of a user when the wheelchair is used, and specifically performs the following operations:
1) Constructing a user attention pattern model, which comprises two conditions of user attention concentration during visual stimulus and user attention non-concentration during visual stimulus, constructing a first support vector machine based on data during attention concentration, constructing a second support vector machine based on data during attention non-concentration, combining the two support vector machines to construct a support vector machine SVM model of the user attention pattern, setting an SVM kernel function type to be a linear type, and setting a kernel function expression to be:
K(x,y)＝x*y
wherein K (x, y) is a kernel function, and x, y are sample data;
2) Classifying the electroencephalogram data of the user through the built Support Vector Machine (SVM) model of the user attention pattern, judging whether the electroencephalogram instruction of the user belongs to visual stimulus received when attention is concentrated or not, and avoiding false triggering of the instruction generated by the visual stimulus when attention is not concentrated.
Further, the intelligent wheelchair control module comprises an early warning and automatic risk avoiding module, a satellite positioning module, a communication module and a driving state detection module, wherein:
the early warning and automatic risk avoiding module detects the running state of the wheelchair in real time by using four radar modules, namely front radar module, rear radar module, left radar module and right radar module, and stops in time when fixed obstacles are detected around and the distance between the fixed obstacles and the obstacles is smaller than a threshold value; the emergency risk avoidance when the approach of a moving object is detected, and the main implementation process comprises the following steps:
a. when detecting that a fixed obstacle exists in the advancing direction of the wheelchair, starting to measure the distance of the obstacle in real time, when the distance is larger than a set first threshold value, controlling the wheelchair to follow the instruction judgment result of a user, when the distance from the wheelchair to the obstacle is smaller than the set first threshold value, decelerating the wheelchair, prompting the user whether to advance in the direction of the obstacle by an operation interface, and when the distance is smaller than a set second threshold value, automatically stopping the wheelchair;
b. when detecting that a moving object is close to the periphery of the wheelchair, carrying out distance measurement and speed measurement on the moving object, and automatically carrying out emergency risk avoidance on the wheelchair when the moving object collides with the wheelchair and the distance is small;
the satellite positioning module and the communication module are used for positioning the wheelchair in a GPS-BDS dual-mode, and uploading the position information of the wheelchair through GPRS, so that the family of the user can inquire the position of the wheelchair in real time; the user can also call for help by one key through the operation interface, and the help calling content can send the position information of the wheelchair to a pre-stored emergency contact mobile phone through the GPRS and short message function;
the running state detection module detects the running state of the wheelchair by using gyroscopes, analyzes angles of the wheelchair in the horizontal direction and the vertical direction respectively by using two groups of gyroscopes, detects the road surface condition of the wheelchair running and gives an early warning in time when the wheelchair turns on one's side.
Further, the instruction judging module is used for constructing an instruction judging model and analyzing results, and specifically executing the following operations:
a. by establishing a Support Vector Machine (SVM) model for an SSVEP electroencephalogram signal acquired during training of a user, setting the SVM model as a classical C-SVC model, setting the kernel function type of the SVM model as a Gaussian kernel function, setting the penalty coefficient of the C-SVC model as 1, and setting the kernel function expression as follows:
wherein, x and y are sample data, gamma is a unique super parameter of a Gaussian kernel function, I x-y I represents a norm of a vector, and K (x, y) is a kernel function of the SVM;
b. after the real-time electroencephalogram data of the user is preprocessed by the data input module, data prediction is carried out according to the built SVM model, a predicted value of each classification condition is obtained, and an instruction corresponding to the largest predicted value is judged to be a final instruction of the user.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the system of the invention does not only analyze the brain electrical signals generated by visual stimulus of the user, but also assist in judging the current state of the user through the attention detection, so that the problem that the user gives an error instruction or false triggers the error instruction due to the fact that the attention of the user is not concentrated and the like is prevented, the whole system is more reliable, and meanwhile, the thought of the user can be better converted into actual actions.
2. The system of the invention is more focused on how to give better experience to users in actual use, and the social mobility of the system is generally weak in consideration of the fact that the used group is mainly a patient suffering from injury, once danger and special conditions are met, people are difficult to call for help like normal people, and the brain-controlled wheelchair has a certain delay in response speed, so that some dangerous conditions can be caused. Therefore, the system of the invention is added with a plurality of sensors on the basis of the brain-controlled wheelchair, thereby realizing the functions of hazard detection, uploading of wheelchair state information and real-time position, emergency obstacle avoidance and the like, and paying more attention to the safety of the wheelchair on the basis of assisting the brain-controlled wheelchair.
3. The system can be used in the medical field, lightens the workload of medical staff and helps people with limb disabilities to live better.
Drawings
FIG. 1 is a schematic diagram of the relationship between the various modules of the system of the present invention.
Figure 2 is a block diagram of the operational interface of the wheelchair of the present invention.
FIG. 3 is a logic diagram of the control module of the intelligent wheelchair according to the present invention.
FIG. 4 is a block diagram of the flow of electroencephalogram data processing in the system of the present invention.
Detailed Description
The invention will be further illustrated with reference to specific examples.
The multifunctional brain-controlled wheelchair system based on SSVEP and attention detection provided by the embodiment is characterized in that an operation interface is a wheelchair operation interface which is developed on QT by using C++ language and runs on Windows equipment, training and classifying of an algorithm model are written in Matlab, the Matlab is called in a QT background, an output result of the Matlab is fed back to the background, and finally an instruction result is sent to a wheelchair control board through Bluetooth by the background to realize wheelchair control. The relation among the modules of the system is shown in figure 1, which comprises the following steps:
the operation interface module is used for controlling and operating the wheelchair by a user, and different function keys of the operation interface can have different visual stimuli for the user;
the data input module is used for collecting SSVEP brain electrical signal data of visual stimulus of a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not, so that the accuracy rate of instruction issuing is increased;
the intelligent wheelchair control module is used for detecting the running state of the wheelchair by using a radar, carrying out danger early warning and automatic danger avoidance, positioning the wheelchair by using a satellite positioning and communication module, realizing the functions of uploading the position of the wheelchair in real time and calling for help, detecting the running state by using the running state detection module, and carrying out early warning and calling for help when the abnormal running state of the wheelchair is detected;
the instruction judging module: training and classifying are carried out by using an SVM algorithm, and the training and classifying are used for judging and outputting instruction results.
The interface operation module mainly comprises basic parameter setting and function control of the wheelchair, and is shown in fig. 2, wherein:
the basic parameter setting comprises basic parameter setting connected with an electroencephalogram acquisition instrument, wherein the basic parameter setting comprises a port number, an IP address and basic setting of a tag label, and the basic parameter setting comprises a communication protocol and the port number. In addition, the operation interface is also provided with a mode setting, the operation interface is set to be a training mode when a user uses the operation interface for the first time, the model is stored immediately after the model meeting the requirements is trained, and the operation interface can be directly set to be a working model in later use.
The functional control of the wheelchair comprises keys for driving and stopping the wheelchair back and forth and left and right, reminding a user when encountering an obstacle and collecting whether the wheelchair needs to continue driving again, wherein the functional keys are in five flashing blocks on a computer display screen. The frequency of each scintillator block was 8.2hz,9hz,10hz,11hz,12hz. The refresh rate of the computer display screen was 60HZ. The function of the flicker block is to provide external stimulus for human eyes through the flicker in the form, so as to induce human brain to generate SSVEP signals.
The data input module comprises a data acquisition module and a data preprocessing module, as shown in fig. 4, wherein:
the data acquisition module acquires multiple groups of training data of visual stimulus of a user on the operation interface, the operation interface starts to acquire training data when a training key of the operation interface is clicked, the training duration of each key is 2s, the rest is 5s, the 5 keys are acquired once, 10 groups of data are required to be acquired in total, and the data can be used for training a model. When the key of the operation interface is clicked, the system can acquire the brain electricity data of the user in real time, intercept the brain electricity data as instruction data in a 2s time period, and the data is in cnt format.
The data preprocessing module firstly carries out time-frequency conversion on data, namely, the acquired data is filtered by using a ten-order finite impulse response digital filter, then the original data is subjected to characteristic extraction according to a channel, and the characteristic extraction method is that the filtered data minus the data average value is divided by the standard value of the data, and the characteristic extraction method is expressed as follows:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, newdata is new data, rowdata is original data, edata is a data average value, and Sdata is a standard deviation. Wherein the preprocessing of the data is performed in Matlab.
The attention detection module is used for detecting the attention state of a user when the wheelchair is used, and specifically performs the following operations:
and constructing an attention model of a user, mainly constructing a model in which the attention of the user is concentrated when visual stimulus exists and the attention of the user is not concentrated when visual stimulus exists, acquiring electroencephalogram signal data of the user in the two conditions, processing the acquired electroencephalogram signal data to obtain an electroencephalogram feature vector, and constructing a Support Vector Machine (SVM) model by utilizing the electroencephalogram feature vector. The method comprises the steps of constructing a first support vector machine by using an electroencephalogram feature vector in a concentration state when visual stimulus exists, constructing a second support vector machine by using an electroencephalogram feature vector in a non-concentration state when visual stimulus exists, combining and constructing support vector machines in two states to obtain a support vector machine SVM model corresponding to a specific user, and judging the concentration degree of the attention of the specific user through the model.
Because the SSVEP signal generation requires the use of a continuous frequency stimulus to the user, false triggers without visual stimulus may be disregarded. And collecting the data of the two states of concentration and non-concentration of the two types of visual stimulus, and constructing a model in a mode of adopting an SVM algorithm. Setting an SVM model as a C-SVC model, setting an SVM kernel function type as a linear type, and setting a kernel function expression as follows:
K(x,y)＝x*y
where K (x, y) is a kernel function and x, y is sample data.
And calculating the data serving as the instruction by using the attention pattern model at the same time, and taking the calculated result as an index of whether the user focuses on the instruction or not.
The intelligent wheelchair control module has a structural logic shown in fig. 3, and comprises an early warning and automatic risk avoiding module, a satellite positioning module, a communication module and a driving state detection module, wherein:
the early warning and automatic risk avoidance module detects the running state of the wheelchair in real time by using four radar modules, namely HB100, the radar module has stronger anti-interference capability, when the obstacle is detected, the obstacle is firstly judged to be fixed or moved, when the obstacle is fixed, if the obstacle is the advancing direction of the wheelchair, the real-time test distance is started, when the distance is smaller than a threshold value 1, an operation interface prompts a user whether to continue advancing towards the current direction, if the distance is smaller than the threshold value 2, the operation interface continues advancing and decelerating, and the advancing is stopped when the distance is smaller than the threshold value 1. And when the obstacle is judged, the obstacle is moved, the distance measurement is carried out on the moving obstacle, and when the obstacle is judged to possibly conflict with the current wheelchair running, the emergency avoidance is started. The threshold 1 is set to 1.5m and the threshold 2 is set to 60cm.
The satellite positioning module and the communication module are used for positioning the wheelchair in a GPS and BDS dual mode, the satellite module is used for MC20, the communication module is used for SIM900A, and after the data card is inserted, the position of the wheelchair can be uploaded by GPRS, so that the family of a user can inquire the position of the wheelchair in real time. The user can also call for help by one key through the operation interface, and the help calling content can send the position information of the wheelchair to a prestored emergency contact mobile phone through the GPRS and short message function.
The running state detection module detects the running state j of the wheelchair by using the model MPU9050, two groups of gyroscopes are used for respectively analyzing the angles of the wheelchair in the horizontal direction and the vertical direction, when the change of the horizontal angle and the change of the vertical angle are rapidly changed and have larger change amplitude, whether the wheelchair is turned over or not can be detected, if the wheelchair is turned over, the satellite communication and positioning module can timely early warn in time, and help seeking information is sent to a prestored mobile phone number. And also includes detection of the driving surface.
The instruction judging module is used for training and classifying the electroencephalogram instructions of the user and finally calculating the instructions selected by the user, and the instructions are specifically as follows:
when a user is trained, a Support Vector Machine (SVM) model is built through the collected SSVEP electroencephalogram signals, 90% of samples in training samples are used as training sets, 10% of samples are used as test sets to carry out result testing in the background, each sample is used as a test set in sequence, the rest samples are used as training sets to carry out cyclic testing, and when the test results pass, the model is stored as a qualified model. Setting a support SVM model as a classical C-SVC model, setting a SVM model kernel function type as a Gaussian kernel function, setting a penalty coefficient of the C-SVC model as 1, and setting a kernel function expression as follows:
wherein x, y is sample data, gamma is a unique hyper-parameter of a Gaussian kernel function, set to 0.001, ||x-y|| represents the norm of the vector, and K (x, y) is the kernel function of the SVM.
When a user starts to control the wheelchair, the collected data are subjected to pretreatment and then are simultaneously calculated by using the SVM model of the SSVEP, wherein the SVM model of the SSVEP can obtain a matrix containing probability estimated values of each class through calculation, and a label corresponding to the maximum value in the obtained matrix is the final result of instruction judgment.
The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention, so variations in shape and principles of the present invention should be covered.
Claims (3)
1. Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection, which is characterized by comprising:
the operation interface module is used for controlling and operating the wheelchair by a user, and different function keys of the operation interface can have different visual stimuli for the user;
the data input module is used for collecting SSVEP brain electrical signal data of visual stimulus of a user and preprocessing the data;
the attention detection module is used for training and classifying by using an SVM algorithm and is used for carrying out auxiliary judgment on whether the instruction is a false triggering instruction or not, so that the accuracy rate of instruction issuing is increased;
the intelligent wheelchair control module is used for detecting the running state of the wheelchair by using a radar, carrying out danger early warning and automatic danger avoidance, positioning the wheelchair by using a satellite positioning and communication module, realizing the functions of uploading the position of the wheelchair in real time and calling for help, detecting the running state by using the running state detection module, and carrying out early warning and calling for help when the abnormal running state of the wheelchair is detected;
the instruction judging module is used for training and classifying by using an SVM algorithm and is used for judging and outputting instruction results;
the attention detection module is used for detecting the attention state of a user when the wheelchair is used, and specifically performs the following operations:
1) Constructing a user attention pattern model, which comprises two conditions of user attention concentration during visual stimulus and user attention non-concentration during visual stimulus, constructing a first support vector machine based on data during attention concentration, constructing a second support vector machine based on data during attention non-concentration, combining the two support vector machines to construct a support vector machine SVM model of the user attention pattern, setting an SVM kernel function type to be a linear type, and setting a kernel function expression to be:
K(x,y)＝x*y
wherein K (x, y) is a kernel function, and x, y are sample data;
2) Classifying the electroencephalogram data of the user through a built Support Vector Machine (SVM) model of a user attention pattern, judging whether an electroencephalogram instruction of the user belongs to visual stimulus received when attention is concentrated or not, and avoiding false triggering of the instruction generated by the visual stimulus when attention is not concentrated;
the intelligent wheelchair control module comprises an early warning and automatic risk avoiding module, a satellite positioning module, a communication module and a driving state detection module, wherein:
the early warning and automatic risk avoiding module detects the running state of the wheelchair in real time by using four radar modules, namely front radar module, rear radar module, left radar module and right radar module, and stops in time when fixed obstacles are detected around and the distance between the fixed obstacles and the obstacles is smaller than a threshold value; the emergency risk avoidance when the approach of a moving object is detected, and the main implementation process comprises the following steps:
a. when detecting that a fixed obstacle exists in the advancing direction of the wheelchair, starting to measure the distance of the obstacle in real time, when the distance is larger than a set first threshold value, controlling the wheelchair to follow the instruction judgment result of a user, when the distance from the wheelchair to the obstacle is smaller than the set first threshold value, decelerating the wheelchair, prompting the user whether to advance in the direction of the obstacle by an operation interface, and when the distance is smaller than a set second threshold value, automatically stopping the wheelchair;
b. when detecting that a moving object is close to the periphery of the wheelchair, carrying out distance measurement and speed measurement on the moving object, and automatically carrying out emergency risk avoidance on the wheelchair when the moving object collides with the wheelchair and the distance is small;
the satellite positioning module and the communication module are used for positioning the wheelchair in a GPS-BDS dual-mode, and uploading the position information of the wheelchair through GPRS, so that the family of the user can inquire the position of the wheelchair in real time; the user can also call for help by one key through the operation interface, and the help calling content can send the position information of the wheelchair to a pre-stored emergency contact mobile phone through the GPRS and short message function;
the running state detection module is used for detecting the running state of the wheelchair by using gyroscopes, analyzing angles of the wheelchair in the horizontal direction and the vertical direction respectively by using two groups of gyroscopes, detecting the road surface condition of the wheelchair running and early warning in time when the wheelchair turns on one's side;
the instruction judging module is used for constructing an instruction judging model and analyzing results, and specifically executes the following operations:
a. by establishing a Support Vector Machine (SVM) model for an SSVEP electroencephalogram signal acquired during training of a user, setting the SVM model as a classical C-SVC model, setting the kernel function type of the SVM model as a Gaussian kernel function, setting the penalty coefficient of the C-SVC model as 1, and setting the kernel function expression as follows:
wherein, x and y are sample data, gamma is a unique super parameter of a Gaussian kernel function, I x-y I represents a norm of a vector, and K (x, y) is a kernel function of the SVM;
b. after the real-time electroencephalogram data of the user is preprocessed by the data input module, data prediction is carried out according to the built SVM model, a predicted value of each classification condition is obtained, and an instruction corresponding to the largest predicted value is judged to be a final instruction of the user.
2. The SSVEP and attention detection-based multi-functional brain-controlled wheelchair system of claim 1, wherein: the operation interface module comprises the following parts:
a. the method comprises the steps of setting basic parameters, including basic parameter setting and operation mode selection connected with an electroencephalogram acquisition instrument, including a training mode before use and a use mode after successful training, wherein a user needs to train for the first time, and the model is automatically stored after training according with requirements so as to be directly used in the later period;
b. the functional control of the wheelchair comprises the control of using keys to drive and stop the wheelchair in the front, back, left and right directions, reminding a user and inquiring whether the user needs to continue driving when encountering an obstacle, wherein the representation form of the keys on a computer display screen is five square blocks which continuously flash at the set frequency, the user can generate different SSVEP electroencephalograms when watching the square blocks with different flashing frequencies, and the flashing frequencies of the five square blocks are respectively 8.2HZ,9HZ,10HZ,11HZ and 12HZ.
3. The SSVEP and attention detection-based multi-functional brain-controlled wheelchair system of claim 1, wherein: the data input module comprises a data acquisition module and a data preprocessing module, wherein:
the data acquisition module acquires SSVEP electroencephalogram data generated when a user is stimulated by the vision of the operation interface, the data are in cnt format, and the training data comprise data sets of each visual stimulation activity of the user;
the data preprocessing module performs time-frequency conversion on data, filters the acquired data by using a ten-order finite impulse response digital filter, and performs standardized processing on the filtered data according to a line, wherein the standardized processing method comprises the following steps of:
Newdata＝(Rowdata-Edata)/Sdata
in the formula, newdata is new data, rowdata is original data, edata is a data average value, and Sdata is a standard deviation; wherein the preprocessing of the data is performed in Matlab.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210155156.1A CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210155156.1A CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114652532A CN114652532A (en) | 2022-06-24 |
| CN114652532B true CN114652532B (en) | 2023-07-18 |
Family
ID=82027861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210155156.1A Active CN114652532B (en) | 2022-02-21 | 2022-02-21 | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114652532B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116631150B (en) * | 2023-05-12 | 2024-01-23 | 小舟科技有限公司 | Brain-controlled wheelchair anti-collision early warning method and device, equipment and storage medium |
| CN116617011B (en) * | 2023-07-21 | 2023-09-15 | 小舟科技有限公司 | Wheelchair control method, device, terminal and medium based on physiological signals |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976115A (en) * | 2010-10-15 | 2011-02-16 | 华南理工大学 | Motor imagery and P300 electroencephalographic potential-based functional key selection method |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| WO2012035739A1 (en) * | 2010-09-13 | 2012-03-22 | パナソニック株式会社 | Boarded mobile body and method for controlling boarded mobile body |
| CN102799274A (en) * | 2012-07-17 | 2012-11-28 | 华南理工大学 | Method of asynchronous brain switch based on steady state visual evoked potentials |
| WO2015192610A1 (en) * | 2014-06-17 | 2015-12-23 | 华南理工大学 | Intelligent wheel chair control method based on brain computer interface and automatic driving technology |
| RU2627075C1 (en) * | 2016-10-28 | 2017-08-03 | Ассоциация "Некоммерческое партнерство "Центр развития делового и культурного сотрудничества "Эксперт" | Neuro computer system for selecting commands based on brain activity registration |
| CN107981997A (en) * | 2017-11-23 | 2018-05-04 | 郑州布恩科技有限公司 | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention |
| CN112070141A (en) * | 2020-09-01 | 2020-12-11 | 燕山大学 | SSVEP asynchronous classification method fused with attention detection |
| CN112116422A (en) * | 2020-09-10 | 2020-12-22 | 湖南工商大学 | Online shopping system and method based on brain-computer interface |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103263324B (en) * | 2013-05-06 | 2015-07-22 | 西安电子科技大学 | Intelligent wheelchair system based on SSVEP (steady-state visual evoked potential) |
| US20160370774A1 (en) * | 2015-06-17 | 2016-12-22 | Universite du Sud - Toulon - Var Universite du Sud - Toulon - Var | Process for controlling a mobile device |
| CN112370259A (en) * | 2020-11-16 | 2021-02-19 | 吉林大学 | Control system of brain-controlled wheelchair based on SSVEP |
| CN113208634A (en) * | 2021-04-07 | 2021-08-06 | 北京脑陆科技有限公司 | Attention detection method and system based on EEG brain waves |
-
2022
- 2022-02-21 CN CN202210155156.1A patent/CN114652532B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012035739A1 (en) * | 2010-09-13 | 2012-03-22 | パナソニック株式会社 | Boarded mobile body and method for controlling boarded mobile body |
| CN101976115A (en) * | 2010-10-15 | 2011-02-16 | 华南理工大学 | Motor imagery and P300 electroencephalographic potential-based functional key selection method |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN102799274A (en) * | 2012-07-17 | 2012-11-28 | 华南理工大学 | Method of asynchronous brain switch based on steady state visual evoked potentials |
| WO2015192610A1 (en) * | 2014-06-17 | 2015-12-23 | 华南理工大学 | Intelligent wheel chair control method based on brain computer interface and automatic driving technology |
| RU2627075C1 (en) * | 2016-10-28 | 2017-08-03 | Ассоциация "Некоммерческое партнерство "Центр развития делового и культурного сотрудничества "Эксперт" | Neuro computer system for selecting commands based on brain activity registration |
| CN108604124A (en) * | 2016-10-28 | 2018-09-28 | 商业与文化合作专家发展中心（非商业合伙协会） | The neurocomputer system of select command is recorded based on cerebration |
| CN107981997A (en) * | 2017-11-23 | 2018-05-04 | 郑州布恩科技有限公司 | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention |
| CN112070141A (en) * | 2020-09-01 | 2020-12-11 | 燕山大学 | SSVEP asynchronous classification method fused with attention detection |
| CN112116422A (en) * | 2020-09-10 | 2020-12-22 | 湖南工商大学 | Online shopping system and method based on brain-computer interface |
Non-Patent Citations (1)
| Title |
|---|
| 基于Matlab环境的脑控轮椅搭建与实验验证;刘明;王康宁;陈小刚;王瑶;王慧泉;蒲江波;谢小波;王金海;徐圣普;;北京生物医学工程(02);第84-91页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114652532A (en) | 2022-06-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114652532B (en) | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection | |
| CN207979669U (en) | Vehicle multi-mode formula biological response system | |
| CN105078449B (en) | Senile dementia monitor system based on health service robot | |
| CN105078445B (en) | Senior health and fitness's service system based on health service robot | |
| CN105595996B (en) | A kind of fatigue driving eeg monitoring method of electricity and brain electricity comprehensive judgement | |
| CN103699226B (en) | A kind of three mode serial brain-computer interface methods based on Multi-information acquisition | |
| CN104799984B (en) | Assistance system for disabled people based on brain control mobile eye and control method for assistance system | |
| CN112141118B (en) | Intelligent driving system and control method | |
| CN112114670B (en) | Man-machine co-driving system based on hybrid brain-computer interface and control method thereof | |
| CN110334592A (en) | A kind of monitoring of driver's abnormal behaviour and safety control system and safety control method | |
| CN105615878A (en) | Fatigue driving electroencephalographic monitoring method | |
| CN110013249A (en) | A kind of Portable adjustable wears seizure monitoring instrument | |
| CN106205048B (en) | Stupor automatic alarm system and alarm method based on brain-computer interface | |
| CN110584898B (en) | Brain-controlled wheelchair automatic obstacle avoidance method based on multiple sensors | |
| CN110658742A (en) | Multi-mode cooperative control wheelchair control system and method | |
| CN105411580A (en) | Brain control wheelchair system based on touch and auditory evoked potential | |
| CN105595997A (en) | Driving fatigue electroencephalogram monitoring method based on stepped fatigue determination | |
| CN111613329A (en) | Driver state monitoring system | |
| CN113069125A (en) | Head-mounted equipment control system, method and medium based on brain wave and eye movement tracking | |
| CN113156861A (en) | Intelligent wheelchair control system | |
| CN109714419A (en) | A kind of vehicle-mounted health detection system and its method | |
| CN115454238A (en) | Human-vehicle interaction control method and device based on SSVEP-MI fusion and automobile | |
| CN110688013A (en) | English keyboard spelling system and method based on SSVEP | |
| CN113491520A (en) | Driving fatigue detection method and device | |
| CN114460958A (en) | Brain-computer fusion flight control system based on hierarchical architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |