CN113274032A - Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface - Google Patents
Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface Download PDFInfo
- Publication number
- CN113274032A CN113274032A CN202110471569.6A CN202110471569A CN113274032A CN 113274032 A CN113274032 A CN 113274032A CN 202110471569 A CN202110471569 A CN 202110471569A CN 113274032 A CN113274032 A CN 113274032A
- Authority
- CN
- China
- Prior art keywords
- patient
- interface
- training
- rehabilitation
- ssvep
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 219
- 238000000034 method Methods 0.000 title claims abstract description 25
- 208000006011 Stroke Diseases 0.000 title claims description 43
- 230000002490 cerebral effect Effects 0.000 title claims description 8
- 206010008190 Cerebrovascular accident Diseases 0.000 title claims description 7
- 230000000638 stimulation Effects 0.000 claims abstract description 70
- 230000002452 interceptive effect Effects 0.000 claims abstract description 42
- 230000002093 peripheral effect Effects 0.000 claims abstract description 39
- 239000011521 glass Substances 0.000 claims abstract description 32
- 210000004556 brain Anatomy 0.000 claims abstract description 22
- 230000000875 corresponding effect Effects 0.000 claims description 35
- 238000007781 pre-processing Methods 0.000 claims description 26
- 238000012790 confirmation Methods 0.000 claims description 24
- 238000004891 communication Methods 0.000 claims description 16
- 230000001953 sensory effect Effects 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 13
- 238000002360 preparation method Methods 0.000 claims description 12
- 208000033986 Device capturing issue Diseases 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 235000005911 diet Nutrition 0.000 claims description 3
- 230000037213 diet Effects 0.000 claims description 3
- 230000005611 electricity Effects 0.000 claims description 3
- 238000010187 selection method Methods 0.000 claims description 3
- 230000033001 locomotion Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 abstract description 6
- 230000003930 cognitive ability Effects 0.000 abstract description 4
- 238000011084 recovery Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000006467 substitution reaction Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 241000234295 Musa Species 0.000 description 2
- 235000018290 Musa x paradisiaca Nutrition 0.000 description 2
- 235000014443 Pyrus communis Nutrition 0.000 description 2
- 235000013361 beverage Nutrition 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 235000008429 bread Nutrition 0.000 description 2
- 230000035622 drinking Effects 0.000 description 2
- 235000013601 eggs Nutrition 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 229910052500 inorganic mineral Inorganic materials 0.000 description 2
- 239000011707 mineral Substances 0.000 description 2
- 230000007659 motor function Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 206010037714 Quadriplegia Diseases 0.000 description 1
- 235000015895 biscuits Nutrition 0.000 description 1
- 238000002507 cathodic stripping potentiometry Methods 0.000 description 1
- 235000014510 cooky Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 239000011664 nicotinic acid Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009861 stroke prevention Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/36003—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation of motor muscles, e.g. for walking assistance
Abstract
The invention relates to a stroke rehabilitation training system and method based on SSVEP + MI brain-computer interface, the system includes: the multifunctional electric brain rehabilitation robot comprises a computer (1), an electroencephalogram cap (2), a mechanical arm (3), AR glasses (4), an electric stimulation feedback instrument (5), a rehabilitation pneumatic hand (6), an SSVEP stimulation interface (7), an SSVEP online training unit (8), an MI interactive interface (9), an MI offline training unit (10) and an MI online training unit (11). The method comprises the following steps: the SSVEP brain-computer interface is combined with the AR and the mechanical arm in the first stage of rehabilitation training, the SSVEP-EEG is decoded through the FBCCA algorithm, the intention of the patient can be expressed, the life can be assisted, the attention and the cognitive ability of the patient can be trained, and a foundation is laid for the second stage of rehabilitation training. In the second stage of rehabilitation training, an MI brain-computer interface is combined with AR and rehabilitation peripherals, MI-EEG is decoded through an FBCSP algorithm, a closed-loop rehabilitation loop can be formed in multiple feedback modes, the function control connection between the external limb and the brain is repaired, and the recovery of limb movement functions of a patient is facilitated.
Description
Technical Field
The invention relates to the technical field of bioelectrical signal processing, in particular to a stroke rehabilitation training system and method based on an SSVEP + MI brain-computer interface.
Background
The stroke is also called as stroke, is the first cause of death and disability of adults in China, and has the five characteristics of high morbidity, high disability rate, high mortality, high recurrence rate and high economic burden. According to the Chinese stroke prevention and treatment report 2019, the following results are shown: the number of people who suffer from stroke is 1318 ten thousand for people older than 40 years old, and more than 190 people die of stroke every year. Modern rehabilitation theory and practice prove that: if the cerebral apoplexy patient carries out rehabilitation training in time, can effectively resume the limb motion function that suffers from. After the vital signs of the stroke patient are stable, the stroke patient needs to perform rehabilitation training as early as possible, but the stroke patient hardly expresses intention in the early period of the rehabilitation training and has difficulty in rehabilitation training due to inconvenience in movement. In addition, the vast majority of cerebral apoplexy rehabilitation training devices are all recovered passively, not only consume great manpower, lack interesting and patient's initiative and participate in, more lack cerebral motor nervous system's direct participation moreover, can't help cerebral apoplexy effectively recovered.
Disclosure of Invention
The invention aims to provide a stroke rehabilitation training system and method based on an SSVEP + MI brain-computer interface aiming at the defects of the prior art, so that the defects of the existing stroke rehabilitation training device and method are overcome, and the recovery of a stroke patient is promoted.
In order to achieve the purpose, the invention adopts the following technical scheme:
a stroke rehabilitation training system based on an SSVEP + MI brain-computer interface comprises a computer, an electroencephalogram cap, a mechanical arm and a rehabilitation peripheral, wherein the computer comprises an SSVEP stimulation interface, an SSVEP online training unit, an MI interactive interface, an MI offline training unit and an MI online training unit;
the electroencephalogram cap is used for collecting electroencephalogram signals generated when a patient watches the corresponding block of the SSVEP stimulation interface and sending the electroencephalogram signals to the computer;
the SSVEP on-line training unit in the computer is used for preprocessing the electroencephalogram signal, identifying the electroencephalogram signal by using an FBCCA algorithm and acquiring intention information of a patient;
the computer is further used for generating a mechanical arm control instruction according to the intention information and sending the mechanical arm control instruction to the mechanical arm so that the mechanical arm executes corresponding actions according to the mechanical arm control instruction;
the mechanical arm is used for executing corresponding actions according to the mechanical arm control instruction;
the electroencephalogram cap is further used for acquiring an offline training electroencephalogram signal generated when a patient watches the MI interactive interface to perform motor imagery during offline training;
the MI off-line training unit in the computer is used for preprocessing the off-line training electroencephalogram signal and generating a training model by using an FBCSP algorithm;
the electroencephalogram cap is also used for acquiring online training electroencephalogram signals generated when a patient watches the MI interactive interface to perform motor imagery during online training;
the MI on-line training unit in the computer is used for preprocessing the on-line training electroencephalogram signals and classifying and identifying the on-line training electroencephalogram signals according to the training model and the FBCSP algorithm to obtain the motor imagery intention of the patient;
the computer is also used for generating a control instruction of a rehabilitation peripheral according to the motor imagery intention and sending the control instruction of the rehabilitation peripheral to the rehabilitation peripheral so as to enable the rehabilitation peripheral to generate and send sensory feedback to a patient;
the rehabilitation peripheral is used for generating and sending sensory feedback to the patient.
The rehabilitation peripheral comprises: the AR glasses are in communication connection with the computer through a TCP, the electric stimulation feedback instrument is in communication with the computer through a serial port, and the rehabilitation pneumatic hand is in communication with the computer through the serial port; the sensory feedback includes: auditory feedback, visual feedback, tactile feedback; wherein:
the AR glasses to send auditory and visual feedback to the patient;
the rehabilitation pneumatic hand is used for sending tactile feedback to a patient;
the electrical stimulation feedback instrument is used for sending tactile feedback to the patient.
The SSVEP stimulation interface comprises an intention expression block, an auxiliary life block and a confirmation block;
wherein the secondary interface of the intent expression comprises a plurality of life scene interface blocks, a confirmation block and an exit block; the secondary life-support interface includes a plurality of diet blocks, confirmation blocks, and exit blocks.
The MI interactive interface is divided into a preparation interface, a database interface and a functional interface; wherein:
the preparation interface is used for connecting the electroencephalogram cap, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand, and selecting and previewing an AR scene;
the database interface is used for registering or inquiring personal information of the patient and storing rehabilitation training data of the patient;
the functional interface is used for displaying the quality of motor imagery electroencephalogram signals of a patient, setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand, and guiding the patient to perform off-line training and on-line training.
The rehabilitation training is divided into two stages:
in the first stage of rehabilitation training, the computer generates an SSVEP stimulation interface, a patient can watch the corresponding flickering block on the SSVEP stimulation interface according to the requirement of the patient, and the computer executes the SSVEP online training unit. After preprocessing, FBCCA algorithm and classified recognition, the electroencephalogram signals can be used for analyzing the intention of the patient, and the intention of the patient can be converted into a control instruction of a mechanical arm, so that the effect of assisting the life of the patient is achieved. The SSVEP brain-computer interface is combined with AR glasses and mechanical arms to carry out rehabilitation training, so that the attention and the cognitive ability of a patient can be improved, and a foundation is laid for motor imagery rehabilitation training. A stroke multi-stage rehabilitation training system and method based on SSVEP + MI brain-computer interface is provided, wherein the first stage rehabilitation training is operated by adopting the system, and the operation steps are as follows:
(1-1) the medical staff helps the patient to wear the electroencephalogram cap and the AR glasses and open the SSVEP stimulation interface;
(1-2) when a stimulation flicker appears on a primary interface of the SSVEP stimulation interface, a patient firstly watches the intention expression or the auxiliary life block of the primary interface and then watches the confirmation block, and then the corresponding secondary interface can be entered;
(1-3) if the intention expression is selected, the patient looks at the life need block first and then at the confirmation block. If the auxiliary life is selected, the patient watches the required object block first and then watches the confirmation block. If the user watches the exit block, returning to the first-level interface;
(1-4) when the patient watches the flickering block, the electroencephalogram acquisition device acquires electroencephalogram signals of the patient and transmits the electroencephalogram signals to the SSVEP online training unit of the computer through TCP communication;
(1-5) after the SSVEP online training unit carries out preprocessing, FBCCA algorithm and classification recognition on the electroencephalogram signals, a block watched by a patient can be analyzed;
(1-6) if the intention expression is selected, the intention of the patient can be directly analyzed. If the auxiliary life is selected, the computer sends a control instruction to the mechanical arm, and the mechanical arm helps the patient to grab the corresponding article.
The primary interface of the SSVEP stimulation interface in the step (1-2) comprises 3 blocks of intention expression, life assistance and confirmation. The secondary interface of the SSVEP stimulation interface in the step 2) is divided into an intention expression secondary interface and an auxiliary life secondary interface. The second level interface of the intention expression comprises 12 blocks of eating, drinking, massaging, chatting, sleeping, washing, urinating, defecating, watching TV, listening to music, confirming and quitting; the secondary interface for assisting life comprises 12 blocks including apple, banana, orange, pear, egg, bread, steamed stuffed bun, biscuit, mineral water, beverage, affirmation and quit.
The SSVEP online training unit in the step (1-5) comprises the following specific steps:
(1-5-1) carrying out channel selection on the original electroencephalogram signal, removing baseline drift, and removing power frequency interference and artifact interference;
(1-5-2) carrying out filter bank analysis on the preprocessed electroencephalogram signals, and decomposing the SSVEP electroencephalogram signals through a plurality of different frequency bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
(1-5-3) carrying out correlation analysis on each subband component obtained by filter bank analysis and a standard sine and cosine reference signal;
(1-5-4) the frequency corresponding to the maximum correlation is the identification result.
In the second stage of rehabilitation training, the computer generates an MI interface for guiding the patient to perform off-line training and on-line training. The patient first performs MI offline training according to the prompts in the AR glasses, and the computer executes the MI offline training unit. The electroencephalogram signals can provide a model for the on-line training of patients after being preprocessed, processed by an FBCSP algorithm and trained and modeled. When MI is trained on line, the patient only needs to imagine limb movement, and the computer executes the MI on-line training unit. After preprocessing, FBCSP algorithm and classified recognition, the electroencephalogram signals can analyze the motor imagery intentions of the patient and convert the motor imagery intentions into control instructions of rehabilitation peripherals, AR glasses give visual and auditory feedback to the patient, and an electrical stimulation feedback instrument and a rehabilitation pneumatic hand give tactile feedback to the patient. The motor imagery rehabilitation training with multiple feedbacks can reshape the nerve center of the brain, repair the function control connection between the external limbs and the brain, and recover the limb motor function of the patient. A stroke multi-stage rehabilitation training system and method based on SSVEP + MI brain-computer interface is provided, the second stage rehabilitation training is operated by adopting the system, and the operation steps are as follows:
(2-1) the medical staff helps the patient to wear the electroencephalogram cap and the AR glasses and open the MI interactive interface;
(2-2) firstly, opening a preparation interface of the MI interactive interface, connecting the electroencephalogram cap, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand, and selecting an AR scene;
(2-3) re-entering a database interface of the MI interactive interface, and registering or inquiring personal information of the patient;
(2-4) entering a functional interface of the MI interactive interface, and setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand;
(2-5) enabling the patient to perform motor imagery according to the prompt in the AR glasses, then performing training modeling by the MI offline training unit, and providing a model for the MI online training unit;
(2-6) performing on-line training on the patient, and analyzing the motor imagery intention of the patient by the MI on-line training unit and converting the motor imagery intention into control instructions of the AR glasses, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand;
(2-7) the AR glasses give visual and auditory feedback to the patient, and the electro-stimulation feedback device and the rehabilitating pneumatic hand give tactile feedback to the patient.
The specific steps of the MI offline training unit and the MI online training unit in the step (2-5) are as follows:
(2-5-1) carrying out channel selection on the original electroencephalogram signal, removing baseline drift, and removing power frequency interference and artifact interference;
(2-5-2) decomposing the brain electrical signals after the preprocessing into signals of a plurality of frequency bands by using a filter bank;
(2-5-3) carrying out feature extraction on the electroencephalogram signals on a plurality of frequency bands by using a CSP algorithm;
(2-5-4) selecting an optimal frequency band which can minimize the error of the classification result by adopting a feature selection method, and selecting the optimal features in the features extracted by the CSP algorithm;
(2-5-5) if the training is off-line, carrying out training modeling by using the extracted features and the motor imagery label to generate a model;
and (2-5-6) if the training is on-line, carrying out classification and recognition by using the extracted features and a model provided by off-line training, and analyzing the motor imagery intention of the patient.
The invention discloses a second aspect provides a stroke rehabilitation training method based on SSVEP + MI brain-computer interface, comprising the following steps:
acquiring electroencephalogram signals generated by a corresponding block when a patient watches an SSVEP stimulation interface; preprocessing the electroencephalogram signals, identifying by using an FBCCA algorithm, and acquiring intention information of a patient; generating a mechanical arm control instruction according to the intention information; and sending the mechanical arm control instruction to the mechanical arm so that the mechanical arm can grab the corresponding article according to the mechanical arm control instruction.
Acquiring electroencephalogram signals generated by corresponding blocks when a patient watches an MI interactive interface to perform motor imagery during off-line training; preprocessing the electroencephalogram signals of the patient during off-line training, and generating a training model by using an FBCSP algorithm; acquiring electroencephalogram signals generated by a corresponding block when a patient watches an MI interactive interface to perform motor imagery when the patient is online; after preprocessing the on-line electroencephalogram signals of the patient, classifying and identifying the on-line electroencephalogram signals of the patient according to the training model and the FBCSP algorithm to obtain the motor imagery intention of the patient; generating a control instruction of a rehabilitation peripheral according to the motor imagery intention; sending control instructions of the rehabilitation peripheral to a rehabilitation peripheral to cause the rehabilitation peripheral to generate and send sensory feedback to a patient, wherein the sensory feedback comprises: visual feedback, auditory feedback, and tactile feedback.
Compared with the prior art, the invention has the following obvious prominent substantive characteristics and obvious advantages:
1. the invention provides a stroke multi-stage rehabilitation training system and method based on an SSVEP + MI brain-computer interface, wherein in the first stage, the SSVEP brain-computer interface is adopted to combine AR and a mechanical arm to perform rehabilitation training: the SSVEP can be applied to the intention expression of the patient, the patient can watch the flickering picture on the intention expression interface according to the requirement of the patient, and the system can decode the intention of the patient;
2. the invention combines the mechanical arm, and can help the patient to grab the corresponding article, thereby playing the role of assisting the daily life of the patient;
3. the invention adopts AR, which not only can improve the training enthusiasm of the patient, but also can facilitate the rehabilitation training of the hemiplegic patient. The first stage of rehabilitation training can improve the attention and cognitive ability of the patient, and lays a foundation for the second stage of rehabilitation training. In the second stage, the MI brain-computer interface is adopted to combine with AR and rehabilitation peripherals to carry out rehabilitation training: the patient leads to the quadriplegia owing to the brain nerve region that governs limb movement receives the damage, and when using this system, the patient only needs imagination limb movement, thereby the system can drive patient's limb movement through recovered peripheral hardware and give patient tactile feedback, AR can improve patient's training enthusiasm and can give patient's vision and hearing feedback, through multiple feedback mode, form closed loop rehabilitation return circuit, rebuild the cortex of impaired brain, restore the function control connection between outside limbs and the brain, promote the recovery of cerebral apoplexy patient.
Drawings
FIG. 1 is a block diagram of the system architecture of the present invention.
Fig. 2 is a flow chart of the first stage rehabilitation training of the present invention.
Fig. 3 is a flow chart of the second stage rehabilitation training of the present invention.
FIG. 4 is the primary interface of the SSVEP stimulation interface of the present invention.
FIG. 5 is an intended expression secondary interface of the SSVEP stimulation interface of the present invention.
FIG. 6 is an assisted living secondary interface of the SSVEP stimulation interface of the present invention.
Fig. 7 is a preparation interface of the MI interactive interface of the present invention.
Fig. 8 is a database interface of the MI interactive interface of the present invention.
Fig. 9 is a functional interface of the MI interactive interface of the present invention.
FIG. 10 is a block diagram of the SSVEP online training unit of the present invention.
Fig. 11 is a block diagram of an MI offline training unit and an MI online training unit of the present invention.
Detailed Description
The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings:
the first embodiment is as follows:
referring to fig. 1, a stroke rehabilitation training system based on an SSVEP + MI brain-computer interface includes a computer 1, an electroencephalogram cap 2, a mechanical arm 3, and a rehabilitation peripheral, where the computer 1 includes: the brain-computer-controlled rehabilitation training system comprises an SSVEP stimulation interface 7, an SSVEP online training unit 8, an MI interactive interface 9, an MI offline training unit 10 and an MI online training unit 11, wherein the brain-computer cap 2 is in communication connection with the computer 1 through a TCP (transmission control protocol), the mechanical arm 3 is connected with the computer 1 through a network cable, and the rehabilitation peripheral is connected with the computer 1 through a TCP or a communication serial port;
the electroencephalogram cap 2 is used for collecting electroencephalogram signals generated when a patient watches the corresponding block of the SSVEP stimulation interface 7 and sending the electroencephalogram signals to the computer 1;
the SSVEP online training unit 9 in the computer 1 is used for preprocessing the electroencephalogram signal, identifying the electroencephalogram signal by using an FBCCA algorithm, and acquiring intention information of a patient;
the computer 1 is further configured to generate a mechanical arm control instruction according to the intention information, and send the mechanical arm control instruction to the mechanical arm 3, so that the mechanical arm executes a corresponding action according to the mechanical arm control instruction;
the mechanical arm 3 is used for executing corresponding actions according to the mechanical arm control instruction;
the electroencephalogram cap 2 is further used for acquiring an offline training electroencephalogram signal generated when a patient exercises offline and watches the MI interactive interface 9 for motor imagery;
an MI offline training unit 10 in the computer 1 is used for preprocessing the offline training electroencephalogram signal and generating a training model by using an FBCSP algorithm;
the electroencephalogram cap 2 is also used for acquiring online training electroencephalogram signals generated when a patient watches the MI interactive interface 9 for motor imagery during online training;
the MI on-line training unit 8 in the computer 1 is used for preprocessing the on-line training electroencephalogram signals, classifying and identifying the on-line training electroencephalogram signals according to the training model and the FBCSP algorithm, and obtaining the motor imagery intention of the patient;
the computer 1 is further configured to generate a control instruction of a rehabilitation peripheral according to the motor imagery intention, and send the control instruction of the rehabilitation peripheral to the rehabilitation peripheral so that the rehabilitation peripheral generates and sends sensory feedback to a patient;
the rehabilitation peripheral is used for generating and sending sensory feedback to the patient.
The stroke rehabilitation training system based on the SSVEP + MI brain machine interface improves the defects of the existing stroke rehabilitation training equipment and method, and promotes the rehabilitation of stroke patients.
Example two:
this embodiment is substantially the same as the first embodiment, and is characterized in that:
the rehabilitation peripheral comprises: the system comprises AR glasses 4, an electrical stimulation feedback instrument 5 and a rehabilitation pneumatic hand 6, wherein the AR glasses 4 are in communication connection with the computer 1 through a TCP, the electrical stimulation feedback instrument 5 is in communication with the computer 1 through a serial port, and the rehabilitation pneumatic hand 6 is in communication with the computer 1 through a serial port; the sensory feedback includes: auditory feedback, visual feedback, tactile feedback; wherein:
the AR glasses 4 are used for sending auditory feedback and visual feedback to the patient;
the rehabilitation pneumatic hand 6 is used for sending tactile feedback to the patient;
the electrical stimulation feedback instrument 5 is used for sending tactile feedback to the patient.
The SSVEP stimulation interface comprises an intention expression block, an auxiliary life block and a confirmation block;
wherein the secondary interface of the intent expression comprises a plurality of life scene interface blocks, a confirmation block and an exit block; the secondary life-support interface includes a plurality of diet blocks, confirmation blocks, and exit blocks.
The MI interactive interface is divided into a preparation interface, a database interface and a functional interface; wherein:
the preparation interface is used for connecting the electroencephalogram cap, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand, and selecting and previewing an AR scene;
the database interface is used for registering or inquiring personal information of the patient and storing rehabilitation training data of the patient;
the functional interface is used for displaying the quality of motor imagery electroencephalogram signals of a patient, setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand, and guiding the patient to perform off-line training and on-line training.
The multistage cerebral stroke rehabilitation training system based on the SSVEP + MI brain-computer interface in the embodiment adopts the SSVEP brain-computer interface to combine AR and the mechanical arm to perform rehabilitation training: the SSVEP can be applied to the intention expression of the patient, the patient can watch the flickering picture on the intention expression interface according to the requirement of the patient, and the system can decode the intention of the patient.
Example three:
in this embodiment, as shown in fig. 1, a multistage stroke rehabilitation training system and method based on an SSVEP + MI brain-computer interface includes a computer, an electroencephalogram cap, a mechanical arm, AR glasses, an electrical stimulation feedback instrument, a pneumatic rehabilitation hand, an SSVEP stimulation interface, an SSVEP online training unit, an MI interactive interface, an MI offline training unit, and an MI online training unit. The method is characterized in that: the electroencephalogram cap is communicated with a computer through a TCP (transmission control protocol), the mechanical arm is communicated with the computer through a network cable, the AR glasses are communicated with the computer through the TCP, the electrical stimulation feedback instrument is communicated with the computer through a serial port, and the rehabilitation pneumatic hand is communicated with the computer through the serial port.
During the first stage of rehabilitation training, the computer generates an SSVEP stimulation interface. The patient can watch the corresponding flickering block on the SSVEP stimulation interface according to the requirement of the patient, and the computer can execute the SSVEP on-line training unit. After preprocessing, FBCCA algorithm and classified recognition, the electroencephalogram signals can be used for analyzing the intentions of the patients, the intentions of the patients can be converted into control instructions of the mechanical arm, and the mechanical arm helps the patients to grab corresponding articles and plays a role in assisting the lives of the patients. The first stage of rehabilitation training can improve the attention and cognitive ability of the patient, and lays a foundation for the second stage of rehabilitation training.
During the second phase of rehabilitation, the computer generates an MI interface for guiding the patient through off-line and on-line training. The patient first performs MI offline training according to the prompts in the AR glasses, and the computer executes the MI offline training unit. The electroencephalogram signals can provide a model for the on-line training of patients after being preprocessed, processed by an FBCSP algorithm and trained and modeled. When MI is trained on line, the patient only needs to imagine limb movement, and the computer executes the MI on-line training unit. After preprocessing, FBCSP algorithm and classification recognition, the electroencephalogram signals can analyze the motor imagery intention of the patient, and the intention can be converted into control instructions of AR glasses, so that visual and auditory feedback is given to the patient; can be converted into a control instruction of an electrical stimulation feedback instrument to give tactile feedback to the patient so as to train the limb of the patient; can be converted into control commands of a rehabilitation pneumatic hand, and gives tactile feedback to a patient so as to train the finger of the patient. The second stage of rehabilitation training can remodel the nerve centre of the brain, repair the functional control connection between the external limbs and the brain and recover the limb motor function of the patient.
Example four:
the present embodiment is basically the same as the third embodiment, and is characterized in that:
fig. 2 is a flowchart of the first stage rehabilitation training, and the specific operation steps are as follows:
1) the medical staff helps the patient to wear the electroencephalogram cap and the AR, and the SSVEP stimulation interface is opened;
2) when the first-level interface of the SSVEP stimulation interface has stimulation flicker, a patient firstly watches the intention expression or the auxiliary living block of the first-level interface and then watches the confirmation block, and then the corresponding second-level interface can be accessed;
3) if the intent expression is selected, the patient looks at the life need block first, and then looks at the confirmation block. If the auxiliary life is selected, the patient watches the required object block first and then watches the confirmation block. If the user watches the exit block, returning to the first-level interface;
4) when a patient watches the flickering block, the electroencephalogram acquisition device acquires electroencephalogram signals of the patient and transmits the electroencephalogram signals to the SSVEP online training unit of the computer through TCP communication;
5) after the SSVEP online training unit carries out preprocessing, FBCCA algorithm and classification recognition on the electroencephalogram signals, a block watched by a patient can be analyzed;
6) if intent expression is selected, the intent of the patient can be analyzed directly. If the auxiliary life is selected, the computer sends a control instruction to the mechanical arm, and the mechanical arm helps the patient to grab the corresponding article.
Fig. 3 is a flow chart of the second stage rehabilitation training, and the specific operation steps are as follows:
1) medical staff help patients wear electroencephalogram caps and AR glasses and open MI interactive interfaces;
2) firstly, opening a preparation interface of an MI interactive interface, connecting an electroencephalogram cap, an electrical stimulation feedback instrument and a rehabilitation pneumatic hand, and selecting an AR scene;
3) then entering a database interface of the MI interactive interface to register or inquire personal information of the patient;
4) then entering a functional interface of an MI interactive interface, and setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand;
5) enabling a patient to perform motor imagery according to prompts in AR glasses, and then training and modeling by the MI offline training unit to provide a model for the MI online training unit;
6) then the patient is trained on line, the MI on-line training unit can analyze the motor imagery intention of the patient and convert the motor imagery intention into control instructions of AR glasses, an electrical stimulation feedback instrument and a rehabilitation pneumatic hand;
7) the AR glasses give visual and auditory feedback to the patient, and the electro-stimulation feedback instruments and the rehabilitating pneumatic hands give tactile feedback to the patient.
Example five:
this embodiment is substantially the same as the previous embodiment, and is characterized in that:
the primary interface of the SSVEP stimulation interface is shown in fig. 4, and includes 3 blocks of intent expression, assisted living, and confirmation. The secondary interface of the SSVEP stimulation interface is divided into an intention expression secondary interface and an auxiliary life secondary interface. The intent expression secondary interface of the SSVEP stimulus interface is shown in fig. 5, and includes 12 blocks including eating, drinking, massaging, chatting, sleeping, washing, urinating, defecating, watching tv, listening to music, confirming, and quitting; fig. 6 shows secondary interfaces of assisted living in SSVEP stimulation interfaces, including apple, banana, orange, pear, egg, bread, steamed stuffed bun, cookie, mineral water, beverage, confirmation and withdrawal, for 12 blocks. The patient can enter the corresponding secondary interface by watching the intention expression of the primary interface or the auxiliary living block and then watching the confirmation block. If the intention expression is selected, the patient firstly watches the life demand block and then watches the confirmation block, and then the intention expression can be realized. Returning to the primary interface if the exit block is watched; if the auxiliary life is selected, the patient firstly watches the required object block and then watches the confirmation block, and the mechanical arm can help the patient to grab the corresponding object. And returning to the primary interface after watching the exit block.
The preparation interface of the MI interactive interface is shown in fig. 7, and the preparation interface of the MI interactive interface is used for connecting an electroencephalogram cap, an electrical stimulation feedback instrument and a rehabilitation pneumatic hand and can also be used for selecting and previewing an AR scene. The database interface of the MI interactive interface is shown in fig. 8, and the database interface of the MI interactive interface can register or query personal information of a patient, and can also store rehabilitation training data of the patient after the rehabilitation training of the patient is finished. The functional interface of the MI interactive interface is shown in fig. 9, the functional interface of the MI interactive interface can display the motor imagery electroencephalogram signal quality of a patient, set the relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand, and guide the patient to perform off-line training and on-line training.
Example six:
this embodiment is substantially the same as the previous embodiment, and is characterized in that:
in this embodiment, as shown in fig. 1, a multistage stroke rehabilitation training system and method based on an SSVEP + MI brain-computer interface includes a computer, an electroencephalogram cap, a mechanical arm, AR glasses, an electrical stimulation feedback instrument, a pneumatic rehabilitation hand, an SSVEP stimulation interface, an SSVEP online training unit, an MI interactive interface, an MI offline training unit, and an MI online training unit. The brain electricity cap adopts the BCR 64 lead brain electricity cap, and the sampling rate is 1000Hz, communicates with the computer through TCP. The mechanical arm adopts Kinova6 axle bionic mechanical arm, communicates through the net twine with the computer. The AR glasses adopt HoloLens second generation and communicate with the computer through TCP. The electrical stimulation feedback instrument adopts an XCH-C1 biofeedback rehabilitation instrument and is communicated with a computer through a serial port. The rehabilitation pneumatic hand adopts a health robot with the function of a Yi Sheng hand and is communicated with a computer through a serial port.
The flicker frequency of 3 blocks in the primary interface of the SSVEP stimulation interface is 9Hz, 13Hz and 11Hz in sequence. Intent of the SSVEP stimulation interface to express that the flicker frequency of the 12 blocks in the secondary interface was 9Hz, 13.5Hz, 11Hz, 13Hz, 14Hz, 9.5Hz, 12.5Hz, 10.5Hz, 12Hz, 14.5Hz, 10Hz, and 11.5Hz in that order. The same is true for the flicker frequency of 12 blocks in the secondary life interface. The flicker frequency of different blocks is different, according to the SSVEP visual induction principle, the patient watches different blocks, namely watches different frequency signals, the brain generates different electroencephalogram signals, and the intention of the patient can be analyzed through the SSVEP online training unit.
The SSVEP online training unit adopts an FBCCA algorithm, the method is an improvement on a CCA (Canonica correlation analysis) algorithm, and a filter bank is added on the basis of the CCA, so that electroencephalogram signal harmonic components which are not fully utilized in the traditional CCA algorithm are utilized, and the algorithm precision is improved. FIG. 10 is a diagram of an SSVEP online training unit structure, which includes the following steps:
1) the channel selection is firstly carried out on the original brain electrical signals, and the SSVEP mainly adopts brain electrical signals of 10 channels (T5, T6, P3, P4, Pz, PO3, PO4, O1, O2 and Oz) corresponding to the occipital area of the brain relevant to vision. Then removing baseline drift, and removing 50Hz power frequency interference by using a notch filter;
2) constructing N band-pass filters, wherein all the band-pass filters cover all the band-pass frequencies as much as possible;
3) suppose that the order is solved for frequency fkThe maximum correlation coefficient of the signal of (2) and the electroencephalogram signal X. Respectively introducing the preprocessed electroencephalogram signal data into N different band-pass filters to obtain N groups of data after the data pass through the band-pass filters;
4) solving a maximum correlation coefficient of a reference signal formed by each group of data and standard sine and cosine by using a CCA algorithm to obtain N phase relation numbers;
5) according to the formula w (n) ═ ae-bn+ c solving harmonic weight and substituting the result into formula
Solving the sum of the weights as frequency fkThe maximum correlation coefficient rho of the signal of (A) and the electroencephalogram signal Xk；
6) And (3) respectively substituting the reference signals with 12 frequencies into the steps 2) to 4) to find the maximum correlation coefficient, wherein the corresponding frequency is the frequency of the SSVEP signal, and then obtaining the identification result.
The MI off-line training unit and the MI on-line training unit adopt an FBCSP algorithm, the method is an improvement on a CSP (Common Spatial Pattern) algorithm, frequency band optimization and feature selection are carried out on the basis of the CSP algorithm, the optimal frequency band which enables the error of a classification result to be minimum can be selected, the optimal feature in the features extracted by the CSP algorithm is selected, and the classification and identification accuracy is improved. Fig. 11 is a structural diagram of an MI offline training unit and an MI online training unit, and the specific steps are as follows:
1) the channel selection is firstly carried out on the original brain electrical signals, and brain electrical signals of 12 channels (Fz, Cz, Pz, Oz, C1, C2, C3, C4, C5, C6, P3 and P4) related to motor imagery are adopted. Then removing baseline drift, and removing 50Hz power frequency interference by using a notch filter;
2) decomposing MI-EEG into signals on 9 frequency bands of 4-8Hz, 8-12Hz, 12-16Hz, … and 36-40Hz within the range of 4-40Hz by adopting a filter bank consisting of ChebyshevII type filters, wherein the bandwidth of the filter bank is 4 Hz;
3) then spatial filtering is performed by using CSPs respectively. Through CSP feature extraction, each sub-band can finally obtain a feature vector fb＝{fb,1,fb,2,…,fb,2mB is more than or equal to 1 and less than or equal to 9, wherein m is the logarithm of the selected characteristics in CSP characteristic extraction;
4) selecting an optimal frequency band which can enable the error of the classification result to be minimum by adopting an MIBIF (Multi Information-based Best Individual Feature based on Mutual Information) algorithm, and selecting corresponding k Feature subsets from 9 x 2m features extracted from CSP features;
5) the last feature classification method adopts an SVM (Support Vector Machine) algorithm, the principle of SVM classification is to find a Support Vector for constructing an optimal classification hyperplane in a training sample, and the Support Vector can be classified into a quadratic rule problem with inequality constraint.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the embodiments, and various changes can be made according to the objects of the invention. All changes, modifications, substitutions, combinations, or simplifications that may be made in accordance with the spirit and principles of the present invention are intended to be equivalent substitutions. The invention also provides a multi-stage stroke rehabilitation training system and method based on the SSVEP + MI brain-computer interface, which are suitable for the purpose of the invention and do not depart from the technical principles and the inventive concept of the system and the method.
Example seven:
a stroke rehabilitation training method based on SSVEP + MI brain machine interface is operated by adopting the system, and is characterized in that:
the SSVEP online training unit comprises the following specific operation steps:
selecting channels of the original electroencephalogram signals, removing baseline drift, and removing power frequency interference and artifact interference;
performing filter bank analysis on the preprocessed electroencephalogram signals, and decomposing the SSVEP electroencephalogram signals through a plurality of different frequency bands of a filter to obtain sub-band signals passing through each sub-band of the filter;
performing correlation analysis on each subband component obtained by filter bank analysis and a standard sine and cosine reference signal;
and taking the block with the frequency corresponding to the maximum correlation as the identification result of the intention information of the patient.
b. Applied to the computer side, the operation steps are as follows:
acquiring an electroencephalogram signal generated when a patient watches a corresponding block of an SSVEP stimulation interface;
preprocessing the electroencephalogram signals, identifying by using an FBCCA algorithm, and acquiring intention information of a patient;
generating a mechanical arm control instruction according to the intention information;
sending the mechanical arm control instruction to the mechanical arm so that the mechanical arm executes corresponding action according to the mechanical arm control instruction;
acquiring an off-line training electroencephalogram signal generated when a patient watches an MI interactive interface to perform motor imagery during off-line training;
preprocessing the off-line training electroencephalogram signal, and generating a training model by using an FBCSP algorithm;
acquiring an online training electroencephalogram signal generated when a patient watches an MI interactive interface to perform motor imagery during online training;
preprocessing the online training electroencephalogram signals, and classifying and identifying the online training electroencephalogram signals according to the training model and the FBCSP algorithm to obtain the motor imagery intention of the patient;
generating a control instruction of a rehabilitation peripheral according to the motor imagery intention;
sending the control instruction of the rehabilitation peripheral to the rehabilitation peripheral so that the rehabilitation peripheral generates and sends sensory feedback to the patient.
The specific operation steps of the MI off-line training unit and the MI on-line training unit are as follows:
selecting channels of the original electroencephalogram signals, removing baseline drift, and removing power frequency interference and artifact interference;
decomposing the preprocessed electroencephalogram signal into signals of a plurality of frequency bands by using a filter bank;
performing feature extraction on the electroencephalogram signals on a plurality of frequency bands by using a CSP (Common Spatial Pattern) algorithm;
selecting an optimal frequency band which can minimize the error of the classification result by adopting a feature selection method, and selecting optimal features in the features extracted by the CSP algorithm;
if the training is off-line, training and modeling are carried out by using the extracted features and the motor imagery label to generate a model;
if the training is on-line, the extracted features and the model provided by off-line training are used for classification and identification, and the motor imagery intention of the patient is analyzed.
The embodiments of the present invention have been described with reference to the accompanying drawings, but the present invention is not limited to the embodiments, and various changes and modifications can be made according to the purpose of the invention, and any changes, modifications, substitutions, combinations or simplifications made according to the spirit and principle of the technical solution of the present invention shall be equivalent substitutions, as long as the purpose of the present invention is met, and the present invention shall fall within the protection scope of the present invention without departing from the technical principle and inventive concept of the present invention.
Claims (8)
1. The utility model provides a cerebral apoplexy rehabilitation training system based on SSVEP + MI brain machine interface, includes computer (1), brain electricity cap (2), arm (3) and recovered peripheral hardware, its characterized in that: the computer (1) comprises an SSVEP stimulation interface (7), an SSVEP online training unit (8), an MI interactive interface (9), an MI offline training unit (10) and an MI online training unit (11), the electroencephalogram cap (2) is in communication connection with the computer (1) through a TCP, the mechanical arm (3) is connected with the computer (1) through a network cable, and the rehabilitation peripheral is connected with the computer (1) through a TCP or a communication serial port;
the electroencephalogram cap (2) is used for collecting electroencephalogram signals generated when a patient watches a corresponding block of the SSVEP stimulation interface (7), and sending the electroencephalogram signals to the computer (1);
the SSVEP online training unit (9) in the computer (1) is used for preprocessing the electroencephalogram signals, identifying the electroencephalogram signals by using an FBCCA algorithm and acquiring intention information of a patient;
the computer (1) is further used for generating a mechanical arm control instruction according to the intention information and sending the mechanical arm control instruction to the mechanical arm (3) so that the mechanical arm executes corresponding actions according to the mechanical arm control instruction;
the mechanical arm (3) is used for executing corresponding actions according to the mechanical arm control instruction;
the electroencephalogram cap (2) is further used for acquiring an offline training electroencephalogram signal generated when a patient watches the MI interactive interface (9) for motor imagery during offline training;
an MI off-line training unit (10) in the computer (1) is used for preprocessing the off-line training electroencephalogram signals and generating a training model by using an FBCSP algorithm;
the electroencephalogram cap (2) is also used for acquiring online training electroencephalogram signals generated when a patient watches the MI interactive interface (9) for motor imagery during online training;
an MI (MI) online training unit (8) in the computer (1) is used for preprocessing the online training electroencephalogram signals, classifying and identifying the online training electroencephalogram signals according to the training model and the FBCSP algorithm, and obtaining the motor imagery intention of the patient;
the computer (1) is also used for generating a control instruction of a rehabilitation peripheral according to the motor imagery intention and sending the control instruction of the rehabilitation peripheral to the rehabilitation peripheral so that the rehabilitation peripheral generates and sends sensory feedback to a patient;
the rehabilitation peripheral is used for generating and sending sensory feedback to the patient.
2. The stroke rehabilitation training system based on the SSVEP + MI brain-computer interface of claim 1, wherein: the rehabilitation peripheral comprises AR glasses (4), an electrical stimulation feedback instrument (5) and a rehabilitation pneumatic hand (6), wherein the AR glasses (4) are in communication connection with the computer (1) through TCP, the electrical stimulation feedback instrument (5) is in communication with the computer (1) through a serial port, and the rehabilitation pneumatic hand (6) is in communication with the computer (1) through the serial port; the sensory feedback includes: auditory feedback, visual feedback, tactile feedback; wherein:
the AR glasses (4) for sending auditory and visual feedback to the patient;
the rehabilitating pneumatic hand (6) for sending tactile feedback to the patient;
the electrical stimulation feedback instrument (5) is used for sending tactile feedback to the patient.
3. The stroke rehabilitation training system based on the SSVEP + MI brain-computer interface of claim 1, wherein:
the SSVEP stimulation interface comprises an intention expression block, an auxiliary life block and a confirmation block;
wherein the secondary interface of the intent expression comprises a plurality of life scene interface blocks, a confirmation block and an exit block; the secondary life-support interface includes a plurality of diet blocks, confirmation blocks, and exit blocks.
4. The stroke rehabilitation training system based on the SSVEP + MI brain-computer interface of claim 1, wherein: the MI interactive interface is divided into a preparation interface, a database interface and a functional interface; wherein:
the preparation interface is used for connecting the electroencephalogram cap, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand, and selecting and previewing an AR scene;
the database interface is used for registering or inquiring personal information of the patient and storing rehabilitation training data of the patient;
the functional interface is used for displaying the quality of motor imagery electroencephalogram signals of a patient, setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand, and guiding the patient to perform off-line training and on-line training.
5. A stroke rehabilitation training method based on SSVEP + MI brain-machine interface, which is operated by the stroke rehabilitation training system based on SSVEP + MI brain-machine interface of claim 1, wherein: the SSVEP online training unit comprises the following specific operation steps:
selecting channels of the original electroencephalogram signals, removing baseline drift, and removing power frequency interference and artifact interference;
performing filter bank analysis on the preprocessed electroencephalogram signals, and decomposing the SSVEP electroencephalogram signals through a plurality of different frequency bands of a filter to obtain sub-band signals passing through each sub-band of the filter;
performing correlation analysis on each subband component obtained by filter bank analysis and a standard sine and cosine reference signal;
and taking the block with the frequency corresponding to the maximum correlation as the identification result of the intention information of the patient.
6. A stroke rehabilitation training method based on SSVEP + MI brain-machine interface, which is operated according to the stroke rehabilitation training system based on SSVEP + MI brain-machine interface of claim 1, wherein: the specific operation steps of the MI off-line training unit and the MI on-line training unit are as follows:
selecting channels of the original electroencephalogram signals, removing baseline drift, and removing power frequency interference and artifact interference;
decomposing the preprocessed electroencephalogram signal into signals of a plurality of frequency bands by using a filter bank;
performing feature extraction on the electroencephalogram signals on a plurality of frequency bands by using a CSP (Common Spatial Pattern) algorithm;
selecting an optimal frequency band which can minimize the error of the classification result by adopting a feature selection method, and selecting optimal features in the features extracted by the CSP algorithm;
if the training is off-line, training and modeling are carried out by using the extracted features and the motor imagery label to generate a model;
if the training is on-line, the extracted features and the model provided by off-line training are used for classification and identification, and the motor imagery intention of the patient is analyzed.
7. A stroke rehabilitation training method based on SSVEP + MI brain-computer interface, which is operated by the stroke rehabilitation training system based on SSVEP + MI brain-computer interface according to claim 1, wherein: applied to the computer side, the operation steps are as follows:
acquiring an electroencephalogram signal generated when a patient watches a corresponding block of an SSVEP stimulation interface;
preprocessing the electroencephalogram signals, identifying by using an FBCCA algorithm, and acquiring intention information of a patient;
generating a mechanical arm control instruction according to the intention information;
sending the mechanical arm control instruction to the mechanical arm so that the mechanical arm executes corresponding action according to the mechanical arm control instruction;
acquiring an off-line training electroencephalogram signal generated when a patient watches an MI interactive interface to perform motor imagery during off-line training;
preprocessing the off-line training electroencephalogram signal, and generating a training model by using an FBCSP algorithm;
acquiring an online training electroencephalogram signal generated when a patient watches an MI interactive interface to perform motor imagery during online training;
preprocessing the online training electroencephalogram signals, and classifying and identifying the online training electroencephalogram signals according to the training model and the FBCSP algorithm to obtain the motor imagery intention of the patient;
generating a control instruction of a rehabilitation peripheral according to the motor imagery intention;
sending the control instruction of the rehabilitation peripheral to the rehabilitation peripheral so that the rehabilitation peripheral generates and sends sensory feedback to the patient.
8. A stroke rehabilitation training method based on SSVEP + MI brain-computer interface, which is operated by the stroke rehabilitation training system based on SSVEP + MI brain-computer interface according to claim 1, and is characterized by comprising the following operation steps:
1) and a first stage of rehabilitation training:
(1-1) the medical staff helps the patient to wear the electroencephalogram cap and the AR glasses and open the SSVEP stimulation interface;
(1-2) when a stimulation flicker appears on a primary interface of the SSVEP stimulation interface, a patient firstly watches the intention expression or the auxiliary life block of the primary interface and then watches the confirmation block, and then the corresponding secondary interface can be entered;
(1-3) if the intention expression is selected, the patient looks at the life need block first and then at the confirmation block. If the auxiliary life is selected, the patient watches the required object block first and then watches the confirmation block. If the user watches the exit block, returning to the first-level interface;
(1-4) when the patient watches the flickering block, the electroencephalogram acquisition device acquires electroencephalogram signals of the patient and transmits the electroencephalogram signals to the SSVEP online training unit of the computer through TCP communication;
(1-5) after the SSVEP online training unit carries out preprocessing, FBCCA algorithm and classification recognition on the electroencephalogram signals, a block watched by a patient can be analyzed;
(1-6) if the intention expression is selected, the intention of the patient can be directly analyzed. If the auxiliary life is selected, the computer sends a control instruction to the mechanical arm, and the mechanical arm helps the patient to grab the corresponding article.
2) And second stage rehabilitation training:
(2-1) the medical staff helps the patient to wear the electroencephalogram cap and the AR glasses and open the MI interactive interface;
(2-2) firstly, opening a preparation interface of the MI interactive interface, connecting the electroencephalogram cap, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand, and selecting an AR scene;
(2-3) re-entering a database interface of the MI interactive interface, and registering or inquiring personal information of the patient;
(2-4) entering a functional interface of the MI interactive interface, and setting relevant parameters of an electrical stimulation feedback instrument and a rehabilitation pneumatic hand;
(2-5) enabling the patient to perform motor imagery according to the prompt in the AR glasses, then performing training modeling by the MI offline training unit, and providing a model for the MI online training unit;
(2-6) performing on-line training on the patient, and analyzing the motor imagery intention of the patient by the MI on-line training unit and converting the motor imagery intention into control instructions of the AR glasses, the electrical stimulation feedback instrument and the rehabilitation pneumatic hand;
(2-7) the AR glasses give visual and auditory feedback to the patient, and the electro-stimulation feedback device and the rehabilitating pneumatic hand give tactile feedback to the patient.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110471569.6A CN113274032A (en) | 2021-04-29 | 2021-04-29 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110471569.6A CN113274032A (en) | 2021-04-29 | 2021-04-29 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113274032A true CN113274032A (en) | 2021-08-20 |
Family
ID=77277615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110471569.6A Pending CN113274032A (en) | 2021-04-29 | 2021-04-29 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113274032A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114146283A (en) * | 2021-08-26 | 2022-03-08 | 上海大学 | Attention training system and method based on target detection and SSVEP |
| CN114546105A (en) * | 2021-12-27 | 2022-05-27 | 宁波慈溪生物医学工程研究所 | Interactive motor imagery brain control method, brain control VR system and computer equipment |
| CN114864044A (en) * | 2022-05-13 | 2022-08-05 | 上海韶脑传感技术有限公司 | Hand function rehabilitation training system integrating eye movement direction classification and augmented reality technology |
| CN115381465A (en) * | 2022-07-28 | 2022-11-25 | 山东海天智能工程有限公司 | Rehabilitation training system based on BCI/VR and AR technologies |
| CN115399787A (en) * | 2022-07-29 | 2022-11-29 | 山东海天智能工程有限公司 | Brain-computer interface and AR/VR interactive technology combined rehabilitation training system |
| CN115624678A (en) * | 2022-10-27 | 2023-01-20 | 上海韶脑传感技术有限公司 | Rehabilitation training system and training method |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130127708A1 (en) * | 2010-05-28 | 2013-05-23 | The Regents Of The University Of California | Cell-phone based wireless and mobile brain-machine interface |
| WO2016002199A1 (en) * | 2014-06-30 | 2016-01-07 | 学校法人 慶應義塾 | Control device for rehabilitation device and rehabilitation device |
| WO2017112679A1 (en) * | 2015-12-22 | 2017-06-29 | University Of Florida Research Foundation, Inc. | Multimodal closed-loop brain-computer interface and peripheral stimulaiton for neuro-rehabilitation |
| CN107315478A (en) * | 2017-07-05 | 2017-11-03 | 中国人民解放军第三军医大学 | A kind of Mental imagery upper limbs intelligent rehabilitation robot system and its training method |
| CN108304068A (en) * | 2018-01-30 | 2018-07-20 | 山东建筑大学 | A kind of upper-limbs rehabilitation training robot control system and method based on brain-computer interface |
| CN109166612A (en) * | 2018-08-14 | 2019-01-08 | 龚映清 | A kind of big game scene rehabilitation system and method based on eye movement and brain electric information |
| CN109657560A (en) * | 2018-11-24 | 2019-04-19 | 天津大学 | Mechanical arm controls online brain-computer interface system and implementation method |
| US20190387995A1 (en) * | 2016-12-20 | 2019-12-26 | South China University Of Technology | Brain-Computer Interface Based Robotic Arm Self-Assisting System and Method |
| CN111110982A (en) * | 2019-12-02 | 2020-05-08 | 燕山大学 | Hand rehabilitation training method based on motor imagery |
| CN111544882A (en) * | 2020-04-30 | 2020-08-18 | 天津大学 | Novel cerebral stroke lower limb active rehabilitation system based on brain-computer interface and application |
| CN111571619A (en) * | 2020-04-17 | 2020-08-25 | 上海大学 | Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing |
| CN111631907A (en) * | 2020-05-31 | 2020-09-08 | 天津大学 | Cerebral apoplexy patient hand rehabilitation system based on brain-computer interaction hybrid intelligence |
| US20210018896A1 (en) * | 2019-07-16 | 2021-01-21 | Carnegie Mellon University | Methods and Systems for Noninvasive Mind-Controlled Devices |
-
2021
- 2021-04-29 CN CN202110471569.6A patent/CN113274032A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130127708A1 (en) * | 2010-05-28 | 2013-05-23 | The Regents Of The University Of California | Cell-phone based wireless and mobile brain-machine interface |
| WO2016002199A1 (en) * | 2014-06-30 | 2016-01-07 | 学校法人 慶應義塾 | Control device for rehabilitation device and rehabilitation device |
| WO2017112679A1 (en) * | 2015-12-22 | 2017-06-29 | University Of Florida Research Foundation, Inc. | Multimodal closed-loop brain-computer interface and peripheral stimulaiton for neuro-rehabilitation |
| US20190387995A1 (en) * | 2016-12-20 | 2019-12-26 | South China University Of Technology | Brain-Computer Interface Based Robotic Arm Self-Assisting System and Method |
| CN107315478A (en) * | 2017-07-05 | 2017-11-03 | 中国人民解放军第三军医大学 | A kind of Mental imagery upper limbs intelligent rehabilitation robot system and its training method |
| CN108304068A (en) * | 2018-01-30 | 2018-07-20 | 山东建筑大学 | A kind of upper-limbs rehabilitation training robot control system and method based on brain-computer interface |
| CN109166612A (en) * | 2018-08-14 | 2019-01-08 | 龚映清 | A kind of big game scene rehabilitation system and method based on eye movement and brain electric information |
| CN109657560A (en) * | 2018-11-24 | 2019-04-19 | 天津大学 | Mechanical arm controls online brain-computer interface system and implementation method |
| US20210018896A1 (en) * | 2019-07-16 | 2021-01-21 | Carnegie Mellon University | Methods and Systems for Noninvasive Mind-Controlled Devices |
| CN111110982A (en) * | 2019-12-02 | 2020-05-08 | 燕山大学 | Hand rehabilitation training method based on motor imagery |
| CN111571619A (en) * | 2020-04-17 | 2020-08-25 | 上海大学 | Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing |
| CN111544882A (en) * | 2020-04-30 | 2020-08-18 | 天津大学 | Novel cerebral stroke lower limb active rehabilitation system based on brain-computer interface and application |
| CN111631907A (en) * | 2020-05-31 | 2020-09-08 | 天津大学 | Cerebral apoplexy patient hand rehabilitation system based on brain-computer interaction hybrid intelligence |
Non-Patent Citations (1)
| Title |
|---|
| 王伟: "面向康复运动想象脑电分析及结合VR训练研究", 《中国优秀硕士学位论文全文数据库医药卫生科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114146283A (en) * | 2021-08-26 | 2022-03-08 | 上海大学 | Attention training system and method based on target detection and SSVEP |
| CN114546105A (en) * | 2021-12-27 | 2022-05-27 | 宁波慈溪生物医学工程研究所 | Interactive motor imagery brain control method, brain control VR system and computer equipment |
| CN114864044A (en) * | 2022-05-13 | 2022-08-05 | 上海韶脑传感技术有限公司 | Hand function rehabilitation training system integrating eye movement direction classification and augmented reality technology |
| CN115381465A (en) * | 2022-07-28 | 2022-11-25 | 山东海天智能工程有限公司 | Rehabilitation training system based on BCI/VR and AR technologies |
| CN115399787A (en) * | 2022-07-29 | 2022-11-29 | 山东海天智能工程有限公司 | Brain-computer interface and AR/VR interactive technology combined rehabilitation training system |
| CN115624678A (en) * | 2022-10-27 | 2023-01-20 | 上海韶脑传感技术有限公司 | Rehabilitation training system and training method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113274032A (en) | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface | |
| CN104173124B (en) | A kind of upper limb healing system based on bio signal | |
| CN107315478B (en) | A kind of Mental imagery upper limb intelligent rehabilitation robot system and its training method | |
| CN108304068B (en) | Upper limb rehabilitation training robot control system and method based on brain-computer interface | |
| CN109366508A (en) | A kind of advanced machine arm control system and its implementation based on BCI | |
| CN103793058B (en) | A kind of active brain-computer interactive system Mental imagery classification of task method and device | |
| CN106726030B (en) | Brain machine interface system and its application based on Clinical EEG Signals control robot movement | |
| CN103845137B (en) | Based on the robot control method of stable state vision inducting brain-machine interface | |
| CN109992113A (en) | A kind of MI-BCI system and its control method induced based on more scenes | |
| CN113398422B (en) | Rehabilitation training system and method based on motor imagery-brain-computer interface and virtual reality | |
| CN111110982A (en) | Hand rehabilitation training method based on motor imagery | |
| CN111938991A (en) | Hand rehabilitation training device and training method in double active control modes | |
| CN110238863A (en) | Based on brain electricity-electromyography signal lower limb rehabilitation robot control method and system | |
| CN110179643A (en) | A kind of neck rehabilitation training system and training method based on annulus sensor | |
| CN102306303B (en) | Electroencephalography signal characteristic extraction method based on small training samples | |
| CN107957783A (en) | A kind of Multimode Intelligent control system and method based on brain electricity with myoelectric information | |
| CN106109164A (en) | Rehabilitation system and the control method of rehabilitation system | |
| CN107483992A (en) | A kind of Intelligent TV remote control method based on SSVEP and Mental imagery | |
| CN113975633B (en) | Electrical stimulation rehabilitation training system based on multi-source information coupling feedback | |
| CN113805694A (en) | Auxiliary grabbing system and method based on brain-computer interface and computer vision | |
| CN111571619A (en) | Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing | |
| CN114021604A (en) | Motion imagery training system based on real-time feedback of 3D virtual reality technology | |
| Feng et al. | Feature extraction algorithm based on csp and wavelet packet for motor imagery eeg signals | |
| CN113359991B (en) | Intelligent brain-controlled mechanical arm auxiliary feeding system and method for disabled people | |
| CN114469641A (en) | Functional electrical stimulation dyskinesia mirror image training method based on myoelectric recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210820 |