CN115444717A - Limb function rehabilitation training method and system based on brain-computer interface - Google Patents

Limb function rehabilitation training method and system based on brain-computer interface Download PDF

Info

Publication number
CN115444717A
CN115444717A CN202211401866.4A CN202211401866A CN115444717A CN 115444717 A CN115444717 A CN 115444717A CN 202211401866 A CN202211401866 A CN 202211401866A CN 115444717 A CN115444717 A CN 115444717A
Authority
CN
China
Prior art keywords
limb
electroencephalogram
patient
real
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211401866.4A
Other languages
Chinese (zh)
Other versions
CN115444717B (en
Inventor
赵玉水
张海峰
赵绍晴
张海燕
郭新峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Haitian Intelligent Engineering Co ltd
Original Assignee
Shandong Haitian Intelligent Engineering Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Haitian Intelligent Engineering Co ltd filed Critical Shandong Haitian Intelligent Engineering Co ltd
Priority to CN202211401866.4A priority Critical patent/CN115444717B/en
Publication of CN115444717A publication Critical patent/CN115444717A/en
Application granted granted Critical
Publication of CN115444717B publication Critical patent/CN115444717B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H1/00Apparatus for passive exercising; Vibrating apparatus; Chiropractic devices, e.g. body impacting devices, external devices for briefly extending or aligning unbroken bones
    • A61H1/02Stretching or bending or torsioning apparatus for exercising
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H1/00Apparatus for passive exercising; Vibrating apparatus; Chiropractic devices, e.g. body impacting devices, external devices for briefly extending or aligning unbroken bones
    • A61H1/02Stretching or bending or torsioning apparatus for exercising
    • A61H1/0274Stretching or bending or torsioning apparatus for exercising for the upper limbs
    • A61H1/0285Hand
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H1/00Apparatus for passive exercising; Vibrating apparatus; Chiropractic devices, e.g. body impacting devices, external devices for briefly extending or aligning unbroken bones
    • A61H1/02Stretching or bending or torsioning apparatus for exercising
    • A61H1/0274Stretching or bending or torsioning apparatus for exercising for the upper limbs
    • A61H1/0285Hand
    • A61H1/0288Fingers
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2230/00Measuring physical parameters of the user
    • A61H2230/08Other bio-electrical signals
    • A61H2230/10Electroencephalographic signals
    • A61H2230/105Electroencephalographic signals used as a control parameter for the apparatus

Landscapes

  • Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • Pain & Pain Management (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Rehabilitation Therapy (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Rehabilitation Tools (AREA)

Abstract

The invention belongs to the field of brain-computer interfaces, and provides a limb function rehabilitation training method and system based on a brain-computer interface, which comprises the steps of performing limb rehabilitation training based on a virtual reality scene, and acquiring an electroencephalogram signal corresponding to limb motor imagery; performing rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device, and decoding the electroencephalogram signals; comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery, and judging the movement intention of the patient; determining a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and controlling the limb rehabilitation robot according to the motion instruction of the limb rehabilitation robot to realize limb rehabilitation training; compared with the traditional passive rehabilitation therapy with fixed treatment mode and fixed time, the invention obviously improves the active participation treatment degree of the patient, and realizes the rehabilitation training of the cerebral apoplexy patient by combining the limb rehabilitation robot to remold the nerve channel.

Description

Limb function rehabilitation training method and system based on brain-computer interface
Technical Field
The invention belongs to the technical field of brain-computer interface medical rehabilitation, and particularly relates to a limb function rehabilitation training method and system based on a brain-computer interface.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The aging situation is getting more and more severe. Cerebral apoplexy is a sudden cerebral blood circulation disorder disease and is one of the biggest threats to the health of the old. The life of the patient is extremely painful and inconvenient, the patient needs to be cared by a specially-assigned person, heavy burden is brought to families and the society, and many families are poverty-caused by diseases. How to help the patients to carry out effective rehabilitation therapy is a problem to be solved urgently in the society at present, and is also a research hotspot of multiple interdisciplines such as artificial intelligence and rehabilitation engineering.
Human upper limb function accounts for 60% of the whole body function, limb function accounts for 90% of the upper limb function, upper limb and hand dysfunction after stroke is one of the most common sequelae, if stroke patients completely lose one-hand function due to hemiplegia, the whole body function can be lost by 27%, and in 6 months after stroke, about 65% of patients still have dysfunction in the affected hand. However, the motor functions of the upper limbs and the hands are complex, the movement is fine, the projection area on the motor cortex of the brain is large, and the brain is unilaterally dominated, so that the rehabilitation treatment of the dysfunction of the upper limbs and the hands after the stroke is always a difficulty.
The existing limb rehabilitation training method after stroke has the following problems:
(1) The method excessively depends on the one-to-one treatment of the patient by a rehabilitation doctor, which not only wastes time and labor, but also has the rehabilitation effect greatly depending on the clinical experience of the rehabilitation doctor.
(2) Training a patient's limb in a predetermined pattern, lack of active participation and feedback by the patient, further lack of direct participation of the cerebral nervous system that innervates limb movement, inability to directly give stimulation to the central nervous system, or inability to optimally activate the cerebral cortex to promote healing.
(3) The rehabilitation training mechanical arm and the robot in the prior art have the problems of complex installation, large volume and no contribution to the wearing and moving of patients,
(4) The finger joint has low flexibility, stiff flexion and extension and unobvious rehabilitation training effect.
Disclosure of Invention
Compared with the traditional passive rehabilitation therapy with fixed treatment mode and fixed time, the invention obviously improves the active participation treatment degree of the patient, and realizes the rehabilitation training of the stroke patient by combining the limb rehabilitation robot to remold the nerve channel.
According to some embodiments, a first aspect of the present invention provides a limb function rehabilitation training method based on a brain-computer interface, which adopts the following technical solutions:
a limb function rehabilitation training method based on a brain-computer interface comprises the following steps:
performing limb rehabilitation training based on the virtual reality scene, and acquiring electroencephalogram signals corresponding to limb motor imagery;
performing rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device, and decoding the electroencephalogram signals;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to the limb motor imagery, and judging the movement intention of the patient, wherein the method comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to the limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
and determining a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and controlling the limb rehabilitation robot according to the motion instruction of the limb rehabilitation robot to realize limb rehabilitation training.
Further, the limb rehabilitation training is performed based on the virtual reality scene, and an electroencephalogram signal corresponding to the limb motor imagery is acquired, specifically:
manufacturing a training scene model, a 3D character model and a hand animation model thereof based on 3D modeling software to form a limb rehabilitation virtual scene training model;
performing limb rehabilitation training based on the limb rehabilitation virtual scene training model;
acquiring an electroencephalogram signal corresponding to the limb motor imagery.
Further, based on the recovered robot of limbs carries out the rehabilitation training of limbs specific motion, utilize brain electricity collection system to obtain patient's real-time brain electrical signal and carry out brain electrical signal decoding, include:
performing rehabilitation training of limb specific actions based on the limb rehabilitation robot;
acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device;
based on the real-time electroencephalogram signals of the patient, eliminating the electro-oculogram, myoelectricity and electrocardio-artifact of the real-time electroencephalogram signals by utilizing a stack type sparse automatic encoder;
and obtaining the decoded real-time electroencephalogram signal.
Further, the acquiring process of the electroencephalogram signals corresponding to the limb motor imagery comprises the following steps:
acquiring an electroencephalogram signal when a patient imagines the same rehabilitation training action for the first time based on a virtual reality scene;
performing first electroencephalogram decoding on the electroencephalogram signals obtained when the same rehabilitation training action is imagined for the first time;
repeatedly acquiring an electroencephalogram signal when the patient imagines the same rehabilitation training action, and performing electroencephalogram decoding;
and obtaining the corresponding relation between the decoded electroencephalogram signal and the motor imagery action according to the rehabilitation training action which is imagined by the patient and the corresponding decoded electroencephalogram signal.
Further, the electroencephalogram acquisition device comprises a first training module, a real-time updating system module and an online simulation training module, and specifically comprises:
the first training module enables a patient to execute the selected motor imagery task, and establishes a first rehabilitation model by performing offline decoding, identification and classification on the acquired electroencephalogram signals;
the real-time updating system module enables a patient to execute a motor imagery task and simultaneously acquire an online real-time electroencephalogram of the patient according to a first rehabilitation model, feeds the online real-time electroencephalogram back to the patient through the limb action of a virtual character, generates an electroencephalogram which is easier to identify, and establishes a real-time updating rehabilitation model by performing off-line decoding on a newly acquired electroencephalogram again; the updating process is repeated until a preset correct rate is obtained and the updating process is stable;
and the online simulation training module carries out real-time online simulation on a specific control object based on the real-time updating rehabilitation model, and prepares for specific practical application.
Further, the stack-type sparse automatic encoder eliminates the electrooculogram, myoelectricity and electrocardio artifact of real-time electroencephalogram signals, and comprises:
based on the real-time electroencephalogram signal of the patient;
and automatically selecting the electroencephalogram signals related to the motor imagery characteristics according to the stack type sparse automatic encoder to output.
According to some embodiments, the second aspect of the present invention provides a limb function rehabilitation training system based on a brain-computer interface, which adopts the following technical solutions:
limb function rehabilitation training system based on brain-computer interface comprises:
the virtual training module is configured to perform limb rehabilitation training based on a virtual reality scene, and acquire an electroencephalogram signal corresponding to limb motor imagery;
the rehabilitation training module is configured to perform rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquire real-time electroencephalogram signals of a patient by using the electroencephalogram acquisition device and decode the electroencephalogram signals;
the movement intention judging module is configured to compare the decoded real-time electroencephalogram signals with electroencephalogram signals corresponding to limb motor imagery and judge the movement intention of a patient, and comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
and the rehabilitation action execution module is configured to determine a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and control the limb rehabilitation robot to realize limb rehabilitation training according to the motion instruction of the limb rehabilitation robot.
According to some embodiments, a third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the brain-computer interface based limb function rehabilitation training method according to the first aspect.
According to some embodiments, a fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the brain-computer interface based limb function rehabilitation training method according to the first aspect when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
the invention integrates the cerebral neuron electric signals reflecting the intention of the patient into a limb rehabilitation manipulator, mainly uses a technology called as a non-invasive brain-computer interface, senses and learns the cerebral neuron electric signal mode of each patient, identifies the consciousness of the brain of a user to specific actions, reads and analyzes the movement intention, and then transmits the movement intention to the autonomous control treatment process of the limb rehabilitation device by a computer language. Compared with the traditional passive rehabilitation therapy with fixed treatment mode and fixed time, the system has the advantages that the active participation treatment degree of a patient is obviously improved, the muscle strength such as the stretching of the back of the wrist and the like is obviously increased by combining the hand function rehabilitation manipulator, and the motion capability of the wrist and the limbs is improved; when the rehabilitation manipulator with the brain-control hand function is applied, different parameters are set, so that the rehabilitation manipulator can be used for treating myasthenia/muscular atrophy and the like caused by nerve/muscle damage, can also improve the motion capability of wrist and finger joints of a stroke patient, and is combined with rehabilitation exercise training to achieve a better rehabilitation effect.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a flowchart of a limb function rehabilitation training method based on a brain-computer interface according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example one
As shown in fig. 1, the embodiment provides a limb function rehabilitation training method based on a brain-computer interface, and the embodiment is exemplified by applying the method to a server, and it is understood that the method may also be applied to a terminal, and may also be applied to a system including a terminal and a server, and is implemented by interaction between the terminal and the server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network server, cloud communication, middleware service, a domain name service, a security service CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein. In this embodiment, the method includes the steps of:
performing limb rehabilitation training based on the virtual reality scene, and acquiring an electroencephalogram signal corresponding to limb motor imagery;
performing rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device, and decoding the electroencephalogram signals;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery, and judging the movement intention of a patient, wherein the method comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to the limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the movement intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
and determining a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and controlling the limb rehabilitation robot according to the motion instruction of the limb rehabilitation robot to realize limb rehabilitation training.
The limb rehabilitation training is carried out based on the virtual reality scene, and electroencephalogram signals corresponding to limb motor imagery are obtained, and the method specifically comprises the following steps:
manufacturing a training scene model, a 3D character model and a hand animation model thereof based on 3D modeling software to form a limb rehabilitation virtual scene training model;
performing limb rehabilitation training based on the limb rehabilitation virtual scene training model;
acquiring an electroencephalogram signal corresponding to the limb motor imagery.
Based on the recovered robot of limbs carries out the rehabilitation training of limbs specific motion, utilize brain electricity collection system to acquire patient's real-time brain electrical signal and carry out brain electrical signal decoding, include:
performing rehabilitation training of limb specific actions based on the limb rehabilitation robot;
acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device;
based on the real-time electroencephalogram signals of the patient, eliminating the electro-oculogram, myoelectricity and electrocardio-artifact of the real-time electroencephalogram signals by utilizing a stack type sparse automatic encoder;
and obtaining the decoded real-time electroencephalogram signal.
The acquisition process of the electroencephalogram signals corresponding to the limb motor imagery comprises the following steps:
step (1): acquiring an electroencephalogram signal when a patient imagines the same rehabilitation training action for the first time based on a virtual reality scene;
step (2): performing first electroencephalogram decoding on the electroencephalogram signals obtained when the same rehabilitation training action is imagined for the first time;
repeating the steps (1) and (2), acquiring an electroencephalogram signal when the patient imagines the same rehabilitation training action, and performing electroencephalogram decoding;
and obtaining the corresponding relation between the decoded electroencephalogram signal and the motor imagery action according to the rehabilitation training action which is imagined by the patient and the corresponding decoded electroencephalogram signal.
It should be noted that the decoded electroencephalogram signal is an electroencephalogram signal when the patient imagines the same rehabilitation training action in the virtual reality scene;
for example, the patient imagines the hand-lifting action of the limb, and the electroencephalogram signal of the patient at the moment is collected and decoded to obtain the hand-lifting action of the limb and the corresponding decoded electroencephalogram signal during hand-lifting;
then, repeatedly imagining the action of lifting the hand by the limbs, and repeatedly acquiring the electroencephalogram signals of the patient for decoding when the action of lifting the hand by the limbs is imagined;
and obtaining the relation between the decoded electroencephalogram signals and the hand-lifting action of the motor imagery limb, namely the corresponding relation between the motor imagery action and the decoded electroencephalogram signals corresponding to the action.
The real-time electroencephalogram signal after decoding is compared with the electroencephalogram signal corresponding to the limb motor imagery, and the movement intention of the patient is judged, and the method comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continue to acquire the real-time brain electrical signals of the patient.
The electroencephalogram acquisition device comprises a first training module, a real-time updating system module and an online simulation training module, and specifically comprises the following steps:
the first training module enables a patient to execute a selected motor imagery task, and establishes a first rehabilitation model by performing offline decoding, identification and classification on the acquired electroencephalogram signals;
the real-time updating system module enables a patient to execute a motor imagery task and simultaneously acquire an online real-time electroencephalogram of the patient according to a first rehabilitation model, feeds the online real-time electroencephalogram back to the patient through the limb action of a virtual character, generates an electroencephalogram which is easier to identify, and establishes a real-time updating rehabilitation model by performing off-line decoding on a newly acquired electroencephalogram again; the updating process is repeated until a preset correct rate is obtained and the updating process is stable;
and the online simulation training module carries out real-time online simulation on a specific control object based on the real-time updating rehabilitation model, and prepares for specific practical application.
The stack type sparse automatic encoder eliminates the ocular, myoelectric and electrocardio artifacts of real-time electroencephalogram signals, and comprises the following steps:
based on the real-time electroencephalogram signal of the patient;
and automatically selecting the electroencephalogram signals related to the motor imagery characteristics according to the stack type sparse automatic encoder to output.
Feedback training system design based on VR: and (3) making a training scene model, a 3D character model and hand animation thereof in 3ds MAX, importing a Unity3D middle design control mode, analyzing EEG signals of a patient according to an effective model through hand motion imagination, and controlling the hand motion of the 3D character in real time. By introducing the patient in a virtual environment, the patient experiences realism in a realistic "thought-as-you-move" visual feedback and training system. The method has the characteristics of strong immersion, personalized training and the like, is expected to promote the remodeling of central nerves, and provides an innovative method for the rehabilitation of hand functions.
Procedure for importing Unity 3D:
1. first set the units in the 3ds Max software and click open the units setup setting in the customize option.
2. The ratio in the display unit and the system unit is set to centimeter.
3. And after the setting is finished, a three-dimensional model is created.
4. Exported in max format, and deposited under the Assets folder in the created Unity project.
5. And starting Unity, seeing the previously created three-dimensional training scene model, the 3D character model and the expected hand animation model in the project view, and dragging the three-dimensional training scene model, the 3D character model and the expected hand animation model into the game view.
6. Then, layout is performed, and the import is completed.
After the virtual training is completed, the actual rehabilitation training process comprises the following steps:
the first step is as follows: the patient imagines the specific movement action of the affected hand, collects a small amount of electrode EEG signals in real time, eliminates EOG artifacts by using SSAE (stacked sparse automatic encoder) without recording EOG, and further performs feature extraction by using EMD-CSP to identify the movement intention of the user.
The original EEG signal is easily distorted by interference, and the interference sources mainly include internal interference (e.g., electro-oculogram signal, electrocardiogram signal, and electromyogram signal) and external interference (e.g., noise, ac power introduced by the acquisition system, and electromagnetic interference in the surrounding environment).
After receiving the electroencephalogram signals input by the electroencephalogram collector, the SSAE automatically selects the electroencephalogram signals (alpha, mu and beta) related to the motor imagery characteristics to output according to the set characteristic parameters (the method of the traditional SSAE), thereby achieving the purposes of better representing input samples, reducing noise and eliminating artifacts.
The second step is that: a software and hardware module is constructed based on ARM, and intentions are converted into commands (the module mainly has the task of converting signals sent by an electroencephalogram acquisition device into languages which can be recognized by a computer).
The third step: and commands are transmitted to the multi-degree-of-freedom mechanical arm through a BCI (brain computer interface), so that the actions completely consistent with the VR virtual scene are completed, and the intended and acting immersion visual and auditory feedback and practical training are realized.
Specifically, based on the recovered robot of limbs carries out the rehabilitation training of limbs specific motion, utilize brain electricity collection system to obtain patient's real-time brain electrical signal and carry out brain electrical signal decoding, include:
performing rehabilitation training of limb specific actions based on the limb rehabilitation robot;
acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device;
based on the real-time electroencephalogram signals of a patient, eliminating the electro-oculogram, myoelectricity and electrocardio-artifact of the real-time electroencephalogram signals by using a stack-type sparse automatic encoder;
and obtaining the decoded real-time electroencephalogram signal.
The electroencephalogram acquisition device comprises a first training module, a real-time updating system module and an online simulation training module, and specifically comprises:
the first training module enables a patient to execute the selected motor imagery task, and establishes a first rehabilitation model by performing offline decoding, identification and classification on the acquired electroencephalogram signals;
the real-time updating system module enables a patient to execute a motor imagery task and simultaneously acquire an online real-time electroencephalogram of the patient according to a first rehabilitation model, feeds the online real-time electroencephalogram back to the patient through the limb action of a virtual character, generates an electroencephalogram which is easier to identify, and establishes a real-time updating rehabilitation model by performing off-line decoding on a newly acquired electroencephalogram again; the updating process is repeated until a preset correct rate is obtained and the updating process is stable;
and the online simulation training module carries out real-time online simulation on a specific control object based on the real-time updating rehabilitation model, and prepares for specific practical application.
Example two
The embodiment provides a limbs function rehabilitation training system based on brain-computer interface, includes:
the virtual training module is configured to perform limb rehabilitation training based on a virtual reality scene, and acquire an electroencephalogram signal corresponding to limb motor imagery;
the rehabilitation training module is configured to perform rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquire real-time electroencephalogram signals of a patient by using the electroencephalogram acquisition device and decode the electroencephalogram signals;
the movement intention judging module is configured to compare the decoded real-time electroencephalogram signals with electroencephalogram signals corresponding to limb motor imagery and judge the movement intention of a patient, and comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
the rehabilitation action execution module is configured to determine a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and control the limb rehabilitation robot to realize limb rehabilitation training according to the motion instruction of the limb rehabilitation robot.
The modules are the same as the corresponding steps in the implementation examples and application scenarios, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In the foregoing embodiments, the description of each embodiment has an emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions in other embodiments.
The proposed system can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules may be combined or integrated into another system, or some features may be omitted, or not executed.
It should be noted that, the rehabilitation training system described in this embodiment trains abnormal state processing: in the normal rehabilitation training process, the rehabilitation training system drives the limbs of the patient to do stretching training according to the set training mode. When abnormal conditions such as twitch, spasm and the like occur to the limbs of the patient, the limb function rehabilitation training system automatically switches to a floating state and sends an abnormal state notification to the doctor end, after the doctor deletes the examination and removes the abnormal state, the system automatically corrects the position of the abnormal state at the occurrence moment and returns to the position of the training interruption, and then the stretching training action is continuously completed according to the set training rule. The judgment of the limb state of the patient is realized through the change rate of the tail end position of the functional rehabilitation training system and the change condition of the interaction force of the limb.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the brain-computer interface-based limb function rehabilitation training method according to the first embodiment.
Example four
The embodiment provides a computer device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the limb function rehabilitation training method based on brain-computer interface as described in the first embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (10)

1. A limb function rehabilitation training method based on a brain-computer interface is characterized by comprising the following steps:
performing limb rehabilitation training based on the virtual reality scene, and acquiring an electroencephalogram signal corresponding to limb motor imagery;
performing rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device, and decoding the electroencephalogram signals;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery, and judging the movement intention of a patient, wherein the method comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to the limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
and determining a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and controlling the limb rehabilitation robot according to the motion instruction of the limb rehabilitation robot to realize limb rehabilitation training.
2. The brain-computer interface-based limb function rehabilitation training method of claim 1, wherein the limb rehabilitation training is performed based on the virtual reality scene to acquire electroencephalogram signals corresponding to limb motor imagery, and specifically comprises the following steps:
manufacturing a training scene model, a 3D character model and a hand animation model thereof based on 3D modeling software to form a limb rehabilitation virtual scene training model;
performing limb rehabilitation training based on the limb rehabilitation virtual scene training model;
acquiring an electroencephalogram signal corresponding to the limb motor imagery.
3. The limb function rehabilitation training method based on brain-computer interface as claimed in claim 1, wherein the rehabilitation training based on limb specific movement of the limb rehabilitation robot, acquiring real-time brain electrical signals of the patient by the brain electrical acquisition device and decoding the brain electrical signals comprises:
performing rehabilitation training of limb specific actions based on the limb rehabilitation robot;
acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device;
based on the real-time electroencephalogram signals of the patient, eliminating the electro-oculogram, myoelectricity and electrocardio-artifact of the real-time electroencephalogram signals by utilizing a stack type sparse automatic encoder;
and obtaining the decoded real-time electroencephalogram signal.
4. The brain-computer interface-based limb function rehabilitation training method as claimed in claim 1, wherein the process of acquiring electroencephalogram signals corresponding to the limb motor imagery comprises:
acquiring an electroencephalogram signal when a patient imagines the same rehabilitation training action for the first time based on a virtual reality scene;
performing first electroencephalogram decoding on the electroencephalogram signals during the first imagination of the same rehabilitation training action;
repeatedly acquiring an electroencephalogram signal when the patient imagines the same rehabilitation training action, and performing electroencephalogram decoding;
and obtaining the corresponding relation between the decoded electroencephalogram signal and the motor imagery action according to the rehabilitation training action which is imagined by the patient and the corresponding decoded electroencephalogram signal.
5. The brain-computer interface-based limb function rehabilitation training method as claimed in claim 1, wherein said electroencephalogram acquisition device comprises a first training module, a real-time updating system module and an on-line simulation training module, and specifically comprises:
the first training module enables a patient to execute the selected motor imagery task, and establishes a first rehabilitation model by performing offline decoding, identification and classification on the acquired electroencephalogram signals;
the real-time updating system module enables a patient to execute a motor imagery task and simultaneously acquire an online real-time electroencephalogram of the patient according to a first rehabilitation model, feeds the online real-time electroencephalogram back to the patient through the limb action of a virtual character, generates an electroencephalogram which is easier to identify, and establishes a real-time updating rehabilitation model by performing off-line decoding on a newly acquired electroencephalogram again; the updating process is repeated until a preset correct rate is obtained and the updating process is stable;
and the online simulation training module carries out real-time online simulation on a specific control object based on the real-time updating rehabilitation model, and prepares for specific practical application.
6. The brain-computer interface based limb function rehabilitation training method according to claim 3, wherein the stack type sparse automatic encoder eliminates electro-oculogram, myoelectricity and electro-cardio artifacts of real-time brain electrical signals, comprising:
based on the real-time electroencephalogram signal of the patient;
and automatically selecting the electroencephalogram signals related to the motor imagery characteristics according to the stack type sparse automatic encoder to output.
7. Limb function rehabilitation training system based on brain-computer interface, its characterized in that includes:
the virtual training module is configured to perform limb rehabilitation training based on a virtual reality scene, and acquire electroencephalogram signals corresponding to limb motor imagery;
the rehabilitation training module is configured to perform rehabilitation training of specific limb actions based on the limb rehabilitation robot, acquire real-time electroencephalogram signals of a patient by using the electroencephalogram acquisition device and decode the electroencephalogram signals;
the movement intention judging module is configured to compare the decoded real-time electroencephalogram signals with electroencephalogram signals corresponding to limb motor imagery and judge the movement intention of a patient, and comprises the following steps:
acquiring a decoded real-time electroencephalogram signal;
comparing the decoded real-time electroencephalogram signal with an electroencephalogram signal corresponding to limb motor imagery;
if the decoded real-time electroencephalogram signal is consistent with the electroencephalogram signal corresponding to the limb motor imagery, the motor intention of the patient is the corresponding motor imagery action;
if not, returning to continuously acquire the real-time electroencephalogram signals of the patient;
and the rehabilitation action execution module is configured to determine a motion instruction of the limb rehabilitation robot based on the motion intention of the patient, and control the limb rehabilitation robot to realize limb rehabilitation training according to the motion instruction of the limb rehabilitation robot.
8. The brain-computer interface based limb function rehabilitation training system of claim 7, wherein the limb rehabilitation robot performs rehabilitation training of limb specific actions, acquires real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device and decodes the electroencephalogram signals, comprising:
performing rehabilitation training of limb specific actions based on the limb rehabilitation robot;
acquiring real-time electroencephalogram signals of a patient by using an electroencephalogram acquisition device;
based on the real-time electroencephalogram signals of the patient, eliminating the electro-oculogram, myoelectricity and electrocardio-artifact of the real-time electroencephalogram signals by utilizing a stack type sparse automatic encoder;
and obtaining the decoded real-time electroencephalogram signal.
9. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the brain-computer interface based limb function rehabilitation training method according to any one of claims 1-7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps in the brain-computer interface based limb function rehabilitation training method according to any one of claims 1-7.
CN202211401866.4A 2022-11-10 2022-11-10 Limb function rehabilitation training method and system based on brain-computer interface Active CN115444717B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211401866.4A CN115444717B (en) 2022-11-10 2022-11-10 Limb function rehabilitation training method and system based on brain-computer interface

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211401866.4A CN115444717B (en) 2022-11-10 2022-11-10 Limb function rehabilitation training method and system based on brain-computer interface

Publications (2)

Publication Number Publication Date
CN115444717A true CN115444717A (en) 2022-12-09
CN115444717B CN115444717B (en) 2023-03-10

Family

ID=84295780

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211401866.4A Active CN115444717B (en) 2022-11-10 2022-11-10 Limb function rehabilitation training method and system based on brain-computer interface

Country Status (1)

Country Link
CN (1) CN115444717B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116849942A (en) * 2023-07-28 2023-10-10 中国医学科学院生物医学工程研究所 Brain-control intelligent lifting and turning-over multifunctional medical care bed

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013063986A1 (en) * 2011-11-03 2013-05-10 深圳Tcl新技术有限公司 Remote control method, device and system based on brain waves
CN106020472A (en) * 2016-05-13 2016-10-12 天津理工大学 Brain computer interface system on basis of motor imageries of different uplifting amplitudes of lower limbs
CN106445155A (en) * 2016-09-29 2017-02-22 珠海市魅族科技有限公司 Controlling method and virtual reality equipment based on electroencephalogram eeg
WO2017084416A1 (en) * 2015-11-17 2017-05-26 天津大学 Feedback system based on motor imagery brain-computer interface
CN107015632A (en) * 2016-01-28 2017-08-04 南开大学 Control method for vehicle, system based on brain electricity driving
CN107157705A (en) * 2017-05-09 2017-09-15 京东方科技集团股份有限公司 rehabilitation system and method
CN111743538A (en) * 2020-07-06 2020-10-09 江苏集萃脑机融合智能技术研究所有限公司 Brain-computer interface alarm method and system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013063986A1 (en) * 2011-11-03 2013-05-10 深圳Tcl新技术有限公司 Remote control method, device and system based on brain waves
WO2017084416A1 (en) * 2015-11-17 2017-05-26 天津大学 Feedback system based on motor imagery brain-computer interface
CN107015632A (en) * 2016-01-28 2017-08-04 南开大学 Control method for vehicle, system based on brain electricity driving
CN106020472A (en) * 2016-05-13 2016-10-12 天津理工大学 Brain computer interface system on basis of motor imageries of different uplifting amplitudes of lower limbs
CN106445155A (en) * 2016-09-29 2017-02-22 珠海市魅族科技有限公司 Controlling method and virtual reality equipment based on electroencephalogram eeg
CN107157705A (en) * 2017-05-09 2017-09-15 京东方科技集团股份有限公司 rehabilitation system and method
CN111743538A (en) * 2020-07-06 2020-10-09 江苏集萃脑机融合智能技术研究所有限公司 Brain-computer interface alarm method and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
孟飞,黄军友,高小榕: "基于脑-机接口技术的上肢康复训练系统" *
徐宝国;彭思;宋爱国;: "基于运动想象脑电的上肢康复机器人" *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116849942A (en) * 2023-07-28 2023-10-10 中国医学科学院生物医学工程研究所 Brain-control intelligent lifting and turning-over multifunctional medical care bed

Also Published As

Publication number Publication date
CN115444717B (en) 2023-03-10

Similar Documents

Publication Publication Date Title
Neuper et al. Motor imagery and EEG-based control of spelling devices and neuroprostheses
Ferreira et al. Human-machine interfaces based on EMG and EEG applied to robotic systems
CN104360730B (en) Man-machine interaction method supported by multi-modal non-implanted brain-computer interface technology
BRPI1100261B1 (en) PROCESS AND DEVICE FOR BRAIN-COMPUTER INTERFACE
Pérez-Velasco et al. EEGSym: Overcoming inter-subject variability in motor imagery based BCIs with deep learning
JP2009531077A (en) Apparatus and method for real time control of effectors
CN115444717B (en) Limb function rehabilitation training method and system based on brain-computer interface
CN111297379A (en) Brain-computer combination system and method based on sensory transmission
Kawala-Janik et al. Method for EEG signals pattern recognition in embedded systems
Jiang et al. Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: A 10-year perspective review
Kawala-Janik Efficiency evaluation of external environments control using bio-signals
Wang et al. Prosthetic control system based on motor imagery
CN113053492B (en) Self-adaptive virtual reality intervention system and method based on user background and emotion
CN111857352B (en) Gesture recognition method based on imagination type brain-computer interface
Li et al. Preliminary study of online real-time control system for lower extremity exoskeletons based on EEG and sEMG fusion
Rajyalakshmi et al. Exploration of recent advances in the field of brain computer interfaces
CN114936574A (en) High-flexibility manipulator system based on BCI and implementation method thereof
Materka et al. High-speed noninvasive brain-computer interfaces
Noel et al. Utilizing Deep Neural Networks for Brain–Computer Interface-Based Prosthesis Control
Kæseler et al. Brain patterns generated while using a tongue control interface: a preliminary study with two individuals with ALS
Wang et al. Multi-user motion recognition using sEMG via discriminative canonical correlation analysis and adaptive dimensionality reduction
Piozin et al. Motion prediction for the sensorimotor control of hand prostheses with a brain-machine interface using EEG
Rao et al. A reliable eye blink based home automation system using false free detection algorithm
Ye et al. The design of multi-task simulation manipulator based on motor imagery EEG
Bolaños et al. Non-invasive control of a intelligent room using EEG signals

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Method and System for Limb Function Rehabilitation Training Based on Brain Computer Interface

Effective date of registration: 20231225

Granted publication date: 20230310

Pledgee: Agricultural Bank of China Limited Tai'an Mount Taishan Sub branch

Pledgor: SHANDONG HAITIAN INTELLIGENT ENGINEERING Co.,Ltd.

Registration number: Y2023980074026