CN112951409A - Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality - Google Patents
Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality Download PDFInfo
- Publication number
- CN112951409A CN112951409A CN202110161666.5A CN202110161666A CN112951409A CN 112951409 A CN112951409 A CN 112951409A CN 202110161666 A CN202110161666 A CN 202110161666A CN 112951409 A CN112951409 A CN 112951409A
- Authority
- CN
- China
- Prior art keywords
- module
- patient
- rehabilitation training
- virtual
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 34
- 206010019468 Hemiplegia Diseases 0.000 title claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 129
- 230000033001 locomotion Effects 0.000 claims abstract description 37
- 238000012544 monitoring process Methods 0.000 claims abstract description 36
- 238000007405 data analysis Methods 0.000 claims abstract description 23
- 238000004458 analytical method Methods 0.000 claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 10
- 238000011156 evaluation Methods 0.000 claims abstract description 9
- 230000009471 action Effects 0.000 claims description 18
- 230000008054 signal transmission Effects 0.000 claims description 13
- 230000003238 somatosensory effect Effects 0.000 claims description 13
- 239000011521 glass Substances 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 7
- 238000011161 development Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000006872 improvement Effects 0.000 claims description 3
- 210000000988 bone and bone Anatomy 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 4
- 208000006011 Stroke Diseases 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000004899 motility Effects 0.000 description 3
- 230000004064 dysfunction Effects 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 241000272525 Anas platyrhynchos Species 0.000 description 1
- 206010008190 Cerebrovascular accident Diseases 0.000 description 1
- 241000287486 Spheniscidae Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 230000002490 cerebral effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1118—Determining activity level
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/021—Measuring pressure in heart or blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
Abstract
The invention discloses a hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality, which comprises a Kinect interaction module, a motion capture module, a virtual rehabilitation training scene module, a body condition monitoring module, a virtual reality module, a training data analysis module and an emergency alarm module. The motion capture module captures the offset motion of the patient, and the Kinect interaction module converts the offset motion of the patient into a motion in a virtual rehabilitation training scene; the training data analysis module records data of the patient in a virtual rehabilitation training scene and performs analysis and evaluation; the physical condition monitoring module monitors the physical condition of the patient in the rehabilitation process; and the emergency alarm module triggers an alarm according to the body data result analyzed by the body condition monitoring module and then alarms in real time. The invention enables the hemiplegic patient to carry out balance rehabilitation training autonomously without accompanying of therapists, saves medical resources and improves the enthusiasm of the hemiplegic patient in the rehabilitation training mode through games.
Description
Technical Field
The invention relates to the technical field of VR rehabilitation, in particular to a hemiplegia patient rehabilitation system, a hemiplegia patient rehabilitation method and a storage medium based on Kinect interaction and virtual reality.
Background
Good balance is of great importance to people because the completion of various basic activities in daily life (such as standing, walking, eating, washing, dressing) depends on the balance of the body. The balance dysfunction is one of the most common dysfunctions of patients with cerebral apoplexy and hemiplegia, directly influences the walking function of the patients and increases the falling risk of the patients. According to research, the falling incidence rate of stroke patients is 25% -75%. This not only significantly affects the patient's ability to self-care of life, reducing their quality of life, but also places a heavy burden on the patient and their families and society. Therefore, it is very important to help the stroke hemiplegia patients to improve the balance function.
At present, there are two main types of rehabilitation exercises for helping a hemiplegic patient to improve a balance function, one is rehabilitation exercise of a hand grip performed by a rehabilitation therapist on the patient according to a principle of 3-level balance progressive exercise. The other type is to rely on a rehabilitation robot or an instrument to perform auxiliary training on the rehabilitation robot or the instrument. However, both of these approaches leave the patient passively treated, ignoring the patient's subjective motility. The rehabilitation training is tedious, and the initiative of the rehabilitation training of the patient can be influenced due to the lack of active cooperation of the patient, so that the treatment effect is influenced. In addition, because the rehabilitation treatment period is long, if a mode that a therapist carries out rehabilitation training on a pair of patients is adopted, a large amount of medical resources are wasted.
Therefore, the prior art has yet to be developed.
Disclosure of Invention
In view of the defects of the prior art, the invention aims to provide a hemiplegia patient rehabilitation system, a hemiplegia patient rehabilitation method and a hemiplegia patient rehabilitation storage medium based on Kinect interaction and virtual reality, which aim to enable a hemiplegia patient to carry out balance rehabilitation training autonomously without accompanying of a therapist, save medical resources and improve the enthusiasm of the hemiplegia patient rehabilitation training in a game training mode.
In order to realize the purpose, the invention adopts the following technical scheme:
a hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality, wherein,
the system comprises a Kinect interaction module, a motion capture module, a virtual rehabilitation training scene module and a physical condition monitoring module which are connected with the Kinect interaction module, a virtual reality module and a training data analysis module which are connected with the virtual rehabilitation training scene module, and an emergency alarm module which is connected with the physical condition monitoring module;
the motion capture module captures the offset motion of the patient, and the Kinect interaction module synchronizes the offset motion of the patient to the 3D object constructed in the virtual rehabilitation training scene module to realize the control of the object;
the Kinect interaction module converts the offset action of the patient into an action in a virtual rehabilitation training scene by calling a Kinect for Unity plug-in tool in Unity3D, so that the patient can control the role object in the virtual rehabilitation training scene, and the balance rehabilitation training of the patient is realized;
the virtual rehabilitation training scene module is provided with a somatosensory game developed by a Unity3D game development engine and specially used for the balance rehabilitation training of the hemiplegic patients;
the training data analysis module records data of the patient in a virtual rehabilitation training scene, evaluates the effect of the patient on the rehabilitation training according to the data, and feeds back the data to a doctor for analysis and improvement of a training plan;
the virtual reality module is composed of VR glasses adapted to a PC end of a computer, and presents a virtual rehabilitation training scene to the patient, so that the hemiplegic patient is immersed in the virtual rehabilitation training scene;
the physical condition monitoring module monitors the physical condition of a patient in the rehabilitation process and stores and analyzes physical data;
and the emergency alarm module triggers an alarm according to the body data result analyzed by the body condition monitoring module and then alarms in real time.
The specific implementation of the function of the Kinect interaction module comprises two steps:
the method comprises the steps that firstly, through an API (application programming interface) provided by a Kinect software development kit, a patient deviation action captured by an action capture module is obtained and converted into a machine language;
and secondly, the Unity3D engine transmits the machine language of the converted patient deviation action to a corresponding game interface in the virtual rehabilitation training scene module to complete the control of the character object in the virtual scene.
The motion sensing game in the virtual rehabilitation training scene module is a 3D racing game which is constructed through a script program and is specially used for the balance rehabilitation training of the hemiplegic patients.
The somatosensory game of the virtual rehabilitation training scene module comprises two training modes of unlocking a level and practicing points.
Wherein, VR glasses are HTC VIVE VR glasses of adaptation computer PC end.
The motion capture module utilizes a sensor carried by the body sensing equipment in the Kinect interaction module to acquire RGB and depth image data in real time, so that the skeleton tracking of a human body is realized, and the posture of the human body is identified.
Wherein the motion capture module captures data of 25 joint points throughout the patient.
The body condition monitoring module comprises a monitoring module, a body data transmission module, a body data analysis module, a body data storage module and a database which are sequentially connected;
the monitoring module is provided with a plurality of sensors for monitoring various data changes of the body of the patient in real time, the data changes are transmitted to the body data analysis module through the body data transmission module, the body data analysis module analyzes the data, the body data and an analysis result are stored through the body data storage module, and meanwhile, the data are recorded into the database to participate in the follow-up rehabilitation effect analysis and evaluation.
The emergency alarm module comprises a main control module, an alarm signal transmission module connected with the main control module, an audible and visual alarm module connected with the alarm signal transmission module and an emergency power-off module;
the main control module receives the analysis result of the body condition monitoring module and triggers an alarm signal, and then transmits the analysis result to the sound and light alarm module through the alarm signal transmission module to give an alarm, and if the situation is serious, transmits the analysis result to the emergency power-off module through the alarm signal transmission module to perform emergency power-off processing.
The virtual rehabilitation training scene module sets a left-right movement collection target of a character object in a virtual scene controlled by a patient through left-right body deviation, and controls the advancing speed of the character object in the virtual scene through front-back body inclination.
Compared with the prior art, the hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality has the following beneficial effects:
1. the invention provides a new method for training the balance function of a hemiplegic patient by utilizing a somatosensory interaction technology and a VR technology, can accurately capture the deviation action of the patient by using a Kinect action capture device, can carry out more specialized rehabilitation guidance on the hemiplegic patient, integrates a somatosensory game with a virtual reality technology by a system, provides an interesting rehabilitation training mode, can fully mobilize the subjective motility of the patient training and provide the efficiency of the rehabilitation training of the patient, can also provide specialized and personalized evaluation for the patient based on the artificial intelligence rehabilitation training data analysis, is convenient for the patient to better recover, and increases the interest of the balance training, so that the patient can actively participate in the rehabilitation training, the system can give real-time feedback in the training process, can more accurately guide the patient to train, and the patient can be better immersed in the training by the application of the virtual reality technology, and then promote rehabilitation training's efficiency and effect, shorten patient's rehabilitation training time, the training level of different degrees of difficulty has still been designed to the system, and the patient of being convenient for carries out individualized rehabilitation training according to the condition of self, can let the hemiplegia patient independently carry out balanced rehabilitation training, and need not therapist's companion, can practice thrift medical resource like this.
2. The invention monitors the physical condition of the patient in the rehabilitation process in real time through the physical condition monitoring module and the emergency alarm module, stores and analyzes the change data of the physical condition, alarms in real time if the change data exceeds a normal body index or is lower than the normal index, and carries out emergency power-off treatment through the emergency power-off module if the change data is serious so as to protect the patient.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
FIG. 1 is a schematic diagram of a hemiplegia patient recovery system based on Kinect interaction and virtual reality according to a first embodiment of the present invention;
FIG. 2 is a diagram of a motion capture module capturing 25 joints of a patient's entire body according to the present invention;
FIG. 3 is a schematic diagram of a virtual rehabilitation training somatosensory game framework of the invention;
FIG. 4 is a schematic diagram of a balance rehabilitation training mode of the present invention;
FIG. 5 is a schematic diagram of the body condition monitoring module of the present invention;
FIG. 6 is a schematic diagram of the emergency alert module of the present invention;
fig. 7 is a schematic diagram of the balance rehabilitation training process of the present invention.
Description of reference numerals:
the system comprises a 100-Kinect interaction module, a 200-motion capture module, a 300-virtual rehabilitation training scene module, a 400-physical condition monitoring module, a 401-monitoring module, a 402-physical data transmission module, a 403-physical data analysis module, a 404-physical data storage module, a 405-database, a 500-virtual reality module, a 600-training data analysis module, a 700-emergency alarm module, a 701-main control module, a 702-alarm signal transmission module, a 703-sound-light alarm module and a 704-emergency power-off module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the invention provides a hemiplegic patient rehabilitation system based on Kinect interaction and virtual reality, which includes a Kinect interaction module 100, a motion capture module 200 connected with the Kinect interaction module 100, a virtual rehabilitation training scene module 300 and a physical condition monitoring module 400, and also includes a virtual reality module 500 connected with the virtual rehabilitation training scene module 300, a training data analysis module 600, and an emergency alarm module 700 connected with the physical condition monitoring module.
The motion capture module 200 captures the patient's offset motion, and the Kinect interaction module 100 synchronizes the patient's offset motion to the 3D object constructed in the virtual rehabilitation training scenario module 300 to realize the control of the object.
Preferably, the motion capture module 200 of the present invention utilizes the sensors of the body sensing device in the Kinect interaction module 100 to acquire RGB and depth image data in real time, so as to realize bone tracking of the human body, thereby recognizing the posture of the human body. Such as left-right movement, front-back movement, etc.
Further, as shown in FIG. 2, the motion capture module 200 of the present invention captures data for 25 joint points throughout the patient. This enables the patient's movements to be accurately identified.
The Kinect interaction module 100 converts the patient deviation action into the action in the virtual rehabilitation training scene by calling a Kinect for Unity plug-in tool in Unity3D, so as to realize the control of the patient on the role object in the virtual rehabilitation training scene, thereby realizing the balanced rehabilitation training of the patient.
In the embodiment of the invention, the Kinect for unity plug-in tool is provided by Microsoft officials.
Specifically, the specific implementation of the function of the Kinect interaction module 100 of the present invention includes two steps:
in the first step, the patient's offset motion captured by the motion capture module 100 is obtained through the API interface provided by the Kinect software development kit, and converted into machine language.
Secondly, the Unity3D engine transmits the machine language of the transformed patient deviation action to the corresponding game interface in the virtual rehabilitation training scene module 300, so as to complete the control of the character object in the virtual scene.
The virtual rehabilitation training scene module 300 is provided with a somatosensory game developed by a Unity3D game development engine and specially used for the balance rehabilitation training of the hemiplegic patients. Fig. 3 is a frame diagram of the virtual balance rehabilitation training motion sensing game according to the present invention.
Preferably, the motion sensing game in the virtual rehabilitation training scene module 300 of the present invention is a 3D racing game constructed by a script program and dedicated to the balance rehabilitation training of the hemiplegic patient. The script program can simplify the development process in the somatosensory game development of rehabilitation training, shorten the development period and facilitate the code updating and maintenance.
Preferably, the script program of the present invention is a C # script.
As an embodiment, the 3D racing game in the virtual rehabilitation training scene module 300 of the present invention is to evaluate the effect of the rehabilitation training of the patient by controlling the character objects (e.g. people or penguins) in the game to slide down from a steep, snow-covered hill and collecting scattered targets (e.g. small fish) around the slide way, and when the patient reaches the end point, the system gives the length of time taken by the patient to complete the track and the number of the collected targets.
As shown in fig. 4, preferably, the motion sensing game of the virtual rehabilitation training scenario module of the present invention includes two training modes, i.e., an unlocking level and a practice point.
In the embodiment of the invention, the level with different levels of difficulty is set in the unlocking level mode, and when the next level is entered, the patient needs to meet the set requirement of the level, namely the terminal is reached in the limited time, and a certain number of targets are collected.
The exercise point mode is mainly characterized in that diversified virtual scenes are set to encourage patients to actively participate in training, the exercise point mode has a plurality of open levels, the patients can select proper levels according to self rehabilitation conditions and continuously enhance self balance sense through repeated exercise, the exercise system can record the best training results of the patients, the patients can continuously break through themselves and refresh the results, and the exercise system also allows the patients to automatically set individual elements of a competition scene, such as day time, night, fine days or foggy days, so that the participation sense of the patients on the rehabilitation training is enhanced.
Preferably, the virtual rehabilitation training scene module 300 of the present invention sets the left-right movement collection target of the character object in the virtual scene controlled by the left-right body shift of the patient, and controls the advancing speed of the character object in the virtual scene by the front-back body tilt. This allows the patient's movements to be faithfully mapped to the game, enhancing the realism of the game.
The training data analysis module 600 records the data of the patient in the virtual rehabilitation training scenario, and evaluates the effect of the patient on the rehabilitation training with the data, and feeds the data back to the doctor for analysis and improvement of the training plan.
After the training data analysis module 600 records the data of the patient in the virtual rehabilitation training scene, the rehabilitation training data is analyzed based on artificial intelligence, so that the patient can be provided with specific and personalized evaluation, meanwhile, in the training process, the training data analysis module 600 can feed the training data and the evaluation result back to the doctor for analysis in time, the doctor can know the rehabilitation condition of the patient conveniently, meanwhile, the doctor can also make a special training plan aiming at the patient balance rehabilitation training condition, so that the patient can be guided to train more accurately, and the patient can be recovered better.
The virtual reality module 500 is composed of VR glasses adapted to the PC end of the computer, and presents a virtual rehabilitation training scene to the patient, so that the hemiplegic patient is immersed in the virtual rehabilitation training scene. The virtual reality module 500 guides the hemiplegic patient to immerse in the virtual balance rehabilitation training scene, so that the patient can complete the rehabilitation training task without the side duck;
preferably, the VR glasses of the present invention are HTC VIVE VR glasses adapted to the PC side of the computer. HTC VIVE VR glasses are relative with ordinary VR glasses, and its field of vision and picture are clear, immerse and feel strong, and patient's experience is effectual.
The physical condition monitoring module 400 monitors the physical condition of the patient during the rehabilitation process, and stores and analyzes the physical data. The physical data includes heart rate, blood pressure, respiration and other physiological data.
Preferably, as shown in fig. 5, the physical status monitoring module 400 of the present invention comprises a monitoring module 401, a physical data transmission module 402, a physical data analysis module 403, a physical data storage module 404 and a database 405 connected in sequence.
The monitoring module 401 is provided with a plurality of sensors for monitoring various data changes of the body of the patient in real time, the data changes are transmitted to the body data analysis module 403 through the body data transmission module 402, the body data analysis module 403 analyzes the data, the body data and an analysis result are stored through the body data storage module 404, and meanwhile, the data are recorded into the database 404 to participate in subsequent rehabilitation effect analysis and evaluation.
The emergency alarm module 700 performs real-time alarm after triggering alarm according to the body data result analyzed by the body condition monitoring module. Preferably, as shown in fig. 6, the emergency alert module 700 of the present invention includes a main control module 701, an alert signal transmission module 702 connected to the main control module 701, an audible and visual alarm module 703 and an emergency power-off module 704 connected to the alert signal transmission module 702.
The main control module 701 receives the analysis result of the physical condition monitoring module 400 and triggers an alarm signal, and then transmits the alarm signal to the sound and light alarm module 703 through the alarm signal transmission module 702 for alarming, and if the condition is serious, transmits the alarm signal to the emergency power-off module 704 through the alarm signal transmission module 702 for emergency power-off processing.
The triggering of the alarm signal may be triggered by the physical condition monitoring module 400 or the main control module 701 of the emergency alarm module 700, and an alarm is given in real time if the result of the physical data exceeds the normal body index or is lower than the normal index. If the alarm is within the preset alarm range, the alarm is given through the audible and visual alarm module 703, and if the alarm is over the preset alarm range, the situation is serious, the emergency power-off processing is performed through the emergency power-off module 704 to ensure the safety of the patient under the unattended condition.
The training process of the balance rehabilitation training somatosensory game of the embodiment of the invention is shown in fig. 7 and comprises the following steps:
1. after a patient enters a virtual rehabilitation training scene, displaying a training mode selection UI (user interface);
2. after the patient selects the training mode, displaying a rehabilitation training main scene;
3. prompting the rehabilitation training task of the patient on a game starting interface;
4. the patient performs action operation according to the prompt to obtain limb input of the patient;
5. judging whether the control role in the game touches a target or not;
6. if the target is touched, adding one to the number of the collected targets, and if the target is not touched, keeping the number of the collected targets unchanged;
7. the patient continues to complete the training task and records the number of targets collected;
8. the patient reaches the end point, and the training performance (the used time and the total collection target) of the patient is displayed;
9. storing the training result of the patient;
10. and outputting a training data file, and entering subsequent rehabilitation result evaluation.
The hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality, which is provided by the invention, provides a new method for training the balance function of a hemiplegia patient by utilizing a somatosensory interaction technology and a VR technology, can accurately capture the offset action of the patient by using a Kinect action capturing device, can carry out more specialized rehabilitation guidance on the hemiplegia patient, integrates a somatosensory game with the virtual reality technology, provides an interesting rehabilitation training mode, can fully mobilize the subjective motility of the patient for training, provides the rehabilitation training efficiency of the patient, can also provide specialized and personalized evaluation for the patient based on the artificial intelligence rehabilitation training data analysis, is convenient for the better rehabilitation of the patient, monitors the change index of each index of the real-time body of the patient in real time by combining a body condition monitoring module and an emergency alarm module, and carries out real-time alarm if the index exceeds the normal body index or is lower than the normal index, if the situation is serious, the emergency power-off module is used for carrying out emergency power-off processing, so that the patient can use the emergency power-off module more safely, and the practicability is higher.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality is characterized in that,
the system comprises a Kinect interaction module, a motion capture module, a virtual rehabilitation training scene module and a physical condition monitoring module which are connected with the Kinect interaction module, a virtual reality module and a training data analysis module which are connected with the virtual rehabilitation training scene module, and an emergency alarm module which is connected with the physical condition monitoring module;
the motion capture module captures the offset motion of the patient, and the Kinect interaction module synchronizes the offset motion of the patient to the 3D object constructed in the virtual rehabilitation training scene module to realize the control of the object;
the Kinect interaction module converts the offset action of the patient into an action in a virtual rehabilitation training scene by calling a Kinect for Unity plug-in tool in Unity3D, so that the patient can control the role object in the virtual rehabilitation training scene, and the balance rehabilitation training of the patient is realized;
the virtual rehabilitation training scene module is provided with a somatosensory game developed by a Unity3D game development engine and specially used for the balance rehabilitation training of the hemiplegic patients;
the training data analysis module records data of the patient in a virtual rehabilitation training scene, evaluates the effect of the patient on the rehabilitation training according to the data, and feeds back the data to a doctor for analysis and improvement of a training plan;
the virtual reality module is composed of VR glasses adapted to a PC end of a computer, and presents a virtual rehabilitation training scene to the patient, so that the hemiplegic patient is immersed in the virtual rehabilitation training scene;
the physical condition monitoring module monitors the physical condition of a patient in the rehabilitation process and stores and analyzes physical data;
and the emergency alarm module triggers an alarm according to the body data result analyzed by the body condition monitoring module and then alarms in real time.
2. The system as claimed in claim 1, wherein the specific implementation of the function of the Kinect interaction module includes two steps:
the method comprises the steps that firstly, through an API (application programming interface) provided by a Kinect software development kit, a patient deviation action captured by an action capture module is obtained and converted into a machine language;
and secondly, the Unity3D engine transmits the machine language of the converted patient deviation action to a corresponding game interface in the virtual rehabilitation training scene module to complete the control of the character object in the virtual scene.
3. The system of claim 1, wherein the somatosensory game in the virtual rehabilitation training scene module is a 3D racing game which is constructed by a script program and is specially used for the balance rehabilitation training of the hemiplegic patients.
4. The system of claim 1, wherein the somatosensory game of the virtual rehabilitation training scenario module comprises two training modes of unlocking a level and exercising a score.
5. The system of claim 1, wherein the VR glasses are HTC VIVE VR glasses adapted to be used on a PC side of a computer.
6. The system of claim 1, wherein the motion capture module utilizes a sensor of the body sensing device in the Kinect interaction module to acquire RGB and depth image data in real time, so as to realize bone tracking of the human body, thereby recognizing the posture of the human body.
7. The system of claim 1, wherein the motion capture module captures data for 25 joint points throughout the patient.
8. The system of claim 1, wherein the physical condition monitoring module comprises a monitoring module, a physical data transmission module, a physical data analysis module, a physical data storage module and a database which are connected in sequence;
the monitoring module is provided with a plurality of sensors for monitoring various data changes of the body of the patient in real time, the data changes are transmitted to the body data analysis module through the body data transmission module, the body data analysis module analyzes the data, the body data and an analysis result are stored through the body data storage module, and meanwhile, the data are recorded into the database to participate in the follow-up rehabilitation effect analysis and evaluation.
9. The system of claim 1, wherein the emergency alarm module comprises a main control module, an alarm signal transmission module connected with the main control module, an audible and visual alarm module and an emergency power-off module connected with the alarm signal transmission module;
the main control module receives the analysis result of the body condition monitoring module and triggers an alarm signal, and then transmits the analysis result to the sound and light alarm module through the alarm signal transmission module to give an alarm, and if the situation is serious, transmits the analysis result to the emergency power-off module through the alarm signal transmission module to perform emergency power-off processing.
10. The system of claim 1, wherein the virtual rehabilitation training scene module sets the collection target for the patient to control the left and right movement of the character object in the virtual scene through the left and right body offset, and controls the traveling speed of the character object in the virtual scene through the front and back body inclination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110161666.5A CN112951409A (en) | 2021-02-05 | 2021-02-05 | Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110161666.5A CN112951409A (en) | 2021-02-05 | 2021-02-05 | Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112951409A true CN112951409A (en) | 2021-06-11 |
Family
ID=76242575
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110161666.5A Pending CN112951409A (en) | 2021-02-05 | 2021-02-05 | Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112951409A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113593671A (en) * | 2021-10-08 | 2021-11-02 | 西安交通大学城市学院 | Automatic adjustment method and device of virtual rehabilitation game based on Leap Motion gesture recognition |
| CN114042296A (en) * | 2021-09-22 | 2022-02-15 | 广州医科大学附属第一医院（广州呼吸中心） | Intelligent training system for rehabilitation of lung surgery patient |
| CN114366557A (en) * | 2021-12-31 | 2022-04-19 | 华南理工大学 | Man-machine interaction system and method for lower limb rehabilitation robot |
| CN114367086A (en) * | 2021-12-31 | 2022-04-19 | 华南理工大学 | Lower limb rehabilitation training game system |
| CN115171457A (en) * | 2022-07-04 | 2022-10-11 | 上海明略信息技术有限公司 | Interactive training and evaluation system for aircraft ground signal command situation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140188009A1 (en) * | 2012-07-06 | 2014-07-03 | University Of Southern California | Customizable activity training and rehabilitation system |
| CN104258539A (en) * | 2014-09-04 | 2015-01-07 | 燕山大学 | Elderly people rehabilitation and monitoring system based on interaction of virtual reality and body movement |
| CN109350923A (en) * | 2018-10-25 | 2019-02-19 | 北京机械设备研究所 | A kind of rehabilitation training of upper limbs system based on VR and more body position sensors |
| CN110201358A (en) * | 2019-07-05 | 2019-09-06 | 中山大学附属第一医院 | Rehabilitation training of upper limbs system and method based on virtual reality and motor relearning |
-
2021
- 2021-02-05 CN CN202110161666.5A patent/CN112951409A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140188009A1 (en) * | 2012-07-06 | 2014-07-03 | University Of Southern California | Customizable activity training and rehabilitation system |
| CN104258539A (en) * | 2014-09-04 | 2015-01-07 | 燕山大学 | Elderly people rehabilitation and monitoring system based on interaction of virtual reality and body movement |
| CN109350923A (en) * | 2018-10-25 | 2019-02-19 | 北京机械设备研究所 | A kind of rehabilitation training of upper limbs system based on VR and more body position sensors |
| CN110201358A (en) * | 2019-07-05 | 2019-09-06 | 中山大学附属第一医院 | Rehabilitation training of upper limbs system and method based on virtual reality and motor relearning |
Non-Patent Citations (4)
| Title |
|---|
| 余楠兮 等: "建筑工人上肢康复训练虚拟现实系统软件设计", 《土木工程与管理学报》 * |
| 唐心宇: "基于云平台的情景交互式康复训练及评估系统", 《中国优秀硕士学位论文全文数据库 医药卫生》 * |
| 喻洪流 等: "《康复器械技术及路线图规划》", 31 July 2014, 东南大学出版社 * |
| 张帅等: "基于Unity3D和Kinect的体感跑酷游戏开发关键技术设计与实现", 《三明学院学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114042296A (en) * | 2021-09-22 | 2022-02-15 | 广州医科大学附属第一医院（广州呼吸中心） | Intelligent training system for rehabilitation of lung surgery patient |
| CN113593671A (en) * | 2021-10-08 | 2021-11-02 | 西安交通大学城市学院 | Automatic adjustment method and device of virtual rehabilitation game based on Leap Motion gesture recognition |
| CN113593671B (en) * | 2021-10-08 | 2022-01-04 | 西安交通大学城市学院 | Automatic adjustment method and device of virtual rehabilitation game based on Leap Motion gesture recognition |
| CN114366557A (en) * | 2021-12-31 | 2022-04-19 | 华南理工大学 | Man-machine interaction system and method for lower limb rehabilitation robot |
| CN114367086A (en) * | 2021-12-31 | 2022-04-19 | 华南理工大学 | Lower limb rehabilitation training game system |
| CN115171457A (en) * | 2022-07-04 | 2022-10-11 | 上海明略信息技术有限公司 | Interactive training and evaluation system for aircraft ground signal command situation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112951409A (en) | Hemiplegia patient rehabilitation system based on Kinect interaction and virtual reality | |
| CN109298779B (en) | Virtual training system and method based on virtual agent interaction | |
| CN104524742A (en) | Cerebral palsy child rehabilitation training method based on Kinect sensor | |
| CN111035840B (en) | VR closed-loop drug addiction self-control force training system | |
| KR101660157B1 (en) | Rehabilitation system based on gaze tracking | |
| CN110232963B (en) | Upper limb movement function evaluation system and method based on stereoscopic display technology | |
| CN106693280A (en) | Virtual-reality-based Parkinsonism training method, system and device | |
| CN109166612B (en) | Large-scale game scene rehabilitation system and method based on eye movement and electroencephalogram information | |
| US20140081432A1 (en) | Method and Apparatus for Rehabilitation Using Adapted Video Games | |
| Muñoz et al. | The biocybernetic loop engine: an integrated tool for creating physiologically adaptive videogames | |
| CN113611388B (en) | Intelligent sports rehabilitation and training system based on exoskeleton | |
| CN104484574A (en) | Real-time human body gesture supervised training correction system based on quaternion | |
| CN113066260B (en) | Parkinson's disease early warning system based on daily behavior analysis | |
| Funabashi et al. | Agar: A vr serious game to support the recovery of post-stroke patients | |
| CA2816955A1 (en) | Computer aided analysis and monitoring of mobility abnormalities in human patients | |
| CN110415357A (en) | A kind of interaction training supervision method based on virtual reality and augmented reality | |
| CN109126045A (en) | intelligent motion analysis and training system | |
| CN113694343A (en) | Immersive anti-stress psychological training system and method based on VR technology | |
| CN113332101B (en) | Control method and device of rehabilitation training device based on brain-computer interface | |
| CN110910984A (en) | System and method for processing state adjustment information of sportsman | |
| Calin et al. | Mira–upper limb rehabilitation system using microsoft kinect | |
| CN111312363B (en) | Double-hand coordination enhancement system based on virtual reality | |
| Yeh et al. | Virtual reality for post-stroke shoulder-arm motor rehabilitation: Training system & assessment method | |
| CN112220651A (en) | Wearable equipment system for rehabilitation training and wearable equipment | |
| CN116059600B (en) | Rehabilitation training device, training method and evaluation system based on interactive projection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210611 |