CN104722056A - Rehabilitation training system and method using virtual reality technology - Google Patents
Rehabilitation training system and method using virtual reality technology Download PDFInfo
- Publication number
- CN104722056A CN104722056A CN201510059881.9A CN201510059881A CN104722056A CN 104722056 A CN104722056 A CN 104722056A CN 201510059881 A CN201510059881 A CN 201510059881A CN 104722056 A CN104722056 A CN 104722056A
- Authority
- CN
- China
- Prior art keywords
- model
- data
- kinect
- skeleton
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a rehabilitation training system and method using the virtual reality technology. The method includes the following stages: (1) skeleton joints of the human body are collected through a somatosensory device Kinect, and a skeleton system is generated; (2) a model is prepared, skeletons are bound, a database environment is constructed, the positions of the skeleton joints are obtained through machine learning of depth information and color information, modeling is achieved through a 3Dmax software tool, the human body skeleton IK/FK is built, and the joints of the model are bound in a one-to-one correspondence mode; (3) data of the Kinect are transmitted into a Unity3D through an interface function, and a scene model is driven to move; (4) a role is led in, and the environment is debugged; (5) the data are written in and read out, and visualization of the data is controlled. According to the rehabilitation training system and method, limitation of sensors is eliminated, and natural limb motion monitoring becomes possible; the somatosensory device Kinect is adopted, so that the system cost is reduced, and the complexity for wearing an input device in the training process is eliminated.
Description
Technical field
The present invention relates to a kind of rehabilitation training system and method, especially a kind ofly design a kind of ingenious, that intelligence degree is high rehabilitation training system and method using virtual reality technology.
Background technology
In current rehabilitation at home, main rehabilitation means are the manipulations based on therapist, or more auxiliary simple exercising devices, the object recovering muscular movement function is namely reached by passive movement, synkinesia, active movement and resistive exercise etc.This method achieves good rehabilitation effect, but simultaneously due to the limitation for the treatment of means, causes patient to be in passive position.And in the training process, action repeatedly, dull, uninteresting, patient is easy to produce boredom, is unfavorable for the continuation for the treatment of and deeply.
In order to overcome the above problems, people start virtual reality technology to use during dyskinesia resumes training.Utilize virtual reality system to carry out dyskinesia rehabilitation training, be allow patient play a role in virtual environment, become the part in virtual environment, undertaken alternately by training action and virtual environment, virtual environment gives patient feedback immediately.This training method can reduce rehabilitation process to therapist and the degree of dependence for the treatment of place, for patient provide accurately, stablize, the training effect evaluation index of personalized training mode and quantification, increase the interest of therapeutic process simultaneously, exciting patient to participate in the enthusiasm of therapeutic process, is that passive treatment becomes active treatment.
In virtual reality system, the movement transition of hand can be the computer instruction of touching object by general data glove, but only has the ability of touching object of stretching out one's hand, not the tactile feedback of touching object, and this is far from being enough in the sense of reality.Recent study personnel add sense of touch and device for force feedback in data glove, strengthen patient to the actual experience of virtual environment, increase the feeling of immersion of patient.The Ruba of Ottawa size uses the data glove with tactile feedback to carry out rehabilitation training, and devises the rehabilitation training virtual scene of different grade of difficulty for patient.Rutgers university and Stanford University develop the bone remote rehabilitation system based on virtual reality.Patient carries out virtual reality rehabilitation training in family, and the control end of this system directly reads the rehabilitation data of database, and professional treatment teacher understands the progress of patients ' recovery training by analyzing data.Zhang Lixun etc. develop the lower limb rehabilitation robot virtual reality emulation system based on matlab.This system can pass through the motion state of network remote monitoring healing robot and the controling parameters of on-line tuning healing robot, and carry out virtual emulation at control end, make professional treatment teacher can get information about the physical training condition of patient at control end, provide platform for professional treatment teacher remote guide patient carries out rehabilitation training.
The appearance of body sense technology, makes people very directly can use limb action, with device or the environment interaction of periphery, and without the need to using the control appliance of any complexity, people just can be allowed to do interaction with content with being personally on the scene.
Utilize virtual display system to carry out dyskinesia rehabilitation training, allow patient play a role in virtual environment, become the part in virtual environment, undertaken alternately by training action and virtual environment, virtual environment gives patient feedback immediately.This training method can reduce rehabilitation process to therapist and the degree of dependence for the treatment of place, for patient provide accurately, stablize, the training effect evaluation index of personalized training mode and quantification, increase the interest of therapeutic process simultaneously, excite patient to participate in the enthusiasm of therapeutic process, make passive treatment become active treatment.At present, virtual reality technology is used widely in dyskinesia rehabilitation training field.But in current virtual reality system, there is many defects:
(1) in the use procedure of training system, needs of patients dresses perception input equipment, as data glove, data suit, the data helmet etc., and from output feedack equipment, obtains the feedback of the multiple sense organs such as vision, the sense of hearing and sense of touch.Donning process is loaded down with trivial details, affects the enthusiasm of user.
(2) patient needs to carry out action correction before use system, and use procedure is loaded down with trivial details, reduces the training interest of user.
(3) wearable device is usually expensive, considerably increases the cost of training system, is unfavorable for that patient introduces the purchase use of family.
(4) input interface is single, is unfavorable for the expansion of system.
Summary of the invention
For solving the problem, the invention provides a kind of rehabilitation training system and the method for using virtual reality technology, comprising following three phases:
First stage: gather skeleton node by body sense equipment Kinect, generate shell system, specifically comprise:
1) depth image is obtained by infrared sensor;
2) find mobile object, carry out Pixel-level assessment, carry out the rejecting shade of background;
3) human body is distinguished by the machine learning system of Kinect;
4) by identification determination human body 20 artis of human body, and shell system is generated.
Second stage: preparation model also binds bone, build database environment, each bone node location is obtained by the machine learning of depth information and colour information, by the modeling of 3Dmax Software tool, set up skeleton IK/FK, the node one_to_one corresponding of Binding Model, by upgrading the node location controlled when a model, being synchronized with the movement of implementation model.
Phase III: import in the middle of Unity3D by interface function by the data of Kinect, model of place is driven to move.
Fourth stage: import role, set up Unity3D scene and gui interface, debugging enironment.
Five-stage: functional module realize part, the write of this part primary responsibility data and reading, by manager script regulation and control data, and control data is visual.
Compared with prior art, patient only needs station before screen to this patent, selects the scene of corresponding grade of difficulty, completes various limb action according to the instruction on scenic picture, screen gets final product the action situation of real-time monitored patient; The action of patient can also carry out interaction with the content in picture, makes rehabilitation training just as playing games, increases the enthusiasm of patient; The 3D image detecting of body sense equipment and skeleton tracer technique, allow and break away from sensor constraint, and realizing the monitoring of nature limb motion becomes possibility; Adopt body sense interactive device Kinect, reduce the cost of system, easy and simple to handle, interactivity is strong, eliminates in training process the triviality of dressing input equipment.In future soon, the sight that rehabilitation and health care enter into family also it is expected to, and will become an important component part of online medical treatment.
Native system uses body sense technology to carry out real-time action seizure, catch the 3 D motion trace of patient's arm, by computer graphical and image technique, develop some game style training scenes for rehabilitation to carry out human-computer interaction, add the interest of therapeutic process and the enthusiasm of patient, make rehabilitation training become the active behavior of patient.Patient can be made to carry out alternately with natural way and the object had in the virtual environment of multiple stimulus to the sense organ.And decrease the danger that patient is caused by faulty operation in true environment, thus better complete rehabilitation training, by continuous arm motion, reach the effect regained the function gradually.
The key point of native system:
Pure three dimensional pattern
Select 3D Max as D modeling tool, model adopts pure three dimensional pattern, and with the addition of bone for manikin, can detect the action of multiple joint parts of visual human.
The game type rehabilitation training scene of different grade of difficulty
System is the game type rehabilitation training virtual scene that rehabilitation training patient devises different grade of difficulty, training process is from the easier to the more advanced, incremental, is furnished with music effect in scene, increases the interest of training, reaches better training effect.
Multiple input equipment interface
System for primary input device, and provides other wearable input equipment interface, as data glove, data suit, the data helmet etc. with body sense interactive device kinect.
Easy and simple to handle, cost is low, and interactivity is strong
Adopt body sense interactive device kinect as input equipment, patient can train without the need to dressing any awareness apparatus.Native system is with low cost, is easy to patient and buys, family use.
Achieve the action data record of training mission
System is by the data transmission with far-end server, patient's training data each time is all saved, and be shown in web page or this locality, the recovery that the understanding patient that professional treatment Shi Gengjia can be made visual and clear is current, also make guidance for later rehabilitation programme.
Detailed description of the invention
Below in conjunction with embodiment, this patent is described further:
Use rehabilitation training system and the method for virtual reality technology, a kind of rehabilitation training system and method using virtual reality technology, comprises following three phases:
First stage: gather skeleton node by body sense equipment Kinect, generate shell system, specifically comprise:
1) depth image is obtained by infrared sensor;
2) find mobile object, carry out Pixel-level assessment, carry out the rejecting shade of background;
3) human body is distinguished by the machine learning system of Kinect;
4) by identification determination human body 20 artis of human body, and shell system is generated.
Second stage: preparation model also binds bone, build database environment, each bone node location is obtained by the machine learning of depth information and colour information, by the modeling of 3Dmax Software tool, set up skeleton IK/FK, the node one_to_one corresponding of Binding Model, by upgrading the node location controlled when a model, being synchronized with the movement of implementation model.
Phase III: import in the middle of Unity3D by interface function by the data of Kinect, model of place is driven to move.
Fourth stage: import role, set up Unity3D scene and gui interface, debugging enironment.
Five-stage: functional module realize part, the write of this part primary responsibility data and reading, by manager script regulation and control data, and control data is visual.
Compared with prior art, patient only needs station before screen to this patent, selects the scene of corresponding grade of difficulty, completes various limb action according to the instruction on scenic picture, screen gets final product the action situation of real-time monitored patient; The action of patient can also carry out interaction with the content in picture, makes rehabilitation training just as playing games, increases the enthusiasm of patient; The 3D image detecting of body sense equipment and skeleton tracer technique, allow and break away from sensor constraint, and realizing the monitoring of nature limb motion becomes possibility; Adopt body sense interactive device Kinect, reduce the cost of system, easy and simple to handle, interactivity is strong, eliminates in training process the triviality of dressing input equipment.In future soon, the sight that rehabilitation and health care enter into family also it is expected to, and will become an important component part of online medical treatment.
Embodiment recited above is only be described the preferred embodiment of the present invention; not protection scope of the present invention is limited; do not departing under the present invention designs spiritual prerequisite; the various distortion that the common engineers and technicians in this area make technical solution of the present invention and improvement, all should fall in protection domain that claims of the present invention determine.
Claims (1)
1. use rehabilitation training system and the method for virtual reality technology, it is characterized in that, comprise following three phases:
First stage: gather skeleton node by body sense equipment Kinect, generate shell system, specifically comprise:
1) depth image is obtained by infrared sensor;
2) find mobile object, carry out Pixel-level assessment, carry out the rejecting shade of background;
3) human body is distinguished by the machine learning system of Kinect;
4) by identification determination human body 20 artis of human body, and shell system is generated;
Second stage: preparation model also binds bone, build database environment, each bone node location is obtained by the machine learning of depth information and colour information, by the modeling of 3Dmax Software tool, set up skeleton IK/FK, the node one_to_one corresponding of Binding Model, by upgrading the node location controlled when a model, being synchronized with the movement of implementation model;
Phase III: import in the middle of Unity3D by interface function by the data of Kinect, model of place is driven to move;
Fourth stage: import role, set up Unity3D scene and gui interface, debugging enironment;
Five-stage: functional module realize part, the write of this part primary responsibility data and reading, by manager script regulation and control data, and control data is visual.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510059881.9A CN104722056A (en) | 2015-02-05 | 2015-02-05 | Rehabilitation training system and method using virtual reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510059881.9A CN104722056A (en) | 2015-02-05 | 2015-02-05 | Rehabilitation training system and method using virtual reality technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104722056A true CN104722056A (en) | 2015-06-24 |
Family
ID=53446840
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510059881.9A Pending CN104722056A (en) | 2015-02-05 | 2015-02-05 | Rehabilitation training system and method using virtual reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104722056A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106601063A (en) * | 2016-12-20 | 2017-04-26 | 张小来 | Method for building visual use electrocardiogram monitor teaching and training system |
| CN106599770A (en) * | 2016-10-20 | 2017-04-26 | 江苏清投视讯科技有限公司 | Skiing scene display method based on body feeling motion identification and image matting |
| CN106650687A (en) * | 2016-12-30 | 2017-05-10 | 山东大学 | Posture correction method based on depth information and skeleton information |
| CN107133489A (en) * | 2017-07-03 | 2017-09-05 | 广东工业大学 | A kind of rehabilitation training appraisal procedure and system based on somatosensory device |
| WO2017161733A1 (en) * | 2016-03-24 | 2017-09-28 | 乐视控股（北京）有限公司 | Rehabilitation training by means of television and somatosensory accessory and system for carrying out same |
| CN107274467A (en) * | 2017-06-29 | 2017-10-20 | 厦门游亨世纪科技有限公司 | A kind of model animation interlock method based on Unity3D |
| CN107537135A (en) * | 2017-01-25 | 2018-01-05 | 上海电气集团股份有限公司 | A kind of lower limb rehabilitation training device and system based on virtual reality technology |
| CN108031110A (en) * | 2017-11-03 | 2018-05-15 | 东莞市新进巧工艺制品有限公司 | A kind of games system based on AR technologies |
| CN108257654A (en) * | 2017-12-21 | 2018-07-06 | 天津大学 | A kind of method that cervical vertebra rehabilitation training is realized by virtual reality technology |
| CN109086755A (en) * | 2018-11-07 | 2018-12-25 | 上海电气集团股份有限公司 | The virtual reality display methods and system of healing robot based on image segmentation |
| CN109545323A (en) * | 2018-10-31 | 2019-03-29 | 贵州医科大学附属医院 | A kind of ankle rehabilitation system with VR simulation walking |
| CN113033242A (en) * | 2019-12-09 | 2021-06-25 | 上海幻电信息科技有限公司 | Action recognition method and system |
| CN113058258A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing expected somatosensory action based on player game |
| CN114005323A (en) * | 2021-10-22 | 2022-02-01 | 南通市第二人民医院 | Virtual reality training system for central integrated rehabilitation therapy technology |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102908772A (en) * | 2012-10-16 | 2013-02-06 | 东南大学 | Upper limb rehabilitation training system by using augmented reality technology |
| CN103230664A (en) * | 2013-04-17 | 2013-08-07 | 南通大学 | Upper limb movement rehabilitation training system and method based on Kinect sensor |
| CN103598889A (en) * | 2013-11-13 | 2014-02-26 | 东南大学常州研究院 | Scene interaction rehabilitation training method based on Kinect vision gait analysis |
-
2015
- 2015-02-05 CN CN201510059881.9A patent/CN104722056A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102908772A (en) * | 2012-10-16 | 2013-02-06 | 东南大学 | Upper limb rehabilitation training system by using augmented reality technology |
| CN103230664A (en) * | 2013-04-17 | 2013-08-07 | 南通大学 | Upper limb movement rehabilitation training system and method based on Kinect sensor |
| CN103598889A (en) * | 2013-11-13 | 2014-02-26 | 东南大学常州研究院 | Scene interaction rehabilitation training method based on Kinect vision gait analysis |
Non-Patent Citations (4)
| Title |
|---|
| DAVID ANT N等: "A Kinect-based telerehabilitation system", 《2013 IEEE 15TH INTERNATIONAL CONFERENCE ON E-HEALTH NETWORKING, APPLICATIONS AND SERVICES 》 * |
| 丁晨等: "Kinect体感交互技术及其在医疗康复领域的应用", 《中国康复理论与实践》 * |
| 乐小燕等: "Kinect与Unity结合的人体骨骼控制方法", 《信息与电脑》 * |
| 王万良等: "基于kinect的高维人体动画实时合成研究", 《计算机应用与软件》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017161733A1 (en) * | 2016-03-24 | 2017-09-28 | 乐视控股（北京）有限公司 | Rehabilitation training by means of television and somatosensory accessory and system for carrying out same |
| CN106599770A (en) * | 2016-10-20 | 2017-04-26 | 江苏清投视讯科技有限公司 | Skiing scene display method based on body feeling motion identification and image matting |
| CN106601063A (en) * | 2016-12-20 | 2017-04-26 | 张小来 | Method for building visual use electrocardiogram monitor teaching and training system |
| CN106650687A (en) * | 2016-12-30 | 2017-05-10 | 山东大学 | Posture correction method based on depth information and skeleton information |
| CN107537135A (en) * | 2017-01-25 | 2018-01-05 | 上海电气集团股份有限公司 | A kind of lower limb rehabilitation training device and system based on virtual reality technology |
| CN107274467A (en) * | 2017-06-29 | 2017-10-20 | 厦门游亨世纪科技有限公司 | A kind of model animation interlock method based on Unity3D |
| CN107133489A (en) * | 2017-07-03 | 2017-09-05 | 广东工业大学 | A kind of rehabilitation training appraisal procedure and system based on somatosensory device |
| CN108031110A (en) * | 2017-11-03 | 2018-05-15 | 东莞市新进巧工艺制品有限公司 | A kind of games system based on AR technologies |
| CN108257654A (en) * | 2017-12-21 | 2018-07-06 | 天津大学 | A kind of method that cervical vertebra rehabilitation training is realized by virtual reality technology |
| CN109545323A (en) * | 2018-10-31 | 2019-03-29 | 贵州医科大学附属医院 | A kind of ankle rehabilitation system with VR simulation walking |
| CN109086755A (en) * | 2018-11-07 | 2018-12-25 | 上海电气集团股份有限公司 | The virtual reality display methods and system of healing robot based on image segmentation |
| CN113033242A (en) * | 2019-12-09 | 2021-06-25 | 上海幻电信息科技有限公司 | Action recognition method and system |
| CN113058258A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing expected somatosensory action based on player game |
| CN113058258B (en) * | 2021-04-22 | 2024-04-19 | 杭州当贝网络科技有限公司 | Method, system and storage medium for identifying motion based on expected motion of player game |
| CN114005323A (en) * | 2021-10-22 | 2022-02-01 | 南通市第二人民医院 | Virtual reality training system for central integrated rehabilitation therapy technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104722056A (en) | Rehabilitation training system and method using virtual reality technology | |
| CN108815804B (en) | VR upper limb rehabilitation training platform and method based on MYO arm ring and mobile terminal | |
| CN108701429A (en) | The virtual and/or augmented reality that physics interactive training is carried out with operating robot is provided | |
| US20090051544A1 (en) | Wearable User Interface Device, System, and Method of Use | |
| CN107067856A (en) | A kind of medical simulation training system and method | |
| CN105374251A (en) | Mine virtual reality training system based on immersion type input and output equipment | |
| Abdur Rahman et al. | Multimedia interactive therapy environment for children having physical disabilities | |
| CN107281728B (en) | Sensor-matched augmented reality skiing auxiliary training system and method | |
| CN105630168B (en) | A kind of information processing method, the first electronic equipment and the second electronic equipment | |
| CN112642133A (en) | Rehabilitation training system based on virtual reality | |
| Rahman et al. | Modeling therapy rehabilitation sessions using non-invasive serious games | |
| US9000899B2 (en) | Body-worn device for dance simulation | |
| KR20150097050A (en) | learning system using clap game for child and developmental disorder child | |
| CN107632702B (en) | Holographic projection system adopting light-sensing data gloves and working method thereof | |
| CN110741327B (en) | Mud toy system and method based on augmented reality and digital image processing | |
| WO2020016225A1 (en) | Method and control unit for controlling a virtual reality display, virtual reality display and virtual reality system | |
| US11436806B1 (en) | Dual perspective rendering in virtual reality | |
| CN211180839U (en) | Motion teaching equipment and motion teaching system | |
| US20230162458A1 (en) | Information processing apparatus, information processing method, and program | |
| CN108969864A (en) | Depression recovery therapeutic equipment and its application method based on VR technology | |
| CN114255511A (en) | Controller and method for gesture recognition and gesture recognition device | |
| Ojeda et al. | Gesture-gross recognition of upper limbs to physical rehabilitation | |
| KR102527513B1 (en) | Wheelchair boarding device capable of rotating, forward and backward, and virtual reality application, and virtual reality application method using the same | |
| CN112245910B (en) | Modeling and limit movement method and system based on Quest head display | |
| KR20190059352A (en) | System for generating virtual and augmented reality images for experience using motion recognition camera and diorama and game method using the system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150624 |
|
| RJ01 | Rejection of invention patent application after publication |