Summary of the invention
The present invention is directed to existing rehabilitation training system, the healing hand function training system that provides a kind of actual situation to merge, cost is low, take up room little, to the patient without injury.
In order to solve the problems of the technologies described above, the present invention is achieved by the following technical solutions:
The healing hand function training system that a kind of actual situation merges comprises:
Image capture device, for many viewpoints of Real-time Collection user hand images, and carry out being sent to background process equipment after pre-service;
Background process equipment, the generating virtual scene also shows output, described many viewpoints pretreatment image is carried out to three-dimensional modeling and generate the hand three-dimensional model, is shown in virtual scene, and sets up the man-machine interaction feedback mechanism.
Further, described image capture device comprises: at least two for the video camera from different viewpoints collection images, be used to the support that supports described video camera and the client that connects one to one with video camera, the image of described client camera acquisition, and to the image pre-service.
For the ease of regulating the angle of video camera, on described support, be provided with and the described video camera The Cloud Terrace of corresponding flexible connection one by one.
Due to needs, gather the multi aspect images of user's hand, described video camera preferably adopts 5.
Further, described background process equipment comprises server and display, described server generating virtual scene, and export display to and show, server receives simultaneously pretreatment image and carries out three-dimensional modeling, and the hand three-dimensional model of foundation is shown in virtual scene, and sets up the man-machine interaction feedback mechanism.
Wherein, described man-machine interaction feedback mechanism is: server carries out computational analysis to the variation of hand three-dimensional model, and virtual scene is made to corresponding change and shown output.
Based on above-mentioned healing hand function training system, the healing hand function training method that the present invention provides a kind of actual situation to merge simultaneously comprises the following steps:
(1), image capture device gathers background information, and stores;
(2), many viewpoints of image capture device Real-time Collection user hand images, and carry out being sent to background process equipment after pre-service;
(3), background process equipment shows the output virtual scene, and to described many viewpoints pretreatment image Real-time modeling set, the hand three-dimensional model of foundation is shown in virtual scene;
(4), virtual scene is made corresponding change and is shown output according to the variation of hand three-dimensional model.
Further, described background process equipment comprises server and display, and step (3) comprises following sub-step:
(31), at the display end, show initial virtual scene;
(32), server receives in real time many viewpoints pretreatment image, and sets up the hand three-dimensional model;
(33), the hand three-dimensional model of foundation is shown in virtual scene.
Further again, in step (3), the Real-time modeling set method is:
(a), according to the Octree modeling principle, initial space is carried out to successively subdivision, obtain the voxel visual shell model of hand;
(b), utilize the cube method of advancing to carry out the gridding processing to voxel visual shell model surface;
(c), utilization is carried out texture based on the visual shell model of the texture mapping method effects on surface gridding of viewpoint.
Further, in step (31), also comprise before the step of the inside and outside parameter of calibrating camera.
Compared with prior art, advantage of the present invention and good effect are: the healing hand function training system that actual situation of the present invention merges, by building a virtual training environment, the guiding user action, make it in actual situation is mutual, realize the training of hand motion ability, accelerate healing hand function, and can not increase the weight of the hand burden, accomplished the hand not damaged; Device structure is simple, takes up room little; Cost of manufacture is low; The healing hand function training method that actual situation of the present invention merges, simple to operate; The user also can suitably train under the guidance without specialist, to the user without injury; Adopt the actual situation interactive system of immersion to train in real time, obtain training effect preferably.
After reading by reference to the accompanying drawings the detailed description of embodiment of the present invention, it is clearer that the other features and advantages of the invention will become.
Embodiment
Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described in further detail.
the present invention likely can cause mechanical damage to the patient in order to solve the auxiliary Rehabilitation training of existing apparatus, and present healing robot apparatus expensive, take up room large, and the problem that practical application has little significance, the healing hand function training system that provides a kind of actual situation to merge, use many viewpoints hand images of a plurality of video camera Real-time Collection users, carry out the hand three-dimensional modeling, generate the hand three-dimensional model and be shown in virtual scene, when actual hand motion, the hand three-dimensional model moves thereupon, and virtual scene is made real-time response, realized the seamless fusion of actual hand and virtual environment, user's feeling of immersion is good, equipment is simple, take up room little, and to user's not damaged.
embodiment mono-, the healing hand function training system that the actual situation of the present embodiment merges, comprise: image capture device and background process equipment, described image capture device Real-time Collection user hand images, and the hand images that gathers is carried out being sent to background process equipment after pre-service, described background process equipment receives pretreatment image, utilize dimensional Modeling Technology, described pretreatment image is carried out to three-dimensional modeling and generate the hand three-dimensional model, and demonstration output virtual scene and hand three-dimensional model, this hand three-dimensional model is placed in virtual scene, with the seamless fusion of virtual scene.Hand images modeling due to image capture device Real-time Collection user, therefore, when user's hand motion, its three-dimensional model is action thereupon also, background process equipment is by setting up the man-machine interaction feedback mechanism, computational analysis is carried out in variation to the hand three-dimensional model, and virtual scene is made to corresponding change and shown output.The healing hand function training system that this actual situation merges is by building a virtual scene, this virtual scene can produce to the variation of hand corresponding feedback, form the rehabilitation training environment of an actual situation fusion, positive guide, the guiding user action, make it in actual situation is mutual, realize the training of hand motion ability, accelerate healing hand function, and can not increase the weight of the hand burden, accomplished the hand not damaged.
As a specific embodiment, shown in Figure 1, described image capture device comprises: be used to video camera 1-1, support 1-2 and the client 1-3 that gathers image, described video camera 1-1 is arranged on support 1-2, by support 1-2 support cameras 1-1, because video camera 1-1 must gather hand information from a plurality of visual angles, therefore video camera 1-1 is at least two, in order to obtain the image of many viewpoints, be convenient to after three-dimensional modeling truer, and consider simultaneously the saving cost, and reducing operand, described video camera 1-1 preferably adopts 5.The number of described client 1-3 equates with the number of video camera 1-1, and client 1-3 and video camera 1-1 connect one to one, and described client 1-3 receives the image that video camera 1-1 gathers, and image is carried out being sent to background process equipment after pre-service.The image pre-service comprises denoising, the extraction of hand foreground information etc.
Described background process equipment comprises server 2-1 and display 2-2, described server 2-1 generating virtual scene, and export display 2-2 to and show, server 2-1 receives the pretreatment image that client 1-3 sends simultaneously, because pretreatment image is multi-view image, therefore server 2-1, by analytical calculation, can adopt certain modeling method to carry out three-dimensional modeling to hand, and the hand three-dimensional model of foundation seamlessly is shown in virtual scene.Due to video camera 1-1 Real-time Collection user's hand images and by server 2-1 Real-time modeling set, therefore, when user's hand motion, its three-dimensional model is action thereupon also, server 2-1, by calculating, makes corresponding change by virtual scene, scene that is virtually reality like reality, realize man-machine interaction, the guiding user better completes the hand function training in virtual training environment.By display 2-2, show virtual scene and hand three-dimensional model, the user can observe the hand exercise situation of oneself, can, according to the guiding positive action of the virtual scene that shows, more be conducive to functional rehabilitation.The communication of client 1-3 and server 2-1 preferably adopts the LAN transmission to realize, therefore, between described client 1-3 and server 2-1, is provided with the network switch 3.
For the ease of regulating the angle of video camera 1-1, on described support 1-2, be provided with and the described video camera The Cloud Terrace 1-4 of corresponding flexible connection one by one.Those skilled in the art can easily be known: The Cloud Terrace is the installation of specialty, the fastening of fixed cameras, can to the supervision angle of video camera, adjust flexibly, at this, does not elaborate.
Based on above-mentioned healing hand function training system, the healing hand function training method that the present embodiment provides a kind of actual situation to merge simultaneously comprises the following steps:
S1, image capture device gather background information, and storage;
Many viewpoints hand images of S2, image capture device Real-time Collection user, and carry out being sent to background process equipment after pre-service;
S3, background process equipment show the output virtual scene, and to described many viewpoints pretreatment image Real-time modeling set, and the hand three-dimensional model of foundation is shown in virtual scene;
S4, virtual scene are made corresponding change and are shown output according to the variation of hand three-dimensional model.
Wherein, image capture device gathers background information, for the ease of in step S2, extracting user's hand images information, at first gather background image, and storage, when user's hand is placed in space, utilize the background subtraction method very easily the hand foreground information to be extracted, the image pre-service comprises the foreground information extraction of image denoising and hand images etc.
As a specific embodiment, described background process equipment comprises server and display, and step S3 comprises following sub-step:
S31, at the display end, show initial virtual scene;
S32, server receive many viewpoints pretreatment image in real time, and set up the hand three-dimensional model;
S33, the hand three-dimensional model of foundation is shown in virtual scene.
In step S31, because virtual scene is set up by server end, and when user's hand motion, its three-dimensional model is action thereupon also, server, by computational analysis, is made corresponding change by virtual scene, and shows output at display, therefore, at first server shows initial virtual scene by display.
Wherein, in step S3, the Real-time modeling set method is:
(a), according to the Octree modeling principle to the space subdivision, obtain the voxel visual shell model of hand;
(b), utilize the cube method of advancing to carry out the gridding processing to voxel visual shell model surface;
(c), utilization is carried out texture based on the visual shell model of the texture mapping method effects on surface gridding of viewpoint.
Due to Octatree technique fast operation in modeling, the present embodiment preferably adopts the Octree modeling principle to carry out modeling, meet the requirement of real-time of the healing hand function training method of this actual situation fusion, Octatree technique is modeled as prior art, at this, does not elaborate.
In order to make the hand of setting up three-dimensional model surface smoothing, precision is higher, and the voxel visual shell model that the present embodiment preferably adopts the cube method of advancing to set up previous step carries out the gridding processing.
In order to make the hand three-dimensional model have more verisimilitude and feeling of immersion, more near real human body skin, the visual shell model that also comprises the effects on surface gridding in the present embodiment carries out texture, the texture source is the many viewpoints hand images after pre-service, make it seem more approaching true, the method of texture is the texture mapping method based on viewpoint, and the method is prior art, at this, does not elaborate.
In addition, because the shooting angle of video camera may be adjusted, some parameters (such as the inside and outside parameter of video camera etc.) can change accordingly, therefore, before step S31, also comprise the step of the inside and outside parameter of calibrating camera, this step is arranged between other steps, can processed offline.Camera calibration can be realized in the following ways: with a uniform black and white checkerboard pattern as destination object, after video camera fixes, gather the image of the template of diverse location and angle, with this group image, carry out extraction and the calculating of angle point, thereby obtain the model of video camera, i.e. the inside and outside parameter of video camera.
The healing hand function training method that the actual situation of the present embodiment merges, simple to operate, the user also can suitably train under the guidance without specialist, to the user without injury, adopt immersion and interactively actual situation to merge scene and train, obtain training effect preferably.
Certainly; this actual situation merges the healing hand function training system and not only is confined to carry out the hand function rehabilitation training; also can be by transformation; for other position function rehabilitation trainings of human body; above-mentioned explanation is not limitation of the present invention; the present invention also is not limited in above-mentioned giving an example, and those skilled in the art make in essential scope of the present invention variation, remodeling, interpolation or replacement, also should belong to protection scope of the present invention.